Each user from a dialog’s occupant_ids field can remove the dialog. It does not mean that this dialog will be removed completely for all users in this dialog. It will be removed only for a current user.
To completely remove a dialog, pass force=1. Only the owner can do it.