What right those who govern have to govern they don't question, they just govern. Whether the people have a right to depose them that doesn't concern them. All they are concerned with is that the people will not be tempted to depose them.
The great virtue of bureaucracy - indeed, perhaps its defining characteristic ~ was that it was an institutional method for applying general rules to specific cases, thereby making the actions of government fair and predictable.