A government not supported by the majority cannot sustain itself without tyranny. They will eventually get rid of the current government and laws, even amend the constitution, if needed, to get their way. Thus it is impossible to maintain libertarianism by restricting voters or their power.