He said it most ideally, where it's a nothing more than a voluntary Union of States. States that are required to uphold the Bill of Rights, but have no other requirements placed on them.
I want 50 experiements in government, where the people are free to move to a more like-minded state and thus get the representation that best suits them. Then we can have an ecomonic competition among states where the people pick winners and looser, and the looser can freely decide to emulate the policy of the winning states.
We'll have such variety that we won't get stuck in century long arguments of policy. If there's an idea then there's a state where someone has tried it and learned the results. Everyone else knows the results and this competition, this plurality, results in a better life for everyone because an entire nation of 310 million people won't be held hostage to a stagnant and ineffective consolidated seat of power in Washington DC.
I'm sorry Jaskalas and Anarchist420 (mainly Anarchist420), that vision isn't ideal at all. See federalist 6, 7, and 8.
Federalist 6:
A man must be far gone in Utopian speculations who can seriously doubt that, if these States should either be wholly disunited, or only united in partial confederacies, the subdivisions into which they might be thrown would have frequent and violent contests with each other. To presume a want of motives for such contests as an argument against their existence, would be to forget that men are ambitious, vindictive, and rapacious. To look for a continuation of harmony between a number of independent, unconnected sovereignties in the same neighborhood, would be to disregard the uniform course of human events, and to set at defiance the accumulated experience of ages
...
what reason can we have to confide in those reveries which would seduce us into an expectation of peace and cordiality between the members of the present confederacy, in a state of separation?
Federalist 7 shows the likelihood of future conflict between states by describing points of conflict.
Federalist 8 shows the inconveniences of such conflict.
In short, such a vision of 50 separate, sovereign states would quickly devolve into war between the states. The national government must have enough power such that the states are unified under one central government.
Yet, in order for government to operate based on the consent of the governed and not to become tyrannical, government must be limited. How? The system of federalism developed in the constitution have proven to be a solution. To prevent the federal government from becoming tyrannical, the constitution specifically enumerates specific powers reserved for the federal government. The rest go to the states. The duty of the citizen is to see to it that this balance of power between federal and state government be preserved, though throughout America's history, we've forever been centralizing. This is why the thinking behind the New Deal and the Great Society were wrong. The federal government has no power to step into those fields. Altogether today, we do not have enough confidence in states being able to solve their own problems, and states today are highly dependent on money from the feds, whose distribution is decided by the centralized power of Washington DC.
Never before the American experiment have we seen limited government extend over so vast a territory as the United States. We ought to pay attention to the political thinking that went into writing the constitution because that theory is the foundation for the governance of this nation.