One thing I do not understand is, why didn't we ever go on the offensive? It seems all we did was play a defensive war. Couldn't the military launch a massive offensive and take Hanoi?
Both the Soviet Union and Red China were courting and supplying North Vietnam as a client state. Had the US invaded, we would definitely have found ourselves fighting China again and possibly also at war with the Soviet Union. Much as I hate fighting a war half-assed, neither of those things should be taken lightly.
As far as the war itself, it's largely a myth that inside South Vietnam we fought under restrictive ROE. We relocated a significant portion of the population to create free-fire zones, developed remote sensors and denuded great stretches of jungle and forest for better visibility, brought to bear massive fire power. Our biggest self-restraint outside of not invading was honoring other nations' boundaries which provided the NVA safe areas to rest and refit, and that largely went away in 1968. We also allowed the South Vietnamese to be as corrupt as they wished as long as they were strongly pro-American, which allowed the Viet Cong to hide within the civilian population. We had problems within our force as well - racial strife, a policy of rotation by individual (kills unit cohesion) rather than by unit, 12 or 13 month deployments (by the time you know what you are doing you are a short-timer more concerned with surviving than with executing a mission), six month commands for officers (gives them zero time to learn their men or their trade or the terrain before pushing to accomplish something career-building and thus needlessly killing a lot of Americans and South Vietnamese - and the generals thought every officer needed to get his chance at command), and overall leadership failed to sell the nation on the need for the war. Thus we had a reluctant force with poor leadership and little motivation fighting the finest light infantry in the world (after '67 the Viet Cong was largely spent, and after Tet '68 they were completely finished and the NVA regulars fielded the only large units opposing us.) Only the Marines really had a model suitable for winning a guerrilla war and McNamara stuck them on his "wall" as targets rather than allowing them to fully implement it.
The main reason we lost is that we lost our will. Tet '68 was a crushing defeat for the NVA even though they achieved strategic surprise, but it was reported as a crushing defeat for America. War is the continuation of policy via violence. Ho Chi Min said "You will kill ten of us, we will kill one of you, but in the end, you will tire of it first." He was absolutely correct. His force and his nation were of one mind, fighting for a cause in which they absolutely believed. Our force was powerful but our nation was not of one mind; fighting communism in a tiny country very few Americans could describe or place on a map simply wasn't motivation enough to support a war. In the end we were out-maneuvered, out-fought strategically and operationally if seldom tactically, out-generalled, and out-gutted.