Don't do anything illegal, and you have nothing to worry about. If you simply abide by the laws, do your job, and don't commit any crimes, what is there to worry about? They are doing this for your protection. There are people that making a living off of committing crimes that other people end up having to pay for, those are the people they are going after.
That's all well and good when you have a just government, but such a thing doesn't last. If you are completely ignorant of human nature, history, and current events, your statement almost makes sense.
Similar statements to "don't do anything illegal, and you have nothing to worry about" have been repeated in every single police state and dictatorship to this day. When secret police start carrying away members of the formerly-free press for being "dissidents", don't say you weren't warned. There are always internal enemies of the state who are imprisoned, tortured, or killed unjustly, and found more easily when the government can easily monitor all communications. It's not just journalists who live in fear in many countries today, it's people who practice other religions (or any religion, as in the past with very heavy-handed Communist nations), outspoken business people, educators who teach other than the approved material, etc.
If that sounds overly dramatic and a farfetched, Orwellian tale, just take a look at the world and history. Broad powers to conduct surveillance covertly with little or no oversight enables dictatorships and oppressive regimes. Not just for relatively tiny states like North Korea or Syria, but massive nations of the recent past and present like the USSR and China. Democracy doesn't mean squat in the long run if you give a few people the power of and over thousands and millions without enough checks on their power. Even Rome was a republic to begin with.
There is no magical barrier preventing the United States from becoming like many of the failed nation states that have come before or that exist today. All it takes is for us to keep shrugging our shoulders. The road to such a state of affairs can be gradual or quick, and technology allows for the much quicker road. If revolutions can take place practically overnight with the aid of Twitter and Facebook - a thing many would have thought unlikely if anyone had even dreamed of it 30 or 40 years ago - dictatorships can rise in a little longer period of time with the restriction and/or surveillance of such technologies, the removal of even the possibility of anonymity, and the monitoring of all traffic.
"Those who cannot remember the past are condemned to repeat it"
Even if you don't want to look at the past or the future, just see what this is doing to the political and economic goodwill that the government and U.S. businesses have with the rest of the world. Especially in high tech segments of the market like cloud storage. Who wants to have their data hosted or routed through the U.S. if it's being examined? And all that infrastructure to do so comes from taxpayer money. That comes from your wallet and mine. Do we even know what the "return" on our investment is? Can the NSA/FBI/whoever even disclose
exactly how effective these programs are? It's not just that the government is spying on us, it's that they are doing so opaquely, even to other parts of the government, with very little accountability. If "we the people" can't hold our government accountable then we really can't be called a democracy. Instead, citizens like Edward Snowden who are trying to hold the government accountable are being held accountable for revealing the truth.
It's sad when the U.S. government attempts to chase down a technology expert-turned-whistleblower across countries and lay off 90% of the NSA's system admins because of revelations of it spying on its own people.