I was "Executive Platinum" level on American for a number of years due to business travel. (100K+ miles/year)
Alot of what was said above is true. At least 90% of 'upgrades' go to frequent flyers. Once you get to 'elite' status (25K+ miles/year on most airlines) it is easier. You can always ask at the gate and if they are nice and 1st isn't full they will bump you. If you want your bump confirmed, you can request it when you reserve your flight but you are gonna give up something. American gives so many upgrades per 10K miles you fly depending on what level you are at. These are free and in addition to earning your regular miles. Each upgrade is worth a 500 mile flight segment. After that, you can buy upgrades at $25 each. These things may have changed or gone up recently.
If the flight is not full, it is usually easy to get bumped if you are flying on a full-fare ticket - just ask.
Anyway, the thing to remember is the gate attendant is god here. The check-in/baggage check person technically can bump you but they normally will get heat if they do. Same thing goes for flight attendants once you have taken off unless the flight is *really empty.
Gate attendant has the power.
There are exceptions of course, but this is what I've noticed over the years.
That being said, I only fly rarely now that I changed jobs and I've lost my 'elite' status. I dread the next time I have to fly. I haven't flown coach in at least 7 years. Scary 😱