Chuck Norris has as much as said, without actually saying it, that Bruce was an actor, and a very good martial artist, but not a fighter.
There's a recent interview where he was asked about Bruce, and how they met after Chuck won the world championship in karate in New York. Struck up a friendship, and went and trained together.
The interviewer asked Chuck if that meant they sparred, and Chuck said no, they practiced kicks and worked on the bags, etc....e.g., "trained".
Pressed about whether they fought or not, Chuck said "no, because I was a professional fighter....", and the interviewer says "and he was not", and Chuck says with a nod, "yeah", then throws in "but he was very good".
Then asked "so you would have beaten him", and Chuck said he didn't want to go there, and neither did Bruce.
So he was being diplomatic, but it's obvious from how Chuck said it that he thinks he would have beaten Lee, no problem.
http://www.youtube.com/watch?v=Mcl81ldNODc&feature=related
And I believe Gene Lebell embarrassed Lee on the set of Green Hornet, which led to Lee learning from Lebell and their friendship.
On a side note, Gene Lebell also choked out Steven Seagal and Seagal shit his pants, literally, when he went to sleep. One BAD old dude.