werepossum, I would have very much appreciated a reference. Allow me to answer every point raised.
Foremost, I would like to say, Lying is prohibited in Islam, except in certain cases. The "except in certain cases falls under the concept of Taqiyya". This is not a novel concept. We lie everyday in certain circumstances. Islam just provides a name to it, so that it does not weigh as a massive sin.
Taqiyya is definitely an Islamic concept. However, to claim that it is a "widely-practiced art" is quite incorrect. I would like to see some substantiation to that claim. The second claim of denying something in English and promoting it in another language is also a claim without substantiation. Please provide substantiation, so that either I can address it or condemn it as the case may be.
The hadith is quite irrelevant to the invasion of Mecca. The history of the treaty and the invastion of Mecca can be found here:
http://en.wikipedia.org/wiki/Treaty_of_Hudaybiyyah
The scope you are referring to is a misrepresentation. Lying to benefit self-esteem is permissible under Islam in cases like where if your wife asks you if she is fat, you say, no darling, you arent. Its not meant to lie by saying stuff like "I make $1 million dollars", just so you boost your self-esteem.
The Lying for "protection of faith" is with respect to situations where your life depends on your religious belief. As we all know, history is rife with examples of where your life was in danger based on your faith. Islam allows you to lie about your faith when a gun is pointed to your temple. This is simple common sense.
As you stated, the broad interpretation of Taqiyya is not the majority view. Can you please provide clarify to which Sunni clerics shun Taqqiya and also endorse the practice in another term?
I am not sure the context to the praiseworthy goal hadith that you quote. If I have time, I will definitely try to educate myself about it and come back with a educated response. I can certainly tell you that no Muslim thinks that Sharia and Islamic clerical rule can be only achieved by lying. Most Muslims are not even in support of Islamic clerical rule.
Hope this helps.