There's talk that smartwatches did hurt the watch market, but primarily the 'entry' market (i.e. the Fossils and Swatches of the world) rather than the Patek Philippes and Omegas. Beyond that, it's more likely to be economic factors at work.
Just remember that we're at the start of a pretty long road for a relatively new industry, assuming smartwatches persist. If you'll recall, the iPhone sold just 1.1 million units in its first full quarter -- it wasn't an immediate fiscal threat to the incumbents. The real danger was over the long term, as Apple took its good critical reception and built up an audience.
There are estimates that Apple will have sold 10 million Watches by the end of 2015. That's not a huge amount, but what happens when there's a second-gen Watch in 2016 and sales amplify? What if Android Wear has its Droid moment (i.e. that moment when it finally captures the public interest) and gets stronger sales as well? They won't seem like niche devices then.
The concern isn't that mechanical watches will quickly be displaced. It's that the smartwatch market will eventually become large and sophisticated enough that it hits the mainstream, changing watch culture in the process. Smartphones used to be reserved purely for businesspeople and geeks, and now just about everyone uses them. What if young adults in 2025 are so accustomed to smartwatches being around that they aspire to owning an Apple Watch or Huawei Watch one day, rather than a Rolex like their parents had? What does Switzerland do if it no longer defines the watch industry?