They were doomed from the start. Didn't matter if they concentrated entirely on the ST/Amiga, there was no way they could compete with the PC. For the ST, its main advantage was price. Sure, its graphics capability at first was better than most PCs, but they caught up quickly after a few years, and then surpassed the ST. But the problem with a cheap computer is that a lot of people that buy one will then not want to buy a lot of expensive software. Therefore, not a lot of developers were willing to write software for the machine. And the software available was often not the highest quality. And piracy made the situation much worse. I still remember buying WordPerfect for the ST. It was one of the few serious business apps for the ST, and it was buggy as hell. They eventually fixed many of the bugs, but I doubt any company would be happy with the initial release.
Same for a lot of other programs, even games. I remember buying SubLogic's Jet, and it was a POS. Average framerate was around 1-2 FPS (that's not a typo). Meanwhile, on the PC, you could buy Falcon AT and later Falcon 3, Wing Commander, Strike Commander, etc. All way better than anything available on the ST. But since sales weren't high enough to justify investing a lot of effort in producing software for it, the machine was eventually ignored. And since there was so little quality software, no one in their right mind would buy the machine no matter how good the hardware (which was not great to begin with.)
The other major issue with the ST and Amiga was that they were integrated machines. That meant Atari and Commodore had to do basically all the R&D. The PC, on the other hand, was modular and expandable. So individual companies could produce parts that were better than what was available on the ST or Amiga. Does the PC have poor graphics? Well, individual companies could just concentrate on making better video cards, and they could be swapped into your PC in minutes. Bad audio quality? Sound card makers soon fixed that. And of course, once the Voodoo was released as an add-in card, that triggered an arms race for 3d graphics that just dwarfed anything the ST or Amiga could do. Same for storage. Same for CPU. Same for memory. And since there was such a large installed base of machines, these companies could actually sell enough to make enough money to put back into R&D, allowing even better versions in the near future.
I upgraded both the OS and memory on my ST. It involved de-soldering the RF shield, prying the individual chips out of the motherboard (an IC puller would have been useful, but since I didn't have one, a screwdriver had to do.), then press the new chips back in, all the while hoping not to bend any of the pins. User friendly it was not. Imagine if we had to upgrade our OS by swapping a ROM chip rather than installing software? It would be a nightmare for pretty much everyone.
So combine a small installed base that couldn't justify 3rd party software support, being responsible for the R&D of the entire machine, basing your machine on a family of CPU that couldn't compete with Intel, and lack of easy expandability and modularity, and the ST was doomed to be a niche machine.
I kept mine for almost 10 years. It served me from the final year in high school all the way through medical school. I couldn't afford to buy a new machine, so I had to live with it while trying not to be jealous of all the games being released for the PC. I almost broke down and bought a PC when I saw Doom for the first time, though. Maybe it was a blessing in disguise that there weren't any good games for the ST, as it meant I could concentrate on my studies.