I can only see one advantage to moving a "big" and "stationary" console to an ARM architecture, and that's to make cross development with mobile games easier. That is a very low priority there because mobile games have rather major differences in interface. The Switch (and the shield/portable that shares much of it's hardware) is an odd situation. They needed to be good both in a fixed installation situation and a mobile one as well. For them, the ARM base architecture made sense as the implementations that were available were highly focused on the mobile market and featured aggressive power efficiency capabilities while still maintaining good performance.
Can an ARM architecture that's optimized from the ground, through the process, all the way to the board be an effective main console? Yes, it's possible. The problem is getting the whole platform optimized for performance first, and then getting the games developers on board. Having Nintendo laying the foundations for developer houses to have those capabilities is a big deal to be sure, but Nintendo also has a long history of wanting exclusivity and focusing on their own in house properties over third party ones. I'm sure that, for the next major generation of consoles, there will be rumors going around that MS and Sony are nosing around ARM based designs. I also don't think that it will amount to much more than keeping AMD and Intel (should the choose to bid for the project, which they might with a renewed focus on GPU performance) competitive on their pricing. I do think that Nvidia, in a continuing effort to diversify their portfolio to protect against renewed focus on APU performance from both AMD and Intel destroying their volume low end business, will present an updated shield at some point that will have impressive performance capabilities for a modest platform price. They will, in combination with Nintendo, attract a few developers to put together nice games for them. Where that goes long term is anyone's guess.