With that logic, does anything matter?
Truth is, its up to the user to decide what they want from a game. To some, graphics are everything. To others, graphics are nothing. And many do not buy graphics cards exclusively to play games at the maximum settings... Lots play to ensure optimal performance at all times. For example, as a hardcore Counter Strike player, all my settings are on low to reduce clutter, but a graphics card is still necessary to run games at 200+ FPS to ensure a consistent framerate.
To most, its a combination of both. Nobody wants to suck at playing games due to terrible framerates. Furthermore, nobody wants to play a butt ugly shell of a game just to ensure they stay competitive.
And for the record, some seem to have gotten themselves to believe that a story in a game is told exclusively through audio, and that at the end of the day a graphics card really is not necessary. And that may have been true, 20 years ago, but this is 2015. Graphics are widely used to tell stories in the few games that actually still have stories to tell.