A LOT of this will depend on the headphones you are using. While I'm certainly not buying into all of the "audiophile" stuff like 1000 dollar coax cables and the like, a decent dac and amp can definitely have an impact on performance. For example, my headphones that I use all day at work and on the train are Shure SE215's. Nothing crazy, but a leap better than the earbuds that come with phones. I just recently purchased a Chord Mojo USB DAC/Amp, and last night I was playing around with whether I could hear an absolute, put-my-finger-on-it difference, or if it was just "better" but in a hazy, whatever kind of way. I didn't test for very long, but I immediately noticed two things:
1. Sound floor. My laptop (Dell Latitude 13 9350) headphone jack has an incredibly audible hiss. It doesn't sound bad if you're used to it, but I went back and forth several times with no music playing, and it wasn't even a questionable thing, it was absolutely there. You could hear it in several quiet passages as I tested back and forth with some music.
2. Lows/Mids: The music I was playing (Snarky Puppy's "Something") uses a lot of drums percussion. I did some A/B testing on three different sections of drum fills, and was surprised that I could actually hear the trail off of the lower toms for significantly longer and for more notes (one, for example, was of a drum fill going from high toms to low toms). I could hear 5 notes on the low toms, with the last two notes being significantly quieter, but only 3 on the onboard.. Could be related to the noise floor, but I'll have to listen when I have more time.
Yes, this is a DAC and amp together, not just a DAC, but it IS worth noting the noise floor is probably related to DAC and not as much amp. I was definitely a doubter, but when I tried out this Chord Mojo I was surprised at things I heard on tracks that Ive listened to 20+ times and never heard before. I would definitely say that there is definitely diminishing returns (comparing a $200 DAC vs a $1000 one, you probably won't hear much) and there is a ton of placebo effect stuff in the audio world, but a lot of the stock stuff is pretty terrible on most peripherals (just look at keyboards and mice, for example).
Anyway, to answer your question, the answer is "it depends". Sound is subjective (mostly), and it depends on your headphones, your computer, and your ears.