Microstutter: The Poll

Discussion in 'Video Cards and Graphics' started by Hitman928, Dec 19, 2012.

?

Was there microstutter?

  1. Yes, but A was the smoothest

  2. Yes, but B was the smoothest

  3. Yes, but C was the smoothest

  4. Yes, but D was the smoothest

  5. Yes, and A was the worst

  6. Yes, and B was the worst

  7. Yes, and C was the worst

  8. Yes, and D was the worst

  9. Yes, they were all about the same

  10. I couldn't see any microstutter

Multiple votes are allowed.
Results are only viewable after voting.
  1. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Ok, so out of curiosity, I put together a little sample of gameplay from Max Payne 3. I recorded a run through of each section and then ran through each section again to benchmark with fraps.

    As I don't have the full version of fraps, I split each run into two sections so that I could get a bigger sample. There is no correlation in the order of the videos to the order of the fraps results, they have been jumbled so that you don't know which fraps graph belongs to which video, only I do. All of the frame time graphs are scaled the same (each grid line represents 5 ms), however, the axis labels have been removed and not all have an origin of zero, so you cannot determine the frame rate by the frame times and cannot compare the frame rate between graphs.

    Each run represents a change in settings/hardware (not saying which run is which until the end). With each run I tried to duplicate the same pattern each time. Each run is not exactly identical but are as close as I could make them. The videos were recorded at 1/2 size but 60 fps, so I recommend actually downloading the source videos and not watching at full screen to compare. This is not a perfect setup, but seeing as how I don't have a high speed camera, it's the best I could do. This is NOT meant to be conclusive or to thoroughly prove anything. I just thought with all the microstutter debate going on, this could be another angle to look at things. Mostly it's an idea that I hope others who can perform a more thorough test here could pick up on.

    So, please watch all the videos, vote honestly in the poll, and then, post which videos you think line up with which frame times. If you can't tell, or only want to guess on a particular video, that's fine too. So for instance, if I thought the 'A' videos lined up with the frame times 1-1 and 1-2 graphs and the 'B' videos lined up with the 2-1 and 2-2 frame times, I would post A with 1, B with 2, etc.

    Thanks for playing :)

    Videos:

    A1 - https://docs.google.com/open?id=0B11hajGAJWuVVmZmNVFxT3VVZ2M
    A2 - https://docs.google.com/open?id=0B11hajGAJWuVV0Z5b3dGUDRqOVE
    B1 - https://docs.google.com/open?id=0B11hajGAJWuVSV9TV0thcXB1d2s
    B2 - https://docs.google.com/open?id=0B11hajGAJWuVRU51eHRfa2dZc2M
    C1 - https://docs.google.com/open?id=0B11hajGAJWuVQlRpYlN1VDdyVFE
    C2 - https://docs.google.com/open?id=0B11hajGAJWuVVHplcEQ4WGxOeWM
    D1 - https://docs.google.com/open?id=0B11hajGAJWuVN2wtNi0ybVE1N1k
    D2 - https://docs.google.com/open?id=0B11hajGAJWuVdXYwV3p2WVlBQmM
     
    #1 Hitman928, Dec 19, 2012
    Last edited: Dec 19, 2012
  2. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Frame times:
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
     
    #2 Hitman928, Dec 19, 2012
    Last edited: Dec 19, 2012
  3. VulgarDisplay

    VulgarDisplay Diamond Member

    Joined:
    Apr 3, 2009
    Messages:
    6,194
    Likes Received:
    1
    This is how these frametime reviews should be conducted. Blind studies with many people providing the results. I'm giddy to see what happens.
     
  4. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Thanks for the encouragement, I figured, if we're really gonna do this, we might as well do it right. This definitely isn't the end game, just trying to get the ball rolling in the right direction. Plus, I'd love to see how people vote, if they are willing to put themselves out there ;)
     
  5. boxleitnerb

    boxleitnerb Platinum Member

    Joined:
    Nov 1, 2011
    Messages:
    2,596
    Likes Received:
    1
    Good idea in principle, but I will not vote because:

    To watch is one thing, to actually play and feel the feedback and smoothness (or lack thereof) is another altogether, especially when differences are not massive and blatantly obvious. I therefore cannot guarantee that my judgement would be accurate enough and in line with the gameplay experience.
     
  6. Final8ty

    Final8ty Golden Member

    Joined:
    Jun 13, 2007
    Messages:
    1,150
    Likes Received:
    0
    They all looked bad to me, sorry.
     
  7. VulgarDisplay

    VulgarDisplay Diamond Member

    Joined:
    Apr 3, 2009
    Messages:
    6,194
    Likes Received:
    1
    Are you playing with a controller? It's making my eyes bleed.
     
  8. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Lol, nope kb/m. I tend to be a bit jerky with a mouse though, sorry about that. I tired to be as smooth as I could, but it's just how I play. I think it's more pronounced because of the third person perspective, but there's plenty of slow mo and game controlled camera sequences to compare as well, that's why I picked MP3. I could do it agin with a fps, it would probably be a lot less influenced by input "jerks". I also found that the transitions in and out of slo mo in this game can be a bit jarring, no matter what the setting.

    Make sure you're dl'ing them as well because the streaming quality sucks.
     
    #8 Hitman928, Dec 19, 2012
    Last edited: Dec 19, 2012
  9. thilanliyan

    thilanliyan Diamond Member

    Joined:
    Jun 21, 2005
    Messages:
    8,884
    Likes Received:
    2
    Thanks for your effort. When I get a chance I will participate...although I don't think I would be able to notice many differences especially since looking at the frametime graphs, there are no consistent spikes.
     
  10. VulgarDisplay

    VulgarDisplay Diamond Member

    Joined:
    Apr 3, 2009
    Messages:
    6,194
    Likes Received:
    1
    Yeah, there are hardly any spikes above 30ms which is I believe what is being said to be the threshold for noticeable stutter.

    One thing I've decided is that google docs is slow. 500kb/s is prehistoric.

    I honestly can't tell the difference. They all have jerky motion, but I think it's just character animations glitching for the most part. It doesn't look like a smooth game in general, but I haven't played it myself so I can't comment.
     
    #10 VulgarDisplay, Dec 19, 2012
    Last edited: Dec 19, 2012
  11. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Yeah, the animations are a bit jerky in general, probably a generation behind most games today, watch the trailer and you'll probably see the same thing you're seeing in the video. It's unfortunate because everything else in the game looks great. Maybe I'll pick a different game, a smoother game in general and repeat this. Could be easier to spot differences. Any suggestions?

    edit: yeah, next run, switching to something else than google, I figured I would try it since I had it so easily available, but there are other, better options.
     
  12. Ben90

    Ben90 Platinum Member

    Joined:
    Jun 14, 2009
    Messages:
    2,866
    Likes Received:
    0
    After playing some 125FPS Quake Live and watching the second clip of each series here is my (edited) response:

    A2: Stutter
    B2: Stutter
    C2: Stutter
    D2: Stutter

    I voted "Yes, they were all about the same"
     
    #12 Ben90, Dec 19, 2012
    Last edited: Dec 19, 2012
  13. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Just out of curiosity, did you download them or just watch them in browser?
     
  14. Elfear

    Elfear VC&G Moderator
    Super Moderator

    Joined:
    May 30, 2004
    Messages:
    6,464
    Likes Received:
    1
    You mind posting without the expletives next time?


    OP - Thanks for taking the time to record all the videos. I'll vote when I get home.
     
  15. WaitingForNehalem

    WaitingForNehalem Platinum Member

    Joined:
    Aug 24, 2008
    Messages:
    2,499
    Likes Received:
    0
    Agreed, please refrain from using the Lord's name in vain. It is almost as if you are purposely trying to be disrespectful since you used it in all four scenarios.
     
  16. ICDP

    ICDP Senior member

    Joined:
    Nov 15, 2012
    Messages:
    709
    Likes Received:
    0
    Thanks for posting these Hitman. Video A looked the worst for me, though it was hard to tell as all looked to have low FPS IMHO. Honestly it doesn't look like microstutter is being observed from your graphs, might just be just low FPS.
     
  17. ICDP

    ICDP Senior member

    Joined:
    Nov 15, 2012
    Messages:
    709
    Likes Received:
    0
    I consider having a passage from a religious book for your signature line to be disrespectful. This is an international forum and is not frequented exclusively by Christians. I agree he or she should be more respectful of other peoples cultures and beliefs, so should you. It is hypocritical of someone with a religious message in his/her sig to moan about being disrespectful IMHO.

    Please don't say "this is an American site", there is no such thing when it comes to an internet site that is aimed at an international audience.

    Enough off topic though.
     
    #17 ICDP, Dec 19, 2012
    Last edited: Dec 19, 2012
  18. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    I'll let you know, it's not low fps. I think I'm going to put this on hold for now though and redo this with a game with better animation and tweak a few things in my process. Looking at the game itself and the videos, there's just a lot of jerkiness in a lot of areas that don't go away whatever the settings. This game is known for really stiff animations though. I'll work on a new set of videos today and should have somthing better by late tonight or tomorrow. Go ahead and view/vote if you want in the mean time.
     
  19. ICDP

    ICDP Senior member

    Joined:
    Nov 15, 2012
    Messages:
    709
    Likes Received:
    0
    Great work and thanks for doing this. I know it is a very labour intensive process and do appreciate it. Good idea doing posting the iharts without GPU details, it makes the test blind.
     
  20. WaitingForNehalem

    WaitingForNehalem Platinum Member

    Joined:
    Aug 24, 2008
    Messages:
    2,499
    Likes Received:
    0
    How is having a Bible verse disrespectful? I'm not sure how that could be taken maliciously. Everyone is free to believe in what they choose and express their beliefs in a respectful, peaceful way. I don't see anything wrong with that.

    But I agree, enough digression, back to the topic at hand.
     
    #20 WaitingForNehalem, Dec 19, 2012
    Last edited: Dec 19, 2012
  21. ICDP

    ICDP Senior member

    Joined:
    Nov 15, 2012
    Messages:
    709
    Likes Received:
    0
    Do you not see the irony in this statement on an international forum? I will make it simple for you, because not everyone on this planet is a Christian and believes the Bible. There are Muslim, Hindu, Atheist, Buddhist etc. I won't tell you what beliefs I have and I certainly wouldn't put it in a sig on an international forum, because that would be disrespecting other peoples beliefs.
     
    #21 ICDP, Dec 19, 2012
    Last edited: Dec 19, 2012
  22. lakedude

    lakedude Golden Member

    Joined:
    Mar 14, 2009
    Messages:
    1,611
    Likes Received:
    0
    This^

    The input is way too herky jerky to really get a feel for microstutter.
     
  23. Greenlepricon

    Greenlepricon Senior member

    Joined:
    Aug 1, 2012
    Messages:
    468
    Likes Received:
    0
    Maybe my eyesight really is pretty bad but they all look decent to me. I do notice some of the stutter but I tend to play jerky too and am not that picky as long as it's relatively playable by my standards. Also I'm on my laptop which doesn't help the quality right now. I'll check again on my desktop and see if my opinion changes at all.
     
  24. blastingcap

    blastingcap Diamond Member

    Joined:
    Sep 16, 2010
    Messages:
    6,653
    Likes Received:
    1
    Not to mention that FRAPS records things in the middle of the graphics chain, not at the end like a high-speed video camera would. So all this frametime talk is useless without a high speed camera to back it up.

    More info here about why FRAPS is insufficient:

    http://techreport.com/review/21516/inside-the-second-a-new-look-at-game-benchmarking/11
     
  25. Hitman928

    Hitman928 Senior member

    Joined:
    Apr 15, 2012
    Messages:
    991
    Likes Received:
    2
    Yes, a high speed camera would be best . . . unfortunately I don't have one. Part of this experiment is to see just how useful, if at all, fraps is to track microstutter.