ComputerBaseVirtual Reality Benchmarks

[pj-]

These raw frame-rate comparisons is akin to a VR experience where you stare at a scene static.

They don't measure the most important metric: motion to photon latency.

Tom's did a test last year on 3dMark's latency test (DX11) and it was awful, every GPU across the field got 35-50ms lag time from when you move your head, to when the scene is visually updated.

We still lack verifiable data and benchmarks for this latency that utilizes LiquidVR and GameWorks VR APIs.

The verifiable data is that the thousands of people who own or have demoed VR aren't barfing all over themselves.

If games were running with 35-50ms latency on the headsets that are coming out this/next month, the impressions you'd be seeing would be less "vr is the future" and more "wait when did I eat cheetos?"

The steam vr test can't measure motion to photon latency because there's no headset attached to the PC, nor is there a high framerate camera pointed at the headset screen. It's not an API test, that kind of thing doesn't need to be widely distributed. It's a simplistic "is my computer powerful enough?" test. It's also probably useful for valve to get statistics on the kind of computers owned by people interested in VR.

 

[Silverforce11]

This is comedy. AP covers GDC and asks all the tough questions that tech journalists are not asking (or hiding, rather, anything potentially negative).

http://bigstory.ap.org/article/dbb8...l-reality-creators-motion-sickness-real-issue

Oculus and Sony both posted health and safety warnings outside their booths on the GDC show floor, cautioning attendees trying the Rift and PlayStation VR that they may feel motion sickness, nausea, disorientation and blurred vision. Those effects were felt by many attendees.

Do you know what some developers are doing to counter this?

They are RESTRICTING the ability to change FoV. A more on-rails kind of experience.

"After a morning's worth of different Rift games, I felt disorientated, a touch nauseous and distinctly headachey," wrote Keza MacDonald on the gaming site Kotaku. "After five hours, I felt like I needed a lie-down in a dark room."

Would two hours in VR be too much?

"With the current technology, it's iffy," said Luckey (Occulus Dev). "It's all technologically solvable. It's not like we're saying, 'Oh no. We can't get any better. This is a dead end.'

I've been hearing it clearly when guys at OC and Valve have been telling each other and their peers, motion to photon latency is a huge deal breaker for many who does more than a short time-demo on VR. They all acknowledge it and are hard at work to solve it.

 

[Piroko]

The verifiable data is that the thousands of people who own or have demoed VR aren't barfing all over themselves.

I remain sceptical. There's a huge amount of positive bias with many tech journalists who want these things to succeed, some even publicly state that they don't want those products to get negative coverage (afair., LinusTechTips were one of those).

 

[Silverforce11]

I remain sceptical. There's a huge amount of positive bias with many tech journalists who want these things to succeed, some even publicly state that they don't want those products to get negative coverage (afair., LinusTechTips were one of those).

I'm sure it will succeed, though not to the level they hope for gaming, until they solve the problem. It will have a ton of applications outside of fast paced gaming in the immediate future.

Some people who are resistant to nausea will also have no issues with it.

 

[ShintaiDK]

(Some) People will be sick no matter what with VR. Simply because your balance center and the visual input doesn't match. Nothing cant fix this. Its a human limitation.

 

[pj-]

I've been hearing it clearly when guys at OC and Valve have been telling each other and their peers, motion to photon latency is a huge deal breaker for many who does more than a short time-demo on VR. They all acknowledge it and are hard at work to solve it.

Latency has not been a problem with these headsets for over a year. The problem is people trying to force games into VR with elements that can make people sick. Using an analog stick to move through a VR world would cause motion sickness for some people even if the motion to photon latency was 0.

One of the games I saw for Rift was basically F-Zero in VR. I don't know how anyone could play that without puking instantly.

(Some) People will be sick no matter what with VR. Simply because your balance center and the visual input doesn't match. Nothing cant fix this. Its a human limitation.

That is only true if the game uses artificial locomotion. I don't think I've heard of anyone getting motion sick in "room scale" games, where any camera movement corresponds directly to your head/body movement. Teleportation is currently the most common way to allow the player to cover large distances without causing sickness.

 

[Silverforce11]

Latency has not been a problem with these headsets for over a year. The problem is people trying to force games into VR with elements that can make people sick. Using an analog stick to move through a VR world would cause motion sickness for some people even if the motion to photon latency was 0.

Why are you ignoring what even Occulus is saying?

Wasn't that clear enough from a direct quote of their developers? 2 hours is not recommended by Occulus.

Tech journalists are saying they need to lie down after spending the morning with the Rift.

It's like some people just want to stick their head in the sand and ignore the biggest problem with VR.

Our brain just cannot cope with the lag time from when it KNOWs your head/eyes have moved, to when it actually sees it. Lots of research is being done on this topic because it's new territory.

 

[3DVagabond]

I'm sure it will succeed, though not to the level they hope for gaming, until they solve the problem. It will have a ton of applications outside of fast paced gaming in the immediate future.

Some people who are resistant to nausea will also have no issues with it.

I just think that these "minimum requirements" won't be enough. Weren't they saying you need ~90fps per eye to avoid the nausea? Now they are saying 40fps? I believe anyway. I can't recall precisely.

 

[Silverforce11]

I just think that these "minimum requirements" won't be enough. Weren't they saying you need ~90fps per eye to avoid the nausea? Now they are saying 40fps? I believe anyway. I can't recall precisely.

Valve said they can get it ok at 45fps, with their post processing to make it look 90.

VR for peasants basically, its not going to end well.

 

[pj-]

Why are you ignoring what even Occulus is saying?

Wasn't that clear enough from a direct quote of their developers? 2 hours is not recommended by Occulus.

Tech journalists are saying they need to lie down after spending the morning with the Rift.

It's like some people just want to stick their head in the sand and ignore the biggest problem with VR.

Our brain just cannot cope with the lag time from when it KNOWs your head/eyes have moved, to when it actually sees it. Lots of research is being done on this topic because it's new territory.

Why are you ignoring everything I said?

The reason some journalists got sick was because Oculus has a lot of games with unnatural locomotion. These are rated "intense" on their comfort scale.

When there is no locomotion, I have not seen any reports of people getting nauseous from VR. This is one of the great benefits of room scale VR.

This is the second result when I search for "htc vive nausea"
http://arstechnica.com/gaming/2016/01/man-survives-48-straight-hours-in-vr-with-no-reported-nausea/

The full context of that oculus founder (not a developer) luckey palmer quote is this:

Would two hours in VR be too much?

"With the current technology, it's iffy," said Luckey. "It's all technologically solvable. It's not like we're saying, 'Oh no. We can't get any better. This is a dead end.' We have tons of ways to make this higher resolution, lighter weight and more comfortable. Eventually, the goal is to make something that's not much heavier than a pair of sunglasses."

For many who've tried VR, it's not an issue at all. Hidden Path Entertainment founder Jeff Pobst said he recently spent 15 hours wearing the Rift headset while playing the VR version of his strategy game, "Defense Grid 2."

The implication is that two hours is uncomfortable for well known reasons such as eye strain from fixed focus on relatively low resolution screens or the weight of having something resting on your cheek bones.

Our brains absolutely can cope with the lag time and it's well understood that under 20ms motion to photon latency is good enough to trick the brain and induce "presence," where your brain buys that you are in the environment. Both vive and rift are under this threshold, rift especially is reported to be at around 7ms currently.

You are conflating the technology with its application. Assuming there was a magic zero latency headset, people would still get motion sickness in "intense" rated VR games (virtual roller coasters for example), because of the unnatural movement they contain. Games such as fantastic contraption or tilt brush will never have this problem.

If anyone is burying their head in the sand its the folks determined to prove for whatever reason that VR is a fad that comes and goes once every decade or so and this time is no different.

 

[Silverforce11]

Our brains absolutely can cope with the lag time and it's well understood that under 20ms motion to photon latency is good enough to trick the brain and induce "presence," where your brain buys that you are in the environment. Both vive and rift are under this threshold, rift especially is reported to be at around 7ms currently.

Absolutely not.

Where is this proof that its at 7ms for motion to photon latency?

That is not possible, even LiquidVR with Async Compute driving Compute Timewarp to speed up that latency is ~10ms.

When those technologies are not used, as Tom's found, motion to photon lag is 35-50ms. WELL ABOVE 20ms.

ps. I am not referring to frame rate or frame latency. If its 7ms for MPL, then Occulus and Rift will not have any concerns at all about motion sickness and they can stop any further research on it, call it in, mission success!

 

[pj-]

Absolutely not.

Where is this proof that its at 7ms for motion to photon latency?

That is not possible, even LiquidVR with Async Compute driving Compute Timewarp to speed up that latency is ~10ms.

When those technologies are not used, as Tom's found, motion to photon lag is 35-50ms. WELL ABOVE 20ms.

ps. I am not referring to frame rate or frame latency. If its 7ms for MPL, then Occulus and Rift will not have any concerns at all about motion sickness and they can stop any further research on it, call it in, mission success!

I don't remember where I saw the 7ms thing but I found this:
http://static.oculus.com/connect/slides/OculusConnect_Mastering_the_SDK.pdf

The time from the latest headset polling data is read and the timewarp is applied is < 7ms, at which time they immediately start presenting

I guess it was 10ms for DK2 whenever they made that presentation. I would assume it has gotten a bit better with SDK improvements and using a 90hz display instead of 75.

Edit: One thing about motion to photon is that it ignores prediction. Good prediction would put the perceived latency at 0, as the image will be rendered in the position where you were headed anyway.

 

[Silverforce11]

Looks like they are talking about the latency for vision tracking? ie. Yaw/Pitch and the resulting detection/tracking by their engine.

That's on a Mac Book Pro with 750M, there is absolute zero chance that it's sub 20ms motion to photon latency.

This is with Maxwell 2.

http://www.geforce.com/whats-new/ar...us-the-only-choice-for-virtual-reality-gaming

The standard VR pipeline from input in (when you move your head) to photons out (when you see the action occur in-game) is about 57 milliseconds (ms). However, for a good VR experience, this latency should be under 20ms.

Presently, a large portion of this is the time it takes the GPU to render the scene and the time it takes to display it on the headset (about 26ms). To reduce this latency we've reduced the number of frames rendered in advance from four to one, removing up to 33ms of latency, and are nearing completion of Asynchronous Warp, a technology that significantly improves head tracking latency, ensuring the delay between your head moving and the result being rendered is unnoticeable.

Combined, and with the addition of further NVIDIA-developer tweaks, the VR pipeline is now only 25ms. With further work and updated technology we expect consumer versions of VR to be even more responsive.

NV's best result is 25ms. This is actually lower than what Tom's Hardware found when they tested with actual measurement on the devices, 35-50ms.

 

[pj-]

That was september 2014..

I don't think ATW was supported at the GPU driver level until fairly recently

 

[Silverforce11]

That was september 2014..

I don't think ATW was supported at the GPU driver level until fairly recently

NV has not said anything since about improving that record 25ms. If so, I didn't see it.

Also, your Occulus PDF is talking about VR testing on a Mac, with a 750M. Do you honestly think it's 7ms Motion to Photon latency?

Haha come on. 7ms headset tracking lag would be good for that thing.

 

[pj-]

NV has not said anything since about improving that record 25ms. If so, I didn't see it.

Also, your Occulus PDF is talking about VR testing on a Mac, with a 750M. Do you honestly think it's 7ms Motion to Photon latency?

Haha come on. 7ms headset tracking lag would be good for that thing.

As long as the scene being rendered is simple enough to hit 90fps, of course I think it could do that.

 

[Silverforce11]

This is with the Rift, December 2015.

Measuring the actual photon display on the headset.

http://www.tomshardware.com/news/vrmark-virtual-reality-benchmark-preview,30820.html

Note that this does not actually fully measure the motion to photon latency, just the draw call submitted by the GPU to when the photon is output on the VR headset. This is already quite high. VRMark as I am aware, is not using LiquidVR or GameWorks VR.

When we add in the 7ms (should be improved in later hardware) head motion to engine tracking pickup, that latency rises. So there's an actual delay, you need to add in tracking time + rendering time (11ms at 90 fps), before you arrive at the draw call submission and finally the display output. Thus, NV and others have said, the typical pipeline is ~50ms. Way too high.

For VR MPL to get below 20ms, they would have to track the movement within 1ms or so. The GPU then gets the last frame, Async Timewarp it to update the FoV, then send it straight out on priority, bypassing the current frame 11ms/90 fps in the engine rendering latency (something GPUs without parallel engines cannot do! It gets stuck in traffic). On top of this, the VR headset needs to be extreme fast in processing/refresh rate.