Have ramdisks matured enough for gaming yet?

[I4AT]

Say you have 16GB+ RAM with ~4GB dedicated to OS/game engine, there's still plenty left over to load most, if not all, game data into a ramdisk.

I've seen a few threads discussing this but they're mostly a few years old and larger quantities are cheaper and more readily available these days.

I'm guessing games aren't optimized for this at all, so how do you tell it to read from the ramdisk, but still write save files etc. to a hard disk? Also, is it possible to 'stream' data I guess you could say, so while the game is running data can be moving into the ramdisk in the background, rather than loading everything before launching the game?

Most SSD's can reach ~30MB/s 4KB random reads on the lower end, so to fully pre-load a 10GB game might take, at worst about 5 minutes, but realistically be much faster right?

How much longer would it take with a mechanical disk? Or is your average SSD fast enough that it's only worthwhile to use a ramdisk with a mechanical drive?

 

[BFG10K]

A RAM disk is just like a standard volume, the only difference being its contents are lost after it’s powered down. It doesn’t do any of those things you mentioned.

Games run exactly the same as they would off a regular drive. Saves are written wherever they usually go, typically My Documents, App Data, or similar. Older games may write to the game’s directory itself.

Here are some load times I tested across multiple drives:

“RAM” is from the disk cache and should be identical to a RAM disk. As you can see, it’s usually not much faster than the SSD (“Intel”). Also those benefits are only apparent the first time you load a level; subsequent reloads come from the disk cache, so the I/O performance is no longer a factor.

I personally wouldn’t bother with RAM disks for games, especially if you already have an SSD.

 

[I4AT]

But if you create a disk in RAM doesn't the data still have to be read from the hard drive before it can be "moved"? Which I'm assuming happens at the maximal read speed of the said drive.

I'm a little confused then about the differences, loading a game means all the assets are being read from the drive into memory, right? And hard drives are obviously slower than SSD's, but shouldn't RAM have read speeds of at least 3-5x that of an SSD? If you have the entire game directory pre-loaded into a ramdisk shouldn't loading be near instantaneous or at least significantly faster than an SSD? With the exception of online games where the network becomes the bottleneck.

I forgot that most newer games write saves to directories outside of the install folder, but what if you have an older game that doesn't? What would be the workaround? Or does the game still write to the C: drive regardless of whether you're reading from a ramdisk?

 

[BFG10K]

But if you create a disk in RAM doesn't the data still have to be read from the hard drive before it can be "moved"? Which I'm assuming happens at the maximal read speed of the said drive.

Again, a RAM disk is exactly like any other volume, except its contents are lost after it’s powered down. So yes, it’s your responsibility to get the data in there, and to save it before it’s lost.

You could do this with startup/shutdown scripting, or you can manually copy the data yourself.

It’s also possible that using hibernate instead of boot/shutdown would preserve it, but you’d still be responsible for managing the data if you ever wanted to replace it with another game.

I'm a little confused then about the differences, loading a game means all the assets are being read from the drive into memory, right? And hard drives are obviously slower than SSD's, but shouldn't RAM have read speeds of at least 3-5x that of an SSD? If you have the entire game directory pre-loaded into a ramdisk shouldn't loading be near instantaneous or at least significantly faster than an SSD?

Games aren’t completely bottlenecked by I/O. The CPU needs to decompress archives, GPU drivers need to upload assets, etc.

Once your I/O is fast enough, you’ll experience no performance gain because there are sequential bottlenecks elsewhere. See the “RAM” score in my graphs? I could increase my I/O performance by 50,000 times, but the load times would never get any faster.

 

[Karsten]

Very few games read a lot of data from the drive once the initial read is complete. Even if they do the read is mostly very small. To go trough the trouble of a RAM drive, the potential for issues with power and such is just not worth it.
Better to get a SSD and use the build in game relocation tool within STEAM, or the like.

 

[I4AT]

I never thought about decompression, I guess that would explain the lack of any benefit between an SSD and a ramdisk in certain games where the CPU would become the bottleneck.

So assuming there was no decompression, unpacking, etc. involved, loads would probably occur in just a few seconds, right? I can't imagine the GPU doing its work should have any noticeable impact or 3D engines simply wouldn't work, because when you move from one position to another in a 3D world all those assets are initialized and drawn on the fly pretty much seamlessly, if that makes sense.

So theoretically, we could decompress/unpack files first, and then move them into a ramdisk and see a pretty significant performance boost? Now that I think about it, I think this used to be done when Doom 3 first came out.

This would probably have a larger effect on open-world/seamless games than level-instanced ones.

 

[Karsten]

I never thought about decompression, I guess that would explain the lack of any benefit between an SSD and a ramdisk in certain games where the CPU would become the bottleneck.

So assuming there was no decompression, unpacking, etc. involved, loads would probably occur in just a few seconds, right? I can't imagine the GPU doing its work should have any noticeable impact or 3D engines simply wouldn't work, because when you move from one position to another in a 3D world all those assets are initialized and drawn on the fly pretty much seamlessly, if that makes sense.

So theoretically, we could decompress/unpack files first, and then move them into a ramdisk and see a pretty significant performance boost? Now that I think about it, I think this used to be done when Doom 3 first came out.

I don't think you will find a lot of games that let you "decompress" their files ahead of time.

 

[PrincessFrosty]

Good question, my last build included 16Gb of RAM, primarily because it's just so cheap at the moment it was silly to pass up. Recently I started tinkering with the idea of RAMdisks especially in the context of gaming.

With 16Gb of RAM installed, my OS + game usage never seem to exceed 6Gb usage so I dedicate 10Gb of RAM to a RAMdisks. I'm using PC12800 RAM, which means theoretical peak speed of the drive is 12.8GB/sec, that's 25x faster than my OCZ Vertex 3 SSD, although for several reasons the real world speeds are slower.

I use ImDisk http://www.ltr-data.se/opencode.html , it's basically just a free and very lite disk driver, it comes with a basic control panel applet for easy configuration but also allows command line configuration which is handy for automating RAMdisk creation. For those who don't know RAM is volatile memory, and you lose all the data stored on a RAMdisk when the RAM loses power.

So my general idea was this; when the computer boots have it run a script to automatically create a RAMdisk and copy my favourite game installation files to that disk. When I later come to load that game it will load faster, essentially you're pre-caching these files at some earlier time which is convenient to you and mitigating loading times later on. It works especially well if you do not shut your PC down often, however if you regularly reboot, for example you only boot your PC to play some games, this would be much less useful as the total time to copy from Disk -> RAMdisk -> RAM is longer than just a single load from Disk -> RAM.

I created a batch file to automate this with ImDisk, if you want to test this for yourself this will help you get started:

1) Download and install ImDisk
2) Create a new script file, right click on your desktop, select "new" and select "text document", then give it the name RAMDISK.bat (make sure you have file extensions showing)
3) Right click the new batch file and click "edit"

In the batch file you want to do several things, first create the RAMDisk, you do this by calling the imdisk.exe file with various parameters to define the type and size of the disk, mine looks like this:

[b]imdisk.exe -a -t vm -o rw -s 10G -p "/fs:ntfs /q /y" -m Z:[/b]

This calls imdisk.exe
[b]-a[/b] attaches a new disk
[b]-t[/b] is the type and [b]vm[/b] means its made in virtual memory
[b]-o[/b] is the option flag and [b]rw[/b] means it's both read/write enabled
[b]-s[/b] is the size [b]10G[/b] means 10Gb, 500M would be a 500mb drive
[b]-p[/b] is the format options [b]"fs:ntfs /q /y"[/b] sets the file system to NTFS
[b]-m[/b] is the mount letter for the drive in this case I picked [b]Z:[/b]

The next line in the batch file we can copy files to our new RAMDisk, for example I copy Battlefield Bad Company 2 for testing.

[b]xcopy "C:\Games\BFBC2" "Z:\" /E /Q /H[/b]

This uses xcopy
[b]"C:\Games\BFBC2"[/b] is my games installation directory
[b]"Z:\"[/b] is the target RAMDisk, make sure this matches your mount point above
[b]/E[/b] forces the copy of all subdirectories
[b]/Q[/b] stops the file copying message
[b]/H[/b] forces the copy of hidden and system files (if they exist)

Once you've edited your parameters save the batch file.

Some tips:
1) When running it make sure you run it as administrator, the format step of RAMDisk creation requires admin elevation and will fail without it.
2) You can use task scheduler to run this batch file at system boot, or whenever you prefer.
3) Do not try and create RAMdisk greater than the amount of Physical RAM installed
4) Do not try and copy a game directory bigger than your RAM disk
5) Investigate if games extract temporary files to their installation directory when running, the size of the installation directory may grow when in use, make sure to allow for this when picking your RAMdisk size
6) RAM DISKS DO NOT SAVE DATA PERMANENTLY!! If you reboot or lose power to your PC whatever is on the RAMDisk is gone forever with zero chance of recovery, never EVER store valuable data here under any circumstance. With respect to running games, make sure single player games are storing saved games into somewhere outside of the RAMdisk, thankfully a lot of modern games store user settings and save games on a per user basis and these get put in your C:\users\<account>\documents\, but always double check first otherwise you risk losing it!

I've not done extensive testing yet, I did very basic test with online gaming using BFBC2 and found almost no loading speed benefit when compared to my SSD, this suggests to me that the bottleneck with this specific game is the CPU. Ideally I need to get this working with steam games since most of my games are on steam, I would be interested to see if anyone can use this method to get much faster loading times and if so, with what games.

*edit*

Ugh, smileys

 

[BFG10K]

So assuming there was no decompression, unpacking, etc. involved, loads would probably occur in just a few seconds, right?

Possibly, but then the other drives would also get faster. So a RAM disk might be 1 second, an SSD might be 3 seconds, and a VelociRaptor might be 5 seconds.

Again, that&#8217;s not worth messing around with a RAM disk. You&#8217;ll spend a hundred times longer administering the thing.

I can't imagine the GPU doing its work should have any noticeable impact or 3D engines simply wouldn't work, because when you move from one position to another in a 3D world all those assets are initialized and drawn on the fly pretty much seamlessly, if that makes sense.

When you load a level, the GPU uploads all textures/models/shader programs/etc that are used in that level. This forms a large chunk of the load times above, and it still happens even when loading from a disk cache/RAM disk.

Afterwards each frame is simply a rasterized version of the content cached in the VRAM, updated to factor in changes in the game world. That's why it's seamless.

Some games always stream certain data, but it&#8217;s usually low enough that most HDDs can easily keep up.

So theoretically, we could decompress/unpack files first, and then move them into a ramdisk and see a pretty significant performance boost? Now that I think about it, I think this used to be done when Doom 3 first came out.

Maybe, but decompressing the files will cause a lot more little files to start existing instead of a few big archives, possibly offsetting some of the savings with decompression

As a general rule, sequential I/O is easier to perform than random I/O, assuming the data doesn&#8217;t have significant fragmentation of course.

 

[I4AT]

Hmm, so if games typically aren't even I/O bound when using an SSD, why do we have RAM that's capable of such speeds? What tasks are the OS and browsers doing that takes advantage of all that extra speed that not even a game utilizes?

Good question, my last build included 16Gb of RAM, primarily because it's just so cheap at the moment it was silly to pass up. Recently I started tinkering with the idea of RAMdisks especially in the context of gaming.

With 16Gb of RAM installed, my OS + game usage never seem to exceed 6Gb usage so I dedicate 10Gb of RAM to a RAMdisks. I'm using PC12800 RAM, which means theoretical peak speed of the drive is 12.8GB/sec, that's 25x faster than my OCZ Vertex 3 SSD, although for several reasons the real world speeds are slower.

How long does it typically take for 10GB of game data to be copied to your ramdisk from your SSD? What about from a mechanical drive, if you have one?

 

[nanobreath]

Hmm, so if games typically aren't even I/O bound when using an SSD, why do we have RAM that's capable of such speeds? What tasks are the OS and browsers doing that takes advantage of all that extra speed that not even a game utilizes?

This timing is completely arbitrary, just trying to get the point across on why ramdrives aren't as useful anymore, and why ram is still needed.

[Compressed data on SSD copied to Ram] - 3 secs
[Compressed data on ram, decompressed] - 10 secs
[Uncompressed data is fed to and rendered by video card] - 20secs
Total time - 33secs

Rendered and uncompressed data is now on ram and ready for access by the game. Anytime the game needs this data it can now access everything in milliseconds. Now compare this to using a ramdrive:

[Compressed data on Ramdrive copied to Ram] - <1 secs
[Compressed data on ram, decompressed] - 10 secs
[Uncompressed data is fed to and rendered by video card] - 20secs
Total time 31secs


See how much time you saved? It took you milliseconds to copy data off your ramdrive, but this is a small fraction of the time needed to "load" a game for playing.

Now this loading of a game is completely different than RUNNING a game. Once the game (or any program for that mater) is running it looks something like this.

You enter a room that has 1000 assests associated with it that are pre-rendered and need to be read and displayed.

With this data sitting in RAM you have this:

[Load assest 1] - 1/1000th of a second
[Load assest 2] - 1/1000th of a second
[Load assest 3] - 1/1000th of a second
[Load assest 4] - 1/1000th of a second
[Load assest 5] - 1/1000th of a second
Etc for 1000 assets, it takes seconds to load and display the room you entered.

Now without RAM and just a SSD :
[Load assest 1] - 1/5th of a second
[Load assest 2] - 1/5th of a second
[Load assest 3] - 1/5th of a second
[Load assest 4] - 1/5th of a second
[Load assest 5] - 1/5th of a second

Etc for 1000 assets, it now takes several minutes to load and display the room you just entered. RAM is a REALLY good idea.

Hopefully this clears some things up for you.

 

[PrincessFrosty]

Hmm, so if games typically aren't even I/O bound when using an SSD, why do we have RAM that's capable of such speeds? What tasks are the OS and browsers doing that takes advantage of all that extra speed that not even a game utilizes?

RAM is the speed it is to make sure the CPU does not have any data read/write bottlenecks when doing calculations. The CPU reads values from RAM, does operations on them and writes the results back to RAM.

How long does it typically take for 10GB of game data to be copied to your ramdisk from your SSD? What about from a mechanical drive, if you have one?

My OCZ Vertex 3 120Gb SSD (on SATA III) just copied my entire 8.2Gb BFBC2 directory to a RAMDisk created with ImDisk, in slightly less than 30 seconds.
My WD Green 1Tb HDD (SATA II) copied the same directory in about 2 minutes 15 seconds

When copying to/from a RAMDisk using a HDD or SSD the bottleneck is the drive itself, a spindle drive will be a fair bit slower, there is no speed advantage copying from SSD/HDD to the RAMdisk.

 

[PrincessFrosty]

This timing is completely arbitrary, just trying to get the point across on why ramdrives aren't as useful anymore, and why ram is still needed.

[Compressed data on SSD copied to Ram] - 3 secs
[Compressed data on ram, decompressed] - 10 secs
[Uncompressed data is fed to and rendered by video card] - 20secs
Total time - 33secs

Rendered and uncompressed data is now on ram and ready for access by the game. Anytime the game needs this data it can now access everything in milliseconds. Now compare this to using a ramdrive:

[Compressed data on Ramdrive copied to Ram] - <1 secs
[Compressed data on ram, decompressed] - 10 secs
[Uncompressed data is fed to and rendered by video card] - 20secs
Total time 31secs


See how much time you saved? It took you milliseconds to copy data off your ramdrive, but this is a small fraction of the time needed to "load" a game for playing.

Remember that this varies from game to game, the bottleneck during the loading period for different games depends on various factors like the amount of compression used in art assets, the total size of the art assets, the speed of the rest of the hardware and even the settings used in game.

The game I initially tested (BFBC2) is clearly not benefiting from this because whatever is happening at load time is not bottlenecked by the data store, that doesn't necessarily hold true for all games. That's what I'm interested in investigating.

I'm extremely interested in the idTech5 engine with a large amount of relatively uncompressed textures, unfortunately I can't fit all of Rage or Max Payne 3 in to RAM since the installs are so big, it might be possible using junction points or other trickery to have a subset of the art assets stored/mirrored on the RAMdisk, upgrading the script to do that shouldn't be very hard at all.

 

[nanobreath]

Remember that this varies from game to game, the bottleneck during the loading period for different games depends on various factors like the amount of compression used in art assets, the total size of the art assets, the speed of the rest of the hardware and even the settings used in game.

The game I initially tested (BFBC2) is clearly not benefiting from this because whatever is happening at load time is not bottlenecked by the data store, that doesn't necessarily hold true for all games. That's what I'm interested in investigating.

I'm extremely interested in the idTech5 engine with a large amount of relatively uncompressed textures, unfortunately I can't fit all of Rage or Max Payne 3 in to RAM since the installs are so big, it might be possible using junction points or other trickery to have a subset of the art assets stored/mirrored on the RAMdisk, upgrading the script to do that shouldn't be very hard at all.

Yeah, I was being VERY generic to explain the situation where ramdisks would not be effective.

For the case of the idTech5 engine, it might produce some benefits, but my main concern is whether or not the benefits would be worth the effort. Don't get me wrong, I fully support doing something to see if it is possible and to evaluate it. Sometimes it is just as much or more fun making this stuff work than the actual game.

 

[PrincessFrosty]

It's true, I'm probably just as interested in finding useful tasks for extremely fast data stores held in RAM as I am in finding ways to speed up game loading times.

Honestly though I think the research and investigation is the most effort, if we can find some decent uses for this, some specific games that happen to benefit from this kind of thing, then we can write scripts to easily automate the "management" of the RAMDisk, it doesn't necessarily have to be a huge pain to manage.

 

[BFG10K]

Hmm, so if games typically aren't even I/O bound when using an SSD, why do we have RAM that's capable of such speeds? What tasks are the OS and browsers doing that takes advantage of all that extra speed that not even a game utilizes?

The RAM is mainly for the CPU side of things (data & instructions). Even though cache hit rates are typically very good, the times you have to go to main memory, it’s extremely lethargic in comparison. If each of those instances had to hit an SSD instead of RAM, it would noticeably degrade system performance across the board.

With that said, a lot of those “extreme” RAM kits are just marketing as the differences aren’t really visible outside certain memory bound benchmarks. So don’t waste money on kits like that.