• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Why are we holding ourselves back with mechanical drives?

V

Valinos

Banned
I'm curious to know why we hold ourselves back with the biggest bottleneck on the computer: the mechanical hard drive. As fast as IDE and SCSI drives are getting, the speed gains are becoming smaller and smaller and there is a limit. Why haven't the big hardware manufacturers moved onto solid-state disks? As flash memory becomes cheaper and cheaper, it seems like it would make more sense to have solid state hard drives that have the speed of regular system RAM...therefore eliminating the mechanical drive bottleneck in both speed and long-term data integrity.

I know I'd pay $1000 for a 5 gig solid-state hard drive. The speed gains would be incredible and aside from external sources...the drive would never fail. It would have no moving parts to deal with. Why haven't we made the jump to these? The results would definately be revolutionary. Hell, we could even eliminate the need for RAM if we had these drives. The need for temporary data storage would be eliminated with the high transfer speeds of flash memory.

Anyone care to enlighten me on why we haven't made this step forward? All comments, thoughts, and ideas welcome.
 
I think a solid state drive is in the works, cant remember which company or group of companies is/are working on it.
 


<< I know I'd pay $1000 for a 5 gig solid-state hard drive. >>


Actually, this is the reason why I wouldn't buy a solid state drive right now, especially after being spoiled with the cheap $/gig mechanical drives of today. Before the consumer market buys into something, the more affluent will have to adopt it first. And your $1k for 5gig figure is a little low... try $74k for 4gigs (summer '01 pricing). Yeah, I *wish* I had that much money to spend on a solid state device. 😉 Another reason was already given by tontod:



<< I think a solid state drive is in the works, cant remember which company or group of companies is/are working on it. >>


From as far back as 10 years that I can remember, there have been a couple of companies that have said "We'll produce a solid state drive." Before the product ever surfaced, most have died away or don't have the marketing to convince industry they needed one of these. Redundancy is cheap with RAID, some speed gains can be had through RAID, then there's the all important performance/price ratio. Will the backend database really get us the same n-fold gains as the n-fold money we invest in it?

But here's one (currently surviving company) with products. I didn't find pricing with a quick google search, but I'm inclined to believe it's still out of my budget.
Bit-Micro SCSI drive
Bit-Micro IDE drive
 
actually there a bit more to be said about the so called bottleneck that u speak of,

yes -- if you are simply copy/moving gigabytes of information from one drive to another you would get a big improvment from using solid state drives, but if you are an average person you probably wouldn't see much improvment at all.

most people refer to hdd's as being the bottleneck simply because it is the slowest part of the computer in terms of data transfer, but they don't really understand what a bottleneck is. In fact, a lot of the time the hard drive isn't a bottleneck at all. I would argue that right now, memory is more of a bottleneck then anything else, especially if we are talking about the P4. Improving latency and bandwidth in the memory subsystem would give a much bigger improvement in performance than adding a solid state disk drive.

The way most current chipsets/memory controllers/cpu's are designed most of the data you need is cached.
You might think that when you turn your computer on most of the boot up process is taken up by reading data from the harddrive, thats simply not true. Solid state drivers have been around for a while now, they've just cost thousands of dollars. But I remember reading that when compared to a regular harddrive the boot up time of a pentium III machine only increased by 5 seconds. (In the days it used to take 60 or so seconds to boot up at best, and by now most hdd's are much faster)

Most of this you can test yourself. For example when using the windows2000/XP hybernate feature it only takes seconds before the computer returns to working state. (after the post obviously) If you have a motherboard that supports STR (suspend to ram) you'll see that it doesn't really take much faster if you store your data in ram instead of the harddrive.

I am obviously not saying that hard drives aren't a lot slower than ram. In fact they are. But it's all about how the computer architecture works. You are only working with megabytes at time for on particular program (i.e. microsoft word, java2, whatever) most of it is cached. Now if you are using your system that needs access to large amounts of data fast such as a database server than a solid state drive would make sense--but then again thats why servers have gigabytes of ram. In a essence they have a solid state drive.

Anyway.. just some thoughts...
(p.s. didn't prove read this so ignore the millions of mistakes which i probably made)
 
Anyone know the bandwidth of common flash? I'm pretty sure it's not up at hd standards and I know it's not DRAM speeds.
 
I can see so many more possibilities with solid state drives though. If they could transfer information as fast as DDR or RDRAM then you'd have so many new things that could be done. Personally, I'm a huge gamer and I can see the possibilities of endless, huge worlds with tons of real-time destructible environments and objects...that could all be written and read easily. If the drives became commonplace you'd see developers for software finally take advantage of the huge speed gains. Video encoding and editing would have HUGE performance gains from something like this. The transfer, writing, and reading of data would be so quick that the efficiency of some applications could skyrocket. Giving you more time.

I'm not a programmer, so I don't know the details, but I could see many doors opening up if you had all your memory into one fast, solid unit. You'd also eliminate the need for as much power, no more noise, and your case would probably be a lot cooler. We wouldn't even have to defrag anymore! woohoo

I can't see something like this costing $74k. As cheap as flash memory and other similar technologies are becoming....I honestly can't see it as being that expensive. You can get 512mb of compact flash for less than $200. How much different would a full fledged, 10 gig solid state drive be? Am I missing something?
 
Current Flash memory technology also has the downside that it tends to wear out after 10000 writes to a sector. Fine for a memory card, but unacceptable for a hard drive. Also, I know that Flash memory cards peak at <2MB/sec, whether this is a limitation of the technology or just the format I have no clue. Bit Micro has a line of solid state Hard Disks. The best I have heard was $16k for 9GB.
 
ibm is developing a new memory technology they dub magnetic ram. it should be faster than regular dram and only requires power for reading and writing.
 
Solid-state drives are regularly used in extemely critical applications. I know that NASA uses lots of them, and so do most large companies with huge networks. As it should naturally, only they could afford them. It's a commodity to the general public for now.... not a necessity.
 


<< I can see so many more possibilities with solid state drives though. If they could transfer information as fast as DDR or RDRAM then you'd have so many new things that could be done. Personally, I'm a huge gamer and I can see the possibilities of endless, huge worlds with tons of real-time destructible environments and objects...that could all be written and read easily. If the drives became commonplace you'd see developers for software finally take advantage of the huge speed gains. Video encoding and editing would have HUGE performance gains from something like this. The transfer, writing, and reading of data would be so quick that the efficiency of some applications could skyrocket. Giving you more time. >>



as far as adding complexity to games and so forth, I really don't think that the current system is all that bad. Say you have a world in a game that takes up 5gb. Well, obviously you can't show all 5gb of that world at the same time! The user can only access a part of it at a time. There's only so much that a processor can handle, and only so much that can be shown on a monitor. As long as the currently-user-accessible part fits within conventional RAM (really shouldn't be a problem when 256MB is common and 1GB is easily within the average user's reach), the processor can get all of the data that it currently needs, and the program should be written in such a way that the information is moved from the HD to the RAM before the information is needed by the processor. That way, the processor is always getting info straight from the RAM and not from the hard drive.

So I really don't think that it would make that much of a difference for the vast majority of users. If good enough prediction algorithms exist, then there's no need for ultra-high-speed hard drives, because everything will be pre-cached into RAM. (Granted, that's a big if, but for games I don't think it's very hard.)

With video editing, some database situations, or other situations which require truly random access across a very large (say >1GB) amount of data, that's where an improvement would be seen. But then, those who do that sort of thing often have access to workstations or servers with huge amounts of RAM. So I don't see as much of a problem with the current system.
 
wbwither you took the words right out of my ... uhh... keyboard 🙂

there would be no advantage at all to games from adding a solid state disk drive.
the bottleneck in games right now is the video card and since we are talking about memory.. its the video cards memory.
you can have big levels and detailed textures that take up 50 GB if you wanted too even right now. The problem is even if all of that was in ram it would be to slow. Even with agp texturing enabled. In the days before texture compression (which basically means all the textures stay in the video cards memory) remember those benchmarks of 32mb geforce vrs 64mb? as soon as the textures weren't able to fit in the 32mb of memory the framerate dropped by about 80% as appose to the 64mb geforce where there was maybe a 1 percent drop.

The video cards memory is much much much faster than regular memory. Remember it's not just the memory speed it's also the width of the data bus. The ddr memory on video cards is 2 times faster than regular ddr memory, but the nvidia geforce 4 has what?? like 8 times the memory bandwith of regular memory?

also what might not be clear here (and please correct me if i am wrong) is the fact that even if everything was the same speedwise --- lets say for arguments sake, the hdd and the ram and the cache were all the same speed, the cache would still be a lot faster, why? because addressing a small amount of memory is very easy.. you can directly map it. on the other hand even when dealing with fast ddr memory it still takes time to access it (thats where the cas and ras delays come from). thats why computer science students learn about binary trees and hashtables and other algorythms for putting data into memory. the point is that a solid state disk drive would still have to be cached, just like regular drives, so speed wouldn't be the key factor of improvment. there are some other reason of course to go for a solid state disk drive. Hell i'd want one, if they weren't so $$$$$.
 


<< that the information is moved from the HD to the RAM before the information is needed by the processor >>



That sounds a lot like prefetching? So you are saying there should be a HD to RAM level prefetching?

I can see things speed up a lot here 😀
 
Cray has one.

224 GB with 80 GB/s transfer rate. Too bad most of us won't make enough money in our lifetime to buy one.

The truth is that they are already available through comapnies like M-systems. The problem is that the popularity isn't there yet. I still think a solid state drive or ramdrive is were computers should be going. Just use the old clunky mechanical drive for backup when you shutdown.

In 5 to 10 years, I'ld bet you'll see alot more solid-state drives. It will just take a while for them to get to the mainstream.

Edit: And they will fail. There MTBF may or may not be higher than mechanical drives. Probably will be higher though.

By the way I use an M-systems Disk On Chip for the OS on my homemade router. It's pretty strange to watch a machine boot up absolutely silently. Zero moving parts in my router computer.
 


<<

<< that the information is moved from the HD to the RAM before the information is needed by the processor >>



That sounds a lot like prefetching? So you are saying there should be a HD to RAM level prefetching?

I can see things speed up a lot here 😀 >>



Yeah, exactly. Is this not done already to some extent? Like, I notice a big difference with, simple example, Adobe Acrobat. The first time I open Acrobat when I click on a pdf file on the Web somewhere, I get a little bit of delay; I hear the HD whirring, and then after a second, the pdf pops up in Acrobat. So I close that pdf (maybe even close Internet Explorer, maybe even for an hour or two), surf around a little bit more (say 5 mins -- although as I've said, I've noticed this even with a few hours' time differnece) and then I click on another pdf. This time, it almost instantaneously pops up on my screen. I don't hear the HD do anything. It's clear that Acrobat has been cached in the memory, even though it has been essentially closed. I guess the OS keeps it in memory until it needs to reallocate that memory for something else? I don't know how it all works, but it's just one thing I've noticed.

So within the next few generations of browser/OS, I would expect that it would SEE the .pdf links on the web page, and anticipate that I was going to open one, and before I do so, Acrobat will have already been read from the HD into the RAM, ready for me to use it, so when I click on a pdf, it opens instantly. The same thing when I go to my web graphics folder in Windows Explorer; what's in there? Tons of PhotoShop files! Why not pre-fetch PhotoShop, since I'm probably going to open one of the files? After all, it's not like the HD is doing anything else anyway. It just sits there 99% of the time. And if I've got the RAM to spare, why not? When I go to the folder where I keep all my WordPerfect documents or Excel spreadsheets, pre-fetch WordPerfect or Excel! It should be able to do this seamlessly, without a noticable lag of any sort.

In a game environment, this becomes much easier. Random access isn't really possible, since semi-real physical worlds are represented, and you can't magically teleport to anywhere on the map (well, in most games at least). And if there's a wormhole or portal of some sort, when I move to within X distance from the portal, the game pre-fetches the info for what's on the other side of the portal from the HD, and into the RAM. So at all times, I have a 'sphere' of space around me, the player, which is cached in RAM and ready to go.

Of course, with all of this discussion, there is still some limitation on the RAM. If I've already got 5 programs open and I go to my PhotoShop folder, it's not realistic for me to expect the OS to pre-load PhotoShop, because the RAM is already used. So it wouldn't happen all of the time, just when the RAM is free.

See, all of this type of thing is what I live for, essentially. Computer technology is improving at a drastic rate. However, it's clear that what we have already, we're not using completely efficiently. Why not maximize the efficiency of what we have now, and gain performance without buying new hardware? Why not simply write more intelligent software? It really wouldn't be that hard to implement this sort of thing.
 
You can always buy your self a cray...



<< I'm curious to know why we hold ourselves back with the biggest bottleneck on the computer: the mechanical hard drive. As fast as IDE and SCSI drives are getting, the speed gains are becoming smaller and smaller and there is a limit. Why haven't the big hardware manufacturers moved onto solid-state disks? As flash memory becomes cheaper and cheaper, it seems like it would make more sense to have solid state hard drives that have the speed of regular system RAM...therefore eliminating the mechanical drive bottleneck in both speed and long-term data integrity.

I know I'd pay $1000 for a 5 gig solid-state hard drive. The speed gains would be incredible and aside from external sources...the drive would never fail. It would have no moving parts to deal with. Why haven't we made the jump to these? The results would definately be revolutionary. Hell, we could even eliminate the need for RAM if we had these drives. The need for temporary data storage would be eliminated with the high transfer speeds of flash memory.

Anyone care to enlighten me on why we haven't made this step forward? All comments, thoughts, and ideas welcome. >>

 
If we were to use RAM-drives then in the event of power loss you would lose every single thing on that drive (unless it had a battery backup that recharges on power-on). RAM prices may be dropping, but ROM isn't. Similar to RAM, ROM (Read-only memory) is a solid state device made of ships and silicon. ROM does not require power to keep information stored in them. A simple way to realize the difference from RAM to ROM is take a Palm Pilot or Pocket PC. If you remove the batteries for a few minutes, you lose everything you installed or saved on it, but the OS stays intact. This is because your data is stored on RAM, while the OS is stored on ROM.
 


<< Check this ram-disk out.
Cenatek

Doug >>



My eval unit should be here any day now <grin>



<< About $1000 for a 1 Gig drive. >>



Only took about a month of badgering, but they finally took my idea and started offering bare units! w00t!
 


<< Current Flash memory technology also has the downside that it tends to wear out after 10000 writes to a sector. Fine for a memory card, but unacceptable for a hard drive. Also, I know that Flash memory cards peak at <2MB/sec, whether this is a limitation of the technology or just the format I have no clue. Bit Micro has a line of solid state Hard Disks. The best I have heard was $16k for 9GB. >>



As far as CF goes, the best I've seen come from Zenergy. They claim 7MB/s reads and 3MB/s writes. Didn't see any seek time specs, but Dandata tested a CF card and it came in at about 2ms. Also claimed in 1,000,000 erases!

Been thinking about doing a project with 8 of them in RAID-0 😀
 
http://www.platypus-technology.com/

Takes 1GB DIMMs, 8 per card and can link multiple cards. UPS powersource as an option.

<< the drive would never fail >>

But your data would. A filesystem can still be corrupted, only now by stray EMI which RAM has always had a problem with. ECC memory is still enormously expensive in large quantities like this, and would probably still crash and corrput the filesystem somehow after a given amount of time, so "never" doesn't apply.
 
You must log in or register to reply here.

TRENDING THREADS

Back
Top