sustainable read speed for SSD possible?

[froky]

I have a project where I need to allow a user to connect an SSD (UASP) to a device (which has an embedded Mini ITX PC) and stream data (uncompressed video) at 370 MByte/second read speed.
I need to allow the user to connect the SSD to his own PC and create any (uncompressed video) content he wants and send it from his PC to the SSD, which he will later connect to the custom device for streaming his content.
In other words, files are going to be saved and deleted from the SSD by the user, however the read speed must remain the same and not go below 370 MByte/s for the device to be able to stream the files.
Is this possible? Leaving any damage or wear from old SSD aside.

 

[mikeymikec]

I think you're in for a world of "fun" if you require a guarantee almost any read or write speed (apart from the absurdly low varieties); all it takes is for the OS or a third party program to decide that it wants some I/O time and there goes the guarantee.

However, I think a decent (ie. not cheapo) and modern SSD would manage to keep up a stream that required that kind of performance (bearing in mind that streaming doesn't necessarily involve a fixed I/O speed). If I wanted to do as much as I could to ensure it, the SSD would only be storing the files that need to be streamed. I probably would go for a PCIE SSD (again playing on the safe side).

 

[froky]

I think you're in for a world of "fun" if you require a guarantee almost any read or write speed

The device has an embedded Mini ITX PC with an OS (I'll probably go with some flavor of Linux) which the user cannot access. So I can set up the OS of the device how I want and it will stay that way.

The reason I asked this question was for concerns of something like fragmentation slowing down the read speed of the SSD over time.

 

[mikeymikec]

Linux has had some TRIM issues that have only recently been fixed, you might want to research that when choosing the OS version.

Fragmentation isn't an issue of the magnitude it is with HDDs because the access times of decent SSDs are an order of magnitude lower than with HDDs, AFAIK.

 

[froky]

Linux has had some TRIM issues that have only recently been fixed, you might want to research that when choosing the OS version.

Fragmentation isn't an issue of the magnitude it is with HDDs because the access times of decent SSDs are an order of magnitude lower than with HDDs, AFAIK.

Thanks for mentioning TRIM. I searched for it, and correct me if I got the wrong info, that will have zero effect on streaming (reading) data. Only thing it will affect is how fast the user can overwrite files on the SSD from his PC, not how fast my device can stream/read the new files he has placed on the SSD.
Am I right?

 

[mikeymikec]

Presumably you will actually be writing some data to this SSD at some point

No TRIM = write amplification = pointless overhead and 'life' needlessly taken from the drive.

However, the bug in question is rather good at corrupting data, as I understand it.

 

[froky]

Presumably you will actually be writing some data to this SSD at some point

I probably wasn't clear, let me try again:

From what I gathered, unlike HDD, SSD is slower at writing data to a space in the memory which has previously been occupied by another file and deleted only by being marked as deleted, not actually ''zeroed''.
TRIM allows to ''zero'' the space when you choose to delete a file, thus when writing to that space again later, the write speed will be the same as if you were writing to a previously unused space.
Is this correct?

Well, since my device only reads (streams) data and my aim is minimal read speed (not write speed), this wont cause slowdowns for my device, which is what I need. It will cause slowdowns when writing the file to the SSD from the user's PC, before the SSD is disconnected from the PC and connected to my device. And here the 370 MByte/s minimum speed is not a requirement, it is when only reading the written data by my device.
Did I get it?

That said, I might go with Windows 7 for other reasons.

 

[mikeymikec]

SSDs cannot write to an area that already has data in (as you say, it has to be erased first), so they don't ever try. Instead they write the data to a new location, then mark the old location as garbage. Whatever garbage collection routine the TRIM has will then clean up the garbage at a later date.

When the OS requests that a file be deleted, a TRIM-aware OS will send a TRIM command to the drive so that the areas used by that file are immediately regarded as garbage and erased. An OS that isn't TRIM aware won't send the TRIM command, so there's a chance that when other 'live' data (which shares the same pages in the SSD as the 'garbage' data) is modified, when the SSD copies that page data to a new (erased) page, the garbage will get taken with it, so more data is being written unnecessary, which is what write amplification is all about.

Getting back to my original point, because there's a bug in the TRIM implementation (which apparently affects not all SSDs) in Linux that has only recently been fixed, it could cause data corruption when the TRIM command has been sent. So it's not a question of performance in the context of that bug. Three choices there as far as I'm aware: Install Linux and disable TRIM at the first opportunity, or use an 'known good' SSD, or use a version of Linux that doesn't exhibit the bug.

Win7 is TRIM aware and I'm not aware of any bugs in its implementation (using the MSAHCI driver and AHCI mode in the BIOS). I'm not aware of any bugs in the AMD or Intel AHCI drivers either with regard to TRIM (I would normally use those rather than MSAHCI - however I would use MSAHCI if I wasn't sufficiently confident that a third party's drivers were mature enough).

 

[froky]

Hm, okay, your warning on possibility of damage when TRIM was not enabled was load and clear. I just wanted to make sure I understood that besides the corruption which it could cause, the slowdown would be noticeable only when writing and not reading.

 

[ClockHound]

370 MByte per second - for how many seconds?

Most SATA3 SSDs that can average better than that rate (500MB/s+). Could start with even the cheap BX100 and work your way up from there.

An NVMe drive, like the Intel 750/Sammy 951 could be a safer choice with a 3X-4X bandwidth cushion.

If all the TRIM/GC tasks are performed while connected to the host system before launching the streaming mode, a decent SATA drive would seem sufficient.

 

[mikeymikec]

@ froky

No, with that bug in play, TRIM being specifically disabled is a workaround to avoid the corruption.

 

[froky]

370 MByte per second - for how many seconds?

Indefinitely. It's streaming raw video data, created by the user, size limited only by the capacity of the SSD. So it can be minute long, or 45 minutes long.

I need something easily pluggable by the user and not confusing.
That's why I'm not going with PCIE connected SSDs and similar.

I'd even prefer USB 3.0 memory sticks because it would be something even more familiar to the user, even though expensive. But none seem to rech those speeds at this time.

 

[mikeymikec]

The only way you're going to possibly manage it with something pluggable would be an SSD connected to USB 3.0, but you're basically putting as many obstacles in the way as possible.

It might manage it, it might not. Maybe in 5 years it'll be more practical, dunno.

 

[AlienTech]

USB3 wont stream data at those speeds, Any hiccup and the speeds would drop drastically and cause dropouts. The only interface I seen work at high speeds consistently is IEEE.. They were designed specially to use with video streaming so maybe thats why there is no drop outs. Remember a hard drive will once in a while do a resync.. Which will cause the transfer to stop and cause drop outs. SSD's as long as they dont get a garbage collection wont have a dropout problem, so then the interface has to be reliable.. I dont think anything interprets a SATA connection data transfer so it should be fine as well. I think there might be video contention if using internal graphics.. Or maybe even data transfer to external video cards which might cause problems so you have to make sure those parts are running fast as well. In essence you need more than just a fast drive.. fast memory, good motherboard, good interface are all necessary.. For many year sI spent a great deal of effort and money trying to get this to work without much success but that was a long time ago and things have gotten much better.. But it still stands that you need hardware which will do the job and they cost an average amount of money for something like this which is not really high end but is tolerance critical.. Many motherboards even top end will has weird problems.. So even check feedback for such things.. Something as simple as someone turning on the fan or lights can cause hiccups and dropouts.. So having good quality keyboards, mouse, power supply and cables are also a must.. Yea pretty much everything has to be speced.. Which might be why I spent years on it.. Getting one thing fixed cause another to fail.. Very simple stuff..

 

[froky]

USB3 wont stream data at those speeds

Well THE SSD connected through USB3 port is not really USB3, https://en.wikipedia.org/wiki/USB_Attached_SCSI
Are we good with SSD?

 

[coercitiv]

Are we good with SSD?

A good SSD does around 500MB/s in sequential read, so you're good with a SSD.

However, why is that you need to stream directly from the attached SSD? Why not make a copy to a more reliable medium inside the unit?

Are there no caches & latencies involved? I find hard to believe you need to enforce a minimum speed rather close to the maximum 500MB/s SATAIII SSDs offer when there must be quite a bit of caching involved in the process (raw data cache, encoded video stream cache etc)

 

[froky]

A good SSD does around 500MB/s in sequential read, so you're good with a SSD.

Okay, good.
When AlienTech previously said that USB3 couldnt handle such speeds, did he mean USB3 memory sticks then? Or also UASP (SSD connected via a USB3 port)?

However, why is that you need to stream directly from the attached SSD?

It's a design choice that has been chosen after long discussions and comprisons of options. Copying to a local memory has also been discussed. Don't worry about it.

 

[mikeymikec]

Trying to stream over USB3 is subject to more potential obstacles than over say AHCI or NVMe; USB3 is a general purpose bus whereas the other two are designed specifically to communicate with storage devices, and I very much doubt that the USB-attached SSD will be the only device connected by USB. There's also a potential question of what the USB buses are sharing bandwidth with.

 

[coercitiv]

When AlienTech previously said that USB3 couldnt handle such speeds, did he mean USB3 memory sticks then? Or also UASP (SSD connected via a USB3 port)?

He meant USB 3 - the interface, the bus. Mikeymikec explained it quite nicely in a post above.

Depending on user/usage even problems risen from wire/socket fatigue might give you trouble at some point.

 

[froky]

What If I design a slot on the device for inserting an SSD directly into?
Any issues with such an approach?
Then the user would need to use a USB adapter only when tranfering his files from his PC to the SSD, where speed isn't an issue.

 

[Phynaz]

What device can consume data at this rate?

How are you going to send data at that rate?

What you want to do is basically impossible without specialized hardware. Expensive specialized hardware.

I don't understand why you wouldn't use a h.264 capture card. What in the world is he doing that requires uncompressed video? Even movie studios don't use uncompressed video except in post production.

 

[froky]

What device can consume data at this rate?

A PC?

How are you going to send data at that rate?

Like we've been discussing this whole thread. I mean cmon, seriously?

 

[Phynaz]

I'm very doubtful a PC could consume video data at that rate.

You said custom device in your op, now you're saying its a PC.

 

[froky]

I'm very doubtful a PC could consume video data at that rate.

At this point you have to define "consume". If you mean read from memory, it can, unless you have a hard drive instead of, well, SSD. Tested it.

You said custom device in your op, now you're saying its a PC.

This is the very first sentence of the very first post here:

I have a project where I need to allow a user to connect an SSD (UASP) to a device (which has an embedded Mini ITX PC)

Not that I'm saying it's a regular PC now. I'm just answering "PC" to your question on what kind of device can do this.

I never asked for a discussion of the project and its design choices. I'm asking a specific question, can you stay on topic?

 

[coercitiv]

I never asked for a discussion of the project and its design choices. I'm asking a specific question, can you stay on topic?

And you have been answered already: modern SSDs can read at 500MB+ per second. What else is there to discuss but design choices? (SATA vs USB is a design choice)

If you are indeed sending uncompressed video on a stream, then there is nothing else we can help you with, and we have reached a conclusion.