Does this storage strategy make sense?

[xsrossiter]

I'll be putting together a system around the Tyan S2665ANF with an Adaptec 39320-R. Off the SCSI card two Seagate 36 GB in RAID 0 for working storage (video/census files I am working on at the moment) and one 18 GB for OS and Apps. Off one of the EIDE channels on the motherboard a WD1200JB for non-working storage (things that I will get to eventually and shift to working storage). Finally, when I've got the finished product burn it to CD-RW or DVD.

The work flow could be as follows:

1 - Shoot 20+ hours of digital video
2 - Firewire footage to WD1200JB
3 - Pull segement of footage to RAID 0 setup
4 - Beat up on footage with dual 2.66 Xeons, Radeon 9700 Pro and 2GB RAM
5 - Burn finished product to DVD or return to WD1200JB

The idea is (1) to keep the OS with its swap file moving fast but seperate from data file manipulations, (2) have a temporary cheap storage bin for on deck projects, (3) rip through video files and census data queries when needed using 10K RAID 0, (4) offload finished product to keep disks available for other work.

I'm not totally sure how video data might perform differtly from census data under this setup. They both come in huge single files but census data is millions of fields whereas video is a binary stream.

Or maybe 2 36 GBs in RAID 1 for OS and Apps, 2 36 GBs in RAID 0 for work in progress and 1 WD1200JB off the EIDE connector for storage of non-critcal data/video. Anything else can be burned to DVD or CD-R.

 

[thesix]

> Does this storage strategy make sense?

Are you making videos for a living ? 🙂

Your configuration is much better than most of the studio shops ( because they're still using Mac ? just kidding 🙂 I guess.

If you shoot miniDV, the data 'density' is about 5MB/sec of 15GB/hour.
A single IDE disk can handle that throughput quite well.
When 'processing' the video, CPU is the bottleneck, not disk, unless you like to
move video files from one disk to another (which is not part of the video 'processing' 🙂.

I can use my miniDV capture TV programs directly to my 5400RPM external firewire disk in real time without losing a single frame.

Since you have two CPUs, use softwares that can utilize both of them, which will save you a lot of time.

 

[ProviaFan]

I am not a video editing pro, but I notice that when doing preview renders in Premiere, the disk seems to be the bottleneck, even with my 7,200 RPM IDE drive. A RAID setup would probably be nice here, but I don't know that SCSI RAID is necessary. Perhaps three or four WD "SE" 120GB or 200GB drives in RAID 5 on a 3Ware IDE RAID controller would be better suited for your application?

 

[Link19]

Hi I am knew to the forums. In fact I have never registered to post on any forums ever before. I have just read other people's posts and replies on forums. I would like to know how I post a comment or a question I might have on the main General hardware forum category and other forum categories.

 

[Link19]

Originally posted by: xsrossiter
I'll be putting together a system around the Tyan S2665ANF with an Adaptec 39320-R. Off the SCSI card two Seagate 36 GB in RAID 0 for working storage (video/census files I am working on at the moment) and one 18 GB for OS and Apps. Off one of the EIDE channels on the motherboard a WD1200JB for non-working storage (things that I will get to eventually and shift to working storage). Finally, when I've got the finished product burn it to CD-RW or DVD.

The work flow could be as follows:

1 - Shoot 20+ hours of digital video
2 - Firewire footage to WD1200JB
3 - Pull segement of footage to RAID 0 setup
4 - Beat up on footage with dual 2.66 Xeons, Radeon 9700 Pro and 2GB RAM
5 - Burn finished product to DVD or return to WD1200JB

The idea is (1) to keep the OS with its swap file moving fast but seperate from data file manipulations, (2) have a temporary cheap storage bin for on deck projects, (3) rip through video files and census data queries when needed using 10K RAID 0, (4) offload finished product to keep disks available for other work.

I'm not totally sure how video data might perform differtly from census data under this setup. They both come in huge single files but census data is millions of fields whereas video is a binary stream.

Or maybe 2 36 GBs in RAID 1 for OS and Apps, 2 36 GBs in RAID 0 for work in progress and 1 WD1200JB off the EIDE connector for storage of non-critcal data/video. Anything else can be burned to DVD or CD-R.

Hi once again I am new and I am trying to find out where a quote gets posted when you select that option

 

[ProviaFan]

Originally posted by: Link19
Hi I am knew to the forums. In fact I have never registered to post on any forums ever before. I have just read other people's posts and replies on forums. I would like to know how I post a comment or a question I might have on the main General hardware forum category and other forum categories.

Click on the new topic button under the Navigation drop down menu at the upper right side of the page.

 

[thesix]

> but I notice that when doing preview renders in Premiere, the disk seems to be the bottleneck

Are you using the same disk for Premiere software, video file, Windows and even the swap space ?

Any good 7200RPM IDE disk can handle 20MB/sec sequential read and 15MB/sec sequential write.
The real challenge here is to keep I/O sequential on that physical device.
For example, if Premiere is writing output to the same device as it's reading source from, neither
read or write is sequential anymore, that's a biggest performance penalty.

All you need is dedicated disks for input and output files ( and tempoaray files etc. )

Let's say you only have 3 disks, one is used as bootdisk for Windows and Programes,
you have two choices for the other two:

1) Use them separately: process raw AVI file on disk B, trim/edit it to a new AVI file on disk C -->
render the new AVI file on disk C and save the MPEG2 file back to disk B, and so on.

2) Setup a RAID 0 from two disks: do all the processing within the single RAID0 disk.

Which one is faster/better ? The first one.

Another simple test:
1. Copy a 1GB file from disk A to disk B
2. Copy a 1GB file to a new file on the same RAID0 disk composed by disk A and disk B

Which one is faster ? The first one.

Enough said about RAID0, now SCSI v.s. IDE:

SCSI only makes sense when you want to have more than 4 disks on your system,
otherwise, the much much higher price doesn' t justify the relatively small performance gain.

The reason I am using SCSI at home is that I got a bounch of 9GB 10RPM Ultra160 disks
for very cheap price a while ago, and I can't resist using them 🙂
The 39260 card and a SCSI enclosure cost me $500 though.

Don't get me wrong, it's good ( not just cool ) to have SCSI for scalability.

Anyway, having 4 separate ATA133 8MB cache disks and being able
to separate data to each of them is *much better* than having one luxury SCSI RAID0 disk.

 

[ProviaFan]

Originally posted by: thesix
> but I notice that when doing preview renders in Premiere, the disk seems to be the bottleneck

Are you using the same disk for Premiere software, video file, Windows and even the swap space ?

Good point. I had one disk for OS / programs, and the other must have had the data files and the scratch files on it at the same time; that might have been the problem.

 

[xsrossiter]

Thanks for the help everybody. It's kind of scary how much you know 😕 but I'm glad for the help. Well, time to contemplate some more.

 

[Fant]

I must agree thats its much more important to have seperate physical drives for each of your files than having faster SCSI drives. Ideal would be both seperate scsi drives. One drive for OS/Apps/Tempspace, one drive for reading data from, and one drive for writing to. It keeps the system moving along much more efficiently. No matter how much we'd like, computers and peripherals are not good at doing two tasks simultaneously.