gigabit network for network rendering

uzuncakmak

Member
Apr 2, 2012
35
0
0
I have around 20 PCs networked and I'd like to use them to help render my projects
Most of the resources for rendering will reside on a network share
then all the nodes read data from there .. render... and then write back the rendered frames....
I must upgrade the current network at least to a Gigabit Ethernet .. What switch options do I have there? Preferable a reliable one like Cisco...
And also ~20 PCs reading data (usually uncompressed video files) from the same network share at the same time will certainly create traffic there, will a Gigabit Ethernet Switch be enough for that or I should be heading towards 10 Gigabit Ethernet?

Can you advice on this matter?

Thanks in advance..
 

OVerLoRDI

Diamond Member
Jan 22, 2006
5,490
4
81
In the situation you are describing it sounds like your NAS will be a bottle kneck as well. Maybe someone with more experience can chime in about the bandwidth requirements for uncompressed video for that many render boxes. You might actually need 10 gigabit....
 

Lithium381

Lifer
May 12, 2001
12,452
2
0

tomt4535

Golden Member
Jan 4, 2004
1,758
0
76
What kind of disks do you have in the server hosting the file share? That could also be a possible bottleneck. 10g is also very expensive, think $1k per port or more, and the NICs, cabling, etc you need to do it right, its not going to be cheap.
 

Rhyseh

Junior Member
Apr 2, 2012
5
0
0
As stated the share is probably going to be the bottleneck, what's the interface speed on server running the share? Also what kind of drives does it have?

From my limited understanding of your environment I would say that 10 gigabit is not going to see any performance increases. What's your budget for this? If you are talking about 10 gigabit connections you really need to get out a networking professional to do a proper assessment and implementation.
 

Lithium381

Lifer
May 12, 2001
12,452
2
0
Depending on your server hardware, assuming you've got two 1gig ports, you could bundle them and get (almost) a 2gig pipe. I agree with Rhyseh though, 1g is pretty common nowadays, but if you're going to bite the bullet and go 10g, it's not cheap and you should probably have someone help you build it so it's done right. that way it scales for the future and doesn't have to be rebuilt again in the short-term future
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
I had spent considerable amount of time explaining what I will do with my network to a tech guy in Cisco last night and he agreed that 1Gb (not 10Gb) ethernet will do the job fine. And he recommended Cisco SG200-26P

http://www.newegg.com/Product/Produc...annel&CMP=OTC-

This is a managed version and I may opt to get an unmanaged version only to cut down the prices.

I have checked and all my NICs support Gigabit Ethernet.

I will also make all the cables CAT5e.

So the only remaining issue is then HDDs on the media server.. which is a 1TB 7200 rpm raid 0 HDD... What should I do on this part? upgrade to 15K?!
 

yinan

Golden Member
Jan 12, 2007
1,801
2
71
At the very least RAID 1 so a single hardware failure does not make you lose all of your data.
 

richaron

Golden Member
Mar 27, 2012
1,357
329
136
At the very least RAID 1 so a single hardware failure does not make you lose all of your data.

what they said.

Can I get more NICs on the server to distribute the traffic to those NICs evenly???

I'm pretty sure you can, I wouldn't know how to set it up though.

I don't even think it would be needed, especially since 1Gb/s full duplex is approaching the limit of your storage anyway. It really depends on the software you are using, how big the local caches are & how much/often they update the server.
 
Last edited:

tuprox

Member
Apr 3, 2012
63
0
61
Just out of curiosity can you tell us what kind of computers you have running? 20 machines running full speed is a lot of energy and a lot of heat to manage - not bad in the fall/winter but a heck of a lot to manage in the summer.

Also with the rendering is it possible to do it with GPU's? Depending upon what you are doing a single machine with a couple powerful video cards will give you the same rendering capability as all 20 machines utilizing CPU's.

Also, you need to look at energy consumption over time. It might be cheaper to get a more powerful machine that could replace a number of your 20 pc's especially if you can utilize GPU's. Just remember that all the energy you use in the computers goes somewhere and it is almost all translated to heat and then you have to deal with the energy for cooling in the summer.
 

skyking

Lifer
Nov 21, 2001
22,543
5,589
146
It is hard to figure some of this without knowing the sizes of the chunks that will get moved, rendered, then written back to storage. What app are you using?
Oh, and welcome to the new guys. :)
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
what they said.



I'm pretty sure you can, I wouldn't know how to set it up though.

I don't even think it would be needed, especially since 1Gb/s full duplex is approaching the limit of your storage anyway. It really depends on the software you are using, how big the local caches are & how much/often they update the server.

I just checked with an Intel guy he told me

http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-i340-server-adapter-brief.html

that 'd give me the four times the bandwidth.. So theoretically it will provide me with 4Gb/s and my drive , if I use a single Intel 520 type SSD is around 6Gb/s, can catch up with the speed I guess. Or I can go with 15K drives in RAID 1 configuration that would also be an option..

Software is : Adobe After Effects CS5.5 x64

Each machine will render a single image (TIFF or PNG, each at its max ~20 MB), then write it back there. So I guess writing does not make much traffic but reading of off there..

I dont know much about local caches???
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
Just out of curiosity can you tell us what kind of computers you have running? 20 machines running full speed is a lot of energy and a lot of heat to manage - not bad in the fall/winter but a heck of a lot to manage in the summer.

Also with the rendering is it possible to do it with GPU's? Depending upon what you are doing a single machine with a couple powerful video cards will give you the same rendering capability as all 20 machines utilizing CPU's.

Also, you need to look at energy consumption over time. It might be cheaper to get a more powerful machine that could replace a number of your 20 pc's especially if you can utilize GPU's. Just remember that all the energy you use in the computers goes somewhere and it is almost all translated to heat and then you have to deal with the energy for cooling in the summer.

All 20 computers are Dell Optiplex 380 with 2.93 GHz Core 2 Duo and 8 GB RAM...

What GPU options can be considered then??
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
It is hard to figure some of this without knowing the sizes of the chunks that will get moved, rendered, then written back to storage. What app are you using?
Oh, and welcome to the new guys. :)

After Effects CS5.5 x 64

Chunks are varying in size anywhere from 1GB up to 80 GB..

Theoretically, if all PCs finish rendering at the same time and try to write back, it will NOT be exceeding 1GB...

Thanks.
 

richaron

Golden Member
Mar 27, 2012
1,357
329
136
Local caches (the amount of memory each machine uses) aren't really that important, they are just an example of how each program handles things differently so the answers to your question might be different.

Still, I don't think it's worth spending more on your network connection.

The more important aspect is your storage. With a gigabit network you have 125MB MAX (you will never get that fast). If your two hard drives (in raid 1) can write at 100MB/s, that means you can handle 5 frames every second. HEAPS fast enough since your computers will spend much more time rendering than they will sending data over the network.

I would use a 1Gb network & the hard drives you have now. If later you realise something is slowing it down, you can change it then.
 
Last edited:

skyking

Lifer
Nov 21, 2001
22,543
5,589
146
If things got a little slow, I'd consider a second server to split up the traffic. The switch backbone can handle far more than any single port, so if you had the jobs on two servers instead of one it would truly double your capacity. You could run a cron job on off hours to back one up to the other in a backup directory.
 

dclive

Elite Member
Oct 23, 2003
5,626
2
81
If things got a little slow, I'd consider a second server to split up the traffic. The switch backbone can handle far more than any single port, so if you had the jobs on two servers instead of one it would truly double your capacity. You could run a cron job on off hours to back one up to the other in a backup directory.

That's what link aggregation is for; one machine can handle two NICs, combined, to double the bandwidth, assuming you have multiple GigE streams coming from multiple GigE clients.

But I suspect with the slow machines doing the rendering that won't be much of a problem to begin with.

OP: Bring in an Adobe AE consultant to design a network. The advice you're getting needs to be focused on the Adobe AE product. It's very, very unlikely the machines you have can come even close to swamping one GigE channel, much less two; I won't worry about it. Just get in an AE expert; you'll save yourself tremendous time and frustration.

It would also probably be cheaper (and roughly as fast), from a licensing point of view, to upgrade say 5-7 client machines to i7 CPUs and render only on them rather than rendering on 20 old machines.
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
Local caches (the amount of memory each machine uses) aren't really that important, they are just an example of how each program handles things differently so the answers to your question might be different.

Still, I don't think it's worth spending more on your network connection.

The more important aspect is your storage. With a gigabit network you have 125MB MAX (you will never get that fast). If your two hard drives (in raid 1) can write at 100MB/s, that means you can handle 5 frames every second. HEAPS fast enough since your computers will spend much more time rendering than they will sending data over the network.

I would use a 1Gb network & the hard drives you have now. If later you realise something is slowing it down, you can change it then.

Nice logic.. First, anyway, I will get a GigE switch and try some tests to see any improvement/lags etc... then I can monitor both the net traffic and disk traffic and decide on which to invest more if need be..

PS: 2 x 15K at RAID 1 (each 4Gb/s) : don't this give me a faster reading then RAID 0 or a single drive???
 

dclive

Elite Member
Oct 23, 2003
5,626
2
81
Nice logic.. First, anyway, I will get a GigE switch and try some tests to see any improvement/lags etc... then I can monitor both the net traffic and disk traffic and decide on which to invest more if need be..

PS: 2 x 15K at RAID 1 (each 4Gb/s) : don't this give me a faster reading then RAID 0 or a single drive???

How about you figure out where your bottleneck is first BEFORE spending lots of money? :)
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
If things got a little slow, I'd consider a second server to split up the traffic. The switch backbone can handle far more than any single port, so if you had the jobs on two servers instead of one it would truly double your capacity. You could run a cron job on off hours to back one up to the other in a backup directory.

my solution to that was to use an Intel server nic adapter

http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-i350-server-adapter-brief.html

which would connect to the switch with 4 separate ports and then by link aggregation I can get 4 times the bandwidth..

copying things around is already quite a hassle for me , so I do wish to use a single network share as much as I can.
 

uzuncakmak

Member
Apr 2, 2012
35
0
0
That's what link aggregation is for; one machine can handle two NICs, combined, to double the bandwidth, assuming you have multiple GigE streams coming from multiple GigE clients.

But I suspect with the slow machines doing the rendering that won't be much of a problem to begin with.

OP: Bring in an Adobe AE consultant to design a network. The advice you're getting needs to be focused on the Adobe AE product. It's very, very unlikely the machines you have can come even close to swamping one GigE channel, much less two; I won't worry about it. Just get in an AE expert; you'll save yourself tremendous time and frustration.

It would also probably be cheaper (and roughly as fast), from a licensing point of view, to upgrade say 5-7 client machines to i7 CPUs and render only on them rather than rendering on 20 old machines.

Until now I have consulted Cisco, Intel and Dell and now I will consult both Adobe and Nvidia (for a GPU/openGL based rendering solution e.g. tesla)...