renderfarms

deadalvs

Member
Dec 2, 2006
44
0
0


hi !

i'm in the 3d field now since many years. but still i am thinking about how a possible low-end setup for a renderfarm actually is started best.
packages like my favourite renderer ( maxwell, http://www.maxwellrender.com ) surely are capable to communicate with any number of slaves via tcp/ip.

so i could basically go out and buy any number of pc-junk and build a farm, but i would not be able to get far because of ram issues. when rendering a little more complex scene, i had my overclocked core 2 quad system on win xp 64 with 4 gigs of ram write out to the disk since ram was full.
well. a chain's just as strong as the weakest element, so 8 gigs per node is the bottom line today. i'd go for 16.

what's the best way to go?

has anyone some experience in that field?


 

deadalvs

Member
Dec 2, 2006
44
0
0

of course, to keep expenses low, a free 64bit linux would be needed. which ones is recommended ?
 

Fullmetal Chocobo

Moderator<br>Distributed Computing
Moderator
May 13, 2003
13,704
7
81
This might be best in General Hardware, but if your goal is to get the most efficiency out of a configuration, DC would be a good place for information. From reading around here, single powerful machines seem to be more efficient than multiple slower machines. I haven't used the program you mention, I did dabble quite a bit in 3D Studio Max R6, and scene setup is all about the video card--render farms won't help you much in this area. The power of render farms comes when it is time to render that scene. I would think the best configuration for you would be a single powerful machine setup for rendering, and then add machines to your render farm as money allows.
 

deadalvs

Member
Dec 2, 2006
44
0
0

hey man.

scene setup is only one side of the coin, and certainly for working on the scene it's best to have a powerhorse. but @ rendertime, pure raw cpu power counts. and maxwell needs A LOT (maybe you also want to work on your computer while rendering...). check out the specifications of the renderer. it basically simulates a full camera setup (every shot has a shutter time set, film material, f-stop, ... all as a real camera), all based on physical laws and optics. and it calculates all in 32bit (.hdr) range. the other important point is that it renders the full frame without any buckets. so the quality of the rendering simply increases over time. the quality itself is measured in SLs (sampling levels) with exponential timesteps from SL to SL. so basically you can render forever on an image.
now. a standard production image in architectural visualization as i've done a lot takes from 4 to 15 hours to render on an overclocked quad today, but i've seen images rendered on multiple nodes over a period of 100 hours to clear up noise. for example images with chromatic aberration, caustics, or subtle diffuse lighting need a long time to render. and as the field of architecture is... a lot of different versions are needed sometimes in a very short time (six images in an afternoon... i've had that.)

certainly one could just pick a different renderer with a biased approach (photon mapping like mental ray, vray, brazil, ...) which render the image itself a lot quicker by sort of faking the lighting. but the actual lighting setup takes MUCH longer (i worked on the lighting setup for an office interior for a full day. just tweaking parameters in mental ray !!). maxwell ? just place light emitters or the sun and you're done. there are no settings for nothing except for the camera options. this is why i love that renderer. same quality every time. no tweaking !

also, the problem with renderfarms is that most of the time they're running at low usage and sometimes, the peaks seem to explode where one wishes to have access to blueGene. ( http://www.top500.org/ )

so the bang-for-buck must be worth more than in a gangbang. money, space and technical knowledge also are limiting factors in the broad field of architecture.


my dad also says that he'd rather pay a lot more for one huge computer (let's say 4xquad xeon under one hood), but this is a waste of money based on raw cpu power that is accessible with lo-price slaves (a rule of thumb: 20% more power costs the double amount of money). of course, there are a lot of nice labs out there that sell very sweet rigs, but at what prices ?? see for example http://www.appro.com/

this is also why i started this thread here, to find out what can be done againts horrific prices:
http://forums.anandtech.com/me...id=27&threadid=2187473

the helmer computer mentioned at this thread there has six nodes and costs around 3500$ ... a great price for that amount of power !

so the main question exactly was... how should those machines be built ? what are the rules behind looking out for possible hardware ?

blades ? off-shelf computers ? self-made lego computers ? single components just mounted to a wooden frame with some metal around ?
difficult... i know !

 

deadalvs

Member
Dec 2, 2006
44
0
0

hi ! thanks for that link... really seems interesting. but do they still exist ... i mean last update of their homepage was 2004 ... also based on cpu speed...


 

networkman

Lifer
Apr 23, 2000
10,436
1
0
I guess we'll soon know - I dropped 'em an email to see if they're still in business and what they've got available. :)
 

deadalvs

Member
Dec 2, 2006
44
0
0

nice, dude !

well, most of the time, the hardware is not such a trouble to find or even finance... it's more the work / brandname / salaries / OS / shipping / and so on that also have to be payed, which can be quite a sum !

nothing's free... i'm interested if they still are in business... i wonder about their prices.


did you check that helmer site (http://helmer.sfe.se/) ? awesome what that dude is doing with his money !

 

networkman

Lifer
Apr 23, 2000
10,436
1
0
As promised, here's the email thread of our conversation, my letter first, followed by Ross's response:

----- Original Message -----
To: <info@rendercube.com>
Sent: Saturday, May 17, 2008 6:15 PM
Subject: Question regarding company status and inventory on-hand

> Hello,
>
> I saw your webpage and was curious if you're still in business, as the
> last model mentioned is from June 2004? Do you have other model systems
> available utilizing dual or quad-core technology?
>
> The elegance and design of the cases really caught my attention. Even
> just being able to get a price on a quad-motherboard capable case and
> power supplies would be of interest to me.
>
> Thanks!
>
> Rich Nagel
>
> PS: My interest isn't so much for rendering as it is for Distributed
> Computing.



Hi Rich,

Thank you for your inquiry.

We are still manufacturing a chassis, however the design has changed
significantly from the now-dated products on the RenderCube website. Most
of the chassis are being used for scientific computing applications. You
can see the latest design configuration at www.pqs-chem.com . Look under
their products page. You will see a two-motherboard system (16 core) design
and further down the page, a four motherboard system (32 core). Both
systems use Intel's latest quad core technology, however the design will
accommodate AMD's processors as well. The straight-through cooling works
quite well and the systems can be used at the desk-side.

The dual quad-core system (four 5400 Xeons, four 300GB drives and 16 GB
memory) is about $7500. Price varies with processor speed and quantity of
memory.

Thanks,
Ross

Ross Lyle, PEng
RenderCube Technologies
2961 Charlotte Drive
Victoria, BC V9B 4L9
Canada


So there you have it -- another website to look at and products to drool over! :p
 

deadalvs

Member
Dec 2, 2006
44
0
0

this is just sweet... it's nearly like some sort of non-profit organization ! i love that...

and they even have academic pricing ... sweet !


 

Fullmetal Chocobo

Moderator<br>Distributed Computing
Moderator
May 13, 2003
13,704
7
81
I was doing some research, and I found this thread over at XS. If you look at post #3, you might want to get in touch with angra. Just thought this might be useful. :)
 

deadalvs

Member
Dec 2, 2006
44
0
0

ah, this is cool... thanks for that link. i guess i`ll have to ask that guy what raytracers he uses...

surely. rendering on GPUs would be much, much faster. but i`ve seen no true quality raytracing software except for gelato or similar that actually runs on GPUs. i`m surely waiting for it, but i guess there are more problems to solve to have this work. like floating point problems.

i guess povray would also work on it... i just never used it.

 

deadalvs

Member
Dec 2, 2006
44
0
0

it seems the cheapest overall solution is always q6600 based with overclocking. how far can the quad q6600 be overclocked with just fans or an active copper sink?
 

Fullmetal Chocobo

Moderator<br>Distributed Computing
Moderator
May 13, 2003
13,704
7
81
Originally posted by: deadalvs

it seems the cheapest overall solution is always q6600 based with overclocking. how far can the quad q6600 be overclocked with just fans or an active copper sink?

According to these (Thread 1 and thread 2)
3.2 - 3.4 is doable fairly easily with an aftermarket air cooler. After 3.4, voltages have to be increased. I don't OC much, but I know a few crunchers do, so they might be able to give better info. :)

 

deadalvs

Member
Dec 2, 2006
44
0
0

i've seen one of the threads searching on my own. so thanks for the other one. :)

it's so tough to choose a rig though. there are so many possible rigs to buy and i'd like to go for components that let me OC for sure up to 3.6 even if i had to run a lot cooler for stability's sake.

by the way... would you overclock in a 1u or 2u rack ? if i really will go for a little more cpu power i'd really be able to put them in one place neatly in cheap 1u or 2u cases. looks way cooler.. :)

when putting a farm together with overclocked 6600s you can get horrific speed @ a pretty good price !

going with a free 64 bit linux and 8 or even 16 gigs of ram ... *dream*

* * *

i'm actually thinking of one day starting an independent renderfarm that is NON-PROFIT oriented. so basically every penny that is coming in is invested in hardware, support, admin work or licenses for the renderer(s)

you think that would work ?? join me... ! *hehe*


 

Fullmetal Chocobo

Moderator<br>Distributed Computing
Moderator
May 13, 2003
13,704
7
81
I would OC in a 1U with this on the procs. But that's about it. ;)
It all depends on the temps--if I needed them in a 1U or 2U, I'd put them in there and see what the temps were like. Some people were OCing to 70c, which is amazing hot to me, and still being stable.
 

deadalvs

Member
Dec 2, 2006
44
0
0

yep... watercooling's cool... but the maintenance and risks (more elements that are able to fail) are a little higher. my main system is in a thermaltake (prebuilt watercooling) case and i had no problems so far with my Q6700 2.66 @ 3.3 GHz.

just when having a farm you cannot have all computers watercooled.

* * *

the PSUs in 2u racks are special ? would you go for a rack case that has a built-in PSU ?

i'm still trying to find the cheapest 1u or 2u case that could support a slightly oc'd q6600...

where do i find such parts best ? newegg.com ?



 

networkman

Lifer
Apr 23, 2000
10,436
1
0
Hey, thought I'd let you know about another possibility. This link came across the listserv of the Linux Users Group that I'm a member of:

http://www.dvhardware.net/article27538.html

"The system uses four NVIDIA GeForce 9800 GX2 graphics cards..."

"This new system is used by the ASTRA research group, part of the Vision Lab of the University of Antwerp, to develop new computational methods for tomography. The guys explain the eight NVIDIA GPUs deliver the same performance for their work as more than 300 Intel Core 2 Duo 2.4GHz processors. On a normal desktop PC their tomography tasks would take several weeks but on this NVIDIA-based "supercomputer" it only takes a couple of hours. The NVIDIA graphics cards do the job very efficiently and consume a lot less power than a real supercomputer cluster."

Specs:

AMD Phenom 9850 processor + Scythe Infinity CPU cooler
4x MSI 9800GX2 graphics card
4x 2GB Corsair Twinx DDR2 PC6400 memory
MSI K9A2 Platinum motherboard
Samsung Spinpoint F1 750GB HDD
ThermalTake Toughpower 1500W Modular PSU
Lian-Li PC-P80 Armorsuit case
Windows XP 64-bit

A quick note for our American readers, a system with similar specifications would cost you around $3,900 at Newegg. Most computer hardware costs a lot more in Europe than in the U.S.


There's more at the link along with some pics and even a short video. Very cool! :D

 

deadalvs

Member
Dec 2, 2006
44
0
0
sweet... 8 GPUs ... :)

actually i've been searching on GPU computing a little in the past too. well. there's only one big problem: software's not adapted. and ordinary software cannot be speeded up with GPUs.


i mean there are renderers around like gelato (http://www.nvidia.com/page/gz_home.html, but they're still extremely young renderers and by thus not tools for every day's work.

i'd surely go for GPU rendering if i had an adapted version for GPUs of maxwell render ! 100% sure !



 

networkman

Lifer
Apr 23, 2000
10,436
1
0
Might at least be worth a look, plus you could contact the people involved and see if they've been considering any other applications for the hardware. ;)