Optimum number of files to FTP at once

[Syringer]

So we have a data feed management service, where we basically process datafeeds containing product information for a particular merchant, and at night we send them out to comparison shopping engines, like shopping.com, pricegrabber, etc.. via FTP.

There's about 1000 files total, and maybe 10 different FTPs that we send to. The files range in size anywhere between 1mb to 100mb.

My question is then is there an optimal number of files to send out at once? Too many files at once create the potential for it to timeout--whereas one at a time will be inefficient especially if it's a big file. So is there a way to determine the happy medium?

 

[Texashiker]

I think its going to be determined by the speed of your connection and the ability of your server. Some web servers can serve up thousands of connections and files every second. While other servers will buckle with just a few dozen connections.

Your probably just going to have to play around with it and work the numbers out for yourself.

 

[imagoon]

Generally:
Huge files: IE say 5MB -> gigs typically 1-2 at a time.
Tons of small ones: 2-10.

Basically lots of small files have a ton of setup and xfer over head so you can get more than 1 at time quickly. Large files (assuming the ftpd doesn't limit the connection) will max out a connection in most cases assuming the hardware can push it.