• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Anything Better than ProFTP?

T

TechBoyJK

Lifer
About to make the final decision to implement ProFTP on a FreeBSD server that will be serving only image files smaller than 150k (gif jpg etc).

This is for a company that plans to put a lot of marketing behind this, and they have a fairly reasonable chance at building a great site (nda in effect here). They are with a host that could easily support eBay, so bandwidth limits on the provider's end are not of concern.

The server itself will be a 3U rack with dual 1.8GHZ P4 Xeon with 512of ddr ram and eight 120 gig Western Digital drives with the eight meg buffer on raid (which ever provides mirrored-for fail safe) It has dual gigE nics as well. If this server's resources get consumed, will ProFTP be able to handle it?? I think it will.

Does anybody have any other ftp programs that would be a better choice? I am going to set the decision to go with Proftp in concrete in about a week.

ps, for a server this specialized (only a few services and only serving/accepting photo files under 150k) do I need any more than 512megs of ram?
 
According to the list on their site (linky) kernel.org uses ProFTPd.

Your choice of hardware seems a shade odd to me. If the site really needed that kind of horsepower, it should probably be load balnced across multiple servers.

bart
 
Just out of curiosity

1.) Why didn't you follow through on HTTP upload?
2.) Are you really gonna use ColdFusion on FreeBSD?
3.) I couldn't find pricing for individual "U" of rackspace at Cybercon. Did you pick a different host, or get a custom deal?
4.) Are the database, router, and firewall questions you've asked related to this project (the image server), or just other stuff you wanted to know about.

bart
 
How many simultaneous connections are you anticipating at both peak and average?

I'm having a difficult time envisioning a realistic FTP serving scenario where those resources you specified are necessary.

edit: You are also going to want to seriously consider your hardware/OS choices here -- I would recommend researching or asking a BSD user here how the SMP and particularly the hyperthreading support is with the latest version of FreeBSD compared to linux -- You'll probably get the most bang for your buck with linux given those hardware specs.
 
please tell me you aren't using IDE drives in a "real" server.... please... if the site will be for a company and could be big then , build it right, go buy a server, dont build one out of parts, and it should at least be a failover cluster of two machines... Compaq makes some nice ones for this
 
IDE Raid? In a "real" server? That is not a great idea.

You can get a GOOD Dell Small Business Server for under a grand.
 
1. This is not the only server in the farm. We will probably need a full 42U
2. This is not the main photo server, this is for the extra bulk that won't see as much traffic
3. FreeBSD/Coldfusion is not on our plate.
4. The servers running database/coldfusion/email/main photo server will all be scsi raid
5. Firewall=Packet Filter/OpenBSD
6. SQL=MS2k Server MS SQL
7. Coldfusion=MS2k Server
8. I've read enough to see that the WD Drives w/8meg buffer is not that much slower than a standard scsi drive. Yea its slower, but its hella more expensive to get 960 gigs worth of scsi drives on a raid config.
9. FreeBSD has MP support, OpenBSD does not. FreeBSD should kick some major ass on this platform.
10. HTTP upload - it just doesn't sound safe, but I will look into it. It would be nice because I could remove the entire ftp service from the server and do everything from Apache 😉

Like I said, this particular server is for the extra stuff that won't see as much traffic as some of the other machines. Storage capacity is the major concern on this puppy.

There are two photo servers, primary/secondary. The primary server will hold the first three of 10 possible photos to be uploaded. This baby is definately scsi.
 
SCSI devices are alot faster in a server type of environment where alot of threads are doing alot of different things, in a desktop environmet, your right, the WD's are about the same.... Secondly, SCSI is proven and VERY reliable over time, which cannot be said for any IDE device I know of. also, the simple crap with master/slave, 18in maximum cable lengths and other issues make using more than 2 or 3 IDE devices a total PITA. The only reasonably fast IDE raid controllers I know of are the adaptec 2400A and the 3ware families.... I've used both and believe me, they cannot compare to a scsi array.... The adaptec will not allow your eight drive config, so you'd be forced to the 3ware... and you cannot use slave devices on it so you going to have 8 IDE cables of no more than 18 inches going to each drive...now thats a mess.


Also, if these things are just FTPing then a P2-350 could handle the load(with ease), little reason for a dual 1.8 config... you might want to let us know all the servers configs and uses and we can give you a thumbs up or thumbs down....
 
Congratulations n0c, spidey, and N11, we've just helped TechBoy be a one man IT department 🙂

Tech, how much more can you tell us about the setup without breaking your employment rules? I always find it really interesting to see other real-world examples.

What is each machines setup? (hardware + OS)
What is each machines purpose?
How does traffic flow between them?
Are they NAT'd by the firewall, or really routed?
How will you get the first three photos to go to one server and the next 12 to the other?
At any point did a SAN appeal to you?
Was coldfusion a legacy decision, or is this a new platform launch?

Just a couple comment:

for that server if storage is really the important part you may want to ditch the whole dual-xeon thing, and go with dual p3's. You won't need DDR (its just fileserving out a 100mbit pipe at most). You won't need the extra CPU horsepower either. Unless you're actually processing the images (like you have a script that autmajically photoshops them down to under 150K) you'll get the same performance.

If you've already got the CPU's, and you're looking to save bandwith, you coud look into mod_gzip. It basically checks to see if the browsers supports recieving compressed files, and if so will gzip them before sending it across the network. This wont help anywhere near as much for images as it does for text/html, but hey if you've got 1.8ghz xeons to spare, try it out.

Are you using any sort of scripting language on that webserver? If you tell us which, somebody could probably cook up a demo upload script for you to see if you like the HTTP upload.

bart
 
A lot of websites are load balanced via an alteon acedirector, cisco localdirector, f5 bigip, etc. If your site does get popular over time it would be beneficial to you to have an easily scalable setup already. Image hosting could could be done on a gige box on the backend serving up pics to the load balanced webservers, which are uploaded via http. 99% of the sites I've seen use http upload for something similar to what you're doing. Your sql box could be a gige box on the backend as well. And also, no matter how much you think you need 960gb for a pics website, you probably don't =)
 
i like pure-ftp
Pure-FTPd is a free (GPL), secure, production-quality and standard-conformant FTP server based upon Troll-FTPd. It doesn't provide useless bells and whistles, but focuses on efficiency and ease of use. It provides simple answers to common needs, plus unique useful features for personal users as well as hosting providers.
Click to expand...

users of pure-ftp
 
Hey guys, just got the network back up and going. Out here in a rural area, so no more broadband! I am going to try and post this info as best as possible.

The bulk of the stuff will come in a following post, soon....

Coldfusion is not a legacy decision. The best and most trustworthy programmer I know that understands this project best and has the most skill in coldfusion is who is hard coding the site. I thought about trying to port the site to PHP, but this won't happen until the entire framework and most of the site is finished, tested, and worthy. Luckilly our programmer comments his code very, very well, and makes comments about his "general" programming logic. This way if someone needed to port it to PHP, they would understand what needs to be done.... i hope... We have already invested about $35,000 in programming.

This system will consist of about 12 servers.

Hint Hint, this will be an auction/trading site, one that is intended to one up eBay. I can't go into much detail, but I have a huge interest in this, but the real weight will be carried by a few partners/investors/owners that I have lined up/locked in. One of the investors/partners and I already hold the record for the largest grossing production auction in eBay's history. I more or less consulted him on the project that was brought to him by eBay, and I actually did most of the work involved with setting up the "computer/internet" portion of the auction, as well as developed the main offline advertisement.

Budget should give me about $75K-$100K to work with, but I want to work with the $50k figure for now. Those who know why I want to do this will understand.

Our main server farm, the one I have to assemble/design, will be housed at Cybercon.com's HQ's in St. Louis. We are looking at about 12 servers to start. Once setup, most of the real maintenance will be done with Cybercon's staff looking over my shoulder or them doing it themselves. As luck would have it, they have to do everything at my say so. ie, I have to approve/disapprove of everything that is done to our servers.

Once we go live, I will be moving to a corporate apartment a few blocks from Cybercon.com. I have talked to my board about having cybercon.com/phone company run a dedicated t1 (or something similar) to the apartment so that I have an actual hardline direct connection to the net. I dunno yet.

I am only 22, and this project, if launched successfully, will help me solidy my place in a circle of great businessman. I started programming when I was 5, and did it seriously until I was about 12/13, when I took a greater interest in music. Go Dream Theater!! >>> just so you know where I am coming from.

Anyways, I spit this up because I haven't been able to post in a day or so, and wanted to lay some info down until I could get the "goodies" up and going. And, yes, if I had the budget, I would use fiber channels. Maybe during phase 2!

----Cybercon gives me an ethernet connection off of their routers. They say I don't need any kind of router in the farm. Only a switch/firewall. They have a great power backup system, so I really don't need to figure in APC battery back up to the farm either, even though i would feel safer if the farm had its own independant power backup.

I will get back to this soon....
 
Actually, I understand programming, but I am much more interested in infrastructure and networking. I am studying for my A+, I-Net+, Network+, Server+, EBiz+, and then TechTrainer+, in that order. Hopefully this time next year, I will have them all. However, I'm not here to defend/promote my personality.... I don't think I'm that interesting.
 
RE: Photoserver

I was actually thinking about using an AMD Athlon XP 1600 (1.4) with 512 ddr ram instead of the dualMP setup. Yea/No?
 
I think that's probably more information than you wanted to give.

Though I see several flaws in this discussion, good luck in whatever you do.
 
Funny thing, I was reading about how ebay started, and their server setup. Pretty gaggle-ish too.

Tech: If you haven't actually bought the server yet, good lord, get a dell/ibm/hp. Make sure you get the 24x7 4hour parts + labor warranty.
That or build two customs, jury rig your own replication setup (could be an cron job'd rsync) and set them up for failover.

bart
 
You must log in or register to reply here.

TRENDING THREADS

Back
Top