TEAMANANDTECH Chessbrain (now we know)

[Insidious]

So, this explains what we have been seeing the last day or two:

Posted: Sun Jun 29, 2003 10:14 am Post subject: We're upgrading the ChessBrain hardware!

--------------------------------------------------------------------------------

The SuperNode server is being moved to an AMD machine with a gig of RAM. You may experience PeerNode inactivity, as we prepare for the transfer. We should have the new server up and running by Monday morning GMT.

The new machine should resolve many of the issues I've written about in the past few weeks

If you have any questions send me an email: cjus@chessbrain.net

 

[winr]

Thanks!!!

 

[panhead49]

wonder what machine was running it before??

 

[Insidious]

CBs weekly update at www.chessbrain.net

Posted: Mon Jun 30, 2003 11:52 am Post subject: Weekly status report: June 30th 2003

--------------------------------------------------------------------------------

This past weekend has been the absolute worst weekend in ChessBrain?s relatively short, two year, history. That said? several miracles did occur to soften what would otherwise be a serious disaster for the project.

Our troubles began last Friday June 27th at about 2 am PST. Those of you who know me? know that I would likely still be awake coding? Well - I was. At that time I noticed database errors appearing on the SSH window that I use to casually monitor the server.

The database errors were due to I/O disk errors. The hard drive on the SuperNode server appeared to be failing. After uttering a heart felt ?oh 'poopies'?, I stopped the server and retrieved the stats database Yes ? stats first! Many of our members will tell you? the server can crash, the hardware can die, the project can ultimately die? and so what! - The stats better survive!

Well, the good news is that the stats are fine.

After I realized the IO problem, I sent out a few urgent emails. Shawn Sullivan at Games Parlor answered my emergency email and quickly responded by offering us a server at www.gamesparlor.com. By morning we had another server to work with! I quickly setup a supernode at Games Parlor, while Gavin Roy at bteg networks worked to move us to another machine.
One problem we have is that the PeerNode clients do not have the ability to move from one server to another by themselves. Instead, the PeerNode operator has to manually edit the cbspn.conf file to enter the new address of the SuperNode server. We?re working on a PeerNode update, which will query the chessbrain.net site for routing information and automatically move between SuperNode servers as needed.
Gavin offered us an Athlon XP 1900+ with 512 Megs of memory, running with Gentoo Linux. We soon discovered an alarming problem. The SuperNode server started complaining that it could not create more than 200 connections (tech note: each connection is on a thread). The problem had me stumped for most of the day. I finally located the answer in a Gentoo forum post by zojas. Zojas has the same problem and discovered that each thread process gets 8 megs of stack space and since there is a 2GB virtual address space you can only have 256 threads per process. { 2GB / 8MB = 256 } The solution involved limiting the stack space per thread using only a few lines of code:
Code:
struct rlimit lim;
lim.rlim_cur = 1 * 256 * 1024;
lim.rlim_max = 1 * 256 * 1024;
setrlimit( RLIMIT_STACK, &lim );

http://forums.gentoo.org/viewtopic.php?t=51289&highlight=pthread
With this final problem resolved ChessBrain is now back online on a good machine with twice the memory. BeoServer is now enjoying the use of a 256MB hash table

I simply can?t thank Gavin Roy and Shawn Sullivan enough for their support over the past weekend. If it were not for their support ChessBrain would be ?dead in the water?.

During the next week or so you may notice erratic PeerNode behavior as we tweak the server and test new versions of the PeerNode client. The new PeerNode will be an optional download ? but chances are good that you?ll want to use it.

Thanks to Gavin, Shawn and Walt Howard we now have three SuperNode servers online and will use them to test the new PeerNode software. The idea is that if one server goes down, the PeerNode will be able to migrate to another server.

We?d like to use the NullSoft installer for all of our Windows applications. If you have experience with the installer and would like to help us package up our wares? please send me an email! Also, I?d like to see a rewrite of the PeerNode client GUI application? if you have Windows programming experience and would like to write the new client ? send me an email!

- CJ