• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

TeAm Stats... Poll and input needed from youz guyz!

GeoffS

Lifer
SO... as many of you know, I've got to pull the plug on the stats server. It's been quietly running in my ex-employer's server room for the last 4 months, however I just got a request to have it out by the 21st. Now, I'm going to see if I can set it up in my basement for a while as an interim solution, however I'm on cable through Adelphia, who blocks port 80 and it's possible that I'll end up with a varying IP address should there be a power failure, etc. Not a big deal, just an inconvenience more than anything else.

However, I think the time has come to move the stats to a hosted site. As many of you know, they have caps on disc space and overage charges, etc. The FaD database is currently running at about 2gig of data (12M records total) and the oldest database, D2OL is also around 2g (about 13M records total). Both database have data from the day I turned them one (Jan-01-2001 for D2OL and Nov-09-2004 for FaD).

The question I have is how much data is actually required? Just totals and today and yesterday? The last week? Month? 2? 6? I dunno... I'm a data guy and like seeing lots of data, but how much of it is really relevent I guess is the question. Not many (if any) of the stats sites I look at (and to be honest, I generally don't look any any unless I've got data issues and am trying to see if my data is right) carry as much, if any, historical data, and maybe that's something that sets my stats apart from others.

The answer to the question on how I proceed with setting up my stats going forward depends heavily on how much data you guys want to see. Without a doubt, at this point, I'm going to need to either host these somewhere, or pay for a pipe and static IP address to my home. Expense is going to become an issue and I'll probably have to resort to a PayPal button on the pages somewhere, and/or GoogleAds... whatever... the free ride from my ex-work was nice while it lasted.

Stats that will be removed very shortly are the LifeMapper (dead project), DPAD (needs a rewrite... my numbers are all wrong), seriously considering canning D2OL and TSC unless there are objections, and SoB has virtually no traffic so I'm not sure there is a need to continue providing those stats (I'm not even sure if they are right it's been so long since I've looked at them and I've had no emails telling me they are wrong). That leaves FaD which is ending in a month.

So, the second part of this is suggestions for new projects... I've been asked to do Rosetta, and I think I owe the DPAD folks stats. I understand that once you do one BOINC stats page, the rest are easy to roll out as the stats format is the same for all of them. If this is true, then we seriously need to look at how much data to retain as I can't afford to have 10 projects each with a gig of disc space hosted! lol...

So, pls give me your input and participate in the poll!

Thanks!
Geoff

[EDIT] Server stats: This MySQL server has been running for 57 days, 6 hours, 47 minutes and 20 seconds. It started up on Sep 15, 2005 at 07:37 AM. Since its startup, 159,388,020 queries have been sent to the server.

Of course, it's done lots more than that... this is just the longest it's run without me rebooting it because of software updates, etc 🙂
 
Ok, I've voted for totals and daily for the past month, and stats for Rosetta sounds good to me 🙂
All Boinc projects would be good but I see your point about storage, especially since new Boinc projects seem to be coming out every other week!
 
i voted for past three months...just seems like aquarter's worth of data should be good.

I agree with sh!t canning the D2OL stats...that project has one foot in the grave anyways. I guess there is no need for the FAD anymore either :disgust: 🙁

Oh well, if you put a paypal button up there I will be sure to hit it since I goto your stats site several times a day!!!

Slatz
 
My stats currently have daily production and daily totals at the user and team level, since the creation of the stats pages. Any summarization I do is (with the exception of overall totals as noted above) is done on the fly. I don't have 'month-end' processes that would accumulate a month's data... and I don't have any automated purge routines, but that's going to have to change! 😛
 
Ideas to consider:

- You could probably steal the graphs from the Seventeen or Bust web site, to at least view graphs of older data.
- You could store a week's worth of data, or only TeAm Anandtech data, or something on the hosted site, and refer people to your dyndns:8080 home site for more.
- You could store only static cached pages of your site on the hosted site, and push that data up to the hosted site automatically on a regular basis. I assume this would be cheaper since you wouldn't have to pay for a database.
 
The database usage is included in the hosting price... it's the disc space that will drive the price up. That's a really interesting idea though to have the database living on my home server while hosting the stats from a hosted server... wonder what that would do to throughput... hmmm....
 
Yeah.... let's wait till the night crew shows up... if there's still minimal interest, then I'll take that as direction...
 
Originally posted by: GeoffS
Yeah.... let's wait till the night crew shows up... if there's still minimal interest, then I'll take that as direction...

Dracula, Frankenstein, Blade?
you might be waiting a long time 😛 he he he!!!
 
Originally posted by: Freewolf
I voted this morning.
I'm sure you will do what's best for everyone and I trust your judgement.

He is the boss 😛

And I agree wholeheartedly 😛

I voted everything just because I am a stats ho!
 
Originally posted by: Freewolf
I voted this morning.
I'm sure you will do what's best for everyone and I trust your judgement.

Sounds very much like a threat 😀
My fleshy bits are already spoken for 😀 😀
 
Just a suggestion for hosting, I'm currently with Dreamhost for hosting, and they provide (from memory) 4.8gb of space and 120gb of monthly bandwidth for just under $10 per month. Have to say they have been excellent for the last 10months I've been with them 🙂
Even if you prune back the database to 6months, that should be plenty of space to do your stuff. You might just have to think about the amount of CPU strain the stats would put on a shared hosting server, if your scripts are CPU intensive...
 
Originally posted by: Coolkid
Just a suggestion for hosting, I'm currently with Dreamhost for hosting, and they provide (from memory) 4.8gb of space and 120gb of monthly bandwidth for just under $10 per month. Have to say they have been excellent for the last 10months I've been with them 🙂
Even if you prune back the database to 6months, that should be plenty of space to do your stuff. You might just have to think about the amount of CPU strain the stats would put on a shared hosting server, if your scripts are CPU intensive...


I'm using DreamHost also. I was thinking of having the bulk of the database on my home server (Dell SC420 3.2G P4, 1g ram, 2x120g SATA drives), and having that machine process the hourly updates, pushing static HTML pages back up to the hosting service (DH) site... and drill-downs from the static pages would hit my database instead of the DH database. It's an overhaul of my stats though as every page I have currently is dynamic. 🙂
 
Geoff, I can get you free hosting, and the space you need (well, I'm 99% sure I can).

I didn't read through this thread much at all (shoot me 🙁), so PM me with exactly what you need, and I'll forward it off to the right guy (his box is at ThePlanet, and should be solid for what you're looking for).
 
You can get static copies of some (or all) of your pages with WGet. Something along the lines of wget -Exkp -i list_of_your_server_URLs.txt should do it. Then the static pages will have links among themselves, but all other links will go to your dynamic pages.
 
Geoff,

If you like some donations to help out with the stats server(hardware or cash - sorry no stats-recording skill here) 😛 just let me know. I'd be happy to help out for the benefit of the TeAm! 🙂
 
Originally posted by: Ken_g6
You can get static copies of some (or all) of your pages with WGet. Something along the lines of wget -Exkp -i list_of_your_server_URLs.txt should do it. Then the static pages will have links among themselves, but all other links will go to your dynamic pages.

Oh! That's pretty cool!!! I was just going to have my daily/hourly process write out the code for the HTML pages, but that sounds like a much more elegent solution! So very cool! I think you just saved me a pile of coding... thanks!

Geoff
 
Originally posted by: networkman
Geoff,

If you like some donations to help out with the stats server(hardware or cash - sorry no stats-recording skill here) 😛 just let me know. I'd be happy to help out for the benefit of the TeAm! 🙂

Thanks very much Rich! You, and several others have made this generous offer of support. I'm not yet at a position where this is necessary, I think... I'm going to run a cat5 cable to my basement and hook the server up there temporarily. I'm going to sign up a new domain since I'm going to have to do stuff to it (redirect to port 8080, change the IP if my cable modem IP changes, etc), something really similar like ta-stats.com, and then, once the BOINC stats are written, move the code to a hosted service, with the database sitting on my machine here at home... it *should* work... but this is, again, new territory for me! lol... I'll only know whether or not I need help once I'm about hip-deep into it!
 
Originally posted by: GeoffS
lol... I'll only know whether or not I need help once I'm about hip-deep into it!

As with so many other things in life I quite understand. 😉

If you need or want the help don't hesitate to ask for it - we'll have your back. 🙂


 
Originally posted by: networkman
Originally posted by: GeoffS
lol... I'll only know whether or not I need help once I'm about hip-deep into it!

As with so many other things in life I quite understand. 😉

If you need or want the help don't hesitate to ask for it - we'll have your back. 🙂

what networkman said!!!

Slatz
 
Back
Top