Stats not showing up on tacube.teamanandtech.com

[NicColt]

Is there anything wrong with it. How often does it update.

I know I see
Last Updated GMT:
18 Jun 2001 01:30

but nothing changes.

 

[Russ]

It's showing:

Last Updated GMT:
18 Jun 2001 02:01

It updates every 30 minutes, but it's 30 minutes behind because the log files are gigantic.

Russ, NCNE

 

[Sloth]

Russ: Have you compressed the logs? Brad has a great utility that will compress the logs down to a single entry for each IP. That way instead of the 9 billion lines each day it will be in the thousands for you guys. Makes the stats faster not having to go through all the info each time.

S.

 

[Russ]

Yeah, I have the compressor. But, I have to set aside the time to go through and compress over 300 days worth one log at a time.

Russ, NCNE

 

[NicColt]

oh ok they will eventually show up then. thanks.

 

[bphantom]

Russ, the compressor does have the ability to compress multiple logs at the same time. Just do ./logcomp.pl -rewind=400. It will compress the logs for the past 400 days. To view all the functions, do ./logcomp.pl --help.

Brad..

 

[NicColt]

I just checked and it's the same as last night.

 

[IJump]

Where can I get the log compressor? I could use it on my pproxy. Mine has been up for just under 100 days and I would like to get a handle on things before they get out of hand.

 

[bphantom]

IJump, here you go: Log Compressor Script .

Brad..

 

[Russ]

Brad,

That rocks! Question, though: How long is it going to take to do 338 days worth and will it interfere/cause problems with the updates while the compression is ocurring?

Nic,

It's been updating every half hour just like it should. Latest is now 18 Jun 2001 15:00. I'm wondering if you have an old page stuck in cache?

Russ, NCNE

 

[bphantom]

Russ, depending on the sizes of your logs, it may take a little bit (hopefully under a half an hour). I would just comment out the cronjob that runs the stats and let the compressor crank at full speed. If I remember correctly your server has dual CPU's. To speed up the process, you could do:

In shell window #1:
./logcomp.pl -offset=-1 -rewind=169

In shell window #2:
./logcomp.pl -offset=-169 -rewind=169

This would split the compressor into two instances and each do half of the 338 days. Just for safety concerns, I would back up all the logs first. Make a note of the current totals from the stats, run the compressor(s), run stats, and verify the compressed totals. Everything will match (I've done quite a few tests myself).

Brad..

 

[HeavyIron]

So with that script I could setup a cronjob './logcomp.pl -offset=-7' to compress the logs that are a week old?
You see compressed logs and the Statsaholic php stats don't get along.
If the log compressor works over a logfile that the php script has a location marker in, it loses it's place and causes an error.
So if I offset it a few days it gives the php script a chance to parse the logs and not lose it's place.
I was only saying this because I don't get OGR results in every day since it is not The Federations primary contest, at least not until RC5 ends.


BTW, I'm still using v1.0.3 and it has worked flawlessly since.... well for about a year

 

[bphantom]

HeavyIron, does ./logcomp.pl --help work? I'm not sure if v1.0.3 includes the -offset/-rewind command line features. But as for your question, yes, that will work. Issuing -offset=-7 will compress the "8th day" log file. The rewind feature is also quite nice. I use this on Mika's RC5 and OGR log files. About twice a week I backup and compress the log files, then use -rewind=-3 or -4. Works GREAT!

Brad..

 

[HeavyIron]

heh heh, I guess in v.1.0.3 the --help switch doesn't exist.
I just tried and what it did was just run the script.
Doh, here comes php errors

v.1.0.3 used the wildcard date setup like this '2000????'

I'll gladly upgrade

wget time

 

[bphantom]

wget is your friend. Especially for embedded Akamai video links.

Brad..

 

[NicColt]

well take a look at this pic of the page <---(stupid tripod) even though the page DOES update, the numbers stay the same. Look at the guy from netcom who has 29 for 883 and 0 secs. It was like that yesterday, I submitted 401 block mid sunday for the first time and they never showed up yet. However I have seen the first page or two update but not page 7 or 8.

 

[Russ]

Now, it HAS stopped updating, as of 09:30 Pacific time. I think I may need to shutdown the pproxy for a bit until I get this straightened out.

Brad,

I just did a test with the log compressor on one log (yesterday) and I get:

Use of uninitialized value at logcomp.pl line 80, 81, 74, 106, 111 and several dozen repeats for line 135. Any idea what I did wrong?

It did create the compressed file (.cmp) and it is MUCH smaller. But, how is ppstats supposed to know to use it, instead of the regular .log file?

Finally, I have two sets of logs. One with the prefix &quot;stats&quot;, which is what I setup when I first setup the pproxy. The other has the prefix &quot;daily&quot;, which you had me create to work with the Baby Bovine script. The &quot;daily&quot; ones are HUGE, about 10MB each, and the &quot;stats&quot; ones are about 2.5MB before compression.

Which of these files should ppstats be using? Which are supposed to be compressed? Why do I have two?

I'm confused.

Russ, NCNE

 

[cpars]

<< I think I may need to shutdown the pproxy for a bit until I get this straightened out. >>



Is my pproxy gonna be ok till you get back online Russ? here is a clip from my .ini

[rc564]
expertmode=1
logfilekeyblock=pproxyrc5
logfilekeyblockrotation=daily
minkeysready=20000
maxkeysready=50000
maxkeysdone=3

 

[Russ]

cpars,

That should tide you over fine.

Russ, NCNE

 

[bphantom]

Russ, what was the parameters you sent it? Looking at those lines in the perl script, that is where it grabs what you entered. As for the .cmp file, in the logcomp.ini you need to set logreplace=&quot;true&quot;. This will overwrite the original log files with the compressed versions. As for the logs, &quot;daily&quot; is only for the sendstatus program. &quot;stats&quot; is for the compressor. If the values in the .ini are set correctly, it should automatically figure out the log filenames. Sorry about the lapse of reply, I was fairly busy this afternoon.

Brad..