Looks like d.net is catching up

[Sinner]

Just checked my stats and it looks up to date.

I moved up nearly 100,000 places!!

It's also showing the work of my entire active herd yesterday versus just the ones on TA proxy.

892 vs 4777 Not to shabby I think.

 

[Chaoot]

Yep..gotta agree..

Personal stats are more or less finished, now its just a matter of (short) time before the team stats will show.. )

 

[IsOs]

DNET stat page is updated up to the 4th. But it looks like those blocks that were missing were gone forever!:|

 

[Russ]

Yep, IsOs, once again the effort gets flushed in the Dnet system. We can now add the 2nd and the 3rd to the list of days where production just disappeared.

Russ, NCNE

 

[dkappos]

Is there any way to backup the blocks before flushing. If mika's proxy could do that it would save us lots of block over time.

 

[IsOs]

<< Just copy your buff-out to before flushing it. >>

One of our co-members advised me that last September.

I was going through the backup tapes and found most of the buff-out files. My problem is I don't know which of these files are not already counted.

 

[Russ]

IsOs,

I think the Dnet keyservers will just ignore the duplicates and count the others. It's sure as hell worth a shot for the over 100,000 blocks of yours that they lost.

Russ, NCNE

 

[IsOs]

There quite a few files, so flushing them will take some time as well. I'll put these files in the harddrive so I'll have easy access to them by next week.

 

[ViRGE]

Isos, just merge them with the client first.

 

[Engineer]

As far as the 2nd and 3rd missing production, there are still over 10,000,000 blocks stuck in the various Dnet proxies Distributed net proxies. They are slowly falling, but I would bet we still have many blocks stuck in there

 

[IsOs]

Virge, how do I merge buff-out file, I know I can merge buff-in using checkpointfile. I'm using v2.7105.432.

I've download the new GUI beta clients but hesitant to run them. Are they faster than the 432 in crunching?

 

[ViRGE]

Just run the client with the -import <filename> command, and it will import <filename> into the right buffer. As for speed, I can't really say. No one's ever compared clients that were that far apart. However, if I had to make a bet, I'd say the beta was a bit faster.

PS I'd reccomend using the same version client as the newest one in your herd; I don't know how clients handle newer version's buffers

 

[IsOs]

Unfortunately, rc5desg.exe or guicntrl.exe did not recognized -import. I guess I c an try installing the beta and try it.

 

[ViRGE]

Ya, just install the beta, and flush with it; that's all that you really need to do.

 

[Sloth]

It looks like they have finally taken care of the problem.

As of 1/6 @ 23:30 there are just the normal 1000 or so rc5 blocks at the proxies and ogr is cleaned up too.

It should be a big day for the team with the old blocks coming through


S.

 

[SuenoNL]

don't forget proxy2.hou.tx.us (owned by dbaker) is still down.
as far as i know, it had ~ 4.000.000 workunits in if before it died.

edit: 3963853 to be exact.

 

[Sukhoi]

I'm kinda confused here...how is DNet loosing all these blocks? As far as I know, none of TA's SETI members have ever lost WU's. There have been times when the WU's weren't credited on the CGI stats page for a few days, but they eventually showed up.

 

[ViRGE]

I think a lot of it has to do with a few things. First, because standard pproxies hold blocks in their memory(they only write a hard copy everyone so often), a crash can make the blocks dissapear permanently. However, this would be a problem with out RR system, and Mikas; since I belive that Dnet uses proxy software that writes everything to a HD. The second factor is people not understanding how Dnet takes care of keyspace and issuing blocks. Not all of Dnet's keyspace is open at any one time. Only a certain range(of which I don't know the number) is open at any given time, meaning that it's much easier than people think to have their randoms not count and/or their blocks not count due to being done allready as randoms. The last, and lease likely factor is the master keyserver. The MK knows what blocks were done, by whom, and possibly a few other things. Every hour, it comiles a list of the past hour, and ships it off to the statsbox. However, due to the hourly nature of the MK, it might be possible that it may have some inadvertant holes in the logs it sends, where a fraction of time is missed. However, since that's a very unlikely situation, I'm willing to bet it's the first 2 that are the causes.

 

[Joe O]

Virge, Actually your third scenario has happened:
:: 13-Dec-2000 18:51 (Wednesday) ::

Due to an operational issue with the keymaster, 35 hours worth of OGR and RC5 log files were lost between Dec. 10 and Dec. 13. This is why the stats for these dates are so low. The work was recorded by the master, but it was not logged to log-files.
We are working on recovering the lost logfile information from the fullserver proxy logs at this time, and will post an update when available.

 

[Moose]

Virge is pretty much right. I would like to just explain one thing. The master works the same way as fullservers. It is constantly writing its buffer information to disk. Not only does it write completed block information (sets a workunit as finished in a subspace bitmap,) but it writes a log entry much like you see in a normal pproxy.

That log is rotated every hour. That is what is sent to the stats server.

As far as the lost logs from early December. They have been recreated from Fullserver logs. It is a matter of merging those logs and importing them into the database. To give you a perspective on how big of a deal that is, 1 day of Fullserver logs are typically 150+ megs compressed. From say 18 fullservers we are talking about alot of disk space needed. These files have to be uncompressed, split, recombined, and sorted then compressed again into hourly logs of all work.

This process is being worked on but is time consuming.

Thanks
paul

 

[Sukhoi]

Ah, makes sense.

 

[IsOs]

Does is mean that there's still hope