Guys, is it possible to lose 50,000 blocks to "dupes".

[Russ]

On the 12th, TA Cube was missing about 50,000 blocks. Dnet actually reported LESS then that which was flushed to our pproxy. We had 223k at the pproxy, and they reported 217k in the stats.

Since we have about 45 to 50k going directly to Dnet, this means that somewhere in the neighborhood of 50,000 blocks VANISHED.

Now, rather then start yet another thread here complaining about missing blocks, I decided to eMail the guys at Dnet to see what was going on. This is the response:



<< Date: Sun, 13 May 2001 23:43:45 -0500
From: &quot;Jim C. Nasby&quot; <jim@nasby.net>
Organization: distributed.net
X-Mailer: Mozilla 4.72 [en] (WinNT; U)
X-Accept-Language: en-US,en
To: Russ <russ@compucheap.com>
Subject: Re: More missing blocks

This is almost certainly due to dupes, which are possible even if you
don't run out of work due to the way that random workunits are generated
by clients. Until we can change that, there's no way to fix the problem.

Russ wrote:
>
> Hi Guys,
>
> Over the last three runs, 10th, 11th and 12th, TA Cube has been missing a
> substantial number of blocks. The first two days, around 20K a day but on
> the 12th about 50,000 vanished. Most of my mini team members flush through
> my pproxy, but we have about 40K to 45K a day that goes directly to your
> keyservers.
>
> On the 12th, we put 223K through my pproxy ALONE, and were credited for
> only 217K by you guys. And, yes, the pproxy was up and communicating with
> Dnet just fine during the entire period. Note the total here:
>
> http://tacube.teamanandtech.com/proxy/statsview/output/byemail.html
>
> And what was credited in your stats run:
>
> http://stats.distributed.net/rc5-64/psummary.php3?id=316304
>
> Add this to the usual amount that we flush directly and just the blocks
> that were LOST would put a participant in the top 50 in the dailies.
>
> Looking at the other &quot;Big Dawg&quot; participants, it appears that they too are
> missing substantial amounts of production.
>
> Any idea what is going on here? Are we going to get this work credited?
>
> Russ >>



So, what he is claiming is that fully 20% of the work disappeared because of dupes? As I told him in my response, I ain't buying it.

If we are losing - and it's not just us, check your totals for that day, there are a TON of blocks missing - that much production, then where is the incentive to even continue busting our humps on their project?

Russ, NCNE

 

[Adul]

mmmm we are only missing 9k from yesterday.

 

[Russ]

We're actually pretty close to the correct total for yesterday. A few missing, perhaps, but I really don't mind a few, or even a few K as I know that's part of the system.

But, 50K is not acceptable.

Russ, NCNE

 

[Choralone]

I too have noticed a ton of work on the 12th missing. My in-laws machine flushed 5K+ of mainly non-randoms (there were a couple hundred) through my Pproxy on the 12th. But when I checked my d.net stats they have me listed as submitting 1097 blocks for the day, a tad above my usual daily production. I haven't checked my proxy log to see exactly how many blocks were flushed but I can't believe that only %20 of all the work submitted through my proxy was good. :| I'll check it when I get home tonight and see exactly how much work was &quot;lost&quot;... This is starting to torque me off too...

 

[BadThad]

I was wondering what was going on! If this keeps up, I'm going back to SETI. :| I'm missing about 4000 wu's. I thought it was just dnet slow to update the stats, but now, it's been quite a few days and my wu's are GONE.

 

[JHutch]

One of the things that is particularly frustrating about randoms is this:

A random block is 2^28 keys large (1 WU)

A normal block can be up to 2^33 keys large (32 WU)

However, if the random block (1 WU) gets in first and is inside the 32 WU range of the legitimate work, ALL of the 32 WU of the legitimate work are discarded! The keyserver doesn't have the ability to go through and chop up a 32 WU block and credit you for 31 WU. It just discards the whole thing.

JHutch

PS Now, this isn't gospel, because I've never managed to get this confirmed by DNet, but a lot of careful examination of things and talk with other &quot;old-timers&quot; leads me to be pretty darn sure about it.