S5 data and New App at Einstein

[RobertE]

Einstein is starting the roll out of the S5 data with a new app. All the juicy details can be foundHere

I'll update this as our generation and testing of our first S5 workunits proceeds
The names of these workunits are of the form h1_XXXX.X_S5R1x_* and l1_XXXX.X_S5R1x_*.

The application running these is called einstein_S5R1. This application incorporates many of the speedups and other code changes suggested by Akos. Note: please do NOT replace these stock executables with custom versions of 'albert'. They are not compatible.

There are two types of workunits: short and long. The short workunits have XXXX.X less than or equal to 0400.0.

There are also two types of data files: short and long. The short data files (l1_XXXX.X) are from the LIGO Livingston Observatory, and are about 4.5MB in size. The long data files (h1_XXXX.X) are from LIGO Hanford and are about 16MB in size. Note: once your computer downloads one of these data files, it should be able to do many workunits for that same file.

We are switching to a new uniform system for awarding credits. All users on all platforms will claim the same credit for each workunit, with an amount of credit proportional to the length of the workunit: 'equal credit for equal work'.

To try and increase the total amount of computing power available to the project, we have changed the target number of results and minimum quorum from 3 to 2. Only if the first two results from different hosts/users do not agree will additional work be generated.

Please be patient with us if we have to sort out last minute problems or other issues. We have been testing this privately for some time, so we are fairly confident that there are no significant issues that remain. Nevertheless, several of us on 'the cutting edge' are rather short on sleep!

Bruce

Added 14/06/2006: please feel free to post questions here, but keep in mind that it may take some time before we have time to answer everything.

Added 15/06/2006: we have now generated about 1500 workunits. So far all is looking well. There are a fairly large number of download errors in downloading data files. But these appear to be mostly due to some problems we had yesterday in replicating data to our mirror sites. There are also interesting errors on *some* Mac OS X PPC systems. I'm sure we'll sort this out quickly.

 

[networkman]

Thanks for posting that, Robert.

I've not looked at the E@H project in a while so maybe I'll give it another chance once I've acheived my goal of 50k in the Leiden Classical project. I'm at 48.3k now so it shouldn't be too long.

 

[BlackMountainCow]

Thx Robert!

:beer:

 

[petrusbroder]

Thanks Robert!

 

[RobertE]

Update 6/16

Added 16/06/2006: We have completed the GENERATION of the remaining S4 workunits. They are in the database and should complete crunching during the next 2 to 3 weeks. We have also begun automatic generation of the S5 workunits, and have updated the server status page to reflect this. I also added an item on this status page to track the progress of the remaining S4 workunits.

You can expect longer run times with the S5 dataset and the new app. The individual WUs are now considerably larger to compensate for the optimizations that Aksof provided.

For example a short WU on a Xeon 3.06 took 1.4 hrs.
A long WU on a P4 3.00 w/HT took 16.4 hrs.

A short WU took 2 hours on my Hot rod Dell P4 2.8 w/ht.

YMMV on your run times.

All credit claims are done server side as well.

 

[BlackMountainCow]

Do you know if there will be an extra optimized app, I mean, even more optimized for different CPUs, as it used to be?

 

[networkman]

Originally posted by: BlackMountainCow
Do you know if there will be an extra optimized app, I mean, even more optimized for different CPUs, as it used to be?

I rather doubt it. Previously, the E@H folks release an app and Akosf released optimized improved versions based off of the source code. Now that the E@H folks have hired Akosf onto their project team, the new app being released includes those optimization designed and tested by Akosf. It'll end up being a bit more fair for all participants as you won't have to know to go to the E@H forums to find an optimized app - it'll already be included when you download initially.

edit: More to the point of your question, I expect any further insights on optimizations that Akosf might discover for particular instruction sets would probably be included on a newer version of the client, rather than as a seperate app. That's just my impression based on what I've read in the E@H forums but would seem most likely.

 

[amdxborg]

OK to be honest I'm really dissapointed about the huge size of the wus, I mean 16megs for a half day of crunching??? QMC gives 1meg wu that lasts a day!! The only thing this means for me is to move most if not all of my E@H machines to another project. This was one of my fav projects, but this really sux!! :|

 

[networkman]

Yeah, I have to agree that 16 meg is rather excessive. Although I understand why they're doing it, I also think they're going to lose a lot of folks who are currently using dial-up or have caps on their available bandwidth or time.

It will be unfortunate to see you backing off on the project, Johan, but I do understand the reason.

 

[RobertE]

Originally posted by: BlackMountainCow
Do you know if there will be an extra optimized app, I mean, even more optimized for different CPUs, as it used to be?

I would expect some CPU specific apps eventually. From what I have gathered, the new app only detects if your CPU has SSE or not. Nothing yet for SSE2/SSE3/3dnow/etc. I'd put my money on Askof coming out with those. I don't think however, that we will see the hugh speedups as before, but you never know.

 

[RobertE]

Originally posted by: amdxborg
OK to be honest I'm really dissapointed about the huge size of the wus, I mean 16megs for a half day of crunching??? QMC gives 1meg wu that lasts a day!! The only thing this means for me is to move most if not all of my E@H machines to another project. This was one of my fav projects, but this really sux!! :|

It is a huge download for dialup/metered service users. One thing I don't know is how many WUs you'll get out of that block of data.

 

[networkman]

Well, according to my current RAC I'll be done with my Leiden goal in about 2.5 days, so I can start up the project then and let you know.

Hmm.. I have 2 machines "off-site" doing the existing E@H project so I guess I'll need to get ahold of those and do the appropriate upgrade. My workstation at work is a dual-core P4 2.8 w/HT and 4 gig of ram so it should be interesting to see how fast it does the WUs.

 

[RobertE]

More info on the size of the downloads.

Any guesstimates on how many WUs will come from each dataset download?

There are 5802 data files, total. With 13 million workunits, this implies about 2240 workunits per data file.

But this number is a bit misleading, because there are far fewer workunits per file at low frequencies and more workunits per file at high frequencies. Note that the frequency is the XXXX.X part of the file name.

Bruce

It looks like each data set download should yield a good amount of WUs & crunch times. A larger cache may be usefull for thoses running E@H and are bandwith sensitive.

 

[petrusbroder]

THanks for the regular info, RobertE! It is apreciated!

 

[networkman]

FYI - I hit my goal of 50k in the Leiden Classical project and am switching back to the E@H project.

Do understand though that it's just three machines for the summer and early fall months, perhaps more power will be brought to bear when the temperatures turn cooler.