DF (Sub)Daily + Feature Request for DF Queue

[m0ti]

Hi,

Hmm, seems my post of the Daily yesterday did not go through!

So I guess that I'll combine it with today's Daily then (later on today).

There weren't a lot of responses to the Daily two days ago, so I'm assuming that most people didn't see it.

I'd noted there, that in another thread, I had promised to make a DF Queue proggie.

Due to my current time constraints I won't be able to get working on it for a while, but remind me to get going on it in min-November / early December please. I'd like to have an early version released in January. Also what features would people want (OS? Configurability? etc).

Let me know folks.

 

[Robor]

What do you mean by a DF "queue proggie"? What would it do? Are you talking about a proxy/queue where internal clients would go through the system running this program or something different?

Thanks!

 

[muttley]

Well are you writing something like a Seti driver? Another VERY nice feature would be to have the driver poll the server queue to tell me its operating or a NOT operating condition.
Another thing is the update queued curent so the bandwidth is reduced to one download distributed to all the other computers. Some place I read about that and my mind flushed it cause I didn't matter at the time to me.
A server queue in Windows is fine but a DF driver linux version commandline would work if it tells me that the program is running and stats are being produced.

tired

muttley

 

[m0ti]

I'm thinking of a queue for uploading results.

"Downloads" for DF aren't like SETI, here they're just different proteins and is handled as part of the regular client update. There's an auto update daemon for computers that want to get updated via the net.

I want to do the upload portion. The queue will be for uploading results back to DF.org, or alternatively, one could request the data transfered to a file.

I figure it'll have three portions: client (sends completed work to the server), server (receives client work), and a stand-alone upload proggie.


Everything will support both manual and automatic modes (for automatic, specific time during the day could be specified, other conditions too, perhaps)


Server Functionality
- queue results
- forward results to another server
- upload results
- copy un-uploaded results to a file (definitely compressed and signed)
- stats?

Client Functionality
- upload results to one of several servers (list of them)
- results could be compressed and signed to save on bandwidth, too (selectable option?)
- check if server is up or not
- stats?

Uploader Functionality
- uploads results from a specified file (the one created by a Server)

Is that good for everyone?

What about OS? Footprint (size in memory)? Need to run as a service? Etc?

muttley - sorry, unfamiliar with Seti Driver. I'll go and have a look see.

Hmm.

I was thinking of using it only as a queue for clients to use to upload, not for processing WU's or anything like that.

I've read of quite a few people that aren't running DF on as many computers as they like because there isn't a queue proggie.

But we'll see how things go. Future versions will have more functionality (and less bugs!).

 

[FoBoT]

Originally posted by: m0ti
I want to do the upload portion. The queue will be for uploading results back to DF.org, or alternatively, one could request the data transfered to a file.

can you really do this??!?

this is a MUCH needed tool for certain sectors of the DF community

its really too bad that howard doesn't just turn the current download daemon into a two way proxy

but if you can make your own, wow!!!

i would love to see it written for linux or freebsd, but winders would be ok

dude this is all good, i am not a programmer, but i have the environment to do testing for you and i will give other support that i can to get this going

 

[Insidious]

I don't know what a "DF Queue" is...............

Please don't make me be a mindless newb any longer.......... What is it?

 

[m0ti]

Insidious (Sid),

thought that I had explained it.

Well, here it is again:

Many people have computers that don't have internet access for security/other reasons.

Thus these people wind up doing one of two things: sneakernetting the results to a computer that does have an internet connection, or not running it on computers without internet access, since they don't want to have to sneakernet everything. (in case you don't know sneaker net was the first network. it works by people physically transferring the results around, by sneaker (or perhaps business casual shoe)).


DF Queue will be a program for these folk. They can set it up to automatically to send the results to a computer that has internet access which will then upload the results for them to DF.org, or if they don't have a computer with internet access, send the results to a central location which will then create a single file for them to sneaker-net to a computer with internet access which will then upload the results.

At the beginning it probably won't do stats, a local log would probably be kept on the client to record successes/failures.

 

[Insidious]

Thanks m0ti, That would be a nice thing to have.

I'm assuming the client portion would never actually attempt to upload and is just creating a file instead........... Hmmmmmmmm any chance of an automated e-mail with that file as an attachment?

 

[m0ti]

not the start, sid.

Why would you want that feature anyway (where would it be useful)?

 

[FoBoT]

Originally posted by: m0ti
At the beginning it probably won't do stats, a local log would probably be kept on the client to record successes/failures.

i think that would be fine

 

[Insidious]

Why would you want that feature anyway (where would it be useful)?

I was thinking of my workplace. The firewall isn't going to let any file transfers happen. I know some of the people there would be happy to crunch if they could get the results out without having to do anything (copy files to disks, etc...) The only way I could think of would be to attach the file to an e-mail. The automated part was just icing.

Anyway, I think its great what you are doing m0ti......... Thanks

 

[m0ti]

hmm,

E-mail's definitely a feature that wouldn't be a first thing.

It'd be a little awkward to implement. And it might complicate things with other stuff (mail servers, etc).

Still, I'll keep it in mind.

I think that it is a good idea though. We'll see if it makes into version 2.

EDIT: Sid, do you have access to regular internet though (port 80)?

 

[muttley]

I'm tooooooo tired to read and comprehend falling asleep at keyboard. Will try to read again tomorrow.

muttley

 

[Insidious]

Sid, do you have access to regular internet though (port 80)?

I really don't know how the internet is accessed at work. When we access it, we get a security login screen. All I really know is that it is heavily filtered and scrutinized pretty closely. They have us all pretty bashful about internet use.

I'll see if I can find someone who can tell me.

(right now, all my crunching is from our homes)

 

[m0ti]

well then I think that they'd still be suspicious of lots of e-mails with attachments (or a few e-mails with large attachments).

In any case, doing it via mail is a little bit problematic because it is difficult for the client to ascertain if the server received the work.

Hmm, sounds like you're on a proxy. DF supports proxies. But in any case, suspicion is bad.

you could crunch at work and have all the computers send to a server you set up on one of them. And then put out the work to a file to be taken home and uploaded.

 

[Insidious]

moti,

well, I think you are right on the money.... I'm SOL at work.

That's OK though, I think the client you are making will still help to get a few more assimilations on the home front.

Thanks!