bittorrent : Why the space allocation?

[HeXploiT]

Why does bittorrent allocate hard drive space for files it's downloading even though it hasn't actually downloaded them yet? As soon as the download starts, if it's a 2gb file i'm downloading it automatically takes up the 2gigs on my drive. I've also noticed that all applications be it acrobat or winamp will attempt to open a file in the bittorrent directory even though the file doesn't actually exist yet. Is there some advantage to this? Sofar it seems like a pain more than anything else.

 

[MercenaryForHire]

Because of the way BT downloads a file, any part of that 2GB chunk could be coming in at any given time. It's got to be preallocated in order to be able to write to it.

And I'd rather know right off the bat that I have to clear out a few more megs of space rather than have it fail at 95%

- M4H

 

[notfred]

Jsut like M4H said, it doesn't download all parts of the file in order. If it's downloading the first and last parts of the file at the same time, it needs to be able to put them in the correct location.

 

[BlueWeasel]

That makes sense. I've been wondering the same thing myself

 

[HeXploiT]

Uuuum...why? Why would it not have a set starting point for the download(beginning/end of the file)? Guess i don't really understand how bittorrent works.

 

[notfred]

Originally posted by: Perry404
Uuuum...why? Why would it not have a set starting point for the download(beginning/end of the file)? Guess i don't really understand how bittorrent works.

Bittorrent breaks up files into a bunch of small sections. Usually 1-4mb. Then, when downlaoding a file, it picks a few sections at random and starts downloading them. This allows for all the peers to share better with each other. Here's an example:

Pretend you have a file with 10 sections.
You connect to some peers with the following sections:
Peer 1 has sections 1, 4, and 8.
Peer 2 has sections 3, 6, and 9.
Peer 3 has section 10.

You can now download something from each of the above sections. Say you get sections 4, 6, and 10.

You can now upload section 4 to Peer 2, section 6 to peer 3, and section 10 to peer 3.

If all downloads occured in order, then:
Peer one has section 1, 2, 3
Peer 2 has section 1, 2, 3
Peer 3 has section 1

You can now only download section 1, and when you downlaod it, you can't do any useful sharing of it, because everyone else already has that part. Your upload bandwidth is wasted. If you had gotten section 4, then you could start uplaoding that to peers, and your bandwidth is put to better use.

 

[Bateluer]

Also makes hdd space management easier.

 

[phlashphire]

Somewhat related to the topic, does this method increase/decrease/not effect hard disk fragmentation?

 

[oLLie]

Originally posted by: phlashphire
Somewhat related to the topic, does this method increase/decrease/not effect hard disk fragmentation?

If you use Azureus client, this method is an option (to reduce HD fragmentation), therefore it seems that it is not a necessity of the bittorrent protocol as is said here, but an option.

*IIRC*

 

[BlueWeasel]

Originally posted by: notfred
Bittorrent breaks up files into a bunch of small sections. Usually 1-4mb. Then, when downlaoding a file, it picks a few sections at random and starts downloading them. This allows for all the peers to share better with each other. Here's an example:

Pretend you have a file with 10 sections.
You connect to some peers with the following sections:
Peer 1 has sections 1, 4, and 8.
Peer 2 has sections 3, 6, and 9.
Peer 3 has section 10.

You can now download something from each of the above sections. Say you get sections 4, 6, and 10.

You can now upload section 4 to Peer 2, section 6 to peer 3, and section 10 to peer 3.

If all downloads occured in order, then:
Peer one has section 1, 2, 3
Peer 2 has section 1, 2, 3
Peer 3 has section 1

You can now only download section 1, and when you downlaod it, you can't do any useful sharing of it, because everyone else already has that part. Your upload bandwidth is wasted. If you had gotten section 4, then you could start uplaoding that to peers, and your bandwidth is put to better use.

Excellent explanation, Mr. Durden

 

[HeXploiT]

That was a good explanation thanks. It seems efficiancy is now king.

 

[cerebusPu]

my bittorrent client doesnt do it. Im running torrent storm.

 

[notfred]

Originally posted by: cerebusPu
my bittorrent client doesnt do it. Im running torrent storm.

Then it keeps track of all the peices of the file it downloads in some other fashion. It still downloads in random order.

 

[Lonyo]

BitTornado (for me) failed to do this.
Consequently, I have some very messed up files, my mp3's are not whole songs, but randomly skip from one bit of one song to another bit of a different song, because they seem to have been mis-reassembled .
Azureus is working fine though.

 

[HeXploiT]

Interesting. I'm using Azureus and many of my mp3s have ended up really choppy & messed up to a degree much higher than from any other filesharing app i've used. I don't know that it can be blamed on the app itself though. mp3s i can see working if put back together incorrectly but if this same corruption occurred with applications/exectables they simply wouldn't work at all.

 

[Nothinman]

Consequently, I have some very messed up files, my mp3's are not whole songs, but randomly skip from one bit of one song to another bit of a different song, because they seem to have been mis-reassembled

I suspect something else caused the problem, I've been using bittornado without preallocation since it was released to Debian sid and I've never had a corrupt file.

Then it keeps track of all the peices of the file it downloads in some other fashion. It still downloads in random order.

The default for bittornado is 'normal' which doesn't preallocate anything but it can be changed to be one of normal, background, pre-allocate or sparse if you want.