Lossless audio formats, why didnt WAV take off?

[Maximilian]

So in conclusion, its all about file size. Neato :thumbsup:

 

[Cogman]

So in conclusion, its all about file size. Neato :thumbsup:

Pretty much. So long as the bitrate is high enough so that you can't tell the difference, you're good (and going to be a smaller filesize than a lossless compression scheme).

 

[Midwayman]

Of course you can train yourself to hear the difference between compressed and uncompressed audio. Compression leaves artifacts that you can hear if you know what you're listening for. Well done compression does it in such a way that its not obvious, but its still there.

For comparison, Some people can't spot the difference between a JPG image and a TGA. If you know what artifacts are likely to occur and where to look, its reasonably easy to spot these.

For audio compression I ran ABX testing on OGG samples to determine what I should compress my CD library with. Ended up about 350kb or so was where I started having a really hard time telling the difference. So, I went slightly higher than that and compessed it all.

In both cases, its not about being physically unable to hear or see the compression. Its about knowing what to look for.

 

[Cogman]

Of course you can train yourself to hear the difference between compressed and uncompressed audio. Compression leaves artifacts that you can hear if you know what you're listening for. Well done compression does it in such a way that its not obvious, but its still there.

For comparison, Some people can't spot the difference between a JPG image and a TGA. If you know what artifacts are likely to occur and where to look, its reasonably easy to spot these.

For audio compression I ran ABX testing on OGG samples to determine what I should compress my CD library with. Ended up about 350kb or so was where I started having a really hard time telling the difference. So, I went slightly higher than that and compessed it all.

In both cases, its not about being physically unable to hear or see the compression. Its about knowing what to look for.

If you throw enough bits at lossy compression, it eventually becomes lossless. Lossy compression gives you the option to determine where it is that you can hear the difference.

It is just silly to say that you can ALWAYS tell the difference between lossy and lossless even if you know what you are looking for.

BTW, I am interested, when did you run this ABX test? 350kb/s for OGG (currently) is really quite high. Also, What was your confidence level?

 

[JSt0rm]

If you throw enough bits at lossy compression, it eventually becomes lossless. Lossy compression gives you the option to determine where it is that you can hear the difference.

It is just silly to say that you can ALWAYS tell the difference between lossy and lossless even if you know what you are looking for.

BTW, I am interested, when did you run this ABX test? 350kb/s for OGG (currently) is really quite high. Also, What was your confidence level?

Depends on where you are at. If you are on the highway with roadnoise? No.

In the room I posted on page 2? yes.

 

[Cogman]

Depends on where you are at. If you are on the highway with roadnoise? No.

In the room I posted on page 2? yes.

Even in the room you posted. With the absolute best sound system money can buy, I can raise the bitrate to a high enough level that the human ear can't tell the difference.

Heck, I can raise the bitrate high enough that no measuring device can detect the difference, because the output will be exactly identical.

Like I said before, if you raise the bitrate high enough on any lossy codec, you end up with a lossless stream.

 

[JSt0rm]

Even in the room you posted. With the absolute best sound system money can buy, I can raise the bitrate to a high enough level that the human ear can't tell the difference.

Heck, I can raise the bitrate high enough that no measuring device can detect the difference, because the output will be exactly identical.

Like I said before, if you raise the bitrate high enough on any lossy codec, you end up with a lossless stream.

ok and in that case it would null. And I always said if the audio nulls it is the same.

 

[Ross Ridge]

If you throw enough bits at lossy compression, it eventually becomes lossless.

That's not true of lossy compression schemes in general. For example, youn't use MP3 or JPEG encoding as a lossless compression format regardless of the bit rate or compression parameters you use.

 

[sdifox]

If you throw enough bits at lossy compression, it eventually becomes lossless. Lossy compression gives you the option to determine where it is that you can hear the difference.

It is just silly to say that you can ALWAYS tell the difference between lossy and lossless even if you know what you are looking for.

BTW, I am interested, when did you run this ABX test? 350kb/s for OGG (currently) is really quite high. Also, What was your confidence level?

err, no. Lossless compression is reversible. Lossy compression never is.

 

[Cogman]

That's not true of lossy compression schemes in general. For example, youn't use MP3 or JPEG encoding as a lossless compression format regardless of the bit rate or compression parameters you use.

Just because it isn't generally done doesn't mean that it isn't possible. Pretty much all lossy compression formats are based off of the DCT, http://en.wikipedia.org/wiki/Discrete_cosine_transform . A transformation that was originally used to generate a wave form for a bunch of discrete signals. The lossy compression comes in because the compressors are throwing away data from the DCT. Raising the bitrate high enough would essentially be saying "Ok, keep everything in the DCT." which is a lossless compression scheme.

Of course, you can end up with a file that is bigger than the source (depending on what other magic they are doing in the background). Which is why lossy compression schemes aren't generally used for lossless compression.

 

[Cogman]

err, no. Lossless compression is reversible. Lossy compression never is.

See my explanation above. Most all lossy compression is based off of algorithms which can easily be turned into lossless compression.

 

[sdifox]

See my explanation above. Most all lossy compression is based off of algorithms which can easily be turned into lossless compression.

Lossy compression operates on the premise of dropping parts of the waveform that you might not notice. But the waveform is changed for sure. Doesn't matter how little compression you use, you are still dropping parts of the waveform.

Take a wave file, plot the waveform, compress, then compare the waveform to the original wave file and you'll see differences. Not so with lossless compression.

 

[Cogman]

Lossy compression operates on the premise of dropping parts of the waveform that you might not notice. But the waveform is changed for sure. Doesn't matter how little compression you use, you are still dropping parts of the waveform.

Take a wave file, plot the waveform, compress, then compare the waveform to the original wave file and you'll see differences. Not so with lossless compression.

If you read my post, you would have seen that I pretty much covered that.

As I said before, when you throw enough bits at a lossy algorithm, and it becomes a lossless algorithm. The data might not be compressed (it may result in a larger file), but it will be lossless.

 

[Midwayman]

If you throw enough bits at lossy compression, it eventually becomes lossless. Lossy compression gives you the option to determine where it is that you can hear the difference.

It is just silly to say that you can ALWAYS tell the difference between lossy and lossless even if you know what you are looking for.

BTW, I am interested, when did you run this ABX test? 350kb/s for OGG (currently) is really quite high. Also, What was your confidence level?

Well, of course if you don't throw away any data and pack it in a lossy container, you will never be able to tell the difference. Of course its not really lossy at that point. Everyone has a different threshold for what they can hear, and more important- what they will accept. I would guess that the vast majority of people could hear the difference between common commercial mp3 bitrates and a CD given a little coaching an good equipment in a quiet environment.

The test was probably 6-8 years ago I think. Its pretty simple test. Piece of software takes a wav and a OGG file compressed to whatever standard and plays one back and asks you to select which you thought it was. After 20 tries, it displays how many you got right. I stopped when I was getting 11-12 right. Probably not the most scientific test, but it was designed to give me the bitrate I needed and nothing more.

http://abx-comparator.berlios.de/ is similar, but not the same piece of software I used.

 

[Ross Ridge]

Just because it isn't generally done doesn't mean that it isn't possible.

It's not possible in the context of the discussion we're having now. You can't encode a arbitrary song as an MP3 at bitrate that would make it lossless. The format simply doesn't support that.

The lossy compression comes in because the compressors are throwing away data from the DCT. Raising the bitrate high enough would essentially be saying "Ok, keep everything in the DCT." which is a lossless compression scheme.

You're assuming the compression format allows encoding an arbitrary DCT without throwing away data. They don't.

 

[Cogman]

It's not possible in the context of the discussion we're having now. You can't encode a arbitrary song as an MP3 at bitrate that would make it lossless. The format simply doesn't support that.



You're assuming the compression format allows encoding an arbitrary DCT without throwing away data. They don't.

True, there is a specification maximum bitrate, however, that is more a "Nobody will ever need this" sort of decision then a "It can't be done" decision.

MP3 has this limitation, AAC does not.

There is some possible (not definite) data loss due to rounding as well.

 

[JSt0rm]

can somebody tell me who the evil audiophile companies are? Im still perplexed by this

 

[Ross Ridge]

True, there is a specification maximum bitrate, however, that is more a "Nobody will ever need this" sort of decision then a "It can't be done" decision.

MP3 has this limitation, AAC does not.

There is some possible (not definite) data loss due to rounding as well.

No, you really don't understand the problem here. When transforming a 16-bit PCM audio stream using some DCT algorithim you get a sequence of floating point values. If you then encoded these floating point values with enough precision and range, then yes you could feed them into the inverse DCT algorithm and get the original 16-bit PCM values back. However, the MP3 and AAC encoding standards do not let you do this.

These standards require that the (M)DCT samples be quantized and then Huffman encoded before they're output. The quantization step is crucially important, this takes the DCT samples and turns them into something that can be compressed well by the Huffman encoder. This quantization step however is what makes the MP3, AAC and another similar compression schemes lossy, and what makes them unavoidably lossy. MP3 requires that quantization produce an integer in the range of –8206 and 8206 (AAC between -8191 and 8192). That's little over 14 bits, and not any where near enough bits to allow the inverse DCT to reconstruct the original 16-bit PCM values in the general or typical case.

 

[OneOfTheseDays]

No, you really don't understand the problem here. When transforming a 16-bit PCM audio stream using some DCT algorithim you get a sequence of floating point values. If you then encoded these floating point values with enough precision and range, then yes you could feed them into the inverse DCT algorithm and get the original 16-bit PCM values back. However, the MP3 and AAC encoding standards do not let you do this.

These standards require that the (M)DCT samples be quantized and then Huffman encoded before they're output. The quantization step is crucially important, this takes the DCT samples and turns them into something that can be compressed well by the Huffman encoder. This quantization step however is what makes the MP3, AAC and another similar compression schemes lossy, and what makes them unavoidably lossy. MP3 requires that quantization produce an integer in the range of –8206 and 8206 (AAC between -8191 and 8192). That's little over 14 bits, and not any where near enough bits to allow the inverse DCT to reconstruct the original 16-bit PCM values in the general or typical case.

Damn. Owned.

Yes, in general anytime quantization is involved you lose the ability to go back to what you originally started with.

 

[Hyperlite]

No, you really don't understand the problem here. When transforming a 16-bit PCM audio stream using some DCT algorithim you get a sequence of floating point values. If you then encoded these floating point values with enough precision and range, then yes you could feed them into the inverse DCT algorithm and get the original 16-bit PCM values back. However, the MP3 and AAC encoding standards do not let you do this.

These standards require that the (M)DCT samples be quantized and then Huffman encoded before they're output. The quantization step is crucially important, this takes the DCT samples and turns them into something that can be compressed well by the Huffman encoder. This quantization step however is what makes the MP3, AAC and another similar compression schemes lossy, and what makes them unavoidably lossy. MP3 requires that quantization produce an integer in the range of –8206 and 8206 (AAC between -8191 and 8192). That's little over 14 bits, and not any where near enough bits to allow the inverse DCT to reconstruct the original 16-bit PCM values in the general or typical case.

Cool. Thanks for the explanation.