• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Isn't it time, that MS patches out the 32GB FAT32 format limitation (USB flash drive)

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.
exFAT is becoming common by force (SD standard), and was released 6 years after the FAT32 formatting size limit was put in place. I wouldn't give MS the credit to think they had it in the pipe that long, much less had plans to replace FAT32 with another FS other than NTFS, that far back.
The project that became exFAT may not have existed at the time, but MS had been working on the matter for many years. It was known for some time that we'd need a lightweight file system to replace FAT32, so it was a matter of figuring out what would work best and what could achieve widespread adoption.

Very interesting stuff, any predictions as to whether exFAT will indeed prevail or be replaced by something open?
exFAT will prevail. It has the right mix of features (data structure sizes and free space bitmaps), simplicity (allowing safe "dirty" unmounting), and broad support that it's the logical successor to FAT32 in both lineage and usage.

Every supported Windows machine on the planet can already use it, as can Mac OS X. The Linux guys will have to hold their noses, but they'll come around.
On my large (and valuable data-wise) memory sticks I use NTFS and have since windows 2000. I also back up their data to my server. I have had zero problems with file loss or errors on them, I cannot say that about fat32.
As much as I like NTFS, it wasn't meant for removable USB storage, which is why MS defaults to exFAT for those devices. NTFS is for fixed disks, either hard disks or SSDs (which have higher quality controllers than what you'd find on a USB flash drive).
 
Last edited:
The Linux guys will have to hold their noses, but they'll come around.
Nah, well just keep using NTFS 😀, unless they come out with a FS that actually is designed for removable media. They are intending to add transactional features in the future, not by original design, which IMO, is just one big *facepalm* (make it proprietary, try to make everyone license it, practically brag about it being patent-encumbered, and then don't implement the key feature that it should have for its purported use, transaction-safety!).

NTFS may not have been designed for removable storage, but to date, there has been no released information indicating that exFAT was, either. It can be optionally implemented halfway-decently, but there's no requirement, and I won't trust any implementation to be better than FAT32, without requiring transaction-safety in the spec (TeXFAT support is optional, though that kind of functionality needs to be fundamental and required).
 
Last edited:
Nah, well just keep using NTFS 😀, unless they come out with a FS that actually is designed for removable media. They are intending to add transactional features in the future, not by original design, which IMO, is just one big *facepalm* (make it proprietary, try to make everyone license it, practically brag about it being patent-encumbered, and then don't implement the key feature that it should have for its purported use, transaction-safety!).

NTFS may not have been designed for removable storage, but to date, there has been no released information indicating that exFAT was, either. It can be optionally implemented halfway-decently, but there's no requirement, and I won't trust any implementation to be better than FAT32, without requiring transaction-safety in the spec (TeXFAT support is optional, though that kind of functionality needs to be fundamental and required).
I have to tell you, I don't trust ntfs-3g one bit. It is a _very_ slow, user space implementation that does not handle non-POSIX NTFS features very well. It is only for the simplest NTFS usage possible. I wouldn't trust my backup being in an NTFS volume via a linux mount. I use windows for NTFS and ext4 for linux. The most reliable for linux would be probably XFS, and maybe btrfs in the future. There are interesting times ahead!
 
I have to tell you, I don't trust ntfs-3g one bit. It is a _very_ slow, user space implementation that does not handle non-POSIX NTFS features very well. It is only for the simplest NTFS usage possible. I wouldn't trust my backup being in an NTFS volume via a linux mount. I use windows for NTFS and ext4 for linux. The most reliable for linux would be probably XFS, and maybe btrfs in the future. There are interesting times ahead!
Of course it's for the simplest use of NTFS. What non-POSIX NTFS features do you generally use on your external HDD? exFAT added a dirty bit, but otherwise doesn't have much really useful for a bad unmount. SD cards not running an OS will basically have to use exFAT, just like they had to use FAT32, and before than, FAT. But just like those, that only affects SD cards. I can count on one finger the number of times I've needed FAT32 for a USB drive, yet many times where I've both needed larger files, and when I was glad that it didn't silently allow bad writes to go unnoticed.

I haven't found it be slow at all, though. It's always been disk-limited, IME.
 
Of course it's for the simplest use of NTFS. What non-POSIX NTFS features do you generally use on your external HDD? exFAT added a dirty bit, but otherwise doesn't have much really useful for a bad unmount.
exFAT doesn't have journaling, file compression, or logging. For a fixed disk (especially a system disk) these are incredibly useful features. But for a removable disk you actually don't want these things, which is why we have exFAT.

On a side note, here's a good MS slide deck on exFAT: http://www.flashmemorysummit.com/English/Collaterals/Proceedings/2011/20110811_T3A_Prewitt.pdf
 
Last edited:
Of course it's for the simplest use of NTFS. What non-POSIX NTFS features do you generally use on your external HDD? exFAT added a dirty bit, but otherwise doesn't have much really useful for a bad unmount. SD cards not running an OS will basically have to use exFAT, just like they had to use FAT32, and before than, FAT. But just like those, that only affects SD cards. I can count on one finger the number of times I've needed FAT32 for a USB drive, yet many times where I've both needed larger files, and when I was glad that it didn't silently allow bad writes to go unnoticed.

I haven't found it be slow at all, though. It's always been disk-limited, IME.

Typically I would like to be able to "cp -a" the system partition of a laptop as backup, but obviously this doesn't work, and using "dd" for the whole partition is wasteful if you have lots of free space. Permissions, junction points and links are f(&*ed up and many linux applications just don't work on NTFS. Then there are these shadow copy inconsistencies people are talking about and sound scary, not sure about specifics.

In my experience, ntfs-3g is CPU limited in the worst possible way, that is single threaded. Overheads for filesystems should be minimal, not bring my sandy bridge on its knees because it's writing zeroes on disk.

exFAT doesn't have journaling, file compression, or logging. For a fixed disk (especially a system disk) these are incredibly useful features. But for a removable disk you actually don't want these things, which is why we have exFAT.

Journalling is a must, without it you risk losing a whole folder just by renaming/moving it when there is a failure.
 
Journalling is a must, without it you risk losing a whole folder just by renaming/moving it when there is a failure.
But journaling is bad on removable flash media. It makes writes take longer (so you can't immediately unplug the things), causes a bunch of small random writes, and consequently is more likely to trigger garbage collection. And in the case you've outlined, a scandisk/fsck would recover the data; the purpose of the journal would be to skip that step and just replay the journal to immediately recover it.

This is all about tradeoffs. Journaling and other advanced features are generally considered a poor tradeoff for removable flash drives.
 
But journaling is bad on removable flash media. It makes writes take longer (so you can't immediately unplug the things), causes a bunch of small random writes, and consequently is more likely to trigger garbage collection. And in the case you've outlined, a scandisk/fsck would recover the data; the purpose of the journal would be to skip that step and just replay the journal to immediately recover it.

This is all about tradeoffs. Journaling and other advanced features are generally considered a poor tradeoff for removable flash drives.

You make a good point, but what if it is an 128GiB SD card or something, then journalling sounds much more necessary.
 
Typically I would like to be able to "cp -a" the system partition of a laptop as backup, but obviously this doesn't work, and using "dd" for the whole partition is wasteful if you have lots of free space. Permissions, junction points and links are f(&*ed up and many linux applications just don't work on NTFS. Then there are these shadow copy inconsistencies people are talking about and sound scary, not sure about specifics.
If you need that stuff to work in Windows, you need NTFS, and in time, ReFS. In Linux, you need a *n*x FS. They are not interchangeable for anything but directories of data files w/o special permissions, and that goes both ways (or would, if Windows had good FS drivers for non-native FSes 😉). IE, use NTFS just like you would FAT32, but take advantage of it being journaled. For anything like using links, or applying permissions at any granularity lower than the whole mount point, an *n*x-native FS is going to be a must.

In my experience, ntfs-3g is CPU limited in the worst possible way, that is single threaded. Overheads for filesystems should be minimal, not bring my sandy bridge on its knees because it's writing zeroes on disk.
Using a fast SSD? It's CPU-heavy, but I've never seen it use more than maybe 30% on old Core Duos and Athlon64s, copying files at 80+MB/s.

I also want an equivalent to journaling on anything I can. For dealing with low-memory embedded systems, I can understand wanting to make something different than NTFS, but a transactional-logging system with block-level copy-on-write (IE, write to a new cluster, then free the old cluster, and update a copy of the metadata, not making it the current metadata until the write is done; IoW, no in-place data changes) should be quite doable, in any number of ways, and offer the benefits of a journal, without a distinct journal, and without needing several MBs of RAM to manage. Performance would only cost a few % of clusters reserved, which would be largely unnoticed. By not overwritng an old chunk of data until a new chunk has been successfully written, most anything could be rolled back, as the new old data would all still be there.

A journaled FS in general is really not ideal for internal HDDs, either, but back in the 80s and 90s, software techniques to do otherwise were still wild crazy hippie ideas, like JIT 🙂.

exFAT doesn't have journaling, file compression, or logging. For a fixed disk (especially a system disk) these are incredibly useful features. But for a removable disk you actually don't want these things, which is why we have exFAT.
I don't care about compression or audit-quality logging, but why wouldn't you want a nominally transaction-safe file-system? At worst, it should be able to know what failed (not merely that something might have), and at best roll it back. Being removable makes me want that more, not less, and is the 2nd main reason I use NTFS, now (the 1st being cross-platform compatibility, but FAT32 has that, too).

It doesn't need to actually be a journal. A journal causes more writes than may be necessary (it's not the MBs, it's the separate addressing, some of which may need to be synchronized, which does slow things down). What's really needed is to be reasonably sure that old data will stick around until after new data has finished writing, however that can be done. exFAT kinda sorta might be able to do that, sometimes, with small volumes and files, due to the legacy FAT table, but that apparently breaks with big files, big cluster sizes, etc. (and the whole point is to be able to use it on media that' hundreds of GBs in size).
 
Last edited:
If you need that stuff to work in Windows, you need NTFS, and in time, ReFS. In Linux, you need a *n*x FS. They are not interchangeable for anything but directories of data files w/o special permissions, and that goes both ways (or would, if Windows had good FS drivers for non-native FSes 😉). IE, use NTFS just like you would FAT32, but take advantage of it being journaled. For anything like using links, or applying permissions at any granularity lower than the whole mount point, an *n*x-native FS is going to be a must.

Using a fast SSD? It's CPU-heavy, but I've never seen it use more than maybe 30% on old Core Duos and Athlon64s, copying files at 80+MB/s.

I also want an equivalent to journaling on anything I can. For dealing with low-memory embedded systems, I can understand wanting to make something different than NTFS, but a transactional-logging system with block-level copy-on-write (IE, write to a new cluster, then free the old cluster, and update a copy of the metadata, not making it the current metadata until the write is done; IoW, no in-place data changes) should be quite doable, in any number of ways, and offer the benefits of a journal, without a distinct journal, and without needing several MBs of RAM to manage. Performance would only cost a few % of clusters reserved, which would be largely unnoticed. By not overwritng an old chunk of data until a new chunk has been successfully written, most anything could be rolled back, as the new old data would all still be there.

A journaled FS in general is really not ideal for internal HDDs, either, but back in the 80s and 90s, software techniques to do otherwise were still wild crazy hippie ideas, like JIT 🙂.
F2FS: https://lkml.org/lkml/2012/10/5/205

I haven't tested it but it's probably what you're looking for.
 
F2FS: https://lkml.org/lkml/2012/10/5/205

I haven't tested it but it's probably what you're looking for.
But where's the Windows driver? :\

That's what really gets me about the SD Association (without which exFAT would be another footnote in MS' history), a constant striving for short-term crap, instead of using their ability to force support to actually do it better (SD standard size limits, rather than making a forward/backward-compatible, like ATA, are another example, which lead to older devices not being able to use newer SD cards--CF, FI, has no such problem, only too many pins).
 
But where's the Windows driver? :\
There is none! 😀
In theory, it can be ported to whatever, because it's GPL (as most cool stuff), but I believe Microsoft does not give developers access to the Windows kernel file-system core, so you cannot add real file-systems, only filter drivers. I could be mistaken though.

Edit: it's GPL and that means Windows needs to be GPL too, forgot about that!
 
Last edited:
microsoft is not the same microsoft anymore...

Ugh... im still gaging at the metro style dashboard on server 2012...
Really?!?!? dashboard on server?!?!?!?!

I think there committed to really pissing us off.

Yea, this is so profanely stupid there has to be a conspiracy involved.
 
Given that USB flash drives are ubiquitous, and they are starting to come out in higher capacities than 32GB (64, 128, and 256GB, even Kingston has a 512GB), and that NTFS sucks for flash drives, and FAT32 is interchangable, WHY OH WHY does MS still insist on arbitrarily limiting FAT32 formatting to 32GB or smaller partitions?
You're supposed to use exFAT. exFAT was designed specifically for flash drives and boasts a number of improvements over FAT32
 
Back
Top