Memory fragmentation

[lopri]

What exactly is 'Memory Fragmentation'? How does it effect performance and how can it be resolved? Does it mean that SSDs can be fragmented as well?

 

[taltamir]

why wouldn't SSD have fragmentations? they are storage devices so a file can fragment externally, and they have sectors composed of more then one bit so a file can fragment internally...
http://en.wikipedia.org/wiki/Memory_fragmentation

As for memory, memory as in RAM? well, I see no reason why ram can't have fragmentation as well, it is a volatile but extremely fast storage system, but it is still a storage system. So by its very nature its fragmentable.

 

[Nothinman]

What exactly is 'Memory Fragmentation'? How does it effect performance and how can it be resolved? Does it mean that SSDs can be fragmented as well?

Virtual memory fragmentation isn't a huge issue because each process has their own virtual space. So even if one process goes nuts and fragments the hell out of their space it won't affect any other processes. Physical memory fragmentation is another issue because the kernel only has one pool of physical memory with which to play. So the memory manager has to be meticulous in keeping track of which pages can be merged into larger chunks because it's simple to allocate 1 4K page but when you start needing 4, 5, 6 contiguous pages of memory it can be a PITA to find a big enough chunk.

It's one of those things that can't really be resolved with current conventions. You're working with a finite set of resources and dividing it up into millions of smaller chunks to be used by other parts of the system. There's no way to guarantee when those chunks will be handed back to you or in what order. Memory is allocated in varied size chunks so when one chunk is freed and it's between two other still allocated chunks you can't allocate a contiguous chunk of memory larger than the freed one at that address. The longer the system stays up and memory is continually allocated and freed fragmentation will become worse and worse.

 

[taltamir]

RAM is "Random Access Memory"... it is built to have fast random access speed. So why would you care if the chunks are in or out of order? heck why or HOW would you even order chunks of data in the ram?

 

[Nothinman]

The issue is being able to allocate large contiguous chunks of memory, not speed. If you can't find more than 2 contiguous pages of memory and you have a driver that wants to DMA more than 8K at a time you're screwed.

 

[taltamir]

why would it want it? why does it matter if it contiguous or not? its ram. BECAUSE it takes the same time to access any single point regardless of order it does not matter if the info is contiguous. And the way addressing works, you need to address each individual block of data regardless of it being contiguous or not.

 

[Nothinman]

Because a lot of devices require contiguous physical addresses for DMA. And because being able to keep large allocations contiguous puts less strain on the memory manager. Accessing memory may be fast but searching a list of millions of pages for a free one isn't.

 

[lopri]

Thanks for the explanation, guys. I was apparently asking about physical RAM.

So basically the way memory fragments is the same as HDD's in theory? (And in practice as well?) If so;

1. Is it a performance-degrading factor or an 'error' factor? In other words, will it make memory less efficient over time, or will it just cause an error at some point?
2. How can one 'defragment' memory in an effective way?

 

[Nothinman]

So basically the way memory fragments is the same as HDD's in theory? (And in practice as well?) If so;

Basically yea and memory gets fragmented a lot faster because memory is constantly being allocated and freed.

1. Is it a performance-degrading factor or an 'error' factor? In other words, will it make memory less efficient over time, or will it just cause an error at some point?

For userland applications it shouldn't ever get to an error condition unless there really is no memory available, virtually contiguous ranges can point to heavily fragmented physical areas without the application caring at all. For kernel level stuff it could result in allocation errors.

2. How can one 'defragment' memory in an effective way?

You really don't. The kernel's memory manager is the only place that could happen, nothing in userland can see physical addresses so there's nothing they can do about it. All of those tools that say they defragment memory are snakeoil and pretty much just do a really large allocation and then free it hoping that the kernel will make things better in the background.

 

[Lorne]

Lopri, There isnt, Even as the system sits idle there is a constant access to memory reading and writing, The only way to make memory keep defragmented would to build that control into the MMU/MCU and that would create more latency then the thearetical fragmented latency would have caused in the firstplace, A simular example is difference in FBDDR (FullyBufferd DDR memory) and non FB DDR that most of us use.

I answered the question in another thred about drefragging SSD, Yes you can defrag, Doesnt improve anything, SSD's dont suffer head movement latencies and inner/out rotational speeds.

 

[nerp]

Memory defraggers, cleaners and optimizers are scams. Don't bother. They're even worse than registry cleaners.

Aside; using a memory cleaner in Vista would be counterproductive, btw. Clearing out your memory would wipe your superfetch cache and that would just cause it to be reloaded. Pointless.

 

[lopri]

This was the thing that got me to delve into fragmentation/defragmentation.

http://technet.microsoft.com/e...nternals/bb897426.aspx