Observations and Questions about RAM usage

[grannyGeek]

I'm puzzled about RAM usage. It's something I do consider when I'm testing new apps, and I tend to choose those with lower impact if features are equal.

I use Windows Task Manager to monitor memory usage ( I know, there are much better dedicated tools out there, what can I say?)
Anyway, something odd came to my attention recently (I guess I'm not as observant as I thought I was). I've always known that many apps' ram usage climbs during configuration, and drops if you close and re-open. Now I've tried a few recently that don't save settings, but simply minimizing to taskbar drops the ram usage (Watch 4 Folder).

I also suddenly found that some applications currently in use might show a certain memory usage, and minimizing and restoring the app window drops the usage dramatically (Opera, VirtuaWin, Qsel, others). Switching users and return to original log-in has the same effect.

So, is this drop in RAM usage an "artificial" result?
If I report memory usage in a comment to an application, would I report the high number or the low number?

[m^(2)]

I'm puzzled about RAM usage. It's something I do consider when I'm testing new apps, and I tend to choose those with lower impact if features are equal.

I use Windows Task Manager to monitor memory usage ( I know, there are much better dedicated tools out there, what can I say?)
Anyway, something odd came to my attention recently (I guess I'm not as observant as I thought I was). I've always known that many apps' ram usage climbs during configuration, and drops if you close and re-open. Now I've tried a few recently that don't save settings, but simply minimizing to taskbar drops the ram usage (Watch 4 Folder).

I also suddenly found that some applications currently in use might show a certain memory usage, and minimizing and restoring the app window drops the usage dramatically (Opera, VirtuaWin, Qsel, others). Switching users and return to original log-in has the same effect.

So, is this drop in RAM usage an "artificial" result?
If I report memory usage in a comment to an application, would I report the high number or the low number?

Probably they notice that they are inactive and drop some caches etc.
I see this happens with Firefox too (XP).
ADDED: I see it happens everywhere. So it's Windows.
Detailed info.

[lyx]

Depends on what kind of memory-usage you're looking at. Normally, when apps get minimized, they swap a good chunk of their memory to disk - they still use that memory, just not now, so that data gets swapped to disk, so that other currently active applications can use it.

On WinXP, windows is very aggressive in doing this, even if you have lots of free RAM. I suspect this has to do with the time in which WinXP was developed - back then RAM was always a shortage, so XP swapped any mem it right now didn't need. Vista/Seven behave more intelligently in that regard - they most of the time only swap to disk when there's an actual RAM-shortage. It's one of the handful of things i like about vista/seven. On the other hand, Vista/Seven themselves need way more memory, so that the end-result may be the same.

Another thing to take into account has to do with memory allocation. It may happen that when applications no longer need memory, it may not immediatelly be released but instead with a delay. When this happens, its done with the idea that if that app quickly after freeing mem needs new mem, the previously freed mem can just be recycled. This is a frequent theme with memory management: how do i avoid the speed penality of constantly having to deallocate and allocate memory, without allocating way more than i actually need?

[m^(2)]

Depends on what kind of memory-usage you're looking at. Normally, when apps get minimized, they swap a good chunk of their memory to disk - they still use that memory, just not now, so that data gets swapped to disk, so that other currently active applications can use it.

On WinXP, windows is very aggressive in doing this, even if you have lots of free RAM. I suspect this has to do with the time in which WinXP was developed - back then RAM was always a shortage, so XP swapped any mem it right now didn't need. Vista/Seven behave more intelligently in that regard - they most of the time only swap to disk when there's an actual RAM-shortage. It's one of the handful of things i like about vista/seven. On the other hand, Vista/Seven themselves need way more memory, so that the end-result may be the same.

As explained in what I linked, it's only trimming of a working set, not swapping memory out.
Windows (or any other reasonable OS) never swaps memory out unless it's necessary, MS programmers are not that dumb.

[lyx]

As explained in what I linked, it's only trimming of a working set, not swapping memory out.
Windows (or any other reasonable OS) never swaps memory out unless it's necessary, MS programmers are not that dumb.

Sadly, thats not what i see in practice. Neither on my machines nor on other's machines. It may be that it doesn't happen when minimizing (may have misatributed that), but on any computer that has an abundance of RAM, i've always seen XP swapping out memory anyways, even if there's 1GB of RAM unused (well, actually partially used as "system cache").

[m^(2)]

As explained in what I linked, it's only trimming of a working set, not swapping memory out.
Windows (or any other reasonable OS) never swaps memory out unless it's necessary, MS programmers are not that dumb.

Sadly, thats not what i see in practice. Neither on my machines nor on other's machines. It may be that it doesn't happen when minimizing (may have misatributed that), but on any computer that has an abundance of RAM, i've always seen XP swapping out memory anyways, even if there's 1GB of RAM unused (well, actually partially used as "system cache").

If system cache is used more heavily than some application memory then no wonder the application memory gets swapped out...I'd say it's correct.
BTW, how do you know what goes to disk and what doesn't?

[lyx]

BTW, how do you know what goes to disk and what doesn't?

There are mutiple ways to do it, but a simple one that works even if the used taskmanager doesn't give detailed information, is this:
Add up all the commited vmem of all processes. Then look at how much RAM is free (again, actually partially used as system cache).

[m^(2)]

BTW, how do you know what goes to disk and what doesn't?

There are mutiple ways to do it, but a simple one that works even if the used taskmanager doesn't give detailed information, is this:
Add up all the commited vmem of all processes. Then look at how much RAM is free (again, actually partially used as system cache).

It can be approximated by adding Private Memory + Shared WS, but this is a lower bound. I see no way to get the upper bound (which is more interesting)...unless with some tool written specially for this purpose.

[SYSTEM]

As explained in what I linked, it's only trimming of a working set, not swapping memory out.
Windows (or any other reasonable OS) never swaps memory out unless it's necessary, MS programmers are not that dumb.

Sadly, thats not what i see in practice. Neither on my machines nor on other's machines. It may be that it doesn't happen when minimizing (may have misatributed that), but on any computer that has an abundance of RAM, i've always seen XP swapping out memory anyways, even if there's 1GB of RAM unused (well, actually partially used as "system cache").

Do you know if Windows swaps memory out when applications are minimized even if page file is disabled? I have page file disabled and am able to reproduce the behavior grannyGeek described.

[lyx]

AFAIK, windows doesn't even access a swapfile, if swapping is completely disabled (you can delete the swapfile and it wont be recreated). I'm running without a swapfile myself (yes, i know that this means BSOD when out of mem. But all my apps are small, and the peak memusage here is just 60% of my installed ram. I've never run out of mem in the 5 years since i run this setup, so i may as well not take chances in how "intelligent" windows manages swapping).

Are you sure that what you're seeing is a drop in "commited" memory, and not just a drop in "allocated" mem? Asking because IF its just allocated mem dropping, then one possible explanation would be, that when apps get minimized, windows frees currently allocated but unused mem of an app (see earlier comment about memory management).

P.S.: Just tried this myself with a handful of apps. What i see in system explorer:
- Total VMem Usage does not change
- Working-Set drastically goes down (often below 1 MB)

[Hydaral]

From what I have read, it is generally considered Not A Good Idea to turn off the swap/paging file, here is a forum thread with a response from a MS MVP:

http://social.answers.microsoft.com/For ... 0937210722

Can you? Yes. Should you? Definitely not!

1. If you don't have a page file, you can't use all the RAM you have.
That's because Windows preallocates virtual memory in anticipation of
a possible need for it, even though that allocated virtual memory may
never be used. Without a page file, that allocation has to be made in
real memory, thus tying up that memory and preventing it from being
used for any purpose.

2. There is never a benefit in not having a page file. If it isn't
needed, it won't be used. Don't confuse allocated memory with used
memory.

And from Mark Russinovich (who knows more about the internals of Windows than most people at Microsoft):

http://blogs.technet.com/b/markrussinov ... 55406.aspx

Some feel having no paging file results in better performance, but in general, having a paging file means Windows can write pages on the modified list (which represent pages that aren’t being accessed actively but have not been saved to disk) out to the paging file, thus making that memory available for more useful purposes (processes or file cache). So while there may be some workloads that perform better with no paging file, in general having one will mean more usable memory being available to the system (never mind that Windows won’t be able to write kernel crash dumps without a paging file sized large enough to hold them).

[SYSTEM]

Are you sure that what you're seeing is a drop in "commited" memory, and not just a drop in "allocated" mem? Asking because IF its just allocated mem dropping, then one possible explanation would be, that when apps get minimized, windows frees currently allocated but unused mem of an app (see earlier comment about memory management).

Windows Task Manager claims that the RAM usage of an application drops a lot when I minimize the application window, even though page file is disabled. That's the only thing I know for sure.

[lyx]

From what I have read, it is generally considered Not A Good Idea to turn off the swap/paging file, here is a forum thread with a response from a MS MVP:

http://social.answers.microsoft.com/For ... 0937210722

Can you? Yes. Should you? Definitely not!

1. If you don't have a page file, you can't use all the RAM you have.
That's because Windows preallocates virtual memory in anticipation of
a possible need for it, even though that allocated virtual memory may
never be used. Without a page file, that allocation has to be made in
real memory, thus tying up that memory and preventing it from being
used for any purpose.

This is both, correct and also misleading. What is correct is that for memory-management strategy reasons, more VMem needs to be allocated than is actually used. With a pagefile, the unneeded vmem can be swapped to disk, without a pagefile, it must be kept in ram.

What he does not mention is that the VMem is limited to 32bits too. So, if the amount of RAM in the computer, plus HW-allocations, does already fill up all the 32bit addressspace, then there simply is no difference! This is because as he mentions, these minimum allocations are made as VIRTUAL MEMORY - the virtual memory is limited to 32bit however. Now, if you have just as much RAM as you have VMem, then you cannot gain more RAM by swapping out unused VMem pages.

Think of it like this: Forget RAM for now. The computer has 32bits (4GB) of conceptual virtual memory. Everything including some hardware stuff needs to fit into that. Parts of this VMem can then either be mapped to disk, or mapped to RAM, but the total amount of VMem cannot exceed 4GB - you cannot have more then 4GB in TOTAL loaded - this limit cannot be exceeded, also not with a swapfile. But if the amount of RAM you have, plus hw-stuff, does already add up to 4GB, then swapping becomes pointless.

[SYSTEM]

Keeping page file enabled would be a good idea if there was a way to fine-tune it (no, DisablePagingExecutive isn't enough alone). Under Linux, there is a setting called swappiness which can be configured between 0 and 100 (0 = avoid swapping as long as possible; 100 = use swap very aggressively).

I think one size doesn't fit all. I'm aware Microsoft has attempted to adjust default settings so that optimal performance can be achieved, but when Windows XP was released, computers had like 128 or 256 megabytes of memory. Two gigabytes (the amount of RAM I have) is eight times 256 megabytes...

[Andrew Lee]

What he does not mention is that the VMem is limited to 32bits too. So, if the amount of RAM in the computer, plus HW-allocations, does already fill up all the 32bit addressspace, then there simply is no difference! This is because as he mentions, these minimum allocations are made as VIRTUAL MEMORY - the virtual memory is limited to 32bit however. Now, if you have just as much RAM as you have VMem, then you cannot gain more RAM by swapping out unused VMem pages.

Think of it like this: Forget RAM for now. The computer has 32bits (4GB) of conceptual virtual memory. Everything including some hardware stuff needs to fit into that. Parts of this VMem can then either be mapped to disk, or mapped to RAM, but the total amount of VMem cannot exceed 4GB - you cannot have more then 4GB in TOTAL loaded - this limit cannot be exceeded, also not with a swapfile. But if the amount of RAM you have, plus hw-stuff, does already add up to 4GB, then swapping becomes pointless.

Actually, that's not true. On a 32-bit machine, the total amount of physical memory/virtual memory _can_ exceed 4GB. What _cannot_ exceed 4GB is the physical address space of each _process_ on the system.

For example, I am running a modified kernel on a 32-bit Win7 machine at work. It has 8GB physical memory, and I see total 8GB physical memory in task manager. Also, I can run a couple of 1.5GB virtual machines without any slowdown in speed. Without the mod kernel, I can only see 3GB physical RAM, and can't start more than 1 VM.

In general, I think it's best to leave swapping on. However, if you are only running small apps and not really taxing the memory limits of your machine, then turning off swapping probably wouldn't be a problem. But once you start a memory intensive app like database server or photoshop, then you are going to encounter problems and have to enable swapping anyway.

Also, I think the memory manager on Win7 is much better than WinXP. The sluggish recovery after minimizing an app on WinXP does not happen on Win7 anymore.