What to Do When You Run Out of Memory
It’s a rare situation, but unfortunately still a very real one for anyone who hasn’t moved to the x64 architecture yet: you have an x86 program that’s memory-hungry, plenty of RAM installed (4-8 GB), but the process, even though it should be able to get around 2 GB, stalls out at 1.7 GB and then either starts swapping heavily (torturing the GC, in the case of .Net) or crashes with OOM errors.
Of course, you could spend some time and do a proper x64 port, but on Windows there’s another workaround that lets a program claim up to 4GB of memory on an x64 system, and up to 3GB on x86. This applies both to .Net and to programs written in C++ and other languages.
editbin.exe OurExe.exe /LARGEADDRESSAWARE
You can verify the change was applied by comparing the binary against the original with the fc /bin command.
On x86 systems it might turn out that there’s no effect at all until the /3gb switch is set in boot.ini:
[operating systems]multi(0)disk(0)rdisk(0)partition(1)\WINDOWS=”Windows Server 2003, Enterprise” /fastdetect /3GB
There’s no special magic or witchcraft involved here, it’s all been documented for a long time on microsoft.com
Originally published 2010-07-19.