Cheap, Fast RAM makes for new SWAP ideas

Many of the systems I've been installing and using over the past couple of years have stuck fairly closely to my "one MHz/1Mbyte" rule of thumb. My workstation is 2.4GHz and 2Gigs of RAM for example.

Despite application bloat (another discussion;) the vast majority of RAM in most of my systems ends up as disk cache. I've limited many of the systems to 10Megs of Swap with one or two spare partitions I can turn on if I'm going to do something that needs more - bringing the total to about 1/2 total RAM in most cases. Old school said you should have 2x total RAM due to need to re-build page tables when RAM got too fragmented. Old Xenix systems (that actually "swapped" as opposed to todays "paging") could get to the point where the OS had to push all but the kernel out to disk and rebuild the RAM image from scratch.
One of the things I looked at (2.4 kernel) was using a file in /dev/shm for "swap" with a high priority and a small (10 megs or so) size, and another larger (partition or file) at lower priority. It turns out that swapon won't let you do this (use /dev/shm/swapfile) at least once the system has been running for a while. I'm loath to stop it to test but will try on another system.

The kernel seems to need at least a couple of Megs of swap and the /dev/shm/swapfile would satisfy that but still make it FAST! I guess I'll try something else.

Having a small, high priority swap plug a larger, lower priority one may be the answer. The lower priority disk cache might then soak up the need for any larger swap, and a periodic "swapoff /dev/swap ; sleep 5 ; swapon -p 32767 /dev/swap" would keep it from getting too old and crusty - using a script that would look at how busy the system was to decide if it should run or not (don't run if busy)

I'm just getting into the 2.6 kernel on a couple of servers and will play with the "swapiness" setting to see what it does in real life.

On my workstation I'm looking to keep all of the (many - 60+ virtual desktops full) terminal sessions and applications I keep open responsive as I move from desktop to desktop.

On many of my small customers' servers in many cases I need to balance the needs of a diverse range of server applications such as Samba, NFS, Mysql, EXIM, POP, Spamassassin, DHCP, etc.

The diverse file activity of the SMTP chain (EXIM, Spamassassin, POP) can soak up a lot of cache, while the need to keep buffer space available for SMB and NFS for fast response has to be balanced with the need for RAM for Mysql's tables.

As one last point, I'm wondering if the use of the old "sticky bit" (which forced the old swapping systems to keep a program in RAM even if it was no longer active on the theory that things like the shell and cron would be more responsive if they didn't have to be loaded from disk each time they were run) wouldn't give us some more control by allowing us to "nail" a particular application into RAM simply by setting it chmod 1755. I believe this would take a change to the kernel to achieve as I don't believe the sticky bit on files has much if any use on a paged system. I'd love to do this for example for rxvt/xterm.

This comes as my posting to a discussion on Slashdot at Is Swap Necessary?