Audacity was considered nonportable because it wrote settings to registry. I found this while searching the internet.
It's in french but you can easily change those settings. The program does write settings to the registry but it loads the settings from a reg file on startup and saves it on exit. It automatically erases the registry settings on close.
I also changed some things from the original package and you can get it from http://www.geocities.com/alpha8888_2/PortableAudacity.zip
It is 2824kb uncompressed. It is configured to use English and I removed the language files to decrease the size. I also put in and configured it to use LAME.
Warning Forever is a top-down view space shooter. It is fun and addicting. It is only 950kb uncompressed. It writes settings, scores to the application folder. It comes in an SFX archive, to play you run wf.exe It is freeware and works in Win9x/ME/2k/XP