Also, with Haller's method comes the risk that if the app crashes, you'll lose data (and leave it in the registry of the machine you're working on).
This isn't exactly accurate yet I hear it really often so I want to clear something up.
Stuff will only be left in the registry if the wrapper crashes, not the program that has been portable-ized. This practically has a zero chance of occuring (the wrapper crashing I mean). If the main program crashes, the registry wrapper will clean up the registry the same as if the main program had been closed normally.
Also, most times a program crashes you'll lose data unless it has some sort of auto-saving feature.
In other words, basically the only time the quoted statement is true is if the OS itself crashes. Even then, simply running the wrapped program again then closing it will get everything back to how it should be. Not as perfect as something that redirects registry calls entirely (or a natively portable program), but it's not an end-of-the-world scenario and not nearly as risky as the quoted statement makes it out to be.