"Orca Browser is an extremely fast and user friendly web browser, designed to add more functions on to the latest version of Firefox. Not only does Orca Browser contains all the features, such as security, low memory usage, spell-checker, built-in download manager, etc. from a typical Gecko based browser, it also improves the speed, adds built-in features like an AD Blocker, Flash Blocker, Online Profile Storage, auto Form-Filler, and a Outlook-Style RSS Reader. The auto Form-Filler helps to memorize or fill different types of web forms and passwords. It can also protect them with a master security password. Online Profile Storage allows users to create their own account. With their username and password, users can choose to save or access their personal data such as bookmarks, AutoFills, and RSS feeds remotely from any computer."
There is a USB version that claims to store all settings in the application folder, but I have not tested