Browsers - Download complete web page tests (unoffical)

[TP109]

Edited 7-15-16:

Seems like the problems I experienced were related to system resources and outdated add-ons for Light, Firefox, and Opera. After a reboot and updating the add-ons, I was able to download complete web pages with all the browsers except for three. I corrected my post below.

I needed to download and save some rather large web pages of about 5MB each for offline browsing and tried a number of tools. I usually use K-Meleon, but it crashed when downloading large complete web pages. I wanted to save the web pages using the same structure as K-Melon and FF, with a main index file and all the other linked files in a single folder. I tried to duplicate that behavior using wget and spent quite a bit of time trying to configure a wget script, but learned there is no way to duplicate it. Httrack can do it, but it's somewhat tedious to configure and updates take forever since it needs to re-index all the links again, where with a browser, updates can be accomplished quickly by saving the new content over the old. Curl is great, but it's better suited to file downloads and management rather than for complete web pages and sites.

So I tried other browsers and it worked with Greenbrowser. I was able to download very large web pages with no problem using File-->Save More-->Save all content, or CTL+Shift+E. Later I tested a few other browsers and these were the results. I used the latest revisions where possible, since I'm using XP.

Unable to download large web pages:
QTWeb (no option to save complete web pages)
Otter Browser (no option to save complete web pages)
K-Meleon (tends to crash)

Able to download large web pages:
Opera (36.x)
Firefox
Light
Opera Presto
SRWIron
Greenbrowser
Midori Portable
Chromium

I tried the suggestions below for the Scrapbook add-on and for the MHTML/MAFF add-on. Scrapbook saves everything to a directory with the index page thrown in with the rest of the files. It also names the directory with its own code, so it's not obvious what is contained in each directory. There are tools to convert Scrapbook format to MHTML and so forth, but it's not worth the trouble.

The MHTML/MAFF options are better as these formats save to a single file using the original name of the web page. There is no significant size savings using the MHTML format, and often it's larger than the raw web page files themselves. My testing shows a slight size savings of less than 10% over the raw files for the MAFF format and up to a 33% size savings over the MHTML format. In spite of this, either of these are good choices for storing web pages as long as the individual files are not needed right away, since web pages saved in these formats need to be extracted to get at the individual files.

[billon]

What about using Mozilla Archive Format or ScrapBook Firefox addons?

[Specular]

So when you say 'unable to download large web pages' you mean specifically in the default Firefox format? It's odd there would be a difference in capability, as I have no such issues saving large pages in the MHT format for example using Firefox. If there is one it's probably a bug worth filing.

Personally I find MHT easier to maintain being a single file, and I can always use ExtractMHT to convert to the individual files. Looking at some search results on my drive I see hundreds of saved web pages up to 175MB each using Firefox. I previously used Opera Presto for this.

[TP109]

hundreds of saved web pages up to 175MB each using Firefox

Wow. And I thought 5MB was a large file.

I got most of my issues resolved and edited my original post.

[Marc]

Just in case... with Mozilla Archive Format it is possible to download many tabs into one file. In addition the Re-Pagination addon can concatenate multiple pages in vertical space. there's s well PageZipper bookmarklet

Code: Select all

javascript:(function(){if(window['pgzp']){_pgzpToggleBookmarklet();}else{window._page_zipper_is_bookmarklet=true;window._page_zipper=document.createElement('script');window._page_zipper.type='text/javascript';window._page_zipper.src='http://www.printwhatyoulike.com/static/pagezipper/pagezipper_10.js';document.getElementsByTagName('head')[0].appendChild(window._page_zipper);}})();

But I prefer Re-Pagination.

[TP109]

Just in case... with Mozilla Archive Format it is possible to download many tabs into one file. In addition the Re-Pagination addon can concatenate multiple pages in vertical space. there's s well PageZipper bookmarklet

Thanks, already knew about PageZipper, but will check out Re-pagination on your recommendation.