Text Editor Performance Tests

[TP109]

Guess I have to eat my words. After making the config changes as suggested by lintalist, there wasn't much of a change in performance for UltraEdit. However, after rebooting the PC, there was a big change. UltraEdit became more responsive. I was able to use the automated setup to test it and it's very fast. Loading the 750MB test file was easily under .5 secs and much faster for the 250MB and 500MB files.

Disregard the current UltraEdit data until I finish analyzing the data and update the files. Although I don't have the final numbers yet, it appears that UltraEdit will be the top performer.

[TP109]

I performed the testing for UltraEdit over the past few days and came across some issues that took awhile to figure out. When using the automated setup, UltraEdit appeared to load the test files quickly, but when opening the same files manually, it was much slower. The problem is that the automated testing system isn't perfect and it looks like it's measuring load time AFTER the application itself is loaded. The setup doesn't work the same way with every editor and that is why it's necessary to verify results manually.

UltraEdit is a very capable editor that is designed to work with large files, but doesn't work the same way as the freeware editors previously tested. It uses much less memory than other editors and is disk-based. According to this source, http://www.ultraedit.com/support/tutori ... dling.html:

UltraEdit is a disk based editor, which means it only loads small portions of the file into memory. As UltraEdit uses 64-Bit handling for the file it has no real limit on file size, and does not use excessive RAM for editing.

.
UltraEdit can handle very large files and it opened all the test files up to 750MB without a problem. The issue is that what I have been testing for is different from UltraEdit's strengths - mainly in load time and large file editing. What I've been testing/measuring is launch time and probably more accurately, launch responsiveness time, which is a more important feature for the average user and where load time isn't really being taken into account.

Load time and maximum file size are important when working with large files and none of the freeware editors would be able to match the capabilities of UltraEdit in this regard. That is why it would be unfair to compare them. UltraEdit has a longer application launch time than any of the other editors tested (about 8 seconds on my system), but it's faster than the other editors when loading large files once the application has launched.

My testing doesn't take into account load time, but only when the application itself is fully launched, regardless of whether the test file is still loading or not. What is being measured is whether the application interface appears with the test file so that the user can verify that the proper file has been/is being loaded and that the application itself isn't in a "frozen" state or causing system problems during that time. A large test file stresses that situation while evaluating the editor's capability to handle large files. Some editors don't appear until after the test file is fully loaded while others render the GUI almost instantaneously while still loading the file. EditPadlite for example, appears very quickly with a 750MB file, but it's still loading it, even though there is no visual indication that it's doing so. However, the file can be scrolled through/edited during that time and Editpadlite's interface is usable. Akelpad also appears very quickly, but is in a frozen state while loading a 250MB test file, so it's not usable during that time, and thus not fully launched or responsive until the test file has completely loaded. So Akelpad has a longer launch responsiveness time than Editpadlite in my tests.

So it's comparing two different categories of editors that are used in different ways. UltraEdit is a configurable professional-grade editor designed to handle large files. It doesn't matter how long it takes to launch UltraEdit itself as long as it can open and edit large files quickly. The freeware editors are general-purpose applications where launch time responsiveness matters for quickly viewing a large number of smaller files or large files of up to 750MB (depending that the editor can handle that size file or not). UltraEdit has too long of a launch responsiveness time to be usable for the same purposes as the general-purpose freeware editors.

Anyway, I'm going to keep the UltraEdit data, but not include it going forward for those reasons.

[Midas]

Excellent write up, TP109. Thanks for everything.

[TP109]

The latest updated files have been uploaded to the original post with the UltraEdit data removed. Getting close to completion. Maybe one more edit left.

[TP109]

I think I'm finished for now. Had to fix VBA macros, make grammar corrections, clarifications and such. Probably more that can be done, but this is close to what I consider final.

I'm not making any judgements, but I will state that only one editor made it though all the tests and that was EditPadLite7, which was also a top performer for the launch and resource usage tests. Since no other editor could load the 750MB test file, I stopped there. It possibly could have went higher, but I didn't see the point since there was nothing else to compare it to.

That doesn't mean that EditPadLite is the best editor, "best" is subjective and it really depends on what is important to the user.

The charts and tables should tell you all you need to know.

I welcome any comments, criticism, or corrections. Review the data and provide some feedback.

I uploaded the latest and hopefully final files to the first post.

Thanks for all the comments so far.

[TP109]

I'm still testing text editors for functionality, but more informally, although I'm documenting issues when they come up. I'm considering creating some kind of document for that in the future. What I do is use an editor exclusively for a while, see how it performs and if issues pop up, document them and then switch to another editor. So far, I've tested or am testing Akelpad, Scite (various versions), PlainEdit, Win32Pad, Editor2, MiniSC Edit, Programmers Notepad, Notepad2-mod, and a few others. I've found issues or limitations with all of them.

I started with Scite because of it's performance, but dumped it because of crashing issues and lost code. It has other problems as well. Right now, I'm using PlainEdit (non-Net) for code, Editor2 for text, Notepad2-mod for .config & .ini files, and EditPadLite7 for log files. Already found some issues with PlainEdit, but I'm continuing with it at the moment to verify if the problems are repeatable.

[Midas]

I, for one, will await patiently whatever observations you'll share with us.

[TP109]

Thanks for your interest. Good to know that someone thinks this stuff might be important. Anyway, the functionality testing along with the performance data will significantly change the rankings, since the performance tests only tested launch speed, load time, and stability. Not really good data by itself for selecting an editor. Kind of like choosing the fastest car, but it doesn't matter if it has AC, a heater, sound system, or other features that many think are necessary.

[Midas]

I agree totally.

Suffice to say that if I had to choose, "Text Editors" would be my number one software category...

I don't think I am overly fussy, but I do like my editors nimble and fast. Come to think of it, I mainly use to different types: what I'll call here the "Notepad Replacement" and the "Text IDE" (which I read as "I Do it Expertly" ).

In the first category, Win32pad divides the crown of my personal preference with Editor2, while UltraEdit reigns supreme in the later -- although I strongly dislike its bloated appearance in versions newer than 16... note also that out of the 3, only Editor2 is natively 64-bits.

[TP109]

Correction for Editor2:
Retested 23 Nov 15:
Editor2
F:\Utilities\editor2\editor2.exe
2.9.1.13 Ansi
Empty - .12s
68MB - 22s


The comments by Midas got me curious again about some editors that keep getting mentioned but were not included in the previous testing, so I decided to do a quick test on some of them. As I stated before, I conducted an unofficial screen-out for those that couldn't handle large files or had excessively long launch times, but I didn't document that process and was relying on memory when asked. So I tested them today and have some data on them below:

NotePad Class Editors - Launch and Load Performance Test
November 4, 2015
Test Platform - Windows XP SP3, 1.6GHz, 2GB RAM

ATpad
F:\Utilities\ATPad\ATPad.exe
2.0.100
Empty - .34s
89MB - 10s
250MB - 30s
500MB - 68s but becomes unstable after loading

Editor2
F:\Utilities\editor2\editor2.exe
2.9.1.13 Ansi
Empty - .91s
89MB - 24s

Geany Portable
F:\Utilities\Geany\GeanyPortable.exe
1.22
excessively long launch time (greater than 10 secs) - dropped

MiniSCEdit
F:\Utilities\RichEdit\Bin\Mini SC Edit.exe
1.2.2.0
Empty - .89s
89MB - Won't load file, freezes, locks up

Notepad2-mod
F:\Utilities\notepad2-mod\Notepad2.exe
4.2.25 r964
Empty - .27s
89MB - 20s
250MB - 135s (usable)

QuickEditor
F:\Utilities\quickeditor\qeditor.exe
3.5
Empty - .15s
89MB - 3s
250MB - 9s
500MB - 50s (usable)
750MB - 97s (usable)

Win32pad
F:\Utilities\win32pad\win32pad.exe
1.5.10.4
Empty - .14s
89MB - 17s

The list displays the Editor Name, the path on the PC, the editor version, empty launch time and loaded launch times in seconds (with the test file size used). As can be seen, all of the editors except Geany Portable had acceptable empty launch times. However, with a load, that changes a lot. The only editor that could handle up to a 750MB file was QuickEditor and it remained stable even then. The rest crashed, froze, locked up, or refused to open depending on the file size. The best performers were QuickEditor, followed by ATPad and Notepad2-mod, but ATPad becomes unstable at 500MB and Notepad2-mod has an excessively long load time with 250MB. Once an editor takes around 15 secs or so to load , I didn't test it any further unless it looked really stable like QuickEditor.

So it looks like I really didn't overlook any editors in my previous tests, because none of the above are in the performance class of the previously tested editors except for QuickEditor, but it's a 2006 version and not really a full featured code editor. And although Notepad2-mod is a code editor, it would have been dropped early in the testing since it takes 20 secs to load a 89MB file.

This should complete the performance testing for the freeware editors.

[Midas]

Seeing these results -- and although I said I wasn't fussy -- I'm afraid I didn't mention a couple of features that I'm very partial for in the "Notepad Replacement" category -- in fact, I consider them absolute "must-haves" ranking way higher than the capacity to handle large files, which I don't value as much:

- time stamping (preferably with user defined format, generally for defining with the ISO date/time standard);
- selection case switching (with lower- and upper-casing as bare minimum); and
- URL activation (done in a sensible way, since otherwise it usually collides with text selection).

I know this is one of the most over-populated software categories out there, nonetheless, I'm really curious about two lesser regulars of TPFC:

• NFOPad (http://www.portablefreeware.com/?id=2164)

• J.A.N.E. (http://www.portablefreeware.com/forums/ ... php?t=6050)

And was AkelPad left out because you had already tested it?

BTW, there's even a TPFC category for this kind of editors: http://www.portablefreeware.com/?sc=63...

[TP109]

As you recall, because of the sheer number of editors out there, early on I limited the original formalized and detailed performance testing to free, portable and fairly popular programming class editors. An unofficial weed-out process was used to reduce the number of editors to be tested and identify those that were the most relevant for testing such as being in common use and more likely to be used with important data. That is why that kind of testing is more important for that class of editor than for the more numerous notepad class editors primarily used to view readme files.

So it was important to identity which editors should be tested up front, so that the test results are complete as possible and less likely to be challenged. As I stated before, the weed-out pretests were not documented and I kept getting questions like, "what about my favorite editor." I expected that, it's called inclusion creep and it's a common trait to all fields. That is why I explained all of this previously. And that is why I did that quick testing yesterday, hopefully to satisfy those kinds of questions and to validate for myself that the selection process used in the formalized testing was correct.

That said, I'm not against more requests and questions, as long as it's non-formalized and doesn't require complex comparisons and analysis. After all, I'm already setup to do that testing quickly - it's just a matter of creating a configuration file and writing up the data in a notepad class editor. So testing these notepad class of editors is kind of a fun comparison test and discussing it provides useful and important information that I may have not been aware of. For instance, I learned yesterday that QuickEditor is fast and stable, a real surprise. Anyway, I could write about the psychology of the testing environment all day long.

Now to answer your question. Yes, Akelpad was included in the formalized testing. See the documentation link provided in the original post of this thread. I will test the two editors you mentioned and post the results later. The important features you mentioned is also useful since that kind of information is relevant to future functionality testing. It would be useful if others posted their important feature list here as well.

[Midas]

Hey, I make no requests, it's good to have your testing as it is. Whatever I post here is just in the spirit of adding something to the conversation -- or at least making it look like one. Thanks for considering my ramblings. Having said that, I really need to go back and review the topic before I blab any more...

[TP109]

Actually, I consider your input valuable and your questions well-informed and relevant. I'm not complaining or picking on anyone in particular, just want to put it out there why the formalized testing was conducted in the manner it was.

Hmm, I just noticed the file attachment feature. Didn't notice that before. That will be useful.

[lintalist]

... UltraEdit ... Editor2 is natively 64-bits.

@midas: fyi UltraEdit now also comes in 64-bit just in case you want to try it out.