Site usability testing results

All suggestions about TPFC should be posted here. Discussions about changes to TPFC will also be carried out here.
Message
Author
User avatar
webfork
Posts: 9788
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Site usability testing results

#1 Post by webfork » Fri Dec 18, 2020 1:58 pm

Somehow I've gone all these years without some proper website usability testing, despite having recommended it at multiple different companies I've worked at. With the caveat that this is both unscientific and not very comprehensive, here are some things that came out of testing:

Recommendations
  • After clicking Download - Move the page down to the "Extract instructions" ... I saw my user struggle with what to do with the program after download. They just assumed you double-click whatever is downloaded.
  • A primary search result - Currently the basic search just grabs any keyword and lists it by the last time it was updated, but a "primary" keyword would mean that typing in Firefox would go to the Firefox entry, with entries that just mention Firefox coming after. This would especially help with our more generically named programs like Types and Autoruns. For simplicity, I'd suggest only one primary keyword or key phrase per entry, e.g. PDFXChange would only apply to one of our entries, probably the "Editor" version, the other would be "PDFXChange Viewer".
Other notes
  • Google custom search my user didn't see this under Detailed Search when trying to find a program so we might reformat that somehow, perhaps just making the text bold.
  • Popular searches - My user hovered over this area for a while, I think just because of it's location and list of keywords.
  • User noted that our site was very text heavy but that its not necessarily a bad thing, just a lot of info. We might consider hiding some elements in the regular database listing like Stealth, Unicode support, License, and Similar/alternative apps. They could hide in the regular listing (what shows up on the front page) or appear if you click "more info".

    As someone who tends to be extra verbose in my entry edits, I took this as an indication that I should try to write shorter entries.
  • Forums - They didn't go anywhere near that section, so either that didn't seem interesting from the front page, or they were looking for something more official. Unclear.

---

Feedback or related ideas are of course welcome. And if you want to watch one of your friends try to use our website and report the results, that's welcome too.
Last edited by webfork on Sat Dec 19, 2020 11:21 pm, edited 3 times in total.
Reason: (fixed bad wording)

thepiney
Posts: 155
Joined: Wed Aug 31, 2011 11:57 am

Re: Site usability testing results

#2 Post by thepiney » Fri Dec 18, 2020 4:28 pm

After clicking Download --> Could a "I hate myself for saying it" popup be added? Maybe one that could be printed or saved as a file with the "download" name +extraction+portability instructions?

A primary search result --> I like the "primary" keyword idea, like have that program/post weighted.

Forums --> Most of the people I've sent to the site will rarely, if ever look at them, but some would definitely check them and the "comments" out to see what others are saying.

User avatar
Andrew Lee
Posts: 2603
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Site usability testing results

#3 Post by Andrew Lee » Fri Dec 18, 2020 6:00 pm

I will respond to those items that I can action upon.
webfork wrote:
Fri Dec 18, 2020 1:58 pm
[*] A primary search result - Currently the basic search just grabs any keyword and lists it by the last time it was updated, but a "primary" keyword would mean that typing in Firefox would go to the Firefox entry, with entries that just mention Firefox coming after. This would especially help with our more generically named programs like Types and Autoruns. For simplicity, I'd suggest only one primary keyword or key phrase per entry, e.g. PDFXChange would only apply to one of our entries, probably the "Editor" version, the other would be "PDFXChange Viewer".
I can probably do something about this by giving priority to title matches, followed by keyword matches, followed by description matches.
webfork wrote:
Fri Dec 18, 2020 1:58 pm
  • Google custom search my user didn't see this under Detailed Search when trying to find a program so we might reformat that somehow, perhaps just making the text bold.
I think I know what's going on with this one. I suspect that's because the custom search section requires a scroll on most displays to reveal itself. Maybe moving it to the top will help?
webfork wrote:
Fri Dec 18, 2020 1:58 pm
[*]User noted that our site was very text heavy but that its not necessarily a bad thing, just a lot of info. We might consider hiding some elements in the regular database listing like Stealth, Unicode support, License, and Similar/alternative apps. They could hide in the regular listing (what shows up on the front page) or appear if you click "more info".
Maybe make the "Compact view" default?

User avatar
Midas
Posts: 5811
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: Site usability testing results

#4 Post by Midas » Sat Dec 19, 2020 4:48 am

Andrew Lee wrote:Maybe make the "Compact view" default?

No position yet on the other changes, but I really like this one. 8)

User avatar
Andrew Lee
Posts: 2603
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Site usability testing results

#5 Post by Andrew Lee » Sat Dec 19, 2020 8:46 pm

Midas wrote:
Sat Dec 19, 2020 4:48 am
Andrew Lee wrote:Maybe make the "Compact view" default?
No position yet on the other changes, but I really like this one. 8)
Done, since this option is quite easy to toggle.

User avatar
webfork
Posts: 9788
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: Site usability testing results

#6 Post by webfork » Sat Dec 19, 2020 11:09 pm

Andrew Lee wrote:
Fri Dec 18, 2020 6:00 pm
I can probably do something about this by giving priority to title matches, followed by keyword matches, followed by description matches.
Glad that was a good idea and potentially something you could actually implement. This was all kind of an elaborate experiment that I wasn't sure would really help.

webfork wrote:
Fri Dec 18, 2020 1:58 pm
Maybe make the "Compact view" default?
I saw that you made the change and I do admit it looks better. My hesitation is that now the important Extract instructions are now even less visible, which might create more confusion. So if someone clicks "download" without looking at the rest of the entry, they may wonder why it doesn't work right. Especially entries like FileZilla and muCommander, where there some detailed steps to make it work portably.

webfork wrote:
Fri Dec 18, 2020 1:58 pm
I suspect that's because the custom search section requires a scroll on most displays to reveal itself. Maybe moving it to the top will help?
I'm reluctant to move it to the top, as search engines will sometimes immediately push users to a specific page in the forum without additional context. So for example, the first post of the Firefox official forum thread is a little vague (talks about an IRC conversation from 2007).

Our local search tool might be flawed, but still reliably points people to database entries vs. forum posts. That's where I'd like people to click first, then goto another search engine, as we have now. Maybe we add a heading in the same size / font as "Detailed Search" that reads "Google Custom Search" with the subheading "can't find what you're looking for?"


---
thepiney wrote:
Fri Dec 18, 2020 4:28 pm
After clicking Download --> Could a "I hate myself for saying it" popup be added? Maybe one that could be printed or saved as a file with the "download" name +extraction+portability instructions?
Yeah that's not an easy thing to solve. I don't know how to put that front-and-center, but some way to point people to clear instructions would be ideal. It definitely wouldn't need to apply to all entries, as some of our extract instructions are very straightforward, while others are very detailed/complex.

It's also possible that we embrace the rationale for having detailed instructions in the first place, and just say our visitors are assumed to be more technical, that they might not follow through with an extract process the first time (as in my test) but maybe in future visits.

thepiney wrote:
Fri Dec 18, 2020 4:28 pm
... some would definitely check them and the "comments" out to see what others are saying.
Agreed ... on many websites, I actually weigh comments higher than the entry sometimes. Even poorly worded reviews are often more reliable and informative than most software descriptions.
Last edited by webfork on Sun Dec 20, 2020 12:20 am, edited 1 time in total.
Reason: (fixed wording)

User avatar
Andrew Lee
Posts: 2603
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Site usability testing results

#7 Post by Andrew Lee » Sun Dec 20, 2020 1:55 am

webfork wrote:
Sat Dec 19, 2020 11:09 pm
Andrew Lee wrote:
Fri Dec 18, 2020 6:00 pm
I can probably do something about this by giving priority to title matches, followed by keyword matches, followed by description matches.
Glad that was a good idea and potentially something you could actually implement. This was all kind of an elaborate experiment that I wasn't sure would really help.
Working on it. This one will take a bit more time to impement.
webfork wrote:
Fri Dec 18, 2020 1:58 pm
I saw that you made the change and I do admit it looks better. My hesitation is that now the important Extract instructions are now even less visible, which might create more confusion. So if someone clicks "download" without looking at the rest of the entry, they may wonder why it doesn't work right. Especially entries like FileZilla and muCommander, where there some detailed steps to make it work portably.
Would an audience of this nature even care about whether it is "portable"? Maybe they are just happy to find an application that meet their need, install/run and move on. :D
webfork wrote:
Fri Dec 18, 2020 1:58 pm
Maybe we add a heading in the same size / font as "Detailed Search" that reads "Google Custom Search" with the subheading "can't find what you're looking for?"
Again, very simple to do (and done). Check it out and let me if that's what you had in mind.

User avatar
webfork
Posts: 9788
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: Site usability testing results

#8 Post by webfork » Mon Dec 21, 2020 8:27 pm

Andrew Lee wrote:
webfork wrote:
Fri Dec 18, 2020 1:58 pm
So if someone clicks "download" without looking at the rest of the entry, they may wonder why it doesn't work right. Especially entries like FileZilla and muCommander.
Would an audience of this nature even care about whether it is "portable"? Maybe they are just happy to find an application that meet their need, install/run and move on. :D
My main issue is that there are some programs on the site that are problematic to skip the Extract instructions on:
  • Some entries have steps that prevent it from being a standard installer, which might make visitors wonder why they didn't just visit a much bigger site like Snapfiles or Softpedia. (Netsetman, FreeOffice, FastCopy, and any program that uses Uniextract)
  • I've seen a lot of fairly technical people balk at a file that ends with 7Z, RAR, or TAR.GZ file (CherryTree, DXwind), so they might just abandon the process. The extract steps generally include a link to a compatible compression program.
  • At least one program (XMind portable) is much larger than the regular installer version.
Also, and this is just an opinion, but I see some of the project of portability as being better software. The admittedly tedious Extract steps are worth it because they enable a kind of program that just doesn't trip over itself trying to be convenient for users. A few steps and you've enabled a versatile way to backup, sync, capture settings, run separate/multiple instances, etc.

webfork wrote:
Fri Dec 18, 2020 1:58 pm
Maybe we add a heading in the same size / font as "Detailed Search" that reads "Google Check it out and let me if that's what you had in mind.
Looks great, thanks :)

User avatar
Andrew Lee
Posts: 2603
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Site usability testing results

#9 Post by Andrew Lee » Tue Dec 22, 2020 2:08 am

The prioritization of title > keywords > description is done.

I tried on a couple of searches like "firefox", "chrome" etc. and the first search result is always the primary one, unlike previously.

Lemme know if you find any bugs.

User avatar
Midas
Posts: 5811
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: Site usability testing results

#10 Post by Midas » Tue Dec 22, 2020 5:17 am

webfork wrote:Also, and this is just an opinion, but I see some of the project of portability as being better software. The admittedly tedious Extract steps are worth it because they enable a kind of program that just doesn't trip over itself trying to be convenient for users. A few steps and you've enabled a versatile way to backup, sync, capture settings, run separate/multiple instances, etc.

FWIW, an opinion I totally share. :thumbsup:

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: Site usability testing results

#11 Post by vevy » Tue Dec 22, 2020 8:26 pm

Andrew Lee wrote:
Tue Dec 22, 2020 2:08 am
The prioritization of title > keywords > description is done.

I tried on a couple of searches like "firefox", "chrome" etc. and the first search result is always the primary one, unlike previously.

Lemme know if you find any bugs.
Could you please apply that to the CLI DB?
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
Andrew Lee
Posts: 2603
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Site usability testing results

#12 Post by Andrew Lee » Wed Dec 23, 2020 3:47 am

vevy wrote:
Tue Dec 22, 2020 8:26 pm
Could you please apply that to the CLI DB?
Done.

However, you may find it does not work as well eg. when you search for "mkdir", because I did not sort on the entire list (all 500 entries), but just the current page (default 5 entries).

I made this hack because the original SQL optimizes the retrieval by only fetching the current page with a certain offset instead of the full 500 entries every time. Since there is no way to codify this "title > keywords > description" prioritization within the SQL, I did the sorting within the current page only, which seems to work well enough for the main database.

In the future, if this approach is found to be insufficient, I might really have to think about fetching the whole she-bang and sorting the entire list. This will of course increase memory and CPU expenditure per retrieval.

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: Site usability testing results

#13 Post by vevy » Wed Dec 23, 2020 4:01 am

Thanks

How about increasing the default number of results/page (in search only) to 10 or more?

As for the SQL query, could it be split into three?:
- title
- use cases/keywords/description/etc
- other fields (i.e. all excluding the above)

then combine in order, deduplicate, and pass to frontend. I guess it won't be as heavy, especially with indexing.
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
Andrew Lee
Posts: 2603
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Site usability testing results

#14 Post by Andrew Lee » Wed Dec 23, 2020 9:03 pm

vevy wrote:
Wed Dec 23, 2020 4:01 am
As for the SQL query, could it be split into three?:
- title
- use cases/keywords/description/etc
- other fields (i.e. all excluding the above)

then combine in order, deduplicate, and pass to frontend. I guess it won't be as heavy, especially with indexing.
In terms of code changes, I suspect it'd be easier to do the 500 instead of splitting the SQL query.

The SQL query actually includes a lot of other stuff, including pagination, ordering, scoring etc. which are built up along various stages, cumulating into a single SQL query. It will be more difficult to split into 3 queries based on the current code.

Anyway, we will cross that bridge when we come to it...

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: Site usability testing results

#15 Post by vevy » Fri Dec 25, 2020 3:58 pm

Andrew Lee wrote:
Wed Dec 23, 2020 9:03 pm
The SQL query actually includes a lot of other stuff, including pagination, ordering, scoring etc. which are built up along various stages, cumulating into a single SQL query.
Isn't it better to leave those to the frontend?
Anyway, we will cross that bridge when we come to it...
You are giving me PTSD! But, alright!
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

Post Reply