CLI Database Discussions

[vevy]

Mini-requests:

• Add [ section ] tag to the entry edit toolbars. (Andrew: To be implemented)
• Add color to monospace/code tag and change font to Consolas (like in the forums). (Andrew: To be implemented)
• Add already-usable (but hidden) tags to the entry edit toolbars (e.g. color, strike-through) (Andrew: To be implemented)
• Remove "v" from Version field. Insert automatically in entry view if needed. (Andrew: Done)
• System requirements: new option: "Untested". (Andrew: Done)
• Rename Open-source checkbox to "Open-source/Public-domain". (Andrew: To be implemented)
• Bug: When adding (not editing) an entry, parent ID disappears in preview.
• Change "System requirements" name to "OS support".
• If an icon file is chosen, there is no way to remove it during adding or editing. Same for screenshot?

------------------------------------------------------------
Use case and format requests to be added later.

ID	Request	Description	Priority (1=min, 3=max, p=postponed)	Andrew: Priority	Andrew: Status
	General
#1	Clearer edit logs	Currently, entry log history is diff-ed by characters, which makes figuring the changes out very difficult. Better do it by filed or paragraph.	2
#2	Adapt main page links to CLI.	Like "About", "Useful Links", etc	2p
#3	Adapt current templates to CLI	"How to extract", "Writes settings to", etc	1p
#4	Stock Icon for TUI tools.	--	1
#5	Promote the best tools.	Make a system for promoting the better tools (Editor's pick, voting, etc)	?
	Listings and Filtering/Sorting	Filtering on one/combined fields mainly under Detailed Search.
#6	Sorting by last edited	Sorting entries by last entry update date (as logged).	2
#7	Filtering by License	e.g. only FOSS results.
#8	Filtering by release date	For example, tools released on or after 2018-01-01	1-2
#9	Filtering by developer	All tools by a given developers. Useful for maintenance and usage.	1-2
#10	Filtering by categories	Useful with large categories when combined with other filters.	1-2
#11	Filtering by use case.	--	2p
#12	Filtering by tool size.	--	1
#13	Filtering by system requirements.		1p Spoiler!    (when such data is entered satisfactorily)
#14	Filtering by entry type/status	Tool, package, Win. package, discontinued, etc.	1-2(p?)
#15	Filtering by dependencies	Needs the field to have a Boolean option of Y/N first (like Stealth). Needs the information to be entered first.	1p
#16	Filtering by stealth	Needs the data first.	1p
#17	Clickable badges	Spoiler!    Make badges (additional status/warning) clickable (to a page filtered by these badges e.g. Win packages only). This will have the side benefit of making categories like "Collection" and "Unix Ports" unneeded.	1-2
#18	Field incomplete message	If a filterable field is optional, show a message atop the "filter results page" that says: "Not all tools contain this type of information. Results may be incomplete."
	Search
#19	One-word synonyms	Handle one-word synonyms: like audio=sound≈music.	1p
#20	Handle aliases	Use cases like overlay video on another=picture in picture effect and formats like jpg=jpeg.	1-2p
#21	Partial matches.	"implement" should also match "implementation".	2p Spoiler!    (I'd want it today, but not easily doable; so postponed for now)
#22	Search terms separately	i.e. not necessarily in order	2p
	New Entry Fields
#23	Developer/Author field (Required)	Show suggestion like "Title" field, but clicking a suggestion enters it into the field (not disabled like "Title").	2-3
#24	Documentation Link (Optional)	--	1p
#25	Documentation Text (Optional)	Collapsed by default.	1p
#26	IsWarningNet (optional)	Spoiler!    Boolean. As text or badge with hover text. Be careful with network-facing tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).	1-2
#27	IsWarningSecurity (optional)	Spoiler!    Boolean. As text or badge with hover text. Be careful with security tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).	1-2
#28	IsWarningDecompression (optional)	Spoiler!    Boolean. As text or badge with hover text. Be careful with decompression tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).	1-2
#29	IsNotArchiver (optional)	Spoiler!    Boolean. As text or badge with hover text. Useful because of the difference in behavior with compression tools in GUI vs CLI. This tool does NOT put multiple files into an archive. It just compresses a single file to reduce its size. You may want to group your files first using an archiver. Also, most compressors DELETES the original file by default after compression).	1
#30	IsNotCompressor (optional)	Spoiler!    Boolean. As text or badge with hover text. Useful because of the difference in behavior with compression tools in GUI vs CLI. This tool only puts multiple files into an archive. It doesn't compress them. You may want to also use a compressor for that.	1
#31	Blurb	One-liner/Short Description field (optional).	1
#32	IsInIndex	Boolean, optional. Added to the CLI Tool Index or not. Useful for managing the additions of tools.	1-2p
#33	Available in:	Optional. Still WIP. Enter delimited IDs of parent tools, e.g. 6,9,11 Will put a field with sentence: Available in UnxUtils, GnuWin, Gow.	2p
#34	Notes	Edit box. Optional.
#51	GUI Front-ends	Edit box. Optional. Link or just name the front-ends available for a tool.	1-2
	Existing Fields
#35	Software title: Suggestions	To avoid duplicate entries. Make suggestions clickable into a new tab.
#36	Move Additional statuses	Next to the title. This should better help clarity and will save space.	2		Done
#52	How to extract: "Standard extraction"	Spoiler!    Add the following template (in code) to the extraction field. Most CLI tools are in simple archives. No need to add the full instructions to each entry because: 1. They don't really change. 2. They'll take space. 3. This allows unification and improving the instructions or adding more/better examples at any time. Code: Select all [url=https://www.portablefreeware.com/forums/viewtopic.php?t=25028&p=97047#p97047][size=2][color=grey][u]Standard extraction[/u][/color][/size][/url]	2-3
#53	How to extract: "See parent"	Spoiler!    Add the following template (in code) to the extraction field. Code: Select all [size=2][color=#969696]See parent package(s) or port(s).[/color][/size]	2-3
#54	Templates as unexpanded tags	Spoiler!    Going with your suggestion about [warning] tags, and in the same spirit of unifying template wording and facilitating global changes, add templates in general as tags. e.g. [extraction_see_parent] in #53 and expand automatically. There are ways to expose the wording to the contributor if you wish to discuss.	2-3
	Additional status badges
#37	Package	--
#38	Collection	--
#39	Win Package	--	2
#40	Tool	--
#41	Unix Port	--
#42	Discontinued	--
#43	CLI-mode GUI	CSS: Spoiler!    Code: Select all span.status.gui_cli { box-shadow:inset 0px 0px 0px 1px #444; border: 1px solid #444; color:#F2F0E6; text-shadow: -1px 0px 1px #000, 1px 0px 1px #000, 0px -1px 1px #000, 0px 1px 1px #000; background:linear-gradient(167deg, #F0F8FF, #F0F8FF 50%, #555 50%, #555); }	2-3
#44	TUI	CSS: Spoiler!    Code: Select all span.status.tui { background-color: #004164; border: 1px solid #009CBA; border-radius: 0; box-shadow:inset 0 0 0 1px #008080, inset 0 4px 0 0 #008080; color: #B8C821; }	2-3
#45	Batch	CSS: Spoiler!    Code: Select all span.status.batch { background-color: white; border: 1px solid #0061a7; border-radius: 0; box-shadow:inset 0 0 0 1px #0061a7, inset 0 4px 0 0 #0061a7; color: #222222; }	2-3
#46	Internal Command	CSS: Spoiler!    Code: Select all span.status.internal_command { background-color: black; border: 1px solid #91979E; border-radius: 0; box-shadow:inset 0 0 0 1px #91979E, inset 0 4px 0 0 #DFE3E6; color: white; }	2-3
#47	Windows-Included	CSS: Spoiler!    Code: Select all span.status.windows_included { border: 1px solid royalblue; padding-left: 24px; background-image: linear-gradient(to right, #F35220, #F35220), linear-gradient(to right, #7DB606, #7DB606), linear-gradient(to right, #05A6F0, #05A6F0), linear-gradient(to right, #FFBA08, #FFBA08), linear-gradient(to right, #FFFFFF, #FFFFFF); background-repeat: no-repeat; background-position: 1px 1px, 11px 1px, 1px 11px, 11px 11px, 0,0; background-size: 9px 9px, 9px 9px, 9px 9px, 9px 9px, 100% 100%; }	2-3
#48	Alternative Shell	CSS: Spoiler!    Code: Select all span.status.alternative_shell { background-image: linear-gradient(to right, green, green), linear-gradient(to right, yellow, yellow), linear-gradient(to right, red, red), linear-gradient(to right, rgba(0,0,0,0), rgba(0,0,0,0)); background-repeat: no-repeat; background-position: right 12px top 1px, right 7px top 1px, right 2px top 1px, 0 0; background-size: 2px 2px, 2px 2px, 2px 2px, 100% 100%; background-color: black; border: 1px solid #aaaaaa; border-radius: 0; box-shadow:inset 0 4px 0 0 #aaaaaa88; color: lightgreen; }	2-3
#49	Shell Extension	CSS: Spoiler!    Code: Select all span.status.shell_extension { padding-left: 24px; background-image: linear-gradient(to right, #ffffff, #ffffff), linear-gradient(to right, #ffffff, #ffffff), linear-gradient(to right, #91979E, #91979E), linear-gradient(to right, #91979E, #91979E), linear-gradient(to right, #91979E, #91979E), linear-gradient(to right, #91979E, #91979E), linear-gradient(to right, #000000, #000000), linear-gradient(to right, #000000, #000000), linear-gradient(to right, rgba(0,0,0,0), rgba(0,0,0,0)); background-repeat: no-repeat; border-radius: 0; background-position: 10px 8px, 7px 11px, 0px 0px, 0px 0px, 19px 0px, 0px 19px, 2px 6px, 0 0; background-size: 2px 8px, 8px 2px, 20px 6px, 2px 20px, 2px 20px, 21px 2px, 17px 13px, 0% 0%; background-color: white; color: black; }	2-3
#50	Parent>Child details	Allow child entries to get details from parent tool (collection)	2-3
	Use Cases	TBC
	Formats	TBC

-------------------------------------------------------------------------------

Spoiler!   

Standing requests:

• Allow child entries to get details from parent tool (collection): (Andrew: Testing)
  • The reasoning for this is:
    • Avoid entering (and updating) the same information in hundreds of child tools.
    • Allow filtering by this criteria. If we simply leave these fields empty in the child tools, they won't show on filtering by these criteria. For example, open-source tools that are updated after 2019-01-01 with use case X.
  • Entering entry id for collection in a field instructs TPFC to grab the details of this field from the parent collection.
  • For example, a release date of "{app=13}" means has the same release date of entry #13.
  • If the release date of app #13 (the parent collection) is updated, this gets reflected in all child entries.
  • Method #2 If it is difficult to change the type of the current fields (like release date), you can:
    • Create a general field for "Parent entry ID".
    • For each needed field (like release date), create a checkbox to use that of parent instead. Select which to use (the original filed or the checkbox) using radio buttons (i.e. make them mutually exclusive).
• New fields:
  • IsWarningNet (Boolean) field (optional).
  • IsWarningDecompression (Boolean) field (optional).
  • IsWarningSecurity (Boolean) field (optional).
  • IsNotArchiver (Boolean) field (optional).
  • IsNotCompressor (Boolean) field (optional).
  These fields can be small in space/checkboxes in one line, etc.
• The table request mentioned here. (Andrew: No solution)

------------------------------------------------------------------

Spoiler!   

To help keep both databases with a unified code base from the start, I made this table to higlight similarities/differences and keep things more manageable.

Property	Main Database	CLI Database	Notes
Existing Fields:
Software title (Required)License (Required)	Enabled	Enabled
Categories (Required)	Enabled	Enabled	Two-level main categories in both.
Website URL (Required)	Enabled	Enabled	Some tools may not have one site (e.g. Unix tool ports).
Size (in bytes) (Required)	Enabled	Enabled	Some tools have variable sizes depending on features (e.g. Cygwin, Yori). Workaround: enter "0".
License (Required)	Enabled	Enabled
System requirements (Required)	Enabled	Enabled
Description (Required)	Enabled	Enabled
Writes settings to (Required)	Enabled	Enabled
How to extract (Required)	Enabled	Enabled	Needs a template for CLI tools (Add to PATH, launch in console, etc).
Version number (Optional)	Enabled	Enabled
Download URL (Optional)	Enabled	Enabled	Some tools are only downloadable as a part of a collection. Solution: Leave empty.
Icon (Optional)	Enabled	?	Most CLI tools don't have icons.
Screenshot (Optional)	Enabled	?	Can be a screenshot of help/usage.
Additional Features (Optional)	Enabled	Disabled?	See proposed system.
Additional status (Optional)	Enabled	Enabled?	We need to discuss adware in CLI tools.
Path portability (Optional)	Enabled	Enabled	Is it a given in CLI tools?
Unicode support (Optional)	Enabled	Enabled
Dependencies (Optional)	Enabled	Enabled
Stealth (Optional)	Enabled	Enabled
Keywords (Optional)	Enabled	Enabled
Forum topic ID (Optional)	Enabled	Enabled
Similar/alternative apps (Optional)	Enabled	?	Replaceable with function tags?
Suggested by (Optional)	Enabled	Enabled
Release date (Optional)	Enabled	Enabled
Proposed Fields:
Functions	?	Enabled	see discussion
Part of	Disabled	Enabled	For tools that come under a main tool/collection. (child)
Parts	Disabled	Enabled	For the collections that contain multiple tools. (parent)
Editor's Pick	Enabled?	Enabled	For the excellent tools that deserve to be highlighted.
Documentation/Link to Documentation	Disabled	Enabled?

------------------------------------------------------------------

Original Post:

Spoiler!   

Hi, everyone

I know Andrew is not big on CLI tools, but they are very useful. I keep discovering new tools and some of them are jaw-dropping in functionality and keep wondering: what else is there that I could be missing?

There are many sites that list CLI software among GUI software. Alternativeto.net has tags that help separate such tools, but their database doesn't have that many.

Also, some software authors have many capable CLI tools on their sites.

What I mean is, they are all-over the place and deserve a separate database.

Is there something like that out there? Eager to here your recommendations (and ideas)

-------------------------------------------

Spoiler!   

Hide footer attachments:

Batch.png (1.41 KiB) Viewed 20 times

extraction_see_parent.png (4.24 KiB) Viewed 20 times

standard-extraction.png (3.95 KiB) Viewed 20 times

TUI.png (741 Bytes) Viewed 20 times

CLI-mode.png (2.94 KiB) Viewed 20 times

Shell Extension.png (1.67 KiB) Viewed 20 times

Alternative Shell.png (1.84 KiB) Viewed 20 times

Windows-Included.png (2.04 KiB) Viewed 20 times

Internal Command.png (1.66 KiB) Viewed 20 times

[SYSTEM]

https://tinyapps.org/ isn't exclusively for command-line apps, but a large portion of their catalog is command-line.

[vevy]

Thanks, System. Nice site, but, by my quick look, it doesn't seem to have many tools (for example ptime, rxrepl, busybox, xpdf, etc) and old versions of others.

Any other suggestions are appreciated.

[billon]

https://www.robvanderwoude.com/

[vevy]

https://www.robvanderwoude.com/

Good find. It has some nice tools and references many other authors. It says though:

This list doesn't even attempt to be complete, as there are literally thousands and thousands of batch file utilities available on the web.

The limited space for the descriptions of these tools doesn't always do them justice, so check out the authors' web sites for more details.

I propose we make a (separate) database here for CLI tools. Current database functions as usual while the other serves as a comprehensive directory for all these good tools that risk going into obscurity.

Andrew, you just create a new database for us and we will populate it . Little work is needed on your part. I don't suppose that is too much to ask? Just link it at the main site's top bar (Next to Latest, Categories) and we will do the work.

What do our veterans think? Is it worth a shot?

[TP109]

A search for command line tools in the TPFC Forums brought up 208 pages of results, so command line tools have been an ongoing discussion for awhile. At one time, there was an entry in the FAQs titled, "Why don't you accept command-line apps?", but it's no longer listed on the FAQ page .

The thread below was dedicated to discussing command line tools in 2016.
TPFC Forums - Great CLI Programs

The page below has a list of where to find some command line tools.
https://cects.com/find-win32-cli-utilities/

I propose we make a (separate) database here for CLI tools. Current database functions as usual while the other serves as a comprehensive directory for all these good tools that risk going into obscurity.

I use command line tools on a daily basis and a database for command line tools would be interesting and useful, but I'm not sure if it would be feasible or appropriate for TPFC to take on that task. I recall that possibility being discussed before, but I haven't located the exact thread.

[Specular]

I think the idea of an index of CLI programs is an interesting one but I also wonder how that would gel with carrying over the existing nature of the PFDB, in that most programs, descriptions and instructions are written and intended for a more general user audience, while CLI tools each have a learning curve of their own and I wonder what such a database would look like if carrying over PFDB principles.

Even things as simple as entry descriptions for example over time are often edited to simplify the verbiage yet with CLI programs it necessitates a better understanding of the technical details of the program, any additional dependencies (some of these are moving targets when changes occur for example), 'easy to use' would also need be judged differently and generally users be expected to be more familiar with command-line use and its limitations (which can trip up even experienced users, in areas like path escaping).

I think PFDB has a pretty clued-in, poweruser-leaning audience based on comments and thread postings but I'd be curious how entries and expectations would be handled.

Edit: and I'd also add that it's uncertain how many would be willing to maintain updates to such a DB, given for the audience we have it's a more niche area and keeping track of many updates is something that tends to be picked up by a smaller handful of users and sporadic contributors. Proposing something is obviously simpler than maintaining it.

[webfork]

What do our veterans think? Is it worth a shot?

Some input off the top of my head: the truth is that most CLI tools are portable, so it could make sense to include here. There's some amazing CLI software that could definitely benefit from some time and attention. Still, there are some concerns:

• Advocates - Ultimately a lot of good software arrived because it had a champion. I can definitely get behind great CLI tools like 7zip's command-line tools, SFK, and PDFBox. All those are amazing programs, but I don't really have a toolset beyond that.

• Maintenance - We have a group that's been here for a long time that helps keep entries up to date, works on descriptions, and generally tries to make sure that we find mirrors for dead software. A software listing without a fairly dedicated crew probably will be a flash in the pan.

• Platforms - Does this include non-Windows CLI tools?

• Dependencies - CLI software does have the same problems that regular software has, including dependencies (cygwin and Java) and system configuration (PowerShell vs. CMD) and all of that would definitely need to be part of any comprehensive site.

Whatever the case, it's a great idea I hope someone can execute on.

[Andrew Lee]

At one time, there was an entry in the FAQs titled, "Why don't you accept command-line apps?", but it's no longer listed on the FAQ page.

As the database evolved into being managed by the community (via voting), at one point I think we decided that the good "taste" of the community will take care of this.

I remember we used to have a lot of debates not only about CLI, but also .Net, Java, etc apps (Electron-based apps were still a rarity then). Now there are a sprinkle of such apps in the database on a case-by-case basis.

I kinda like this more than the black-and-white approach, much like how I didn't agree with the hardline view that only stealth will do for portability. Life is more interesting this way.

Personally, I still think we should have a damn good reason for including a CLI app, since these are mostly portable by default, with very few exceptions (and those exceptions are IMHO plain evil ).

[Midas]

I kinda like this more than the black-and-white approach, much like how I didn't agree with the hardline view that only stealth will do for portability. Life is more interesting this way.

I see what you have done there... And although I am a fan of (some) CLI tools -- FFmpeg and Youtube-dl immediatly spring to my mind from having used them in the last few hours -- I wholeheartedly agree. Principles are good, intelligent discretion is even better.

Apart from those, I'd recommend anyone interested in CLI tools to start by checking the following projects:

• Busybox-w32 (https://frippery.org/busybox/)
• CMDer (https://cmder.net/)
• NirCMD (https://www.nirsoft.net/utils/nircmd.html)
• UnxTools (http://unxutils.sourceforge.net/)
• Yori (http://www.malsmith.net/yori/)

[vevy]

Good discussion. Here are my thoughts and responses (TP109, Specular, webfork, Andrew):
Also, Thanks for the links, TP109 and Midas.

• Windows CLI tools. Unix users have it made when it comes to CLI tools and repos.

• Separate Database. Rationale: Keep the purpose and scope of the main database untouched while also not be constrained by its exact paradigm. Databases with mixed CLI and GUI apps are already there on many bigger sites like Softpedia; so there is no need to repeat that.

• Less maintenance. CLI tools often reach maturity much faster than GUI tools due to their relative simplicity. Many (Most ?) CLI tools are not updated that often or at all because they still function well even after many years and OS versions.

• Effort is cumulative. Even if it becomes a "flash in the pan" effort, the work done is not lost and can serve a base for future improvements. Also many entries won't need updating as in the previous point.

• The audience is there and need is there. This site is known among "power users" and is well primed to unify the fragmented efforts like the sites mentioned before.

• Some sites and tools are going offline. f2ko.de is not accessible anymore and many good but unmaintained tools on google sites, private sites, etc won't be there forever. There is a need to collect them and provide archive.org links for example.

• There is a place for native, ported and platform-dependent (Cygwin, Java) tools. Dependencies should be mentioned when appropriate. This is already the case in the regular database (although I think most tools don't have real dependency issues.)

[vevy]

More thoughts:

• I envision a category for "Collection" CLI tools. Things like Cygwin, Busybox, Swiss File Knife, etc that provide many tools/function under one umbrella. Maybe even split them into "Platforms" like Cygwin, "Collections" like UnxUtils, GnuWin32, Busybox, Sysinternals and multi-function tools like Swiss File Knife and Yori(?)

• Maybe we should use tags more in the new database (like 2ry categories). For example, Xpdf may have tags like "convert", "pdf-to-text", "pdf-to-html". Such function tags would help the users find alternatives for a given task. This is a lot of work though. Still, lets lay the foundation since we are planning.

• Should we add apps that have a CLI component like Inkscape and Handbrake, or should we just refer to their regular database entry, or ignore them altogether?

[Specular]

• Less maintenance. CLI tools often reach maturity much faster than GUI tools due to their relative simplicity. Many (Most ?) CLI tools are not updated that often or at all because they still function well even after many years and OS versions.

I'd say it's a bit more of a mix of update frequency, not unlike GUI programs. There are definitely some that go for long stretches without updates though, particularly for niche use. Some of my most used CLI programs are updated very frequently: ffmpeg (updated daily), youtube-dl (frequently), ImageMagick (every few days).

That said I don't tend to update them myself as regularly as updates are available but it'd become a pretty continuous task for whoever was editing such entries.

One other thing is who would be voting here on such programs? As there are a variety of useful GUI programs that at one point or another didn't receive enough votes and I wonder how CLI programs would fare with this userbase. I know popular programs such as those listed so far would likely be voted upon but I wonder how others that may be popular among a certain audience but not necessarily here would be received. Not sure anyone could determine that beforehand necessarily though

• Should we add apps that have a CLI component like Inkscape and Handbrake, or should we just refer to their regular database entry, or ignore them altogether?

This is a good question. Depends how the entries are written maybe, whether it's just a very similar description of the GUI entry with a CLI screenshot or if entries are envisioned to have more about the CLI specific aspects, like perhaps brief mention about syntax or similar (or whether that aspect would be assumed the user would have to go and find in some documentation, which might only exist in a /help command).

[Midas]

Instead of imposing on Andrew Lee the burden of running yet another site for CLI tools, let me suggest here that someone with a little web savvy could setup a Github repo modeled after TPFC database for the effect.

E.g., Scoop (https://scoop.sh/) seems to have had some success with this approach...

[vevy]

I'd say it's a bit more of a mix of update frequency, not unlike GUI programs. There are definitely some that go for long stretches without updates though, particularly for niche use. Some of my most used CLI programs are updated very frequently: ffmpeg (updated daily), youtube-dl (almost daily), ImageMagick (every few days).

True. I have a couple of thoughts:

• There is a lot of value in the listing itself, regardless of keeping up with updates: to make the tool itself discoverable to the user.

• We can just follow stable updates (every few weeks/months for ffmpeg for example.

• We can simply link to the download page, rather than specific files, so that the user will always be getting the latest update. Again there is value in creating a database of the tools themselves rather than scrambling after version updates.
  
  This is one of the reasons I am suggesting a new database not bound by the exact rules of the main one.

One other thing is who would be voting here on such programs?

Well, it is up to the boss, but I would suggest easing up or removing this restriction at first while filling up the database. Let's say:

• new tools are added without needing votes.

• if an app gets two downvotes (because it doesn't work properly for example), it becomes private until further notice.