CLI Database Discussions

[vevy]

[Mod note: access details for cli.portablefreeware.com can be found at the following post: viewtopic.php?p=95530#p95530]


NOTE: This post refers to and uses elements from CLI Discussions Resources.


Current priorities:

• Mini-requests
• #23: Developer field.
• #43 to #49: Additional statuses.

Mini-requests:

• If an icon file is chosen, there is no way to remove it during adding or editing. Same for screenshot? (Andrew: KIV)
• [_size] tag should also apply to line spacing. (Andrew: KIV)
• Links to fix (usually "cli." is unduly added to them): (Andrew: Done)
  • Edit entry: "Portable Freeware Update" Link.
  • DB search: "Forum topics matching your query"
  • Entry Comments: user profile links.
• Minor correction to span.use-case: padding: 0px 4px 2px 4px; (Andrew: Fixed)
• In edit page, "Beta releases should be posted to..." message should be removed for CLI. (Andrew: Fixed)
• In All use cases page, hover text (title) is just the first letter of the use case. (Andrew: Fixed)
• Use cases assignment isn't reflected in all tools. For example, click any of the use cases under curl or Wget. (Andrew: Fixed)

-------------------------------------------------------------------------
Priority: 1=min, 3=max, p=postponed

ID	Request	Description	Priority	Andrew
		General Requests
#01	Clearer edit logs	Currently, entry log history is diff-ed by characters, which makes figuring the changes out very difficult. Better do it by filed or paragraph.	2	KIV
#02	Adapt main page links to CLI.	Like "About", "Useful Links", etc	2p	Updated to point to main site.
#05	Promote the best tools.	Make a system for promoting the better tools (Editor's pick, voting, etc)	?	KIV. The star system (currently only activated for main site) is scored based on internal and external voting.
#06	Sorting by last edited	Sorting entries by last entry update date (as logged).	2	KIV
#79	Show number of tools in top bar.	Currently, I have to do an empty search to get how many tools are there.	1
		Search
#19	One-word synonyms	Handle one-word synonyms: like audio=sound≈music.	1p
#20	Handle aliases	Use cases like overlay video on another=picture in picture effect and formats like jpg=jpeg.	1-2p
#21	Partial matches.	"implement" should also match "implementation".	2-3p
#22	Search terms separately	i.e. not necessarily in order	2p
#81	Prioritize title matches	Tools with {search_term} in their title should be prioritized.	3
#82	Prioritize use case matches	Tools with {search_term} in their use cases should be prioritized.	2-3
#83	Prioritize recent tools	Recently-updated tools should be prioritized.	2
#66	Sources/Developers table	All the sources sifted for Windows CLI tools. These could be developer sites, Github pages, directories, etc. This allows us to know what was checked and when, and whether it needs to be checked regularly.	2
#67	Index Table	Could be dynamically linked to the database or static. CTRL+F for whether a tool exists in the database and its basic info.	1-2p
		Field Requests
	General
#50	Parent>Child detail inheritance	System to allow child entries to get details from parent tool (collection)
#64	Implement filters as text first.	Especially for maintenance filters. e.g. Instead of a checkbox for "Child tool", append "child:1" to search. mp3 license:foss child:1 extract:"." Easier to implement (?), less UI clutter.		KIV
#80	Reverse Boolean Filters.	e.g. os=off for tools that are NOT open source.	2
#54	Templates as unexpanded tags	Going with your suggestion about [warning] tags, and in the same spirit of unifying template wording and facilitating global changes, add templates in general as tags. e.g. [extraction_see_parent] in #53 and expand automatically. There are ways to expose the wording to the contributor if you wish to discuss.	2p
#18	Field incomplete message	If a filterable field is optional, show a message atop the "filter results page" that says: "Not all tools contain this type of information. Results may be incomplete."		KIV
	Title
#55	Inherited	Possible use: Automatically include parent name between brackets touch [Yori].	1p
#35	Software title: Suggestions	To avoid duplicate entries. Make suggestions clickable into a new tab.
	Categories
#10	Filterable	Useful with large categories when combined with other filters.	1-2	KIV
	Website URL
#56	Always inherited.
	Size
#12	Filterable	--	1	KIV
#57	Conditionally inherited	(if child field is empty)
	License
#07	Filterable	e.g. only FOSS results.
#58	Conditionally inherited
#78	Split license types into radio button groups	Public-domain/Open Source (PD, GPL, BSD, etc). Free. Conditionally Free (Free personal, etc) Limited Free (Freemium, Liteware/Demo, etc.) TBC	?
	Description
	Writes settings to
	How to extract
	Version
#68	"v" in Version field	Insert automatically in entry view?	1p
	Release date
#08	Filterable	For example, tools released on or after 2018-01-01	1-2	KIV
#61	Release date	Always inherited.
	Download URL
	Runs on
#13	Filterable	Postponed till such data is entered satisfactorily.	1p	KIV
#59	Conditionally inherited
	Icon
#72	Accept .ico		1
#60	Conditionally inherited			Done
#04	Stock icon for TUI tools.	Free for commercial. Only attribution.	1	KIV
	Screenshot
	Additional status
#36	Move Additional statuses	Next to the title. This should better help clarity and will save space.		Done
#14	Filterable	Tool, package, Win. package, discontinued, etc.	2	KIV
#17	Clickable badges	Make badges (additional status/warning) clickable (to a page filtered by these badges e.g. Win packages only). This will have the side benefit of making categories like "Collection" and "Unix Ports" unneeded.	1-2	KIV
#85	Split Badges from Additional status	into 3 groups: 1. Tool level 2. Tool type 3. Tool origin	2
	Tool level
#38	Collection
#37	Package
#40	Tool
#46	Subcommand (Internal Command)		2-3
	Tool type
#86	CLI	Assumed by default. No badge.	=
#43	CLI-mode GUI		2-3
#44	TUI		2-3
#45	Batch		2-3
#48	Alternative Shell		2-3
#49	Shell Extension		2-3
	Tool origin
#87	Native Windows	Assumed by default. No badge.	=
#41	Unix Port
#47	Windows-Included		2-3
	Additional Status
#88	Active	Assumed by default. No badge.	==
#42	Discontinued
#89	Discontinued		1-2
	Dependencies
#15	Filterable	Needs the field to have a Boolean option of Y/N first (like Stealth). Needs the information to be entered first.	1p	KIV
	Stealth
#16	Filterable	Needs the data first.	1p	KIV
	Suggested By
#69	Conditionally inherited		2
	Parent ID
#65	Suggest parent name	Upon entering parent ID (to confirm correct ID).	1-2
#63	Filter by IsChild	Useful for maintenance.		KIV
	Use Cases
#11	Filterable	--	2p	KIV
#84	Show number of tools between brackets	e.g. search text (15)	1-2
	Developer/Author
#73	Create (Optional)		2
#23	Suggestions	Show suggestion like "Title" field, but clicking a suggestion enters it into the field (not disabled like "Title").
#09	Filterable	All tools by a given developers. Useful for maintenance and usage.	1-2	KIV
#62	Conditionally inherited?		p (implement field first).
	Download Link (Archive)
#70	Create (Optional)	Archive.org or so in case dev site goes offline.	2
	Documentation Link
#24	Create (Optional)	--	1p
	Documentation Text
#25	Create (Optional)	Collapsed by default.	1p
	Warnings
#74	Create (Optional)	Boolean. As text or badge with hover text.
#75	Filterable
#26	IsWarningNet		1-2
#27	IsWarningSecurity		1-2
#28	IsWarningDecompression		1-2
#29	IsNotArchiver	Useful because of the difference in behavior with compression tools in GUI vs CLI.	1
#30	IsNotCompressor	Useful because of the difference in behavior with compression tools in GUI vs CLI.	1
	Blurb
#31	Create (Optional)	One-liner like GNU blurbs, Debian package desc.	1
	IsInIndex
#32	Create. Boolean, optional.	Added to the CLI Tool Index or not. Useful for managing the additions of tools.	1-2p
	Available in/Part of/Get it from/Parent Entry
#33	Create. Optional.	Still WIP. Enter delimited IDs of parent tools, e.g. 6,9,11 Will put a field with sentence: Available in UnxUtils, GnuWin, Gow.	2p
	Notes
#34	Create (Optional)	Edit box. Not shown in entry view.
	GUI Front-ends
#51	Create (Optional)	Edit box. Link or just name the front-ends available for a tool.	1-2
	Noob-friendliness
#76	Create (Optional)	3-level listbox (Easy, intermediate, Hard). Average votes.	1-2
	Changelog Link
#77	Create (Optional)	Many tools have buried changelogs (NEWS, CHANGES, CHANGELOG, Release Notes, etc)	1-2

------------------------------------------------------------------


Original Post:

Spoiler!   

Hi, everyone

I know Andrew is not big on CLI tools, but they are very useful. I keep discovering new tools and some of them are jaw-dropping in functionality and keep wondering: what else is there that I could be missing?

There are many sites that list CLI software among GUI software. Alternativeto.net has tags that help separate such tools, but their database doesn't have that many.

Also, some software authors have many capable CLI tools on their sites.

What I mean is, they are all-over the place and deserve a separate database.

Is there something like that out there? Eager to here your recommendations (and ideas)

[SYSTEM]

https://tinyapps.org/ isn't exclusively for command-line apps, but a large portion of their catalog is command-line.

[vevy]

Thanks, System. Nice site, but, by my quick look, it doesn't seem to have many tools (for example ptime, rxrepl, busybox, xpdf, etc) and old versions of others.

Any other suggestions are appreciated.

[billon]

https://www.robvanderwoude.com/

[vevy]

https://www.robvanderwoude.com/

Good find. It has some nice tools and references many other authors. It says though:

This list doesn't even attempt to be complete, as there are literally thousands and thousands of batch file utilities available on the web.

The limited space for the descriptions of these tools doesn't always do them justice, so check out the authors' web sites for more details.

I propose we make a (separate) database here for CLI tools. Current database functions as usual while the other serves as a comprehensive directory for all these good tools that risk going into obscurity.

Andrew, you just create a new database for us and we will populate it . Little work is needed on your part. I don't suppose that is too much to ask? Just link it at the main site's top bar (Next to Latest, Categories) and we will do the work.

What do our veterans think? Is it worth a shot?

[TP109]

A search for command line tools in the TPFC Forums brought up 208 pages of results, so command line tools have been an ongoing discussion for awhile. At one time, there was an entry in the FAQs titled, "Why don't you accept command-line apps?", but it's no longer listed on the FAQ page .

The thread below was dedicated to discussing command line tools in 2016.
TPFC Forums - Great CLI Programs

The page below has a list of where to find some command line tools.
https://cects.com/find-win32-cli-utilities/

I propose we make a (separate) database here for CLI tools. Current database functions as usual while the other serves as a comprehensive directory for all these good tools that risk going into obscurity.

I use command line tools on a daily basis and a database for command line tools would be interesting and useful, but I'm not sure if it would be feasible or appropriate for TPFC to take on that task. I recall that possibility being discussed before, but I haven't located the exact thread.

[Specular]

I think the idea of an index of CLI programs is an interesting one but I also wonder how that would gel with carrying over the existing nature of the PFDB, in that most programs, descriptions and instructions are written and intended for a more general user audience, while CLI tools each have a learning curve of their own and I wonder what such a database would look like if carrying over PFDB principles.

Even things as simple as entry descriptions for example over time are often edited to simplify the verbiage yet with CLI programs it necessitates a better understanding of the technical details of the program, any additional dependencies (some of these are moving targets when changes occur for example), 'easy to use' would also need be judged differently and generally users be expected to be more familiar with command-line use and its limitations (which can trip up even experienced users, in areas like path escaping).

I think PFDB has a pretty clued-in, poweruser-leaning audience based on comments and thread postings but I'd be curious how entries and expectations would be handled.

Edit: and I'd also add that it's uncertain how many would be willing to maintain updates to such a DB, given for the audience we have it's a more niche area and keeping track of many updates is something that tends to be picked up by a smaller handful of users and sporadic contributors. Proposing something is obviously simpler than maintaining it.

[webfork]

What do our veterans think? Is it worth a shot?

Some input off the top of my head: the truth is that most CLI tools are portable, so it could make sense to include here. There's some amazing CLI software that could definitely benefit from some time and attention. Still, there are some concerns:

• Advocates - Ultimately a lot of good software arrived because it had a champion. I can definitely get behind great CLI tools like 7zip's command-line tools, SFK, and PDFBox. All those are amazing programs, but I don't really have a toolset beyond that.

• Maintenance - We have a group that's been here for a long time that helps keep entries up to date, works on descriptions, and generally tries to make sure that we find mirrors for dead software. A software listing without a fairly dedicated crew probably will be a flash in the pan.

• Platforms - Does this include non-Windows CLI tools?

• Dependencies - CLI software does have the same problems that regular software has, including dependencies (cygwin and Java) and system configuration (PowerShell vs. CMD) and all of that would definitely need to be part of any comprehensive site.

Whatever the case, it's a great idea I hope someone can execute on.

[Andrew Lee]

At one time, there was an entry in the FAQs titled, "Why don't you accept command-line apps?", but it's no longer listed on the FAQ page.

As the database evolved into being managed by the community (via voting), at one point I think we decided that the good "taste" of the community will take care of this.

I remember we used to have a lot of debates not only about CLI, but also .Net, Java, etc apps (Electron-based apps were still a rarity then). Now there are a sprinkle of such apps in the database on a case-by-case basis.

I kinda like this more than the black-and-white approach, much like how I didn't agree with the hardline view that only stealth will do for portability. Life is more interesting this way.

Personally, I still think we should have a damn good reason for including a CLI app, since these are mostly portable by default, with very few exceptions (and those exceptions are IMHO plain evil ).

[Midas]

I kinda like this more than the black-and-white approach, much like how I didn't agree with the hardline view that only stealth will do for portability. Life is more interesting this way.

I see what you have done there... And although I am a fan of (some) CLI tools -- FFmpeg and Youtube-dl immediatly spring to my mind from having used them in the last few hours -- I wholeheartedly agree. Principles are good, intelligent discretion is even better.

Apart from those, I'd recommend anyone interested in CLI tools to start by checking the following projects:

• Busybox-w32 (https://frippery.org/busybox/)
• CMDer (https://cmder.net/)
• NirCMD (https://www.nirsoft.net/utils/nircmd.html)
• UnxTools (http://unxutils.sourceforge.net/)
• Yori (http://www.malsmith.net/yori/)

[vevy]

Good discussion. Here are my thoughts and responses (TP109, Specular, webfork, Andrew):
Also, Thanks for the links, TP109 and Midas.

• Windows CLI tools. Unix users have it made when it comes to CLI tools and repos.

• Separate Database. Rationale: Keep the purpose and scope of the main database untouched while also not be constrained by its exact paradigm. Databases with mixed CLI and GUI apps are already there on many bigger sites like Softpedia; so there is no need to repeat that.

• Less maintenance. CLI tools often reach maturity much faster than GUI tools due to their relative simplicity. Many (Most ?) CLI tools are not updated that often or at all because they still function well even after many years and OS versions.

• Effort is cumulative. Even if it becomes a "flash in the pan" effort, the work done is not lost and can serve a base for future improvements. Also many entries won't need updating as in the previous point.

• The audience is there and need is there. This site is known among "power users" and is well primed to unify the fragmented efforts like the sites mentioned before.

• Some sites and tools are going offline. f2ko.de is not accessible anymore and many good but unmaintained tools on google sites, private sites, etc won't be there forever. There is a need to collect them and provide archive.org links for example.

• There is a place for native, ported and platform-dependent (Cygwin, Java) tools. Dependencies should be mentioned when appropriate. This is already the case in the regular database (although I think most tools don't have real dependency issues.)

[vevy]

More thoughts:

• I envision a category for "Collection" CLI tools. Things like Cygwin, Busybox, Swiss File Knife, etc that provide many tools/function under one umbrella. Maybe even split them into "Platforms" like Cygwin, "Collections" like UnxUtils, GnuWin32, Busybox, Sysinternals and multi-function tools like Swiss File Knife and Yori(?)

• Maybe we should use tags more in the new database (like 2ry categories). For example, Xpdf may have tags like "convert", "pdf-to-text", "pdf-to-html". Such function tags would help the users find alternatives for a given task. This is a lot of work though. Still, lets lay the foundation since we are planning.

• Should we add apps that have a CLI component like Inkscape and Handbrake, or should we just refer to their regular database entry, or ignore them altogether?

[Specular]

• Less maintenance. CLI tools often reach maturity much faster than GUI tools due to their relative simplicity. Many (Most ?) CLI tools are not updated that often or at all because they still function well even after many years and OS versions.

I'd say it's a bit more of a mix of update frequency, not unlike GUI programs. There are definitely some that go for long stretches without updates though, particularly for niche use. Some of my most used CLI programs are updated very frequently: ffmpeg (updated daily), youtube-dl (frequently), ImageMagick (every few days).

That said I don't tend to update them myself as regularly as updates are available but it'd become a pretty continuous task for whoever was editing such entries.

One other thing is who would be voting here on such programs? As there are a variety of useful GUI programs that at one point or another didn't receive enough votes and I wonder how CLI programs would fare with this userbase. I know popular programs such as those listed so far would likely be voted upon but I wonder how others that may be popular among a certain audience but not necessarily here would be received. Not sure anyone could determine that beforehand necessarily though

• Should we add apps that have a CLI component like Inkscape and Handbrake, or should we just refer to their regular database entry, or ignore them altogether?

This is a good question. Depends how the entries are written maybe, whether it's just a very similar description of the GUI entry with a CLI screenshot or if entries are envisioned to have more about the CLI specific aspects, like perhaps brief mention about syntax or similar (or whether that aspect would be assumed the user would have to go and find in some documentation, which might only exist in a /help command).

[Midas]

Instead of imposing on Andrew Lee the burden of running yet another site for CLI tools, let me suggest here that someone with a little web savvy could setup a Github repo modeled after TPFC database for the effect.

E.g., Scoop (https://scoop.sh/) seems to have had some success with this approach...

[vevy]

I'd say it's a bit more of a mix of update frequency, not unlike GUI programs. There are definitely some that go for long stretches without updates though, particularly for niche use. Some of my most used CLI programs are updated very frequently: ffmpeg (updated daily), youtube-dl (almost daily), ImageMagick (every few days).

True. I have a couple of thoughts:

• There is a lot of value in the listing itself, regardless of keeping up with updates: to make the tool itself discoverable to the user.

• We can just follow stable updates (every few weeks/months for ffmpeg for example.

• We can simply link to the download page, rather than specific files, so that the user will always be getting the latest update. Again there is value in creating a database of the tools themselves rather than scrambling after version updates.
  
  This is one of the reasons I am suggesting a new database not bound by the exact rules of the main one.

One other thing is who would be voting here on such programs?

Well, it is up to the boss, but I would suggest easing up or removing this restriction at first while filling up the database. Let's say:

• new tools are added without needing votes.

• if an app gets two downvotes (because it doesn't work properly for example), it becomes private until further notice.