CLI Database Discussions

Discuss anything related to command line tools here.
Message
Author
User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#196 Post by vevy » Sat Sep 05, 2020 7:52 am

Search doesn't work at all in the CLI database (with the exception of blank search).
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: CLI Database Discussions

#197 Post by Andrew Lee » Sat Sep 05, 2020 9:51 pm

vevy wrote:
Sat Sep 05, 2020 7:52 am
Search doesn't work at all in the CLI database (with the exception of blank search).
Search function has been fixed.

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#198 Post by vevy » Sun Sep 06, 2020 11:29 am

Thanks!

Also, the "tell the community" links still begin with cli. which gives a 500 error. The second link title should read "CLI Submissions" instead of "Portable Freeware Submissions".

Whatever else you can help with I would appreciate. If soon then great!

200
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: CLI Database Discussions

#199 Post by Andrew Lee » Sun Sep 06, 2020 5:21 pm

vevy wrote:
Sun Sep 06, 2020 11:29 am
Also, the "tell the community" links still begin with cli. which gives a 500 error. The second link title should read "CLI Submissions" instead of "Portable Freeware Submissions".
My bad. All fixed now.

User avatar
webfork
Posts: 9799
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: CLI Database Discussions

#200 Post by webfork » Mon Sep 07, 2020 8:42 am

Do we have a timeline for the CLI database going public? Maybe in some kind of "beta" / read-only status? Are we waiting for a specific feature or features to get sorted out?

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#201 Post by vevy » Mon Sep 07, 2020 9:51 am

There are some useful tweaks to the database that should make things easier to maintain.

My own mini-roadmap is:
  • Have the most pressing/helpful of these changes implemented. This depends on Andrew.
  • Finish the Unix port uploads (30-40% done).
  • Upload a batch of Windows-native CLI tools.
  • Upload some of each of the other types, like Windows-included CLI tools.
At this stage we have the basic components of the database in my view. We can then focus on feedback and decide on:
  • how to do curation. Which tools to show/promote/demote/hide etc.
  • any visual changes to the database.
  • what other features to add.
  • set up the use case system well enough that we can trial it and see if it gets fully implemented according to user feedback.
  • other useful but less essential features.
At this point I think we can go public as beta.

Then comes the long term part (months, years):
  • Keep adding other tools we have know of.
  • Correcting mistakes, improving design, filling missing fields.
When it becomes rarer to find other tools to add, and when the database feels mature and solid enough to use with no major omissions or glaring mistakes, we can leave beta.

Again, this is all just my own visualization of the process, not how it will necessarily be decided.

My earnest wish is to get through the first two stages (my own work and Andrew's) within the next couple of weeks if possible, so that I can snack on the next two long-term.

Edited: added a couple of sentences.
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#202 Post by vevy » Mon Sep 07, 2020 4:20 pm

I have a naming issue and I could use some help.

In this post (and preceding ones), we were trying to come up with a name for the Windows equivalent of a Linux "package". We decided on Collection.

The concept here was a group of tools that are distributed together (e.g. in a compressed archive) by the developer. For example, Sysinternals is a "collection" of tools. Likewise, coreutils is a "package" of Linux tools. Individual child tools are marked as "tool".


Why deal with this at all?
In the database, only package entries have information like "website" and "release date". Entries for child tools refer to their parent "package" entries for this information. This is to avoid redundancies and facilitate maintenance.

The issue I am trying to find a suitable solution to is one-tool "packages" like wget or txtproc. In Linux, they are still called packages by convention; no problem here. I can mark the entry as both "tool" and "package".

In Windows, I can't really mark an entry for a single tool a "collection", can I?

So, any suggestions for a better name than "collection" that can cover both cases (multiple included tools/one included tool)?
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
webfork
Posts: 9799
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: CLI Database Discussions

#203 Post by webfork » Mon Sep 07, 2020 9:01 pm

vevy wrote:
Mon Sep 07, 2020 9:51 am
My earnest wish is to get through the first two stages (my own work and Andrew's) within the next couple of weeks if possible, so that I can snack on the next two long-term.
Just looking at it over this weekend, there's already a lot of very useful, well-organized software information. You already have something put together that I don't see other sources addressing, including software cataloging sites and most CLI developers. Because of that, I recommend making it available sooner rather than later, but it's certainly your call.

I'll try to address the naming question in the days ahead if someone doesn't find a solution before me.
Last edited by webfork on Mon Sep 07, 2020 9:26 pm, edited 1 time in total.
Reason: (better wording)

User avatar
webfork
Posts: 9799
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: CLI Database Discussions

#204 Post by webfork » Tue Sep 08, 2020 8:19 pm

Quick question:

So apologies if I missed this in the discussion, but one of the CLI database entries (Git for Windows) https://cli.portablefreeware.com/index.php?id=4 is not portable by the standards of www.portablefreeware.com, as it writes to the userprofile directory. This is according to the entry, I haven't tested this. Are we listing non-portable CLI tools?

As this is actually the first example of a non-portable CLI tool that I've come across (excluding WSL which requires an install), I just wanted to check and see if everyone's on the same page about that.

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#205 Post by vevy » Wed Sep 09, 2020 9:08 am

webfork wrote:
Tue Sep 08, 2020 8:19 pm
Quick question:

So apologies if I missed this in the discussion, but one of the CLI database entries (Git for Windows) https://cli.portablefreeware.com/index.php?id=4 is not portable by the standards of www.portablefreeware.com, as it writes to the userprofile directory. This is according to the entry, I haven't tested this. Are we listing non-portable CLI tools?

As this is actually the first example of a non-portable CLI tool that I've come across (excluding WSL which requires an install), I just wanted to check and see if everyone's on the same page about that.
I'd add/keep it and clearly mark it as not portable. Being comprehensive has been the stated goal from the beginning. Provide all the tools for a job to the user and let them choose, because we don't have some site out there for all CLI tools we can refer to for the other non-portable ones.
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#206 Post by vevy » Wed Sep 09, 2020 12:13 pm

@Andrew

Based on an advice from a helpful member and in order to help you feel less overwhelmed by my requests :oops:, I went ahead and made a few-point list of the most useful requests at this point in the first post of this thread (CLI Database Discussions).

Feel free to ignore other requests from me and focus whatever effort you will volunteer on this list at any given time (i.e. as things are implemented, it should be be regularly updated with the next most relevant requests). :D
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: CLI Database Discussions

#207 Post by Andrew Lee » Wed Sep 09, 2020 5:30 pm

vevy wrote:For example, a release date of "{app=13}" means has the same release date of entry #13.
This method is the easiest to implement for display purposes, but one potential problem is it won't extend to filtering, because it is not something any SQL can handle.

Even for the other method suggested (parent id field, radiobutton to select), it will also be quite tricky to implement filtering in SQL (if you are familiar with SQL, try that mental exercise yourself!).

One possible solution is to isolate all parentable fields into their own separate tables, then make the field itself a reference to those tables. Then one can do a SQL join without having to deal with exceptions. But that would be a huge undertaking in terms of table restructuring.
vevy wrote:
  • IsWarningNet (Boolean) field (optional).
  • IsWarningDecompression (Boolean) field (optional).
  • IsWarningSecurity (Boolean) field (optional).
  • IsNotArchiver (Boolean) field (optional).
  • IsNotCompressor (Boolean) field (optional).
What are these new fields for?

User avatar
vevy
Posts: 752
Joined: Tue Sep 10, 2019 11:17 am

Re: CLI Database Discussions

#208 Post by vevy » Wed Sep 09, 2020 5:38 pm

Andrew Lee wrote:
Wed Sep 09, 2020 5:30 pm
This method is the easiest to implement for display purposes, but one potential problem is it won't extend to filtering, because it is not something any SQL can handle.

Even for the other method suggested (parent id field, radiobutton to select), it will also be quite tricky to implement filtering in SQL (if you are familiar with SQL, try that mental exercise yourself!).
Do you query the SQL database directly when a user uses the database, or use a regularly-updated static export?
vevy wrote:
  • IsWarningNet (Boolean) field (optional).
  • IsWarningDecompression (Boolean) field (optional).
  • IsWarningSecurity (Boolean) field (optional).
  • IsNotArchiver (Boolean) field (optional).
  • IsNotCompressor (Boolean) field (optional).
What are these new fields for?
IsWarningNet:
Be careful with network-facing tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).

IsWarningDecompression:
Be careful with decompression tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).

IsWarningSecurity:
Be careful with security tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).

IsNotArchiver:
This tool does NOT put multiple files into an archive. It just compresses a single file to reduce its size. You may want to group your files first using an archiver.

IsNotCompressor:
This tool only puts multiple files into an archive. It doesn't compress them. You may want to also use a compressor for that.
Help make the comprehensive CLI database happen:
                    Vote for filters/badges!

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: CLI Database Discussions

#209 Post by Andrew Lee » Wed Sep 09, 2020 5:43 pm

vevy wrote:
Wed Sep 09, 2020 5:38 pm
Do you query the SQL database directly when a user uses the database, or a regularly-updated static export?
Directly, since for filtering purposes, the filtering parameter is dynamic. Same with search queries.

But even if the output can be cached, some kind of SQL join needs to be fashioned, otherwise you are just brute-forcing the query.

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: CLI Database Discussions

#210 Post by Andrew Lee » Wed Sep 09, 2020 5:49 pm

vevy wrote: IsWarningNet:
Be careful with network-facing tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).

IsWarningDecompression:
Be careful with decompression tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).

IsWarningSecurity:
Be careful with security tools. Vulnerabilities are discovered from time to time. Try to keep such tools updated (including any third-party components or DLLs included with them).

IsNotArchiver:
This tool does NOT put multiple files into an archive. It just compresses a single file to reduce its size. You may want to group your files first using an archiver.

IsNotCompressor:
This tool only puts multiple files into an archive. It doesn't compress them. You may want to also use a compressor for that.
I don't get it. Why do we need these new fields?

Post Reply