Sort Database by Uncompressed Size of App

All suggestions about TPFC should be posted here. Discussions about changes to TPFC will also be carried out here.
Message
Author
User avatar
__philippe
Posts: 498
Joined: Wed Jun 26, 2013 2:09 am

Re: Sort Database by Uncompressed Size of App

#16 Post by __philippe » Tue Apr 04, 2017 3:09 am

SYSTEM wrote:The dump script now produces invalid CSV. The double-quote character is missing at the end of the size field.
@SYSTEM: puppy's teething trouble...(nothing that a gentle touch of DB dusting wouldn't fix)...;-)

Nirsoft's handy CSVFileView to the rescue:
  • sorting on TPFC.CSV's new column shows exactly 17 records (out of 2,042) affected by the "missing quote" syndrome.
    (actually, there shouldn't be any quote at all in there)
  • sorting again on the new column shows a further 10 records include alphabetic strings instead of numeric value in the new field
  • those same 10 records also have an extraneous column appended...
  • one of those 10 records, namely #1665, even has two additional columns appended, for good measure...
I suspect those freshly uncovered anomalies should be easy enough to rectify, thereafter restoring our prize DB forever back to its pristine condition ?

Further suggestions to Andrew : (if not pushing my luck too far...)
  • 1. Perhaps a "Size" header for the new column would be appropriate icing on the cake ?
    2. Maybe the "Size" column could be defined as a numeric field entity, so as to preserve numbers formatting alignment ?


Thanks @SYSTEM for his pertinent observation, and of course @Andrew for a blindingly fast implementation ! 8)

User avatar
Andrew Lee
Posts: 2228
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Sort Database by Uncompressed Size of App

#17 Post by Andrew Lee » Tue Apr 04, 2017 11:52 pm

__philippe wrote:Nirsoft's handy CSVFileView to the rescue:
  • sorting on TPFC.CSV's new column shows exactly 17 records (out of 2,042) affected by the "missing quote" syndrome.
    (actually, there shouldn't be any quote at all in there)
  • sorting again on the new column shows a further 10 records include alphabetic strings instead of numeric value in the new field
  • those same 10 records also have an extraneous column appended...
  • one of those 10 records, namely #1665, even has two additional columns appended, for good measure...
All fixed, tested using CSVFileView. The missing end-quote on the size field, and the missing "Size" header were stupid errors on my part, attributed to over-confidence and lack-of-checking.

The quote-in-description problem has always been there but has never been pointed out. Easy to fix by escaping each quote with another quote.

User avatar
SYSTEM
Posts: 1780
Joined: Sat Jul 31, 2010 1:19 am
Location: Helsinki, Finland

Re: Sort Database by Uncompressed Size of App

#18 Post by SYSTEM » Wed Apr 05, 2017 1:05 am

Andrew Lee wrote:
__philippe wrote:Nirsoft's handy CSVFileView to the rescue:
  • sorting on TPFC.CSV's new column shows exactly 17 records (out of 2,042) affected by the "missing quote" syndrome.
    (actually, there shouldn't be any quote at all in there)
  • sorting again on the new column shows a further 10 records include alphabetic strings instead of numeric value in the new field
  • those same 10 records also have an extraneous column appended...
  • one of those 10 records, namely #1665, even has two additional columns appended, for good measure...
All fixed, tested using CSVFileView. The missing end-quote on the size field, and the missing "Size" header were stupid errors on my part, attributed to over-confidence and lack-of-checking.

The quote-in-description problem has always been there but has never been pointed out. Easy to fix by escaping each quote with another quote.
Thank you. :)
My YouTube channel | Release date of my 12th playlist: November 1, 2018

User avatar
__philippe
Posts: 498
Joined: Wed Jun 26, 2013 2:09 am

Re: Sort Database by Uncompressed Size of App

#19 Post by __philippe » Wed Apr 05, 2017 2:42 am

Andrew Lee wrote:...All fixed, tested using CSVFileView.
The missing end-quote on the size field, and the missing "Size" header were stupid errors on my part, attributed to over-confidence and lack-of-checking.
The quote-in-description problem has always been there but has never been pointed out. Easy to fix by escaping each quote with another quote.
Andrew rules :!:

All fine and dandy now... :D

One nice feature about using CSVFileView is the sort-within-sort capability, for instance :
  • select "Categories" (click header) as a "Primary" sort column, and then,
  • select "Size" (shift+click header) as a "Secondary" sort column
thus displaying all Apps sorted by Size within a specific Category..., nifty !

Thanks again to Andrew for fixing the tpfc.csv file anomalies.

Cheers,

__philippe
_________________
PS
The DB dump file can also be imported into an Excel sheet, thus allowing to format the Size column entries as numeric values,
complete with right-aligned, comma-separated numbers, which, I gather, cannot be formatted right into the source tpfc.csv ?

User avatar
Andrew Lee
Posts: 2228
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Sort Database by Uncompressed Size of App

#20 Post by Andrew Lee » Wed Apr 05, 2017 3:10 pm

The DB dump file can also be imported into an Excel sheet, thus allowing to format the Size column entries as numeric values,
complete with right-aligned, comma-separated numbers, which, I gather, cannot be formatted right into the source tpfc.csv ?
AFAIK CSV is a text format and does not cater for formatting and alignment.

User avatar
__philippe
Posts: 498
Joined: Wed Jun 26, 2013 2:09 am

Re: Sort Database by Uncompressed Size of App

#21 Post by __philippe » Wed Apr 05, 2017 4:45 pm

@Andrew,

And right you are, of course, I should have thought about it before posting drivel... :roll:

Anyway, it's just as well Nir Sofer does cater nicely for this kind of situation :
(From the CSVFileView help file) :
"If the column values are numeric, CSVFileView automatically makes a numeric sorting instead of string sorting."
...
"Automatic Numeric Sorting:
When this option is turned on, CSFileView automatically uses numeric sorting when numeric values are detected".
Other CSV utilities (such as CSVedit) , powerful as they may be, are not so considerate in this specific case:
numeric contents or not, your columns get sorted as strings, willy-nilly.

I'll gladly stick with the tpfc.csv as it stands now, thank you very much...;-)

User avatar
__philippe
Posts: 498
Joined: Wed Jun 26, 2013 2:09 am

Re: Sort Database by Uncompressed Size of App

#22 Post by __philippe » Mon Apr 17, 2017 3:57 am

__philippe wrote:...
Anyway, it's just as well Nir Sofer does cater nicely for this kind of situation :
(From the CSVFileView help file) :
"If the column values are numeric, CSVFileView automatically makes a numeric sorting instead of string sorting."
...
"Automatic Numeric Sorting:
When this option is turned on, CSFileView automatically uses numeric sorting when numeric values are detected".
Besides, it's just as well Nir does, occasionally, consider some (unidentified) user's trifling update suggestion ... ;-)
...and thereby quietly oblige :

CSVFileView v2.31 (released 16-APR-2017) - New in this version:
  • [*]Added 'Align Numeric Columns To Right' option (under the Options menu).

Goody, goody... 8)

User avatar
__philippe
Posts: 498
Joined: Wed Jun 26, 2013 2:09 am

Re: Sort Database by Uncompressed Size of App

#23 Post by __philippe » Thu Jul 13, 2017 6:34 am

Nirsoft relevant news, belatedly...

CSVFileView v2.32 (released 10-MAY-2017) - New in this version:

Added 'Add Thousands Separator' option (Under the Options menu).


CSVFileView will now neatly display "Right-aligned AND Comma-separated" numeric values... 8)

Exeunt Excel behemoth and runner-up CSVed (right-alignment not supported)... :P

... thanks again to Nir...;-)

Post Reply