It is currently Fri Dec 15, 2017 1:32 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 23 posts ]  Go to page Previous  1, 2
Author Message
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Tue Apr 04, 2017 3:09 am 
Offline

Joined: Wed Jun 26, 2013 2:09 am
Posts: 413
SYSTEM wrote:
The dump script now produces invalid CSV. The double-quote character is missing at the end of the size field.
@SYSTEM: puppy's teething trouble...(nothing that a gentle touch of DB dusting wouldn't fix)...;-)

Nirsoft's handy CSVFileView to the rescue:

  • sorting on TPFC.CSV's new column shows exactly 17 records (out of 2,042) affected by the "missing quote" syndrome.
    (actually, there shouldn't be any quote at all in there)
  • sorting again on the new column shows a further 10 records include alphabetic strings instead of numeric value in the new field
  • those same 10 records also have an extraneous column appended...
  • one of those 10 records, namely #1665, even has two additional columns appended, for good measure...

I suspect those freshly uncovered anomalies should be easy enough to rectify, thereafter restoring our prize DB forever back to its pristine condition ?

Further suggestions to Andrew : (if not pushing my luck too far...)
    1. Perhaps a "Size" header for the new column would be appropriate icing on the cake ?
    2. Maybe the "Size" column could be defined as a numeric field entity, so as to preserve numbers formatting alignment ?

Thanks @SYSTEM for his pertinent observation, and of course @Andrew for a blindingly fast implementation ! 8)


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Tue Apr 04, 2017 11:52 pm 
Offline
User avatar

Joined: Sat Feb 04, 2006 9:19 am
Posts: 2158
__philippe wrote:
Nirsoft's handy CSVFileView to the rescue:

  • sorting on TPFC.CSV's new column shows exactly 17 records (out of 2,042) affected by the "missing quote" syndrome.
    (actually, there shouldn't be any quote at all in there)
  • sorting again on the new column shows a further 10 records include alphabetic strings instead of numeric value in the new field
  • those same 10 records also have an extraneous column appended...
  • one of those 10 records, namely #1665, even has two additional columns appended, for good measure...
All fixed, tested using CSVFileView. The missing end-quote on the size field, and the missing "Size" header were stupid errors on my part, attributed to over-confidence and lack-of-checking.

The quote-in-description problem has always been there but has never been pointed out. Easy to fix by escaping each quote with another quote.


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Wed Apr 05, 2017 1:05 am 
Offline
User avatar

Joined: Sat Jul 31, 2010 1:19 am
Posts: 1719
Location: Helsinki, Finland
Andrew Lee wrote:
__philippe wrote:
Nirsoft's handy CSVFileView to the rescue:

  • sorting on TPFC.CSV's new column shows exactly 17 records (out of 2,042) affected by the "missing quote" syndrome.
    (actually, there shouldn't be any quote at all in there)
  • sorting again on the new column shows a further 10 records include alphabetic strings instead of numeric value in the new field
  • those same 10 records also have an extraneous column appended...
  • one of those 10 records, namely #1665, even has two additional columns appended, for good measure...
All fixed, tested using CSVFileView. The missing end-quote on the size field, and the missing "Size" header were stupid errors on my part, attributed to over-confidence and lack-of-checking.

The quote-in-description problem has always been there but has never been pointed out. Easy to fix by escaping each quote with another quote.


Thank you. :)

_________________
My YouTube channel | Release date of my 11th playlist: January 26, 2018


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Wed Apr 05, 2017 2:42 am 
Offline

Joined: Wed Jun 26, 2013 2:09 am
Posts: 413
Andrew Lee wrote:
...All fixed, tested using CSVFileView.
The missing end-quote on the size field, and the missing "Size" header were stupid errors on my part, attributed to over-confidence and lack-of-checking.
The quote-in-description problem has always been there but has never been pointed out. Easy to fix by escaping each quote with another quote.
Andrew rules :!:

All fine and dandy now... :D

One nice feature about using CSVFileView is the sort-within-sort capability, for instance :

  • select "Categories" (click header) as a "Primary" sort column, and then,
  • select "Size" (shift+click header) as a "Secondary" sort column

thus displaying all Apps sorted by Size within a specific Category..., nifty !

Thanks again to Andrew for fixing the tpfc.csv file anomalies.

Cheers,

__philippe
_________________
PS
The DB dump file can also be imported into an Excel sheet, thus allowing to format the Size column entries as numeric values,
complete with right-aligned, comma-separated numbers, which, I gather, cannot be formatted right into the source tpfc.csv ?


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Wed Apr 05, 2017 3:10 pm 
Offline
User avatar

Joined: Sat Feb 04, 2006 9:19 am
Posts: 2158
Quote:
The DB dump file can also be imported into an Excel sheet, thus allowing to format the Size column entries as numeric values,
complete with right-aligned, comma-separated numbers, which, I gather, cannot be formatted right into the source tpfc.csv ?
AFAIK CSV is a text format and does not cater for formatting and alignment.


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Wed Apr 05, 2017 4:45 pm 
Offline

Joined: Wed Jun 26, 2013 2:09 am
Posts: 413
@Andrew,

And right you are, of course, I should have thought about it before posting drivel... :roll:

Anyway, it's just as well Nir Sofer does cater nicely for this kind of situation :
(From the CSVFileView help file) :
Quote:
"If the column values are numeric, CSVFileView automatically makes a numeric sorting instead of string sorting."
...
"Automatic Numeric Sorting:
When this option is turned on, CSFileView automatically uses numeric sorting when numeric values are detected".

Other CSV utilities (such as CSVedit) , powerful as they may be, are not so considerate in this specific case:
numeric contents or not, your columns get sorted as strings, willy-nilly.

I'll gladly stick with the tpfc.csv as it stands now, thank you very much...;-)


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Mon Apr 17, 2017 3:57 am 
Offline

Joined: Wed Jun 26, 2013 2:09 am
Posts: 413
__philippe wrote:
...
Anyway, it's just as well Nir Sofer does cater nicely for this kind of situation :
(From the CSVFileView help file) :
Quote:
"If the column values are numeric, CSVFileView automatically makes a numeric sorting instead of string sorting."
...
"Automatic Numeric Sorting:
When this option is turned on, CSFileView automatically uses numeric sorting when numeric values are detected".

Besides, it's just as well Nir does, occasionally, consider some (unidentified) user's trifling update suggestion ... ;-)
...and thereby quietly oblige :

CSVFileView v2.31 (released 16-APR-2017) - New in this version:
        
  • Added 'Align Numeric Columns To Right' option (under the Options menu).

Goody, goody... 8)


Top
 Profile  
 
 Post subject: Re: Sort Database by Uncompressed Size of App
PostPosted: Thu Jul 13, 2017 6:34 am 
Offline

Joined: Wed Jun 26, 2013 2:09 am
Posts: 413
Nirsoft relevant news, belatedly...

CSVFileView v2.32 (released 10-MAY-2017) - New in this version:

    Added 'Add Thousands Separator' option (Under the Options menu).


CSVFileView will now neatly display "Right-aligned AND Comma-separated" numeric values... 8)

Exeunt Excel behemoth and runner-up CSVed (right-alignment not supported)... :P

... thanks again to Nir...;-)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 23 posts ]  Go to page Previous  1, 2

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group