Extract multiple files from USB HDD

[spicydog]

I have to extract certain files (only a handful of extensions) from a USB HDD that contains more than 2 million files (in all possible formats and nested in several layers of subfolders).

Can anybody suggest me a nice utility (irrelevant whether portable or not... Freeware would be better but it's not a must) to take care of this extremely time consuming task?

Ideal scenario: I set the parameters (the actual extensions that I really need), I run the tool and the morning after I find on my USB HDD only the few folders (named something like "doc & docx", "xls & xlsx", "pdf", "zip"... etc) that contain my files... Nothing else, no empty folders and no other files except these having the extensions that I indicated. Eventual filename conflicts to be solved with an appended progressive number (or custom text)... Error logging could be a plus (since I have the exact same copy on a second HDD from which I could eventually "pluck out" some accidentally deleted files/folders). I will take care of duplicates at a later stage but if this "wonder tool" could reliably do that I would obviously not mind

Thanks for any creative suggestion

[Midas]

The hardcore console way:

Put the code below in a text file an rename it to locate.bat (or .cmd). Move it somewhere on your %PATH% (%WINDIR% is the easiest choice). Open a windows console anywhere on the USB HDD and type "locate *.[extension]" and it will list all relevant files in the console -- append "> list.txt" and a list will be generated inside list.txt in the current folder...

Code: Select all

@echo off
pushd \
echo.
echo PLEASE STAND BY: FINDING INSTANCES OF %1 ...
echo.
dir %1 /b /s /p
popd

[guinness]

Or NirSoft has a search tool, but prefer the dir command too.

[spicydog]

Sorry friends but I have the feeling that I am somehow being misunderstood here

I have 2+ million files on a USB HDD and I only need the few hundreds .doc, .pdf, .xls (and a couple of other formats)... I do not need to make a list of files nor I need to search for files... I only want to nuke all but the few dozens files that I intend to keep.

I thought that probably something like DropIt would help me to sort files and place them into subfolders based on their extension but I fear that when I instruct DropIt to sort (and place into folders named after the extensions) so many files, my system will loose responsiveness and just crash.

I did a small test (on one of the 2 drives) and transferred all files in the 1st level (so on that HDD now I have all files in one single folder)... In such a scenario Windows Explorer needs some 15-20 minutes just to respond to a right-click!

Please confirm if the situation is now clear and THANKS already for your good intentions and support

[guinness]

Search on ss64.com for "ForFiles", you can create a pattern to delete certain filetypes. Did I understand now?

It's possible with dir and the list idea, just think outside the box e.g. loop through the list and use del. I personally would have created an AutoIt/VBScript, maybe it would be good practice.

[spicydog]

Search on ss64.com for "ForFiles", you can create a pattern to delete certain filetypes. Did I understand now?

It's possible with dir and the list idea, just think outside the box e.g. loop through the list and use del. I personally would have created an AutoIt/VBScript, maybe it would be good practice.

So you are saying that I should create a script that recursively deletes all files from my external HDD except certain specific file formats... This starts sounding more promising ... Unless I misunderstood you this time, lol.

[guinness]

Yes. In AutoIt search on the Forums for RecFileListToArray, this allows you to specify which file-types to exclude.

[joby_toss]

Use Everything?

[SYSTEM]

Hi spicydog,

I have written a Ruby script for you.

How to use:

1. You need a Ruby interpreter and libraries to run the script. So, go to http://rubyinstaller.org/downloads/ and download the 7-Zip archive for Ruby 1.9.3.
2. Extract the 7-Zip archive with your favorite extractor.
3. Download the script from https://docs.google.com/file/d/0B0P8wSx ... l3RnM/edit to the directory ruby-1.9.3-p374-i386-mingw32\bin.
4. Open command prompt and navigate to the directory where you just put the script.
5. Enter the command ruby copy.rb.
6. Answer the questions (source directory, target directory, list of extensions)
7. Sit back and watch...

For the programming minded, I have posted the script inline below.

Code: Select all

require 'fileutils'

FILE_NAME_SUFFIX_SEPARATOR = "-"

begin
  print "Enter the source directory: "
  source_directory = gets.chomp
end while not File.directory?(source_directory)

begin
  print "Enter the target directory: "
  target_directory = gets.chomp
end while not File.directory?(target_directory)

begin
  puts "Enter extensions of files you want to copy."
  puts "Separate them by spaces. Example: doc odt txt"
  extensions = gets.split
end while extensions == []

Dir.chdir(source_directory)
begin
  log = File.open(File.join(target_directory, "errors.log"), "w")
rescue Exception => e
  puts "Failed to create the log file!"
  puts "Reason: #{e.message}"
  exit
end

extensions.each do |extension|
  target_subdirectory = File.join(target_directory, extension)
  begin
    Dir.mkdir(target_subdirectory)
  rescue Exception => e
    log.puts "Failed to create the directory #{target_subdirectory}."
    log.puts "Reason: #{e.message}"
    next
  end
  Dir.glob(File.join("**", "*." + extension)) do |filename|
    i = 0
    target_filename = File.join(target_subdirectory, File.basename(filename))
    # In case a file with the same name exists already, find a name that is not used yet.
    while File.exists?(target_filename)
      filename_suffix = FILE_NAME_SUFFIX_SEPARATOR + i.to_s
      i += 1
      target_filename = File.join(target_subdirectory,
        File.basename(filename, "." + extension) + filename_suffix + "." + extension)
    end
    begin
      FileUtils.copy(filename, target_filename, :verbose => true)
    rescue Exception => e
      log.puts "Failed to copy the file #{filename}."
      log.puts "Reason: #{e.message}"
    end
  end
end

log.close

[spicydog]

Hey JT, nice of you to join

So you mean:

1. Run Everything (monitoring only the letter of that USB HDD) and let it create a database
2. Set the exclusions filter to the few formats that I need
3. Run a search for *.*
4. Delete all what Everything finds

Did I understand you correctly?

[joby_toss]

Perfectly!

[guinness]

Is this a code off SYSTEM? Name the time and the place

[SYSTEM]

Is this a code off SYSTEM? Name the time and the place

Please rephrase.

[spicydog]

Is this a code off SYSTEM? Name the time and the place

Please rephrase.

Hey SYSTEM I think that guinness just invited you to a duel, hehe.

Thanks a lot for the Ruby script you wrote... Please tell me if I understood correctly, in the way you suggest the selected files (the ones with the specified extensions) will be copied to a designated path either on the local or on the (same) external drive... So your script "plucks" out of the 2+ million files only the thousand (+/-) files that I need and copies them to a specific location, right?

I was wondering if the same cannot be achieved by a (simpler) batch script any comments/tips?

Kiitos

[spicydog]

Perfectly!

Yours sounds like the simplest solution so far but I was just wondering about one aspect.

If I am not wrong Everything relies on Windows Explorer's context menu, so how is that going to be more responsive than Windows Explorer* itself?

Thanks

*(which turns almost dead when handling so many files in one go)