Beta Testing for Ransomware Detection in File Share

I could use some beta testers for an application I created.  This application/windows service detects ransomware on a Windows File Server or Windows Share.  Please comment to this post on any bugs, issues, or successful use of the program.

Issue:  

When staff members get ransomware, you need to respond quickly to get their computer shutdown as soon as possible.  If you respond quickly enough, you can shut down the offending computer before other file shares become encrypted.  Anti-virus programs currently do not detect encrypted files written by ransomware.  Not knowing that a ransomware virus is on your network is a big problem.  The sooner you get the offending computer shutdown and restore your backups of files shares the better.  If you do not notice an encrypted file share, you can lose your opportunity to restore from backup or cause your users to use a much older backup than necessary.  Anti-virus programs are always a few days behind in detecting new viruses.

Files screens will detect files new files modified or created by old ransomware, but how do you find ransomware files that already exist? You got hit with ransomware but you did not have the Compare tab setup in time. How do you detect where the new ransomware hit your file shares? How do you find ransomware files with folder or file names with long path names?  How do you determine which files are corrupted/encrypted? How do you detect a ransomware that does not create a ransom note in the file share or modify the file name?

The difficulty in detecting ransomware comes from the many different types of ransomware and changes caused by ransomware.  Some ransomware change files extensions, some don’t change the file extensions, all create ransom notes but with different file names, some maintain the same file size of encrypted files after encryption, and some maintain the file properties of files encrypted.  This makes detection difficult for older ransomware and nearly impossible for future ransomware.

  • How do I monitor my windows file shares for ransomware with minimal performance impact? (Compare tab and a few example files in the SourcePath)
  • How do I detect a ransomware that does not create a ransom note in the file share or modify the file names in the share? (Compare tab)
  • How do I automatically stop an infection from encrypting more files and only stop the user that was infected? (Compare tab – CommandProgram and the StopRansomwareInfectedUserPublic.ps1 script)
  • What files and how many files are corrupted in my windows file shares? (Audit Files tab)
  • What files are still good in my windows file shares? (Audit Files tab)
  • What files have been recently changed or created since that last good backup? (Audit Files tab, or Compare tab for full binary comparison)
  • How do I detect encrypted or corrupted zip files, word documents, excel files, or powerpoint files? (Audit Files tab – ValidateZipFiles option)
  • What files and how many were repeatedly created by the virus? (Find Ransom Files tab)
  • How do I delete the ransom note files created by the virus? (Find Ransom Files tab and Find Filters tab)
  • How do I replace the corrupted files and keep the newest good files? (Audit Files tab)
  • How do I detect ransomware no matter what type of ransom files it creates? (Compare tab)
  • How do I quickly stop the Windows file server from sharing files during a virus outbreak? (“Stop File Sharing” button)
  • How do I restore files when long file paths are involved? (Audit Files tab, or FastCopy)
  • How do I find out what files have file permissions corrupted or files that are inaccessible? (Audit Files tab – ExportUnknownToCSV)
  • What files were created or modified when compared to a previous backup? (Audit Files tab or Compare tab for full comparison)

Solution:

https://github.com/prestoncooper/RansomwareDetectionService/

Monitor file shares for encrypted and renamed files using a source folder of example files that you want to monitor copied into each persons home/documents folder or file share.  This service will detect ransomware via entrapment.  The ransomware will mistakenly encrypt a few sample files that we are monitoring for changes.  This program can monitor many local drive paths and UNC paths.  If you use the example files detection method the program runs very quickly because it only checks a few files per main folder and immediate sub folders.

There are options if you want the program to copy the files for you into the main folder or the immediate sub folders.  If you leave these options checked only one error will be logged/emailed and the files will be recopied if they do not exist.  Changed files will continue to log an error/email until restored or the files are replaced manually.

This will detect all current and future ransomware affected files on any Windows file share.

The “Find Ransomware Files” tab after the fact detection of ransomware created files.  This will search the specified directories for all file filters specified and email the list of the files found.  This should not be run during business hours because large file shares can take a long time. I just added the ability to delete any ransomware created files per file filter for cleanup purposes.

Audit Files tab will traverse a directory, compare file signatures for expected file extensions, and create a verified files list, unverified files list (possible corrupted/encrypted files), unknown files list, and prohibited files list. This helps to determine the damage caused by a ransomware. The lists will aide the restore of encrypted/corrupted files. The FixUnverifiedFilesFromBackup option will fix corrupted files by replacing the bad files from the restored backup.

Source Code (Transparency):

I copied the source code into the github project https://github.com/prestoncooper/RansomwareDetectionService/ .  If you want to review the code and compile it yourself (for transparency and if you want to make sure the program is safe before testing. I used BSD License for the code)

The code is multi-threaded and now supports long path names as well.

I just finished alpha testing and could use some more testing.  To test all you have to do is change or rename a file in the FilePathToCheck that the SourcePath copied over in a previous run of the service with CopySourceFiles or CopySourceFilesSubFolders checked.  Add several paths to check as separate rows in the table.

To monitor documents share get the main path and check mark CheckSubFolders and CopySourceFilesSubFolders.  The first run will error from the missing files, but subsequent runs will be successful until there is a change to a file or a missing file.

Ransomware Detection Service (Compare tab)  

 

Ransomware Detection Overview

Options for detection:

Ransomware Detection Options


Compare (Monitor File Shares and Detect Ransomware or files that go missing)

This will copy source files into the file path to check and then on a schedule check to see if the source files have changed or went missing.  Alternatively, use compare to verify contents of a file share and detect changes to critical files. There are two ways to test for ransomware.  First, create a folder in the SourcePath with a few small files with files of the type that you are concerned (XLS, XLSX, DOC, DOCX, PDF, JPG, PNG).  Copy this directory to each folder that you want to monitor or use CopySourceFiles or CopySourceFilesSubFolders options in order to copy the SourcePath files (only needs to run with these options once, uncheck after first run).  If these files change or get encrypted then you will receive an error in the error log and possibly an email if setup.  Secondly you could put a copy of important files into the SourcePath and have it monitoring for changes (This will take longer but you will know when important files are changed).  The files are compared by binary differences and if a file goes missing an error is returned as well.

Scheduling Options (Interval Type):

  • Hourly:  Enter start time in military time,  end time in military time, select hourly interval type,  and enter “interval” in minutes.
  • Daily:  Set a start time in military format,  select days and months you want to run and it will run at that time.
  • Monthy:  Set Interval to 1-31 to run on a specific day of the month,  specify -1 up to -5 and select a day to set the nth day of the month (e.g. -1 Mon would run on the 1st Monday of the month.)

 

 

Find Ransomware Files  (Search for Ransomware created files)

The “Find Ransomware Files” tab searches all the specified directories for the ransomware file filters that you specify in the “Ransomware File Filters” tab.  This solves the following two problems.

  • Files screens will detect files new files modified or created by old ransomware, but how do you find ransomware files that already exist. How do you detect where the new ransomware hit your file shares? How do you find ransomware files with folder or file names with long path names?

 Find Ransomware Files Options:

  • FilePathToCheck:  This is the file share that you want to monitor for ransomware or monitor the files for changes
  • CheckSubFolders: Recursively check all the sub folders of FilePathToCheck.
  • SendEmailOnFailure: Sends summary email when files are changed or if files are missing each time the directory is compared.
  • SendEmailOnSuccess: Sends summary email notifying you that the file path was checked.
  • ExcludedFolders:  Excludes list of folders separated by semicolon from FilePathToCheck. Any folder matching the exact name will be excluded.
  • Ransomware File Filters (tab)
    • Enabled:   Search for this FileFilter
    • Title:  Name of ransomware to find or description of search
    • FileFilter:  Enter in file filter search expected by windows (e.g. *recover*.txt, HELP_RESTORE_FILES.txt, or *.ecc)
    • DeleteFilesFound: Delete all files found by the file filter. (Only check mark this after you have verified the files you want to delete by a previous run and no false positives will be deleted.  Uncheck this after it has run once.  I recommend using a very specific file filter with this option.)
    • Comment: a comment regarding the file filter

 

 

Audit Files (Search for Ransomware Affected Files)

If a ransomware changes files in your windows file shares it is important to know the extent of the damage caused by the virus. The “Audit Files” tab will search specified directories and compare the file header/signature vs known file headers for the file extension. If a compared file does not match the header it is outputted into the UnverifiedFiles.csv file. If a file extension is not known or an error occurs then the file is outputted into the UnknownFiles.csv file. Files that match the known file header/signature will output into the VerifiedFiles.csv file. If a signature if flagged as prohibited then the file will be listed in the file ProhibitedFiles.csv file as well as the file VerifiedFiles.csv. Custom file signatures can be added later in the Audit Signatures tab (This overrrides the stock signatures and all signatures have to be added). If the Audit Signatures table rows are deleted entirely then the stock signatures are used.
Audit Files Options:

Audit Files Options:

  • FilePathToCheck: This is the file share that you want to monitor for ransomware or monitor the files for changes
  • CheckSubFolders: Recursively check all the sub folders of FilePathToCheck.
  • ExcludedFolders: Excludes list of folders separated by semicolon from FilePathToCheck. Any folder matching the exact name will be excluded.
  • ExportCSVPath: The path where the csv files will be saved (UnknownFiles.csv, UnVerifiedFiles.csv, and VerifiedFiles.csv)
  • ValidateZipFiles: Any file starting with zip file header will be test extracted to confirm that the file is not corrupted or encrypted. (zip, docx, xlsx, pptx, xps, oxps, epub, etc are all tested)
  • ExportUnVerifiedToCSV: Saves unverified (Possible ransomware affected files) to csv file
  • ExportVerifiedToCSV: Saves file header verified list of files to csv file. (Prohibited files will also be in this list if the signature matches the file extension)
  • ExportUnknownToCSV: Saves unknown (extension is unknown or error on reading the file) list of files to csv file.
  • ExportProhibitedToCSV: If any signatures and extensions are flagged as prohibited then they will be added to the prohibited csv file.
  • ProhibitedFilesIgnoreFileExtension: If a file signature is flagged as prohibited, then this option if checked will detect files even when file extension has been changed to hide the file.
  • FixUnverifiedFilesFromBackup: Replace unverified files with files from restored backup. Make sure to run a full backup of FilePathToCheck folder before using this option. Run audit of FilePathToCheck folder with FixUnverifiedFilesFromBackup unchecked at least once and fix as many unknown files as possible to yield a better result. This makes the process of leaving good files alone and replacing corrupted/encrypted files with a backup file a lot easier.
  • RestoredFilesPath: If FixUnverifiedFilesFromBackup is checked then this is the Path to restored backup of FilePathToCheck. These files must have the same folder structure as FilePathToCheck and will overwrite any unverified files.
  • SendEmailOnFailure: Sends summary email of files that were possibly affected by ransomware.
  • SendEmailOnSuccess: Sends summary email notifying you that the file path was audited.
  • Audit Signatures tab (If no signatures are listed then the stock signatures are used)
    • Enabled: Whether signature check is enabled
    • ByteOffset: Number of bytes to ignore previous to the Hexadecimal Pattern.
    • FirstNumberOfBytesToRead: Number of bytes to read from the file to compare with the HexPattern. (0 will default to 100 or HexPattern.Length + ByteOffset whichever is greater)
    • HexPattern: The hexadecimal pattern to find within the first 100 bytes of a file.
      SignaturesName: The file type title or signature name
    • FileExtensions: a semicolon separated list of file extensions to match with the signatures include the period with each file extension e.g. .doc;.docx;xls;xlsx
    • Prohibited: If prohibited then any file with the extension and signature will be added to the prohibited list. The file will also be listed in the verified list if signature hexadecimal pattern matches the file extension.
    • Comment: A comment for the signature.

Installation:

  1. Download both Installer Files (setup.exe and RansomwareDetectionServiceInstaller.msi) into the same directory and run setup.exe as administrator https://github.com/prestoncooper/RansomwareDetectionService/blob/master/RansomwareDetectionService2.0.4.6.zip
  2. Run the installation setup.exe downloaded from step 1 (Username for the service will be requested before installing the Windows service (username must to be in “username@domain” or “domain\user” or “computername\user” format.)

 

Caveat:

  • Train or notify users to not delete the files/folders that get copied from the SourcePath.  Deleted files will cause a false positive missing files error message or email.
  • If you are using the important files method then you will receive error messages for all changed files (even when changed normally).
  • “Find Ransomware Files” tab and Audit Files tab for large directories with many files will be slow and should be run during off hours. Compare (Detect Ransomware) is fast and can be run during business hours.

 

Compare Beta Test:

  • Create some sample files in a folder in the SourcePath (pdf, xls, xlsx, doc, docx, txt, etc.)
  • Specify the email settings
  • Test send an email (File -> Test Send Email)
  • Specify the copy options or copy the sourcepath files manually if desired.
  • Monitor some files shares
  • Schedule the compare to run on a schedule (have it run once with copy options or manually copy files before testing)
  • Delete a few of the sample files in the file share after it ran once (verify email was sent if option was checked and error logged)
  • Modify a few of the sample files in the file share after it ran once. (verify email was sent if option was checked and error logged)
  • Test some local paths and unc paths
  • Test some long paths (longer that 1000 characters deep)

Ransomware Find Files Beta Test:

  • Specify the email settings
  • Test send an email (File -> Test Send Email)
  • Specify the any additional file filters
  • Create a text file with a file filter that was specified in the file share you are monitoring.
  • Monitor some file shares and schedule it to run
  • Verify email if checked was sent and error was logged.

If you have any strange errors or if your tests succeed let me know by posting a reply to this blog post.

 

Source Code Compilation:

  • Visual Studio 2010 click on the main solution and click on Build Solution or Rebuild Solution.  The installer project will have the msi and setup.exe in the debug folder. You might have to point some of the static files to their new locations in the setup project.

 

Alternative Resources:

File Screens to Monitor File Shares for Ransomware

15 thoughts on “Beta Testing for Ransomware Detection in File Share

  1. Preston Post author

    I just detected a ransomware in a fileshare that my file screen would not have caught. So awesome that i was able to respond quickly and know about the infection.

  2. Preston Post author

    I added “Find Ransomware Files” tab to address finding new ransomware files that a file screen was not setup to detect. This makes cleanup and searching your network for a new ransomware recently detected easier.

  3. Preston Post author

    I recommend using Files Screens and this Ransomware Detection service be used in conjunction. “Find Ransomware Files” tab is too slow to run during business hours so Files Screens fill the gap.

  4. Pingback: How To Prevent Ransomware Infections | Question Driven

    1. Preston Post author

      The username and password is for an account on your domain or a local computer account. This account is used as the account the windows service runs under. Read the pdf documentation or online documentation for more information.

  5. Preston Post author

    The Ransomware Detection Service now during auditing will overwrite corrupted files with restored backup files. I also successfully detected an unknown ransomware in my file shares yesterday.

  6. Dan

    Hey – thanks for creating this. I’m happy to help with testing. I’ve just successfully setup mail and running through 1st run of the ‘Compare’ function with the default ‘Test Home Directory Only’ settings. I’ve noticed that although the alloted hourly interval has passed, and the ‘CopySourceFiles’ option is ticked, the C:\temp folder and files have not been autocreated. Service is running as domain admin user, and the paths have not been altered from default.

  7. Dan

    Made some chages for test purposes and set source to c:\source and ‘path to check’ to c:\temp. Created folders and files manually and amended one of the files in ‘path to check’ to see if the file change was spotted. It was, and I have the email with details now. The log subfolder (c:\temp\exportCSV) folder and log file did not autocreate. I will create the folder and see if the log file is there on the next pass.

  8. Dan

    Creating the ExportCSV folder manually worked. csv files autocreated and populated on latest pass.

  9. Preston Post author

    Thank you to the people that tested. I changed the current version to stable. It has been working like a charm for me for over 6 months. Over that time I have caught three different ransomware. Two of them didn’t create ransom notes, and one didn’t change file names or file sizes. All of them were detected early and I was able to do restores quickly. I have had a few false positives from users deleting the detection folder, but that is a training issue and not a bug.

  10. Pingback: Ransomware Detection Service Released | Question Driven

Leave a Reply

%d bloggers like this: