If you want to detect when a file is created, modified, or deleted, you can use the FileSystemWatcher class in .NET. However, this is only available when the application is running. What if you want to detect changes that occurred while the application was not running? The USN Journal is a feature of the NTFS and ReFS file systems that keeps track of changes to files and directories. It allows applications to quickly identify changes to the file system.
The USN journal contains 3 kinds of records:
- Record v2 and v3: These records are used to track changes to files and directories (creation, deletion, modification, etc.).
- Record v4: These records tracks modified ranges of a file (offset and length). These records are not enabled by default. You can enable them using
fsutil usn enablerangetracking. Also, the actual range may be smaller than the one reported in the record (depends on the options set when enabling range tracking).
Each entry contains a Unique Sequence Number (USN), which is a unique identifier for the change. The USN is incremented each time a change is made to a file or directory. You can can store the last USN you processed and use it to read only the changes that occurred since that time.
Each entry also contains a file reference number, which is a unique identifier for the file or directory that was changed. This is a 64 or 128 bits number that uniquely identifies the file or directory on the volume. You can use this number to identify the file or directory that was changed. Note that if you delete a file and create a new file with the same name, the file reference number will be different.
Note that the USN Journal has a limited size. When the journal reaches its maximum size, the oldest records are deleted to make room for new ones. This means that you may miss changes if the journal is full. By default, the size on the system drive is about 32MB. With all the writes in the temp folder or AppData, you can quickly fill the journal. You can increase the size of the journal if needed (may require admin permissions).
Most operations on the USN Journal require administrative permissions. However, you can open the journal in unprivileged mode to read the journal. In this mode, you can only read the journal and not make any changes to it. Also, the file names may not be available in unprivileged mode. You can still use the file reference number to identify the file or directory that was changed.
To read the USN journal in .NET you can use the Meziantou.Framework.Win32.ChangeJournal NuGet package. This package provides a managed API to read the USN journal. Here's an example of how to use it:
C#
using Meziantou.Framework.Win32;
var drive = new DriveInfo("D:");
using var changeJournal = ChangeJournal.Open(drive, unprivileged: true);
var filePath = @"D:\test.txt";
File.WriteAllText(filePath, "Hello, World!");
var entry = ChangeJournal.GetEntry(filePath);
// entry.UniqueSequenceNumber
// entry.FileReferenceNumber
// Update the file
File.WriteAllText(filePath, "Hello, World! 2");
// Enumerate all entries from the last change
var entries = changeJournal.GetEntries(entry.UniqueSequenceNumber, ChangeReason.All, returnOnlyOnClose: false, TimeSpan.FromSeconds(10));
foreach (var entry in entries)
{
if (entry is ChangeJournalEntryVersion2or3 entry2or3)
{
Console.WriteLine($"{entry2or3.UniqueSequenceNumber}; version: {entry.Version}; file id: {entry2or3.ReferenceNumber:X8}; reason: {entry2or3.Reason}; name: {entry2or3.Name}");
}
else if (entry is ChangeJournalEntryVersion4 entry4)
{
Console.WriteLine($"{entry4.UniqueSequenceNumber}; version: {entry.Version}; file id: {entry4.ReferenceNumber:X8}; reason: {entry4.Reason}; remaining: {entry4.RemainingExtents}");
foreach (var extent in entry4.Extents)
{
Console.WriteLine(" - " + extent);
}
}
}
// Enumerable all entries from the beginning
var entries = changeJournal.GetEntries(ChangeReason.All, returnOnlyOnClose: false, TimeSpan.FromSeconds(10));
{
// TODO
}
The library also let you control the change journal (may require administrative permissions). For instance, you can increase the size of the journal by deleting it and recreating it with the desired size. You can also enable range tracking to track which part of a file was changed. Note that you cannot disable this settings once enabled. You will need to delete the journal and recreate it.
C#
changeJournal.Create(maximumSize: 10_000_000, allocationDelta: 1_000_000);
changeJournal.EnableTrackModifiedRanges(chunkSize: 10_000, fileSizeThreshold: 100_000_000);
changeJournal.Delete();
#Additional resources
Do you have a question or a suggestion about this post? Contact me!
