birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[AssetProcessor] Refactor the FileWatcher to use only one watch thread

Browse Source 
This change reworks the AssetProcessor's FileWatcher so that it only uses
one thread. This is motivated by getting better support for inotify on
Linux. The previous architecture required calling `inotify_init` once for
each directory that was being watched, and using separate inotify instances
for each watched tree. In addition, having separate threads per watched
tree is not necessary, and just consumes system resources. Each platform
supports watching multiple directories with the same platform-specific
watcher API, so each platform has been updated accordingly.

The interface to the FileWatcher class is greatly simplified. Previously,
it supported client-supplied filtering of the paths that would generate
notifications. This was done by subclassing `FolderWatchBase` and
implementing `OnFileChange`. However, only one filter was ever used, so
that filter is now hard-coded in the FileWatcher class, and the classes
driving the old filtering mechanism are removed. Users of the interface
now have a much easier time, they just call `AddFolderWatch` with the path
to watch, and only have to connect to one set of signals, instead of
separate signals per watched directory.

Signed-off-by: Chris Burel <burelc@amazon.com>
This commit is contained in:
Chris Burel
2021-11-15 11:30:37 -08:00
parent 7ac5bc3d5c
commit ce0bb1ca2b
22 changed files with 658 additions and 932 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Tools/AssetProcessor/native/unittests/FileWatcherUnitTests.cpp
+7 -9
View File
@@ -24,14 +24,12 @@ void FileWatcherUnitTestRunner::StartTest()
FileWatcher fileWatcher;
FolderWatchCallbackEx folderWatch(tempPath, "", true);
fileWatcher.AddFolderWatch(&folderWatch);
fileWatcher.AddFolderWatch(tempPath);
fileWatcher.StartWatching();
{ // test a single file create/write
bool foundFile = false;
auto connection = QObject::connect(&folderWatch, &FolderWatchCallbackEx::fileAdded, this, [&](QString filename)
auto connection = QObject::connect(&fileWatcher, &FileWatcher::fileAdded, this, [&](QString filename)
{
AZ_TracePrintf(AssetProcessor::DebugChannel, "Single file test Found asset: %s.\n", filename.toUtf8().data());
foundFile = true;
@@ -66,7 +64,7 @@ void FileWatcherUnitTestRunner::StartTest()
const unsigned long maxFiles = 10000;
QSet<QString> outstandingFiles;
auto connection = QObject::connect(&folderWatch, &FolderWatchCallbackEx::fileAdded, this, [&](QString filename)
auto connection = QObject::connect(&fileWatcher, &FileWatcher::fileAdded, this, [&](QString filename)
{
outstandingFiles.remove(filename);
});
@@ -122,7 +120,7 @@ void FileWatcherUnitTestRunner::StartTest()
{ // test deletion
bool foundFile = false;
auto connection = QObject::connect(&folderWatch, &FolderWatchCallbackEx::fileRemoved, this, [&](QString filename)
auto connection = QObject::connect(&fileWatcher, &FileWatcher::fileRemoved, this, [&](QString filename)
{
AZ_TracePrintf(AssetProcessor::DebugChannel, "Deleted asset: %s...\n", filename.toUtf8().data());
foundFile = true;
@@ -155,7 +153,7 @@ void FileWatcherUnitTestRunner::StartTest()
{
bool fileAddCalled = false;
QString fileAddName;
auto connectionAdd = QObject::connect(&folderWatch, &FolderWatchCallbackEx::fileAdded, this, [&](QString filename)
auto connectionAdd = QObject::connect(&fileWatcher, &FileWatcher::fileAdded, this, [&](QString filename)
{
fileAddCalled = true;
fileAddName = filename;
@@ -163,7 +161,7 @@ void FileWatcherUnitTestRunner::StartTest()
bool fileRemoveCalled = false;
QString fileRemoveName;
auto connectionRemove = QObject::connect(&folderWatch, &FolderWatchCallbackEx::fileRemoved, this, [&](QString filename)
auto connectionRemove = QObject::connect(&fileWatcher, &FileWatcher::fileRemoved, this, [&](QString filename)
{
fileRemoveCalled = true;
fileRemoveName = filename;
@@ -171,7 +169,7 @@ void FileWatcherUnitTestRunner::StartTest()
QStringList fileModifiedNames;
bool fileModifiedCalled = false;
auto connectionModified = QObject::connect(&folderWatch, &FolderWatchCallbackEx::fileModified, this, [&](QString filename)
auto connectionModified = QObject::connect(&fileWatcher, &FileWatcher::fileModified, this, [&](QString filename)
{
fileModifiedCalled = true;
fileModifiedNames.append(filename);