birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

LYN-2537 engine assets (#254)

Browse Source 
* LYN-2537 Moved the Engine and Editor folder to be within the EngineAssets folder

* Fixed Documentation in bootstrap.cfg to correct the path to the user project specific registry file

* Adding a newline to the output of AssetCatalog 'Registering asset..., but type is not set' message

* Updating the AssetProcessorPlatformConfig.setreg Scan Folder to detect
the @ENGINEROOT@/EngineAssets/Engine path for engine runtime assets and
@ENGINEROOT@/EngineAssets/Editor path for engine tool assets

* Updating references to Icons and other assets to account for moving the
Engine and Editor folder under a single EngineAssets folder

* Moving the Engine Settings Registry folder from Engine/Registry -> Registry

* Removed the LY_PROJECT_CMAKE_PATH define as it is not portable to other locations. It is hard coded to the project location that was used for the CMake configuration. Furthermore it paths with backslashes within it are treated as escape characters and not a path separator

* Updated the LyTestTools asset_processor.py script to copy the exclude.filetag from the EngineAssets/Engine directory now

* Fixed Atom Shader Preprocessing when running using an External Project

* Updated the TSGenerateAction.cpp to fix the build error with using a renamed variable

* Updated the Install_Common.cmake ly_setup_others function to install the
EngineAssets directory and the each of the Gem's Assets directory while
maintaining the relative directory structure to the Engine Root
Also updated the install step to install the Registry folder at the
engine root

* Fixed the copying of the Registry folder to be in the install root, instead of under a second 'Registry' folder

* Moving the AssetProcessorPlatformConfig.setreg file over to the Registry folder

* Updated the LyTestTools and C++ code to point that the new location of
the AssetProcessorPlatformConfig.setreg file inside of the Registry
folder

* Renamed Test AssetProcessor*Config.ini files to have the .setreg extension

* Converted the AssetProcessor test setreg files from ini format to json
format using the SerializeContextTools convert-ini command

* Updated the AssetProcessor CMakeLists.txt to copy over the test setreg files to the build folder

* Updated the assetprocessor test file list to point at the renamed AsssetProcessor*Config setreg filenames

* Removed the Output Prefix code from the AssetProcessor. The complexity that it brought to the AP code is not needed, as users can replicate the behavior by just moving there assets underneath a another folder, underneath the scan folder

* Adding back support to read the AssetProcessorPlatformConfig.setreg file from the asset root. This is only needed for C++ UnitTests as they run in an environment where the accessing the Engine Settings Registry is not available

* Updating the Install_common.cmake logic to copy any "Assets" folder to
the install layout.
The Script has also been updated to copy over the "Assets" folder in the
Engine Root to the install layout instead of an "EngineAssets" folder

* Updating References to EngineAssets source asset folder in code to be the Assets source folder

* Moved the Engine Source Asset folder of 'EngineAssets' to a new folder name of 'Assets'. This is inline with the naming scheme we use for Gem asset folders

* Adding the EngineFinder.cmake to the AutomatedTesting project to allow it to work in a project centric manner

* Updating the LyTestTools copy_assets_to_project function to be able to copy assets with folders to the temporary project root
Fixed an issue in LyTestTools where the temporary log directory could have shutil.rmtree being called twice on it leading to an exception which fails an automated test

Updated the asset_procesor_gui_tests_2 AddScanFolder test to not use the
output prefix, but instead place the source asset root into a
subdirectory

* Correct the AssetProcessorPlatformConfig Scan Folders for the EngineAssets directory to point at the Assets directory

* Updated the asset procesor batch dependency test scan folder to point at the 'Assets' folder instead of 'EngineAssets'
This commit is contained in:
lumberyard-employee-dm
2021-04-28 21:38:43 -05:00
committed by GitHub
parent ed74bb9166
commit 3dec5d3b71
2087 changed files with 1208 additions and 3655 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Tools/AssetProcessor/native/tests/AssetCatalog/AssetCatalogUnitTests.cpp
+6 -7
View File
@@ -217,7 +217,6 @@ namespace AssetProcessor
scanFolderInfo.ScanPath().toStdString().c_str(),
scanFolderInfo.GetDisplayName().toStdString().c_str(),
scanFolderInfo.GetPortableKey().toStdString().c_str(),
scanFolderInfo.GetOutputPrefix().toStdString().c_str(),
scanFolderInfo.IsRoot());
dbConn->SetScanFolder(newScanFolder);
}
@@ -230,12 +229,12 @@ namespace AssetProcessor
config.EnablePlatform({ "fandango" ,{ "console", "renderer" } }, false);
AZStd::vector<AssetBuilderSDK::PlatformInfo> platforms;
config.PopulatePlatformsForScanFolder(platforms);
// PATH DisplayName PortKey outputfolder root recurse platforms order
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder4"), "subfolder4", "subfolder4", "", false, false, platforms, -6), config, dbConn); // subfolder 4 overrides subfolder3
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder3"), "subfolder3", "subfolder3", "", false, false, platforms, -5), config, dbConn); // subfolder 3 overrides subfolder2
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "subfolder2", "subfolder2", "", false, true, platforms, -2), config, dbConn); // subfolder 2 overrides subfolder1
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "subfolder1", "subfolder1", "", false, true, platforms, -1), config, dbConn); // subfolder1 overrides root
AddScanFolder(ScanFolderInfo(tempPath.absolutePath(), "temp", "tempfolder", "", true, false, platforms, 0), config, dbConn); // add the root
// PATH DisplayName PortKey root recurse platforms order
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder4"), "subfolder4", "subfolder4", false, false, platforms, -6), config, dbConn); // subfolder 4 overrides subfolder3
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder3"), "subfolder3", "subfolder3", false, false, platforms, -5), config, dbConn); // subfolder 3 overrides subfolder2
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "subfolder2", "subfolder2", false, true, platforms, -2), config, dbConn); // subfolder 2 overrides subfolder1
AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "subfolder1", "subfolder1", false, true, platforms, -1), config, dbConn); // subfolder1 overrides root
AddScanFolder(ScanFolderInfo(tempPath.absolutePath(), "temp", "tempfolder", true, false, platforms, 0), config, dbConn); // add the root
config.AddMetaDataType("exportsettings", QString());
Code/Tools/AssetProcessor/native/tests/AssetProcessorMessagesTests.cpp
+1 -1
View File
@@ -317,7 +317,7 @@ namespace AssetProcessorMessagesTests
using namespace AzToolsFramework::AssetDatabase;
// Setup the database with all needed info
ScanFolderDatabaseEntry scanfolder1("scanfolder1", "scanfolder1", "scanfolder1", "");
ScanFolderDatabaseEntry scanfolder1("scanfolder1", "scanfolder1", "scanfolder1");
ASSERT_TRUE(m_dbConn.SetScanFolder(scanfolder1));
SourceDatabaseEntry source1(scanfolder1.m_scanFolderID, "source1.png", AZ::Uuid::CreateRandom(), "Fingerprint");
Code/Tools/AssetProcessor/native/tests/FileProcessor/FileProcessorTests.cpp
+2 -2
View File
@@ -46,10 +46,10 @@ namespace UnitTests
m_data->m_fileProcessor = AZStd::make_unique<FileProcessor>(m_data->m_config.get());
m_data->m_scanFolder = { m_data->m_temporarySourceDir.absolutePath().toUtf8().constData(), "dev", "rootportkey", "" };
m_data->m_scanFolder = { m_data->m_temporarySourceDir.absolutePath().toUtf8().constData(), "dev", "rootportkey" };
ASSERT_TRUE(m_data->m_connection.SetScanFolder(m_data->m_scanFolder));
m_data->m_config->AddScanFolder(ScanFolderInfo(m_data->m_temporarySourceDir.absolutePath(), "dev", "rootportkey", "", false, true, m_data->m_config->GetEnabledPlatforms(), 0, m_data->m_scanFolder.m_scanFolderID));
m_data->m_config->AddScanFolder(ScanFolderInfo(m_data->m_temporarySourceDir.absolutePath(), "dev", "rootportkey", false, true, m_data->m_config->GetEnabledPlatforms(), 0, m_data->m_scanFolder.m_scanFolderID));
for (int index = 0; index < 10; ++index)
{
Code/Tools/AssetProcessor/native/tests/SourceFileRelocatorTests.cpp
+10 -12
View File
@@ -84,8 +84,8 @@ namespace UnitTests
AZStd::vector<AssetBuilderSDK::PlatformInfo> platforms;
m_data->m_platformConfig.PopulatePlatformsForScanFolder(platforms);
m_data->m_scanFolder1 = { tempPath.absoluteFilePath("dev").toUtf8().constData(), "dev", "devKey", ""};
m_data->m_scanFolder2 = { tempPath.absoluteFilePath("folder").toUtf8().constData(), "folder", "folderKey", "prefix" };
m_data->m_scanFolder1 = { tempPath.absoluteFilePath("dev").toUtf8().constData(), "dev", "devKey"};
m_data->m_scanFolder2 = { tempPath.absoluteFilePath("folder").toUtf8().constData(), "folder", "folderKey"};
ASSERT_TRUE(m_data->m_connection->SetScanFolder(m_data->m_scanFolder1));
ASSERT_TRUE(m_data->m_connection->SetScanFolder(m_data->m_scanFolder2));
@@ -93,21 +93,19 @@ namespace UnitTests
ScanFolderInfo scanFolder1(m_data->m_scanFolder1.m_scanFolder.c_str(),
m_data->m_scanFolder1.m_displayName.c_str(),
m_data->m_scanFolder1.m_portableKey.c_str(),
m_data->m_scanFolder1.m_outputPrefix.c_str(),
false, true, platforms, 0, m_data->m_scanFolder1.m_scanFolderID);
m_data->m_platformConfig.AddScanFolder(scanFolder1);
ScanFolderInfo scanFolder2(m_data->m_scanFolder2.m_scanFolder.c_str(),
m_data->m_scanFolder2.m_displayName.c_str(),
m_data->m_scanFolder2.m_portableKey.c_str(),
m_data->m_scanFolder2.m_outputPrefix.c_str(),
false, true, platforms, 0, m_data->m_scanFolder2.m_scanFolderID);
m_data->m_platformConfig.AddScanFolder(scanFolder2);
SourceDatabaseEntry sourceFile1 = { m_data->m_scanFolder1.m_scanFolderID, "subfolder1/somefile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile2 = { m_data->m_scanFolder1.m_scanFolderID, "subfolder1/otherfile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile3 = { m_data->m_scanFolder2.m_scanFolderID, "prefix/otherfile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile4 = { m_data->m_scanFolder2.m_scanFolderID, "prefix/a/b/c/d/otherfile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile3 = { m_data->m_scanFolder2.m_scanFolderID, "otherfile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile4 = { m_data->m_scanFolder2.m_scanFolderID, "a/b/c/d/otherfile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile5 = { m_data->m_scanFolder1.m_scanFolderID, "duplicate/file1.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile6 = { m_data->m_scanFolder2.m_scanFolderID, "duplicate/file1.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
SourceDatabaseEntry sourceFile7 = { m_data->m_scanFolder1.m_scanFolderID, "subfolder2/file.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint" };
@@ -128,7 +126,7 @@ namespace UnitTests
ASSERT_TRUE(m_data->m_connection->SetSource(sourceFile11));
SourceFileDependencyEntry dependency1 = { AZ::Uuid::CreateRandom(), "subfolder1/somefile.tif", "subfolder1/otherfile.tif", SourceFileDependencyEntry::TypeOfDependency::DEP_SourceToSource, false };
SourceFileDependencyEntry dependency2 = { AZ::Uuid::CreateRandom(), "subfolder1/otherfile.tif", "prefix/otherfile.tif", SourceFileDependencyEntry::TypeOfDependency::DEP_JobToJob, false };
SourceFileDependencyEntry dependency2 = { AZ::Uuid::CreateRandom(), "subfolder1/otherfile.tif", "otherfile.tif", SourceFileDependencyEntry::TypeOfDependency::DEP_JobToJob, false };
ASSERT_TRUE(m_data->m_connection->SetSourceFileDependency(dependency1));
ASSERT_TRUE(m_data->m_connection->SetSourceFileDependency(dependency2));
@@ -252,7 +250,7 @@ namespace UnitTests
const ScanFolderInfo* info;
auto result = m_data->m_reporter->GetSourcesByPath(source, relocationContainer, info, excludeMetaDataFiles);
ASSERT_EQ(result.IsSuccess(), expectSuccess) << result.GetError().c_str();
ASSERT_EQ(result.IsSuccess(), expectSuccess) << (!result.IsSuccess() ? result.GetError().c_str() : "");
if (expectSuccess)
{
@@ -423,13 +421,13 @@ namespace UnitTests
TEST_F(SourceFileRelocatorTest, GetSources_PrefixedFile_Succeeds)
{
TestGetSourcesByPath("otherfile.tif", { "prefix/otherfile.tif" });
TestGetSourcesByPath("otherfile.tif", { "otherfile.tif" });
}
TEST_F(SourceFileRelocatorTest, GetSources_PrefixedAbsFile_Succeeds)
{
QDir tempDir(m_tempDir.path());
TestGetSourcesByPath( tempDir.absoluteFilePath("folder/otherfile.tif").toUtf8().constData(), { "prefix/otherfile.tif" });
TestGetSourcesByPath(tempDir.absoluteFilePath("folder/otherfile.tif").toUtf8().constData(), { "otherfile.tif" });
}
TEST_F(SourceFileRelocatorTest, GetSources_Folder_Fails)
@@ -664,7 +662,7 @@ namespace UnitTests
ASSERT_TRUE(m_data->m_reporter->GetSourcesByPath(tempPath.absoluteFilePath(AZStd::string("folder/o*").c_str()).toUtf8().constData(), entryContainer, info));
ASSERT_EQ(entryContainer.size(), 1);
ASSERT_STREQ(entryContainer[0].m_sourceEntry.m_sourceName.c_str(), "prefix/otherfile.tif");
ASSERT_STREQ(entryContainer[0].m_sourceEntry.m_sourceName.c_str(), "otherfile.tif");
m_data->m_reporter->PopulateDependencies(entryContainer);
@@ -803,7 +801,7 @@ namespace UnitTests
FileUpdateTasks updateTasks;
ASSERT_TRUE(m_data->m_reporter->GetSourcesByPath(tempPath.absoluteFilePath("folder/*").toUtf8().constData(), entryContainer, info));
ASSERT_EQ(entryContainer.size(), 2);
ASSERT_EQ(entryContainer.size(), 3);
m_data->m_reporter->ComputeDestination(entryContainer, info, "*", "someOtherPlace/*", destInfo);
m_data->m_reporter->PopulateDependencies(entryContainer);
Code/Tools/AssetProcessor/native/tests/assetdatabase/AssetDatabaseTest.cpp
+4 -4
View File
@@ -81,7 +81,7 @@ namespace UnitTests
// Product: "someproduct4.dds" subid: 4
void CreateCoverageTestData()
{
m_data->m_scanFolder = { "c:/O3DE/dev", "dev", "rootportkey", "" };
m_data->m_scanFolder = { "c:/O3DE/dev", "dev", "rootportkey" };
ASSERT_TRUE(m_data->m_connection.SetScanFolder(m_data->m_scanFolder));
m_data->m_sourceFile1 = { m_data->m_scanFolder.m_scanFolderID, "somefile.tif", AZ::Uuid::CreateRandom(), "AnalysisFingerprint1"};
@@ -239,7 +239,7 @@ namespace UnitTests
// we'll create all of those first (except product) before starting the product test.
//add a scanfolder. None of this has to exist in real disk, this is a db test only.
ScanFolderDatabaseEntry scanFolder {"c:/O3DE/dev", "dev", "rootportkey", ""};
ScanFolderDatabaseEntry scanFolder{ "c:/O3DE/dev", "dev", "rootportkey" };
EXPECT_TRUE(m_data->m_connection.SetScanFolder(scanFolder));
ASSERT_NE(scanFolder.m_scanFolderID, AzToolsFramework::AssetDatabase::InvalidEntryId);
@@ -278,7 +278,7 @@ namespace UnitTests
// to add a product legitimately you have to have a full chain of primary keys, chain is:
// ScanFolder --> Source --> job --> product.
// we'll create all of those first (except product) before starting the product test.
ScanFolderDatabaseEntry scanFolder{ "c:/O3DE/dev", "dev", "rootportkey", "" };
ScanFolderDatabaseEntry scanFolder{ "c:/O3DE/dev", "dev", "rootportkey" };
ASSERT_TRUE(m_data->m_connection.SetScanFolder(scanFolder));
SourceDatabaseEntry sourceEntry{ scanFolder.m_scanFolderID, "somefile.tif", AZ::Uuid::CreateRandom(), "fingerprint1" };
@@ -323,7 +323,7 @@ namespace UnitTests
// this is actually a very common case (same job id, same subID)
TEST_F(AssetDatabaseTest, SetProduct_SpecificPK_Succeeds_SameSubID_SameJobID)
{
ScanFolderDatabaseEntry scanFolder{ "c:/O3DE/dev", "dev", "rootportkey", "" };
ScanFolderDatabaseEntry scanFolder{ "c:/O3DE/dev", "dev", "rootportkey"};
ASSERT_TRUE(m_data->m_connection.SetScanFolder(scanFolder));
SourceDatabaseEntry sourceEntry{ scanFolder.m_scanFolderID, "somefile.tif", AZ::Uuid::CreateRandom(), "fingerprint1" };
ASSERT_TRUE(m_data->m_connection.SetSource(sourceEntry));
Code/Tools/AssetProcessor/native/tests/assetmanager/AssetProcessorManagerTest.cpp
+17 -670
View File
@@ -63,13 +63,6 @@ public:
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WildcardMissingFiles_ByName_UpdatesWhenTheyAppear);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, JobDependencyOrderOnce_MultipleJobs_EmitOK);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_BasicTest);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_UpdateTest);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByUuid);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByName);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByUuid_UpdatesWhenTheyAppear);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByName_UpdatesWhenTheyAppear);
friend class GTEST_TEST_CLASS_NAME_(AssetProcessorManagerTest, SourceFileProcessFailure_ClearsFingerprint);
friend class GTEST_TEST_CLASS_NAME_(AbsolutePathProductDependencyTest, UnresolvedProductPathDependency_AssetProcessedTwice_DoesNotDuplicateDependency);
@@ -243,15 +236,10 @@ void AssetProcessorManagerTest::SetUp()
m_config->EnablePlatform({ "pc", { "host", "renderer", "desktop" } }, true);
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "subfolder1", "subfolder1", "", false, true, m_config->GetEnabledPlatforms() ));
// add a scan folder that has an output prefix:
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "subfolder2", "subfolder2", "redirected", false, true, m_config->GetEnabledPlatforms()));
//adding another scan folder with an output prefix
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder3"), "subfolder3", "subfolder3", "dummy", false, true, m_config->GetEnabledPlatforms()));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder4"), "subfolder4", "subfolder4", "", false, true, m_config->GetEnabledPlatforms()));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "subfolder1", "subfolder1", false, true, m_config->GetEnabledPlatforms(), 1));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2/redirected"), "subfolder2", "subfolder2", false, true, m_config->GetEnabledPlatforms()));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder3"), "subfolder3", "subfolder3", false, true, m_config->GetEnabledPlatforms(), 1));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder4"), "subfolder4", "subfolder4", false, true, m_config->GetEnabledPlatforms(), 1));
m_config->AddMetaDataType("assetinfo", "");
AssetRecognizer rec;
AssetPlatformSpec specpc;
@@ -459,72 +447,6 @@ TEST_F(AssetProcessorManagerTest, DeleteFolder_SignalsDeleteOfContainedFiles)
EXPECT_EQ(1, count);
}
TEST_F(AssetProcessorManagerTest, UnitTestForOutPutPrefix)
{
using namespace AssetProcessor;
QDir tempPath(m_tempDir.path());
UnitTestUtils::CreateDummyFile(tempPath.absoluteFilePath("subfolder2/test.txt"));
UnitTestUtils::CreateDummyFile(tempPath.absoluteFilePath("subfolder3/test.txt"));
int count = 0;
auto connection = QObject::connect(m_assetProcessorManager.get(), &AssetProcessorManager::AssetToProcess, [&count](JobDetails jobDetails)
{
if (jobDetails.m_jobEntry.m_databaseSourceName.compare("redirected/test.txt", Qt::CaseInsensitive) == 0 ||
jobDetails.m_jobEntry.m_databaseSourceName.compare("dummy/test.txt", Qt::CaseInsensitive) == 0 )
{
count++;
}
});
m_isIdling = false;
// tell the APM about the files:
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder2/test.txt")));
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder3/test.txt")));
ASSERT_TRUE(BlockUntilIdle(5000));
EXPECT_EQ(2, count);
EXPECT_TRUE(QFile::remove(tempPath.absoluteFilePath("subfolder2/test.txt")));
EXPECT_TRUE(QFile::remove(tempPath.absoluteFilePath("subfolder3/test.txt")));
}
TEST_F(AssetProcessorManagerTest, UnitTestForOutPutPrefixWithMultipleNotification)
{
using namespace AssetProcessor;
QDir tempPath(m_tempDir.path());
UnitTestUtils::CreateDummyFile(tempPath.absoluteFilePath("subfolder2/test.txt"));
UnitTestUtils::CreateDummyFile(tempPath.absoluteFilePath("subfolder3/test.txt"));
int count = 0;
auto connection = QObject::connect(m_assetProcessorManager.get(), &AssetProcessorManager::AssetToProcess, [&count](JobDetails jobDetails)
{
if (jobDetails.m_jobEntry.m_databaseSourceName.compare("redirected/test.txt", Qt::CaseInsensitive) == 0 ||
jobDetails.m_jobEntry.m_databaseSourceName.compare("dummy/test.txt", Qt::CaseInsensitive) == 0)
{
count++;
}
});
m_isIdling = false;
// we are putting multiple entries in the pipeline here and testing that we still receive only unique notifications for processing those assets
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder2/test.txt")));
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder2/test.txt")));
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder2/test.txt")));
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder3/test.txt")));
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder3/test.txt")));
QMetaObject::invokeMethod(m_assetProcessorManager.get(), "AssessModifiedFile", Qt::QueuedConnection, Q_ARG(QString, tempPath.absoluteFilePath("subfolder3/test.txt")));
ASSERT_TRUE(BlockUntilIdle(5000));
EXPECT_EQ(2, count);
EXPECT_TRUE(QFile::remove(tempPath.absoluteFilePath("subfolder2/test.txt")));
EXPECT_TRUE(QFile::remove(tempPath.absoluteFilePath("subfolder3/test.txt")));
}
TEST_F(AssetProcessorManagerTest, UnitTestForGettingJobInfoBySourceUUIDFailure)
{
using namespace AzToolsFramework::AssetSystem;
@@ -1191,7 +1113,7 @@ TEST_F(AssetProcessorManagerTest, BuilderSDK_API_CreateJobs_HasValidParameters_W
{
QDir tempPath(m_tempDir.path());
// here we push a file change through APM and make sure that "CreateJobs" has correct parameters, with no output redirection
QString absPath(tempPath.absoluteFilePath("subfolder2/test_text.txt"));
QString absPath(tempPath.absoluteFilePath("subfolder2/redirected/test_text.txt"));
UnitTestUtils::CreateDummyFile(absPath);
m_mockApplicationManager->ResetMockBuilderCreateJobCalls();
@@ -1212,7 +1134,7 @@ TEST_F(AssetProcessorManagerTest, BuilderSDK_API_CreateJobs_HasValidParameters_W
// subfolder2 has its output redirected in the cache
// this test makes sure that the CreateJobs API is completely unaffected by that and none of the internal database stuff
// is reflected by the API.
EXPECT_STREQ(req.m_watchFolder.c_str(), tempPath.absoluteFilePath("subfolder2").toUtf8().constData());
EXPECT_STREQ(req.m_watchFolder.c_str(), tempPath.absoluteFilePath("subfolder2/redirected").toUtf8().constData());
EXPECT_STREQ(req.m_sourceFile.c_str(), "test_text.txt"); // only the name should be there, no output prefix.
EXPECT_NE(req.m_sourceFileUUID, AZ::Uuid::CreateNull());
@@ -2354,7 +2276,7 @@ TEST_F(PathDependencyTest, AbsoluteDependencies_Deferred_ResolveCorrectly)
// When an absolute path matches a scan folder, the portion of the path matching that scan folder
// is replaced with the scan folder's ID.
AZStd::string absPathDep1WithScanfolder(AZStd::string::format("$4$%s", relativePathDep1.c_str()));
QString absPathDep2(tempPath.absoluteFilePath("subfolder2/dep2.txt"));
QString absPathDep2(tempPath.absoluteFilePath("subfolder2/redirected/dep2.txt"));
QString absPathDep3(tempPath.absoluteFilePath("subfolder1/dep3.txt"));
// -------- Make main test asset, with dependencies on products that don't exist yet -----
@@ -2375,7 +2297,7 @@ TEST_F(PathDependencyTest, AbsoluteDependencies_Deferred_ResolveCorrectly)
// Different scanfolder, same relative file name. This should *not* trigger the dependency. We can't test with another asset in the proper scanfolder because AssetIds are based on relative file name,
// which means both assets have the same AssetId and there would be no way to tell which one matched
ASSERT_TRUE(ProcessAsset(dep1, { {".asset1"}, {".asset2"} }, {}, "subfolder1/"));
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/")); // subfolder2 has a prefix of "redirected"
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/redirected/"));
ASSERT_TRUE(ProcessAsset(dep3, { {".asset1"}, {".asset2"} }, {}, "subfolder1/")); // test a normal dependency with no prefix
// ---------- Verify that the dependency was recorded, and did not keep the path after resolution ----------
@@ -2550,14 +2472,14 @@ TEST_F(PathDependencyTest, AssetProcessed_Impl_DeferredPathResolution_CorrectlyM
// -------- Make main test asset, with dependencies on products that don't exist yet -----
TestAsset primaryFile("test_text");
ASSERT_TRUE(ProcessAsset(primaryFile, { { ".asset" }, {} }, { {"test.asset1", ProductPathDependencyType::ProductFile}, {"redirected/test.asset2", ProductPathDependencyType::ProductFile} }));
ASSERT_TRUE(ProcessAsset(primaryFile, { { ".asset" }, {} }, { {"test.asset1", ProductPathDependencyType::ProductFile}, {"test.asset2", ProductPathDependencyType::ProductFile} }));
// create dependees
TestAsset dep1("test");
TestAsset dep2("test");
ASSERT_TRUE(ProcessAsset(dep1, { {".asset1"}, {".asset2"} }, {}, "subfolder1/"));
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/")); // subfolder2 has a prefix of "redirected"
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/redirected/"));
// ---------- Verify that the dependency was recorded, and did not keep the path after resolution ----------
AzToolsFramework::AssetDatabase::ProductDependencyDatabaseEntryContainer dependencyContainer;
@@ -2575,14 +2497,14 @@ TEST_F(PathDependencyTest, SourceFileDependencyWithPrefix_Deferred_ResolvesCorre
// -------- Make main test asset, with dependencies on products that don't exist yet -----
TestAsset primaryFile("test_text");
ASSERT_TRUE(ProcessAsset(primaryFile, { { ".asset" }, {} }, { {"redirected/test.txt", ProductPathDependencyType::SourceFile} }));
ASSERT_TRUE(ProcessAsset(primaryFile, { { ".asset" }, {} }, { {"test.txt", ProductPathDependencyType::SourceFile} }));
// create dependees
TestAsset dep1("test");
TestAsset dep2("test");
ASSERT_TRUE(ProcessAsset(dep1, { {".asset1"}, {".asset2"} }, {}, "subfolder1/"));
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/")); // subfolder2 has a prefix of "redirected"
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/redirected/"));
// ---------- Verify that the dependency was recorded, and did not keep the path after resolution ----------
AzToolsFramework::AssetDatabase::ProductDependencyDatabaseEntryContainer dependencyContainer;
@@ -2606,12 +2528,12 @@ TEST_F(PathDependencyTest, SourceFileDependencyWithPrefix_Existing_ResolvesCorre
TestAsset dep2("test");
ASSERT_TRUE(ProcessAsset(dep1, { {".asset1"}, {".asset2"} }, {}, "subfolder1/"));
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/")); // subfolder2 has a prefix of "redirected"
ASSERT_TRUE(ProcessAsset(dep2, { {".asset1"}, {".asset2"} }, {}, "subfolder2/redirected/"));
// -------- Make main test asset, with dependencies on products we just created -----
TestAsset primaryFile("test_text");
ASSERT_TRUE(ProcessAsset(primaryFile, { { ".asset" }, {} }, { {"redirected/test.txt", ProductPathDependencyType::SourceFile} }));
ASSERT_TRUE(ProcessAsset(primaryFile, { { ".asset" }, {} }, { {"test.txt", ProductPathDependencyType::SourceFile} }));
// ---------- Verify that the dependency was recorded, and did not keep the path after resolution ----------
AzToolsFramework::AssetDatabase::ProductDependencyDatabaseEntryContainer dependencyContainer;
@@ -2633,13 +2555,9 @@ void MultiplatformPathDependencyTest::SetUp()
m_config->EnablePlatform({ "provo",{ "console" } }, true);
QDir tempPath(m_tempDir.path());
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "subfolder1", "subfolder1", "", false, true, m_config->GetEnabledPlatforms() ));
// add a scan folder that has an output prefix:
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "subfolder2", "subfolder2", "redirected", false, true, m_config->GetEnabledPlatforms()));
//adding another scan folder with an output prefix
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder3"), "subfolder3", "subfolder3", "dummy", false, true, m_config->GetEnabledPlatforms()));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "subfolder1", "subfolder1", false, true, m_config->GetEnabledPlatforms() ));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "subfolder2", "subfolder2", false, true, m_config->GetEnabledPlatforms()));
m_config->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder3"), "subfolder3", "subfolder3", false, true, m_config->GetEnabledPlatforms()));
m_assetProcessorManager = nullptr; // we need to destroy the previous instance before creating a new one
m_assetProcessorManager = AZStd::make_unique<AssetProcessorManager_Test>(m_config.get());
@@ -3741,577 +3659,6 @@ TEST_F(AssetProcessorManagerTest, JobDependencyOrderOnce_MultipleJobs_EmitOK)
EXPECT_EQ(jobDetails[1].m_jobDependencyList[0].m_jobDependency.m_sourceFile.m_sourceFileDependencyPath, secondRelSourceFile); // there should only be one job dependency
}
// ------------------------------------------------------------------------------------------------------------
// The same suite of tests as above, but using a folder with an output prefix instead
// ------------------------------------------------------------------------------------------------------------
TEST_F(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_BasicTest)
{
// make sure that if we publish some dependencies, they appear:
AZ::Uuid dummyBuilderUUID = AZ::Uuid::CreateRandom();
QDir tempPath(m_tempDir.path());
QString inputDatabasePath = "redirected/outputprefixtest.txt";
UnitTestUtils::CreateDummyFile(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
QString absPath(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
QString watchFolderPath = tempPath.absoluteFilePath("subfolder2");
const ScanFolderInfo* scanFolder = m_config->GetScanFolderByPath(watchFolderPath);
ASSERT_NE(scanFolder, nullptr);
ASSERT_STREQ(scanFolder->GetOutputPrefix().toUtf8().constData(), "redirected");
// the above file (assetProcessorManagerTest.txt) will depend on these four files
QString dependsOnFile1_Source = tempPath.absoluteFilePath("subfolder2/a.txt");
QString dependsOnFile2_Source = tempPath.absoluteFilePath("subfolder2/b.txt");
QString dependsOnFile1_Job = tempPath.absoluteFilePath("subfolder2/c.txt");
QString dependsOnFile2_Job = tempPath.absoluteFilePath("subfolder2/d.txt");
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Job, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Job, QString("tempdata\n")));
// construct the dummy job to feed to the database updater function:
AssetProcessorManager::JobToProcessEntry job;
job.m_sourceFileInfo.m_databasePath = inputDatabasePath;
job.m_sourceFileInfo.m_pathRelativeToScanFolder = "outputprefixtest.txt";
job.m_sourceFileInfo.m_scanFolder = scanFolder;
job.m_sourceFileInfo.m_uuid = AssetUtilities::CreateSafeSourceUUIDFromName(job.m_sourceFileInfo.m_databasePath.toUtf8().data());
// note that we have to "prime" the map with the UUIDs to the source info for this to work:
AZ::Uuid uuidOfB = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/b.txt");
AZ::Uuid uuidOfD = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/d.txt");
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfB] = { watchFolderPath, "b.txt", "redirected/b.txt" };
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfD] = { watchFolderPath, "d.txt", "redirected/d.txt" };
// each file we will take a different approach to publishing: rel path, and UUID:
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "a.txt", AZ::Uuid::CreateNull() }));
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "", uuidOfB }));
// it is currently assumed that the only fields that we care about in JobDetails is the builder busId and the job dependencies themselves:
JobDetails newDetails;
newDetails.m_assetBuilderDesc.m_busId = dummyBuilderUUID;
AssetBuilderSDK::SourceFileDependency dep1 = { "c.txt", AZ::Uuid::CreateNull() };
AssetBuilderSDK::JobDependency jobDep1("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep1);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep1));
AssetBuilderSDK::SourceFileDependency dep2 = { "",uuidOfD };
AssetBuilderSDK::JobDependency jobDep2("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep2);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep2));
job.m_jobsToAnalyze.push_back(newDetails);
// this is the one line that this unit test is really testing:
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// the rest of this test now performs a series of queries to verify the database was correctly set.
// this indirectly verifies the QueryAbsolutePathDependenciesRecursive function also but it has its own dedicated tests, above.
AssetProcessor::SourceFilesForFingerprintingContainer deps;
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_SourceToSource, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end());
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_JobToJob, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end());
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_Any, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 5);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end());
// make sure the corresponding values in the map are also correct (ie, database path only)
EXPECT_STREQ(deps[absPath.toUtf8().constData()].c_str(), "redirected/outputprefixtest.txt");
EXPECT_STREQ(deps[dependsOnFile1_Source.toUtf8().constData()].c_str(), "redirected/a.txt");
EXPECT_STREQ(deps[dependsOnFile2_Source.toUtf8().constData()].c_str(), "redirected/b.txt");
EXPECT_STREQ(deps[dependsOnFile1_Job.toUtf8().constData()].c_str(), "redirected/c.txt");
EXPECT_STREQ(deps[dependsOnFile2_Job.toUtf8().constData()].c_str(), "redirected/d.txt");
}
TEST_F(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_UpdateTest)
{
// make sure that if we remove dependencies that are published, they disappear.
// so the first part of this test is to put some data in there, the same as before:
AZ::Uuid dummyBuilderUUID = AZ::Uuid::CreateRandom();
QDir tempPath(m_tempDir.path());
QString absPath(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
UnitTestUtils::CreateDummyFile(absPath);
QString inputDatabasePath = "redirected/outputprefixtest.txt";
QString watchFolderPath = tempPath.absoluteFilePath("subfolder2");
const ScanFolderInfo* scanFolder = m_config->GetScanFolderByPath(watchFolderPath);
ASSERT_NE(scanFolder, nullptr);
// the above file (assetProcessorManagerTest.txt) will depend on these four files:
QString dependsOnFile1_Source = tempPath.absoluteFilePath("subfolder2/a.txt");
QString dependsOnFile2_Source = tempPath.absoluteFilePath("subfolder2/b.txt");
QString dependsOnFile1_Job = tempPath.absoluteFilePath("subfolder2/c.txt");
QString dependsOnFile2_Job = tempPath.absoluteFilePath("subfolder2/d.txt");
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Job, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Job, QString("tempdata\n")));
// construct the dummy job to feed to the database updater function:
AssetProcessorManager::JobToProcessEntry job;
job.m_sourceFileInfo.m_databasePath = inputDatabasePath;
job.m_sourceFileInfo.m_pathRelativeToScanFolder = "outputprefixtest.txt";
job.m_sourceFileInfo.m_scanFolder = scanFolder;
job.m_sourceFileInfo.m_uuid = AssetUtilities::CreateSafeSourceUUIDFromName(job.m_sourceFileInfo.m_databasePath.toUtf8().data());
// note that we have to "prime" the map with the UUIDs to the source info for this to work:
AZ::Uuid uuidOfB = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/b.txt");
AZ::Uuid uuidOfD = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/d.txt");
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfB] = { watchFolderPath, "b.txt", "redirected/b.txt" };
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfD] = { watchFolderPath, "d.txt", "redirected/d.txt" };
// each file we will take a different approach to publishing: rel path, and UUID:
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "a.txt", AZ::Uuid::CreateNull() }));
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "", uuidOfB }));
// it is currently assumed that the only fields that we care about in JobDetails is the builder busId and the job dependencies themselves:
JobDetails newDetails;
newDetails.m_assetBuilderDesc.m_busId = dummyBuilderUUID;
AssetBuilderSDK::SourceFileDependency dep1 = { "c.txt", AZ::Uuid::CreateNull() };
AssetBuilderSDK::JobDependency jobDep1("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep1);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep1));
AssetBuilderSDK::SourceFileDependency dep2 = { "",uuidOfD };
AssetBuilderSDK::JobDependency jobDep2("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep2);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep2));
job.m_jobsToAnalyze.push_back(newDetails);
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// in this test, though, we delete some after pushing them in there, and update it again:
job.m_sourceFileDependencies.pop_back(); // erase the 'b' dependency.
job.m_jobsToAnalyze[0].m_jobDependencyList.pop_back(); // erase the 'd' dependency, which is by guid.
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// now make sure that the same queries omit b and d:
AssetProcessor::SourceFilesForFingerprintingContainer deps;
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_SourceToSource, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end());
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_JobToJob, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end());
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_Any, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end());
}
TEST_F(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByUuid)
{
// make sure that if we publish some dependencies, they do not appear if missing.
AZ::Uuid dummyBuilderUUID = AZ::Uuid::CreateRandom();
QDir tempPath(m_tempDir.path());
QString absPath(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
UnitTestUtils::CreateDummyFile(absPath);
QString inputDatabasePath = "redirected/outputprefixtest.txt";
QString watchFolderPath = tempPath.absoluteFilePath("subfolder2");
const ScanFolderInfo* scanFolder = m_config->GetScanFolderByPath(watchFolderPath);
ASSERT_NE(scanFolder, nullptr);
// the above file (assetProcessorManagerTest.txt) will depend on these four files:
QString dependsOnFile1_Source = tempPath.absoluteFilePath("subfolder2/a.txt");
QString dependsOnFile2_Source = tempPath.absoluteFilePath("subfolder2/b.txt");
QString dependsOnFile1_Job = tempPath.absoluteFilePath("subfolder2/c.txt");
QString dependsOnFile2_Job = tempPath.absoluteFilePath("subfolder2/d.txt");
// in this case, we are only creating file b, and d (which are input by UUID)
// and we will be missing a and c, which are input by name.
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Job, QString("tempdata\n")));
// construct the dummy job to feed to the database updater function:
AssetProcessorManager::JobToProcessEntry job;
job.m_sourceFileInfo.m_databasePath = inputDatabasePath;
job.m_sourceFileInfo.m_pathRelativeToScanFolder = "outputprefixtest.txt";
job.m_sourceFileInfo.m_scanFolder = scanFolder;
job.m_sourceFileInfo.m_uuid = AssetUtilities::CreateSafeSourceUUIDFromName(job.m_sourceFileInfo.m_databasePath.toUtf8().data());
// note that we have to "prime" the map with the UUIDs to the source info for this to work:
AZ::Uuid uuidOfB = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/b.txt");
AZ::Uuid uuidOfD = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/d.txt");
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfB] = { watchFolderPath, "b.txt", "redirected/b.txt" };
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfD] = { watchFolderPath, "d.txt", "redirected/d.txt" };
// each file we will take a different approach to publishing: rel path, and UUID:
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "a.txt", AZ::Uuid::CreateNull() }));
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "", uuidOfB }));
// it is currently assumed that the only fields that we care about in JobDetails is the builder busId and the job dependencies themselves:
JobDetails newDetails;
newDetails.m_assetBuilderDesc.m_busId = dummyBuilderUUID;
AssetBuilderSDK::SourceFileDependency dep1 = { "c.txt", AZ::Uuid::CreateNull() };
AssetBuilderSDK::JobDependency jobDep1("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep1);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep1));
AssetBuilderSDK::SourceFileDependency dep2 = { "",uuidOfD };
AssetBuilderSDK::JobDependency jobDep2("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep2);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep2));
job.m_jobsToAnalyze.push_back(newDetails);
// this is the one line that this unit test is really testing:
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// the rest of this test now performs a series of queries to verify the database was correctly set.
// this indirectly verifies the QueryAbsolutePathDependenciesRecursive function also but it has its own dedicated tests, above.
AssetProcessor::SourceFilesForFingerprintingContainer deps;
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_SourceToSource, false);
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end()); // b
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_JobToJob, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end()); // d
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_Any, false);
// the above function includes the actual source, as an absolute path.
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end()); // b
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end()); // d
}
TEST_F(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByName)
{
AZ::Uuid dummyBuilderUUID = AZ::Uuid::CreateRandom();
QDir tempPath(m_tempDir.path());
QString absPath(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
UnitTestUtils::CreateDummyFile(absPath);
QString inputDatabasePath = "redirected/outputprefixtest.txt";
QString watchFolderPath = tempPath.absoluteFilePath("subfolder2");
const ScanFolderInfo* scanFolder = m_config->GetScanFolderByPath(watchFolderPath);
ASSERT_NE(scanFolder, nullptr);
// the above file (assetProcessorManagerTest.txt) will depend on these four files:
QString dependsOnFile1_Source = tempPath.absoluteFilePath("subfolder2/a.txt");
QString dependsOnFile2_Source = tempPath.absoluteFilePath("subfolder2/b.txt");
QString dependsOnFile1_Job = tempPath.absoluteFilePath("subfolder2/c.txt");
QString dependsOnFile2_Job = tempPath.absoluteFilePath("subfolder2/d.txt");
// in this case, we are only creating file a, and c, which are input by name
// and we we will be making b and d missing, which are input by UUID.
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Job, QString("tempdata\n")));
// construct the dummy job to feed to the database updater function:
AssetProcessorManager::JobToProcessEntry job;
job.m_sourceFileInfo.m_databasePath = inputDatabasePath;
job.m_sourceFileInfo.m_pathRelativeToScanFolder = "outputprefixtest.txt";
job.m_sourceFileInfo.m_scanFolder = scanFolder;
job.m_sourceFileInfo.m_uuid = AssetUtilities::CreateSafeSourceUUIDFromName(job.m_sourceFileInfo.m_databasePath.toUtf8().data());
// note that we have to "prime" the map with the UUIDs to the source info for this to work:
AZ::Uuid uuidOfB = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/b.txt");
AZ::Uuid uuidOfD = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/d.txt");
// each file we will take a different approach to publishing: rel path, and UUID:
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "a.txt", AZ::Uuid::CreateNull() }));
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "", uuidOfB }));
// it is currently assumed that the only fields that we care about in JobDetails is the builder busId and the job dependencies themselves:
JobDetails newDetails;
newDetails.m_assetBuilderDesc.m_busId = dummyBuilderUUID;
AssetBuilderSDK::SourceFileDependency dep1 = { "c.txt", AZ::Uuid::CreateNull() };
AssetBuilderSDK::JobDependency jobDep1("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep1);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep1));
AssetBuilderSDK::SourceFileDependency dep2 = { "",uuidOfD };
AssetBuilderSDK::JobDependency jobDep2("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep2);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep2));
job.m_jobsToAnalyze.push_back(newDetails);
// this is the one line that this unit test is really testing:
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// the rest of this test now performs a series of queries to verify the database was correctly set.
// this indirectly verifies the QueryAbsolutePathDependenciesRecursive function also but it has its own dedicated tests, above.
AssetProcessor::SourceFilesForFingerprintingContainer deps;
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_SourceToSource, false);
// we should find all of the deps, but none of the placeholder
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end()); // a
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_JobToJob, false);
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end()); // c
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_Any, false);
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end()); // a
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end()); // c
}
TEST_F(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByUuid_UpdatesWhenTheyAppear)
{
// this test makes sure that when files DO appear that were previously placeholders, the database is updated
// so the strategy here is to have files b, and d missing, which are declared as dependencies by UUID.
// then, we make them re-appear later, and check that the database has updated them appropriately.
AZ::Uuid dummyBuilderUUID = AZ::Uuid::CreateRandom();
QDir tempPath(m_tempDir.path());
QString absPath(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
UnitTestUtils::CreateDummyFile(absPath);
QString inputDatabasePath = "redirected/outputprefixtest.txt";
QString watchFolderPath = tempPath.absoluteFilePath("subfolder2");
const ScanFolderInfo* scanFolder = m_config->GetScanFolderByPath(watchFolderPath);
ASSERT_NE(scanFolder, nullptr);
// the above file (assetProcessorManagerTest.txt) will depend on these four files:
QString dependsOnFile1_Source = tempPath.absoluteFilePath("subfolder2/a.txt");
QString dependsOnFile2_Source = tempPath.absoluteFilePath("subfolder2/b.txt");
QString dependsOnFile1_Job = tempPath.absoluteFilePath("subfolder2/c.txt");
QString dependsOnFile2_Job = tempPath.absoluteFilePath("subfolder2/d.txt");
// in this case, we are only creating file b, and d, which are addressed by UUID.
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Job, QString("tempdata\n")));
// construct the dummy job to feed to the database updater function:
AssetProcessorManager::JobToProcessEntry job;
job.m_sourceFileInfo.m_databasePath = inputDatabasePath;
job.m_sourceFileInfo.m_pathRelativeToScanFolder = "outputprefixtest.txt";
job.m_sourceFileInfo.m_scanFolder = scanFolder;
job.m_sourceFileInfo.m_uuid = AssetUtilities::CreateSafeSourceUUIDFromName(job.m_sourceFileInfo.m_databasePath.toUtf8().data());
AZ::Uuid uuidOfD = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/d.txt");
AZ::Uuid uuidOfB = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/b.txt");
// each file we will take a different approach to publishing: rel path, and UUID:
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "a.txt", AZ::Uuid::CreateNull() }));
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "", uuidOfB }));
// it is currently assumed that the only fields that we care about in JobDetails is the builder busId and the job dependencies themselves:
JobDetails newDetails;
newDetails.m_assetBuilderDesc.m_busId = dummyBuilderUUID;
AssetBuilderSDK::SourceFileDependency dep1 = { "c.txt", AZ::Uuid::CreateNull() };
AssetBuilderSDK::JobDependency jobDep1("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep1);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep1));
AssetBuilderSDK::SourceFileDependency dep2 = { "",uuidOfD };
AssetBuilderSDK::JobDependency jobDep2("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep2);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep2));
job.m_jobsToAnalyze.push_back(newDetails);
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// so at this point, the database should be in the same state as after the UpdateSourceFileDependenciesDatabase_MissingFiles_ByUuid test
// which was already verified, by that test.
// now that the database has placeholders, we expect them to resolve themselves when we provide the actual files:
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Source, QString("tempdata\n")));
// now that B exists, we pretend a job came in to process B. (it doesn't require dependencies to be declared)
// note that we have to "prime" the map with the UUIDs to the source info for this to work:
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfB] = { watchFolderPath, "b.txt", "redirected/b.txt" };
AssetProcessorManager::JobToProcessEntry job2;
job2.m_sourceFileInfo.m_databasePath = "redirected/b.txt";
job2.m_sourceFileInfo.m_pathRelativeToScanFolder = "b.txt";
job2.m_sourceFileInfo.m_scanFolder = scanFolder;
job2.m_sourceFileInfo.m_uuid = uuidOfB;
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job2);
// b should no longer be a placeholder, so both A and B should be present as their actual path.
AssetProcessor::SourceFilesForFingerprintingContainer deps;
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_SourceToSource, false);
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end()); // a
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end()); // b
// but d should still be a placeholder, since we have not declared it yet.
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_JobToJob, false);
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end()); // c
// in addition, we expect to have the original file that depends on B appear in the analysis queue, since something it depends on appeared:
QString normalizedSourcePath = AssetUtilities::NormalizeFilePath(absPath);
EXPECT_TRUE(m_assetProcessorManager->m_alreadyActiveFiles.contains(normalizedSourcePath));
// now make d exist too and pretend a job came in to process it:
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Job, QString("tempdata\n"))); // create file D
AssetProcessorManager::JobToProcessEntry job3;
job3.m_sourceFileInfo.m_databasePath = "redirected/d.txt";
job3.m_sourceFileInfo.m_pathRelativeToScanFolder = "d.txt";
job3.m_sourceFileInfo.m_scanFolder = scanFolder;
job3.m_sourceFileInfo.m_uuid = uuidOfD;
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfD] = { watchFolderPath, "d.txt", "redirected/d.txt" };
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job3);
// all files should now be present:
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_Any, false);
EXPECT_EQ(deps.size(), 5);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end());
}
TEST_F(AssetProcessorManagerTest, UpdateSourceFileDependenciesDatabase_WithOutputPrefix_MissingFiles_ByName_UpdatesWhenTheyAppear)
{
// this test makes sure that when files DO appear that were previously placeholders, the database is updated
// so the strategy here is to have files a, and c missing, which are declared as dependencies by name.
// then, we make them re-appear later, and check that the database has updated them appropriately.
AZ::Uuid dummyBuilderUUID = AZ::Uuid::CreateRandom();
QDir tempPath(m_tempDir.path());
QString absPath(tempPath.absoluteFilePath("subfolder2/outputprefixtest.txt"));
UnitTestUtils::CreateDummyFile(absPath);
QString inputDatabasePath = "redirected/outputprefixtest.txt";
QString watchFolderPath = tempPath.absoluteFilePath("subfolder2");
const ScanFolderInfo* scanFolder = m_config->GetScanFolderByPath(watchFolderPath);
ASSERT_NE(scanFolder, nullptr);
// the above file (assetProcessorManagerTest.txt) will depend on these four files:
QString dependsOnFile1_Source = tempPath.absoluteFilePath("subfolder2/a.txt");
QString dependsOnFile2_Source = tempPath.absoluteFilePath("subfolder2/b.txt");
QString dependsOnFile1_Job = tempPath.absoluteFilePath("subfolder2/c.txt");
QString dependsOnFile2_Job = tempPath.absoluteFilePath("subfolder2/d.txt");
// in this case, we are only creating file b, and d, which are addressed by UUID.
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Source, QString("tempdata\n")));
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile2_Job, QString("tempdata\n")));
// construct the dummy job to feed to the database updater function:
AssetProcessorManager::JobToProcessEntry job;
job.m_sourceFileInfo.m_databasePath = inputDatabasePath;
job.m_sourceFileInfo.m_pathRelativeToScanFolder = "outputprefixtest.txt";
job.m_sourceFileInfo.m_scanFolder = scanFolder;
job.m_sourceFileInfo.m_uuid = AssetUtilities::CreateSafeSourceUUIDFromName(job.m_sourceFileInfo.m_databasePath.toUtf8().data());
// note that we have to "prime" the map with the UUIDs to the source info for this to work:
AZ::Uuid uuidOfB = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/b.txt");
AZ::Uuid uuidOfD = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/d.txt");
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfB] = { watchFolderPath, "b.txt", "redirected/b.txt" };
m_assetProcessorManager->m_sourceUUIDToSourceInfoMap[uuidOfD] = { watchFolderPath, "d.txt", "redirected/d.txt" };
// each file we will take a different approach to publishing: rel path, and UUID:
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "a.txt", AZ::Uuid::CreateNull() }));
job.m_sourceFileDependencies.push_back(AZStd::make_pair<AZ::Uuid, AssetBuilderSDK::SourceFileDependency>(dummyBuilderUUID, { "", uuidOfB }));
// it is currently assumed that the only fields that we care about in JobDetails is the builder busId and the job dependencies themselves:
JobDetails newDetails;
newDetails.m_assetBuilderDesc.m_busId = dummyBuilderUUID;
AssetBuilderSDK::SourceFileDependency dep1 = { "c.txt", AZ::Uuid::CreateNull() };
AssetBuilderSDK::JobDependency jobDep1("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep1);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep1));
AssetBuilderSDK::SourceFileDependency dep2 = { "",uuidOfD };
AssetBuilderSDK::JobDependency jobDep2("pc build", "pc", AssetBuilderSDK::JobDependencyType::Order, dep2);
newDetails.m_jobDependencyList.push_back(JobDependencyInternal(jobDep2));
job.m_jobsToAnalyze.push_back(newDetails);
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job);
// so at this point, the database should be in the same state as after the UpdateSourceFileDependenciesDatabase_MissingFiles_ByUuid test
// which was already verified, by that test.
// now that the database has placeholders, we expect them to resolve themselves when we provide the actual files:
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Source, QString("tempdata\n")));
// now that A exists, we pretend a job came in to process a. (it doesn't require dependencies to be declared)
AZ::Uuid uuidOfA = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/a.txt");
AssetProcessorManager::JobToProcessEntry job2;
job2.m_sourceFileInfo.m_databasePath = "redirected/a.txt";
job2.m_sourceFileInfo.m_pathRelativeToScanFolder = "a.txt";
job2.m_sourceFileInfo.m_scanFolder = scanFolder;
job2.m_sourceFileInfo.m_uuid = uuidOfA;
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job2);
// a should no longer be a placeholder
AssetProcessor::SourceFilesForFingerprintingContainer deps;
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_SourceToSource, false);
EXPECT_EQ(deps.size(), 3);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end()); // a
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end()); // b
deps.clear();
// c should still be a placeholder and thus should not appear in the results.
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_JobToJob, false);
EXPECT_EQ(deps.size(), 2);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end()); // d
// in addition, we expect to have the original file that depends on A appear in the analysis queue, since something it depends on appeared:
QString normalizedSourcePath = AssetUtilities::NormalizeFilePath(absPath);
EXPECT_TRUE(m_assetProcessorManager->m_alreadyActiveFiles.contains(normalizedSourcePath));
// now make c exist too and pretend a job came in to process it:
ASSERT_TRUE(UnitTestUtils::CreateDummyFile(dependsOnFile1_Job, QString("tempdata\n")));
AZ::Uuid uuidOfC = AssetUtilities::CreateSafeSourceUUIDFromName("redirected/c.txt");
AssetProcessorManager::JobToProcessEntry job3;
job3.m_sourceFileInfo.m_databasePath = "redirected/c.txt";
job3.m_sourceFileInfo.m_pathRelativeToScanFolder = "c.txt";
job3.m_sourceFileInfo.m_scanFolder = scanFolder;
job3.m_sourceFileInfo.m_uuid = uuidOfC;
m_assetProcessorManager.get()->UpdateSourceFileDependenciesDatabase(job3);
// all files should now be present:
deps.clear();
m_assetProcessorManager.get()->QueryAbsolutePathDependenciesRecursive(inputDatabasePath, deps, AzToolsFramework::AssetDatabase::SourceFileDependencyEntry::DEP_Any, false);
EXPECT_EQ(deps.size(), 5);
EXPECT_NE(deps.find(absPath.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Source.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile1_Job.toUtf8().constData()), deps.end());
EXPECT_NE(deps.find(dependsOnFile2_Job.toUtf8().constData()), deps.end());
}
TEST_F(AssetProcessorManagerTest, SourceFile_With_NonASCII_Characters_Fail_Job_OK)
{
// This test ensures that asset processor manager detects a source file that has non-ASCII characters
Code/Tools/AssetProcessor/native/tests/assetscanner/AssetScannerTests.cpp
+4 -4
View File
@@ -74,10 +74,10 @@ namespace AssetProcessor
m_platformConfig.reset(new AssetProcessor::PlatformConfiguration());
AZStd::vector<AssetBuilderSDK::PlatformInfo> platforms;
m_platformConfig.get()->PopulatePlatformsForScanFolder(platforms);
// PATH DisplayName PortKey outputfolder root recurse platforms
m_platformConfig.get()->AddScanFolder(ScanFolderInfo(tempPath.absolutePath(), "", "ap1", "", true, false, platforms));
m_platformConfig.get()->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "", "ap2", "", false, true, platforms));
m_platformConfig.get()->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "", "ap3", "", false, true, platforms));
// PATH DisplayName PortKey root recurse platforms
m_platformConfig.get()->AddScanFolder(ScanFolderInfo(tempPath.absolutePath(), "", "ap1", true, false, platforms));
m_platformConfig.get()->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder1"), "", "ap2", false, true, platforms));
m_platformConfig.get()->AddScanFolder(ScanFolderInfo(tempPath.filePath("subfolder2"), "", "ap3", false, true, platforms));
m_assetScanner.reset(new AssetScanner_Test(m_platformConfig.get()));
Code/Tools/AssetProcessor/native/tests/platformconfiguration/platformconfigurationtests.cpp
+18 -23
View File
@@ -170,7 +170,7 @@ TEST_F(PlatformConfigurationUnitTests, TestReadScanFolderRoot_FromSettingsRegist
EXPECT_TRUE(settingsRegistry->Get(scanOrder, AZ::SettingsRegistryInterface::FixedValueString(AssetProcessor::AssetProcessorSettingsKey)
+ "/ScanFolder SettingsRegistryTest/order"));
// These test values come from the <dev_root>/Engine/Registry/AssetProcessorPlatformConfig.setreg file
// These test values come from the <dev_root>/Registry/AssetProcessorPlatformConfig.setreg file
EXPECT_STREQ("_TestPath", watchPath.c_str());
EXPECT_FALSE(recurseScanFolder);
EXPECT_EQ(20000, scanOrder);
@@ -200,7 +200,7 @@ public:
m_config.reset();
PlatformConfigurationUnitTests::TearDown();
}
AZStd::vector<AssetBuilderSDK::PlatformInfo> m_platforms;
AZStd::unique_ptr<UnitTestPlatformConfiguration> m_config;
AZStd::unique_ptr<QTemporaryDir> m_tempEngineRoot = nullptr; // this actually creates the folder in its constructor, so hold off until setup..
@@ -213,8 +213,8 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, GetScanFolderForFile_Roo
using namespace AzToolsFramework::AssetSystem;
using namespace AssetProcessor;
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1"), "ScanFolder1", "sf1", "", true, false, m_platforms), true); // a root folder that has watched subfolders, not recursive
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1/Editor"), "Editor", "sf2", "", false, true, m_platforms), true); // a child folder that exists within that scan folder.
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1"), "ScanFolder1", "sf1", true, false, m_platforms), true); // a root folder that has watched subfolders, not recursive
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1/Editor"), "Editor", "sf2", false, true, m_platforms), true); // a child folder that exists within that scan folder.
const ScanFolderInfo* info = m_config->GetScanFolderForFile(m_tempPath.filePath("scanfolder1/something.txt"));
@@ -229,8 +229,8 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, GetScanFolderForFile_Sub
using namespace AzToolsFramework::AssetSystem;
using namespace AssetProcessor;
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1"), "ScanFolder1", "sf1", "", true, false, m_platforms), true); // a root folder that has watched subfolders, not recursive
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1/Editor"), "Editor ScanFolder", "sf2", "", false, true, m_platforms), true); // a child folder that exists within that scan folder.
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1"), "ScanFolder1", "sf1", true, false, m_platforms), true); // a root folder that has watched subfolders, not recursive
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1/Editor"), "Editor ScanFolder", "sf2", false, true, m_platforms), true); // a child folder that exists within that scan folder.
const ScanFolderInfo* info = m_config->GetScanFolderForFile(m_tempPath.filePath("scanfolder1/Editor/something.txt"));
@@ -254,8 +254,8 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, GetOverridingFile_Exists
QString differentCaseDummyFileName = m_tempPath.absoluteFilePath("scanfolder2/testcase.txt");
UnitTestUtils::CreateDummyFile(caseSensitiveDummyFileName, QString("testcase1\n"));
UnitTestUtils::CreateDummyFile(differentCaseDummyFileName, QString("testcase2\n"));
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", false, true, m_platforms), true);
// Perform the test by asking it whether anyone overrides "testcase" (lowercase) in scanfolder 2.
QString overrider = m_config->GetOverridingFile("testcase.txt", scanfolder2Path);
@@ -278,9 +278,9 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, GetOverridingFile_Exists
QString differentCaseDummyFileName = m_tempPath.absoluteFilePath("scanfolder2/testcase.txt");
UnitTestUtils::CreateDummyFile(caseSensitiveDummyFileName, QString("testcase1\n"));
UnitTestUtils::CreateDummyFile(differentCaseDummyFileName, QString("testcase2\n"));
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", false, true, m_platforms), true);
// Perform the test by asking it whether the existing real winning file is being overridden by anyone.
QString overrider = m_config->GetOverridingFile("TestCase.tXt", scanfolder1Path);
@@ -297,8 +297,8 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, GetOverridingFile_DoesNo
QString scanfolder1Path = m_tempPath.filePath("scanfolder1");
QString scanfolder2Path = m_tempPath.filePath("scanfolder2");
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", false, true, m_platforms), true);
// Perform the test by asking it whether anyone overrides "testcase" (lowercase) in scanfolder 2.
QString overrider = m_config->GetOverridingFile("doesntExist.txt", scanfolder2Path);
@@ -314,8 +314,8 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, FindFirstMatchingFile_Do
// create two scan folders, since its order dependent, the ScanFolder1 is the "winner" in tie breakers.
QString scanfolder1Path = m_tempPath.filePath("scanfolder1");
QString scanfolder2Path = m_tempPath.filePath("scanfolder2");
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", "", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder1Path, "ScanFolder1", "sf1", false, true, m_platforms), true);
m_config->AddScanFolder(ScanFolderInfo(scanfolder2Path, "ScanFolder2", "sf2", false, true, m_platforms), true);
// Perform the test by asking it whether anyone overrides "testcase" (lowercase) in scanfolder 2.
QString foundFile = m_config->FindFirstMatchingFile("doesntExist.txt");
@@ -325,7 +325,7 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, FindFirstMatchingFile_Do
// note that we do not guarantee that FindFirstMatchingFile always returns the correct case, as it is a super hot path
// function, and the only time case could be incorrect is in the situation where a file with different case overrides
// an underlying file, ie,
// an underlying file, ie,
// Engine/EngineAssets/Textures/StartScreen.tif
// MyGame/EngineAssets/textures/startscreen.tif <-- would override the above because game has higher / more important priority.
@@ -335,8 +335,8 @@ TEST_F(PlatformConfigurationUnitTests_OnePCHostFixture, GetScanFolderForFile_Sub
using namespace AzToolsFramework::AssetSystem;
using namespace AssetProcessor;
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1"), "ScanFolder1", "sf1", "", true, false, m_platforms), true); // a root folder that has watched subfolders, not recursive
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1/Editor"), "Editor ScanFolder", "sf2", "", false, true, m_platforms), true); // a child folder that exists within that scan folder.
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1"), "ScanFolder1", "sf1", true, false, m_platforms), true); // a root folder that has watched subfolders, not recursive
m_config->AddScanFolder(ScanFolderInfo(m_tempPath.filePath("scanfolder1/Editor"), "Editor ScanFolder", "sf2", false, true, m_platforms), true); // a child folder that exists within that scan folder.
const ScanFolderInfo* info = m_config->GetScanFolderForFile(m_tempPath.filePath("scanfolder1/Editor"));
ASSERT_TRUE(info);
@@ -363,20 +363,17 @@ TEST_F(PlatformConfigurationUnitTests, TestFailReadConfigFile_RegularScanfolder)
ASSERT_EQ(config.GetScanFolderCount(), 3); // the two, and then the one that has the same data as prior but different identifier.
QString scanName = AssetUtilities::ComputeProjectPath() + " Scan Folder";
ASSERT_EQ(config.GetScanFolderAt(0).GetDisplayName(), scanName);
ASSERT_EQ(config.GetScanFolderAt(0).GetOutputPrefix(), QString());
ASSERT_EQ(config.GetScanFolderAt(0).RecurseSubFolders(), true);
ASSERT_EQ(config.GetScanFolderAt(0).GetOrder(), 0);
ASSERT_EQ(config.GetScanFolderAt(0).GetPortableKey(), QString("Game"));
ASSERT_EQ(config.GetScanFolderAt(1).GetDisplayName(), QString("FeatureTests"));
ASSERT_EQ(config.GetScanFolderAt(1).GetOutputPrefix(), QString("featuretestsoutputfolder")); // to prove its not related to display name
ASSERT_EQ(config.GetScanFolderAt(1).RecurseSubFolders(), false);
ASSERT_EQ(config.GetScanFolderAt(1).GetOrder(), 5000);
// this proves that the featuretests name is used instead of the output prefix
ASSERT_EQ(config.GetScanFolderAt(1).GetPortableKey(), QString("FeatureTests"));
ASSERT_EQ(config.GetScanFolderAt(2).GetDisplayName(), QString("FeatureTests2"));
ASSERT_EQ(config.GetScanFolderAt(2).GetOutputPrefix(), QString("featuretestsoutputfolder")); // to prove its not related to display name
ASSERT_EQ(config.GetScanFolderAt(2).RecurseSubFolders(), false);
ASSERT_EQ(config.GetScanFolderAt(2).GetOrder(), 6000);
// this proves that the featuretests name is used instead of the output prefix
@@ -613,7 +610,6 @@ TEST_F(PlatformConfigurationUnitTests, Test_GemHandling)
ASSERT_EQ(2, config.GetScanFolderCount());
EXPECT_FALSE(config.GetScanFolderAt(0).IsRoot());
EXPECT_TRUE(config.GetScanFolderAt(0).GetOutputPrefix().isEmpty());
EXPECT_TRUE(config.GetScanFolderAt(0).RecurseSubFolders());
// the first one is a game gem, so its order should be above 1 but below 100.
EXPECT_GE(config.GetScanFolderAt(0).GetOrder(), 100);
@@ -623,7 +619,6 @@ TEST_F(PlatformConfigurationUnitTests, Test_GemHandling)
expectedScanFolder = tempPath.absoluteFilePath("Gems/LmbrCentral/v2/Assets");
EXPECT_FALSE(config.GetScanFolderAt(1).IsRoot() );
EXPECT_TRUE(config.GetScanFolderAt(1).GetOutputPrefix().isEmpty());
EXPECT_TRUE(config.GetScanFolderAt(1).RecurseSubFolders());
EXPECT_GT(config.GetScanFolderAt(1).GetOrder(), config.GetScanFolderAt(0).GetOrder());
EXPECT_EQ(0, config.GetScanFolderAt(1).ScanPath().compare(expectedScanFolder, Qt::CaseInsensitive));
Code/Tools/AssetProcessor/native/tests/resourcecompiler/RCControllerTest.cpp
-51
View File
@@ -247,54 +247,3 @@ TEST_F(RCcontrollerTest_Simple, SameJobIsCompletedMultipleTimes_CompletesWithout
ASSERT_EQ(m_errorAbsorber->m_numAssertsAbsorbed, 4); // Expected that there are 4 errors related to the files not existing on disk. Error message: GenerateFingerprint was called but no input files were requested for fingerprinting.
ASSERT_EQ(m_errorAbsorber->m_numErrorsAbsorbed, 0);
}
// makes sure to expose parts of RCJob to the unit test
class TestRCJob : public AssetProcessor::RCJob
{
friend class GTEST_TEST_CLASS_NAME_(RCcontrollerTest, BuilderSDK_API_ProcessJob_HasValidParameters_WithOutputFolder);
public:
explicit TestRCJob(QObject* parent = 0) : AssetProcessor::RCJob(parent) {};
};
TEST_F(RCcontrollerTest, BuilderSDK_API_ProcessJob_HasValidParameters_WithOutputFolder)
{
// this test makes sure that the BuilderSDK API is not exposed to any database internals.
AZ::Uuid sourceUUID = AZ::Uuid::CreateRandom();
AZ::Uuid builderGuid = AZ::Uuid::CreateRandom();
AssetBuilderSDK::ProcessJobRequest req;
{
// note that this scope is intentional. job.Init(jobDetails) below is actually a by-ref operation that destroys JobDetails in the process.
AssetProcessor::JobDetails jobDetails;
// the crux of this test: the database source name differs from the relative path to watch folder:
jobDetails.m_jobEntry.m_pathRelativeToWatchFolder = "SomeThing.tif"; // case sensitive
// Note that while this is an OS-SPECIFIC path, this unit test does not actually invoke the file system at all
// and only operates on in-memory structures, so it should work on every platform.
jobDetails.m_jobEntry.m_watchFolderPath = "c:/test/a/B/c"; // just to make sure case is preserved
jobDetails.m_jobEntry.m_databaseSourceName = "somepath/SomeThing.tif"; // case sensitive but outputprefixes are generally lowcase
jobDetails.m_jobEntry.m_sourceFileUUID = sourceUUID;
jobDetails.m_jobEntry.m_platformInfo = { "ios",{ "mobile", "renderer" } };
jobDetails.m_jobEntry.m_jobRunKey = 1;
jobDetails.m_jobEntry.m_jobKey = "tiff";
jobDetails.m_jobEntry.m_builderGuid = builderGuid;
TestRCJob job;
job.Init(jobDetails);
job.PopulateProcessJobRequest(req);
}
EXPECT_STREQ(req.m_sourceFile.c_str(), "SomeThing.tif");
EXPECT_STREQ(req.m_watchFolder.c_str(), "c:/test/a/B/c");
// the crux of the test, tested: make sure that it does not contain 'somepath' in there just becuase its part of the Database Source Name.
EXPECT_STREQ(req.m_fullPath.c_str(), "c:/test/a/B/c/SomeThing.tif");
EXPECT_EQ(req.m_builderGuid, builderGuid);
EXPECT_TRUE(req.m_platformInfo.HasTag("renderer"));
EXPECT_TRUE(req.m_platformInfo.HasTag("mobile"));
EXPECT_STREQ(req.m_platformInfo.m_identifier.c_str(), "ios");
EXPECT_EQ(req.m_sourceFileUUID, sourceUUID);
}