birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

AP: product dependency optimization (#6619)

Browse Source 
* Initial pass at optimizing product path dependency resolution

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add version of StripAssetPlatform that doesn't allocate or copy strings.  Re-add missing test and fix up compile errors.  Add benchmark test

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Change UpdateProductDependencies to directly call s_InsertProductDependencyQuery.BindAndStep

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add test for same filename on multiple platforms

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Rework search logic to keep track of the source of a search path (source vs product) and keep track of which search matches which dependency to avoid doing another search through every product later on

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Clean up code, expand test

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Fix paths not being lowercased by SanitizeForDatabase.  Fix UpdateProductDependencies not updating existing dependencies

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add test for duplicate dependency matches.  Fix saving duplicates

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Clean up code

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Separate test into test and benchmark versions

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Cleanup include

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Fix includes, switch hardcoded job manager setup to use JobManagerComponent instead

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Replaced wildcard_match with PathView::Match.  Changed StripAssetPlatformNoCopy to use TokenizeNext.  Removed Environment Create/Destroy calls.  Made ScopedAllocatorFixture a base class of ScopedAllocatorSetupFixture

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add AZ Environment create/destroy on AP test environment

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add missing asserts on database functions

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Fix incorrect usage of StripAssetPlatformNoCopy

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Fix source/product dependency type being ignored.  Removed need for unordered_set for list of resolved dependencies.  Updated unit tests

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Better variable names

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Remove testing code

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Fix missing includes and namespaces

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
This commit is contained in:
amzn-mike
2022-01-13 08:56:24 -06:00
committed by GitHub
parent 6d1a2382e8
commit fed1278fe6
14 changed files with 742 additions and 237 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Tools/AssetProcessor/native/utilities/PlatformConfiguration.cpp
+8 -6
View File
@@ -51,7 +51,7 @@ namespace AssetProcessor
case AZ::SettingsRegistryInterface::VisitAction::Begin:
{
// Only continue traversal if the path is exactly the AssetProcessorSettingsKey (which indicates the start of traversal)
// or if a "Platform *" object and it's children are being traversed
// or if a "Platform *" object and it's children are being traversed
if (jsonPath == AssetProcessorSettingsKey)
{
return AZ::SettingsRegistryInterface::VisitResponse::Continue;
@@ -631,7 +631,7 @@ namespace AssetProcessor
for (const AssetBuilderSDK::PlatformInfo& platform : m_enabledPlatforms)
{
AZStd::string_view currentRCParams = assetRecognizer.m_defaultParams;
// The "/Amazon/AssetProcessor/Settings/RC */<platform>" entry will be queried
// The "/Amazon/AssetProcessor/Settings/RC */<platform>" entry will be queried
AZ::IO::Path overrideParamsKey = AZ::IO::Path(AZ::IO::PosixPathSeparator);
overrideParamsKey /= path;
overrideParamsKey /= platform.m_identifier;
@@ -644,7 +644,7 @@ namespace AssetProcessor
}
else
{
// otherwise check for tags associated with the platform
// otherwise check for tags associated with the platform
for (const AZStd::string& tag : platform.m_tags)
{
overrideParamsKey.ReplaceFilename(AZ::IO::PathView(tag));
@@ -1416,6 +1416,8 @@ namespace AssetProcessor
return QString();
}
auto* fileStateInterface = AZ::Interface<AssetProcessor::IFileStateRequests>::Get();
for (int pathIdx = 0; pathIdx < m_scanFolders.size(); ++pathIdx)
{
AssetProcessor::ScanFolderInfo scanFolderInfo = m_scanFolders[pathIdx];
@@ -1430,7 +1432,7 @@ namespace AssetProcessor
QDir rooted(scanFolderInfo.ScanPath());
QString absolutePath = rooted.absoluteFilePath(tempRelativeName);
AssetProcessor::FileStateInfo fileStateInfo;
auto* fileStateInterface = AZ::Interface<AssetProcessor::IFileStateRequests>::Get();
if (fileStateInterface)
{
if (fileStateInterface->GetFileInfo(absolutePath, &fileStateInfo))
@@ -1499,7 +1501,7 @@ namespace AssetProcessor
QRegExp nameMatch{ posixRelativeName, Qt::CaseInsensitive, QRegExp::Wildcard };
AZStd::stack<QString> dirs;
dirs.push(sourceFolderDir.absolutePath());
while (!dirs.empty())
{
QString absolutePath = dirs.top();
@@ -1528,7 +1530,7 @@ namespace AssetProcessor
continue;
}
}
QString pathMatch{ sourceFolderDir.relativeFilePath(dirIterator.filePath()) };
if (nameMatch.exactMatch(pathMatch))
{