birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Moved the Asset Catalog loading from LmbrCentral to the AzFramework::Application (#4568)

Browse Source 
* Moved the loading of the AssetCatalog from LmbrCentralSystemComponent to AzFramework Application

Modified the AssetCatalog::InitializeCatalog function to no longer rely on the TickBus to send out the `AssetCatalogEventBus::OnCatalogLoaded` event.
It now queues a function on the AssetCatalogRequestBus to send the OnCatalogLoaded event as soon as the dispatching for the AssetCatalogRequestBus has completed on the current thread.
This is done by updating the AssetCatalogRequestBus to use EBus ThreadDispatchPolicy to add a callback to invoke any queued function has soon a thread has finished dispatching and has released its DispatchMutex

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AssetCatalogRequestBus to add a custom DispatchLockGuard

The AssetCatalogRequestBus uses the custom lock guard to dispatch queued
events after it has unlocked it's context mutex for the current thread.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed GetContext call from the
AssetCatalogRequests::PostThreadDispatchInvoker

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the definition of FileTagQueryManager::GetDefaultFileTagFilePath
function to return a path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AZ_CONSOLEFREEFUNC macro to actually use the _NAME

The _NAME parameter was not being used before, resulting in the Console
stringified name of the function being used.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed CrySystem dependencies from the BundlingSystemComponent

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Moved the loading of the AssetCatalog from LmbrCentralSystemComponent to AzFramework Application

Modified the AssetCatalog::InitializeCatalog function to no longer rely on the TickBus to send out the `AssetCatalogEventBus::OnCatalogLoaded` event.
It now queues a function on the AssetCatalogRequestBus to send the OnCatalogLoaded event as soon as the dispatching for the AssetCatalogRequestBus has completed on the current thread.
This is done by updating the AssetCatalogRequestBus to use EBus ThreadDispatchPolicy to add a callback to invoke any queued function has soon a thread has finished dispatching and has released its DispatchMutex

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AssetCatalogRequestBus to add a custom DispatchLockGuard

The AssetCatalogRequestBus uses the custom lock guard to dispatch queued
events after it has unlocked it's context mutex for the current thread.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed GetContext call from the
AssetCatalogRequests::PostThreadDispatchInvoker

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the definition of FileTagQueryManager::GetDefaultFileTagFilePath
function to return a path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AZ_CONSOLEFREEFUNC macro to actually use the _NAME

The _NAME parameter was not being used before, resulting in the Console
stringified name of the function being used.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed CrySystem dependencies from the BundlingSystemComponent

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Addded missing template parameter to AssetCatalogRequests

The fixes the compile error.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Adding AssetBus::MultiHandler::BusDisconnect call

The BlastSystemComponent was connecting to the Bus, but not
disconnecting from it, causing an assert to fire to it being a
multi-thread bus

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added support for DataDrive lifecycle events to the ComponentApplication

The events are using the SettingsRegistry NotifyEvent to track when
certain keys are modified to signal handlers.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Corrected invalid JSON creation in ModuleManager::DeactivateEntities

Resolved clang warning about used type alias

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix for dangling reference in lambda registered to the SettingsRegistry
Notifier event

This was causing the EditorPythonBinding tests to crash due to the
following circumstances.

First Python has created an instance of a SettingsRegistryProxy
Second the SettingsRegistry sends an event during the time when the
SettingsRegistryProxy exists.

This issue was exposed due to the ComponentApplication Lifecycle events
using the SettingsRegistry to dispatch during various times of the
application workflow.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
Signed-off-by: Gene Walters <genewalt@amazon.com>
This commit is contained in:
lumberyard-employee-dm
2021-10-26 10:31:31 -05:00
committed by Gene Walters
parent 0c7df470dd
commit ebfea8a7e0
25 changed files with 390 additions and 173 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Framework/AzFramework/AzFramework/Application/Application.cpp
+38 -1
View File
@@ -10,6 +10,7 @@
#include <AzCore/IO/SystemFile.h>
#include <AzCore/Math/Crc.h>
#include <AzCore/Component/ComponentApplication.h>
#include <AzCore/Component/ComponentApplicationLifecycle.h>
#include <AzCore/Component/NonUniformScaleBus.h>
#include <AzCore/Debug/Profiler.h>
#include <AzCore/Memory/MemoryComponent.h>
@@ -120,6 +121,11 @@ namespace AzFramework
m_archiveFileIO = AZStd::make_unique<AZ::IO::ArchiveFileIO>(m_archive.get());
AZ::IO::FileIOBase::SetInstance(m_archiveFileIO.get());
SetFileIOAliases();
// The FileIOAvailable event needs to be registered here as this event is sent out
// before the settings registry has merged the .setreg files from the <engine-root>
// (That happens in MergeSettingsToRegistry
AZ::ComponentApplicationLifecycle::RegisterEvent(*m_settingsRegistry, "FileIOAvailable");
AZ::ComponentApplicationLifecycle::SignalEvent(*m_settingsRegistry, "FileIOAvailable", R"({})");
}
if (auto nativeUI = AZ::Interface<AZ::NativeUI::NativeUIRequests>::Get(); nativeUI == nullptr)
@@ -172,6 +178,8 @@ namespace AzFramework
// Archive classes relies on the FileIOBase DirectInstance to close
// files properly
m_directFileIO.reset();
AZ::ComponentApplicationLifecycle::SignalEvent(*m_settingsRegistry, "FileIOUnavailable", R"({})");
}
void Application::Start(const Descriptor& descriptor, const StartupParameters& startupParameters)
@@ -196,7 +204,25 @@ namespace AzFramework
systemEntity->Activate();
AZ_Assert(systemEntity->GetState() == AZ::Entity::State::Active, "System Entity failed to activate.");
m_isStarted = (systemEntity->GetState() == AZ::Entity::State::Active);
if (m_isStarted = (systemEntity->GetState() == AZ::Entity::State::Active); m_isStarted)
{
if (m_startupParameters.m_loadAssetCatalog)
{
// Start Monitoring Asset changes over the network and load the AssetCatalog
auto StartMonitoringAssetsAndLoadCatalog = [this](AZ::Data::AssetCatalogRequests* assetCatalogRequests)
{
if (AZ::IO::FixedMaxPath assetCatalogPath;
m_settingsRegistry->Get(assetCatalogPath.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_CacheRootFolder))
{
assetCatalogPath /= "assetcatalog.xml";
assetCatalogRequests->LoadCatalog(assetCatalogPath.c_str());
}
};
using AssetCatalogBus = AZ::Data::AssetCatalogRequestBus;
AssetCatalogBus::Broadcast(AZStd::move(StartMonitoringAssetsAndLoadCatalog));
}
}
}
void Application::PreModuleLoad()
@@ -210,6 +236,17 @@ namespace AzFramework
{
if (m_isStarted)
{
if (m_startupParameters.m_loadAssetCatalog)
{
// Stop Monitoring Assets changes
auto StopMonitoringAssets = [](AZ::Data::AssetCatalogRequests* assetCatalogRequests)
{
assetCatalogRequests->StopMonitoringAssets();
};
using AssetCatalogBus = AZ::Data::AssetCatalogRequestBus;
AssetCatalogBus::Broadcast(AZStd::move(StopMonitoringAssets));
}
ApplicationLifecycleEvents::Bus::Broadcast(&ApplicationLifecycleEvents::OnApplicationAboutToStop);
m_pimpl.reset();
Code/Framework/AzFramework/AzFramework/Asset/AssetCatalog.cpp
+8 -7
View File
@@ -565,7 +565,7 @@ namespace AzFramework
if (!bytes.empty())
{
AZStd::shared_ptr < AzFramework::AssetRegistry> prevRegistry;
AZStd::shared_ptr<AzFramework::AssetRegistry> prevRegistry;
if (!m_initialized)
{
// First time initialization may have updates already processed which we want to apply
@@ -589,7 +589,6 @@ namespace AzFramework
AZ_TracePrintf("AssetCatalog", "Loaded registry containing %u assets.\n", m_registry->m_assetIdToInfo.size());
// It's currently possible in tools for us to have received updates from AP which were applied before the catalog was ready to load
// due to CryPak and CrySystem coming online later than our components
if (!m_initialized)
{
ApplyDeltaCatalog(prevRegistry);
@@ -611,12 +610,13 @@ namespace AzFramework
// the mutex. If the listener tries to perform a blocking asset load via GetAsset() / BlockUntilLoadComplete(), the spawned asset
// thread will make a call to the AssetCatalogRequestBus and block on the held mutex. This would cause a deadlock, since the listener
// won't free the mutex until the load is complete.
// So instead, queue the notification until the next tick, so that it doesn't occur within the AssetCatalogRequestBus mutex, and also
// So instead, queue the notification until after the AssetCatalogRequestBus mutex is unlocked for the current thread, and also
// so that the entire AssetCatalog initialization is complete.
AZ::TickBus::QueueFunction([catalogRegistryString = AZStd::string(catalogRegistryFile)]()
{
AssetCatalogEventBus::Broadcast(&AssetCatalogEventBus::Events::OnCatalogLoaded, catalogRegistryString.c_str());
});
auto OnCatalogLoaded = [catalogRegistryString = AZStd::string(catalogRegistryFile)]()
{
AssetCatalogEventBus::Broadcast(&AssetCatalogEventBus::Events::OnCatalogLoaded, catalogRegistryString.c_str());
};
AZ::Data::AssetCatalogRequestBus::QueueFunction(AZStd::move(OnCatalogLoaded));
}
}
@@ -978,6 +978,7 @@ namespace AzFramework
AZStd::lock_guard<AZStd::recursive_mutex> lock(m_registryMutex);
m_registry->Clear();
m_initialized = false;
}
Code/Framework/AzFramework/AzFramework/Asset/AssetRegistry.cpp
+1
View File
@@ -61,6 +61,7 @@ namespace AzFramework
//=========================================================================
void AssetRegistry::Clear()
{
m_assetDependencies = {};
m_assetIdToInfo = AssetIdToInfoMap();
m_assetPathToId = AssetPathToIdMap();
}
Code/Framework/AzFramework/AzFramework/FileTag/FileTag.cpp
+13 -8
View File
@@ -10,11 +10,11 @@
#include <AzCore/IO/FileIO.h>
#include <AzCore/IO/Path/Path.h>
#include <AzCore/IO/SystemFile.h>
#include <AzCore/Settings/SettingsRegistryMergeUtils.h>
#include <AzCore/std/algorithm.h>
#include <AzCore/std/string/wildcard.h>
#include <AzCore/std/string/regex.h>
#include <AzCore/std/string/conversions.h>
#include <AzCore/Utils/Utils.h>
#include <AzCore/XML/rapidxml.h>
#include <AzFramework/API/ApplicationAPI.h>
#include <AzFramework/Asset/FileTagAsset.h>
@@ -89,19 +89,19 @@ namespace AzFramework
bool FileTagManager::Save(FileTagType fileTagType, const AZStd::string& destinationFilePath = AZStd::string())
{
AzFramework::FileTag::FileTagAsset* fileTagAsset = GetFileTagAsset(fileTagType);
AZStd::string filePathToSave = destinationFilePath;
AZ::IO::Path filePathToSave = destinationFilePath;
if (filePathToSave.empty())
{
filePathToSave = FileTagQueryManager::GetDefaultFileTagFilePath(fileTagType);
}
if (!AzFramework::StringFunc::EndsWith(filePathToSave, AzFramework::FileTag::FileTagAsset::Extension()))
if (!filePathToSave.Extension().Native().ends_with(AzFramework::FileTag::FileTagAsset::Extension()))
{
AZ_Error("FileTag", false, "Unable to save tag file (%s). Invalid file extension, file tag can only have (%s) extension.\n", filePathToSave.c_str(), AzFramework::FileTag::FileTagAsset::Extension());
return false;
}
return AZ::Utils::SaveObjectToFile(filePathToSave, AZ::DataStream::StreamType::ST_XML, fileTagAsset);
return AZ::Utils::SaveObjectToFile(filePathToSave.Native(), AZ::DataStream::StreamType::ST_XML, fileTagAsset);
}
AZ::Outcome<AZStd::string, AZStd::string> FileTagManager::AddTagsInternal(AZStd::string filePath, FileTagType fileTagType, AZStd::vector<AZStd::string> fileTags, AzFramework::FileTag::FilePatternType filePatternType)
@@ -239,17 +239,22 @@ namespace AzFramework
QueryFileTagsEventBus::Handler::BusDisconnect();
}
AZStd::string FileTagQueryManager::GetDefaultFileTagFilePath(FileTagType fileTagType)
AZ::IO::Path FileTagQueryManager::GetDefaultFileTagFilePath(FileTagType fileTagType)
{
auto destinationFilePath = AZ::IO::FixedMaxPath(AZ::Utils::GetEnginePath()) / EngineAssetSourceRelPath;
AZ::IO::Path destinationFilePath;
if (auto settingsRegistry = AZ::SettingsRegistry::Get(); settingsRegistry != nullptr)
{
settingsRegistry->Get(destinationFilePath.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_EngineRootFolder);
}
destinationFilePath /= EngineAssetSourceRelPath;
destinationFilePath /= fileTagType == FileTagType::Exclude ? ExcludeFileName : IncludeFileName;
destinationFilePath.ReplaceExtension(AzFramework::FileTag::FileTagAsset::Extension());
return destinationFilePath.String();
return destinationFilePath;
}
bool FileTagQueryManager::Load(const AZStd::string& filePath)
{
AZStd::string fileToLoad = filePath;
AZ::IO::Path fileToLoad = filePath;
if (fileToLoad.empty())
{
fileToLoad = GetDefaultFileTagFilePath(m_fileTagType);
Code/Framework/AzFramework/AzFramework/FileTag/FileTag.h
+2 -1
View File
@@ -11,6 +11,7 @@
#include <AzCore/std/containers/map.h>
#include <AzCore/std/containers/set.h>
#include <AzCore/std/string/string.h>
#include <AzCore/IO/Path/Path_fwd.h>
#include <AzFramework/FileTag/FileTagBus.h>
namespace AzFramework
@@ -88,7 +89,7 @@ namespace AzFramework
/////////////////////////////////////////////////////////////////////////
static AZStd::string GetDefaultFileTagFilePath(FileTagType fileTagType);
static AZ::IO::Path GetDefaultFileTagFilePath(FileTagType fileTagType);
protected:
Code/Framework/AzFramework/AzFramework/FileTag/FileTagComponent.cpp
+3 -1
View File
@@ -16,6 +16,7 @@
#include <AzCore/std/string/wildcard.h>
#include <AzCore/IO/FileIO.h>
#include <AzCore/IO/Path/Path.h>
#include <AzCore/XML/rapidxml.h>
namespace AzFramework
@@ -66,7 +67,8 @@ namespace AzFramework
m_excludeFileQueryManager.reset(aznew FileTagQueryManager(FileTagType::Exclude));
if (!m_excludeFileQueryManager.get()->Load())
{
AZ_Error("FileTagQueryComponent", false, "Not able to load default exclude file (%s). Please make sure that it exists on disk.\n", FileTagQueryManager::GetDefaultFileTagFilePath(FileTagType::Exclude).c_str());
AZ_Error("FileTagQueryComponent", false, "Not able to load default exclude file (%s). Please make sure that it exists on disk.\n",
FileTagQueryManager::GetDefaultFileTagFilePath(FileTagType::Exclude).c_str());
}
AzFramework::AssetCatalogEventBus::Handler::BusConnect();
Code/Framework/AzFramework/Tests/AssetCatalog.cpp
+3 -1
View File
@@ -308,7 +308,9 @@ namespace UnitTest
registry->Set(projectPathKey, "AutomatedTesting");
AZ::SettingsRegistryMergeUtils::MergeSettingsToRegistry_AddRuntimeFilePaths(*registry);
m_app->Start(desc);
AZ::ComponentApplication::StartupParameters startupParameters;
startupParameters.m_loadAssetCatalog = false;
m_app->Start(desc, startupParameters);
// Without this, the user settings component would attempt to save on finalize/shutdown. Since the file is
// shared across the whole engine, if multiple tests are run in parallel, the saving could cause a crash