birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Asset catalog lock inversion fix. (#7045)

Browse Source 
* Fix lock inversion.
This method was calling AssetCatalog->AssetManager at the same time that loading threads are calling AssetManager->AssetCatalog, causing a deadlock due to lock inversion. The fix is to make this method call AssetManager *outside* of the AssetCatalog call.

Signed-off-by: Mike Balfour <82224783+mbalfour-amzn@users.noreply.github.com>

* Fixing the root cause in EnumerateAssets.
Also added a unit test that failed with the previous EnumerateAssets logic, and succeeds with the new logic.

Signed-off-by: Mike Balfour <82224783+mbalfour-amzn@users.noreply.github.com>

* Make sure not to hold the secondary mutex lock either.

Signed-off-by: Mike Balfour <82224783+mbalfour-amzn@users.noreply.github.com>

* Remove unused alias.

Signed-off-by: Mike Balfour <82224783+mbalfour-amzn@users.noreply.github.com>
This commit is contained in:
Mike Balfour
2022-01-21 10:34:00 -06:00
committed by GitHub
parent 9328f053bd
commit 145e3646a4
2 changed files with 132 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Framework/AzFramework/AzFramework/Asset/AssetCatalog.cpp
+27 -2
View File
@@ -489,6 +489,21 @@ namespace AzFramework
//=========================================================================
void AssetCatalog::EnumerateAssets(BeginAssetEnumerationCB beginCB, AssetEnumerationCB enumerateCB, EndAssetEnumerationCB endCB)
{
using AssetCatalogRequestBusContext = typename AZ::Data::AssetCatalogRequestBus::Context;
// Setting trackCallstack to true causes the context mutex to attempt to re-lock.
// That is being avoided here as the code only wants to unlock the Context mutex if it is in a dispatch.
constexpr bool trackCallstack = false;
AssetCatalogRequestBusContext* assetCatalogContext = AZ::Data::AssetCatalogRequestBus::GetContext(trackCallstack);
bool hasAssetCatalogMutex = false;
if (assetCatalogContext != nullptr && AZ::Data::AssetCatalogRequestBus::IsInDispatchThisThread(assetCatalogContext))
{
hasAssetCatalogMutex = true;
// Unlock the dispatch mutex for the AssetCatalogRequestBus
assetCatalogContext->m_contextMutex.unlock();
}
if (beginCB)
{
beginCB();
@@ -496,9 +511,13 @@ namespace AzFramework
if (enumerateCB)
{
AZStd::lock_guard<AZStd::recursive_mutex> lock(m_registryMutex);
// Make sure we don't hold on to any locks during the enumerateCB, so copy the registry info to a local variable
// and unlock the registryMutex before calling the callback.
m_registryMutex.lock();
auto assetIdToInfoCopy = m_registry->m_assetIdToInfo;
m_registryMutex.unlock();
for (auto& it : m_registry->m_assetIdToInfo)
for (auto& it : assetIdToInfoCopy)
{
enumerateCB(it.first, it.second);
}
@@ -508,6 +527,12 @@ namespace AzFramework
{
endCB();
}
if (hasAssetCatalogMutex)
{
// Relock the mutex if it was unlocked earlier
assetCatalogContext->m_contextMutex.lock();
}
}
//=========================================================================