birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

{LYN-4224} Fix for the file scan slowdown (#1252)

Browse Source 
* {LYN-4224} Fix for the file scan slowdown (#1183)

* {LYN-4224} Fix for the file scan slowdown

* Fixed a slowdown in the file scanning logic
* Improved the file scanning logic from previous code by 40%

Tests:
Using Testing\Pytest\AutomatedTesting_BlastTest

old code:
=== 7 passed in 96.13s (0:01:36) ===

current code:
=== 7 passed in 160.45s (0:02:40) ====

newest code:
=== 7 passed in 52.91s ===

* fixing a unit test compile error

* unit test fixes

* another file improvement

* fix for legacy level loading taking too long

* making an enum for the search types

* switched the enum to "allow" types to make the input more clear

* got rid of orphaned const variables
This commit is contained in:
jackalbe
2021-06-11 12:25:45 -05:00
committed by GitHub
parent 6584e1290b
commit 4818d1ce80
9 changed files with 79 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Framework/AzFramework/AzFramework/Archive/Archive.cpp
+21 -3
View File
@@ -1290,7 +1290,7 @@ namespace AZ::IO
//////////////////////////////////////////////////////////////////////////
AZ::IO::ArchiveFileIterator Archive::FindFirst(AZStd::string_view pDir, [[maybe_unused]] uint32_t nPathFlags, bool bAllowUseFileSystem)
AZ::IO::ArchiveFileIterator Archive::FindFirst(AZStd::string_view pDir, EFileSearchType searchType)
{
auto szFullPath = AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(pDir);
if (!szFullPath)
@@ -1299,8 +1299,26 @@ namespace AZ::IO
return {};
}
bool bScanZips{};
bool bAllowUseFileSystem{};
switch (searchType)
{
case IArchive::eFileSearchType_AllowInZipsOnly:
bAllowUseFileSystem = false;
bScanZips = true;
break;
case IArchive::eFileSearchType_AllowOnDiskAndInZips:
bAllowUseFileSystem = true;
bScanZips = true;
break;
case IArchive::eFileSearchType_AllowOnDiskOnly:
bAllowUseFileSystem = true;
bScanZips = false;
break;
}
AZStd::intrusive_ptr<AZ::IO::FindData> pFindData = new AZ::IO::FindData();
pFindData->Scan(this, szFullPath->Native(), bAllowUseFileSystem);
pFindData->Scan(this, szFullPath->Native(), bAllowUseFileSystem, bScanZips);
return pFindData->Fetch();
}
@@ -1676,7 +1694,7 @@ namespace AZ::IO
return true;
}
if (AZ::IO::ArchiveFileIterator fileIterator = FindFirst(pWildcardIn, 0, true); fileIterator)
if (AZ::IO::ArchiveFileIterator fileIterator = FindFirst(pWildcardIn, IArchive::eFileSearchType_AllowOnDiskOnly); fileIterator)
{
AZStd::vector<AZStd::string> files;
do