Update the FileIO Aliases (#4186)

* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear

The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`

The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`

The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated use of devassets and devroot properties in python

Those properties now use projectroot and engroot

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed hardcoded size of 9 for the product asset alias.

The ResolvePath function now just appends the @projectproductassets@
alias with the input path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix for typos in Hydra python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a deprecated alias map to the FileIO System

When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated python test scripts to use the projectroot binding

Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to devroot and devgame within the codebase

The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated <engroot>/AutomatedTesting references to projectroot


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Correct casing in emfxworkspace file


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance

The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed unused parameter warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Undid change to the LexicallyProximate function to set the path result to the base path.

It needs to return the *this path if the pathResult is empty

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils

Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Changed the AWSMetricsGemMock to use a temporary for writing test assets

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"

IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed whitespacing issues in Application.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to the Project Asset Cache in the PythonTests.

Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed the FileIO Deprecated Alias test on Windows

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removing @projectsourceassets@ alias, as it is only used once.

Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced @projectproductassets@ alias with @products@

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Rollback changes to the PhysX EditorSystemComponent.cpp

The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
This commit is contained in:
lumberyard-employee-dm
2021-10-05 15:12:53 -05:00
committed by GitHub
parent 643bd84739
commit f648cb1fd8
209 changed files with 1365 additions and 1678 deletions
@@ -27,7 +27,7 @@ namespace AssetProcessor
, m_registryBuiltOnce(false)
, m_registriesMutex(QMutex::Recursive)
{
for (const AssetBuilderSDK::PlatformInfo& info : m_platformConfig->GetEnabledPlatforms())
{
m_platforms.push_back(QString::fromUtf8(info.m_identifier.c_str()));
@@ -38,17 +38,9 @@ namespace AssetProcessor
// save 30mb for this. Really large projects do get this big (and bigger)
// if you don't do this, things get fragmented very fast.
m_saveBuffer.reserve(1024 * 1024 * 30);
m_absoluteDevFolderPath[0] = 0;
m_absoluteDevGameFolderPath[0] = 0;
m_saveBuffer.reserve(1024 * 1024 * 30);
AZStd::string engineRoot;
AzFramework::ApplicationRequests::Bus::BroadcastResult(engineRoot, &AzFramework::ApplicationRequests::GetEngineRoot);
azstrcpy(m_absoluteDevFolderPath, AZ_MAX_PATH_LEN, engineRoot.c_str());
AZStd::string gameFolderPath{AssetUtilities::ComputeProjectPath().toUtf8().constData()};
azstrcpy(m_absoluteDevGameFolderPath, AZ_MAX_PATH_LEN, gameFolderPath.c_str());
AssetUtilities::ComputeProjectPath();
AssetUtilities::ComputeProjectCacheRoot(m_cacheRootDir);
@@ -359,7 +351,7 @@ namespace AssetProcessor
[[maybe_unused]] bool makeDirResult = AZ::IO::SystemFile::CreateDir(absPath.toUtf8().constData());
AZ_Warning(AssetProcessor::ConsoleChannel, makeDirResult, "Failed create folder %s", platformCacheDir.toUtf8().constData());
}
// if we succeeded in doing this, then use "rename" to move the file over the previous copy.
bool moved = AssetUtilities::MoveFileWithTimeout(tempRegistryFile, actualRegistryFile, 3);
allCatalogsSaved = allCatalogsSaved && moved;
@@ -382,7 +374,7 @@ namespace AssetProcessor
}
}
}
{
// scoped to minimize the duration of this mutex lock
QMutexLocker locker(&m_savingRegistryMutex);
@@ -605,7 +597,7 @@ namespace AssetProcessor
AZStd::string nameForMap(relativeFilePath.toUtf8().constData());
AZStd::to_lower(nameForMap.begin(), nameForMap.end());
m_sourceNameToSourceUUIDMap.insert({ nameForMap, sourceUuid });
}
@@ -627,16 +619,6 @@ namespace AssetProcessor
//////////////////////////////////////////////////////////////////////////
const char* AssetCatalog::GetAbsoluteDevGameFolderPath()
{
return m_absoluteDevGameFolderPath;
}
const char* AssetCatalog::GetAbsoluteDevRootFolderPath()
{
return m_absoluteDevFolderPath;
}
bool AssetCatalog::GetRelativeProductPathFromFullSourceOrProductPath(const AZStd::string& fullSourceOrProductPath, AZStd::string& relativeProductPath)
{
ProcessGetRelativeProductPathFromFullSourceOrProductPathRequest(fullSourceOrProductPath, relativeProductPath);
@@ -976,7 +958,7 @@ namespace AssetProcessor
// regardless of which way we come into this function we must always use ConvertToRelativePath
// to convert from whatever the input format is to a database path (which may include output prefix)
QString databaseName;
QString databaseName;
QString scanFolder;
if (!AzFramework::StringFunc::Path::IsRelative(sourcePath))
{
@@ -1163,7 +1145,7 @@ namespace AssetProcessor
AZStd::string AssetCatalog::GetAssetPathById(const AZ::Data::AssetId& id)
{
return GetAssetInfoById(id).m_relativePath;
}
AZ::Data::AssetId AssetCatalog::GetAssetIdByPath(const char* path, const AZ::Data::AssetType& typeToRegister, bool autoRegisterIfNotFound)
@@ -1175,7 +1157,7 @@ namespace AssetProcessor
AZStd::string relProductPath;
GetRelativeProductPathFromFullSourceOrProductPath(path, relProductPath);
QString tempPlatformName = GetDefaultAssetPlatform();
AZ::Data::AssetId assetId;
{
QMutexLocker locker(&m_registriesMutex);
@@ -1344,7 +1326,7 @@ namespace AssetProcessor
//remove aliases if present
normalisedAssetPath = AssetUtilities::NormalizeAndRemoveAlias(normalisedAssetPath);
if (!normalisedAssetPath.isEmpty()) // this happens if it comes in as just for example "@assets@/"
if (!normalisedAssetPath.isEmpty()) // this happens if it comes in as just for example "@products@/"
{
AZStd::lock_guard<AZStd::mutex> lock(m_databaseMutex);
@@ -1439,7 +1421,7 @@ namespace AssetProcessor
relativePath = entry.m_sourceName;
watchFolder = scanEntry.m_scanFolder;
return true;
}
@@ -1489,7 +1471,7 @@ namespace AssetProcessor
{
return foundIter->second;
}
// we did not find it - try the backup mapping!
AssetId legacyMapping = registryToUse.GetAssetIdByLegacyAssetId(assetId);
if (legacyMapping.IsValid())
@@ -1533,7 +1515,7 @@ namespace AssetProcessor
return !assetInfo.m_relativePath.empty();
}
// Asset isn't in the DB or in the APM queue, we don't know what this asset ID is
return false;
}
@@ -1588,7 +1570,7 @@ namespace AssetProcessor
AZStd::string sourceFileFullPath;
AzFramework::StringFunc::Path::Join(watchFolder.c_str(), assetInfo.m_relativePath.c_str(), sourceFileFullPath);
assetInfo.m_sizeBytes = AZ::IO::SystemFile::Length(sourceFileFullPath.c_str());
assetInfo.m_assetType = AZ::Uuid::CreateNull(); // most source files don't have a type!
// Go through the list of source assets and see if this asset's file path matches any of the filters