birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update the FileIO Aliases (#4186)

Browse Source 
* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear

The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`

The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`

The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated use of devassets and devroot properties in python

Those properties now use projectroot and engroot

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed hardcoded size of 9 for the product asset alias.

The ResolvePath function now just appends the @projectproductassets@
alias with the input path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix for typos in Hydra python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a deprecated alias map to the FileIO System

When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated python test scripts to use the projectroot binding

Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to devroot and devgame within the codebase

The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated <engroot>/AutomatedTesting references to projectroot


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Correct casing in emfxworkspace file


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance

The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed unused parameter warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Undid change to the LexicallyProximate function to set the path result to the base path.

It needs to return the *this path if the pathResult is empty

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils

Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Changed the AWSMetricsGemMock to use a temporary for writing test assets

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"

IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed whitespacing issues in Application.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to the Project Asset Cache in the PythonTests.

Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed the FileIO Deprecated Alias test on Windows

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removing @projectsourceassets@ alias, as it is only used once.

Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced @projectproductassets@ alias with @products@

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Rollback changes to the PhysX EditorSystemComponent.cpp

The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
This commit is contained in:
lumberyard-employee-dm
2021-10-05 15:12:53 -05:00
committed by GitHub
parent 643bd84739
commit f648cb1fd8
209 changed files with 1365 additions and 1678 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Editor/Util/PathUtil.cpp
+33 -167
View File
@@ -11,9 +11,9 @@
#include "PathUtil.h"
#include <AzCore/IO/SystemFile.h> // for AZ_MAX_PATH_LEN
#include <AzCore/Settings/SettingsRegistryMergeUtils.h>
#include <AzCore/Utils/Utils.h>
#include <AzToolsFramework/API/EditorAssetSystemAPI.h> // for ebus events
#include <AzFramework/StringFunc/StringFunc.h>
#include <AzFramework/API/ApplicationAPI.h>
#include <AzCore/std/string/conversions.h>
#include <AzFramework/IO/LocalFileIO.h>
@@ -179,7 +179,7 @@ namespace Path
EBUS_EVENT_RESULT(engineRoot, AzFramework::ApplicationRequests::Bus, GetEngineRoot);
return QString(engineRoot);
}
//////////////////////////////////////////////////////////////////////////
QString& ReplaceFilename(const QString& strFilepath, const QString& strFilename, QString& strOutputFilename, bool bCallCaselessPath)
{
@@ -216,30 +216,21 @@ namespace Path
//////////////////////////////////////////////////////////////////////////
QString GetResolvedUserSandboxFolder()
{
char resolvedPath[AZ_MAX_PATH_LEN] = { 0 };
gEnv->pFileIO->ResolvePath(GetUserSandboxFolder().toUtf8().data(), resolvedPath, AZ_MAX_PATH_LEN);
return QString::fromLatin1(resolvedPath);
AZ::IO::FixedMaxPath userSandboxFolderPath;
gEnv->pFileIO->ResolvePath(userSandboxFolderPath, GetUserSandboxFolder().toUtf8().constData());
return QString::fromUtf8(userSandboxFolderPath.c_str(), static_cast<int>(userSandboxFolderPath.Native().size()));
}
// internal function, you should use GetEditingGameDataFolder instead.
AZStd::string GetGameAssetsFolder()
{
const char* resultValue = nullptr;
EBUS_EVENT_RESULT(resultValue, AzToolsFramework::AssetSystemRequestBus, GetAbsoluteDevGameFolderPath);
if (!resultValue)
AZ::IO::Path projectPath;
if (auto settingsRegistry = AZ::SettingsRegistry::Get(); settingsRegistry != nullptr)
{
if ((gEnv) && (gEnv->pFileIO))
{
resultValue = gEnv->pFileIO->GetAlias("@devassets@");
}
settingsRegistry->Get(projectPath.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_ProjectPath);
}
if (!resultValue)
{
resultValue = ".";
}
return resultValue;
return projectPath.Native();
}
/// Get the data folder
@@ -258,26 +249,6 @@ namespace Path
return str;
}
//! Get the root folder (in source control or other writable assets) where you should save root data.
AZStd::string GetEditingRootFolder()
{
const char* resultValue = nullptr;
EBUS_EVENT_RESULT(resultValue, AzToolsFramework::AssetSystemRequestBus, GetAbsoluteDevRootFolderPath);
if (!resultValue)
{
if ((gEnv) && (gEnv->pFileIO))
{
resultValue = gEnv->pFileIO->GetAlias("@devassets@");
}
}
if (!resultValue)
{
resultValue = ".";
}
return resultValue;
}
AZStd::string MakeModPathFromGamePath(const char* relGamePath)
{
@@ -335,165 +306,60 @@ namespace Path
return "";
}
bool relPathfound = false;
bool relPathFound = false;
AZStd::string relativePath;
AZStd::string fullAssetPath(fullPath.toUtf8().data());
EBUS_EVENT_RESULT(relPathfound, AzToolsFramework::AssetSystemRequestBus, GetRelativeProductPathFromFullSourceOrProductPath, fullAssetPath, relativePath);
EBUS_EVENT_RESULT(relPathFound, AzToolsFramework::AssetSystemRequestBus, GetRelativeProductPathFromFullSourceOrProductPath, fullAssetPath, relativePath);
if (relPathfound)
if (relPathFound)
{
// do not normalize this path, it will already be an appropriate asset ID.
return CaselessPaths(relativePath.c_str());
}
char rootpath[_MAX_PATH] = { 0 };
azstrcpy(rootpath, _MAX_PATH, Path::GetEditingRootFolder().c_str());
if (bRelativeToGameFolder)
{
azstrcpy(rootpath, _MAX_PATH, Path::GetEditingGameDataFolder().c_str());
}
QString rootPathNormalized(rootpath);
QString srcPathNormalized(fullPath);
#if defined(AZ_PLATFORM_WINDOWS)
// avoid confusing PathRelativePathTo
rootPathNormalized.replace('/', '\\');
srcPathNormalized.replace('/', '\\');
#endif
AZ::IO::FixedMaxPath rootPath = bRelativeToGameFolder ? AZ::Utils::GetProjectPath() : AZ::Utils::GetEnginePath();
AZ::IO::FixedMaxPath resolvedFullPath;
AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(resolvedFullPath, fullPath.toUtf8().constData());
// Create relative path
char resolvedSrcPath[AZ_MAX_PATH_LEN] = { 0 };
AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(srcPathNormalized.toUtf8().data(), resolvedSrcPath, AZ_MAX_PATH_LEN);
QByteArray path = QDir(rootPathNormalized).relativeFilePath(resolvedSrcPath).toUtf8();
if (path.isEmpty())
{
return fullPath;
}
// The following code is required because the windows PathRelativePathTo function will always return "./SomePath" instead of just "SomePath"
// Only remove single dot (.) and slash parts of a path, never the double dot (..)
const char* pBuffer = path.data();
bool bHasDot = false;
while (*pBuffer && pBuffer != path.end())
{
switch (*pBuffer)
{
case '.':
if (bHasDot)
{
// Found a double dot, rewind and stop removing
pBuffer--;
break;
}
// Fall through intended
case '/':
case '\\':
bHasDot = (*pBuffer == '.');
pBuffer++;
continue;
}
break;
}
QString relPath = pBuffer;
return CaselessPaths(relPath);
return CaselessPaths(resolvedFullPath.LexicallyProximate(rootPath).MakePreferred().c_str());
}
QString GamePathToFullPath(const QString& path)
{
using namespace AzToolsFramework;
AZ_Warning("GamePathToFullPath", path.size() <= AZ_MAX_PATH_LEN, "Path exceeds maximum path length of %d", AZ_MAX_PATH_LEN);
if ((gEnv) && (gEnv->pFileIO) && gEnv->pCryPak && path.size() <= AZ_MAX_PATH_LEN)
AZ_Warning("GamePathToFullPath", path.size() <= AZ::IO::MaxPathLength, "Path exceeds maximum path length of %zu", AZ::IO::MaxPathLength);
if (path.size() <= AZ::IO::MaxPathLength)
{
// first, adjust the file name for mods:
bool fullPathfound = false;
AZStd::string assetFullPath;
AZStd::string adjustedFilePath = path.toUtf8().data();
AssetSystemRequestBus::BroadcastResult(fullPathfound, &AssetSystemRequestBus::Events::GetFullSourcePathFromRelativeProductPath, adjustedFilePath, assetFullPath);
if (fullPathfound)
bool fullPathFound = false;
AZ::IO::Path assetFullPath;
AZ::IO::Path adjustedFilePath = path.toUtf8().constData();
AssetSystemRequestBus::BroadcastResult(fullPathFound, &AssetSystemRequestBus::Events::GetFullSourcePathFromRelativeProductPath,
adjustedFilePath.Native(), assetFullPath.Native());
if (fullPathFound)
{
//if the bus message succeeds than normalize and lowercase the path
AzFramework::StringFunc::Path::Normalize(assetFullPath);
return assetFullPath.c_str();
//if the bus message succeeds than normalize
return assetFullPath.LexicallyNormal().c_str();
}
// if the bus message didn't succeed, 'guess' the source assets:
// if the bus message didn't succeed, check if he path exist as a resolved path
else
{
// Not all systems have been converted to use local paths. Some editor files save XML files directly, and a full or correctly aliased path is already passed in.
// If the path passed in exists already, then return the resolved filepath
if (AZ::IO::FileIOBase::GetDirectInstance()->Exists(adjustedFilePath.c_str()))
{
char resolvedPath[AZ_MAX_PATH_LEN + PathUtil::maxAliasLength] = { 0 };
AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(adjustedFilePath.c_str(), resolvedPath, AZ_MAX_PATH_LEN + PathUtil::maxAliasLength);
return QString::fromUtf8(resolvedPath);
AZ::IO::FixedMaxPath resolvedPath;
AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(resolvedPath, adjustedFilePath);
return QString::fromUtf8(resolvedPath.c_str(), static_cast<int>(resolvedPath.Native().size()));
}
// if we get here it means that the Asset Processor does not know about this file. most of the time we should never get here
// the rest of this code just does a bunch of heuristic guesses in case of missing files or if the user has hand-edited
// the asset cache by moving files in via some other means or external process.
if (adjustedFilePath[0] != '@')
{
const char* prefix = (adjustedFilePath[0] == '/' || adjustedFilePath[0] == '\\') ? "@devassets@" : "@devassets@/";
adjustedFilePath = prefix + adjustedFilePath;
}
char szAdjustedFile[AZ_MAX_PATH_LEN + PathUtil::maxAliasLength] = { 0 };
gEnv->pFileIO->ResolvePath(adjustedFilePath.c_str(), szAdjustedFile, AZ_ARRAY_SIZE(szAdjustedFile));
if ((azstrnicmp(szAdjustedFile, "@devassets@", 11) == 0) && ((szAdjustedFile[11] == '/') || (szAdjustedFile[11] == '\\')))
{
if (!gEnv->pCryPak->IsFileExist(szAdjustedFile))
{
AZStd::string newName(szAdjustedFile);
AzFramework::StringFunc::Replace(newName, "@devassets@", "@devroot@/engine", false);
if (gEnv->pCryPak->IsFileExist(newName.c_str()))
{
azstrcpy(szAdjustedFile, AZ_ARRAY_SIZE(szAdjustedFile), newName.c_str());
}
else
{
// getting tricky here, try @devroot@ alone, in case its 'editor'
AzFramework::StringFunc::Replace(newName, "@devassets@", "@devroot@", false);
if (gEnv->pCryPak->IsFileExist(szAdjustedFile))
{
azstrcpy(szAdjustedFile, AZ_ARRAY_SIZE(szAdjustedFile), newName.c_str());
}
// give up, best guess is just @devassets@
}
}
}
// we should very rarely actually get to this point in the code.
// szAdjustedFile may contain an alias at this point. (@assets@/blah.whatever)
// there is a case in which the loose asset exists only within a pak file for some reason
// this is not recommended but it is possible.in that case, we want to return the original szAdjustedFile
// without touching it or resolving it so that crypak can open it successfully.
char adjustedPath[AZ_MAX_PATH_LEN + PathUtil::maxAliasLength] = { 0 };
if (gEnv->pFileIO->ResolvePath(szAdjustedFile, adjustedPath, AZ_MAX_PATH_LEN + PathUtil::maxAliasLength)) // resolve to full path
{
if ((gEnv->pCryPak->IsFileExist(adjustedPath)) || (!gEnv->pCryPak->IsFileExist(szAdjustedFile)))
{
// note that if we get here, then EITHER
// the file exists as a loose asset in the actual adjusted path
// OR the file does not exist in the original passed-in aliased name (like '@assets@/whatever')
// in which case we may as well just resolve the path to a full path and return it.
assetFullPath = adjustedPath;
AzFramework::StringFunc::Path::Normalize(assetFullPath);
azstrcpy(szAdjustedFile, AZ_MAX_PATH_LEN + PathUtil::maxAliasLength, assetFullPath.c_str());
}
// if the above case succeeded then it means that the file does NOT exist loose
// but DOES exist in a pak, in which case we leave szAdjustedFile with the alias on the front of it, meaning
// fopens via crypak will actually succeed.
}
return szAdjustedFile;
return path;
}
}
else
{
return "";
return QString{};
}
}