Update the FileIO Aliases (#4186)

* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear

The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`

The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`

The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated use of devassets and devroot properties in python

Those properties now use projectroot and engroot

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed hardcoded size of 9 for the product asset alias.

The ResolvePath function now just appends the @projectproductassets@
alias with the input path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix for typos in Hydra python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a deprecated alias map to the FileIO System

When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated python test scripts to use the projectroot binding

Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to devroot and devgame within the codebase

The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated <engroot>/AutomatedTesting references to projectroot


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Correct casing in emfxworkspace file


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance

The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed unused parameter warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Undid change to the LexicallyProximate function to set the path result to the base path.

It needs to return the *this path if the pathResult is empty

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils

Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Changed the AWSMetricsGemMock to use a temporary for writing test assets

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"

IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed whitespacing issues in Application.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to the Project Asset Cache in the PythonTests.

Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed the FileIO Deprecated Alias test on Windows

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removing @projectsourceassets@ alias, as it is only used once.

Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced @projectproductassets@ alias with @products@

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Rollback changes to the PhysX EditorSystemComponent.cpp

The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
This commit is contained in:
lumberyard-employee-dm
2021-10-05 15:12:53 -05:00
committed by GitHub
parent 643bd84739
commit f648cb1fd8
209 changed files with 1365 additions and 1678 deletions
@@ -77,11 +77,15 @@
namespace AzFramework
{
namespace ApplicationInternal
{
static constexpr const char s_prefabSystemKey[] = "/Amazon/Preferences/EnablePrefabSystem";
static constexpr const char s_prefabWipSystemKey[] = "/Amazon/Preferences/EnablePrefabSystemWipFeatures";
static constexpr const char s_legacySlicesAssertKey[] = "/Amazon/Preferences/ShouldAssertForLegacySlicesUsage";
static constexpr const char* DeprecatedFileIOAliasesRoot = "/O3DE/AzCore/FileIO/DeprecatedAliases";
static constexpr const char* DeprecatedFileIOAliasesOldAliasKey = "OldAlias";
static constexpr const char* DeprecatedFileIOAliasesNewAliasKey = "NewAlias";
}
Application::Application()
@@ -563,6 +567,68 @@ namespace AzFramework
}
}
struct DeprecatedAliasesKeyVisitor
: AZ::SettingsRegistryInterface::Visitor
{
using VisitResponse = AZ::SettingsRegistryInterface::VisitResponse;
using VisitAction = AZ::SettingsRegistryInterface::VisitAction;
using Type = AZ::SettingsRegistryInterface::Type;
using AZ::SettingsRegistryInterface::Visitor::Visit;
VisitResponse Traverse(AZStd::string_view path, AZStd::string_view,
VisitAction action, Type type) override
{
if (action == AZ::SettingsRegistryInterface::VisitAction::Begin)
{
if (type == AZ::SettingsRegistryInterface::Type::Array)
{
m_parentArrayPath = path;
}
// Strip off last path segment from json path and check if is a child element of the array
if (AZ::StringFunc::TokenizeLast(path, '/');
m_parentArrayPath == path)
{
m_aliases.emplace_back();
}
}
else if (action == AZ::SettingsRegistryInterface::VisitAction::End)
{
if (type == AZ::SettingsRegistryInterface::Type::Array)
{
m_parentArrayPath = AZStd::string{};
}
}
return AZ::SettingsRegistryInterface::VisitResponse::Continue;
}
void Visit(AZStd::string_view, AZStd::string_view valueName, Type, AZStd::string_view value) override
{
if (!m_aliases.empty())
{
if (valueName == ApplicationInternal::DeprecatedFileIOAliasesOldAliasKey)
{
m_aliases.back().m_oldAlias = value;
}
else if (valueName == ApplicationInternal::DeprecatedFileIOAliasesNewAliasKey)
{
m_aliases.back().m_newAlias = value;
}
}
}
struct AliasPair
{
AZStd::string m_oldAlias;
AZStd::string m_newAlias;
};
AZStd::vector<AliasPair> m_aliases;
private:
AZStd::string m_parentArrayPath;
};
static void CreateUserCache(const AZ::IO::FixedMaxPath& cacheUserPath, AZ::IO::FileIOBase& fileIoBase)
{
@@ -610,9 +676,8 @@ namespace AzFramework
void Application::SetFileIOAliases()
{
if (m_archiveFileIO)
if (auto fileIoBase = m_archiveFileIO.get(); fileIoBase)
{
auto fileIoBase = m_archiveFileIO.get();
// Set up the default file aliases based on the settings registry
fileIoBase->SetAlias("@engroot@", GetEngineRoot());
fileIoBase->SetAlias("@projectroot@", GetEngineRoot());
@@ -620,29 +685,20 @@ namespace AzFramework
{
AZ::IO::FixedMaxPath pathAliases;
if (m_settingsRegistry->Get(pathAliases.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_CacheProjectRootFolder))
{
fileIoBase->SetAlias("@projectcache@", pathAliases.c_str());
}
pathAliases.clear();
if (m_settingsRegistry->Get(pathAliases.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_CacheRootFolder))
{
fileIoBase->SetAlias("@assets@", pathAliases.c_str());
fileIoBase->SetAlias("@projectplatformcache@", pathAliases.c_str());
fileIoBase->SetAlias("@root@", pathAliases.c_str()); // Deprecated Use @projectplatformcache@
fileIoBase->SetAlias("@products@", pathAliases.c_str());
}
pathAliases.clear();
if (m_settingsRegistry->Get(pathAliases.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_EngineRootFolder))
{
fileIoBase->SetAlias("@engroot@", pathAliases.c_str());
fileIoBase->SetAlias("@devroot@", pathAliases.c_str()); // Deprecated - Use @engroot@
}
pathAliases.clear();
if (m_settingsRegistry->Get(pathAliases.Native(), AZ::SettingsRegistryMergeUtils::FilePathKey_ProjectPath))
{
fileIoBase->SetAlias("@devassets@", pathAliases.c_str()); // Deprecated - Use @projectsourceassets@
fileIoBase->SetAlias("@projectroot@", pathAliases.c_str());
fileIoBase->SetAlias("@projectsourceassets@", (pathAliases / "Assets").c_str());
}
}
@@ -663,6 +719,15 @@ namespace AzFramework
}
fileIoBase->SetAlias("@log@", projectLogPath.c_str());
fileIoBase->CreatePath(projectLogPath.c_str());
DeprecatedAliasesKeyVisitor visitor;
if (m_settingsRegistry->Visit(visitor, ApplicationInternal::DeprecatedFileIOAliasesRoot))
{
for (const auto& [oldAlias, newAlias] : visitor.m_aliases)
{
fileIoBase->SetDeprecatedAlias(oldAlias, newAlias);
}
}
}
}
@@ -1121,7 +1121,7 @@ namespace AZ::IO
if (AZ::IO::FixedMaxPath pathBindRoot; !AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(pathBindRoot, szBindRoot))
{
AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(pathBindRoot, "@assets@");
AZ::IO::FileIOBase::GetDirectInstance()->ResolvePath(pathBindRoot, "@products@");
desc.m_pathBindRoot = pathBindRoot.LexicallyNormal().String();
}
else
@@ -1807,9 +1807,9 @@ namespace AZ::IO
if (m_eRecordFileOpenList != IArchive::RFOM_Disabled)
{
// we only want to record ASSET access
// assets are identified as files that are relative to the resolved @assets@ alias path
// assets are identified as files that are relative to the resolved @products@ alias path
auto fileIoBase = AZ::IO::FileIOBase::GetInstance();
const char* aliasValue = fileIoBase->GetAlias("@assets@");
const char* aliasValue = fileIoBase->GetAlias("@products@");
if (AZ::IO::FixedMaxPath resolvedFilePath;
fileIoBase->ResolvePath(resolvedFilePath, szFilename)
@@ -546,6 +546,16 @@ namespace AZ::IO
realUnderlyingFileIO->GetAlias(alias);
}
void ArchiveFileIO::SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias)
{
FileIOBase* realUnderlyingFileIO = FileIOBase::GetDirectInstance();
if (!realUnderlyingFileIO)
{
return;
}
realUnderlyingFileIO->SetDeprecatedAlias(oldAlias, newAlias);
}
AZStd::optional<AZ::u64> ArchiveFileIO::ConvertToAlias(char* inOutBuffer, AZ::u64 bufferLength) const
{
if ((!inOutBuffer) || (bufferLength == 0))
@@ -63,6 +63,7 @@ namespace AZ::IO
IO::Result FindFiles(const char* filePath, const char* filter, FindFilesCallbackType callback) override;
void SetAlias(const char* alias, const char* path) override;
void ClearAlias(const char* alias) override;
void SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias) override;
AZStd::optional<AZ::u64> ConvertToAlias(char* inOutBuffer, AZ::u64 bufferLength) const override;
bool ConvertToAlias(AZ::IO::FixedMaxPath& convertedPath, const AZ::IO::PathView& path) const override;
using FileIOBase::ConvertToAlias;
@@ -186,8 +186,8 @@ namespace AZ::IO
{
// filter out the stuff which does not match.
// the problem here is that szDir might be something like "@assets@/levels/*"
// but our archive might be mounted at the root, or at some other folder at like "@assets@" or "@assets@/levels/mylevel"
// the problem here is that szDir might be something like "@products@/levels/*"
// but our archive might be mounted at the root, or at some other folder at like "@products@" or "@products@/levels/mylevel"
// so there's really no way to filter out opening the pack and looking at the files inside.
// however, the bind root is not part of the inner zip entry name either
// and the ZipDir::FindFile actually expects just the chopped off piece.
@@ -202,22 +202,22 @@ namespace AZ::IO
// Example:
// "@assets@\\levels\\*" <--- szDir
// "@assets@\\" <--- mount point
// "@products@\\levels\\*" <--- szDir
// "@products@\\" <--- mount point
// ~~~~~~~~~~~ Common part
// "levels\\*" <---- remainder that is not in common
// "" <--- mount point remainder. In this case, we should scan the contents of the pak for the remainder
// Example:
// "@assets@\\levels\\*" <--- szDir
// "@assets@\\levels\\mylevel\\" <--- mount point (its level.pak)
// "@products@\\levels\\*" <--- szDir
// "@products@\\levels\\mylevel\\" <--- mount point (its level.pak)
// ~~~~~~~~~~~~~~~~~~ common part
// "*" <---- remainder that is not in common
// "mylevel\\" <--- mount point remainder.
// example:
// "@assets@\\levels\\otherlevel\\*" <--- szDir
// "@assets@\\levels\\mylevel\\" <--- mount point (its level.pak)
// "@products@\\levels\\otherlevel\\*" <--- szDir
// "@products@\\levels\\mylevel\\" <--- mount point (its level.pak)
// "otherlevel\\*" <---- remainder
// "mylevel\\" <--- mount point remainder.
@@ -249,7 +249,7 @@ namespace AZ::IO
// which means we may search inside the pack.
ScanInZip(it->pZip.get(), sourcePathRemainder.Native());
}
}
}
@@ -94,7 +94,7 @@ namespace AZ::IO::Internal
}
AZStd::smatch matches;
const AZStd::regex lodRegex("@assets@\\\\(.*)_lod[0-9]+(\\.cgfm?)");
const AZStd::regex lodRegex("@products@\\\\(.*)_lod[0-9]+(\\.cgfm?)");
if (!AZStd::regex_match(szPath, matches, lodRegex) || matches.size() != 3)
{
// The current file is not a valid LOD file
@@ -725,7 +725,7 @@ namespace AzFramework
if (!info.m_relativePath.empty())
{
const char* devAssetRoot = fileIO->GetAlias("@devassets@");
const char* devAssetRoot = fileIO->GetAlias("@projectroot@");
if (devAssetRoot)
{
AZ::Data::AssetStreamInfo streamInfo;
@@ -61,7 +61,7 @@ namespace AzFramework
AZ::IO::Path& gemAbsPath = gemInfo.m_absoluteSourcePaths.emplace_back(value);
// Resolve any file aliases first - Do not use ResolvePath() as that assumes
// any relative path is underneath the @assets@ alias
// any relative path is underneath the @products@ alias
if (auto fileIoBase = AZ::IO::FileIOBase::GetInstance(); fileIoBase != nullptr)
{
AZ::IO::FixedMaxPath replacedAliasPath;
@@ -12,10 +12,12 @@
#include <AzCore/IO/Path/Path.h>
#include <AzCore/Casting/numeric_cast.h>
#include <AzCore/Casting/lossy_cast.h>
#include <AzCore/std/containers/fixed_unordered_set.h>
#include <AzCore/std/functional.h>
#include <AzCore/std/string/conversions.h>
#include <AzCore/std/string/string_view.h>
#include <AzCore/StringFunc/StringFunc.h>
#include <AzCore/Utils/Utils.h>
#include <cctype>
namespace AZ
@@ -292,7 +294,7 @@ namespace AZ
void LocalFileIO::CheckInvalidWrite([[maybe_unused]] const char* path)
{
#if defined(AZ_ENABLE_TRACING)
const char* assetAliasPath = GetAlias("@assets@");
const char* assetAliasPath = GetAlias("@products@");
if (path && assetAliasPath)
{
const AZ::IO::PathView pathView(path);
@@ -478,17 +480,15 @@ namespace AZ
return false;
}
if (IsAbsolutePath(path))
if (AZ::IO::PathView(path).HasRootPath())
{
size_t pathLen = strlen(path);
if (pathLen + 1 < resolvedPathSize)
{
azstrncpy(resolvedPath, resolvedPathSize, path, pathLen + 1);
//see if the absolute path uses @assets@ or @root@, if it does lowercase the relative part
[[maybe_unused]] bool lowercasePath = LowerIfBeginsWith(resolvedPath, resolvedPathSize, GetAlias("@assets@"))
|| LowerIfBeginsWith(resolvedPath, resolvedPathSize, GetAlias("@root@"))
|| LowerIfBeginsWith(resolvedPath, resolvedPathSize, GetAlias("@projectplatformcache@"));
//see if the absolute path matches the resolved value of @products@, if it does lowercase the relative part
LowerIfBeginsWith(resolvedPath, resolvedPathSize, GetAlias("@products@"));
ToUnixSlashes(resolvedPath, resolvedPathSize);
return true;
@@ -499,34 +499,39 @@ namespace AZ
}
}
char rootedPathBuffer[AZ_MAX_PATH_LEN] = {0};
constexpr AZStd::string_view productAssetAlias = "@products@";
// Add plus one for the path separator: <alias>/<path>
constexpr size_t MaxPathSizeWithProductAssetAlias = AZ::IO::MaxPathLength + productAssetAlias.size() + 1;
using RootedPathString = AZStd::fixed_string<MaxPathSizeWithProductAssetAlias>;
RootedPathString rootedPathBuffer;
const char* rootedPath = path;
// if the path does not begin with an alias, then it is assumed to begin with @assets@
// if the path does not begin with an alias, then it is assumed to begin with @products@
if (path[0] != '@')
{
if (GetAlias("@assets@"))
if (GetAlias("@products@"))
{
const int rootLength = 9;// strlen("@assets@/")
azstrncpy(rootedPathBuffer, AZ_MAX_PATH_LEN, "@assets@/", rootLength);
size_t pathLen = strlen(path);
size_t rootedPathBufferlength = rootLength + pathLen + 1;// +1 for null terminator
if (rootedPathBufferlength > resolvedPathSize)
if (const size_t requiredSize = productAssetAlias.size() + strlen(path) + 1;
requiredSize > rootedPathBuffer.capacity())
{
AZ_Assert(rootedPathBufferlength < resolvedPathSize, "Constructed path length is wrong:%s", rootedPathBuffer);//path constructed is wrong
size_t remainingSize = resolvedPathSize - rootLength - 1;// - 1 for null terminator
azstrncpy(rootedPathBuffer + rootLength, AZ_MAX_PATH_LEN, path, remainingSize);
rootedPathBuffer[resolvedPathSize - 1] = '\0';
AZ_Error("FileIO", false, "Prepending the %.*s alias to the input path results in a path longer than the"
" AZ::IO::MaxPathLength + the alias size of %zu. The size of the potential failed path is %zu",
AZ_STRING_ARG(productAssetAlias), rootedPathBuffer.capacity(), requiredSize)
}
else
{
azstrncpy(rootedPathBuffer + rootLength, AZ_MAX_PATH_LEN - rootLength, path, pathLen + 1);
rootedPathBuffer = RootedPathString::format("%.*s/%s", AZ_STRING_ARG(productAssetAlias), path);
}
}
else
{
ConvertToAbsolutePath(path, rootedPathBuffer, AZ_MAX_PATH_LEN);
if (ConvertToAbsolutePath(path, rootedPathBuffer.data(), rootedPathBuffer.capacity()))
{
// Recalculate the internal string length
rootedPathBuffer.resize_no_construct(AZStd::char_traits<char>::length(rootedPathBuffer.data()));
}
}
rootedPath = rootedPathBuffer;
rootedPath = rootedPathBuffer.c_str();
}
if (ResolveAliases(rootedPath, resolvedPath, resolvedPathSize))
@@ -561,11 +566,57 @@ namespace AZ
const char* LocalFileIO::GetAlias(const char* key) const
{
const auto it = m_aliases.find(key);
if (it != m_aliases.end())
if (const auto it = m_aliases.find(key); it != m_aliases.end())
{
return it->second.c_str();
}
else if (const auto deprecatedIt = m_deprecatedAliases.find(key);
deprecatedIt != m_deprecatedAliases.end())
{
AZ_Error("FileIO", false, R"(Alias "%s" is deprecated. Please use alias "%s" instead)",
key, deprecatedIt->second.c_str());
AZStd::string_view aliasValue = deprecatedIt->second;
// Contains the list of aliases resolved so far
// If max_size is hit, than an error is logged and nullptr is returned
using VisitedAliasSet = AZStd::fixed_unordered_set<AZStd::string_view, 8, 8>;
VisitedAliasSet visitedAliasSet;
while (aliasValue.starts_with("@"))
{
if (visitedAliasSet.contains(aliasValue))
{
AZ_Error("FileIO", false, "Cycle found with for alias %.*s when trying to resolve deprecated alias %s",
AZ_STRING_ARG(aliasValue), key);
return nullptr;
}
if(visitedAliasSet.size() == visitedAliasSet.max_size())
{
AZ_Error("FileIO", false, "Unable to resolve path to deprecated alias %s within %zu steps",
key, visitedAliasSet.max_size());
return nullptr;
}
// Add the current alias value to the visited set
visitedAliasSet.emplace(aliasValue);
// Check if the alias value corresponds to another alias
if (auto resolvedIter = m_aliases.find(aliasValue); resolvedIter != m_aliases.end())
{
aliasValue = resolvedIter->second;
}
else if (resolvedIter = m_deprecatedAliases.find(aliasValue);
resolvedIter != m_deprecatedAliases.end())
{
aliasValue = resolvedIter->second;
}
else
{
return nullptr;
}
}
return aliasValue.data();
}
return nullptr;
}
@@ -574,6 +625,11 @@ namespace AZ
m_aliases.erase(key);
}
void LocalFileIO::SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias)
{
m_deprecatedAliases[oldAlias] = newAlias;
}
AZStd::optional<AZ::u64> LocalFileIO::ConvertToAliasBuffer(char* outBuffer, AZ::u64 outBufferLength, AZStd::string_view inBuffer) const
{
size_t longestMatch = 0;
@@ -675,7 +731,9 @@ namespace AZ
: string_view_pair{};
size_t requiredResolvedPathSize = pathView.size() - aliasKey.size() + aliasValue.size() + 1;
AZ_Assert(path != resolvedPath && resolvedPathSize >= requiredResolvedPathSize, "Resolved path is incorrect");
AZ_Assert(path != resolvedPath, "ResolveAliases does not support inplace update of the path");
AZ_Assert(resolvedPathSize >= requiredResolvedPathSize, "Resolved path size %llu not large enough. It needs to be %zu",
resolvedPathSize, requiredResolvedPathSize);
// we assert above, but we also need to properly handle the case when the resolvedPath buffer size
// is too small to copy the source into.
if (path == resolvedPath || (resolvedPathSize < requiredResolvedPathSize))
@@ -699,13 +757,9 @@ namespace AZ
resolvedPath[resolvedPathLen] = '\0';
// If the path started with one of the "asset cache" path aliases, lowercase the path
const char* assetAliasPath = GetAlias("@assets@");
const char* rootAliasPath = GetAlias("@root@");
const char* projectPlatformCacheAliasPath = GetAlias("@projectplatformcache@");
const char* projectPlatformCacheAliasPath = GetAlias("@products@");
const bool lowercasePath = (assetAliasPath != nullptr && AZ::StringFunc::StartsWith(resolvedPath, assetAliasPath)) ||
(rootAliasPath != nullptr && AZ::StringFunc::StartsWith(resolvedPath, rootAliasPath)) ||
(projectPlatformCacheAliasPath != nullptr && AZ::StringFunc::StartsWith(resolvedPath, projectPlatformCacheAliasPath));
const bool lowercasePath = projectPlatformCacheAliasPath != nullptr && AZ::StringFunc::StartsWith(resolvedPath, projectPlatformCacheAliasPath);
if (lowercasePath)
{
@@ -822,5 +876,10 @@ namespace AZ
return pathStr + "/";
}
bool LocalFileIO::ConvertToAbsolutePath(const char* path, char* absolutePath, AZ::u64 maxLength) const
{
return AZ::Utils::ConvertToAbsolutePath(path, absolutePath, maxLength);
}
} // namespace IO
} // namespace AZ
@@ -61,6 +61,8 @@ namespace AZ
void SetAlias(const char* alias, const char* path) override;
void ClearAlias(const char* alias) override;
const char* GetAlias(const char* alias) const override;
void SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias) override;
AZStd::optional<AZ::u64> ConvertToAlias(char* inOutBuffer, AZ::u64 bufferLength) const override;
bool ConvertToAlias(AZ::IO::FixedMaxPath& convertedPath, const AZ::IO::PathView& path) const override;
using FileIOBase::ConvertToAlias;
@@ -71,7 +73,7 @@ namespace AZ
bool GetFilename(HandleType fileHandle, char* filename, AZ::u64 filenameSize) const override;
bool ConvertToAbsolutePath(const char* path, char* absolutePath, AZ::u64 maxLength) const;
private:
SystemFile* GetFilePointerFromHandle(HandleType fileHandle);
@@ -79,7 +81,6 @@ namespace AZ
AZStd::optional<AZ::u64> ConvertToAliasBuffer(char* outBuffer, AZ::u64 outBufferLength, AZStd::string_view inBuffer) const;
bool ResolveAliases(const char* path, char* resolvedPath, AZ::u64 resolvedPathSize) const;
bool IsAbsolutePath(const char* path) const;
bool LowerIfBeginsWith(char* inOutBuffer, AZ::u64 bufferLen, const char* alias) const;
@@ -91,6 +92,7 @@ namespace AZ
AZStd::atomic<HandleType> m_nextHandle;
AZStd::unordered_map<HandleType, SystemFile> m_openFiles;
AZStd::unordered_map<AZStd::string, AZStd::string> m_aliases;
AZStd::unordered_map<AZStd::string, AZStd::string> m_deprecatedAliases;
void CheckInvalidWrite(const char* path);
};
@@ -49,14 +49,14 @@ namespace AZ
s_IOLog.append(m_name);
s_IOLog.append("\r\n");
}
void Append(const char* line)
{
s_IOLog.append(AZStd::string::format("%u ", m_fileOperation));
s_IOLog.append(line);
s_IOLog.append("\r\n");
}
~LogCall()
{
s_IOLog.append(AZStd::string::format("%u End ", m_fileOperation));
@@ -251,7 +251,7 @@ namespace AZ
REMOTEFILE_LOG_APPEND(AZStd::string::format("NetworkFileIO::Size(filePath=%s) size request failed. return Error", filePath).c_str());
return ResultCode::Error;
}
size = response.m_size;
REMOTEFILE_LOG_APPEND(AZStd::string::format("NetworkFileIO::Size(filePath=%s) size=%u. return Success", filePath, size).c_str());
return ResultCode::Success;
@@ -793,6 +793,12 @@ namespace AZ
REMOTEFILE_LOG_CALL(AZStd::string::format("NetworkFileIO()::ClearAlias(alias=%s)", alias?alias:"nullptr").c_str());
}
void NetworkFileIO::SetDeprecatedAlias([[maybe_unused]] AZStd::string_view oldAlias, [[maybe_unused]] AZStd::string_view newAlias)
{
REMOTEFILE_LOG_CALL(AZStd::string::format("NetworkFileIO()::SetDeprecatedAlias(oldAlias=%.*s, newAlias=%.*s)",
AZ_STRING_ARG(oldAlias), AZ_STRING_ARG(newAlias)).c_str());
}
AZStd::optional<AZ::u64> NetworkFileIO::ConvertToAlias(char* inOutBuffer, [[maybe_unused]] AZ::u64 bufferLength) const
{
REMOTEFILE_LOG_CALL(AZStd::string::format("NetworkFileIO()::ConvertToAlias(inOutBuffer=%s, bufferLength=%u)", inOutBuffer?inOutBuffer:"nullptr", bufferLength).c_str());
@@ -927,7 +933,7 @@ namespace AZ
{
m_cacheLookaheadPos = filePosition - CacheStartFilePosition();
}
void RemoteFileCache::SyncCheck()
{
#ifdef REMOTEFILEIO_SYNC_CHECK
@@ -955,7 +961,7 @@ namespace AZ
AZ_TracePrintf(RemoteFileCacheChannel, "RemoteFileCache::SyncCheck(m_fileHandle=%u) tell request failed.", m_fileHandle);
REMOTEFILE_LOG_APPEND(AZStd::string::format("RemoteFileCache::SyncCheck(m_fileHandle=%u) tell request failed.", m_fileHandle).c_str());
}
if (responce.m_offset != m_filePosition)
{
AZ_TracePrintf(RemoteFileCacheChannel, "RemoteFileCache::SyncCheck(m_fileHandle=%u) failed!!! m_filePosition=%u tell=%u", m_fileHandle, m_filePosition, responce.m_offset);
@@ -1028,7 +1034,7 @@ namespace AZ
{
REMOTEFILE_LOG_CALL(AZStd::string::format("RemoteFileIO()::Close(fileHandle=%u)", fileHandle).c_str());
Result returnValue = NetworkFileIO::Close(fileHandle);
if (returnValue == ResultCode::Success)
{
AZStd::lock_guard<AZStd::recursive_mutex> lock(m_remoteFileCacheGuard);
@@ -1160,7 +1166,7 @@ namespace AZ
REMOTEFILE_LOG_CALL(AZStd::string::format("RemoteFileIO()::Read(fileHandle=%u, buffer=OUT, size=%u, failOnFewerThanSizeBytesRead=%s, bytesRead=OUT)", fileHandle, size, failOnFewerThanSizeBytesRead ? "True" : "False").c_str());
AZStd::lock_guard<AZStd::recursive_mutex> lock(m_remoteFileCacheGuard);
RemoteFileCache& cache = GetCache(fileHandle);
AZ::u64 remainingBytesToRead = size;
AZ::u64 bytesReadFromCache = 0;
AZ::u64 remainingBytesInCache = cache.RemainingBytes();
@@ -1263,7 +1269,7 @@ namespace AZ
RemoteFileCache& cache = GetCache(fileHandle);
if (cache.m_cacheLookaheadBuffer.size() && cache.RemainingBytes())
{
// find out where we are
// find out where we are
AZ::u64 seekPosition = cache.CacheFilePosition();
// note, seeks are predicted, and do not ask for a response.
@@ -1361,6 +1367,14 @@ namespace AZ
}
}
void RemoteFileIO::SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias)
{
if (m_excludedFileIO)
{
m_excludedFileIO->SetDeprecatedAlias(oldAlias, newAlias);
}
}
AZStd::optional<AZ::u64> RemoteFileIO::ConvertToAlias(char* inOutBuffer, AZ::u64 bufferLength) const
{
return m_excludedFileIO ? m_excludedFileIO->ConvertToAlias(inOutBuffer, bufferLength) : strlen(inOutBuffer);
@@ -102,6 +102,7 @@ namespace AZ
Result FindFiles(const char* filePath, const char* filter, FindFilesCallbackType callback) override;
void SetAlias(const char* alias, const char* path) override;
void ClearAlias(const char* alias) override;
void SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias) override;
AZStd::optional<AZ::u64> ConvertToAlias(char* inOutBuffer, AZ::u64 bufferLength) const override;
bool ConvertToAlias(AZ::IO::FixedMaxPath& convertedPath, const AZ::IO::PathView& path) const override;
using FileIOBase::ConvertToAlias;
@@ -194,6 +195,7 @@ namespace AZ
void SetAlias(const char* alias, const char* path) override;
const char* GetAlias(const char* alias) const override;
void ClearAlias(const char* alias) override;
void SetDeprecatedAlias(AZStd::string_view oldAlias, AZStd::string_view newAlias) override;
AZStd::optional<AZ::u64> ConvertToAlias(char* inOutBuffer, AZ::u64 bufferLength) const override;
bool ConvertToAlias(AZ::IO::FixedMaxPath& convertedPath, const AZ::IO::PathView& path) const override;
using FileIOBase::ConvertToAlias;