Update the FileIO Aliases (#4186)

* Update the FileIOAlias naming to make the cache, project root and engine root paths more clear The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been collapsed to `@projectproductassets@` The alias of `@devroot@` and `@engroot@` has been collapsed to `@engroot@` The alias of `@devassets@` and `@projectroot@` has been collapsed to `@projectroot@` Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated use of devassets and devroot properties in python Those properties now use projectroot and engroot Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Removed hardcoded size of 9 for the product asset alias. The ResolvePath function now just appends the @projectproductassets@ alias with the input path Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Fix for typos in Hydra python test Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset cache write errors for Test file creation. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Added a deprecated alias map to the FileIO System When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated python test scripts to use the projectroot binding Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated references to devroot and devgame within the codebase The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated <engroot>/AutomatedTesting references to projectroot Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Correct casing in emfxworkspace file Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Removed newly added AppendPathParts function Removed the Path constructors which accepts a PathIterable instance The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Fixed unused parameter warning Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Undid change to the LexicallyProximate function to set the path result to the base path. It needs to return the *this path if the pathResult is empty Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms). Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Changed the AWSMetricsGemMock to use a temporary for writing test assets Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/" IsAbsolute was not the corect check as the path of "D:" is a relative path. To be absolute according to the Windows the path must have a root directory. i.e "D:/" or "D:\" Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated the ScriptCanvas Upgrade Tool to output backed up files to the Project User Path instead of the engine root Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Fixed whitespacing issues in Application.cpp Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Modified testSeedFolder variable in the SeedBuilderTests to use the @engroot@ alias instead of @projectproductassets@/.. alias when referring to the LmbrCentral Gem source path Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Updated references to the Project Asset Cache in the PythonTests. Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Fixed the FileIO Deprecated Alias test on Windows Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Removing @projectsourceassets@ alias, as it is only used once. Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath form the SettingsRegistry. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Replaced @projectproductassets@ alias with @products@ Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com> * Rollback changes to the PhysX EditorSystemComponent.cpp The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497 Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-10-05 15:12:53 -05:00
parent 643bd84739
commit f648cb1fd8
209 changed files with 1365 additions and 1678 deletions
@@ -13,7 +13,6 @@
 #include <AzCore/Android/Utils.h>
 #include <AzCore/IO/IOUtils.h>
 #include <AzCore/IO/Path/Path.h>
-#include <AzCore/IO/SystemFile.h>
 #include <AzCore/std/functional.h>

 #include <android/api-level.h>
@@ -42,10 +41,10 @@ namespace AZ
    {
        Result LocalFileIO::Copy(const char* sourceFilePath, const char* destinationFilePath)
        {
-            char resolvedSourcePath[AZ_MAX_PATH_LEN];
-            char resolvedDestPath[AZ_MAX_PATH_LEN];
-            ResolvePath(sourceFilePath, resolvedSourcePath, AZ_MAX_PATH_LEN);
-            ResolvePath(destinationFilePath, resolvedDestPath, AZ_MAX_PATH_LEN);
+            char resolvedSourcePath[AZ::IO::MaxPathLength];
+            char resolvedDestPath[AZ::IO::MaxPathLength];
+            ResolvePath(sourceFilePath, resolvedSourcePath, AZ::IO::MaxPathLength);
+            ResolvePath(destinationFilePath, resolvedDestPath, AZ::IO::MaxPathLength);

            if (AZ::Android::Utils::IsApkPath(sourceFilePath) || AZ::Android::Utils::IsApkPath(destinationFilePath))
            {
@@ -77,18 +76,17 @@ namespace AZ
        {
            ANDROID_IO_PROFILE_SECTION_ARGS("FindFiles:%s", filePath);

-            char resolvedPath[AZ_MAX_PATH_LEN];
-            ResolvePath(filePath, resolvedPath, AZ_MAX_PATH_LEN);
+            char resolvedPath[AZ::IO::MaxPathLength];
+            ResolvePath(filePath, resolvedPath, AZ::IO::MaxPathLength);

            AZStd::string pathWithoutSlash = RemoveTrailingSlash(resolvedPath);
            bool isInAPK = AZ::Android::Utils::IsApkPath(pathWithoutSlash.c_str());

+            AZ::IO::FixedMaxPath tempBuffer;
            if (isInAPK)
            {
                AZ::IO::FixedMaxPath strippedPath = AZ::Android::Utils::StripApkPrefix(pathWithoutSlash.c_str());

-                char tempBuffer[AZ_MAX_PATH_LEN] = {0};
-
                AZ::Android::APKFileHandler::ParseDirectory(strippedPath.c_str(), [&](const char* name)
                    {
                        AZStd::string_view filenameView = name;
@@ -98,10 +96,9 @@ namespace AZ
                            AZStd::string foundFilePath = CheckForTrailingSlash(resolvedPath);
                            foundFilePath += name;
                            // if aliased, de-alias!
-                            azstrcpy(tempBuffer, AZ_MAX_PATH_LEN, foundFilePath.c_str());
-                            ConvertToAlias(tempBuffer, AZ_MAX_PATH_LEN);
+                            ConvertToAlias(tempBuffer, AZ::IO::PathView{ foundFilePath });

-                            if (!callback(tempBuffer))
+                            if (!callback(tempBuffer.c_str()))
                            {
                                return false;
                            }
@@ -115,10 +112,6 @@ namespace AZ

                if (dir != nullptr)
                {
-                    // because the absolute path might actually be SHORTER than the alias ("c:/r/dev" -> "@devroot@"), we need to
-                    // use a static buffer here.
-                    char tempBuffer[AZ_MAX_PATH_LEN];
-
                    // clear the errno state so we can distinguish between errors and end of stream
                    errno = 0;
                    struct dirent* entry = readdir(dir);
@@ -133,10 +126,9 @@ namespace AZ
                            AZStd::string foundFilePath = CheckForTrailingSlash(resolvedPath);
                            foundFilePath += entry->d_name;
                            // if aliased, de-alias!
-                            azstrcpy(tempBuffer, AZ_MAX_PATH_LEN, foundFilePath.c_str());
-                            ConvertToAlias(tempBuffer, AZ_MAX_PATH_LEN);
+                            ConvertToAlias(tempBuffer, AZ::IO::PathView{ foundFilePath });

-                            if (!callback(tempBuffer))
+                            if (!callback(tempBuffer.c_str()))
                            {
                                break;
                            }
@@ -163,8 +155,8 @@ namespace AZ

        Result LocalFileIO::CreatePath(const char* filePath)
        {
-            char resolvedPath[AZ_MAX_PATH_LEN];
-            ResolvePath(filePath, resolvedPath, AZ_MAX_PATH_LEN);
+            char resolvedPath[AZ::IO::MaxPathLength];
+            ResolvePath(filePath, resolvedPath, AZ::IO::MaxPathLength);

            if (AZ::Android::Utils::IsApkPath(resolvedPath))
            {
@@ -201,33 +193,5 @@ namespace AZ
            mkdir(pathBuffer.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
            return IsDirectory(resolvedPath) ? ResultCode::Success : ResultCode::Error;
        }
-
-        bool LocalFileIO::IsAbsolutePath(const char* path) const
-        {
-            return path && path[0] == '/';
-        }
-
-        bool LocalFileIO::ConvertToAbsolutePath(const char* path, char* absolutePath, AZ::u64 maxLength) const
-        {
-            if (AZ::Android::Utils::IsApkPath(path))
-            {
-                azstrncpy(absolutePath, maxLength, path, maxLength);
-                return true;
-            }
-            AZ_Assert(maxLength >= AZ_MAX_PATH_LEN, "Path length is larger than AZ_MAX_PATH_LEN");
-            if (!IsAbsolutePath(path))
-            {
-                // note that realpath fails if the path does not exist and actually changes the return value
-                // to be the actual place that FAILED, which we don't want.
-                // if we fail, we'd prefer to fall through and at least use the original path.
-                const char* result = realpath(path, absolutePath);
-                if (result)
-                {
-                    return true;
-                }
-            }
-            azstrcpy(absolutePath, maxLength, path);
-            return IsAbsolutePath(absolutePath);
-        }
    } // namespace IO
 }//namespace AZ
@@ -10,7 +10,7 @@
 #include <dirent.h>
 #include <unistd.h>
 #include <AzFramework/IO/LocalFileIO.h>
-#include <AzCore/IO/SystemFile.h>
+#include <AzCore/IO/Path/Path.h>
 #include <AzCore/std/functional.h>

 namespace AZ
@@ -19,11 +19,11 @@ namespace AZ
    {
        Result LocalFileIO::Copy(const char* sourceFilePath, const char* destinationFilePath)
        {
-            char resolvedSourceFilePath[AZ_MAX_PATH_LEN] = {0};
-            ResolvePath(sourceFilePath, resolvedSourceFilePath, AZ_MAX_PATH_LEN);
+            char resolvedSourceFilePath[AZ::IO::MaxPathLength] = {0};
+            ResolvePath(sourceFilePath, resolvedSourceFilePath, AZ::IO::MaxPathLength);

-            char resolvedDestinationFilePath[AZ_MAX_PATH_LEN] = {0};
-            ResolvePath(destinationFilePath, resolvedDestinationFilePath, AZ_MAX_PATH_LEN);
+            char resolvedDestinationFilePath[AZ::IO::MaxPathLength] = {0};
+            ResolvePath(destinationFilePath, resolvedDestinationFilePath, AZ::IO::MaxPathLength);

            // Use standard C++ method of file copy.
            {
@@ -45,17 +45,15 @@ namespace AZ

        Result LocalFileIO::FindFiles(const char* filePath, const char* filter, FindFilesCallbackType callback)
        {
-            char resolvedPath[AZ_MAX_PATH_LEN] = {0};
-            ResolvePath(filePath, resolvedPath, AZ_MAX_PATH_LEN);
+            char resolvedPath[AZ::IO::MaxPathLength] = {0};
+            ResolvePath(filePath, resolvedPath, AZ::IO::MaxPathLength);

            AZStd::string withoutSlash = RemoveTrailingSlash(resolvedPath);
            DIR* dir = opendir(withoutSlash.c_str());

            if (dir != nullptr)
            {
-                // because the absolute path might actually be SHORTER than the alias ("c:/r/dev" -> "@devroot@"), we need to
-                // use a static buffer here.
-                char tempBuffer[AZ_MAX_PATH_LEN];
+                AZ::IO::FixedMaxPath tempBuffer;

                errno = 0;
                struct dirent* entry = readdir(dir);
@@ -70,10 +68,9 @@ namespace AZ
                        AZStd::string foundFilePath = CheckForTrailingSlash(resolvedPath);
                        foundFilePath += entry->d_name;
                        // if aliased, dealias!
-                        azstrcpy(tempBuffer, AZ_MAX_PATH_LEN, foundFilePath.c_str());
-                        ConvertToAlias(tempBuffer, AZ_MAX_PATH_LEN);
+                        ConvertToAlias(tempBuffer, AZ::IO::PathView{ foundFilePath });

-                        if (!callback(tempBuffer))
+                        if (!callback(tempBuffer.c_str()))
                        {
                            break;
                        }
@@ -92,8 +89,8 @@ namespace AZ

        Result LocalFileIO::CreatePath(const char* filePath)
        {
-            char resolvedPath[AZ_MAX_PATH_LEN] = {0};
-            ResolvePath(filePath, resolvedPath, AZ_MAX_PATH_LEN);
+            char resolvedPath[AZ::IO::MaxPathLength] = {0};
+            ResolvePath(filePath, resolvedPath, AZ::IO::MaxPathLength);

            // create all paths up to that directory.
            // its not an error if the path exists.
@@ -125,28 +122,5 @@ namespace AZ
            mkdir(buf.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
            return IsDirectory(resolvedPath) ? ResultCode::Success : ResultCode::Error;
        }
-
-        bool LocalFileIO::IsAbsolutePath(const char* path) const
-        {
-            return path && path[0] == '/';
-        }
-
-        bool LocalFileIO::ConvertToAbsolutePath(const char* path, char* absolutePath, AZ::u64 maxLength) const
-        {
-            AZ_Assert(maxLength >= AZ_MAX_PATH_LEN, "Path length is larger than AZ_MAX_PATH_LEN");
-            if (!IsAbsolutePath(path))
-            {
-                // note that realpath fails if the path does not exist and actually changes the return value
-                // to be the actual place that FAILED, which we don't want.
-                // if we fail, we'd prefer to fall through and at least use the original path.
-                const char* result = realpath(path, absolutePath);
-                if (result)
-                {
-                    return true;
-                }
-            }
-            azstrcpy(absolutePath, maxLength, path);
-            return IsAbsolutePath(absolutePath);
-        }
    } // namespace IO
 } // namespace AZ
@@ -47,7 +47,7 @@ namespace AZ

            if (hFind != INVALID_HANDLE_VALUE)
            {
-                // because the absolute path might actually be SHORTER than the alias ("c:/r/dev" -> "@devroot@"), we need to
+                // because the absolute path might actually be SHORTER than the alias ("D:/o3de" -> "@engroot@"), we need to
                // use a static buffer here.
                char tempBuffer[AZ_MAX_PATH_LEN];
                do
@@ -133,36 +133,5 @@ namespace AZ

            return SystemFile::CreateDir(buf.c_str()) ? ResultCode::Success : ResultCode::Error;
        }
-
-        bool LocalFileIO::ConvertToAbsolutePath(const char* path, char* absolutePath, AZ::u64 maxLength) const
-        {
-            char* result = _fullpath(absolutePath, path, maxLength);
-            size_t len = ::strlen(absolutePath);
-            if (len > 0)
-            {
-                // strip trailing slash
-                if (absolutePath[len - 1] == '/' || absolutePath[len - 1] == '\\')
-                {
-                    absolutePath[len - 1] = 0;
-                }
-
-                // For some reason, at least on windows, _fullpath returns a lowercase drive letter even though other systems like Qt, use upper case.
-                if (len > 2)
-                {
-                    if (absolutePath[1] == ':')
-                    {
-                        absolutePath[0] = (char)toupper(absolutePath[0]);
-                    }
-                }
-            }
-            return result != nullptr;
-        }
-
-        bool LocalFileIO::IsAbsolutePath(const char* path) const
-        {
-            char drive[16] = { 0 };
-            _splitpath_s(path, drive, 16, nullptr, 0, nullptr, 0, nullptr, 0);
-            return strlen(drive) > 0;
-        }
    } // namespace IO
 }//namespace AZ
@@ -7,26 +7,24 @@
 */
 #include <AzCore/PlatformIncl.h>
 #include <AzFramework/IO/LocalFileIO.h>
+#include <AzCore/std/string/conversions.h>
+#include <AzCore/IO/Path/Path.h>
 #include <AzCore/IO/SystemFile.h>

-namespace AZ
+namespace AZ::IO
 {
-    namespace IO
+    Result LocalFileIO::Copy(const char* sourceFilePath, const char* destinationFilePath)
    {
+        AZ::IO::FixedMaxPath resolvedSourcePath;
+        ResolvePath(resolvedSourcePath, sourceFilePath);
+        AZ::IO::FixedMaxPath resolvedDestPath;
+        ResolvePath(resolvedDestPath, destinationFilePath);

-        Result LocalFileIO::Copy(const char* sourceFilePath, const char* destinationFilePath)
-        {
-            char resolvedSourcePath[AZ_MAX_PATH_LEN];
-            ResolvePath(sourceFilePath, resolvedSourcePath, AZ_MAX_PATH_LEN);
-            char resolvedDestPath[AZ_MAX_PATH_LEN];
-            ResolvePath(destinationFilePath, resolvedDestPath, AZ_MAX_PATH_LEN);
+        AZStd::fixed_wstring<AZ::IO::MaxPathLength> resolvedSourcePathW;
+        AZStd::fixed_wstring<AZ::IO::MaxPathLength> resolvedDestPathW;
+        AZStd::to_wstring(resolvedSourcePathW, resolvedSourcePath.Native());
+        AZStd::to_wstring(resolvedDestPathW, resolvedDestPath.Native());

-            if (::CopyFileA(resolvedSourcePath, resolvedDestPath, false) == 0)
-            {
-                return ResultCode::Error;
-            }
-
-            return ResultCode::Success;
-        }
-    } // namespace IO
-}//namespace AZ
+        return ::CopyFileW(resolvedSourcePathW.c_str(), resolvedDestPathW.c_str(), false) != 0 ? ResultCode::Success : ResultCode::Error;
+    }
+}//namespace AZ::IO