Commit Graph

11 Commits (e56396a817c295ddf19e25ba612e617943d6d87a)

Author SHA1 Message Date
lumberyard-employee-dm f648cb1fd8
Update the FileIO Aliases (#4186)
* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear

The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`

The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`

The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated use of devassets and devroot properties in python

Those properties now use projectroot and engroot

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed hardcoded size of 9 for the product asset alias.

The ResolvePath function now just appends the @projectproductassets@
alias with the input path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix for typos in Hydra python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a deprecated alias map to the FileIO System

When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated python test scripts to use the projectroot binding

Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to devroot and devgame within the codebase

The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated <engroot>/AutomatedTesting references to projectroot


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Correct casing in emfxworkspace file


Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance

The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed unused parameter warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Undid change to the LexicallyProximate function to set the path result to the base path.

It needs to return the *this path if the pathResult is empty

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils

Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Changed the AWSMetricsGemMock to use a temporary for writing test assets

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"

IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed whitespacing issues in Application.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated references to the Project Asset Cache in the PythonTests.

Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed the FileIO Deprecated Alias test on Windows

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removing @projectsourceassets@ alias, as it is only used once.

Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replaced @projectproductassets@ alias with @products@

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Rollback changes to the PhysX EditorSystemComponent.cpp

The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
4 years ago
amzn-phist fec77632ef
Archive Component - Rewrite and additional work on Archive and Asset Bundler (#4332)
* Fix issues with seedlist for AutomatedTesting

Fixes error reporting so it will show the file hint in the tool.
Removes any missing assets from the .seed file.
Remove an unnecessary dependency from AutomatedTesting dependencies
file.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Make ArchiveComponent use AZ::IO::IArchive

Initial changes that will get the sychronous calls in ArchiveComponent
to use IArchive interface rather than external zip/7z tools.

Some of the asynchronous api are still in place, anything that wasn't
being used has been removed for now.  This may change later if we move
towards all the api being asynchronous.  Until then, we can't remove the
reliance upon the external archive tools completely.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Updates AZStd::thread constructors

Adds a variadic constructor which forwards args to the functor.
Because of our thread_desc extension, there was confusion on the
arugments, so the args were reordered to take the thread_desc first,
before the functor and args.
Also the thread_desc is taken as reference rather than by pointer.
Update callsites to account for this change.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Async operation of ArchiveComponent api

This sets up the ArchiveComponent to operate asynchronously.  It uses
promise/future to transfer results to caller.

This is still broken, there's a few things that need to get fixed up,
but this is a good checkpoint for the work as it solidifies the api,
cleans up a bunch of unused code, and compiles.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Removes the platform-specific ArchiveComponen

These are no longer needed, as they control the direct interaction with
host OS tools like 7za.exe or /bin/zip.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Removes the platform-specific files from cmake

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Removes the 7za.exe (and legal notice)

This tool is no longer needed in the repo.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fixes usage of IArchive::GetFullPath()

This changed to return a PathView, updated to reflect that.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix promises and threads

Make sure promises are only set exactly once.  This meant reworking some
of the initial error checking.
Detach threads when created.  Adds [[nodiscard]] to the functions that
return a future.  Since threads are detached, the future is the main way
to get communication from the thread.
Clean up interface, add comments.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* More edits to thread creation

Changes to thread construction to account for parameter change.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix some remaining issues with ArchiveComponent

Put created threads inside a container, then join them at Deactivate.
Fix asset bundler case when injecting a file with no working directory.
Fix thread constructor that applies args to a function.
Fix lambdas to take string args by value rather than reference.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fixes some remaining bugs in ArchiveComponent

Open archive as read-only during extract & list operations.
Fixes paths issues.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix initialize of opaque thread handle in thread_UnixLike.h

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed unused variable in AssetBundleComponent.cpp to fix compiler warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix some issues with archives

File paths in the CDR and the local headers need to match, but there
were issues with path separators and case that made it possible to get
invalid headers errors in some archives.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Adds some new ArchiveComponent unit tests

Adds new tests for extraction of archive and adding files from a file
list to an archive.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix file data offset issues when opening archives

When opening an INestedArchive it would run through the CDR headers to
create file entries in the zip cache.  The offsets to the compressed
data were being calculated incorrectly because they were using the CDR
headers rather than jumping to the local file headers and getting
offsets from those sizes.

Removed and refactored some archive validation flow and zip cache
factory init methods to either init default or init w/ additional
validation checks.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Addresses PR feedback

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address more points of feedback in PR

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address additional PR feedback

 Fixes up some error checks and uses of strings vs paths.
 Enable archive component tests on Linux so we can see if they will
 pass.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address PR feedback

Change the INestedArchive interface to list files as AZ::IO::Path.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Disabling the ArchiveComponent tests on Linux

They failed so we will revisit them to attempt a fix.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Rename a member variable to be more accurate

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address feedback on PR

Bump version of Archive Components for serialize context.
Improve error messages during archive open and validation.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Revert recent changes

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
4 years ago
Steve Pham 38261d0800
Shorten copyright headers by splitting into 2 lines (#2213)
* Updated all copyright headers to split the longer original copyright line into 2 shorter lines

Signed-off-by: Steve Pham <spham@amazon.com>
4 years ago
Esteban Papp 4aebea61af Fixes for Jenkins
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Esteban Papp ae46eea1e8 Fix tests in Linux and discover path comparison problem in Windows (fixed)
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Esteban Papp 4a5ac0e204 Fix for ArchiveTest where the assets alias was not being set
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Steve Pham b4a2edec6a
Final update copyright headers to reference license files at the repo root (#1693)
* Final update copyright headers to reference license files at the repo root

Signed-off-by: spham <spham@amazon.com>

* Fix copyright validator unit tests to support the stale O3DE header scenario

Signed-off-by: spham <spham@amazon.com>
5 years ago
Steve Pham 70042fcdcd
O3DE Copyright Updates for Linux Foundation (#1504) 5 years ago
alexpete c2cbd430fe Integrating up through commit 90f050496 5 years ago
alexpete 75dc720198 Integrating latest 47acbe8 5 years ago
alexpete a10351f38d Initial commit 5 years ago