Add cvars to control the number of worker threads created by the TaskGraph and Job systems. The cvars are scales on the hw concurrency of the cpu.
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear
The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`
The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`
The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated use of devassets and devroot properties in python
Those properties now use projectroot and engroot
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed hardcoded size of 9 for the product asset alias.
The ResolvePath function now just appends the @projectproductassets@
alias with the input path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix for typos in Hydra python test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a deprecated alias map to the FileIO System
When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated python test scripts to use the projectroot binding
Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated references to devroot and devgame within the codebase
The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated <engroot>/AutomatedTesting references to projectroot
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Correct casing in emfxworkspace file
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance
The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed unused parameter warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Undid change to the LexicallyProximate function to set the path result to the base path.
It needs to return the *this path if the pathResult is empty
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils
Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Changed the AWSMetricsGemMock to use a temporary for writing test assets
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"
IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed whitespacing issues in Application.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated references to the Project Asset Cache in the PythonTests.
Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed the FileIO Deprecated Alias test on Windows
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removing @projectsourceassets@ alias, as it is only used once.
Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced @projectproductassets@ alias with @products@
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Rollback changes to the PhysX EditorSystemComponent.cpp
The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added IsDirectory function to SystemFile
This takes the implementation in LocalFileIO and uses it for SystemFile
and then just has LocalFileIO call the SystemFile implementation
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed logic to detect the WinApi FILE_ATTRIBUTE_DIRECTORY attribute
Updated the FileIO.cpp test to use AZ::IO::Path and removed direct uses
of AZStd::string
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Adding googletest printers for string and Path classes
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the SystemFile_WinAPI functions to use AZStd::to_wstring
This makes the the SystemFile function convert from UTF-8 to UTF-16
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix issues with seedlist for AutomatedTesting
Fixes error reporting so it will show the file hint in the tool.
Removes any missing assets from the .seed file.
Remove an unnecessary dependency from AutomatedTesting dependencies
file.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Make ArchiveComponent use AZ::IO::IArchive
Initial changes that will get the sychronous calls in ArchiveComponent
to use IArchive interface rather than external zip/7z tools.
Some of the asynchronous api are still in place, anything that wasn't
being used has been removed for now. This may change later if we move
towards all the api being asynchronous. Until then, we can't remove the
reliance upon the external archive tools completely.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Updates AZStd::thread constructors
Adds a variadic constructor which forwards args to the functor.
Because of our thread_desc extension, there was confusion on the
arugments, so the args were reordered to take the thread_desc first,
before the functor and args.
Also the thread_desc is taken as reference rather than by pointer.
Update callsites to account for this change.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Async operation of ArchiveComponent api
This sets up the ArchiveComponent to operate asynchronously. It uses
promise/future to transfer results to caller.
This is still broken, there's a few things that need to get fixed up,
but this is a good checkpoint for the work as it solidifies the api,
cleans up a bunch of unused code, and compiles.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the platform-specific ArchiveComponen
These are no longer needed, as they control the direct interaction with
host OS tools like 7za.exe or /bin/zip.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the platform-specific files from cmake
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the 7za.exe (and legal notice)
This tool is no longer needed in the repo.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fixes usage of IArchive::GetFullPath()
This changed to return a PathView, updated to reflect that.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix promises and threads
Make sure promises are only set exactly once. This meant reworking some
of the initial error checking.
Detach threads when created. Adds [[nodiscard]] to the functions that
return a future. Since threads are detached, the future is the main way
to get communication from the thread.
Clean up interface, add comments.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* More edits to thread creation
Changes to thread construction to account for parameter change.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix some remaining issues with ArchiveComponent
Put created threads inside a container, then join them at Deactivate.
Fix asset bundler case when injecting a file with no working directory.
Fix thread constructor that applies args to a function.
Fix lambdas to take string args by value rather than reference.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fixes some remaining bugs in ArchiveComponent
Open archive as read-only during extract & list operations.
Fixes paths issues.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix initialize of opaque thread handle in thread_UnixLike.h
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed unused variable in AssetBundleComponent.cpp to fix compiler warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix some issues with archives
File paths in the CDR and the local headers need to match, but there
were issues with path separators and case that made it possible to get
invalid headers errors in some archives.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Adds some new ArchiveComponent unit tests
Adds new tests for extraction of archive and adding files from a file
list to an archive.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix file data offset issues when opening archives
When opening an INestedArchive it would run through the CDR headers to
create file entries in the zip cache. The offsets to the compressed
data were being calculated incorrectly because they were using the CDR
headers rather than jumping to the local file headers and getting
offsets from those sizes.
Removed and refactored some archive validation flow and zip cache
factory init methods to either init default or init w/ additional
validation checks.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Addresses PR feedback
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address more points of feedback in PR
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address additional PR feedback
Fixes up some error checks and uses of strings vs paths.
Enable archive component tests on Linux so we can see if they will
pass.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address PR feedback
Change the INestedArchive interface to list files as AZ::IO::Path.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Disabling the ArchiveComponent tests on Linux
They failed so we will revisit them to attempt a fix.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Rename a member variable to be more accurate
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address feedback on PR
Bump version of Archive Components for serialize context.
Improve error messages during archive open and validation.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Revert recent changes
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* [Mac] Initial support for building with hardened runtime enabled and code signing the binaries and bundles generated by the CMake install process.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Move call to ly_post_install_step
2. Entitlements should only be added for executables
3. Change use of CMake exec_program to newer execute_process
4. Remove broken symlinks from embedded Python frameworks
5. Run post install code signing only if hardened runtime is enabled
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove unnecessary flag
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove unnecessary additional call to condesign python inside a bundle
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Move commonly used install functions for codesigning, copying files, and fixing frameworks to a utility script
2. Remove unnecessary wait in the Editor/AP launchers I added earlier.
3. Codesign 3rd party libraries for distribution.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Codesigning on 3rd party libs should only happen when hardened runtime is enabled.
2. Change the order of the if blocks in Editor's main_dummy.cpp. This was causing strange notarization issues because it wass too similar to AP's main executable.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add new line to end of file
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Move architecture specification to PAL_mac cmake file. 2. Codesign failure should be fatal.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Address some PR feedback
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove unnecessary comment. Change if to use IN_LIST.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* HOME may not always be defined. Adding alternate POSIX way of determining HOME.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Checking in partial work to get O3DE SDK built as an app bundle. Has a bunch of debug code that needs to be cleaned up.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove this and add it back later with fixed casing.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Adding file back with fixed case
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Add entitlements sparingly(only when necessary)
2. Convert entitlements to plist files which we can directly pass to codesign
3. Install python site-packages in the o3de_sdk launcher and then launch the project manger.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Move hardened runtime check to codesigning functions only. This way, non-hardened runtime install is identical to the former except for codesign. Makes it easy for QA to test internally.
2. Move cmake min version for install to the pre-install steps.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Remove the dummy launchers for AssetProcessor and Editor
2. Add loader_path to the rpaths of binaries outside an app bundle so that the dynamic loader can load their dependencies if any.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove file named main_dummy.cpp
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add O3DE SDK launcher
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add missing runtime dependencies to gems
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Update the path to binaries when codesigning to the correct one.
2. Remove some debug messages.
3. Move installed binary path setreg generation to the target install function. This way, we get the correct path to the bundle accounting for different configs and subdirectories.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add explanatory comments.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. ly_install_add_install_path_setreg cannot be called during install target because the runtime dependencies are already processed by then.
2. The SDK launcher now uses the ProjectManager's bundle setreg to find the path to the installed binaries
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Update path to install relative binaries after merge from dev
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Only one config of the SDK launcher needs to be installed. Preinstall steps should not be run per config, but only once.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Install python dependencies using the get_python.sh script.
2. Replace any reference to hard-coded package name/version numbers with variables.
3. Add one more missing runtime dependency.
4. Misc. PR feedback.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Remove the need for setreg files in all tool bundle. Project binary path can be used instead.
2. Move O3DE_SDK_Launcher to Code/Tools/BundleLauncher
3. Add ly_install_run_script() function for install(SCRIPT) functionality.
4. Address some other PR feedback.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add source permission when installing O3DE_SDK executable.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Rename setreg file to add specialization tag.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove LY_BUILD_PERMUTATION that's not needed
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Add BinariesInstallPath.setreg to all our bundles like before. This is now only added during the install process though.
2. Fix path in Install_common.cmake
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Fix comment
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Avoid redefining `PRI` macros on Linux
The fixed width 64-bit integer types are defined differently per platform.
Some platforms define it as "long", others define it as "long long". For
consistency, `AZ::u64` or `AZ::s64` is always defined to "long long".
However, this causes problems for formatting those types, because on
platforms where `uint64_t` is a `long`, `PRIu64` gives the wrong format
string for `AZ::u64`. Previously this was fixed by redefining the `PRI`
macros so that they work for `AZ::u64`, but that breaks the ability to
format `uint64_t`.
We could add an AZ-specific version of the `PRI` macros for 64-bit integer
types, but we don't really need to, since they are `long long` on every
platform we support.
* Use `%ll` for `AZ::u64`
* Use `PRIu64` for `uint64_t`
Signed-off-by: Chris Burel <burelc@amazon.com>
* Avoid redefining `PRI` macros in CryCommon
Signed-off-by: Chris Burel <burelc@amazon.com>
* Modernization + small fix.
Modernize ( `bool`/`override`/other) code in AzCore, AzFramework, AzQtComponents, AzToolsFramework, etc.
Replaced a `bind` or two, use `using` in a few places as well.
Fix nullptr comparison of AZStd::function.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Apply review-based changes
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Release build fix for Windows
* Release build fix for Android
* Release build fix for Windows
* Release build fix for Android
* Release build fix for Linux
* Release build fix for Mac
* Release build fix for iOS
Signed-off-by: Steve Pham <spham@amazon.com>
* Minimal TypeInfo header/reduce std interdependencies.
TypeInfoSimple.h is a small header that can replace the use of
TypeInfo.h in some cases.
Signed-off-by: Nemerle <nemerle5+git@gmail.com>
* Windows build fixed
Removed algorithm.h from string_view.h
smoke-test passed
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Resotore dynamic_pointer_cast in intrusive_ptr
Requested by reviewer.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Fix CI build
string.h - missed alogorithm.h, since it was removed from string_view
NodeWrapper.h - missing smart_ptr.h
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
Co-authored-by: Nemerle <nemerle5+git@gmail.com>
* 1. Initial support for loading dylibs outside the bundle.
2. Child processes inherit parent's environment if no environment is explicitly specified(should change to append the parent's environment even if environment variables are explicitly specified).
3. Update some time functions to use system uptime instead of wall clock time when computing elapsed time. This fixes false timeouts when the OS goes to sleep.
4. Increase wait times for AssetBuilders and some Atom tools to connect to the AssetProcessor. This is needed because GateKeeper slows down first time bootup which results in asset processing failures.
With this change we'll be able to run Editor and AssetProcessor from an install on Mac and we will also be able to build and run projects using the installed engine as an SDK.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Remove debug messages.
2. Fix license
3. Pass parent's environment variables to child processes by default(on Mac).
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. Add more detailed comments.2. Use a custom ly_copy for Mac and leave the default as is.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Address some feedback from review
Signed-off-by: amzn-sj <srikkant@amazon.com>
Esteban Papp
rgba16f
lumberyard-employee-dm
Yaakuro
amzn-phist
SJ
Chris Burel
bosnichd
Artur K
AMZN-AlexOteiza
Steve Pham
Jeremy Ong
Jeremy Ong