* Fix issues with seedlist for AutomatedTesting
Fixes error reporting so it will show the file hint in the tool.
Removes any missing assets from the .seed file.
Remove an unnecessary dependency from AutomatedTesting dependencies
file.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Make ArchiveComponent use AZ::IO::IArchive
Initial changes that will get the sychronous calls in ArchiveComponent
to use IArchive interface rather than external zip/7z tools.
Some of the asynchronous api are still in place, anything that wasn't
being used has been removed for now. This may change later if we move
towards all the api being asynchronous. Until then, we can't remove the
reliance upon the external archive tools completely.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Updates AZStd::thread constructors
Adds a variadic constructor which forwards args to the functor.
Because of our thread_desc extension, there was confusion on the
arugments, so the args were reordered to take the thread_desc first,
before the functor and args.
Also the thread_desc is taken as reference rather than by pointer.
Update callsites to account for this change.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Async operation of ArchiveComponent api
This sets up the ArchiveComponent to operate asynchronously. It uses
promise/future to transfer results to caller.
This is still broken, there's a few things that need to get fixed up,
but this is a good checkpoint for the work as it solidifies the api,
cleans up a bunch of unused code, and compiles.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the platform-specific ArchiveComponen
These are no longer needed, as they control the direct interaction with
host OS tools like 7za.exe or /bin/zip.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the platform-specific files from cmake
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the 7za.exe (and legal notice)
This tool is no longer needed in the repo.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fixes usage of IArchive::GetFullPath()
This changed to return a PathView, updated to reflect that.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix promises and threads
Make sure promises are only set exactly once. This meant reworking some
of the initial error checking.
Detach threads when created. Adds [[nodiscard]] to the functions that
return a future. Since threads are detached, the future is the main way
to get communication from the thread.
Clean up interface, add comments.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* More edits to thread creation
Changes to thread construction to account for parameter change.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix some remaining issues with ArchiveComponent
Put created threads inside a container, then join them at Deactivate.
Fix asset bundler case when injecting a file with no working directory.
Fix thread constructor that applies args to a function.
Fix lambdas to take string args by value rather than reference.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fixes some remaining bugs in ArchiveComponent
Open archive as read-only during extract & list operations.
Fixes paths issues.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix initialize of opaque thread handle in thread_UnixLike.h
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed unused variable in AssetBundleComponent.cpp to fix compiler warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix some issues with archives
File paths in the CDR and the local headers need to match, but there
were issues with path separators and case that made it possible to get
invalid headers errors in some archives.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Adds some new ArchiveComponent unit tests
Adds new tests for extraction of archive and adding files from a file
list to an archive.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix file data offset issues when opening archives
When opening an INestedArchive it would run through the CDR headers to
create file entries in the zip cache. The offsets to the compressed
data were being calculated incorrectly because they were using the CDR
headers rather than jumping to the local file headers and getting
offsets from those sizes.
Removed and refactored some archive validation flow and zip cache
factory init methods to either init default or init w/ additional
validation checks.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Addresses PR feedback
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address more points of feedback in PR
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address additional PR feedback
Fixes up some error checks and uses of strings vs paths.
Enable archive component tests on Linux so we can see if they will
pass.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address PR feedback
Change the INestedArchive interface to list files as AZ::IO::Path.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Disabling the ArchiveComponent tests on Linux
They failed so we will revisit them to attempt a fix.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Rename a member variable to be more accurate
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address feedback on PR
Bump version of Archive Components for serialize context.
Improve error messages during archive open and validation.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Revert recent changes
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Intial attempt to convert the Atom/RHI/FrameScheduler to use the new TaskGraph api
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Avoid enqueuing work on the active task thread if the submitted task
graph is waitable
When submitting a task graph, supplying a wait event implies that
dependent jobs must occur on threads that do not wait on the event (in
the absence of work stealing). This change prevents this by adding a
notion of a task thread enable/disable state, and prohibiting dependent
jobs from being enqueued on waiting threads.
Signed-off-by: Jeremy Ong <jcong@amazon.com>
* Convert RPI/Scene to use TaskGraph pass 1, Culling jobs remain on the old system
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* RemoveTask Graph changes from the FrameScheduler::ExecuteGroups, use old job system instead
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Per review, removing commented out code
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Cleanup debug code, & build fix
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Add a cvar & interface to query whether to use jobs or task graph
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Make TaskGraph assert if you try to wait inside a job
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Fix TaskTest SpawnSubgraph to account for the new TaskGraphEvent assert on wait in a running task
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* 3 minor cleanups. 1) Events always store a ptr to their executor 2) Fix clang compile error 3) remove an early out.
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Fix double group end that was causing assert/crash plus misc minor diff's with development
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Fix deallocation failure on deactivation of the TaskGraphSystemComponent. Also make the system component account for multiple creation in Unit Tests.
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Update with PR feedback
1) Rename UseTaskGraph to IsTaskGraphActive & update related code
2) prefer TaskExecutor::SetInstance
3) add comments and remove commented out code
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Fix incorrect RTTI name for TaskGraphActiveInterface
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Move TaskGraphSystemComponent CRC calculation to a shared variable
Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
Co-authored-by: Jeremy Ong <jcong@amazon.com>
* Removed ununeeded includes from EBus EBus.h and Policies.h
Updated the locations which needed those includes
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Adding missing include for <memory> to AWsClientAuthBus.h
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove the while true loop in the EBusQueuePolicy Execute() function
The while true loop in Execute was for allowing additional functions to be queued in the middle of execution of current list of functions.
That functionality was dangerous, because if a queued function added itself during execution unconditionally, then it would result in an infinite loop
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AssetManager::DispatchEvents function to pump the AssetBus event queue until empty
Queued Events on the AssetBus is able to queue additional events on that Bus during execution of those events.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Changed the AssetManager::DispatchEvents function to only execute the
AssetBus queued events once
Changed the AssetJobsFloodTest.AssetWithNoLoadReference_LoadDependencies_BehaviorObeyed test to dispatch events until the OnAssetContainerReady callback is signaled.
This happens after every asset load to make sure that the
expiring AssetContainer instances are removed from `AssetManager::m_ownedAssetContainer` container before retrying to load the same asset.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a MaxTimeoutSeconds constant for the maximum amount of the time to run a single DispatchEvents loop
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
The new FileReader class allows using either a FileIOBase or SystemFile
classes for reading files.
If a non-nullptr FileIOBase is supplied, the FileIOBase is used,
otherwise SystemFile is used.
Updated the SettingsRegistryMergeUtils
`MergeSettingsToRegistry_ConfigFile` function to use the FileReader.
This allows .cfg such as the autoexec.cfg to be read using FileIO,
therefore allowing it to be read from the engine.pak file.
Because the AZ::Console::ExecuteConfigFile uses the
`MergeSettingsToRegistry_ConfigFile` function for merging *.cfg files to
the SettingsRegistry, this allows the AZ::Console to run console
commands from files within pak files thanks to the ArchiveFileIO being
the default FileIOBase instance
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added string and string_view class constructor overloads which is deleted that prevents initializing them from a nullptr or an integer type
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed locations where string and string_view were initialized with
nullptr
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix IArchive::IsInstalledToHDD signature
Updated calls to ConvertAbsolutePathToAliasedPath to use an
list initialization instead of nullptr
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed issue where two Paths could compare equal to each other, but hash
differently
This issue is caused by the Path comparison logic using the path
separator of the left path in a comparison of two paths(left and right)
to determine whether the PathComparison is case-sensitive or not.
The logic has been updated to only perform a non-case-sensitive path
comparison if both paths are using the WindowsPathSeperator of `\`
Also fixed issue with the Hashing algorihtm of the Path class to always
hash the root directory as if it is `/`.
This allows a path of "C:\foo" and "C:/foo" to hash to the equivalent
value.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* MS Build Tools 14.29 workaround around suppressing warnings using the external header feature
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a max_size function to all AZStd container style allocator
functions
The max_size functions returns the maximum value that a single contiguous
allocation value returns
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the BestFitExternalMapSchema and MallocSchema
GetMaxContiguousAllocationSize function
Those functions now return a Max allocation size of
AZ_CORE_MAX_ALLOCATOR size to indicate the maximum size for a single
allocation
Changed the IAllocatorAllocator::GetMaxContiguousAllocationSize function
from a pure virtual function to regular virtual function
Removed the left over String.cpp test to validate that the issue with
the allocator::max_size() function was occuring
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed PathView `MakeRelativeTo` and `Append` functions path segment
comparisons when using the Windows path separator of '\'
The PathSegment comparisons were case-sensitive in both those functions
and now use `Internal::ComparePathSegments` function to perform the
appropriate case comparison based on the path separator value of the
Path class
Reverted the LocalFileIO::CheckInvalidWrite function back to not
lowercasing the assets alias and input path before invoking
`PathView::IsRelativeTo`
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Simplified the LocalFileIO::ConvertToAliasBuffer logic
Fix for the ArchiveTest
`IResourceList_Add_AbsolutePath_RemovesAndReplacesWithAlias` and
`TestArchiveViaFileIO` test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a PathIterable structure stores a non-heap container of normalized path segments of an input path.
Moved the PathParser logic to a PathParser.inl file
Removed dependency of the PathView::IsRelativeTo logic on FixedMaxPath
There is no longer a 1024 character limit when determining if a path is relative to a base
Added a GetNormalPathParts and AppendNormalPathParts to function and removed LexicallyNormalInplace to share the logic for creating a normalized path between IsRelativeTo and LexicallyNormal
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Clang PathIterable.inl build fix
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed Normalize and Relative Path functions initialize the result paths
With the correct path separator for the paths being transformed
Ported over the Custom Path Root Separator logic to the PathParser.inl
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated he Shader Preprocessor include path gather.
It now uses AZ::IO::Path for the path operations and checks if the path
exist before adding it to the list of include paths.
Finally the set logic has been removed for a simpler find_if check to
see if the include path already since in the project include paths
This fixes the Asset Processing issues with shader includes due to the
Path.inl changes
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed tail recursion call to AppendNormalPathParts to supply a PathView with the same path separator as the parent call
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Adding reference qualifier overloads to the Path class Native function
Removed the conversion operators from the Path class for converting to a
string_type&/const string_type&
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Move a few specializations from AzStdOnDemandReflection.inl to cpp file
This reduces compilation time and produced profile/debug file sizes.
The specializations for string and string_view are only implemented for 'char' type, since
others are not used anywhere.
Extracted `Reflect` method from `ClientAuthAWSCredentials` to a cpp file.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Windows build fixes.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Added missing license.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Fix missing spaces in template argument lists
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Clang format on ClientAuthAWSCredentials.cpp
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Merge upstream development and fix linux build.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* add a new implementation for cone/ray intersect to simplify and fix existing issue
Signed-off-by: hultonha <hultonha@amazon.co.uk>
* move new ray/cone intersection function to AzToolsFramework - repond to PR comments
Signed-off-by: hultonha <hultonha@amazon.co.uk>
* updates following PR feedback
Signed-off-by: hultonha <hultonha@amazon.co.uk>
* add additional comment to give more context to the intersection function
Signed-off-by: hultonha <hultonha@amazon.co.uk>
* update google test expect usage
Signed-off-by: hultonha <hultonha@amazon.co.uk>
* Modernization + small fix.
Modernize ( `bool`/`override`/other) code in AzCore, AzFramework, AzQtComponents, AzToolsFramework, etc.
Replaced a `bind` or two, use `using` in a few places as well.
Fix nullptr comparison of AZStd::function.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Apply review-based changes
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
NOTE: The memory driller is still intact for now to avoid needing to
modify allocators, but the frame/cpu portions of driller and the
standalone executable are now gone.
Signed-off-by: Jeremy Ong <jcong@amazon.com>
* Added a deferred queue to the AZ Console class
An AZ Console instance will now store any console commands that could be
dispatched from a configuration file into a deferred queue, that can be
invoked later.
This can be used to defer execution of console commands in configuration files such as
.cfg, .setreg and .setregpatch files that are defined in gem modules that
have not been loaded yet.
The defered execution can then be invoked at any point later in the
application
Updated the Component Application CreateCommon function to invoke deferred console commands
after all the gems have loaded
fixes#2062
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed variable shadowing in the Console Deferred Command Test
Updated commit for the ClearDeferredQueue function to just mention
clearing the queue
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Filtered out execution of the ConsoleRootCommandKey as a console command
The AZ::Console notification handler is tracking changes to the fields
of "/Amazon/AzCore/Runtime/ConsoleCommands" and it's children.
Now the "/Amazon/AzCore/Runtime/ConsoleCommands" field is the
ConsoleRootCommandKey and not an actual console command so it shouldn't
attempt to be invoked
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the execution of deferred console commands after linking deferred
functors
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the execution of deferred console commands into CreateModuleClass
hook
Any module that loads using the ModuleManager system will attempt to
execute any deferred console commands to allow newly registered commands
from that module to be dispatched.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
amzn-phist
rgba16f [Amazon]
lumberyard-employee-dm
Artur K
Esteban Papp
hultonha
Jeremy Ong
carlitosan
Jeremy Ong