* Adds simple stats tracking to AssetProcessor
The system captures cumulative and individual processing times.
It avoids touching any part of the app which affects decision
making or flow, or altering any structures such as JobEntry which
are involved in processing, in order to keep it as simple and
small as possible.
Signed-off-by: lawsonamzn <70027408+lawsonamzn@users.noreply.github.com>
* Follow up fix for ATOM-13512
Update the file mask mapping in ImageBuilder.settings.
Change Reflectance.preset to use BC4 format since it only needs once channel.
Signed-off-by: Qing Tao <55564570+VickyAtAZ@users.noreply.github.com>
The AssetProcessor on Linux uses `inotify` to monitor for file updates.
The AssetProcessor also uses a `ProcessWatcher` to launch child
AssetBuilder processes. `ProcessWatcher` accomplishes this by calling
`fork()`, which duplicates the current process, then calling `exec()`.
The `fork()` call also appears to duplicate any inotify fds. This
results in the subprocess consuming duplicate inotify watches, which is
a limited system resource (Ubunut 20 defaults
`/proc/sys/fs/inotify/max_user_watches` to 65536). The AssetProcessor
still has issues with this max, but this should free up at least half of
the uses.
Signed-off-by: Chris Burel <burelc@amazon.com>
Changes include:
- Move config files from ImageProcessingAtom/Config/ folder to ImageProcessingAtom/Assets/Config/ folder so it can be watched as source depencies.
- Change GetSuggestedPreset function so it can return certain preset for certain file name.
- Move file mask mappings from preset to ImageBuilder.settings. But the file masks in preset can still be used.
- Changed from using project config folder's imageBuilder.Settings for override to using it as json merge patch.
- Expose GetFileHash function in AssetBuilderSDK api.
Signed-off-by: Qing Tao <55564570+VickyAtAZ@users.noreply.github.com>
* Cleaning up errors with default assets, used in bundled release builds
Signed-off-by: AMZN-stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Updated simple asset references to be to the product, not source assets
Signed-off-by: AMZN-stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
Leaving in the print for absorbed asserts to avoid running into future situations where important asserts are accidentally absorbed
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Enable relocation of the Project Game Release Layout
Relocating the Project Game Release Layout to another directory on the file system failed due to the querying of the engine root failing due to the ComponentApplication::m_engineRoot not using the project path stored in the SettingsRegisry if the engine root cannot be detected
Removed the ApplicationRequestBus GetEngineRoot function.
The ComponentApplicationRequestBus has a function of the same name that returns the same path.
Removed the deprecated GetAppRoot function.
The path it returns has no defined value. It was not the engine root or the project root.
Removed unused CFileUtil and CFileUtil_impl functions that were invoking the ApplicationREquestBus GetEngineRoot function.
On the way to update the functions it was discovered that they aren't called
Added a CalculateBranchToken overload that can populate a fixed_string to avoid heap allocations
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Protect against an empty list of artifacts to remove when generating the
engine.pak
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix test thread being created multiple times
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Update test to not use a callback
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add some more comments
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add back the callback, remove the use of a thread/sleep
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
(cherry picked from commit 73202c2091)
* Fixed the return value of the ConvertToAbsolutePath function
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added the generated cmake_dependencies.*.setreg files to engine.pak (#5073)
* Copied the generated cmake_dependencies.*.setreg file to the Cache
directory
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed the platform name from the bootstrap.game.*.setreg
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Update how the project related file paths are determined when not
supplied.
The project-path determination now goes back to only detecting a "project.json" file.
It no longer attempts to detect a "Cache" directory
The project-cache-path determination now in addition to checking the
project_cache_path key searches for a "Cache" directory.
The project-path defaults to executable folder if it cannot be detected.
The copying of generated executable folder Registry directory contents
to the product cache is now removed after the archive step.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the invocation of the AssetProcessor in Jenkins to supply an
absolute path to the project.
The project-path is no longer treated as relative to the engine root,
but instead relative to the current working directory at application
startup.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added constant for the storing the name of Cache directory
Fixed typos and grammatical errors in the SettingsRegistryMergeUtils.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated UnitTest prepend the EngineRoot path to "AutomatedTesting" when
setting the project path.
This is needed now that the project-path isn't treated relative to the
EngineRoot if it is not absolute.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix AssetSeedManagerTest and PlatformAddressedAssetCatalogManagerTest
Instead of trying to used the AutomatedTesting directory as the project root, the temp directory created during the test is used as the project root.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the setting of the project cache root folder and project
asset platform root folder into the `if (!projectCachePath.empty())`
block
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removing the scan up logic for the "Cache" directory.
This is no longer needed to locate the project cache path in a Project Game Release Layout.
Because the project path defaults to the executable directory if, it is not found, the Cache directory will be set to the "Cache" directory within the executable directory.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix test thread being created multiple times
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Update test to not use a callback
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add some more comments
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add back the callback, remove the use of a thread/sleep
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Fixed some files missed when groundplane_521 was renamed to 512 (#4958)
* Fixed references to 521x521 to reference the correct 512x512 FBX file
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Fixed asset hints
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Moved the Asset Catalog loading from LmbrCentral to the AzFramework::Application (#4568)
* Moved the loading of the AssetCatalog from LmbrCentralSystemComponent to AzFramework Application
Modified the AssetCatalog::InitializeCatalog function to no longer rely on the TickBus to send out the `AssetCatalogEventBus::OnCatalogLoaded` event.
It now queues a function on the AssetCatalogRequestBus to send the OnCatalogLoaded event as soon as the dispatching for the AssetCatalogRequestBus has completed on the current thread.
This is done by updating the AssetCatalogRequestBus to use EBus ThreadDispatchPolicy to add a callback to invoke any queued function has soon a thread has finished dispatching and has released its DispatchMutex
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AssetCatalogRequestBus to add a custom DispatchLockGuard
The AssetCatalogRequestBus uses the custom lock guard to dispatch queued
events after it has unlocked it's context mutex for the current thread.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed GetContext call from the
AssetCatalogRequests::PostThreadDispatchInvoker
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the definition of FileTagQueryManager::GetDefaultFileTagFilePath
function to return a path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AZ_CONSOLEFREEFUNC macro to actually use the _NAME
The _NAME parameter was not being used before, resulting in the Console
stringified name of the function being used.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed CrySystem dependencies from the BundlingSystemComponent
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the loading of the AssetCatalog from LmbrCentralSystemComponent to AzFramework Application
Modified the AssetCatalog::InitializeCatalog function to no longer rely on the TickBus to send out the `AssetCatalogEventBus::OnCatalogLoaded` event.
It now queues a function on the AssetCatalogRequestBus to send the OnCatalogLoaded event as soon as the dispatching for the AssetCatalogRequestBus has completed on the current thread.
This is done by updating the AssetCatalogRequestBus to use EBus ThreadDispatchPolicy to add a callback to invoke any queued function has soon a thread has finished dispatching and has released its DispatchMutex
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AssetCatalogRequestBus to add a custom DispatchLockGuard
The AssetCatalogRequestBus uses the custom lock guard to dispatch queued
events after it has unlocked it's context mutex for the current thread.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed GetContext call from the
AssetCatalogRequests::PostThreadDispatchInvoker
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the definition of FileTagQueryManager::GetDefaultFileTagFilePath
function to return a path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AZ_CONSOLEFREEFUNC macro to actually use the _NAME
The _NAME parameter was not being used before, resulting in the Console
stringified name of the function being used.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed CrySystem dependencies from the BundlingSystemComponent
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Addded missing template parameter to AssetCatalogRequests
The fixes the compile error.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Adding AssetBus::MultiHandler::BusDisconnect call
The BlastSystemComponent was connecting to the Bus, but not
disconnecting from it, causing an assert to fire to it being a
multi-thread bus
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added support for DataDrive lifecycle events to the ComponentApplication
The events are using the SettingsRegistry NotifyEvent to track when
certain keys are modified to signal handlers.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Corrected invalid JSON creation in ModuleManager::DeactivateEntities
Resolved clang warning about used type alias
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix for dangling reference in lambda registered to the SettingsRegistry
Notifier event
This was causing the EditorPythonBinding tests to crash due to the
following circumstances.
First Python has created an instance of a SettingsRegistryProxy
Second the SettingsRegistry sends an event during the time when the
SettingsRegistryProxy exists.
This issue was exposed due to the ComponentApplication Lifecycle events
using the SettingsRegistry to dispatch during various times of the
application workflow.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added the generated cmake_dependencies.*.setreg files to engine.pak (#5073)
* Copied the generated cmake_dependencies.*.setreg file to the Cache
directory
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed the platform name from the bootstrap.game.*.setreg
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixes for release builds with DCO fix (#5164)
* This set of changes is work toward allowing release builds to work with asset bundler generated bundles and legacy, non-prefab levels. This requires some other in-flight changes before this work is complete.
Updated engine seed list + fixed automated test
ComponentApplicationLifecycle has the ability to automatically register events if asked to register a handler and the event doesn't exist. This is only intended for cases where you need to register a handler early in startup before the settings registry file is loaded.
Added two new lifecycle events: One after the system entity has been activated, and one after the system interface has been created.
If you load an archive before the system entity has been activated, archive.cpp caches information about those archives until that time, so it can finish registration. This is because the serialization system and BundlingSystemComponent both need to be available to do this registration, but the bundles have to be loaded before those are initialized so that the settings registry file can be loaded.
Fixed an error were mounted pak files were searching for levels.pak and not level.pak, and not finding them. I'm pretty sure this logic doesn't do anything functional either way, but I've been testing legacy levels with this change and they work now.
Moved wildcard pak loading to where engine.pak is loaded. This is because the settings registry file that defines the IO stack to spin up must be available early in application startup, and this file must be within a mounted pak file. If you're using asset bundler generated bundles, they need to be loaded at this time so that file can be loaded.
Atom's BootstrapSystemComponent.cpp no longer initializes on AssetCatalogLoaded, and instead initializes on the ApplicationLifecycle event SystemInterfaceCreated. This is because the base assetcatalog.xml file is really just a development time concept, this file should not be used in packaged release builds, because those builds will make use of delta catalogs in each bundle loaded. The asset catalog contains the list of all assets that were in the cache at development time, and this contains content that developers don't want to ship, and they may want to specifically hide from their customers, so data miners don't find secrets about upcoming game content.
Recovering from a branch that had incorrect DCO
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Fixed an incorrect ebus disconnect and removed an include that's no longer needed
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Fixed a copy and paste typo from trying to recover the previous pull request
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Updated product IDs for the settings registry builder to no longer collide with the JSON builder. Now they are based on a hash of the configuration.
Updated the engine default seed list to include the new asset ID info for the renamed bootstrap file
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Updated the path to the application lifecycle events, because runtime settings aren't included in the merged bootstrap file.
Addressed some feedback on printing out a string view on an error
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Removed a test that uses old assets that aren't relevant. We may not need this test anymore, but if we do we've backlogged a task to create a new test to cover this behavior without using old assets.
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Renamed SystemInterfaceCreated event to LegacySystemInterfaceCreated
Removed SystemEntityActivated event. Now that I have the rest of the fixes in this pull request, this new event wasn't needed, the already existing SystemComponentsActivated event does what I need.
Changed list to vector
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added newline to debug prints. Added additional debug prints based on previous segfault log
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add more debug messaging
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add db path to output
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Fix handling of absolute path dependencies when a newly added file satisfies a previously added dependency
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Fixed relative path wildcard dependencies matching absolute paths
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Update extra unit test to only run on windows since this problem doesn't apply to other OSes
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* This set of changes is work toward allowing release builds to work with asset bundler generated bundles and legacy, non-prefab levels. This requires some other in-flight changes before this work is complete.
Updated engine seed list + fixed automated test
ComponentApplicationLifecycle has the ability to automatically register events if asked to register a handler and the event doesn't exist. This is only intended for cases where you need to register a handler early in startup before the settings registry file is loaded.
Added two new lifecycle events: One after the system entity has been activated, and one after the system interface has been created.
If you load an archive before the system entity has been activated, archive.cpp caches information about those archives until that time, so it can finish registration. This is because the serialization system and BundlingSystemComponent both need to be available to do this registration, but the bundles have to be loaded before those are initialized so that the settings registry file can be loaded.
Fixed an error were mounted pak files were searching for levels.pak and not level.pak, and not finding them. I'm pretty sure this logic doesn't do anything functional either way, but I've been testing legacy levels with this change and they work now.
Moved wildcard pak loading to where engine.pak is loaded. This is because the settings registry file that defines the IO stack to spin up must be available early in application startup, and this file must be within a mounted pak file. If you're using asset bundler generated bundles, they need to be loaded at this time so that file can be loaded.
Atom's BootstrapSystemComponent.cpp no longer initializes on AssetCatalogLoaded, and instead initializes on the ApplicationLifecycle event SystemInterfaceCreated. This is because the base assetcatalog.xml file is really just a development time concept, this file should not be used in packaged release builds, because those builds will make use of delta catalogs in each bundle loaded. The asset catalog contains the list of all assets that were in the cache at development time, and this contains content that developers don't want to ship, and they may want to specifically hide from their customers, so data miners don't find secrets about upcoming game content.
Recovering from a branch that had incorrect DCO
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Fixed an incorrect ebus disconnect and removed an include that's no longer needed
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Fixed a copy and paste typo from trying to recover the previous pull request
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Updated product IDs for the settings registry builder to no longer collide with the JSON builder. Now they are based on a hash of the configuration.
Updated the engine default seed list to include the new asset ID info for the renamed bootstrap file
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Updated the path to the application lifecycle events, because runtime settings aren't included in the merged bootstrap file.
Addressed some feedback on printing out a string view on an error
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Removed a test that uses old assets that aren't relevant. We may not need this test anymore, but if we do we've backlogged a task to create a new test to cover this behavior without using old assets.
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Renamed SystemInterfaceCreated event to LegacySystemInterfaceCreated
Removed SystemEntityActivated event. Now that I have the rest of the fixes in this pull request, this new event wasn't needed, the already existing SystemComponentsActivated event does what I need.
Changed list to vector
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Copied the generated cmake_dependencies.*.setreg file to the Cache
directory
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed the platform name from the bootstrap.game.*.setreg
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Pass relative path IsFileExcluded so that only paths relative to the root scan folder are matched against the exclude filters.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Revert previous change. Remove the exclude filter for the Install directory.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Pass in relative path to the exclude filter as before. Fix the AssetScanner tests.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Prepend a ./ to the relative path in order to match the exclude patterns
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove hack to prepend ./. Update the exclude patterns so that the hack is no longer required.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add missing ? and remove whitespace
Signed-off-by: amzn-sj <srikkant@amazon.com>
* 1. IsFileExcluded() now converts the input path to a path that's relative to its corresponding scan folder.
2. Update regex patterns in gems and AutomatedTesting as well.
3. Remove unnecessary escaping for '/'.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Use ConvertToRelativePath() function to compute path relative to a scan folder.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* More fixes to regex patterns
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove test case which tests a hypothetical scenario that cannot occur. Fix another test case by adding scan folder.
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Remove assert that's not needed since it's a valid scenario
Signed-off-by: amzn-sj <srikkant@amazon.com>
* Add unit tests for ResolveSourceFileDependencyPath
Fix crash when file is not in scan folder
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Switch to using vector of AZ strings for better error output
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Switch to using UnorderedElementsAre
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add some comments and fix typo
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add an error message to AP when bad project path
Produce a log error or a dialog box error when the project path for AP
does not have a project.json and is invalid.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix a failing unit test - AssetProcessorMessages
Adding a check for 'project.json' caused BeforeRun() in a test fixture
to fail. Teardown of the fixture was also broken if the test failed to
fully startup the application manager, so added null checks there.
Added an assert to the fixture's Setup to check the status of BeforeRun().
Added additional settings registry setup to the fixture to make sure the
project path and branch token are configured before BeforeRun() is
called.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Convert resolved wildcard paths to relative path before saving in database.
Warn if file could not be converted to a relative path.
Fix FindWildcardMatches path handling that could result in pathMatch missing the first character
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Handle abs path wildcard dependencies
Remove dependencies outside of scan folder
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Switch to AZ::IO::PathView for abs path check
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Made code a little more clear
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add "Registry" folders as scan folders
For projects, Gems, and Engine, add the "Registry" folder as scan
folders.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Change params for adding scan folder
Also add trailing newlines to setreg files.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Adjust the priority order for project templates
Scan folder orders were way too high, they should be very low to become
highest priority order.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes change of default assets folder to Assets
Need to revert this change so it can be done separately.
Renamed the scan folder to 'Project/Assets' to prep for Assets folder
change later on.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Update platform configuration test to pass
Adjusted expectation of scan folder count from 1 to 2 per Gem for
'Assets' and 'Registry' now.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear
The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`
The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`
The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated use of devassets and devroot properties in python
Those properties now use projectroot and engroot
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed hardcoded size of 9 for the product asset alias.
The ResolvePath function now just appends the @projectproductassets@
alias with the input path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix for typos in Hydra python test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a deprecated alias map to the FileIO System
When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated python test scripts to use the projectroot binding
Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated references to devroot and devgame within the codebase
The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated <engroot>/AutomatedTesting references to projectroot
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Correct casing in emfxworkspace file
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance
The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed unused parameter warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Undid change to the LexicallyProximate function to set the path result to the base path.
It needs to return the *this path if the pathResult is empty
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils
Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Changed the AWSMetricsGemMock to use a temporary for writing test assets
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"
IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed whitespacing issues in Application.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated references to the Project Asset Cache in the PythonTests.
Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed the FileIO Deprecated Alias test on Windows
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removing @projectsourceassets@ alias, as it is only used once.
Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced @projectproductassets@ alias with @products@
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Rollback changes to the PhysX EditorSystemComponent.cpp
The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix "index out of range" error
When the parent is the tree root element, beginInsertRows must be
called with an invalid (but legal) index. A QModelIndex with a row
index of zero when the parent has no children is an illegal index
and will result in "undefined behavior", like the "index out of
range" one.
Therefore, if our parent is the tree root element, we use QModelIndex()
instead.
Fixes#2343
Signed-off-by: Dennis Brakhane <brakhane@gmail.com>
* Use QModelIndex() instead of createIndex(-1, -1)
Both do the same, but the former is Qt best practise.
Signed-off-by: Dennis Brakhane <brakhane@gmail.com>
* add some sanity checks in debug mode
Using illegal ModelIndices can result in hard to debug problems later
on, so add a few checks to help spotting them sooner.
Signed-off-by: Dennis Brakhane <brakhane@gmail.com>
* Updated the AsetProcessorManagerTest LockFileTest to check the SourceFileNotificationMessage
If the SourceFileNotificationMessage indicates that the source file was
removed, it triggers the callback to spin off a thread to remove the
product file.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Changed SourceFileNotificationMessage cast to use azrtti_cast
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* adding Windows/release to PR-validation builds
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* changing trace back to expand to nothing for release
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* typo
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* more fixes
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* fixing some more unused variable cases
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* renaming file in ScriptCanvas that causes a msbuild warning
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* reverting a previous change
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* Fix issues with seedlist for AutomatedTesting
Fixes error reporting so it will show the file hint in the tool.
Removes any missing assets from the .seed file.
Remove an unnecessary dependency from AutomatedTesting dependencies
file.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Make ArchiveComponent use AZ::IO::IArchive
Initial changes that will get the sychronous calls in ArchiveComponent
to use IArchive interface rather than external zip/7z tools.
Some of the asynchronous api are still in place, anything that wasn't
being used has been removed for now. This may change later if we move
towards all the api being asynchronous. Until then, we can't remove the
reliance upon the external archive tools completely.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Updates AZStd::thread constructors
Adds a variadic constructor which forwards args to the functor.
Because of our thread_desc extension, there was confusion on the
arugments, so the args were reordered to take the thread_desc first,
before the functor and args.
Also the thread_desc is taken as reference rather than by pointer.
Update callsites to account for this change.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Async operation of ArchiveComponent api
This sets up the ArchiveComponent to operate asynchronously. It uses
promise/future to transfer results to caller.
This is still broken, there's a few things that need to get fixed up,
but this is a good checkpoint for the work as it solidifies the api,
cleans up a bunch of unused code, and compiles.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the platform-specific ArchiveComponen
These are no longer needed, as they control the direct interaction with
host OS tools like 7za.exe or /bin/zip.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the platform-specific files from cmake
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Removes the 7za.exe (and legal notice)
This tool is no longer needed in the repo.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fixes usage of IArchive::GetFullPath()
This changed to return a PathView, updated to reflect that.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix promises and threads
Make sure promises are only set exactly once. This meant reworking some
of the initial error checking.
Detach threads when created. Adds [[nodiscard]] to the functions that
return a future. Since threads are detached, the future is the main way
to get communication from the thread.
Clean up interface, add comments.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* More edits to thread creation
Changes to thread construction to account for parameter change.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix some remaining issues with ArchiveComponent
Put created threads inside a container, then join them at Deactivate.
Fix asset bundler case when injecting a file with no working directory.
Fix thread constructor that applies args to a function.
Fix lambdas to take string args by value rather than reference.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fixes some remaining bugs in ArchiveComponent
Open archive as read-only during extract & list operations.
Fixes paths issues.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix initialize of opaque thread handle in thread_UnixLike.h
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed unused variable in AssetBundleComponent.cpp to fix compiler warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix some issues with archives
File paths in the CDR and the local headers need to match, but there
were issues with path separators and case that made it possible to get
invalid headers errors in some archives.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Adds some new ArchiveComponent unit tests
Adds new tests for extraction of archive and adding files from a file
list to an archive.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Fix file data offset issues when opening archives
When opening an INestedArchive it would run through the CDR headers to
create file entries in the zip cache. The offsets to the compressed
data were being calculated incorrectly because they were using the CDR
headers rather than jumping to the local file headers and getting
offsets from those sizes.
Removed and refactored some archive validation flow and zip cache
factory init methods to either init default or init w/ additional
validation checks.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Addresses PR feedback
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address more points of feedback in PR
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address additional PR feedback
Fixes up some error checks and uses of strings vs paths.
Enable archive component tests on Linux so we can see if they will
pass.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address PR feedback
Change the INestedArchive interface to list files as AZ::IO::Path.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Disabling the ArchiveComponent tests on Linux
They failed so we will revisit them to attempt a fix.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Rename a member variable to be more accurate
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Address feedback on PR
Bump version of Archive Components for serialize context.
Improve error messages during archive open and validation.
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
* Revert recent changes
Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed ununeeded includes from EBus EBus.h and Policies.h
Updated the locations which needed those includes
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Adding missing include for <memory> to AWsClientAuthBus.h
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove the while true loop in the EBusQueuePolicy Execute() function
The while true loop in Execute was for allowing additional functions to be queued in the middle of execution of current list of functions.
That functionality was dangerous, because if a queued function added itself during execution unconditionally, then it would result in an infinite loop
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AssetManager::DispatchEvents function to pump the AssetBus event queue until empty
Queued Events on the AssetBus is able to queue additional events on that Bus during execution of those events.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Changed the AssetManager::DispatchEvents function to only execute the
AssetBus queued events once
Changed the AssetJobsFloodTest.AssetWithNoLoadReference_LoadDependencies_BehaviorObeyed test to dispatch events until the OnAssetContainerReady callback is signaled.
This happens after every asset load to make sure that the
expiring AssetContainer instances are removed from `AssetManager::m_ownedAssetContainer` container before retrying to load the same asset.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a MaxTimeoutSeconds constant for the maximum amount of the time to run a single DispatchEvents loop
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>