* Add linux equivalents for all the places that enumerate platforms
Signed-off-by: Chris Burel <burelc@amazon.com>
* Fix the AssetFileInfoListComparisonTest fixture to not hardcode the PC platform
Signed-off-by: Chris Burel <burelc@amazon.com>
By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.
Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.
Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows ScenePI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.
Testing:
- Changing the registery setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switcihng the material conversion registry setting from true to false. (Details below)
- TODO: Will merge this change to a customer's fork and test on their existing content.
Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
- Entity 1 material assignments
- Blue_Zaxis: left as-is
- Green_Yaxis: exported the material
- Red_Xaxis: exported the material, and changed the material instance color to pink
- StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
- With_Texture: selected an existing brick material, changed the material instance color to red.
- Entity 2 material assignments
- Default Material: set to an existing brick material
- Blue_Zaxis: manually assigned built-in material that was converted from fbx
- Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Engine improvements/fixes
Fixed behavior that made the editor automated test to be sometimes stuck if lost the focus is lost.
Added support for specifying multiple tests to in batch to the editor, this is achieved by passing --runpythontest with the tests separated by ';'
Added new cmdline argument --project-user-path for overriding the user path. This allows to have multiple editors running writing logs and crash logs in different locations.
Moved responsability of exiting after a test finishes/passes out of ExecuteByFilenameAsTest, callers will use the bool return to know if the test passed.
Editor test batch and parallelization implementation:
Now the external python portion of the editor tests will be specified via test specs which will generate the test. Requiring no code. This is almost a data-driven approach.
Tests can be specified as single tests, parallel, batchable or batchable+parallel
Command line arguments for pytest to override the maximum number of editors, disable parallelization or batching.
Automated tests for testing this new editor testing utility
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
Co-authored-by: Garcia Ruiz <aljanru@amazon.co.uk>
* Final update copyright headers to reference license files at the repo root
Signed-off-by: spham <spham@amazon.com>
* Fix copyright validator unit tests to support the stale O3DE header scenario
Signed-off-by: spham <spham@amazon.com>
- On Windows the game now has file read sharing enabled for debug and profile builds. On Windows 10 the performance impact is neglectable, so it's been turned on to remove the need for the "cl_streamerDevMode" command line option.
- The drive that handles the virtual file system has been added to the game by default for debug and profile builds. Previously this required using "cl_streamerDevMode" which can now be omitted. A previous update already made it so that the drive would only be added if /Amazon/AzCore/Bootstrap/remote_filesystem was set to 1, but the configuration wasn't updated to reflect this.
- Removed some comments to keep the setreg files clean. The non-specialized versions of the same setreg files still have the comments.
- Removed the "DevMode" configuration (used by cl_streamerDevMode) from the game except for debug and profile. This configuration contained development tools which are not needed for release builds.
* [ATOM-15472] Shader Build Pipeline: Remove Deprecated Files And Functions That
Predate The Shader Supervariants
These are the essential impactful changes as a result of deprecating the
ShaderResourceGroupAsset.
* Addressed feedback by @moudgils. Better comments in header files.
* More updates related with deprecation of ShaderResourceGroupAsset
* Deleted the temporary version 2 classes.
* Updated version of the shader asset builders.
* Updated version of all the shader related classes impacted
by the Supervariant concept and deprecation of ShaderResourceGroupAsset
* Changes to *.pass and DGI, Reflections and RayTracing.
* changes to material related assets
* changes to core lights
* Changes to auxgeom/dynamic draw.
* changes to decals, lyshine, imguipass
* changes to RPI Pass classes
* Shader for SceneSrg, ViewSrg and ForwardPass Srgs.
* changes to mesh, skinned mesh, Morphtarget.
* Fixes to RayTracingPass.cpp & now allow empty srg in shaders.
* Updated Atom_RPI.Tests
* Simplified InstanceDatabase by removing AddHandler
------------------------------------------------------------------------------------
* Updated DiffuseGI precompiled shaders.
Added RayTracingSceneSrg and RayTracingMaterialSrg shader asset.
Updated ShaderAssetCreator::Clone to handle the supervariant when processing root variants.
Co-authored-by: Doug McDiarmid <dmcdiar@amazon.com>
------------------------------------------------------------------------------------
* Changed semantics for some PassSrg to SRG_PerPass_WithFallback.
AuxGeom/FixedShapeProcessor.cpp requires SRG_PerDraw on ObjectSrg.
Removed names of SceneSrg and ViewSrg from RPISystemDescriptor.cpp
* Moved ShaderLib/Atom/Features/DummyEntryFunctions.azsli
To Gems/Atom/RPI/Assets/ShaderLib/Atom/RPI/DummyEntryFunctions.azsli
Removed redundant checking for finalization in
ShaderResourceGroupLayout.cpp
* Fixed race condition bug for Shader::FindOrCreate.
InstanceDatabase<>::CreateInstance() needs to be atomic
for instance creation and initialization.
Added optional InstanceHandler::CreateFunctionWithParams to accomodate
to the needs of Instances that need more than an asset reference
to be able to be created an initialzed.
Removed ShaderResourceGroup::FindOrCreate() only ::Create is available
now.
* Renamed scene_and_view_srgs.* as SceneAndViewSrgs.*
Changed GetAzslFileOfOrigin for GetUniqueId
* Fixed unit tests.
* Reverted the serialization name of m_uniqueId back to
"m_azslFileOfOrigin" so precompiled shaders don't fail
in layout comparison.
* Fixed AtomCore.Tests
Removed non-applicable test. InstanceDatabase.AddHandler() is not
available anymore.
* The Null rhi is re-enabled for shader compilation.
Signed-off-by: garrieta <garrieta@amazon.com>
This set of changes enables conversions for singly-nested slices. Multiple nesting hierarchies are only partially supported at this point. Conversion is also significantly more deterministic, which makes it easier to convert single slices without needing to reconvert every slice or level that relies on it as well.
Changes:
- Added version of Instance::AddInstance() that takes in an alias to allow for deterministic aliases
- Added a "SliceConverterEditorEntityContextComponent" that's used to specifically disable entity activation on creation. The disabling is done this way vs adding a new public API, because the disable shouldn't be required in any normal case outside of this tool.
- Disabled more AWS gems for the SliceConverter, as they're unneeded and cause issues if they're around in the tool.
- Added a small null check to the Camera Controller.
- Added the actual support for slice instance conversion. This instantiates the entities, applies the data patches, turns them into a prefab instance, and generates a JSON patch out of the changes.
* Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)"
This reverts commit 58adcf168f.
* Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter"
This reverts commit 798d96f1a2, reversing
changes made to eb31d90ad9.
* Revert "Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter""
This reverts commit c1124f26d957388e88cc4990021314b5af247e1d.
* Revert "Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)""
This reverts commit 978477097892a22e83519646527ff52ba6532f35.
* Fixed how FbxImportRequestHandler is loaded
* Bumped version to force FBX to rebuild + removed unused variable
* Revert "Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)""
This reverts commit 978477097892a22e83519646527ff52ba6532f35.
* Revert "Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter""
This reverts commit c1124f26d957388e88cc4990021314b5af247e1d.
* Fixed a bad revert
* Better error reporting at AP launch
* AZ_CRC -> AZ_CRC_CE and removed delayed reload of settings registry file now that it's available at startup
* fixed typo in comment
The priority threshold to consider a task high priority can now be configured through the Settings Registry under key "/O3DE/AzFramework/Spawnables/HighPriorityThreshold".
Nested slices are now detected, converted into prefabs, and the top-level prefab will get linked to the nested prefabs with the proper number of instances. However, the nested prefabs won't retain any of the slice override values or parent entity hierarchy. That will (hopefully) be added in a separate PR.
This also adds support for better relative source paths for nested prefabs. To support this, the tool now needs to connect/disconnect with the AssetProcessor to be able to turn a slice asset ID into a relative source path, so that nested templates can be looked up and converted correctly.
* Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)"
This reverts commit 58adcf168f.
* Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter"
This reverts commit 798d96f1a2, reversing
changes made to eb31d90ad9.
* Updated the DynamicModuleHandle code to search within the
SettingsRegistry for the FilePathKey_ProjectBuildPath setting in order
to determine the binary directory for the Project.
This is used to locate shared libraries and executables built by the
project when running and Engine SDK binary from within the Engine SDK
* Added a generation step within the Projects.cmake file to generate a
.setreg file containing the CMake build directory root.
The file is output to the <project-root>/user/Registry/build_path.setreg
file.
This occurs only on non-host platforms when configuring for
non-Monolithic builds, since the Build Directory is used to located the
project binary directory in order to located the project's generated
${CMAKE_BINARY_DIR}/bin/$<CONFIG>/Registry directory containing the
cmake_dependencies.<project-name>.<application-name>.setreg file
containing the list of gem modules to load for a given application
Updated the SettingsRegistryMergeUtils AddRuntimeFilePaths function to
be read in the new "/Amazon/Project/Settings/Build/project_build_path"
and use that to form an absolute path to the project build directory by
appending it to the FilePathKey_ProjectPath key.
That key is set in the 'FilePathKey_ProjectBuildPath' constant
Next updated the SettingsRegistryMergeUtils
MergeSettingsToRegistry_TargetBuildDependencyRegistry function to look
within the project build directory to locate the
cmake_dependencies.*.setreg file to load
Tweaked the Settings Registry merge order of the ComponentApplication,
GameApplication and Settings Registry builder to merge the command line
after merging the global user registry and after merging the project
user registry.
Moved the call to MergeSettingsToRegistry_TargetBuildDependencyRegistry
to occur after the above calls to make sure the properly overriden
projects' user registry was merged in order for the correct project
build path to be stored in the SettingsRegistry
* Added a ProjectConfigurationBinPath key which contains the path to the <build-dir>/bin/$<CONFIG> directory for a project which is used to load gem dlls and the Registry/cmake_dependencies.*.setreg files when using an pre-built Editor/AssetProcessor on an external project
* Fixed variable reference to fileNamePath variable
* Removing the default Project Build Path from the Settings Registry
Runtime Filepaths.
Any paths would have to be set explicitly via .setreg/.setregpatch file
or the --project-build-path parameter
Updated the setting of the project build path and project binary
directory to perform existance checks on the paths before setting the
keys of /Amazon/AzCore/Runtime/FilePaths/ProjectBuildPath and
/Amazon/AzCore/Runtime/FilePaths/ProjectConfigurationBinPath
Added a backup project binary path of
<project-build-path>/bin/$<PLATFORM>/$<CONFIG> which is used if the path
of <project-build-path>/bin/$<CONFIG> has not been found
Fixed compile error in DynamicModuleHandle_Apple.cpp
* UnixLike Platform Build fix for the DynamicModuleHandle code
- Uses ToolsApplication instead of ComponentApplication so that built-in Editor components are recognized and read in correctly
- Starts up all the DynamicModules immediately so that the System Components are activated, which registers asset handlers and allows asset references to serialize in correctly
- Adds a -specialization command-line flag to specify which project specialization to use (editor, game, etc)
- Removes the filter to ignore unknown classes since they should all now be "known"
- Adds a few gem autoload flags and a null thumbnail service so that Qt and Python systems can be skipped, as they aren't needed for the data conversions and would bring additional overhead and complications
* LYN-2537 Updated the AssetBundler code to looks for the AssetSeedList
files within the Assets/Engine directory
Updated the MissingDependencyScanner GetXMLDependenciesFile functions to
use the Assets/Engine directory as well
Also fixed the MissingDependencyScanner to properly located dependency
xml files within gem directories
* Adding back input argument validation for the AssetBundler command options.
Also added an application_options settings registry file that contains the list of valid command options for the ComponentApplication
* Adding missing end of file newline for applications_options.setreg
* Fixed the AssetBundler help output for the bundleSeed command
* LYN-2537 Moved the Engine and Editor folder to be within the EngineAssets folder
* Fixed Documentation in bootstrap.cfg to correct the path to the user project specific registry file
* Adding a newline to the output of AssetCatalog 'Registering asset..., but type is not set' message
* Updating the AssetProcessorPlatformConfig.setreg Scan Folder to detect
the @ENGINEROOT@/EngineAssets/Engine path for engine runtime assets and
@ENGINEROOT@/EngineAssets/Editor path for engine tool assets
* Updating references to Icons and other assets to account for moving the
Engine and Editor folder under a single EngineAssets folder
* Moving the Engine Settings Registry folder from Engine/Registry -> Registry
* Removed the LY_PROJECT_CMAKE_PATH define as it is not portable to other locations. It is hard coded to the project location that was used for the CMake configuration. Furthermore it paths with backslashes within it are treated as escape characters and not a path separator
* Updated the LyTestTools asset_processor.py script to copy the exclude.filetag from the EngineAssets/Engine directory now
* Fixed Atom Shader Preprocessing when running using an External Project
* Updated the TSGenerateAction.cpp to fix the build error with using a renamed variable
* Updated the Install_Common.cmake ly_setup_others function to install the
EngineAssets directory and the each of the Gem's Assets directory while
maintaining the relative directory structure to the Engine Root
Also updated the install step to install the Registry folder at the
engine root
* Fixed the copying of the Registry folder to be in the install root, instead of under a second 'Registry' folder
* Moving the AssetProcessorPlatformConfig.setreg file over to the Registry folder
* Updated the LyTestTools and C++ code to point that the new location of
the AssetProcessorPlatformConfig.setreg file inside of the Registry
folder
* Renamed Test AssetProcessor*Config.ini files to have the .setreg extension
* Converted the AssetProcessor test setreg files from ini format to json
format using the SerializeContextTools convert-ini command
* Updated the AssetProcessor CMakeLists.txt to copy over the test setreg files to the build folder
* Updated the assetprocessor test file list to point at the renamed AsssetProcessor*Config setreg filenames
* Removed the Output Prefix code from the AssetProcessor. The complexity that it brought to the AP code is not needed, as users can replicate the behavior by just moving there assets underneath a another folder, underneath the scan folder
* Adding back support to read the AssetProcessorPlatformConfig.setreg file from the asset root. This is only needed for C++ UnitTests as they run in an environment where the accessing the Engine Settings Registry is not available
* Updating the Install_common.cmake logic to copy any "Assets" folder to
the install layout.
The Script has also been updated to copy over the "Assets" folder in the
Engine Root to the install layout instead of an "EngineAssets" folder
* Updating References to EngineAssets source asset folder in code to be the Assets source folder
* Moved the Engine Source Asset folder of 'EngineAssets' to a new folder name of 'Assets'. This is inline with the naming scheme we use for Gem asset folders
* Adding the EngineFinder.cmake to the AutomatedTesting project to allow it to work in a project centric manner
* Updating the LyTestTools copy_assets_to_project function to be able to copy assets with folders to the temporary project root
Fixed an issue in LyTestTools where the temporary log directory could have shutil.rmtree being called twice on it leading to an exception which fails an automated test
Updated the asset_procesor_gui_tests_2 AddScanFolder test to not use the
output prefix, but instead place the source asset root into a
subdirectory
* Correct the AssetProcessorPlatformConfig Scan Folders for the EngineAssets directory to point at the Assets directory
* Updated the asset procesor batch dependency test scan folder to point at the 'Assets' folder instead of 'EngineAssets'
Esteban Papp
Chris Burel
Chris Santora
Ken Pruiksma
AMZN-AlexOteiza
Steve Pham
Chris Galvan
sharmajs-amzn
Gene Walters
AMZN-koppersr
mnaumov
galibzon
amzn-sj
lumberyard-employee-dm
Mike Balfour
AMZN-stankowi
Terry Michaels
amzn-mike
mbalfour
Nicholas Lawson