* Profiler: implement loading from saved capture
Adds functionality for finding a saved capture on disk and then
deserializing it using rapidjson's built-in buffered stream reader. This
does require use of raw file pointers since saved captures can be
hundreds of megabytes.
Actually showing the data in the visualizer is TODO.
* Profiler: use heap buffer over stack buffer
* Profiler: move deserialization logic to ImGuiCpuProfiler
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: Implement continuous capture internals
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: extend ProfileCaptureSystemComponent
Implements dumping of saved CPU profiling data to a local file, blocking
call.
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: Working IO thread for serialization
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: switch to AZ::JobFunction for IO
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: move to a ring buffer for storage
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: switch back to IO thread
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: add TODO
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Profiler: correct thread safety issues
Signed-off-by: Jacob Hilliard <jhlliar@amazon.com>
* Add CaptureCpuFrameTime method to ProfilingCaptureSystemComponent for monitoring CPU performance.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* ly_test_tools: Refactor benchmark data aggregator in preparation for CPU frame times.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* performance benchmarks: Aggregate and report CPU frame times based on JSON data.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* AutomatedTesting: Capture CPU frame time in AtomFeatureIntegrationBenchmark.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.
Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.
Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows ScenePI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.
Testing:
- Changing the registery setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switcihng the material conversion registry setting from true to false. (Details below)
- TODO: Will merge this change to a customer's fork and test on their existing content.
Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
- Entity 1 material assignments
- Blue_Zaxis: left as-is
- Green_Yaxis: exported the material
- Red_Xaxis: exported the material, and changed the material instance color to pink
- StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
- With_Texture: selected an existing brick material, changed the material instance color to red.
- Entity 2 material assignments
- Default Material: set to an existing brick material
- Blue_Zaxis: manually assigned built-in material that was converted from fbx
- Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Repro steps:
- Create two entities.
- Entity 1
- Add a mesh component and assign a model with multiple sub-meshes
- Add a material component. The material component looks correct.
- Entity 2
- Add a mesh component and assign the same model as the other entity
- Add a material component. The material component shows "<unknown>" for all material slot names
The problem was that ReflectedPropertyEditor creates a new Asset<> reference with the correct ID but does not load it. This asset is passed to EditorMaterialComponent, MaterialComponentController, and MeshFeatureProcessor and none of these tell the Asset to load. The MeshFeatureProcessor was not loading the Asset or connecting to the AssetBus because the instance already existed in the InstanceDatabse so from the FP's perspecive there was no need. But for the FP's GetModelAsset() API to function correctly it needs to have the asset initialized to the available AssetData pointer. So we updated the MeshFeatureProcessor to always connect to the AssetBus so it will find the available AssetData via the OnAssetReady callback.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
I had to update the ModelLod class to take in both the ModelLodAsset and ModelAsset for initialization so it can fetch the slots for each mesh.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Here we inttroduce a new struct ModelMaterialSlot which formalizes the concept of material slot, with an ID, display name, and default material assignment. The ID still comes from the MaterialUid like before. The display name is built-in, rather than being parsed out from the asset file name. And the default material assignment can be any material asset, it doesn't have to come from the FBX (or other scene file).
This commit is just the preliminary set of changes. Cursory testing shows that it works pretty well but more testing is needed (and likely some fixes) before merging.
Here is what's left to do...
Add serialization version converters to preserve prior prefab data.
See if we can get rid of GetLabelByAssetId function only rely on the display name inside ModelMaterialSlot.
I'm not sure if the condition for enabling the "Edit Material Instance..." context menu item is correct.
Test actors
Lots more testing in general
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Expose shadow bias to component & feature processors. Shadow bias now works more consistently with various near / far shadow planes and caster positions. Bias now also affects esm shadows which helps eliminate acne in certain situations.
Signed-off-by: Ken Pruiksma <pruiksma@amazon.com>
* Adding jira comment to light configuration serialization version. Improved comment on final adjustment to bias before its sent to the shader.
Signed-off-by: Ken Pruiksma <pruiksma@amazon.com>
* Hooking up bias to behavior context.
Signed-off-by: Ken Pruiksma <pruiksma@amazon.com>
* fix: Correct typo in profiling capture system.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* Move benchmark data aggregation from ASV into ly_test_tools module.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
Added an initialization delay before starting the DiffuseProbeGrid texture readbacks, to allow the textures to settle.
Signed-off-by: dmcdiar <dmcdiar@amazon.com>
* Switching to Buffer<float> instead of Buffer<float3> in the skinning shader because metal doesn't support float3 buffers
Signed-off-by: amzn-tommy <waltont@amazon.com>
* Adding a comment to LinearSkinningPassSRG pointing out that positions, normals, and bitangents are using float buffers to work on Metal
Signed-off-by: amzn-tommy <waltont@amazon.com>
* ATOM-15939 Add support to capture attachment for ParentPass
- Moved the attachment read back support to Pass class so it supports both ParentPass and RenderPass.
- Added support to output input or output state of an InputOutput attachment.
- Enabled showing ParentPass attachments in PassTree tool.
Signed-off-by: Tao <qingtao@amazon.com>
* Final update copyright headers to reference license files at the repo root
Signed-off-by: spham <spham@amazon.com>
* Fix copyright validator unit tests to support the stale O3DE header scenario
Signed-off-by: spham <spham@amazon.com>
Fix writing to the correct mip in Metal
Set default m_outputScale to 1 in order to fix a 1/0 error
Add support for logging/printing errors pertaining to GPU crashes.
Setting Release queue's collection latency to MaxFrames.
Fix managed mem synchronization related offset bug
-Re-purposed an unused boolean in RPI::Cullable for previous frame's visiblity to instead represent objects that are hidden in the simulation.
-Updated MeshFeatureProcessor::SetVisible to set this value on the cullable.
-Updated the MeshComponent to handle visiblity changes by not rendering the mesh instead of deactivating and/or reactivating the component.
-Updated the AtomActorInstance to handle changes to the visibility from the ActorComponent.
Tested by creating two entities with static mesh components, on entity hidden and the other visible. Plus three entities with actor components, one where the actor is visible, one where the entity is visible but the 'render character' setting on the actor component is disabled, and one where the 'render character' setting is enabled, but the entity is not visible.
For each of these 5 entities, I added them as 5 loose entities, 5 entities that were children to a parent entity, and a slice with all 5 as children to a parent entity, and tested toggling visibility of the parent entities.
For each of these 3 sets of 5 entities, I added them directly to the level, added them all to a layer where the layer was visible, and added them all to a layer where the layer was not visible, and tested toggling the visibility of the layers.
* ATOM-15859 AuxGeom rendering in editor is too expensive
- The OrphanBuffer calls is the main reason that AuxGeom FP render is slow.
- Switched to use DynamicBuffer for buffers used in DynamicPrimitiveProcessor
- Added some profiling marks.
- Removed DynamicPrimitiveProcessor per view which was added because of OrphanBuffer can only be called once per frame.
[ATOM-14344][ATOM-13908] Merging skinned and static mesh motion shaders using an optional vertext stream. This removes the log spam "Mesh does not have all the required input streams. Missing 'POSITIONT0'." and allows Material::GetShaderCollection() to be const only as it was intended. The MeshFeatureProcessor also no longer needs to decide which motion vector shader to use, and therefore no longer has m_skinnedMeshWithMotion in the descriptor to acquire a mesh.
* Move SetUseForwardPassIblSpecular to happen when acquiring a mesh, instead of immediately after so that we don't build the drawpacket twice for every mesh
* Update the MeshFeatureProcessor to use the booleans from the descriptor directly instead of having loose booleans in the MeshDataInstance
* m_excludeFromReflectionCubeMaps is not (and does not need to be) part of the descriptor, since setting is cheap and doesn't duplicate work that is done when acquiring the mesh
Added a ShaderSystem supervariant to provide a system-wide supervariant name.
Changed ShaderAsset to append the system-wide supervariant name when searching for supervariants.
Added the NoMSAA supervariant to several shaders.
* Fixed organization of the AssetProcessor SourceAssetBrowser
Assets within the Engine Root were grouped under a '/' entry.
That has been fixed to use the relative path within the engine root for
those assets
Assets outside of the Engine Root, but on the same drive were using
absolute paths before. Now there are child entries that navigate up the
directory hierarchy to those asset locations
* Added ly_enable_gems call to Atom gems targets that are required
The DefaultLevel.prefab contains several Atom components, that require
the Atom RHI, RPI, Common_Feature, ShaderBuilder and AtomLyIntegration CommonFeatures
gems to be enabled in order to successfully process in the
AssetProcessor.
* Added ly_enable_gems call to make the Camera gem required in Tools,
Builders and Clients.
This is needed as the DefaultLevel.prefab contains an Editor Camera
Component
* Adding the ly_enable_gem call to make the Maestro gem required
CrySystem currently requires Maestro to be enabled in order to
initialize
* Added ly_enable_gems call to the SceneProcessing gem to make it required
The SceneCore and SceneData libraries that are part of the core engine
Code folder requires the SceneProcessing gem to be enabled in order to
invoke the InitializeDynamicModule hooks in DllMain.cpp in order to
initialize those libraries.
* Fixed bad argument in comment for Prefab CMakeLists.txt
* Fixed Assert in Asset Builders due to the Atom RPI Builder
The Atom RPI Builder was enabling the Asset Catalog for the ScriptAsset a second time
The Atom Feature Common EditorSystemCommonComponent.cpp which also loads
in the AssetBuilder is enabling the Asset Catalog for the ScriptAsset
Added BehaviorContext reflection to the OutputDeviceTransformType enum
to fix the BehaviorContext errors about reflecting a method that returns
such an enum
* Added TypeId output to the JsonDeserializer report message about missing
ClassData
Previously the report callback would indicate that the target type was
missing Serialization class data, but didn't indicate the TypeId of the
target type
* Added support to the ly_enable_gems function to be able to support
0 gems being enabled.
Updated the Install step for CMake to propagate any ly_enable_gems
within a CMakeLists.txt for a target into the generated CMakeLists.txt
that is made for each installed IMPORTED target
* Adding newline to the end of the Camera Gem CMakeLists.txt
* Fixing target TYPE parameter for actual Gem Modules to use the GEM_MODULE tag instead of MODULE
* Reverting change to the DESTINATION directory for the installed CMakeLists.txt to use the relative path to the installed directory
* Adding the Atom_Bootstrap gem as a required gem
The Client and GameLaunchers required the Atom_Bootstrap gem in order to create the NativeWindow
Added Atom_Feature_Common client module as a runtime dependency of the AtomLyIntegration CommonsFeature client module
* Fixed register.py --all-projects-path and --all-gems-path arguments to
NOT register projects or gems that are within a template folder
Fixed reading of old pre-1.0 o3de_manifest.json files where the
"engines" key was a json array
* Changed how the relative target source directory is calculated when that source directroy resides outside of the engine root.
The final dirname component is used with a unique SHA256 has to form a <dirname>-<8 char SHA256> folder for installing files into
* Adding newline to the end of Atom_Bootstrap CMakeLists.txt
* Moving ly_enable_gems variants for Tools and Builders inside of PAL_TRAIT_BUILD_HOST_TOOLS block
* Adding a comment to AWSCore.ResourceMappingTool target to indicate that it is not a GEM_MODULE.
Furthermore it cannot be loaded with the Gem system because the library is in a different directory the executable
Bone transforms buffer is not valid when using Null renderer, which caused the test to fail since it reported an error and ultimately crashing as well. For now it's been worked around by checking the pointer is valid and not printing the error when null renderer is used, a task for the Atom team has been created to fix this properly in the future (ATOM-15807).
- Added a pause button in imgui cpu profiler.
- Added a capture button to save cpu profiling data to a data file.
- Added some profile marks in both RPI and RHI.
Added new shader reinitialization signaling
This was done while working on "ATOM-15728 Shader Hot Reload Fails in Debug Build", but it turned out these changes did not actually fix the issue (or any other known hot-reload issue). Still, these improvements are appropriate as they correct logical oversights.
ShaderVariant was not listening to asset reloads. It needs to know when the ShaderVariantAsset reload happens so it can reinitialize it's members as well as propagate reinitialization messages. I added a member for the ShaderAsset as the class needs this to reinitialize itself. So now the class listens for reloads of both the ShaderVariantAsset and the ShaderAsset.
Shader was not listening for ShaderAsset reinitialization events.
Updated the API for ShaderReloadNotificationBus's OnShaderVariantReinitialized to include the ShaderVariant which is the most relevant information (the other information wasn't really being used anyway).
Testing: Ran ASV full test suite in dx12 and vulkan, saw only known issues. Tested hot reload in Material Editor and main Editor.
See also aws-lumberyard/o3de-atom-sampleviewer#118
This was done while working on "ATOM-15728 Shader Hot Reload Fails in Debug Build", but it turned out these changes did not actually fix the issue (or any other known hot-reload issue). Still, these improvements are appropriate as they correct logical oversights.
ShaderVariant was not listening to asset reloads. It needs to know when the ShaderVariantAsset reload happens so it can reinitialize it's members as well as propagate reinitialization messages. I added a member for the ShaderAsset as the class needs this to reinitialize itself. So now the class listens for reloads of both the ShaderVariantAsset and the ShaderAsset.
Shader was not listening for ShaderAsset reinitialization events.
Updated the API for ShaderReloadNotificationBus's OnShaderVariantReinitialized to include the ShaderVariant which is the most relevant information (the other information wasn't really being used anyway).
* Extend MeshFeatureProcessor to allow changing the mesh bbox, which requires re-compute the culling data for that mesh
* Update actor mesh bbox when EMFX actor instance bbox changes.
Also use the actor instance global bbox to compute the local bbox for the skinned render mesh, instead of the using the static bounds based bbox, as not every actor instance is going to be using the static bounds bbox.
* Store per-instance mesh AABB in the right place.
In the MeshInstanceData, which is unique per instance, instead of in the Model, which is shared between all instances.
For greater clarity, also remove Model::m_aabb and the corresponding getter and setter, as it isn't immediately obvious whether this gets the model asset bbox or the mesh instance bbox. Callers should instead be explicit about which bbox they want.
* Bug fix: model asset is not necessarily ready in AcquireMesh
* Remove now-unused forward declaration
* Update MockMeshFeatureProcessor with SetLocalAabb/GetLocalAabb
Checked for a valid quality level in DiffuseGlobalIlluminationFeatureProcessor::SetQualityLevel.
Initialized the quality level to Low in DiffuseGlobalIlluminationComponentConfig.
* [ATOM-15472] Shader Build Pipeline: Remove Deprecated Files And Functions That
Predate The Shader Supervariants
These are the essential impactful changes as a result of deprecating the
ShaderResourceGroupAsset.
* Addressed feedback by @moudgils. Better comments in header files.
* More updates related with deprecation of ShaderResourceGroupAsset
* Deleted the temporary version 2 classes.
* Updated version of the shader asset builders.
* Updated version of all the shader related classes impacted
by the Supervariant concept and deprecation of ShaderResourceGroupAsset
* Changes to *.pass and DGI, Reflections and RayTracing.
* changes to material related assets
* changes to core lights
* Changes to auxgeom/dynamic draw.
* changes to decals, lyshine, imguipass
* changes to RPI Pass classes
* Shader for SceneSrg, ViewSrg and ForwardPass Srgs.
* changes to mesh, skinned mesh, Morphtarget.
* Fixes to RayTracingPass.cpp & now allow empty srg in shaders.
* Updated Atom_RPI.Tests
* Simplified InstanceDatabase by removing AddHandler
------------------------------------------------------------------------------------
* Updated DiffuseGI precompiled shaders.
Added RayTracingSceneSrg and RayTracingMaterialSrg shader asset.
Updated ShaderAssetCreator::Clone to handle the supervariant when processing root variants.
Co-authored-by: Doug McDiarmid <dmcdiar@amazon.com>
------------------------------------------------------------------------------------
* Changed semantics for some PassSrg to SRG_PerPass_WithFallback.
AuxGeom/FixedShapeProcessor.cpp requires SRG_PerDraw on ObjectSrg.
Removed names of SceneSrg and ViewSrg from RPISystemDescriptor.cpp
* Moved ShaderLib/Atom/Features/DummyEntryFunctions.azsli
To Gems/Atom/RPI/Assets/ShaderLib/Atom/RPI/DummyEntryFunctions.azsli
Removed redundant checking for finalization in
ShaderResourceGroupLayout.cpp
* Fixed race condition bug for Shader::FindOrCreate.
InstanceDatabase<>::CreateInstance() needs to be atomic
for instance creation and initialization.
Added optional InstanceHandler::CreateFunctionWithParams to accomodate
to the needs of Instances that need more than an asset reference
to be able to be created an initialzed.
Removed ShaderResourceGroup::FindOrCreate() only ::Create is available
now.
* Renamed scene_and_view_srgs.* as SceneAndViewSrgs.*
Changed GetAzslFileOfOrigin for GetUniqueId
* Fixed unit tests.
* Reverted the serialization name of m_uniqueId back to
"m_azslFileOfOrigin" so precompiled shaders don't fail
in layout comparison.
* Fixed AtomCore.Tests
Removed non-applicable test. InstanceDatabase.AddHandler() is not
available anymore.
* The Null rhi is re-enabled for shader compilation.
Signed-off-by: garrieta <garrieta@amazon.com>
First version of temporal antialiasing and contrast adaptive sharpening for GA. Works well in most cases but still has a few issues that will need additional time. This is only the passes and shaders with no exposure to the editor. TAA and CAS can be turned on by enabling their respective passes in the pipeline.
All of the code has been previously reviewed in smaller PRs into the taa_staging branch:
aws-lumberyard-dev#29
aws-lumberyard-dev#53
aws-lumberyard-dev#73
aws-lumberyard-dev#79
aws-lumberyard-dev#84
Main issues:
- Bloom doesn't play nice with TAA and seems to greatly amplify any flickering
- AuxGeom jitters with the camera, so TAA doesn't currently work well in editor
- Transparencies don't have correct motion vectors. History rectification keeps this from looking too bad, but could still be improved
- There is still more that could be done to inhibit flickering, usually from specular aliasing
- Motion vectors aren't correct on POM unless PDO is turned on, which can result in some blurring during motion.
- SSAO can contribute to flickering in its default half res configuration. Changing this to full res mitigates the problem.
Squashed merge of the following:
* [ATOM-13987] Initial checkin of Taa pass.
* TAA pass setup WIP. (does not work yet due to pass configuration issues).
* Taa WIP - Camera motion vectors fixed and hooked up. TAA does simple reprojection and rejection based on depth.
* Small update to use lerp and add some comments.
* Fix issue with attachments not being set up on bindings at initialization. Fixing issue with half-pixel offsets in TAA shader
* - Motion vector passes now use the same output with mesh motion vectors overwriting camera motion vectors.
- Taa pass now works with multiple pipelines.
- Cleaned up TAA shader a bit.
* Fixes from PR review.
* Adding check for multiple attachments of the same name with different resources in Pass::ImportAttachments().
* Adding camera jitter with configurable position count. Updated TAA to blend in tonemapped space.
* Fixes from PR review. Fixing camera motion vectors for background (infinite distance)
* Updates to taa shader from PR review
* Adding a rcp input color size.
* Fix comment on PassAttachment::Update()
* Updates for PR review.
* Fixing missing const on the FrameAttachment* in Pass's call to FindAttachment()
* Taa WIP - Adding filtering to both the current pixel and history. Adding rectification based on variance clipping. Adding some basic anti-flickering. Removing rejection based on depth.
* Updates from PR code review. Mostly better commenting and naming.
* Adding contrast adaptive sharpening based on AMD FidelityFX CAS to help with the softness added by TAA.
* Changing to using luminance for sharpening instead of just green. Added some comments.
* Moving Taa's NaN check to a better location. Disabling TAA and sharpening in prep for check in.
* Updates from PR feedback.
* ATOM-15658 Better option of CreateCommonBuffer requires unique buffer name
- Change the CreateCommonBuffer function to not require an unique name by default.
- Remove the code for generating unique buffer names.
- Add buffer name to BufferAsset so it can be used for device object name instead of using asset file name.
- Change RPI::Buffer to use BufferName_AssetUuid as attachment id.
* provide the ability for component adapters to support multiple components per entity
* add missing explicit keywords
* updates following review feedback - update how template logic works
* small updats (fix typo, remove redundant includes)
* add missing this->
* naming change, common -> controller
* add [[maybe_unused]]
chcurran
Esteban Papp
Jacob Hilliard
Kyle B
pappeste
michabr
antonmic
Cynthia Lin
Yuriy Toporovskyy
Chris Santora
santorac
Jeremy Ong
Guthrie Adams
Ken Pruiksma
moudgils
dmcdiar
Steve Pham
Chris Galvan
dmcdiarmid-ly
Tommy Walton
Qing Tao
amzn-sean
Cynthia Lin
mrieggeramzn
Riegger
hershey5045
Gene Walters
amzn-mike
mbalfour
cgalvan
nvsickle
lumberyard-employee-dm
Aaron Ruiz Mora
AMZN-mnaumov
mnaumov
Roman
galibzon
antonmic
moudgils
greerdv
greerdv
Tom Hulton-Harrop
jiaweig
jiaweig
AMZN-stankowi