* The Build Time Stamp of ShaderAsset And
ShaderVariantAsset Should Be Based On
GetTimeUTCMilliSecond()
GetTimeNowMicroseconds() is useful to measure time stamp differences.
GetTimeUTCMilliSecond() is for time stamps based on absolute clock/wall time.
* Updated DiffuseGlobalIllumination precompiled shaders
Co-authored-by: dmcdiar <dmcdiar@amazon.com>
Signed-off-by: galibzon <66021303+galibzon@users.noreply.github.com>
Changed material document to load a source data are for the parent material as well. It was also a previously loading the parent material products asset which would be out of date compared to the source data.
Changed material document to track source file dependency changes instead of product asset changes.
Fixed a bug or copy paste error in the document manager that was using the same container to track documents the modified externally and from other dependency changes.
Returning source data dependencies when creating a material asset from source.
Signed-off-by: Guthrie Adams <guthadam@amazon.com>
This RenderStates is used to override the values in the final draw packet, if the values are valid; it's supposed to be initialized to invalid values, but it wasn't. So the depth compare function was getting set to Less instead of GreaterEqual. This wasn't a problem when using serialized assets from disk, because the deserialization uses the default constructor which did initialize m_renderStatesOverlay. No all Item constructors initialize m_renderStatesOverlay.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Signed-off-by: Guthrie Adams <guthadam@amazon.com>
(It's a simple enough change to make manually, and making .materialtype is an uncommon workflow, so not worth doing this automatically).
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Also improved the MaterialAssetTests UpgradeMaterialAsset() to focus on testing the inputs and outputs of the class rather than the private internal data.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Also rearranged some logic to simplify the code that loops over rename actions, avoiding making unnecessary additional maps.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
- Added MaterialSourceData::ApplyVersionUpdates() for updating the properties. This should be called by tools after loading the MaterialSourceData. (But can be omitted if a tool wants to read the data exactly as it appears in the .material file).
- Updated MaterialTypeSourceData::FindProperty to support applying version update renames, including a ApplyPropertyRenames utility function, which are necessary for MaterialSourceData to be able to find the necessary property definitons while loading.
- Added a new context struct to JsonMaterialPropertyValueSerializer for passing down the material type version number, to help with applying property renames.
- Renamed the .material file format "propertyLayoutVersion" to "materialTypeVersion" which is more accurate. This shouldn't hurt existing data as this field wasn't actually used for anything before.
- Updated Material Editor to again store the material type version number in .material files.
MaterialSourceDataTests updates...
- Updated to include both a .materialtype file and a MaterialTypeAsset for the test material type. Both are used by the MaterialTypeSourceData class.
- The default test material type now includes some version update steps; these are only used for version update tests and won't impact the other test functions.
- Updated the path for storing temp files to disk, to just be in a "temp" folder in the exe path. (Originally they were saved to the gem folder near MaterialSourceDataTests.cpp, but at some point someone changed it to be under the exe folder, so there's no reason to use the full gem path anymore).
MaterialTypeSourceDataTests updates...
- Moved some code that was accidentally added to LoadAllFieldsUsingOldFormat but should have been in LoadAndStoreJson_AllFields.
- Added test cases for unsupported version update operations
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
This required the use of custom serializers, because the JSON serialization system does not have any means of supporting field name aliases through SerializeContext.
Testing: RPI unit test pass and AtomSampleViewer material screenshot test script passes.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
This revealed that the approach of reflecting both the old "id" and the new "name" would not work, because whenn saving it would write out both fields. So I decied to just give up on backward compatibility. This will be much cleaner than trying to continue supporting "id" as a field name, it is uncommon for users to make their own material types at this point, and if they have made some it is very easy to search and replace "id" with "name" update their files.
All .materialtype files have been updated. RPI unit tests now pass. ASV still passes.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
A "property name" is the name of the just the property without regard to the group that it's in. A "group name" is the name of the group. And a "property ID" is the full unique name of a property in the form "groupName.propertyName". This is important preparation for upcoming changes where property sets can contain other property sets, and property IDs can be arbitrarily long like "layer1.baseColor.factor" for example.
The naming changes include variables, some code comments, and the .materialtype file format. I was able to make these changes in a backward compatible way so a property or group "id" field has been replaced with a "name" field, but "id" is still supported for compatibility. StandardPBR, EnhancedPBR, StandardMultilayerPBR, and Skin have all been updated. Note that MinimalPBR has not been updated, proving that backward compatibility works. (We can update this one too at some point though).
Testing:
Opened up materials in the material editor.
Ran AtomSampleViewer in dx12 and vulkan with no new failures.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Add material property names to material assets, disable FBX dependency on materialtype files.
Signed-off-by: Robin <rbarrand@amazon.com>
* Add reflection for MaterialAssets. Update member variable comment.
Signed-off-by: Robin <rbarrand@amazon.com>
* Switch cvar to using bus value. Refine comments.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor functions and refine comments.
Signed-off-by: Robin <rbarrand@amazon.com>
* Realign property values when material property names are populated.
Signed-off-by: Robin <rbarrand@amazon.com>
* Switch PostLoadInit check to on asset status ready. Add realign property values code to PostLoadInit as well.
Signed-off-by: Robin <rbarrand@amazon.com>
* Stash@{1} code.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor realignment code into the right places.
Signed-off-by: Robin <rbarrand@amazon.com>
* Remove pragma optmize off.
Signed-off-by: Robin <rbarrand@amazon.com>
* More refactoring.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor comments and remove code no longer needed.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor comments and remove unused include.
Signed-off-by: Robin <rbarrand@amazon.com>
* Comment refactor, corrected some code.
Signed-off-by: Robin <rbarrand@amazon.com>
Co-authored-by: Robin <rbarrand@amazon.com>
* Final update copyright headers to reference license files at the repo root
Signed-off-by: spham <spham@amazon.com>
* Fix copyright validator unit tests to support the stale O3DE header scenario
Signed-off-by: spham <spham@amazon.com>
ATOM-15837 Remove Unused Shader System Related *2 Classes From Stabilization Branch.
Testing: ASV full test suite on dx12 and vulkan. Only saw known issues.
Cherrypicked b7f6b57e16 from stabilization branch because I knew it would have conflicts I needed to resolve.
See https://github.com/aws-lumberyard/o3de/pull/1462
This removes mostly benign (but noisy) messages about "Failed to find builder dependency".
ATOM-15136 Builder dependency errors reported in mainline
ATOM-15134 Replace GetPossibleDepenencyPaths Approach with Source Dependencies
SrgLayoutBuilder.cpp conflicted because it had been removed on development. The changes in this file are no longer relevant.
ShaderVariantAssetBuilder.cpp conflicted only because there were formatting changes inside the AddShaderAssetJobDependency() function.
This removes mostly benign (but noisy) messages about "Failed to find builder dependency".
Changed AssetUtils::GetPossibleDepenencyPaths to return all possible source paths, rather than stopping when one is found. This function is now used to report a list of all possible source dependencies, so that CreateJobs will get called by the AP whenever a file shows up at one of those locations. If a file was missing before and then appears, this will cause the builders to wake up and add the appropriate job dependencies on the new files. In ShaderVariantAssetBuilder and MaterialBuilder, we now use GetPossibleDepenencyPaths to report source dependencies rather than job dependencies. We only report a job dependency when the actual source file has been identified. This should all now be consistent with the intended design of the AP's dependency systems (the prior approach was a hack based on misunderstanding of what source dependencies are).
SrgLayoutBuilder's change is a bit tricky. The above changes did not fix all of the "Failed to find builder dependency" messages because AzslBuilder sometimes skips particular files in CreateJobs. When this happens, it is invalid to report an AzslBuilder job dependency on that file. So I copied the same conditional code that is used to skip an azsli file in CreateJobs, and used that to skip AddAzslBuilderJobDependency() in SrgLayoutBuilder as well.
With all these changes combined, I do think we've solved the issue where jobs fail to evict outdated jobs, as described in ATOM-15134. However, we are not yet seeing the iteration time improvements we were hoping for. Before these changes I was seeing roughly a 0.5 minute delay for the initial change, and a 2 minute delay for a subsequent change. With these changes it's more like 0.5 mibutes and 1.5 minutes. It appears that the AP scan is being starved by all the AssetBuilder processing going on, and perhaps IO contention. I suspect that this will be greatly improved on the development branch where we no longer have AzslBuilder and SrgLayoutBuilder slowing things down.
ATOM-15136 Builder dependency errors reported in mainline
ATOM-15134 Replace GetPossibleDepenencyPaths Approach with Source Dependencies
* [ATOM-15472] Shader Build Pipeline: Remove Deprecated Files And Functions That
Predate The Shader Supervariants
These are the essential impactful changes as a result of deprecating the
ShaderResourceGroupAsset.
* Addressed feedback by @moudgils. Better comments in header files.
* More updates related with deprecation of ShaderResourceGroupAsset
* Deleted the temporary version 2 classes.
* Updated version of the shader asset builders.
* Updated version of all the shader related classes impacted
by the Supervariant concept and deprecation of ShaderResourceGroupAsset
* Changes to *.pass and DGI, Reflections and RayTracing.
* changes to material related assets
* changes to core lights
* Changes to auxgeom/dynamic draw.
* changes to decals, lyshine, imguipass
* changes to RPI Pass classes
* Shader for SceneSrg, ViewSrg and ForwardPass Srgs.
* changes to mesh, skinned mesh, Morphtarget.
* Fixes to RayTracingPass.cpp & now allow empty srg in shaders.
* Updated Atom_RPI.Tests
* Simplified InstanceDatabase by removing AddHandler
------------------------------------------------------------------------------------
* Updated DiffuseGI precompiled shaders.
Added RayTracingSceneSrg and RayTracingMaterialSrg shader asset.
Updated ShaderAssetCreator::Clone to handle the supervariant when processing root variants.
Co-authored-by: Doug McDiarmid <dmcdiar@amazon.com>
------------------------------------------------------------------------------------
* Changed semantics for some PassSrg to SRG_PerPass_WithFallback.
AuxGeom/FixedShapeProcessor.cpp requires SRG_PerDraw on ObjectSrg.
Removed names of SceneSrg and ViewSrg from RPISystemDescriptor.cpp
* Moved ShaderLib/Atom/Features/DummyEntryFunctions.azsli
To Gems/Atom/RPI/Assets/ShaderLib/Atom/RPI/DummyEntryFunctions.azsli
Removed redundant checking for finalization in
ShaderResourceGroupLayout.cpp
* Fixed race condition bug for Shader::FindOrCreate.
InstanceDatabase<>::CreateInstance() needs to be atomic
for instance creation and initialization.
Added optional InstanceHandler::CreateFunctionWithParams to accomodate
to the needs of Instances that need more than an asset reference
to be able to be created an initialzed.
Removed ShaderResourceGroup::FindOrCreate() only ::Create is available
now.
* Renamed scene_and_view_srgs.* as SceneAndViewSrgs.*
Changed GetAzslFileOfOrigin for GetUniqueId
* Fixed unit tests.
* Reverted the serialization name of m_uniqueId back to
"m_azslFileOfOrigin" so precompiled shaders don't fail
in layout comparison.
* Fixed AtomCore.Tests
Removed non-applicable test. InstanceDatabase.AddHandler() is not
available anymore.
* The Null rhi is re-enabled for shader compilation.
Signed-off-by: garrieta <garrieta@amazon.com>
* [ATOM-15618] Shader Build Pipeline: Add UnitTest To Validate Shader Compiler
Argument Processing Introduced With The New Supervariant System
- Added new test suite in
Gems/Atom/Asset/Shader/Code/Tests/SupervariantCmdArgumentTests.cpp
- Refactored and improved the previously existing classes:
GlobalBuildOptions, PreprocessorOptions and ShaderCompilerArguments
to work well with the new ShaderSourceData::SupervariantInfo.
- Moved command line argument processing function out of
ShaderCompilerArguments and into its own utility namespace in
Atom/RHI.Edit/Utils.h
Signed-off-by: garrieta <garrieta@amazon.com>
* [ATOM-15276] Shader Build Pipeline: Add Shader Supervariant System.
Added ShaderAssetBuilder2 & ShaderVariantAssetBuilder2.
Added ShaderAsset2, ShaderVariantAsset2.
Eventually they will be the only builders. AzslBuilder &
SrgLayoutBuilder will be removed.
ShaderResourceGroupAsset will be removed. ShaderAssetBuilder &
ShaderVariantAssetBuilder will be replaced.
Signed-off-by: garrieta <garrieta@amazon.com>
Added a loop to the skin shader that will sample from wrinkle masks, multiply them by a weight, combine them, and use them instead of vertex colors for wrinkle map blending
Added an array of masks, an array of weights, and a wrinkle mask count to the DefaultObjectSrg. -Will create a follow up task to handle this a better way.
Removed motion vector (for now) from skin.materialtype since we're not using them, and removed depthtransparent since skin doesn't support transparency
Added an interface to the MeshFeatureProcessor to get the object srg
Wrapped srg->Compile in if(srg->IsQueuedForCompile()) to prevent compiling twice --This doesn't stop a race condition if both happen at the same time, but that is at least far less likely. It will need a better solution later.
Added a function to the MorphTargetExporter that will check to see if a texture that matches the blend shape name exists in a particular folder, and adds a reference to that image to the MorphTargetMetaAsset --Only supports .tif, and doesn't automatically re-process the .fbx if the folder is updated. These can be improved in later iterations
Added a null check in MaterialTypeSourceData.cpp to fix a crash I ran into
Added a for loop in two places to look for the first submesh that has a morph target, instead of just using the first to check if a lod has morph targets or not. --I have a better fix for this, but it involves more areas of the code, so I'm saving that for another change.
Modified AtomActorInstance to look for any morph targets that have a wrinkle mask reference
Then each frame, for any morph targets with non-zero weights that also have wrinkle masks, it updates the mask array, weights, and count on the object srg.
Guthrie Adams
galibzon
Guthrie Adams
santorac
hershey5045
Robin
Artur K
srikappa-amzn
srikappa-amzn
chcurran
Steve Pham
Chris Galvan
Gene Walters
Chris Santora
Tommy Walton
jonbeer
Chris Burel
alexpete