There were conflicts in MaterialDocument.cpp which I had resolve by just taking development. This will not compile. I will make additional fixes in another commit.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
This is needed to support deeply nested material property groups, it just makes the serialization code a lot simpler than trying to support nested groups in the .material file. It also makes the file more readable and easier to search all files for particular properties.
I also updated MaterialSourceData to hide the property values behind a clean API.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Changes to AttachmentImage related classes to have better support using render target as texture.
- Updated AttachmentImageAsset to support an unique name which can be used as AttachmentId.
- Functions in ImageSystemInterface to find an AttachmentImage by its unique name.
- Added InsertChild function to ParentPass class so it can insert a child pass at any location.
- Change to ImageAttachmentPreviewPass so it can preview cached pass attachment even the source pass was disabled.
Signed-off-by: Qing Tao <55564570+VickyAtAZ@users.noreply.github.com>
Fixed the MaterialBuilder to report source dependencies on imported JSON files, so the material type will rebuild when you edit an imported file that contains shared property data.
Fixed a spot in LuaMaterialFunctorSourceData where it was only applying the old "prefix" feature and not the new name context.
Updated a couple built-in material functors to take advantage of the name context (they were using RuntimeContext::GetShaderResourceGroupLayout instead of the RuntimeContext::FindShaderInputConstantIndex wrapper utility function).
Fixed an issue with EnumeratePropertyGroups where it wasn't passing the right name context, so Material Editor wasn't able to load some material types.
Reordered the parameters in the MaterialTypeSourceData enumerate callback functions, I felt this order was more natural.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
The main addition here is the MaterialNameContext class which represents the concept of a namespace for properties, shader options, and SRG fields. This concept was already somewhat supported in LuaMaterialFunctor through bespoke "prefix" fields, but I have generalized it be available for all material functors. Note that I have not yet updated the other material functor types to ensure they take advantage of this feature, that will be in another commit.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
I also noticed that JsonFileLoadContext was no longer used (see https://github.com/o3de/o3de/pull/7010) so I was able to remove all that code.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
In some cases it may be acceptable to do non-const things in a const function as long as it is only manipulating internal data, and the public facing API returns the same values as before. But in this case, the IsFinalized function is a public facing API that would have a different result after GetPropertyValues was called.
I also updated a couple other minor things from code review feedback.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
I made MaterialAsset::Finalize private so I could add some parameters specifically for MaterialAssetCreator to use. Now MaterialAssetCreator::Begin has an option to finalize the material or not.
Moved MaterialAssetCreatorCommon::ValidateDataType to MaterialPropertyDescriptor as "ValidateMaterialPropertyDataType" so that MaterialAsset::Finalize could use it too
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Before, the material builder was loading the MaterialTypeAsset and doing some processing with it, but was avoiding declaring job dependencies that would cause reprocessing lots of assets when a shader or .materialtype file changes. Reading the asset data isn't safe when not declaring a job dependency (or when declaring a weak job dependency like OrderOnce which is the case here). This caused to several known bugs.
The main change here is it no longer loads the MaterialTypeAsset at all; all other changes flow from there.
The biggest changes (when deferred material processing is enabled) are ...
1) MaterialSourceData no longer loads MaterialTypeAsset. All it really needs is to determine whether a string is an image file reference or an enum value, which is easy to do by just looking for the "." for the extension.
2) MaterialAssetCreator no longer produces a finalized material asset. It no longer uses MaterialAssetCreatorCommon because that only produces a non-finalized MaterialAsset, which has very different needs for the SetPropertyValue function. (We could consider merging MaterialAssetCreatorCommon into MaterialTypeAssetCreator since that's the only subclass at this point). And it doesn't do any validation against the properties layout since that can be done at runtime.
3) Moved processing of enum property values from MaterialSourceData to MaterialAsset::Finalize (this was the only thing being done in the builder that actually needed to read the material type asset data).
Also...
- Updated the MaterialAsset class mostly to clarify and formalize the two different modes it can be in: whether it is finalized or not.
- Merged the separate "IncludeMaterialPropertyNames" registry settings from MaterialConverterSystemComponent and MaterialBuilder into one "FinalizeMaterialAssets" setting used for both.
- Removed MaterialSourceData::ApplyVersionUpdates. Now the flow of data is the same regardless of whether the materials are finalized by the AP or at runtime. Version updates are always applied on the MaterialAsset.
- Added a validation check to MaterialTypeAssetCreator ensuring that once a property is renamed, the old name can never be used again for a new property. This assumption was already made previously, but not formalized, in that Material::FindPropertyIndex does not expect every caller to provide a version number for the material property name, also the material asset's list of raw property names was never versioned. The only way for this to be a safe assumption is to prevent reuse of old names.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Average incoming skin influences when multiple vertices have been welded. This is one option which will average out the weights even if two welded vertices have differing boneIds, but we probably also need to add the influences earlier in the process and enforce a process where the vertices do not get welded if they have influences with differing boneIds
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Move skin influences from MeshBuilderSkinningInfo to the vertex attribute layers so they are considered when choosing which vertices can be welded and so they are not duplicated when compatible vertices have been welded
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Updating unit tests
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Remove unused functions
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Update based on feedback from burelc
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Multiple cleanups ( tidy etc. )
Coalesce nested namespaces.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Multiple cleanups ( tidy etc. ) cntd.
Converted Uuid into POD ( defaulted the constructor )
Add `&/const &` to `for` loops that benefit from their use
Some Qt optimizations ( string ref, prevent container detaches, etc. )
Replace `::bind` in a few places.
Replaced the use of AZ_CRC with AZ_CRC_CE in a few places.
Replace a few `typedef`s with `using`s
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Linux compilation fix.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Apply review suggestions.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Fix vs2019 build
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Small clang re-format in StringFunc.cpp
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Apply reviewer's suggestions.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
The main reason for this is to give consistent results between the AP and Material Editor, where a placholder texture can be used if a texture is missing. Otherwise, you could get a placeholder texture in Material Editor and stale data in the runtime; this inconsistency would be confusing.
As a consequence, it is possible for example that the user could mess up the name of a property in a .material file and not notice the problem because it is now a warning instead of an error. If warnings-as-errors is desirable, you can enable the new "/O3DE/Atom/RPI/MaterialBuilder/WarningsAsErrors" registry setting.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* fix brute force mesh intersection function
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* add test for brute force ray intersection fix
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* refactor tests to remove as much duplication and provide API for future tests if required
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* small updates after review feedback
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* update following review feedback
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* fix for pointer offset
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
Using a normal job dependency was overkill. As a result, modifying a shader would trigger a pass to rebuild, and that would cause the entire render pipeline to reinitialize. This causes a lot of unnecessary churn that made debugging shader hot reloads difficult.
When a builder depends on knowing the UUID of another asset, it does need to have a job dependency on that asset. But when the builder doesn’t actually consume any data from inside the asset, and it just needs the ID, it is best to use JobDependencyType::OrderOnce to simply make sure the AP knows about the asset.
Also, the call to GetSourceInfoBySourcePath was an unnecessary check because the AP does not require the file to exist at the time a dependency is reported. GetSourceInfoBySourcePath will only succeed after the AP has scanned the requested source file. We can’t just assume that a false result indicates the file does not exist, it just isn’t known yet.
There are a couple additional use cases to be aware of (from @antonmic)...
1. Passes are critical assets, and some passes depend on shaders, so those shaders need to be processed, that's one reason for the dependency (otherwise you can start the engine, load all critical passes, try to load the shader, shader isn't ready, and bad things happen)
2. If a shader is deleted/removed, the pass should try to rebuild and fail. This was a recent issue that I fixed, you could have some odd condition where you delete a shader, but the pass that references that shader doesn't update or throw any errors because it doesn't rebuild, so the user is unaware that they need to change the .pass file and bad things happen
OrderOnce is sufficient for both these cases. #1 is related to first time processing which exactly the time when OrderOnce would be applied. I tested #2 as well; deleting a .shader file with OrderOnce dependency did trigger the .pass file to rebuild and fail.
Testing:
Ran an ASV pass test with both an exiting cache and after deleting the local cache.
Started the Editor with a clean cache and didn't encounter any startup issues. Was able to successfully load a level the first time.
Deleted SkyBox.shader and saw SkyBox.pass fail in the AP.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Adding shaders and attimage files as runtime depenencies for pass files, so that they are included in asset bundles. Also using the correct job key for attimage files.
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Use a reference to avoid a copy
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Bumping the AnyAsset builder version
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Revert "Bumping the AnyAsset builder version"
This reverts commit 778798ae9cdd93ebe93248b3113e4cfb7609020d.
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Adding shaders and attimage files as runtime depenencies for pass files, so that they are included in asset bundles. Also using the correct job key for attimage files.
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Use a reference to avoid a copy
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Bumping the AnyAsset builder version
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Revert "Bumping the AnyAsset builder version"
This reverts commit 778798ae9cdd93ebe93248b3113e4cfb7609020d.
Signed-off-by: Tommy Walton <waltont@amazon.com>
(It's a simple enough change to make manually, and making .materialtype is an uncommon workflow, so not worth doing this automatically).
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
- Added MaterialSourceData::ApplyVersionUpdates() for updating the properties. This should be called by tools after loading the MaterialSourceData. (But can be omitted if a tool wants to read the data exactly as it appears in the .material file).
- Updated MaterialTypeSourceData::FindProperty to support applying version update renames, including a ApplyPropertyRenames utility function, which are necessary for MaterialSourceData to be able to find the necessary property definitons while loading.
- Added a new context struct to JsonMaterialPropertyValueSerializer for passing down the material type version number, to help with applying property renames.
- Renamed the .material file format "propertyLayoutVersion" to "materialTypeVersion" which is more accurate. This shouldn't hurt existing data as this field wasn't actually used for anything before.
- Updated Material Editor to again store the material type version number in .material files.
MaterialSourceDataTests updates...
- Updated to include both a .materialtype file and a MaterialTypeAsset for the test material type. Both are used by the MaterialTypeSourceData class.
- The default test material type now includes some version update steps; these are only used for version update tests and won't impact the other test functions.
- Updated the path for storing temp files to disk, to just be in a "temp" folder in the exe path. (Originally they were saved to the gem folder near MaterialSourceDataTests.cpp, but at some point someone changed it to be under the exe folder, so there's no reason to use the full gem path anymore).
MaterialTypeSourceDataTests updates...
- Moved some code that was accidentally added to LoadAllFieldsUsingOldFormat but should have been in LoadAndStoreJson_AllFields.
- Added test cases for unsupported version update operations
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* adding Windows/release to PR-validation builds
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* changing trace back to expand to nothing for release
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* typo
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* more fixes
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* fixing some more unused variable cases
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* renaming file in ScriptCanvas that causes a msbuild warning
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* reverting a previous change
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
This revealed that the approach of reflecting both the old "id" and the new "name" would not work, because whenn saving it would write out both fields. So I decied to just give up on backward compatibility. This will be much cleaner than trying to continue supporting "id" as a field name, it is uncommon for users to make their own material types at this point, and if they have made some it is very easy to search and replace "id" with "name" update their files.
All .materialtype files have been updated. RPI unit tests now pass. ASV still passes.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Add material property names to material assets, disable FBX dependency on materialtype files.
Signed-off-by: Robin <rbarrand@amazon.com>
* Add reflection for MaterialAssets. Update member variable comment.
Signed-off-by: Robin <rbarrand@amazon.com>
* Switch cvar to using bus value. Refine comments.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor functions and refine comments.
Signed-off-by: Robin <rbarrand@amazon.com>
* Realign property values when material property names are populated.
Signed-off-by: Robin <rbarrand@amazon.com>
* Switch PostLoadInit check to on asset status ready. Add realign property values code to PostLoadInit as well.
Signed-off-by: Robin <rbarrand@amazon.com>
* Stash@{1} code.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor realignment code into the right places.
Signed-off-by: Robin <rbarrand@amazon.com>
* Remove pragma optmize off.
Signed-off-by: Robin <rbarrand@amazon.com>
* More refactoring.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor comments and remove code no longer needed.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor comments and remove unused include.
Signed-off-by: Robin <rbarrand@amazon.com>
* Comment refactor, corrected some code.
Signed-off-by: Robin <rbarrand@amazon.com>
Co-authored-by: Robin <rbarrand@amazon.com>
santorac
Qing Tao
Esteban Papp
Tommy Walton
puvvadar
Artur K
Tom Hulton-Harrop
Tommy Walton
Robin
srikappa-amzn
AMZN-koppersr
srikappa-amzn
Steve Pham
hershey5045