In some cases it may be acceptable to do non-const things in a const function as long as it is only manipulating internal data, and the public facing API returns the same values as before. But in this case, the IsFinalized function is a public facing API that would have a different result after GetPropertyValues was called.
I also updated a couple other minor things from code review feedback.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
I made MaterialAsset::Finalize private so I could add some parameters specifically for MaterialAssetCreator to use. Now MaterialAssetCreator::Begin has an option to finalize the material or not.
Moved MaterialAssetCreatorCommon::ValidateDataType to MaterialPropertyDescriptor as "ValidateMaterialPropertyDataType" so that MaterialAsset::Finalize could use it too
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
Before, the material builder was loading the MaterialTypeAsset and doing some processing with it, but was avoiding declaring job dependencies that would cause reprocessing lots of assets when a shader or .materialtype file changes. Reading the asset data isn't safe when not declaring a job dependency (or when declaring a weak job dependency like OrderOnce which is the case here). This caused to several known bugs.
The main change here is it no longer loads the MaterialTypeAsset at all; all other changes flow from there.
The biggest changes (when deferred material processing is enabled) are ...
1) MaterialSourceData no longer loads MaterialTypeAsset. All it really needs is to determine whether a string is an image file reference or an enum value, which is easy to do by just looking for the "." for the extension.
2) MaterialAssetCreator no longer produces a finalized material asset. It no longer uses MaterialAssetCreatorCommon because that only produces a non-finalized MaterialAsset, which has very different needs for the SetPropertyValue function. (We could consider merging MaterialAssetCreatorCommon into MaterialTypeAssetCreator since that's the only subclass at this point). And it doesn't do any validation against the properties layout since that can be done at runtime.
3) Moved processing of enum property values from MaterialSourceData to MaterialAsset::Finalize (this was the only thing being done in the builder that actually needed to read the material type asset data).
Also...
- Updated the MaterialAsset class mostly to clarify and formalize the two different modes it can be in: whether it is finalized or not.
- Merged the separate "IncludeMaterialPropertyNames" registry settings from MaterialConverterSystemComponent and MaterialBuilder into one "FinalizeMaterialAssets" setting used for both.
- Removed MaterialSourceData::ApplyVersionUpdates. Now the flow of data is the same regardless of whether the materials are finalized by the AP or at runtime. Version updates are always applied on the MaterialAsset.
- Added a validation check to MaterialTypeAssetCreator ensuring that once a property is renamed, the old name can never be used again for a new property. This assumption was already made previously, but not formalized, in that Material::FindPropertyIndex does not expect every caller to provide a version number for the material property name, also the material asset's list of raw property names was never versioned. The only way for this to be a safe assumption is to prevent reuse of old names.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Average incoming skin influences when multiple vertices have been welded. This is one option which will average out the weights even if two welded vertices have differing boneIds, but we probably also need to add the influences earlier in the process and enforce a process where the vertices do not get welded if they have influences with differing boneIds
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Move skin influences from MeshBuilderSkinningInfo to the vertex attribute layers so they are considered when choosing which vertices can be welded and so they are not duplicated when compatible vertices have been welded
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Updating unit tests
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Remove unused functions
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Update based on feedback from burelc
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Multiple cleanups ( tidy etc. )
Coalesce nested namespaces.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Multiple cleanups ( tidy etc. ) cntd.
Converted Uuid into POD ( defaulted the constructor )
Add `&/const &` to `for` loops that benefit from their use
Some Qt optimizations ( string ref, prevent container detaches, etc. )
Replace `::bind` in a few places.
Replaced the use of AZ_CRC with AZ_CRC_CE in a few places.
Replace a few `typedef`s with `using`s
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Linux compilation fix.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Apply review suggestions.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Fix vs2019 build
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Small clang re-format in StringFunc.cpp
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Apply reviewer's suggestions.
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
The main reason for this is to give consistent results between the AP and Material Editor, where a placholder texture can be used if a texture is missing. Otherwise, you could get a placeholder texture in Material Editor and stale data in the runtime; this inconsistency would be confusing.
As a consequence, it is possible for example that the user could mess up the name of a property in a .material file and not notice the problem because it is now a warning instead of an error. If warnings-as-errors is desirable, you can enable the new "/O3DE/Atom/RPI/MaterialBuilder/WarningsAsErrors" registry setting.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* fix brute force mesh intersection function
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* add test for brute force ray intersection fix
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* refactor tests to remove as much duplication and provide API for future tests if required
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* small updates after review feedback
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* update following review feedback
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
* fix for pointer offset
Signed-off-by: Tom Hulton-Harrop <82228511+hultonha@users.noreply.github.com>
Using a normal job dependency was overkill. As a result, modifying a shader would trigger a pass to rebuild, and that would cause the entire render pipeline to reinitialize. This causes a lot of unnecessary churn that made debugging shader hot reloads difficult.
When a builder depends on knowing the UUID of another asset, it does need to have a job dependency on that asset. But when the builder doesn’t actually consume any data from inside the asset, and it just needs the ID, it is best to use JobDependencyType::OrderOnce to simply make sure the AP knows about the asset.
Also, the call to GetSourceInfoBySourcePath was an unnecessary check because the AP does not require the file to exist at the time a dependency is reported. GetSourceInfoBySourcePath will only succeed after the AP has scanned the requested source file. We can’t just assume that a false result indicates the file does not exist, it just isn’t known yet.
There are a couple additional use cases to be aware of (from @antonmic)...
1. Passes are critical assets, and some passes depend on shaders, so those shaders need to be processed, that's one reason for the dependency (otherwise you can start the engine, load all critical passes, try to load the shader, shader isn't ready, and bad things happen)
2. If a shader is deleted/removed, the pass should try to rebuild and fail. This was a recent issue that I fixed, you could have some odd condition where you delete a shader, but the pass that references that shader doesn't update or throw any errors because it doesn't rebuild, so the user is unaware that they need to change the .pass file and bad things happen
OrderOnce is sufficient for both these cases. #1 is related to first time processing which exactly the time when OrderOnce would be applied. I tested #2 as well; deleting a .shader file with OrderOnce dependency did trigger the .pass file to rebuild and fail.
Testing:
Ran an ASV pass test with both an exiting cache and after deleting the local cache.
Started the Editor with a clean cache and didn't encounter any startup issues. Was able to successfully load a level the first time.
Deleted SkyBox.shader and saw SkyBox.pass fail in the AP.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Adding shaders and attimage files as runtime depenencies for pass files, so that they are included in asset bundles. Also using the correct job key for attimage files.
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Use a reference to avoid a copy
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Bumping the AnyAsset builder version
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Revert "Bumping the AnyAsset builder version"
This reverts commit 778798ae9cdd93ebe93248b3113e4cfb7609020d.
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Adding shaders and attimage files as runtime depenencies for pass files, so that they are included in asset bundles. Also using the correct job key for attimage files.
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Use a reference to avoid a copy
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Bumping the AnyAsset builder version
Signed-off-by: Tommy Walton <waltont@amazon.com>
* Revert "Bumping the AnyAsset builder version"
This reverts commit 778798ae9cdd93ebe93248b3113e4cfb7609020d.
Signed-off-by: Tommy Walton <waltont@amazon.com>
(It's a simple enough change to make manually, and making .materialtype is an uncommon workflow, so not worth doing this automatically).
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
- Added MaterialSourceData::ApplyVersionUpdates() for updating the properties. This should be called by tools after loading the MaterialSourceData. (But can be omitted if a tool wants to read the data exactly as it appears in the .material file).
- Updated MaterialTypeSourceData::FindProperty to support applying version update renames, including a ApplyPropertyRenames utility function, which are necessary for MaterialSourceData to be able to find the necessary property definitons while loading.
- Added a new context struct to JsonMaterialPropertyValueSerializer for passing down the material type version number, to help with applying property renames.
- Renamed the .material file format "propertyLayoutVersion" to "materialTypeVersion" which is more accurate. This shouldn't hurt existing data as this field wasn't actually used for anything before.
- Updated Material Editor to again store the material type version number in .material files.
MaterialSourceDataTests updates...
- Updated to include both a .materialtype file and a MaterialTypeAsset for the test material type. Both are used by the MaterialTypeSourceData class.
- The default test material type now includes some version update steps; these are only used for version update tests and won't impact the other test functions.
- Updated the path for storing temp files to disk, to just be in a "temp" folder in the exe path. (Originally they were saved to the gem folder near MaterialSourceDataTests.cpp, but at some point someone changed it to be under the exe folder, so there's no reason to use the full gem path anymore).
MaterialTypeSourceDataTests updates...
- Moved some code that was accidentally added to LoadAllFieldsUsingOldFormat but should have been in LoadAndStoreJson_AllFields.
- Added test cases for unsupported version update operations
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* adding Windows/release to PR-validation builds
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* changing trace back to expand to nothing for release
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* typo
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* more fixes
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* fixing some more unused variable cases
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* renaming file in ScriptCanvas that causes a msbuild warning
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* reverting a previous change
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
This revealed that the approach of reflecting both the old "id" and the new "name" would not work, because whenn saving it would write out both fields. So I decied to just give up on backward compatibility. This will be much cleaner than trying to continue supporting "id" as a field name, it is uncommon for users to make their own material types at this point, and if they have made some it is very easy to search and replace "id" with "name" update their files.
All .materialtype files have been updated. RPI unit tests now pass. ASV still passes.
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Add material property names to material assets, disable FBX dependency on materialtype files.
Signed-off-by: Robin <rbarrand@amazon.com>
* Add reflection for MaterialAssets. Update member variable comment.
Signed-off-by: Robin <rbarrand@amazon.com>
* Switch cvar to using bus value. Refine comments.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor functions and refine comments.
Signed-off-by: Robin <rbarrand@amazon.com>
* Realign property values when material property names are populated.
Signed-off-by: Robin <rbarrand@amazon.com>
* Switch PostLoadInit check to on asset status ready. Add realign property values code to PostLoadInit as well.
Signed-off-by: Robin <rbarrand@amazon.com>
* Stash@{1} code.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor realignment code into the right places.
Signed-off-by: Robin <rbarrand@amazon.com>
* Remove pragma optmize off.
Signed-off-by: Robin <rbarrand@amazon.com>
* More refactoring.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor comments and remove code no longer needed.
Signed-off-by: Robin <rbarrand@amazon.com>
* Refactor comments and remove unused include.
Signed-off-by: Robin <rbarrand@amazon.com>
* Comment refactor, corrected some code.
Signed-off-by: Robin <rbarrand@amazon.com>
Co-authored-by: Robin <rbarrand@amazon.com>
Made Model Material Conversion Optional
Added a new registry setting that disables automatic conversion of materials from model files like FBX.
By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.
Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.
Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows SceneAPI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.
Testing:
- Changing the registry setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switching the material conversion registry setting from true to false. (Details below)
- Merged this change to a customer's fork and tested on their existing content.
Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
- Entity 1 material assignments
- Blue_Zaxis: left as-is
- Green_Yaxis: exported the material
- Red_Xaxis: exported the material, and changed the material instance color to pink
- StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
- With_Texture: selected an existing brick material, changed the material instance color to red.
- Entity 2 material assignments
- Default Material: set to an existing brick material
- Blue_Zaxis: manually assigned built-in material that was converted from fbx
- Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
santorac
Esteban Papp
Tommy Walton
puvvadar
Artur K
Tom Hulton-Harrop
Tommy Walton
Robin
srikappa-amzn
AMZN-koppersr
srikappa-amzn
Steve Pham
hershey5045
chcurran