Made Model Material Conversion Optional
Added a new registry setting that disables automatic conversion of materials from model files like FBX.
By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.
Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.
Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows SceneAPI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.
Testing:
- Changing the registry setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switching the material conversion registry setting from true to false. (Details below)
- Merged this change to a customer's fork and tested on their existing content.
Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
- Entity 1 material assignments
- Blue_Zaxis: left as-is
- Green_Yaxis: exported the material
- Red_Xaxis: exported the material, and changed the material instance color to pink
- StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
- With_Texture: selected an existing brick material, changed the material instance color to red.
- Entity 2 material assignments
- Default Material: set to an existing brick material
- Blue_Zaxis: manually assigned built-in material that was converted from fbx
- Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
Moved BuildDebugSceneGraph out of SceneProcessing gem and into DebugOutput class to allow the function to be shared, which ensures the same dbgsg format is outputted among any calls to the function.
* Move BuildDebugSceneGraph function, update all calls to this function
Signed-off-by: Victor Huang <huavicto@amazon.com>
* Remove unused includes
Signed-off-by: Victor Huang <huavicto@amazon.com>
* Remove z
Signed-off-by: Victor Huang <huavicto@amazon.com>
These changes allow for usage of different asset import SDKs to process scene files.
Move AssImp specific code out of node, scene & material wrapper parent classes and into child wrapper classes (AssImpNodeWrapper, etc.), allowing child classes to expose import SDK code. Allows for more convenient implementation of other import SDK's elsewhere (such as in a gem).
Add a loadingComponentUuid parameter to LoadSceneFromVerifiedPath to allow for usage of different loading components. Changed tests and all calls to this function accordingly.
* Move AssImp specific code out of wrapper parent classes and into child classes for gem usage
Signed-off-by: Victor Huang <huavicto@amazon.com>
* Add loadingComponentUuid parameter to LoadSceneFromVerifiedPath function
Signed-off-by: Victor Huang <huavicto@amazon.com>
* Make wrapper members protected, change pointer cast
Signed-off-by: Victor Huang <huavicto@amazon.com>
* Adding spaces to fix style
Signed-off-by: Victor Huang <huavicto@amazon.com>
* Fix for pointer cast causing test failures
Signed-off-by: Victor Huang <huavicto@amazon.com>
By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.
Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.
Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows ScenePI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.
Testing:
- Changing the registery setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switcihng the material conversion registry setting from true to false. (Details below)
- TODO: Will merge this change to a customer's fork and test on their existing content.
Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
- Entity 1 material assignments
- Blue_Zaxis: left as-is
- Green_Yaxis: exported the material
- Red_Xaxis: exported the material, and changed the material instance color to pink
- StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
- With_Texture: selected an existing brick material, changed the material instance color to red.
- Entity 2 material assignments
- Default Material: set to an existing brick material
- Blue_Zaxis: manually assigned built-in material that was converted from fbx
- Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Added TSpace method setting which is only visible for MikkT generation.
* Fixed a bug with generating tangents for blend shapes.
* Renamed tangent space into generation method.
* Some code cleaning
Signed-off-by: Benjamin Jillich <jillich@amazon.com>
The previous version returned the incorrect size from the position map when
blendshapes are present in the model. When there are blendshapes, the
vertex welding is disabled, and nothing is inserted into the position map.
However, the position map's size was being used to dictate how many
elements to create in the skinning info. The skinning info tries to
compensate for an incorrect max vertex index by resizing its underlying
vector when adding an influence, but that was using the index as the new
size of the vector, so it was off by one. This fixes both errors.
Signed-off-by: Chris Burel <burelc@amazon.com>
* Default setting when no tangents rule: MikkT tangents.
* Calculates tangents/bitangents for all available uv sets.
* Creates tangent/bitangent data in in case they are not existing yet (as in: the source scene contains tangents/bitangents)
* Overwrites the tangent/bitangent data from the source scene in case MikkT is wished.
* Added helper functions to create tangent/bitangent scene nodes, finding tangent/bitangent data for a given uv layer and calculating the number of uv layers provided by the mesh scene node.
Signed-off-by: Benjamin Jillich <jillich@amazon.com>
This fixes a bug where a Python script file would be run on a scene file
that didn't have a script file set.
Added a general case version to SceneBuilderWorker.cpp, to make it easy
to mark all scene files as dirty.
Automated tests for this will come in a separate pull request.
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
The thought behind Array2D was that it would be more efficient from a
memory allocation perspective to have one fixed buffer that grows than
it would be to have a vector of vectors. In reality, the runtime of
inserting into the middle of one large buffer, and shifting all the
resulting elements, ends up far outweighing any memory allocation
overhead.
In the mesh optimizer, things are added to the end of each sub-vector
one by one. It isn't known up front how many elements each sub-vector
will have. With vertex welding enabled, it is far more likely that a
given vertex will be influenced by a large number of joints. When using
the Array2D to store the influences, and a vertex with a low index has
more than 4 influences, those influences have to be inserted into close
to the front of the big vector, and all the other elements shifted.
Array2D was doing this with a linear shift, shifting the elements by 1
at a time, and not providing any exponential growth on the amount of
elements reserved by each sub-vector. The result is a linear insertion
time.
By contrast, using vector<vector<Influence>> instead gives us back the
amortized constant insertion time. Since the skin influences are just
added to the end of each sub-vector, no shifting of the elements is
necessary. And since the amount of original vertex indices is known up
front, the number of sub-vectors can still be pre-created, so the
sub-vectors themselves never need to shift.
Signed-off-by: Chris Burel <burelc@amazon.com>
The Assimp library does not expose the FBX control point indices. This
change causes vertices that are close enough in their position to be
considered as coming from the same control point. This allows the mesh
optimizer to consider vertices with the same control point index (or
"original vertex index" as it is called in the code) for deduplication.
Signed-off-by: Chris Burel <burelc@amazon.com>
* Determine the original vertex index based on the position
The Assimp library does not expose the FBX control point indices. This
change causes vertices that are close enough in their position to be
considered as coming from the same control point. This allows the mesh
optimizer to consider vertices with the same control point index (or
"original vertex index" as it is called in the code) for deduplication.
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a filter view instead of reimplementing a filter view
Signed-off-by: Chris Burel <burelc@amazon.com>
* Don't attempt to weld similar vertices if there's blendshapes
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add test for the mesh optimizer's ability to weld nearby vertices
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add logging call to show mesh optimizer effect on vertex count
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a bunch of temporaries in order to make `position` `const`
Signed-off-by: Chris Burel <burelc@amazon.com>
* Supply the vertex index remapping to the optimized skin weights
This ensures that the optimized skin weights use the vertex indexes from
the optimized mesh
Signed-off-by: Chris Burel <burelc@amazon.com>
* Determine the original vertex index based on the position
The Assimp library does not expose the FBX control point indices. This
change causes vertices that are close enough in their position to be
considered as coming from the same control point. This allows the mesh
optimizer to consider vertices with the same control point index (or
"original vertex index" as it is called in the code) for deduplication.
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a filter view instead of reimplementing a filter view
Signed-off-by: Chris Burel <burelc@amazon.com>
* Don't attempt to weld similar vertices if there's blendshapes
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add test for the mesh optimizer's ability to weld nearby vertices
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add logging call to show mesh optimizer effect on vertex count
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a bunch of temporaries in order to make `position` `const`
Signed-off-by: Chris Burel <burelc@amazon.com>
* First pass FBX -> Scene File conversion.
This is the simple pass, minimizing code changes and focused on comments.
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Step 1 of part 2 of the FBX -> Scene rename
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Renaming FbxSceneBuilder folder to SceneBuilder
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Renamed files
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* More FBX -> Scene
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Final update copyright headers to reference license files at the repo root
Signed-off-by: spham <spham@amazon.com>
* Fix copyright validator unit tests to support the stale O3DE header scenario
Signed-off-by: spham <spham@amazon.com>
* Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)"
This reverts commit 58adcf168f.
* Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter"
This reverts commit 798d96f1a2, reversing
changes made to eb31d90ad9.
* Revert "Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter""
This reverts commit c1124f26d957388e88cc4990021314b5af247e1d.
* Revert "Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)""
This reverts commit 978477097892a22e83519646527ff52ba6532f35.
* Fixed how FbxImportRequestHandler is loaded
* Bumped version to force FBX to rebuild + removed unused variable
* Revert "Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)""
This reverts commit 978477097892a22e83519646527ff52ba6532f35.
* Revert "Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter""
This reverts commit c1124f26d957388e88cc4990021314b5af247e1d.
* Fixed a bad revert
* Better error reporting at AP launch
* AZ_CRC -> AZ_CRC_CE and removed delayed reload of settings registry file now that it's available at startup
* fixed typo in comment
* Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)"
This reverts commit 58adcf168f.
* Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter"
This reverts commit 798d96f1a2, reversing
changes made to eb31d90ad9.
* Helios - LYN-3250 - Fixed morph targets for meshes that had multiple materials (#374)
Fixed morph targets for meshes that had multiple materials and were split by AssImp: Recombined them into one mesh in the O3DE scene graph, so the behavior would match FBX SDK.
This is cherry-picked from #311
When processing meshes with blend shapes, the mesh optimizer disables the
optimize duplicates setting, to prevent potential vertex reodering that
could cause the base mesh vertices to become out of sync with the blend
shape. However, it will still reorder vertices based on their material.
It places all triangles that use the same material in the same submesh,
grouping them together in the resulting mesh. The SceneAPI does not track
material ids for blend shapes. To ensure that the blend shape triangles are
reordered in the same way as the base shape, this change makes the blend
shape optimization use the material id from the base shape.
- Remove some references to gEnv->pRenderer/GetIEditor()->GetRenderer() that is now always null.
- Restore the debug console to existence.
- Stop building the following in preparation for their removal:
Code/CryEngine/Cry3DEngine/*
Code/CryEngine/RenderDll/*
Code/Tools/CryFXC/*
Code/Tools/HLSLCrossCompiler/*
Code/Tools/HLSLCrossCompilerMETAL/*
Code/Tools/RC/*
Code/Tools/ShaderCacheGen/*
Tools/CrySCompileServer/*
- Reexport cloth assets with AssImp ON. These was necessary because AssImp collects a different name for the color streams than FbxSDK and therefore they needed to be reassigned in the cloth rule.
- Adding '_optimized' string to a global variable and using StringFunc RChop to remove it for a string.
Chris Galvan
santorac
Esteban Papp
Jeremy Ong
amzn-victor
Benjamin Jillich
Chris Santora
pappeste
Chris Burel
Benjamin Jillich
AMZN-stankowi
Steve Pham
hultonha
AMZN-stankowi
amzn-mike
bosnichd
Tommy Walton
Aaron Ruiz Mora