By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.
Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.
Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows ScenePI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.
Testing:
- Changing the registery setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switcihng the material conversion registry setting from true to false. (Details below)
- TODO: Will merge this change to a customer's fork and test on their existing content.
Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
- Entity 1 material assignments
- Blue_Zaxis: left as-is
- Green_Yaxis: exported the material
- Red_Xaxis: exported the material, and changed the material instance color to pink
- StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
- With_Texture: selected an existing brick material, changed the material instance color to red.
- Entity 2 material assignments
- Default Material: set to an existing brick material
- Blue_Zaxis: manually assigned built-in material that was converted from fbx
- Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
Signed-off-by: santorac <55155825+santorac@users.noreply.github.com>
* Added TSpace method setting which is only visible for MikkT generation.
* Fixed a bug with generating tangents for blend shapes.
* Renamed tangent space into generation method.
* Some code cleaning
Signed-off-by: Benjamin Jillich <jillich@amazon.com>
* Disabled to skip exporting a node in case there are child bones underneath, this broke motion extraction as we skipped the motion extraction node.
* Added several helper methods for getting the local space bind pose transform, finding all bones, getting the first bone for a given node name and a recursive has child bones.
* Unified the cloned get all bone methods.
Signed-off-by: Benjamin Jillich <jillich@amazon.com>
* Default setting when no tangents rule: MikkT tangents.
* Calculates tangents/bitangents for all available uv sets.
* Creates tangent/bitangent data in in case they are not existing yet (as in: the source scene contains tangents/bitangents)
* Overwrites the tangent/bitangent data from the source scene in case MikkT is wished.
* Added helper functions to create tangent/bitangent scene nodes, finding tangent/bitangent data for a given uv layer and calculating the number of uv layers provided by the mesh scene node.
Signed-off-by: Benjamin Jillich <jillich@amazon.com>
This fixes a bug where a Python script file would be run on a scene file
that didn't have a script file set.
Added a general case version to SceneBuilderWorker.cpp, to make it easy
to mark all scene files as dirty.
Automated tests for this will come in a separate pull request.
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Update fbx test dbgsg files
Remove rc products from fbx test expected output
Update warning counts for 2 of the tests - these are due to missing material properties
Add SkipAtomOutput setting to disable atom material and mesh processing in scene as these pull in a lot of external dependencies to Process
Removed BlendShapeData and MeshData debug output of positions/normals/faces
Updated ManifestImportRequestHandler to remove warning about Joining 2 absolute paths
Updated AssImpMaterialImporter to use new GenerateRelativeSourcePath API to fix issue where relative path generation failed with tmp project directories
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add comment for setting
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Add override dbgsg file to python test
The mesh optimizer specifically avoids creating optimized nodes for nodes that are not selected, resulting in a different scene graph that requires a different dbgsg file to compare with
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Replace hardcoded passing of specific registry setting to instead pass all registry settings to AssetBuilder
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
* Determine the original vertex index based on the position
The Assimp library does not expose the FBX control point indices. This
change causes vertices that are close enough in their position to be
considered as coming from the same control point. This allows the mesh
optimizer to consider vertices with the same control point index (or
"original vertex index" as it is called in the code) for deduplication.
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a filter view instead of reimplementing a filter view
Signed-off-by: Chris Burel <burelc@amazon.com>
* Don't attempt to weld similar vertices if there's blendshapes
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add test for the mesh optimizer's ability to weld nearby vertices
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add logging call to show mesh optimizer effect on vertex count
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a bunch of temporaries in order to make `position` `const`
Signed-off-by: Chris Burel <burelc@amazon.com>
* Supply the vertex index remapping to the optimized skin weights
This ensures that the optimized skin weights use the vertex indexes from
the optimized mesh
Signed-off-by: Chris Burel <burelc@amazon.com>
* {LYN-4514} Engine updates to enable PAB for the Blast Gem
* Engine updates to enable Python Asset Building for the Blast Gem
* added API to detect IsPythonActive()
* ExportProductList behavior
* scene manifest usage of generated assetinfo
* updated ScriptProcessorRuleBehavior to handle OnPrepareForExport
Tests: new tests for scene behavior via ExportProduct
Signed-off-by: Jackson <23512001+jackalbe@users.noreply.github.com>
* updated base on PR feedback
added more comments for IsPythonActive()
added a serializeContext for export product list
Signed-off-by: Jackson <23512001+jackalbe@users.noreply.github.com>
* Determine the original vertex index based on the position
The Assimp library does not expose the FBX control point indices. This
change causes vertices that are close enough in their position to be
considered as coming from the same control point. This allows the mesh
optimizer to consider vertices with the same control point index (or
"original vertex index" as it is called in the code) for deduplication.
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a filter view instead of reimplementing a filter view
Signed-off-by: Chris Burel <burelc@amazon.com>
* Don't attempt to weld similar vertices if there's blendshapes
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add test for the mesh optimizer's ability to weld nearby vertices
Signed-off-by: Chris Burel <burelc@amazon.com>
* Add logging call to show mesh optimizer effect on vertex count
Signed-off-by: Chris Burel <burelc@amazon.com>
* Use a bunch of temporaries in order to make `position` `const`
Signed-off-by: Chris Burel <burelc@amazon.com>
* First pass FBX -> Scene File conversion.
This is the simple pass, minimizing code changes and focused on comments.
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Step 1 of part 2 of the FBX -> Scene rename
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Renaming FbxSceneBuilder folder to SceneBuilder
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Renamed files
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* More FBX -> Scene
Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
* Final update copyright headers to reference license files at the repo root
Signed-off-by: spham <spham@amazon.com>
* Fix copyright validator unit tests to support the stale O3DE header scenario
Signed-off-by: spham <spham@amazon.com>
Some nodes have aiBones created by AssImp even though they aren't bones. Filter these out by looking through the node graph and only considering
Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
Use first offsetMatrix for a bone (there may be multiple) and calculate using offsetOfParent * inverseRootOffset * rootOffset * inverseOffset
(cherry picked from commit a8eae976dda39ad28f9aa235e1e6ba34715ac48a)
# Conflicts:
# Code/Tools/SceneAPI/FbxSceneBuilder/Importers/AssImpTransformImporter.cpp
For bone nodes, the Transform is computed by multiplying the parent offsetMatrix by the inverse of the node's offsetMatrix
Note that this currently disables the LimitBoneWeights option since that results in the removal of bone nodes that are not attached to a mesh.
Without the bones there is no way to retrieve the offsetMatrix, so the Transform cannot be computed correctly
Fixes LYN-3755
* Abort calls in AssImp, which occur when an assert is hit in builds that have asserts enabled (like debug) no longer generate a popup. Instead, they are captured as errors and an asset processing failure.
* Added missing include
* Added check for _WRITE_ABORT_MSG, so platforms that don't have it but have signals enabled (Linux profile) compile correctly
* Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)"
This reverts commit 58adcf168f.
* Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter"
This reverts commit 798d96f1a2, reversing
changes made to eb31d90ad9.
* Revert "Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter""
This reverts commit c1124f26d957388e88cc4990021314b5af247e1d.
* Revert "Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)""
This reverts commit 978477097892a22e83519646527ff52ba6532f35.
* Fixed how FbxImportRequestHandler is loaded
* Bumped version to force FBX to rebuild + removed unused variable
* Revert "Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)""
This reverts commit 978477097892a22e83519646527ff52ba6532f35.
* Revert "Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter""
This reverts commit c1124f26d957388e88cc4990021314b5af247e1d.
* Fixed a bad revert
* Better error reporting at AP launch
* AZ_CRC -> AZ_CRC_CE and removed delayed reload of settings registry file now that it's available at startup
* fixed typo in comment
* Revert "FBX settings can be opened again: g_fbxImporter is set, and if the ex… (#878)"
This reverts commit 58adcf168f.
* Revert "Merge pull request #753 from aws-lumberyard-dev/Helios_DataDrivenAssetImporter"
This reverts commit 798d96f1a2, reversing
changes made to eb31d90ad9.
* FBX settings can be opened again: g_fbxImporter is set, and if the extension list is empty, it is reloaded.
* auto -> auto*, .size() == 0 -> .empty()
Previously FbxImportRequestHandler used to be activated as part of DllMain init and never had CreateDescriptor called, which meant reflect was not called. BehaviorComponents get created and Activated as part of special SceneCore logic. Since this component now needs to be activated as part of the normal flow, reflecting it caused it to be picked up by the SceneCore activate logic, causing it to be created/activated twice
* Helios - LYN-3250 - Fixed morph targets for meshes that had multiple materials (#374)
Fixed morph targets for meshes that had multiple materials and were split by AssImp: Recombined them into one mesh in the O3DE scene graph, so the behavior would match FBX SDK.
* This simple fix pops any node that can not be registered using it name and unique ID
* Any scene file (i.e FBX) with the three or more nodes with the same name and parent will sort of work
* After the 2nd child node with the same name and parent will be dropped on export
Tests: added the new AutomatedTesting/Assets/BadAssets/three_same_named_nodes.fbx to regress test this
ATOM-15465: Fix issue with incorrect rotation on models.
* Revert "{LYN-3229} Update AssImp package with latest AssImp 3rd party source changes (#545)"
This reverts commit 06d2050ac4fe6b9d0d0c52e759cf73a9a3f7d3eb.
* bumping version to force assets to reprocess
* {LYN-2185} Helios - Added GraphObjectProxy::GetMethodList() function
* Updated the EditorPythonConsoleInterface to get Python type name info
* Added PythonBehaviorInfo to the GraphObjectProxy object to reflect abstract calls that can be invoked
* removed SCENE_DATA_API from Reflect() function
Jira: https://jira.agscollab.com/browse/LYN-2185
Tests: Added tests GraphObjectProxy_GetClassInfo_Loads & GraphObjectProxy_GetClassInfo_CorrectFormats to regress test
* fix compile error
* etchPythonTypeName() returns AZStd::string
* put None into a const char
* GraphObjectProxy is now hidden from Script Canvas
* the IGraphObject has been added to the BC
Jira: https://jira.agscollab.com/browse/LYN-3365
Tests: manual testing the node type is not in the SC editor
* LYN-2537 Moved the Engine and Editor folder to be within the EngineAssets folder
* Fixed Documentation in bootstrap.cfg to correct the path to the user project specific registry file
* Adding a newline to the output of AssetCatalog 'Registering asset..., but type is not set' message
* Updating the AssetProcessorPlatformConfig.setreg Scan Folder to detect
the @ENGINEROOT@/EngineAssets/Engine path for engine runtime assets and
@ENGINEROOT@/EngineAssets/Editor path for engine tool assets
* Updating references to Icons and other assets to account for moving the
Engine and Editor folder under a single EngineAssets folder
* Moving the Engine Settings Registry folder from Engine/Registry -> Registry
* Removed the LY_PROJECT_CMAKE_PATH define as it is not portable to other locations. It is hard coded to the project location that was used for the CMake configuration. Furthermore it paths with backslashes within it are treated as escape characters and not a path separator
* Updated the LyTestTools asset_processor.py script to copy the exclude.filetag from the EngineAssets/Engine directory now
* Fixed Atom Shader Preprocessing when running using an External Project
* Updated the TSGenerateAction.cpp to fix the build error with using a renamed variable
* Updated the Install_Common.cmake ly_setup_others function to install the
EngineAssets directory and the each of the Gem's Assets directory while
maintaining the relative directory structure to the Engine Root
Also updated the install step to install the Registry folder at the
engine root
* Fixed the copying of the Registry folder to be in the install root, instead of under a second 'Registry' folder
* Moving the AssetProcessorPlatformConfig.setreg file over to the Registry folder
* Updated the LyTestTools and C++ code to point that the new location of
the AssetProcessorPlatformConfig.setreg file inside of the Registry
folder
* Renamed Test AssetProcessor*Config.ini files to have the .setreg extension
* Converted the AssetProcessor test setreg files from ini format to json
format using the SerializeContextTools convert-ini command
* Updated the AssetProcessor CMakeLists.txt to copy over the test setreg files to the build folder
* Updated the assetprocessor test file list to point at the renamed AsssetProcessor*Config setreg filenames
* Removed the Output Prefix code from the AssetProcessor. The complexity that it brought to the AP code is not needed, as users can replicate the behavior by just moving there assets underneath a another folder, underneath the scan folder
* Adding back support to read the AssetProcessorPlatformConfig.setreg file from the asset root. This is only needed for C++ UnitTests as they run in an environment where the accessing the Engine Settings Registry is not available
* Updating the Install_common.cmake logic to copy any "Assets" folder to
the install layout.
The Script has also been updated to copy over the "Assets" folder in the
Engine Root to the install layout instead of an "EngineAssets" folder
* Updating References to EngineAssets source asset folder in code to be the Assets source folder
* Moved the Engine Source Asset folder of 'EngineAssets' to a new folder name of 'Assets'. This is inline with the naming scheme we use for Gem asset folders
* Adding the EngineFinder.cmake to the AutomatedTesting project to allow it to work in a project centric manner
* Updating the LyTestTools copy_assets_to_project function to be able to copy assets with folders to the temporary project root
Fixed an issue in LyTestTools where the temporary log directory could have shutil.rmtree being called twice on it leading to an exception which fails an automated test
Updated the asset_procesor_gui_tests_2 AddScanFolder test to not use the
output prefix, but instead place the source asset root into a
subdirectory
* Correct the AssetProcessorPlatformConfig Scan Folders for the EngineAssets directory to point at the Assets directory
* Updated the asset procesor batch dependency test scan folder to point at the 'Assets' folder instead of 'EngineAssets'
{LYN-2074} Add Animation data types Behavior for the scene graph (#253)
* https://jira.agscollab.com/browse/LYN-2074
* moved scene API color to centeralized location
BlendShapeDataFace
BlendShapeData
- Reexport cloth assets with AssImp ON. These was necessary because AssImp collects a different name for the color streams than FbxSDK and therefore they needed to be reassigned in the cloth rule.
- Adding '_optimized' string to a global variable and using StringFunc RChop to remove it for a string.
* AssImp set to be the default FBX processor (#78)
If you encounter issues, reach out to the Helios team with details, and then switch back to FBX SDK locally by changing FBXImporter.h
* Merge pull request #219 from aws-lumberyard-dev/sceneapi_scripting
LYN-3030: Fix to export_chunks_builder.py to match the AssImp node paths
* Hide some automated test folders from the asset processor for automatedtesting. This is necessary because these assets should only be processed when running the test, and not any time AP is launched for this project.
* Putting these test assets back to visible to Asset Processor. These tests need to be updated at some point to handle that, but they won't work with this change for now. Note that until this is addressed, these tests may randomly time out if they're the first tests run on a clean asset processor, and these tests launch AP without using a unique port, so the test can also fail if an Asset Processor executable is hanging open.
Grabbed the change I missed from the 1.0 branch merge, no idea how this got lost.
* Moved from main to periodic. Allen and Fuzzy were already on board, and I think with the potential flakiness in this test, we don't want this in main.
Co-authored-by: jackalbe <23512001+jackalbe@users.noreply.github.com>
* Added error reporting for data integrity issues for non-uniform motion data.
* The actual issue was a mismatch between the end times of the morph and the skeletal animations. They need to match in EMotionFX.
* The morph target animation exported a keyframe too much.
* renaming and organizing files
* removed unused files
* Removing unnecessary file
* moved file
* reverting movement of 3rdparty associations from gems to global
* removing unnecessary calls to ly_add_external_target_path
* fixing install prefix of ci_build
* Fixes to get 3rdparties declared in gems to be installed
* Allowing to install just one configuration
* Adding empty line at the end
* removing commented code
* setting IMPORETD_LOCATION_<CONFIG> and defaulting IMPORTED_LOCATION to the profile config in case other configs are not installed
Esteban Papp
Chris Santora
pappeste
Benjamin Jillich
Benjamin Jillich
AMZN-stankowi
sharmajs-amzn
amzn-mike
nemerle
Chris Burel
Steve Pham
jackalbe
AMZN-stankowi
yuriy0
greerdv
karlberg
lumberyard-employee-dm
Aaron Ruiz Mora
Terry Michaels