birkeh/o3de
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #2746 from aws-lumberyard-dev/Atom/santorac/OptionalSceneApiMaterialConversion3

Browse Source 
Made Model Material Conversion Optional

Added a new registry setting that disables automatic conversion of materials from model files like FBX.

By default, processing of model files (like FBX) automatically convert the included materials to Atom materials, using StandardPBR. This adds a job dependency on StandardPBR.materialtype, which propagates to any related azsl files as well. Thus any change to azsl code will cause all model files in the project to rebuild.

Some game teams have no interest in using the auto-converted materials; they always use a Material Component to apply material overrides for every mesh. This new setting allows teams to disable material auto-conversion for the entire project, thus removing the job dependency on StandardPBR.materialtype. Instead, every mesh will be assigned the same default material. Any change to azsl code will cause that one default material to rebuild, but this will not trigger any models to rebuild.

Details:
- Added /O3DE/SceneAPI/MaterialConverter registry settings for configuring the scene material converter. It includes an enable flag, and a default material to use when conversion is disabled.
- Added SceneBuilderDependencyRequests::AddFingerprintInfo which allows SceneAPI components to modify the scene builder analysis fingerprint. We use this to reprocess scene files when the material converter settings change.
- Updated SceneAPI's material asset builder to skip the StandardPBR dependency when material conversion is disabled.
- Added some code to MaterialComponentController to handle an edge case that may when disabling material conversion on an existing project, and assigned materials disappear.

Testing:
- Changing the registry setting does trigger a rebuild of the fbx files.
- When material conversion is disabled, changing an azsl file does not cause fbx files to rebuild, but the shader still reloads as expected.
- Made a test level using multiple models with multiple meshes, made various adjustments to the material slots for each mesh, and tried switching the material conversion registry setting from true to false. (Details below)
- Merged this change to a customer's fork and tested on their existing content.

Details about my test level:
- Made a new test level AtomTest project
- Added two entities, both using multi-mat_mesh-groups_1m_cubes.fbx
- Added a material component to both entities
  - Entity 1 material assignments
    - Blue_Zaxis: left as-is
    - Green_Yaxis: exported the material
    - Red_Xaxis: exported the material, and changed the material instance color to pink
    - StingrayPBS1: exported the material, scaled the UVs in the exported material source, and changed the material instance color to green.
    - With_Texture: selected an existing brick material, changed the material instance color to red.
  - Entity 2 material assignments
    - Default Material: set to an existing brick material
    - Blue_Zaxis: manually assigned built-in material that was converted from fbx
    - Green_Yaxis: manually assigned built-in material that was converted from fbx, and changed the material instance color to orange
This commit is contained in:
santorac
2021-08-23 15:24:21 -07:00
committed by GitHub
parent 6ab776a3a6 b6dfb86007
commit 60d137c5b6
14 changed files with 241 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Code/Tools/SceneAPI/SceneCore/SceneBuilderDependencyBus.h
+6 -1
View File
@@ -24,7 +24,12 @@ namespace AZ
: public AZ::EBusTraits
{
public:
virtual void ReportJobDependencies(JobDependencyList& jobDependencyList, const char* platformIdentifier) = 0;
//! Builders can implement this function to add job dependencies on other assets that may be used in the scene file conversion process.
virtual void ReportJobDependencies(JobDependencyList& jobDependencyList, const char* platformIdentifier) { AZ_UNUSED(jobDependencyList); AZ_UNUSED(platformIdentifier); }
//! Builders can implement this function to append to the job analysis fingerprint. This can be used to trigger rebuilds when global configuration changes.
//! See also AssetBuilderDesc::m_analysisFingerprint.
virtual void AddFingerprintInfo(AZStd::set<AZStd::string>& fingerprintInfo) { AZ_UNUSED(fingerprintInfo); }
};
using SceneBuilderDependencyBus = EBus<SceneBuilderDependencyRequests>;
} // namespace SceneAPI
Gems/Atom/Feature/Common/Code/Source/Material/MaterialConverterSystemComponent.cpp
+48 -1
View File
@@ -12,12 +12,26 @@
#include <AzCore/Math/Color.h>
#include <AzCore/Serialization/EditContext.h>
#include <AzCore/Serialization/SerializeContext.h>
#include <AzCore/Settings/SettingsRegistry.h>
#include <AzToolsFramework/API/EditorAssetSystemAPI.h>
#include <Atom/RPI.Reflect/Material/MaterialAsset.h>
namespace AZ
{
namespace Render
{
void MaterialConverterSettings::Reflect(AZ::ReflectContext* context)
{
if (auto serializeContext = azrtti_cast<AZ::SerializeContext*>(context))
{
serializeContext->Class<MaterialConverterSettings>()
->Version(1)
->Field("Enable", &MaterialConverterSettings::m_enable)
->Field("DefaultMaterial", &MaterialConverterSettings::m_defaultMaterial);
}
}
void MaterialConverterSystemComponent::Reflect(AZ::ReflectContext* context)
{
if (auto* serialize = azrtti_cast<SerializeContext*>(context))
@@ -26,10 +40,22 @@ namespace AZ
->Version(3)
->Attribute(Edit::Attributes::SystemComponentTags, AZStd::vector<Crc32>({ AssetBuilderSDK::ComponentTags::AssetBuilder }));
}
MaterialConverterSettings::Reflect(context);
}
void MaterialConverterSystemComponent::GetProvidedServices(AZ::ComponentDescriptor::DependencyArrayType& services)
{
services.emplace_back(AZ_CRC_CE("FingerprintModification"));
}
void MaterialConverterSystemComponent::Activate()
{
if (auto* settingsRegistry = AZ::SettingsRegistry::Get())
{
settingsRegistry->GetObject(m_settings, "/O3DE/SceneAPI/MaterialConverter");
}
RPI::MaterialConverterBus::Handler::BusConnect();
}
@@ -37,11 +63,21 @@ namespace AZ
{
RPI::MaterialConverterBus::Handler::BusDisconnect();
}
bool MaterialConverterSystemComponent::IsEnabled() const
{
return m_settings.m_enable;
}
bool MaterialConverterSystemComponent::ConvertMaterial(
const AZ::SceneAPI::DataTypes::IMaterialData& materialData, RPI::MaterialSourceData& sourceData)
{
using namespace AZ::RPI;
if (!m_settings.m_enable)
{
return false;
}
// The source data for generating material asset
sourceData.m_materialType = GetMaterialTypePath();
@@ -140,9 +176,20 @@ namespace AZ
return true;
}
const char* MaterialConverterSystemComponent::GetMaterialTypePath() const
AZStd::string MaterialConverterSystemComponent::GetMaterialTypePath() const
{
return "Materials/Types/StandardPBR.materialtype";
}
AZStd::string MaterialConverterSystemComponent::GetDefaultMaterialPath() const
{
if (m_settings.m_defaultMaterial.empty())
{
AZ_Error("MaterialConverterSystemComponent", m_settings.m_enable,
"Material conversion is disabled but a default material not specified in registry /O3DE/SceneAPI/MaterialConverter/DefaultMaterial");
}
return m_settings.m_defaultMaterial;
}
}
}
Gems/Atom/Feature/Common/Code/Source/Material/MaterialConverterSystemComponent.h
+18 -1
View File
@@ -18,6 +18,16 @@ namespace AZ
{
namespace Render
{
struct MaterialConverterSettings
{
AZ_TYPE_INFO(MaterialConverterSettings, "{8D91601D-570A-4557-99C8-631DB4928040}");
static void Reflect(AZ::ReflectContext* context);
bool m_enable = true;
AZStd::string m_defaultMaterial;
};
//! Atom's implementation of converting SceneAPI data into Atom's default material: StandardPBR
class MaterialConverterSystemComponent final
: public AZ::Component
@@ -27,13 +37,20 @@ namespace AZ
AZ_COMPONENT(MaterialConverterSystemComponent, "{C2338D45-6456-4521-B469-B000A13F2493}");
static void Reflect(AZ::ReflectContext* context);
static void GetProvidedServices(AZ::ComponentDescriptor::DependencyArrayType& services);
void Activate() override;
void Deactivate() override;
// MaterialConverterBus overrides ...
bool IsEnabled() const override;
bool ConvertMaterial(const AZ::SceneAPI::DataTypes::IMaterialData& materialData, RPI::MaterialSourceData& out) override;
const char* GetMaterialTypePath() const override;
AZStd::string GetMaterialTypePath() const override;
AZStd::string GetDefaultMaterialPath() const override;
private:
MaterialConverterSettings m_settings;
};
}
}
Gems/Atom/RPI/Code/Include/Atom/RPI.Edit/Material/MaterialConverterBus.h
+12 -3
View File
@@ -29,10 +29,19 @@ namespace AZ
: public AZ::EBusTraits
{
public:
//! Returns true if the converion was successful
virtual bool IsEnabled() const = 0;
//! Converts data from a IMaterialData object to an Atom MaterialSourceData.
//! Only works when IsEnabled() is true.
//! @return true if the MaterialSourceData output was populated with converted material data.
virtual bool ConvertMaterial(const AZ::SceneAPI::DataTypes::IMaterialData& materialData, MaterialSourceData& out) = 0;
//! Returns the path to the .materialtype file that the materials are based on, such as StandardPBR.materialtype, etc.
virtual const char* GetMaterialTypePath() const = 0;
//! Returns the path to the .materialtype file that the converted materials are based on, such as StandardPBR.materialtype, etc.
virtual AZStd::string GetMaterialTypePath() const = 0;
//! Returns the path to a .material file to use as the default material when conversion is disabled.
virtual AZStd::string GetDefaultMaterialPath() const = 0;
};
using MaterialConverterBus = AZ::EBus<MaterialConverterRequests>;
Gems/Atom/RPI/Code/Include/Atom/RPI.Reflect/Model/ModelMaterialSlot.h
+2
View File
@@ -25,6 +25,8 @@ namespace AZ
static void Reflect(AZ::ReflectContext* context);
// Note that StableId is uint32_t for legacy reasons: we used to use AssetId::m_subId as the material slot ID. But actually the original MaterialUid
// is 64 bit so we might want to switch this to be uint64_t at some point.
using StableId = uint32_t;
static const StableId InvalidStableId;
Gems/Atom/RPI/Code/Source/RPI.Builders/Model/MaterialAssetBuilderComponent.cpp
+117 -12
View File
@@ -28,6 +28,7 @@
#include <Atom/RPI.Edit/Material/MaterialSourceData.h>
#include <Atom/RPI.Edit/Material/MaterialConverterBus.h>
#include <Atom/RPI.Edit/Common/AssetUtils.h>
#include <Atom/RPI.Reflect/Material/MaterialAsset.h>
#include <AzCore/Settings/SettingsRegistry.h>
@@ -70,28 +71,74 @@ namespace AZ
void MaterialAssetDependenciesComponent::ReportJobDependencies(SceneAPI::JobDependencyList& jobDependencyList, const char* platformIdentifier)
{
AssetBuilderSDK::SourceFileDependency materialTypeSource;
bool conversionEnabled = false;
RPI::MaterialConverterBus::BroadcastResult(conversionEnabled, &RPI::MaterialConverterBus::Events::IsEnabled);
// Right now, scene file importing only supports a single material type, once that changes, this will have to be re-designed, see ATOM-3554
RPI::MaterialConverterBus::BroadcastResult(materialTypeSource.m_sourceFileDependencyPath, &RPI::MaterialConverterBus::Events::GetMaterialTypePath);
AZStd::string materialTypePath;
RPI::MaterialConverterBus::BroadcastResult(materialTypePath, &RPI::MaterialConverterBus::Events::GetMaterialTypePath);
AssetBuilderSDK::JobDependency jobDependency;
jobDependency.m_jobKey = "Atom Material Builder";
jobDependency.m_sourceFile = materialTypeSource;
jobDependency.m_platformIdentifier = platformIdentifier;
jobDependency.m_type = AssetBuilderSDK::JobDependencyType::Order;
if (!materialTypeSource.m_sourceFileDependencyPath.empty())
if (conversionEnabled && !materialTypePath.empty())
{
AssetBuilderSDK::SourceFileDependency materialTypeSource;
materialTypeSource.m_sourceFileDependencyPath = materialTypePath;
AssetBuilderSDK::JobDependency jobDependency;
jobDependency.m_jobKey = "Atom Material Builder";
jobDependency.m_sourceFile = materialTypeSource;
jobDependency.m_platformIdentifier = platformIdentifier;
jobDependency.m_type = AssetBuilderSDK::JobDependencyType::Order;
jobDependencyList.push_back(jobDependency);
}
}
void MaterialAssetDependenciesComponent::AddFingerprintInfo(AZStd::set<AZStd::string>& fingerprintInfo)
{
// This will cause scene files to be reprocessed whenever the global MaterialConverter settings change.
bool conversionEnabled = false;
RPI::MaterialConverterBus::BroadcastResult(conversionEnabled, &RPI::MaterialConverterBus::Events::IsEnabled);
fingerprintInfo.insert(AZStd::string::format("[MaterialConverter enabled=%d]", conversionEnabled));
if (!conversionEnabled)
{
AZStd::string defaultMaterialPath;
RPI::MaterialConverterBus::BroadcastResult(defaultMaterialPath, &RPI::MaterialConverterBus::Events::GetDefaultMaterialPath);
fingerprintInfo.insert(AZStd::string::format("[MaterialConverter defaultMaterial=%s]", defaultMaterialPath.c_str()));
}
}
void MaterialAssetBuilderComponent::Reflect(ReflectContext* context)
{
if (auto* serialize = azrtti_cast<SerializeContext*>(context))
{
serialize->Class<MaterialAssetBuilderComponent, SceneAPI::SceneCore::ExportingComponent>()
->Version(16); // Optional material conversion
->Version(16); // Optional material conversion
}
}
Data::Asset<MaterialAsset> MaterialAssetBuilderComponent::GetDefaultMaterialAsset() const
{
AZStd::string defaultMaterialPath;
RPI::MaterialConverterBus::BroadcastResult(defaultMaterialPath, &RPI::MaterialConverterBus::Events::GetDefaultMaterialPath);
if (defaultMaterialPath.empty())
{
return {};
}
else
{
auto defaultMaterialAssetId = RPI::AssetUtils::MakeAssetId(defaultMaterialPath, 0);
if (!defaultMaterialAssetId.IsSuccess())
{
AZ_Error("MaterialAssetBuilderComponent", false, "Could not find asset '%s'", defaultMaterialPath.c_str());
return {};
}
else
{
return Data::AssetManager::Instance().CreateAsset<RPI::MaterialAsset>(defaultMaterialAssetId.GetValue(), Data::AssetLoadBehaviorNamespace::PreLoad);
}
}
}
@@ -119,8 +166,8 @@ namespace AZ
BindToCall(&MaterialAssetBuilderComponent::BuildMaterials);
}
SceneAPI::Events::ProcessingResult MaterialAssetBuilderComponent::BuildMaterials(MaterialAssetBuilderContext& context) const
SceneAPI::Events::ProcessingResult MaterialAssetBuilderComponent::ConvertMaterials(MaterialAssetBuilderContext& context) const
{
const auto& scene = context.m_scene;
const Uuid sourceSceneUuid = scene.GetSourceGuid();
@@ -193,6 +240,64 @@ namespace AZ
return SceneAPI::Events::ProcessingResult::Success;
}
SceneAPI::Events::ProcessingResult MaterialAssetBuilderComponent::AssignDefaultMaterials(MaterialAssetBuilderContext& context) const
{
Data::Asset<MaterialAsset> defaultMaterialAsset = GetDefaultMaterialAsset();
if (!defaultMaterialAsset.GetId().IsValid())
{
AZ_Warning("MaterialAssetBuilderComponent", false, "Material conversion is disabled but no default material was provided. The model will likely be invisible by default.");
// Return success because it's just a warning.
return SceneAPI::Events::ProcessingResult::Success;
}
const auto& scene = context.m_scene;
const Uuid sourceSceneUuid = scene.GetSourceGuid();
const auto& sceneGraph = scene.GetGraph();
auto names = sceneGraph.GetNameStorage();
auto content = sceneGraph.GetContentStorage();
auto pairView = SceneAPI::Containers::Views::MakePairView(names, content);
auto view = SceneAPI::Containers::Views::MakeSceneGraphDownwardsView<
SceneAPI::Containers::Views::BreadthFirst>(
sceneGraph, sceneGraph.GetRoot(), pairView.cbegin(), true);
for (const auto& viewIt : view)
{
if (viewIt.second == nullptr)
{
continue;
}
if (azrtti_istypeof<SceneAPI::DataTypes::IMaterialData>(viewIt.second.get()))
{
auto materialData = AZStd::static_pointer_cast<const SceneAPI::DataTypes::IMaterialData>(viewIt.second);
uint64_t materialUid = materialData->GetUniqueId();
context.m_outputMaterialsByUid[materialUid] = { defaultMaterialAsset, materialData->GetMaterialName() };
}
}
return SceneAPI::Events::ProcessingResult::Success;
}
SceneAPI::Events::ProcessingResult MaterialAssetBuilderComponent::BuildMaterials(MaterialAssetBuilderContext& context) const
{
bool conversionEnabled = false;
RPI::MaterialConverterBus::BroadcastResult(conversionEnabled, &RPI::MaterialConverterBus::Events::IsEnabled);
if (conversionEnabled)
{
return ConvertMaterials(context);
}
else
{
return AssignDefaultMaterials(context);
}
}
} // namespace RPI
} // namespace AZ
Gems/Atom/RPI/Code/Source/RPI.Builders/Model/MaterialAssetBuilderComponent.h
+8
View File
@@ -39,6 +39,13 @@ namespace AZ
// Required for ExportingComponent
static void Reflect(AZ::ReflectContext* context);
private:
SceneAPI::Events::ProcessingResult ConvertMaterials(MaterialAssetBuilderContext& context) const;
SceneAPI::Events::ProcessingResult AssignDefaultMaterials(MaterialAssetBuilderContext& context) const;
Data::Asset<MaterialAsset> GetDefaultMaterialAsset() const;
};
/**
@@ -65,6 +72,7 @@ namespace AZ
// SceneAPI::SceneBuilderDependencyBus::Handler overrides...
void ReportJobDependencies(SceneAPI::JobDependencyList& jobDependencyList, const char* platformIdentifier) override;
void AddFingerprintInfo(AZStd::set<AZStd::string>& fingerprintInfo) override;
};
} // namespace RPI
} // namespace AZ
Gems/Atom/RPI/Code/Source/RPI.Builders/Model/ModelExporterComponent.cpp
+9 -1
View File
@@ -78,9 +78,17 @@ namespace AZ
//Export MaterialAssets
for (auto& materialPair : materialsByUid)
{
const Data::Asset<MaterialAsset>& asset = materialPair.second.m_asset;
// MaterialAssetBuilderContext could attach an independent material asset rather than
// generate one using the scene data, so we must skip the export step in that case.
if (asset.GetId().m_guid != exportEventContext.GetScene().GetSourceGuid())
{
continue;
}
uint64_t materialUid = materialPair.first;
const AZStd::string& sceneName = exportEventContext.GetScene().GetName();
const Data::Asset<MaterialAsset>& asset = materialPair.second.m_asset;
// escape the material name acceptable for a filename
AZStd::string materialName = materialPair.second.m_name;
Gems/AtomLyIntegration/CommonFeatures/Code/Source/Material/MaterialComponentController.cpp
+4 -2
View File
@@ -8,6 +8,7 @@
#include <Material/MaterialComponentController.h>
#include <Atom/RPI.Reflect/Material/MaterialAsset.h>
#include <Atom/RPI.Reflect/Asset/AssetUtils.h>
#include <AzCore/Serialization/SerializeContext.h>
#include <AtomCore/Instance/InstanceDatabase.h>
@@ -133,7 +134,7 @@ namespace AZ
{
InitializeMaterialInstance(asset);
}
void MaterialComponentController::OnTick([[maybe_unused]] float deltaTime, [[maybe_unused]] AZ::ScriptTimePoint time)
{
AZStd::unordered_set<MaterialAssignmentId> propertyOverrides;
@@ -208,6 +209,7 @@ namespace AZ
for (auto& materialPair : m_configuration.m_materials)
{
auto& materialAsset = materialPair.second.m_materialAsset;
if (materialAsset.GetId().IsValid() && !Data::AssetBus::MultiHandler::BusIsConnectedId(materialAsset.GetId()))
{
anyQueued = true;
@@ -221,7 +223,7 @@ namespace AZ
ReleaseMaterials();
}
}
void MaterialComponentController::InitializeMaterialInstance(const Data::Asset<Data::AssetData>& asset)
{
bool allReady = true;
Gems/AtomLyIntegration/CommonFeatures/Code/Source/Material/MaterialComponentController.h
+2 -2
View File
@@ -92,11 +92,11 @@ namespace AZ
AZ_DISABLE_COPY(MaterialComponentController);
//! Data::AssetBus interface
//! Data::AssetBus overrides...
void OnAssetReady(Data::Asset<Data::AssetData> asset) override;
void OnAssetReloaded(Data::Asset<Data::AssetData> asset) override;
//! AZ::TickBus interface implementation
// AZ::TickBus overrides...
void OnTick(float deltaTime, AZ::ScriptTimePoint time) override;
void LoadMaterials();
Gems/SceneProcessing/Code/Source/SceneBuilder/SceneBuilderComponent.cpp
+7
View File
@@ -73,6 +73,13 @@ namespace SceneBuilder
required.emplace_back(AZ_CRC_CE("AssetImportRequestHandler"));
}
void BuilderPluginComponent::GetDependentServices(AZ::ComponentDescriptor::DependencyArrayType& services)
{
// Any components that can modify the analysis fingerprint via SceneBuilderDependencyRequests::AddFingerprintInfo must be activated first,
// so they contribute to the fingerprint calculated in BuilderPluginComponent::Activate().
services.emplace_back(AZ_CRC_CE("FingerprintModification"));
}
void BuilderPluginComponent::Reflect(AZ::ReflectContext* context)
{
AZ::SerializeContext* serializeContext = azrtti_cast<AZ::SerializeContext*>(context);
Gems/SceneProcessing/Code/Source/SceneBuilder/SceneBuilderComponent.h
+1
View File
@@ -29,6 +29,7 @@ namespace SceneBuilder
void Deactivate() override;
static void GetRequiredServices(AZ::ComponentDescriptor::DependencyArrayType& required);
static void GetDependentServices(AZ::ComponentDescriptor::DependencyArrayType& services);
private:
SceneBuilderWorker m_sceneBuilder;
Gems/SceneProcessing/Code/Source/SceneBuilder/SceneBuilderWorker.cpp
+2
View File
@@ -67,6 +67,8 @@ namespace SceneBuilder
context->EnumerateDerived(callback, azrtti_typeid<AZ::SceneAPI::SceneCore::GenerationComponent>(), azrtti_typeid<AZ::SceneAPI::SceneCore::GenerationComponent>());
context->EnumerateDerived(callback, azrtti_typeid<AZ::SceneAPI::SceneCore::LoadingComponent>(), azrtti_typeid<AZ::SceneAPI::SceneCore::LoadingComponent>());
}
AZ::SceneAPI::SceneBuilderDependencyBus::Broadcast(&AZ::SceneAPI::SceneBuilderDependencyRequests::AddFingerprintInfo, fragments);
for (const AZStd::string& element : fragments)
{
Registry/sceneassetimporter.setreg
+5
View File
@@ -10,6 +10,11 @@
".fbx",
".stl"
]
},
"MaterialConverter":
{
"Enable": true,
"DefaultMaterial": "Materials/Presets/PBR/default_grid.material"
}
}
}