* The Build Time Stamp of ShaderAsset And
ShaderVariantAsset Should Be Based On
GetTimeUTCMilliSecond()
GetTimeNowMicroseconds() is useful to measure time stamp differences.
GetTimeUTCMilliSecond() is for time stamps based on absolute clock/wall time.
* Updated DiffuseGlobalIllumination precompiled shaders
Co-authored-by: dmcdiar <dmcdiar@amazon.com>
Signed-off-by: galibzon <66021303+galibzon@users.noreply.github.com>
* Changed references to "InlineConstant" to "RootConstant".
Updated AZSLC to version 1.7.34 for mac, linux & windows
Signed-off-by: garrieta <garrieta@amazon.com>
* Update the FileIOAlias naming to make the cache, project root and engine
root paths more clear
The alias of `@root@`, `@assets@`, and `@projectplatformcache@` has been
collapsed to `@projectproductassets@`
The alias of `@devroot@` and `@engroot@` has been collapsed to
`@engroot@`
The alias of `@devassets@` and `@projectroot@` has been collapsed to
`@projectroot@`
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated use of devassets and devroot properties in python
Those properties now use projectroot and engroot
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updating the alias @engroot@ alias path comment in each platform specific LocalFileIO_*.cpp file
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed hardcoded size of 9 for the product asset alias.
The ResolvePath function now just appends the @projectproductassets@
alias with the input path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove duplicate @projectproductassets@ check in ProcessFileTreeRequest
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fix for typos in Hydra python test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated LocalFileIO::Copy call on Windows to use the Unicode aware CopyFileW API
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the AWSMetreicsGemAllocatorFixture to properly suppress asset
cache write errors for Test file creation.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed unneeded call to set the @projectproductasstes@ alias at the bottom of the AssetSeedManagerTest SetUp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Added a deprecated alias map to the FileIO System
When a deprecated alias is accessed, the FileIO System logs an AZ_Error and indicates the alias that should be used
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated python test scripts to use the projectroot binding
Retrieving the AutomatedTesting project path based on "<devroot>/AutomatedTesting" has been removed.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated references to devroot and devgame within the codebase
The GetAbsoluteDevGameFolderPath functions has been replaced with direct call to AZ::Utils::GetProjectPath
The GetAbsoluteDevRootFolderPath functions has been replaced with direct calls to AZ::Utils::GetEnginePath
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated <engroot>/AutomatedTesting references to projectroot
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced references that assumes the project path is <engroot>/AutomatedTesting with <projectroot> in the AutomatedTesting python test
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Correct casing in emfxworkspace file
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed newly added AppendPathParts function
Removed the Path constructors which accepts a PathIterable instance
The PathIterable isn't safe to return to a user of the Path class as it might be referencing temporary paths supplied via PathView arguments
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed unused parameter warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Undid change to the LexicallyProximate function to set the path result to the base path.
It needs to return the *this path if the pathResult is empty
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Moved the LocalFileIO ConvertToAbsolutePath implementations to AZ::Utils
Fixed the ConvertToAbsolutePath implementation for Unix platforms to use a buffer that is size PATH_MAX(4096 on all our supported Unix platforms).
Because the buffer before was AZ::IO::MaxPathLength which as a size of 1024, this was resulting in the Unix `realpath` function smashing the stack when it converts a path to an absolute path that is longer than 1024 characters
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the EditorCore.Tests to attach the AZ Environment to the EditorCore shared library that is statically loaded on launch.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed for DeprecatedAlaisesKeyVisitor Visit function causing the non string_view overloads being hidden causing a hidden base class function warning
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Changed the AWSMetricsGemMock to use a temporary for writing test assets
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the LocalFileIO::ResolvePath function to use HasRootPath to determine if a path starts with a root path such as "D:", "D:/" or "/"
IsAbsolute was not the corect check as the path of "D:" is a relative
path.
To be absolute according to the Windows the path must have a root
directory. i.e "D:/" or "D:\"
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removed absolute path comment from LocalFile_UnixLike.cpp and LocalFile_Android.cpp FindFiles implementations
Updated the ConvertToAlias to supply an AZ::IO::FixedMaxPath
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced usage of the @projectproductassets@ alias with @engroot@ when referring to the LmbrCentral source folder in the CopyDependencyBuilderTest and the SeedBuilderTests
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated the ScriptCanvas Upgrade Tool to output backed up files to the
Project User Path instead of the engine root
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed whitespacing issues in Application.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Remove unnecessary creation of a FixedMaxPath in the UpgradeTool.cpp
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Modified testSeedFolder variable in the SeedBuilderTests to use the
@engroot@ alias instead of @projectproductassets@/.. alias when
referring to the LmbrCentral Gem source path
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Updated references to the Project Asset Cache in the PythonTests.
Those tests no longer use the logic `azlmbr.paths.projectroot / "Cache" / "pc"` to retrieve a path to the cache root but instead the `azlmbr.paths.projectproductassets` constant
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Fixed the FileIO Deprecated Alias test on Windows
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Removing @projectsourceassets@ alias, as it is only used once.
Updated the PhysX EditorSystemComponent.cpp to query the ProjectPath
form the SettingsRegistry.
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Replaced @projectproductassets@ alias with @products@
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Rollback changes to the PhysX EditorSystemComponent.cpp
The changes to use the ProjectPath from the SettingsRegistry has been implemented in PR #4497
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Release build fix for Windows
* Release build fix for Android
* Release build fix for Windows
* Release build fix for Android
* Release build fix for Linux
* Release build fix for Mac
* Release build fix for iOS
Signed-off-by: Steve Pham <spham@amazon.com>
* Final update copyright headers to reference license files at the repo root
Signed-off-by: spham <spham@amazon.com>
* Fix copyright validator unit tests to support the stale O3DE header scenario
Signed-off-by: spham <spham@amazon.com>
Cherrypicked b7f6b57e16 from stabilization branch because I knew it would have conflicts I needed to resolve.
See https://github.com/aws-lumberyard/o3de/pull/1462
This removes mostly benign (but noisy) messages about "Failed to find builder dependency".
ATOM-15136 Builder dependency errors reported in mainline
ATOM-15134 Replace GetPossibleDepenencyPaths Approach with Source Dependencies
SrgLayoutBuilder.cpp conflicted because it had been removed on development. The changes in this file are no longer relevant.
ShaderVariantAssetBuilder.cpp conflicted only because there were formatting changes inside the AddShaderAssetJobDependency() function.
This removes mostly benign (but noisy) messages about "Failed to find builder dependency".
Changed AssetUtils::GetPossibleDepenencyPaths to return all possible source paths, rather than stopping when one is found. This function is now used to report a list of all possible source dependencies, so that CreateJobs will get called by the AP whenever a file shows up at one of those locations. If a file was missing before and then appears, this will cause the builders to wake up and add the appropriate job dependencies on the new files. In ShaderVariantAssetBuilder and MaterialBuilder, we now use GetPossibleDepenencyPaths to report source dependencies rather than job dependencies. We only report a job dependency when the actual source file has been identified. This should all now be consistent with the intended design of the AP's dependency systems (the prior approach was a hack based on misunderstanding of what source dependencies are).
SrgLayoutBuilder's change is a bit tricky. The above changes did not fix all of the "Failed to find builder dependency" messages because AzslBuilder sometimes skips particular files in CreateJobs. When this happens, it is invalid to report an AzslBuilder job dependency on that file. So I copied the same conditional code that is used to skip an azsli file in CreateJobs, and used that to skip AddAzslBuilderJobDependency() in SrgLayoutBuilder as well.
With all these changes combined, I do think we've solved the issue where jobs fail to evict outdated jobs, as described in ATOM-15134. However, we are not yet seeing the iteration time improvements we were hoping for. Before these changes I was seeing roughly a 0.5 minute delay for the initial change, and a 2 minute delay for a subsequent change. With these changes it's more like 0.5 mibutes and 1.5 minutes. It appears that the AP scan is being starved by all the AssetBuilder processing going on, and perhaps IO contention. I suspect that this will be greatly improved on the development branch where we no longer have AzslBuilder and SrgLayoutBuilder slowing things down.
ATOM-15136 Builder dependency errors reported in mainline
ATOM-15134 Replace GetPossibleDepenencyPaths Approach with Source Dependencies
* [ATOM-15472] Shader Build Pipeline: Remove Deprecated Files And Functions That
Predate The Shader Supervariants
These are the essential impactful changes as a result of deprecating the
ShaderResourceGroupAsset.
* Addressed feedback by @moudgils. Better comments in header files.
* More updates related with deprecation of ShaderResourceGroupAsset
* Deleted the temporary version 2 classes.
* Updated version of the shader asset builders.
* Updated version of all the shader related classes impacted
by the Supervariant concept and deprecation of ShaderResourceGroupAsset
* Changes to *.pass and DGI, Reflections and RayTracing.
* changes to material related assets
* changes to core lights
* Changes to auxgeom/dynamic draw.
* changes to decals, lyshine, imguipass
* changes to RPI Pass classes
* Shader for SceneSrg, ViewSrg and ForwardPass Srgs.
* changes to mesh, skinned mesh, Morphtarget.
* Fixes to RayTracingPass.cpp & now allow empty srg in shaders.
* Updated Atom_RPI.Tests
* Simplified InstanceDatabase by removing AddHandler
------------------------------------------------------------------------------------
* Updated DiffuseGI precompiled shaders.
Added RayTracingSceneSrg and RayTracingMaterialSrg shader asset.
Updated ShaderAssetCreator::Clone to handle the supervariant when processing root variants.
Co-authored-by: Doug McDiarmid <dmcdiar@amazon.com>
------------------------------------------------------------------------------------
* Changed semantics for some PassSrg to SRG_PerPass_WithFallback.
AuxGeom/FixedShapeProcessor.cpp requires SRG_PerDraw on ObjectSrg.
Removed names of SceneSrg and ViewSrg from RPISystemDescriptor.cpp
* Moved ShaderLib/Atom/Features/DummyEntryFunctions.azsli
To Gems/Atom/RPI/Assets/ShaderLib/Atom/RPI/DummyEntryFunctions.azsli
Removed redundant checking for finalization in
ShaderResourceGroupLayout.cpp
* Fixed race condition bug for Shader::FindOrCreate.
InstanceDatabase<>::CreateInstance() needs to be atomic
for instance creation and initialization.
Added optional InstanceHandler::CreateFunctionWithParams to accomodate
to the needs of Instances that need more than an asset reference
to be able to be created an initialzed.
Removed ShaderResourceGroup::FindOrCreate() only ::Create is available
now.
* Renamed scene_and_view_srgs.* as SceneAndViewSrgs.*
Changed GetAzslFileOfOrigin for GetUniqueId
* Fixed unit tests.
* Reverted the serialization name of m_uniqueId back to
"m_azslFileOfOrigin" so precompiled shaders don't fail
in layout comparison.
* Fixed AtomCore.Tests
Removed non-applicable test. InstanceDatabase.AddHandler() is not
available anymore.
* The Null rhi is re-enabled for shader compilation.
Signed-off-by: garrieta <garrieta@amazon.com>
* [ATOM-15276] Shader Build Pipeline: Add Shader Supervariant System.
Added ShaderAssetBuilder2 & ShaderVariantAssetBuilder2.
Added ShaderAsset2, ShaderVariantAsset2.
Eventually they will be the only builders. AzslBuilder &
SrgLayoutBuilder will be removed.
ShaderResourceGroupAsset will be removed. ShaderAssetBuilder &
ShaderVariantAssetBuilder will be replaced.
Signed-off-by: garrieta <garrieta@amazon.com>
ShaderVariantAssetBuilder::ProcessJob was not handling the should-exit-early flag, described at ValidateShaderVariantListLocation(). It was assuming that this flag would be set only during error conditions, which is not the case. I just had to rearrange the logic to handle deferred errors and should-exit-early separately.
Testing:
The problematic job now finishes successfully. I also edited the shadervariantlist file to give it a bad path to the shader and confirmed that we still get the "Error during CreateJobs" message.
galibzon
lumberyard-employee-dm
srikappa-amzn
srikappa-amzn
Steve Pham
santorac
chcurran
Chris Galvan
Chris Burel
Gene Walters
Chris Santora
garrieta
hershey5045
alexpete