Commit Graph

351 Commits (2d1ed48fea6e923c62a48ec12fcdf65fe3c6ea2f)

Author SHA1 Message Date
Cynthia Lin c1a0b5c686
performance benchmarks: Aggregate and report CPU frame times. (#2939)
* Add CaptureCpuFrameTime method to ProfilingCaptureSystemComponent for monitoring CPU performance.

Signed-off-by: Cynthia Lin <cyntlin@amazon.com>

* ly_test_tools: Refactor benchmark data aggregator in preparation for CPU frame times.

Signed-off-by: Cynthia Lin <cyntlin@amazon.com>

* performance benchmarks: Aggregate and report CPU frame times based on JSON data.

Signed-off-by: Cynthia Lin <cyntlin@amazon.com>

* AutomatedTesting: Capture CPU frame time in AtomFeatureIntegrationBenchmark.

Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
4 years ago
Sean Sweeney 462e4d5e5e
Merge pull request #2368 from aws-lumberyard-dev/smoke_test_fix
Moving load level test from sandbox to smoke with fixes
4 years ago
Junbo Liang c7397fe4a2
Fix AWSTests.periodic failing on Linux due to a File Not Found error
Signed-off-by: junbo <junbo@amazon.com>
4 years ago
carlitosan aca59c6b88
Merge pull request #2946 from aws-lumberyard-dev/carlitosan/development
Fix for failing periodic test due to editor properties name change
4 years ago
Nicholas Lawson 12579e5d2e
Fixes incorrect build dependencies for tests (#2959)
The serialize context tools were not being auto built by CI because of
this.
Added missing dependency on AzTestRunner.

Also moved the AssetBundlerBatch dependency directly in there
No reason to add it to additional_dependnecies, since its only
used for conditional inclusions.

Signed-off-by: lawsonamzn <70027408+lawsonamzn@users.noreply.github.com>
4 years ago
chcurran 522e284b3d fix for failing periodic test
Signed-off-by: chcurran <82187351+carlitosan@users.noreply.github.com>
4 years ago
amzn-sean 6d2765ef42
moved default location of surfacetypemateriallibrary.physmaterial (#2786)
from 'project root' to 'project root/Assets/Physics'

The functionality of creating / using the default physmaterial file has only change in related to the file location, other functionality is unchanged.

The following situations can occur:
    This will not affect have any project that uses a custom physmaterial file.
    This will not affect have any project that uses the default from the old location, as the configuration will still point there.
    New projects created will get the default physmaterial file at the new location.
    A Project that fails to load (or deletes) the selected physmaterial file, will get the default physmaterial file at the new location (this happens only on startup of the editor).

Issue: #2765

Signed-off-by: amzn-sean 75276488+amzn-sean@users.noreply.github.com
4 years ago
jromnoa a0f3379999
Adds Light component tests (non-GPU portion) to AutomatedTesting from AtomTest (#2758)
* Fixed Vegetation Layer Spawner documentation link.

Signed-off-by: Chris Galvan <chgalvan@amazon.com>
Signed-off-by: jromnoa <jromnoa@amazon.com>

* add remaining non-GPU test portions for Light component test

Signed-off-by: jromnoa <jromnoa@amazon.com>

* make non-GPU light component test more robust

Signed-off-by: jromnoa <jromnoa@amazon.com>

* remove redundant logging, convert LIGHT_TYPES from list to dict, remove redundant f-string

Signed-off-by: jromnoa <jromnoa@amazon.com>

Co-authored-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
sharmajs-amzn 69bde80de3
Nighly build test fixes (#2727)
Signed-off-by: sharmajs-amzn <82233357+sharmajs-amzn@users.noreply.github.com>
4 years ago
evanchia 65704110ad removing pytest skips on smoke tests
Signed-off-by: evanchia <evanchia@amazon.com>
4 years ago
AMZN-stankowi 9ee9730294
Automated test for scene files with and without python scripts running python incorrectly (#2373)
* Cleared m_scriptFilename between scene files.
This fixes a bug where a Python script file would be run on a scene file
that didn't have a script file set.
Added a general case version to SceneBuilderWorker.cpp, to make it easy
to mark all scene files as dirty.
Automated tests for this will come in a separate pull request.

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Work in progress automated tests

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Python test done

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Sorted jobs work now. This may sort too aggressively, I'll remove the additional sorting after some testing.

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Cleaned up test

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Fixed stray '

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Removed temp code from test

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Command line help options for AP
Removed job sorting that wasn't actually sorting jobs

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>

* Changed constant variable names to match coding standards

Signed-off-by: stankowi <4838196+AMZN-stankowi@users.noreply.github.com>
4 years ago
Cynthia Lin 567b4a7f28
[ATOM-16016] Add initial level for AtomFeatureIntegrationBenchmark to AutomatedTesting. (#2405)
* [ATOM-16016] Add initial level for AtomFeatureIntegrationBenchmark to AutomatedTesting.

Signed-off-by: Cynthia Lin <cyntlin@amazon.com>

* [ATOM-16016] Add new AtomFeatureIntegrationBenchmark to AutomatedTesting.

Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
4 years ago
evanchia 05728920f7 Replaced with imported functions for smoke tests
Signed-off-by: evanchia <evanchia@amazon.com>
4 years ago
evanchia d08cbd2c33 Moving load level test from sandbox to smoke with fixes
Signed-off-by: evanchia <evanchia@amazon.com>
4 years ago
AMZN-AlexOteiza b815c203da
(Continuation) Implemented automation paralellization & standarization (#1718)
Engine improvements/fixes

Fixed behavior that made the editor automated test to be sometimes stuck if lost the focus is lost.
Added support for specifying multiple tests to in batch to the editor, this is achieved by passing --runpythontest with the tests separated by ';'
Added new cmdline argument --project-user-path for overriding the user path. This allows to have multiple editors running writing logs and crash logs in different locations.
Moved responsability of exiting after a test finishes/passes out of ExecuteByFilenameAsTest, callers will use the bool return to know if the test passed.
Editor test batch and parallelization implementation:

Now the external python portion of the editor tests will be specified via test specs which will generate the test. Requiring no code. This is almost a data-driven approach.
Tests can be specified as single tests, parallel, batchable or batchable+parallel
Command line arguments for pytest to override the maximum number of editors, disable parallelization or batching.
Automated tests for testing this new editor testing utility

Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>

Co-authored-by: Garcia Ruiz <aljanru@amazon.co.uk>
4 years ago
moraaar fbb9ed6618
Merge pull request #2333 from aws-lumberyard-dev/Move_Undo_Redo_Test_Sandbox
Moving test_C15425929_Undo_Redo flaky test to sandbox test suite
4 years ago
moraaar caa4e30b38 Marking test_C15425929_Undo_Redo as xfail.
Signed-off-by: moraaar <moraaar@amazon.com>
4 years ago
moraaar a918b3d31a Moving test_C15425929_Undo_Redo flaky test to sandbox test suite
Signed-off-by: moraaar <moraaar@amazon.com>
4 years ago
jckand-amzn dccab7df54 Merge branch 'development' of https://github.com/o3de/o3de into jckand/DynVegXfailFixes 4 years ago
jckand-amzn e0c94dcf45 Updating xfail reason for tests currently failing due to instance spawning with null renderer
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn 6fcf9aa9e1 Updated component configuration for new uniform scale in SlopeAlignmentModifier tests
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
Junbo Liang a32a521a80
Update automation tests for the AWS gems to reduce the run time (#2148)
Simplify the automation tests to reduce the time they cost on Jenkins
4 years ago
jckand-amzn 2173693462 Updating component configuration and expected instance counts for SlopeFilter test
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn d9874431e2 Moving test back to xfail due to remaining issues with instance planting on meshes while null renderer is enabled
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn 605c17072e Updating test suite for Position Modifier test
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn b7cb9fa5e3 Re-configured some component properties to account for uniform scaling in PositionModifier tests
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
amzn-mike fc75dd5fda
Fix python fbx tests (#2171)
* Update fbx test dbgsg files

Remove rc products from fbx test expected output
Update warning counts for 2 of the tests - these are due to missing material properties
Add SkipAtomOutput setting to disable atom material and mesh processing in scene as these pull in a lot of external dependencies to Process
Removed BlendShapeData and MeshData debug output of positions/normals/faces
Updated ManifestImportRequestHandler to remove warning about Joining 2 absolute paths
Updated AssImpMaterialImporter to use new GenerateRelativeSourcePath API to fix issue where relative path generation failed with tmp project directories

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add comment for setting

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Add override dbgsg file to python test

The mesh optimizer specifically avoids creating optimized nodes for nodes that are not selected, resulting in a different scene graph that requires a different dbgsg file to compare with

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>

* Replace hardcoded passing of specific registry setting to instead pass all registry settings to AssetBuilder

Signed-off-by: amzn-mike <80125227+amzn-mike@users.noreply.github.com>
4 years ago
Steve Pham 38261d0800
Shorten copyright headers by splitting into 2 lines (#2213)
* Updated all copyright headers to split the longer original copyright line into 2 shorter lines

Signed-off-by: Steve Pham <spham@amazon.com>
4 years ago
jckand-amzn ed788090d3
Adding xfail mark to test_LandscapeCanvas_GraphClosed_OnEntityDelete due to #2201 (#2208)
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn 4c8befe759
Removing obsolete Vegetation Debugger CVar test. CVars now function without the component present (#2200)
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
Esteban Papp 900b82cb36 missing dependencies
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
jckand-amzn 9fcc9f7b41 Merge branch 'development' of https://github.com/o3de/o3de into jckand/DynVegXfailFixes 4 years ago
jromnoa 900f37644e
fix Atom GPU test by enabling autotest_mode (#2117)
Signed-off-by: jromnoa <jromnoa@amazon.com>
4 years ago
Chris Galvan df9358c492
Merge pull request #2109 from aws-lumberyard-dev/cgalvan/RemoveDeprecatedGraphicsMenuItems
Removed deprecated graphics settings/performance menu options. Updated automated test accordingly.
4 years ago
Chris Galvan f278bbcd65
Merge pull request #2094 from aws-lumberyard-dev/cgalvan/gitflow_210712
Merged stabilization/2106 to development
4 years ago
Chris Galvan 2fc1062802 Removed deprecated graphics settings/performance menu options. Updated automated test accordingly.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
jromnoa 6a6aa69762
increase test timeout, if it still times out I need to get RDP access to the node to fix this since a pop-up or alert is halting+timing out automation (#2041)
Signed-off-by: jromnoa <jromnoa@amazon.com>
4 years ago
Chris Galvan e924ed0b86 Merged stabilization/2106 to development; Resolved merge conflicts
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
jckand-amzn ceae858f09 Updating to use of psutil directly
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn 35a60cdf1a Removing unnecessary Mesh component refreshes and updating calls to scale meshes
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn ea99ed11a3 Updating mesh surface creation, filtering, and expected instance counts
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jckand-amzn 9f5e91be88 Updating utility path for test import, and updating linked issue to disabled test
Signed-off-by: jckand-amzn <jckand@amazon.com>
4 years ago
jromnoa a40c979baa move test to sandbox instead of disabling it
Signed-off-by: jromnoa <jromnoa@amazon.com>
4 years ago
Junbo Liang ededb85da2
Merge AWSCore automation tests to the development branch (#2011)
* Create AWS Core automation for the interacting with AWS resources via the AWS Core gem.
Signed-off-by: junbo <junbo@amazon.com>
Co-authored-by: clujames <clujames@amazon.com>
4 years ago
jckand-amzn 3679530562 Merge branch 'development' of https://github.com/o3de/o3de into jckand/DynVegXfailFixes 4 years ago
jromnoa 2555c88fe7 this test needs to be disabled in main, it was supposed to be merged to development before break but got merged to stabilization and then main
Signed-off-by: jromnoa <jromnoa@amazon.com>
4 years ago
Chris Galvan 3b1873b045 Merged stabilization/2106 -> development (resolved merge conflicts).
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
5 years ago
jckand-amzn c64f3fa5f9 Merge branch 'development' of https://github.com/o3de/o3de into jckand/DynVegXfailFixes 5 years ago
jckand-amzn ea0e38aaf4 Updating expected View menu options for test_Menus_ViewMenuOptions_Work
Signed-off-by: jckand-amzn <jckand@amazon.com>
5 years ago
jckand-amzn 0a68bbdf7c Updating camera position and null renderer args for launcher test
Signed-off-by: jckand-amzn <jckand@amazon.com>
5 years ago