* adding more time via ap_max_activate_time so that the Debug
version of AP-GUI can load all the builder DLLs
Signed-off-by: jackalbe <23512001+jackalbe@users.noreply.github.com>
* Fixed warnings of unused marks, renamed ctest_pytest.ini to pytest.ini to better consistency on runs
* Fixed some test suites to run propertly
* Fix missing arguments
* Fixed missing cmakelists and renamed missing file
* Temp disable editor_testing_tests as timeout in jenkins
* Remove many unused variables and unused setting files
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Remove a few more dead config vars
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* fix android test_ConfigureSettings_DefaultValues_SetsValues
Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
* Commit
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
* Removed xfail
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
* Fix for ap not closing for optimized tests
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
* renamed tests to _Optimized
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
* Added xfail so the test will run in jenkins without affecting outcomes
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
Co-authored-by: Garcia Ruiz <aljanru@amazon.co.uk>
* Add CaptureCpuFrameTime method to ProfilingCaptureSystemComponent for monitoring CPU performance.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* ly_test_tools: Refactor benchmark data aggregator in preparation for CPU frame times.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* performance benchmarks: Aggregate and report CPU frame times based on JSON data.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* AutomatedTesting: Capture CPU frame time in AtomFeatureIntegrationBenchmark.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* fix: Correct typo in profiling capture system.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
* Move benchmark data aggregation from ASV into ly_test_tools module.
Signed-off-by: Cynthia Lin <cyntlin@amazon.com>
Engine improvements/fixes
Fixed behavior that made the editor automated test to be sometimes stuck if lost the focus is lost.
Added support for specifying multiple tests to in batch to the editor, this is achieved by passing --runpythontest with the tests separated by ';'
Added new cmdline argument --project-user-path for overriding the user path. This allows to have multiple editors running writing logs and crash logs in different locations.
Moved responsability of exiting after a test finishes/passes out of ExecuteByFilenameAsTest, callers will use the bool return to know if the test passed.
Editor test batch and parallelization implementation:
Now the external python portion of the editor tests will be specified via test specs which will generate the test. Requiring no code. This is almost a data-driven approach.
Tests can be specified as single tests, parallel, batchable or batchable+parallel
Command line arguments for pytest to override the maximum number of editors, disable parallelization or batching.
Automated tests for testing this new editor testing utility
Signed-off-by: Garcia Ruiz <aljanru@amazon.co.uk>
Co-authored-by: Garcia Ruiz <aljanru@amazon.co.uk>
Allen Jackson
Mike Balfour
Sean Sweeney
evanchia
AMZN-AlexOteiza
Artur K
jckand-amzn
Esteban Papp
Cynthia Lin
sharmajs-amzn
Steve Pham
Royal OBrien