Commit Graph

423 Commits

Author SHA1 Message Date
jromnoa 5c4ce8cd06 Upload test screenshots to s3 on test failure. (#3815)
* add s3 upload on screenshot test failure, should only apply to the test_gpu_profile_vs2019 job on nightly runs

* adds support for zipping screenshot files up prior to uploading to the s3 bucket and also adds the ACL extra arg to the upload_to_s3.py execution

* remove unused json import

* remove regex to use .endswith() instead and rename variables to be more clear (PR feedback)

* rename create_zip_archive to create_screenshots_archive

Signed-off-by: jromnoa <jromnoa@amazon.com>
2021-09-08 06:11:44 -05:00
Junbo Liang 8edcc504bd Add AWSI automation tests to a new test suite and pipe (#3913)
Signed-off-by: junbo <junbo@amazon.com>
2021-09-07 13:02:13 -07:00
bosnichd 82659f24e9 Add support for a custom path root separator using a trait. (#3678)
* - Add support for a custom path root separator using a trait.
- Ensure to set both FilePathKey_ProjectUserPath and FilePathKey_ProjectLogPath to a writeable storage location on non-host platforms.

Signed-off-by: bosnichd <bosnichd@amazon.com>

* Remove az_trait_validator.py, as our PAL guidance no longer dictates that all traits must be defined for all platforms.

Signed-off-by: bosnichd <bosnichd@amazon.com>

* Updated based on review feedback.

Signed-off-by: bosnichd <bosnichd@amazon.com>
2021-09-01 10:24:29 -06:00
Shirang Jia c1c6fa3615 Auto-cut Github issue on build failures from main/development (#3730)
* Send SNS topic on build failure

* send build failure root cause data to SNS topic that triggeres lambda function to automatically create Github issues

* Change SNS topic subject to Build failure and add build url to the message

* Change SNS topic name
2021-08-31 17:37:25 -07:00
Sean Sweeney 873654a9fc Merge pull request #3116 from aws-lumberyard-dev/benchmark_metrics_jenkins
Adds the MARS benchmark script in AR
2021-08-31 15:58:20 -07:00
Mike Chang 2bf83ad9fb Ubuntu 20.04 package update for clang-12 (#3727)
* Updated CMake, added libxkbcommon-dev and clang-12

* Add Corretto JDK 11 to packages for Ubuntu 20 and 18

* Add mountpoint creation command

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-31 13:47:57 -07:00
Mike Chang 6e3b03dfaa Install and configure Ubuntu nodes with Nvidia drivers and NiceDCV (#3728)
These scripts will configure build and test Ubuntu 18/20 nodes with:

Nvidia gaming drivers for GPU instances (g4dn)
lightdm for Ubuntu 18 and gdm3 for Ubuntu 20 (wayland is disabled here, but should have xcb capabilities)
NiceDCV for remote console access

Note: Does not include user setup and firewall configurations

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-31 09:47:57 -07:00
evanchia 5e4ec8c421 adding missing param to Jenkins benchmark step
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-27 10:14:22 -07:00
lumberyard-employee-dm b53182fdb6 Fixed invalid variable access in register.py (#3433)
* Fixed invalid variable access in register.py

The register_restricted_path and register_template_path methods did not specify the parameter for project path

Updated register_o3de_object_path function to pass in absolute paths to the engine or project in order to make relative paths to the registered o3de object to it.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed register_*_path resolve calls in register.py

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-08-25 13:29:24 -05:00
Esteban Papp f94043dc83 Adds functions so targets can define install directories/files 2021-08-24 15:12:02 -07:00
Chris Burel 86770deefa Add linux equivalents for all the places that enumerate platforms (#3325)
* Add linux equivalents for all the places that enumerate platforms

Signed-off-by: Chris Burel <burelc@amazon.com>

* Fix the AssetFileInfoListComparisonTest fixture to not hardcode the PC platform

Signed-off-by: Chris Burel <burelc@amazon.com>
2021-08-24 14:10:59 -07:00
Shirang Jia ce85bb3b7d Add awsi deployment as a non-blocking step in nightly pipeline (#3397)
* Add awsi deployment as a non-blocking step in nightly pipeline

* Make step fail if it's not a non-blocking step
2021-08-24 10:43:17 -07:00
Esteban Papp dc8b48601c PR comments/improvements
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-24 10:36:31 -07:00
Esteban Papp b609d5e2cb supporting multiple directories
automatically excluding obvious things
cleanup and verification

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-23 19:41:57 -07:00
Mike Chang a92684c0b8 Jenkinsfile LFS fix for iOS/Mac pulls (#3396)
* Add git lfs install and pull into Jenkins Git pull stage

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-23 16:07:02 -07:00
Esteban Papp 8bc9ed3d01 removing some rad leftovers (#3366)
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-20 13:39:51 -07:00
hultonha a529256274 Add extra logging for CHANGE_ID and COMMIT_ID - TIAF
Merge pull request #3328 from aws-lumberyard-dev/jenkins_commit_hash_log
2021-08-20 10:45:41 +01:00
Junbo Liang c93a18ab82 Update the AWS automation tests to use existing CloudFormation stacks (#3092) 2021-08-19 14:05:25 -07:00
John dcde3f3e36 Add extra logging for CHANGE_ID and COMMIT_ID.
Signed-off-by: John <jonawals@amazon.com>
2021-08-19 18:18:45 +01:00
Esteban Papp a1b59e5e5e install reorg/cleanup
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-19 08:51:29 -07:00
Shirang Jia e48798fdbf Add missing {} for string interpolation (#3291) 2021-08-18 10:02:24 -07:00
Shirang Jia 3e51240a05 Allow customized Jenkins parameters for different pipelines and add aws integration deployment pipeline (#3248)
Allow customized Jenkins parameters for different pipelines so that we can define different Jenkins parameters for a new pipeline and doesn't affect AR build parameters.

Add aws integration deployment pipeline.
2021-08-17 15:25:32 -07:00
jonawals 252c268ffd Fix corrupt merge. (#3142)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
2021-08-16 16:09:36 +01:00
jonawals 5925bd22f6 Further subdivide TIAF data by suite. (#3128)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
2021-08-16 14:15:02 +01:00
evanchia 78d34e6960 Adds the MARS benchmark script in AR
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-13 14:10:43 -07:00
John a1d855a920 Invert corner case flag for readability.
Signed-off-by: John <jonawals@amazon.com>
2021-08-13 10:03:09 +01:00
John 4dd39c3929 Address commit re-run corner case.
Signed-off-by: John <jonawals@amazon.com>
2021-08-13 09:55:06 +01:00
jonawals ff659fbbb6 Tiaf bucket top level fix (#3085)
* Revert to regular run when invalid commits used.

* Cirrect s3 logging of last commit hash storage

* Add s3 top level url and build number script params.

* Use existing REPOSITORY_NAME env var.

Signed-off-by: John <jonawals@amazon.com>
2021-08-12 12:48:23 -07:00
John 199273b880 Remove positional argument from change list generaiton.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 12:00:51 +01:00
John 9601db55aa Add sanity checking to last commit hash.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 11:19:48 +01:00
John 268a7f547e Add additional logging to s3 storage.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 11:19:27 +01:00
jonawals f4e8ecd35e TIAF script fixes (#3044)
* Fix typo in MARS document key.

Signed-off-by: John <jonawals@amazon.com>

* Add diff support for commits on different branches.

Signed-off-by: John <jonawals@amazon.com>

* Restore archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Correctly handle commit diffs for PR builds.

Signed-off-by: John <jonawals@amazon.com>

* Comment out archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Fix python typo

Signed-off-by: John <jonawals@amazon.com>

* Remove comment pertaining to s3 bucket.

Signed-off-by: John <jonawals@amazon.com>

* Fix test selection efficiency for MARS.

Signed-off-by: John <jonawals@amazon.com>
2021-08-12 10:21:33 +01:00
Chris Galvan 0f8ea17235 Merge pull request #3051 from aws-lumberyard-dev/cgalvan/gitflow_210811_dev
Merged stabilization/2106 to development
2021-08-11 17:41:29 -05:00
Junbo Liang c6714595b1 Define a new Jenkins build script to deploy AWS resources (#2983)
* Define a new Jenkins job to deploy AWS resources

Signed-off-by: junbo <junbo@amazon.com>

* Address PR comments

Signed-off-by: junbo <junbo@amazon.com>

* Add error checking for all the calls

Signed-off-by: junbo <junbo@amazon.com>

* Remove the weekly-build-metrics tag

Signed-off-by: junbo <junbo@amazon.com>

* Parameterize ARN/Region and move it to a Jenkins envvar.

Signed-off-by: junbo <junbo@amazon.com>

* Revert the changes for build_config.json

Signed-off-by: junbo <junbo@amazon.com>
2021-08-11 13:59:29 -07:00
Chris Galvan 5b0046ce78 Resolved conflicts in merge from stabilization/2106 -> development.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
2021-08-11 15:33:50 -05:00
jonawals a7a0e8d2ef Fix for PathLib unlink wrong version parameter. (#3020)
* Fix for PathLib unlink wrong version parameter.

* Fix dst target for TIAF.

* Remove TIAF historic data arhciving on s3 buckets

* Change permissions for TIAF bucket upload

Signed-off-by: John <jonawals@amazon.com>
2021-08-10 18:43:44 -07:00
John 5dbbb93a06 Fix path for subprocess
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:49:19 +01:00
John e3221224be Add traceback to tiaf exception handler.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:34:27 +01:00
John 2be043ca96 Fix None comparisons in TIAF scripts.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:13:19 +01:00
John c2a2d1c5c7 Add changes from TIF/Feature branch.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 14:20:46 +01:00
lumberyard-employee-dm 9afbd07a88 Added new feature to the register command to auto detect manifest file based on input path (#2967)
* Added new feature to the register command to auto detect the correct
o3de manifest file to write to based on the supplied registration path

Updated the create-gem command to register the gem on creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added new feature to the register command to auto detect the correct
o3de manifest file to write to based on the supplied registration path

Updated the create-gem command to register the gem on creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix incorrect variable reference in register_project_path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed o3de python package unit test
The enable_gem, register and engine template test have been updated to
account for the logic to register a gem after creation

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a --no-register arg to engine-template.py
This allows registration of gems/projects using the create-project and
create-gem commands to be skipped

Prevented registration of a gems and projects in the engine_template
command test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Wrapped first parameter to find_ancestor_dir_containing_file with
pathlib.PurePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-08-09 16:58:19 -05:00
Sean Sweeney 54b93d7c25 Merge pull request #2885 from aws-lumberyard-dev/test_metrics_env_var_fix
Escape character in Jenkins for test metrics
2021-08-09 14:03:02 -07:00
Scott Romero 0999a7f6e4 [development] s3 upload script now replicates directory structure relative to search root (#2947)
Previously the script was using the full local path as the artifact key when uploading to s3. This was causing the installer artifacts to be uploaded incorrectly and non-functional for normal use.

Signed-off-by: AMZN-ScottR 24445312+AMZN-ScottR@users.noreply.github.com
2021-08-09 08:48:51 -07:00
Shirang Jia 4f9382e8c6 Include build failure root cause in email notification (#2491) (#2888)
Signed-off-by: shiranj <shiranj@amazon.com>
2021-08-06 12:52:04 -07:00
AMZN-nggieber 2c7f6f9742 Adds Links to Gem Directory and Documentation for Gems (#2922)
* Changed blue text to white that is not meant as a link, made 'View in Director' link work for gems in the inspector, added parsing for gem documentation link

Signed-off-by: nggieber <nggieber@amazon.com>

* Added documentation links for gems, changed markup for urls in summaries and requirements so they are clickable

Signed-off-by: nggieber <nggieber@amazon.com>

* Fixed a couple of the documentation links

Signed-off-by: nggieber <nggieber@amazon.com>

* Added documentation url to edit gem properties script and updated unit tests

Signed-off-by: nggieber <nggieber@amazon.com>
2021-08-06 10:19:35 -05:00
evanchia 3a76f2e6ce Fixing incorrect string replacement in Jenkinsfile
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-05 17:41:06 -07:00
lumberyard-employee-dm a65b326558 Fixed the enable-gem command unit test (#2880)
* Fixed the enable-gem command unit test

The enable-gem unit test would fail if there wasn't an
o3de_manifest.json file in the users $HOME/.o3de directory

The change now is to patch the call to load the o3de manifest

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Adjustments to the enable-gem command unit test to pass on Linux

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-08-05 16:58:57 -05:00
evanchia 574bef0cd8 Escape character in Jenkins for test metrics
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-05 10:19:15 -07:00
Shirang Jia 4b817a6483 Include build failure root cause in email notification (#2491)
Signed-off-by: shiranj <shiranj@amazon.com>
2021-08-04 17:57:47 -07:00
lumberyard-employee-dm 21ca3a4aea The enable-gem command registers gem with project if not registered (#2817)
* The enable_gems command now registers the gem with the project if only registered with o3de_manifest.json

Updated the `enable_gems` command to register the gem with the project if the gem is not registered with either the project or the engine being used.
This allows the gem to be added to the build system if it wasn't registered before.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Adding quoting around the invocation of the OpenProjectManager command

The --project-path parameter now is able to pass in a path with spaces
to the invocation of the Project Manager.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-08-04 13:13:28 -05:00