Commit Graph

430 Commits

Author SHA1 Message Date
Brian Herrera aaf292f8cd Merge pull request #4250 from aws-lumberyard-dev/pipelines/retry-pipeline
Retry pipeline stages when a node goes offline
2021-09-22 09:11:16 -07:00
brianherrera eaddd4eee7 Throw exception for build stage
This is required so Jenkins will properly mark the build failed on valid script errors.

Signed-off-by: brianherrera <briher@amazon.com>
2021-09-21 16:05:25 -07:00
Mike Chang 39a4e05492 Change installer domain to o3debinaries.org (#4244)
* Change installer domain to o3debinaries.org

* Removed 3p license parameter

Signed-off-by: Mike Chang <changml@amazon.com>
2021-09-21 15:24:14 -07:00
brianherrera d44cc844d1 Retry pipeline stages when a node goes offline
This change is required to support spot instances in the AR pipeline.

- When a node goes offline the job is retried on a new node
- Valid script errors are handled properly and the status is updated in Jenkins

Signed-off-by: brianherrera <briher@amazon.com>
2021-09-21 15:06:31 -07:00
Chris Burel 3812e60ab6 Add support for keyboard events from xcb (#3971)
Signed-off-by: Chris Burel <burelc@amazon.com>
2021-09-20 09:09:50 -07:00
brianherrera 0d0b6c8033 Add support for nvme volumes
Signed-off-by: brianherrera <briher@amazon.com>
2021-09-13 17:04:47 -07:00
Junbo Liang 3535d6b6b3 [AWS Automation] Stack tear down mechanism for the deployment on Jenkins (#3656)
Make awsi_test_profile_vs2019 a nonblocking step to make sure that resources are cleaned up even though automation tests failed

Signed-off-by: Junbo Liang <junbo@amazon.com>
2021-09-10 12:56:41 -07:00
jromnoa 5c4ce8cd06 Upload test screenshots to s3 on test failure. (#3815)
* add s3 upload on screenshot test failure, should only apply to the test_gpu_profile_vs2019 job on nightly runs

* adds support for zipping screenshot files up prior to uploading to the s3 bucket and also adds the ACL extra arg to the upload_to_s3.py execution

* remove unused json import

* remove regex to use .endswith() instead and rename variables to be more clear (PR feedback)

* rename create_zip_archive to create_screenshots_archive

Signed-off-by: jromnoa <jromnoa@amazon.com>
2021-09-08 06:11:44 -05:00
Junbo Liang 8edcc504bd Add AWSI automation tests to a new test suite and pipe (#3913)
Signed-off-by: junbo <junbo@amazon.com>
2021-09-07 13:02:13 -07:00
bosnichd 82659f24e9 Add support for a custom path root separator using a trait. (#3678)
* - Add support for a custom path root separator using a trait.
- Ensure to set both FilePathKey_ProjectUserPath and FilePathKey_ProjectLogPath to a writeable storage location on non-host platforms.

Signed-off-by: bosnichd <bosnichd@amazon.com>

* Remove az_trait_validator.py, as our PAL guidance no longer dictates that all traits must be defined for all platforms.

Signed-off-by: bosnichd <bosnichd@amazon.com>

* Updated based on review feedback.

Signed-off-by: bosnichd <bosnichd@amazon.com>
2021-09-01 10:24:29 -06:00
Shirang Jia c1c6fa3615 Auto-cut Github issue on build failures from main/development (#3730)
* Send SNS topic on build failure

* send build failure root cause data to SNS topic that triggeres lambda function to automatically create Github issues

* Change SNS topic subject to Build failure and add build url to the message

* Change SNS topic name
2021-08-31 17:37:25 -07:00
Sean Sweeney 873654a9fc Merge pull request #3116 from aws-lumberyard-dev/benchmark_metrics_jenkins
Adds the MARS benchmark script in AR
2021-08-31 15:58:20 -07:00
Mike Chang 2bf83ad9fb Ubuntu 20.04 package update for clang-12 (#3727)
* Updated CMake, added libxkbcommon-dev and clang-12

* Add Corretto JDK 11 to packages for Ubuntu 20 and 18

* Add mountpoint creation command

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-31 13:47:57 -07:00
Mike Chang 6e3b03dfaa Install and configure Ubuntu nodes with Nvidia drivers and NiceDCV (#3728)
These scripts will configure build and test Ubuntu 18/20 nodes with:

Nvidia gaming drivers for GPU instances (g4dn)
lightdm for Ubuntu 18 and gdm3 for Ubuntu 20 (wayland is disabled here, but should have xcb capabilities)
NiceDCV for remote console access

Note: Does not include user setup and firewall configurations

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-31 09:47:57 -07:00
evanchia 5e4ec8c421 adding missing param to Jenkins benchmark step
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-27 10:14:22 -07:00
lumberyard-employee-dm b53182fdb6 Fixed invalid variable access in register.py (#3433)
* Fixed invalid variable access in register.py

The register_restricted_path and register_template_path methods did not specify the parameter for project path

Updated register_o3de_object_path function to pass in absolute paths to the engine or project in order to make relative paths to the registered o3de object to it.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed register_*_path resolve calls in register.py

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-08-25 13:29:24 -05:00
Esteban Papp f94043dc83 Adds functions so targets can define install directories/files 2021-08-24 15:12:02 -07:00
Chris Burel 86770deefa Add linux equivalents for all the places that enumerate platforms (#3325)
* Add linux equivalents for all the places that enumerate platforms

Signed-off-by: Chris Burel <burelc@amazon.com>

* Fix the AssetFileInfoListComparisonTest fixture to not hardcode the PC platform

Signed-off-by: Chris Burel <burelc@amazon.com>
2021-08-24 14:10:59 -07:00
Shirang Jia ce85bb3b7d Add awsi deployment as a non-blocking step in nightly pipeline (#3397)
* Add awsi deployment as a non-blocking step in nightly pipeline

* Make step fail if it's not a non-blocking step
2021-08-24 10:43:17 -07:00
Esteban Papp dc8b48601c PR comments/improvements
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-24 10:36:31 -07:00
Esteban Papp b609d5e2cb supporting multiple directories
automatically excluding obvious things
cleanup and verification

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-23 19:41:57 -07:00
Mike Chang a92684c0b8 Jenkinsfile LFS fix for iOS/Mac pulls (#3396)
* Add git lfs install and pull into Jenkins Git pull stage

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-23 16:07:02 -07:00
Esteban Papp 8bc9ed3d01 removing some rad leftovers (#3366)
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-20 13:39:51 -07:00
hultonha a529256274 Add extra logging for CHANGE_ID and COMMIT_ID - TIAF
Merge pull request #3328 from aws-lumberyard-dev/jenkins_commit_hash_log
2021-08-20 10:45:41 +01:00
Junbo Liang c93a18ab82 Update the AWS automation tests to use existing CloudFormation stacks (#3092) 2021-08-19 14:05:25 -07:00
John dcde3f3e36 Add extra logging for CHANGE_ID and COMMIT_ID.
Signed-off-by: John <jonawals@amazon.com>
2021-08-19 18:18:45 +01:00
Esteban Papp a1b59e5e5e install reorg/cleanup
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-19 08:51:29 -07:00
Shirang Jia e48798fdbf Add missing {} for string interpolation (#3291) 2021-08-18 10:02:24 -07:00
Shirang Jia 3e51240a05 Allow customized Jenkins parameters for different pipelines and add aws integration deployment pipeline (#3248)
Allow customized Jenkins parameters for different pipelines so that we can define different Jenkins parameters for a new pipeline and doesn't affect AR build parameters.

Add aws integration deployment pipeline.
2021-08-17 15:25:32 -07:00
jonawals 252c268ffd Fix corrupt merge. (#3142)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
2021-08-16 16:09:36 +01:00
jonawals 5925bd22f6 Further subdivide TIAF data by suite. (#3128)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
2021-08-16 14:15:02 +01:00
evanchia 78d34e6960 Adds the MARS benchmark script in AR
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-13 14:10:43 -07:00
John a1d855a920 Invert corner case flag for readability.
Signed-off-by: John <jonawals@amazon.com>
2021-08-13 10:03:09 +01:00
John 4dd39c3929 Address commit re-run corner case.
Signed-off-by: John <jonawals@amazon.com>
2021-08-13 09:55:06 +01:00
jonawals ff659fbbb6 Tiaf bucket top level fix (#3085)
* Revert to regular run when invalid commits used.

* Cirrect s3 logging of last commit hash storage

* Add s3 top level url and build number script params.

* Use existing REPOSITORY_NAME env var.

Signed-off-by: John <jonawals@amazon.com>
2021-08-12 12:48:23 -07:00
John 199273b880 Remove positional argument from change list generaiton.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 12:00:51 +01:00
John 9601db55aa Add sanity checking to last commit hash.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 11:19:48 +01:00
John 268a7f547e Add additional logging to s3 storage.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 11:19:27 +01:00
jonawals f4e8ecd35e TIAF script fixes (#3044)
* Fix typo in MARS document key.

Signed-off-by: John <jonawals@amazon.com>

* Add diff support for commits on different branches.

Signed-off-by: John <jonawals@amazon.com>

* Restore archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Correctly handle commit diffs for PR builds.

Signed-off-by: John <jonawals@amazon.com>

* Comment out archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Fix python typo

Signed-off-by: John <jonawals@amazon.com>

* Remove comment pertaining to s3 bucket.

Signed-off-by: John <jonawals@amazon.com>

* Fix test selection efficiency for MARS.

Signed-off-by: John <jonawals@amazon.com>
2021-08-12 10:21:33 +01:00
Chris Galvan 0f8ea17235 Merge pull request #3051 from aws-lumberyard-dev/cgalvan/gitflow_210811_dev
Merged stabilization/2106 to development
2021-08-11 17:41:29 -05:00
Junbo Liang c6714595b1 Define a new Jenkins build script to deploy AWS resources (#2983)
* Define a new Jenkins job to deploy AWS resources

Signed-off-by: junbo <junbo@amazon.com>

* Address PR comments

Signed-off-by: junbo <junbo@amazon.com>

* Add error checking for all the calls

Signed-off-by: junbo <junbo@amazon.com>

* Remove the weekly-build-metrics tag

Signed-off-by: junbo <junbo@amazon.com>

* Parameterize ARN/Region and move it to a Jenkins envvar.

Signed-off-by: junbo <junbo@amazon.com>

* Revert the changes for build_config.json

Signed-off-by: junbo <junbo@amazon.com>
2021-08-11 13:59:29 -07:00
Chris Galvan 5b0046ce78 Resolved conflicts in merge from stabilization/2106 -> development.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
2021-08-11 15:33:50 -05:00
jonawals a7a0e8d2ef Fix for PathLib unlink wrong version parameter. (#3020)
* Fix for PathLib unlink wrong version parameter.

* Fix dst target for TIAF.

* Remove TIAF historic data arhciving on s3 buckets

* Change permissions for TIAF bucket upload

Signed-off-by: John <jonawals@amazon.com>
2021-08-10 18:43:44 -07:00
John 5dbbb93a06 Fix path for subprocess
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:49:19 +01:00
John e3221224be Add traceback to tiaf exception handler.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:34:27 +01:00
John 2be043ca96 Fix None comparisons in TIAF scripts.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:13:19 +01:00
John c2a2d1c5c7 Add changes from TIF/Feature branch.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 14:20:46 +01:00
lumberyard-employee-dm 9afbd07a88 Added new feature to the register command to auto detect manifest file based on input path (#2967)
* Added new feature to the register command to auto detect the correct
o3de manifest file to write to based on the supplied registration path

Updated the create-gem command to register the gem on creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added new feature to the register command to auto detect the correct
o3de manifest file to write to based on the supplied registration path

Updated the create-gem command to register the gem on creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix incorrect variable reference in register_project_path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed o3de python package unit test
The enable_gem, register and engine template test have been updated to
account for the logic to register a gem after creation

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a --no-register arg to engine-template.py
This allows registration of gems/projects using the create-project and
create-gem commands to be skipped

Prevented registration of a gems and projects in the engine_template
command test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Wrapped first parameter to find_ancestor_dir_containing_file with
pathlib.PurePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-08-09 16:58:19 -05:00
Sean Sweeney 54b93d7c25 Merge pull request #2885 from aws-lumberyard-dev/test_metrics_env_var_fix
Escape character in Jenkins for test metrics
2021-08-09 14:03:02 -07:00
Scott Romero 0999a7f6e4 [development] s3 upload script now replicates directory structure relative to search root (#2947)
Previously the script was using the full local path as the artifact key when uploading to s3. This was causing the installer artifacts to be uploaded incorrectly and non-functional for normal use.

Signed-off-by: AMZN-ScottR 24445312+AMZN-ScottR@users.noreply.github.com
2021-08-09 08:48:51 -07:00