Commit Graph

294 Commits

Author SHA1 Message Date
Junbo Liang 3535d6b6b3 [AWS Automation] Stack tear down mechanism for the deployment on Jenkins (#3656)
Make awsi_test_profile_vs2019 a nonblocking step to make sure that resources are cleaned up even though automation tests failed

Signed-off-by: Junbo Liang <junbo@amazon.com>
2021-09-10 12:56:41 -07:00
jromnoa 5c4ce8cd06 Upload test screenshots to s3 on test failure. (#3815)
* add s3 upload on screenshot test failure, should only apply to the test_gpu_profile_vs2019 job on nightly runs

* adds support for zipping screenshot files up prior to uploading to the s3 bucket and also adds the ACL extra arg to the upload_to_s3.py execution

* remove unused json import

* remove regex to use .endswith() instead and rename variables to be more clear (PR feedback)

* rename create_zip_archive to create_screenshots_archive

Signed-off-by: jromnoa <jromnoa@amazon.com>
2021-09-08 06:11:44 -05:00
Junbo Liang 8edcc504bd Add AWSI automation tests to a new test suite and pipe (#3913)
Signed-off-by: junbo <junbo@amazon.com>
2021-09-07 13:02:13 -07:00
Shirang Jia c1c6fa3615 Auto-cut Github issue on build failures from main/development (#3730)
* Send SNS topic on build failure

* send build failure root cause data to SNS topic that triggeres lambda function to automatically create Github issues

* Change SNS topic subject to Build failure and add build url to the message

* Change SNS topic name
2021-08-31 17:37:25 -07:00
Sean Sweeney 873654a9fc Merge pull request #3116 from aws-lumberyard-dev/benchmark_metrics_jenkins
Adds the MARS benchmark script in AR
2021-08-31 15:58:20 -07:00
Mike Chang 2bf83ad9fb Ubuntu 20.04 package update for clang-12 (#3727)
* Updated CMake, added libxkbcommon-dev and clang-12

* Add Corretto JDK 11 to packages for Ubuntu 20 and 18

* Add mountpoint creation command

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-31 13:47:57 -07:00
Mike Chang 6e3b03dfaa Install and configure Ubuntu nodes with Nvidia drivers and NiceDCV (#3728)
These scripts will configure build and test Ubuntu 18/20 nodes with:

Nvidia gaming drivers for GPU instances (g4dn)
lightdm for Ubuntu 18 and gdm3 for Ubuntu 20 (wayland is disabled here, but should have xcb capabilities)
NiceDCV for remote console access

Note: Does not include user setup and firewall configurations

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-31 09:47:57 -07:00
evanchia 5e4ec8c421 adding missing param to Jenkins benchmark step
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-27 10:14:22 -07:00
Chris Burel 86770deefa Add linux equivalents for all the places that enumerate platforms (#3325)
* Add linux equivalents for all the places that enumerate platforms

Signed-off-by: Chris Burel <burelc@amazon.com>

* Fix the AssetFileInfoListComparisonTest fixture to not hardcode the PC platform

Signed-off-by: Chris Burel <burelc@amazon.com>
2021-08-24 14:10:59 -07:00
Shirang Jia ce85bb3b7d Add awsi deployment as a non-blocking step in nightly pipeline (#3397)
* Add awsi deployment as a non-blocking step in nightly pipeline

* Make step fail if it's not a non-blocking step
2021-08-24 10:43:17 -07:00
Mike Chang a92684c0b8 Jenkinsfile LFS fix for iOS/Mac pulls (#3396)
* Add git lfs install and pull into Jenkins Git pull stage

Signed-off-by: Mike Chang <changml@amazon.com>
2021-08-23 16:07:02 -07:00
Esteban Papp 8bc9ed3d01 removing some rad leftovers (#3366)
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-08-20 13:39:51 -07:00
hultonha a529256274 Add extra logging for CHANGE_ID and COMMIT_ID - TIAF
Merge pull request #3328 from aws-lumberyard-dev/jenkins_commit_hash_log
2021-08-20 10:45:41 +01:00
Junbo Liang c93a18ab82 Update the AWS automation tests to use existing CloudFormation stacks (#3092) 2021-08-19 14:05:25 -07:00
John dcde3f3e36 Add extra logging for CHANGE_ID and COMMIT_ID.
Signed-off-by: John <jonawals@amazon.com>
2021-08-19 18:18:45 +01:00
Shirang Jia e48798fdbf Add missing {} for string interpolation (#3291) 2021-08-18 10:02:24 -07:00
Shirang Jia 3e51240a05 Allow customized Jenkins parameters for different pipelines and add aws integration deployment pipeline (#3248)
Allow customized Jenkins parameters for different pipelines so that we can define different Jenkins parameters for a new pipeline and doesn't affect AR build parameters.

Add aws integration deployment pipeline.
2021-08-17 15:25:32 -07:00
jonawals 252c268ffd Fix corrupt merge. (#3142)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
2021-08-16 16:09:36 +01:00
jonawals 5925bd22f6 Further subdivide TIAF data by suite. (#3128)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
2021-08-16 14:15:02 +01:00
evanchia 78d34e6960 Adds the MARS benchmark script in AR
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-13 14:10:43 -07:00
John a1d855a920 Invert corner case flag for readability.
Signed-off-by: John <jonawals@amazon.com>
2021-08-13 10:03:09 +01:00
John 4dd39c3929 Address commit re-run corner case.
Signed-off-by: John <jonawals@amazon.com>
2021-08-13 09:55:06 +01:00
jonawals ff659fbbb6 Tiaf bucket top level fix (#3085)
* Revert to regular run when invalid commits used.

* Cirrect s3 logging of last commit hash storage

* Add s3 top level url and build number script params.

* Use existing REPOSITORY_NAME env var.

Signed-off-by: John <jonawals@amazon.com>
2021-08-12 12:48:23 -07:00
John 199273b880 Remove positional argument from change list generaiton.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 12:00:51 +01:00
John 9601db55aa Add sanity checking to last commit hash.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 11:19:48 +01:00
John 268a7f547e Add additional logging to s3 storage.
Signed-off-by: John <jonawals@amazon.com>
2021-08-12 11:19:27 +01:00
jonawals f4e8ecd35e TIAF script fixes (#3044)
* Fix typo in MARS document key.

Signed-off-by: John <jonawals@amazon.com>

* Add diff support for commits on different branches.

Signed-off-by: John <jonawals@amazon.com>

* Restore archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Correctly handle commit diffs for PR builds.

Signed-off-by: John <jonawals@amazon.com>

* Comment out archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Fix python typo

Signed-off-by: John <jonawals@amazon.com>

* Remove comment pertaining to s3 bucket.

Signed-off-by: John <jonawals@amazon.com>

* Fix test selection efficiency for MARS.

Signed-off-by: John <jonawals@amazon.com>
2021-08-12 10:21:33 +01:00
Junbo Liang c6714595b1 Define a new Jenkins build script to deploy AWS resources (#2983)
* Define a new Jenkins job to deploy AWS resources

Signed-off-by: junbo <junbo@amazon.com>

* Address PR comments

Signed-off-by: junbo <junbo@amazon.com>

* Add error checking for all the calls

Signed-off-by: junbo <junbo@amazon.com>

* Remove the weekly-build-metrics tag

Signed-off-by: junbo <junbo@amazon.com>

* Parameterize ARN/Region and move it to a Jenkins envvar.

Signed-off-by: junbo <junbo@amazon.com>

* Revert the changes for build_config.json

Signed-off-by: junbo <junbo@amazon.com>
2021-08-11 13:59:29 -07:00
jonawals a7a0e8d2ef Fix for PathLib unlink wrong version parameter. (#3020)
* Fix for PathLib unlink wrong version parameter.

* Fix dst target for TIAF.

* Remove TIAF historic data arhciving on s3 buckets

* Change permissions for TIAF bucket upload

Signed-off-by: John <jonawals@amazon.com>
2021-08-10 18:43:44 -07:00
John 5dbbb93a06 Fix path for subprocess
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:49:19 +01:00
John e3221224be Add traceback to tiaf exception handler.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:34:27 +01:00
John 2be043ca96 Fix None comparisons in TIAF scripts.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 15:13:19 +01:00
John c2a2d1c5c7 Add changes from TIF/Feature branch.
Signed-off-by: John <jonawals@amazon.com>
2021-08-10 14:20:46 +01:00
Sean Sweeney 54b93d7c25 Merge pull request #2885 from aws-lumberyard-dev/test_metrics_env_var_fix
Escape character in Jenkins for test metrics
2021-08-09 14:03:02 -07:00
Scott Romero 0999a7f6e4 [development] s3 upload script now replicates directory structure relative to search root (#2947)
Previously the script was using the full local path as the artifact key when uploading to s3. This was causing the installer artifacts to be uploaded incorrectly and non-functional for normal use.

Signed-off-by: AMZN-ScottR 24445312+AMZN-ScottR@users.noreply.github.com
2021-08-09 08:48:51 -07:00
Shirang Jia 4f9382e8c6 Include build failure root cause in email notification (#2491) (#2888)
Signed-off-by: shiranj <shiranj@amazon.com>
2021-08-06 12:52:04 -07:00
evanchia 3a76f2e6ce Fixing incorrect string replacement in Jenkinsfile
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-05 17:41:06 -07:00
evanchia 574bef0cd8 Escape character in Jenkins for test metrics
Signed-off-by: evanchia <evanchia@amazon.com>
2021-08-05 10:19:15 -07:00
Esteban Papp da474357f3 Some var cleanup so it shows better-organized in cmake-gui. Some vars… (#2361)
* Some var cleanup so it shows better-organized in cmake-gui. Some vars were also not following the namign convention we are using
Removed some unnecessary messaging
Fixed a TIF bug where it would report the wrong test in a message, fixed a message that was being triggered
Changed TIF to be enabled just by the binary so running the ci_build scripts locally doesnt trigger TIF messaging
Removed `LY_ENABLE_MULTIPLAYER_COMPRESSION`, it was not being used

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* handling case where a parameter can be empty

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* needs to be var name, not contents

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
2021-07-26 14:14:56 -07:00
amzn-sean 3ade098742 support node switching in a pipe (#2345)
Signed-off-by: amzn-sean <75276488+amzn-sean@users.noreply.github.com>
2021-07-26 11:16:13 +01:00
Chris Burel f3cb0ee94d Remvoe Jenkins build configuration for unsupported VS2017 compiler
Signed-off-by: Chris Burel <burelc@amazon.com>
2021-07-19 13:30:35 -07:00
Terry Michaels d9ec159f0e Merge pull request #2235 from aws-lumberyard-dev/daimini/gitflow_210716_o3de
Gitflow 7/16/21 - O3DE
2021-07-16 17:50:44 -05:00
Steve Pham 38261d0800 Shorten copyright headers by splitting into 2 lines (#2213)
* Updated all copyright headers to split the longer original copyright line into 2 shorter lines

Signed-off-by: Steve Pham <spham@amazon.com>
2021-07-16 15:25:48 -07:00
Danilo Aimini e9270e491a Merge branch 'stabilization/2106' into daimini/gitflow_210716_o3de
Signed-off-by: Danilo Aimini <82231674+AMZN-daimini@users.noreply.github.com>

# Conflicts:
#	scripts/build/bootstrap/incremental_build_util.py
2021-07-16 13:21:55 -07:00
Shirang Jia 23f690cd55 Cleanup incremental build scripts (#2004)
* Cleanup incremental build scripts

Signed-off-by: shiranj <shiranj@amazon.com>

* Fix timeout function since signal.SIGALRM is not available on Windows

Signed-off-by: shiranj <shiranj@amazon.com>

* Convert missing fstrings

Signed-off-by: shiranj <shiranj@amazon.com>

* Add PipelineAndBranch back to EBS volume tag

Signed-off-by: shiranj <shiranj@amazon.com>
2021-07-12 16:58:33 -07:00
brianherrera 41e864de95 Remove separate git lfs steps.
The steps to remove git lfs hooks and inject creds are no longer required with the public repo.

Signed-off-by: brianherrera <briher@amazon.com>
2021-07-12 11:12:26 -07:00
brianherrera 1121bf6e60 Remove separate git lfs steps.
The steps to remove git lfs hooks and inject creds are no longer required with the public repo.

Signed-off-by: brianherrera <briher@amazon.com>
2021-07-12 09:14:16 -07:00
Steve Pham b8f857a270 Ubuntu package updates (#1985)
Add required libfontconfig1-dev which is needed by Linux/Qt/QtFontDatabaseSupport (dependency of Qt5XcbQpa)

Signed-off-by: spham-amzn <spham@amazon.com>
2021-07-09 15:01:53 -07:00
jonawals 54138bb04c Properly handle creation of TIAF historic workspace directory (#2006)
* Fix historic working dir creation

Signed-off-by: John <jonawals@amazon.com>

* Add new line at EOF

Signed-off-by: John <jonawals@amazon.com>

* Removing extra whitespace at end of file

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Log global exception caught by TIAF driver

Signed-off-by: John <jonawals@amazon.com>

Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
2021-07-08 22:13:44 -05:00
Terry Michaels 9ea9cbf589 Merge pull request #1989 from aws-lumberyard-dev/cgalvan/gitflow_210708
Merged stabilization/2106 to development
2021-07-08 18:03:30 -05:00