Commit Graph

527 Commits (6d206f69212306bfd3ec36f5f9c385d02407901b)

Author SHA1 Message Date
AMZN-Phil 961c412179 Fix adding local repos due to JSON not being able to serialize Windows paths
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
AMZN-Phil fdce8a3085
Merge pull request #4773 from aws-lumberyard-dev/ReturnCachedGemJsons
Helper function to return cached gem json paths
4 years ago
Alex Peterson 1c409e0a23
Change default folders for user content (#4333)
old vs. new defaults:
%USERHOME%/.o3de/Projects-> %USERHOME%/O3DE/Projects
%USERHOME%/.o3de/Gems-> %USERHOME%/O3DE/Gems
%USERHOME%/.o3de/Templates-> %USERHOME%/O3DE/Templates


Signed-off-by: AMZN-alexpete <26804013+AMZN-alexpete@users.noreply.github.com>
4 years ago
AMZN-Phil e8cd6add51 Merge remote-tracking branch 'upstream/development' into ReturnCachedGemJsons 4 years ago
Mike Chang 8ab1cfacf5
Fixes for Test Screenshot upload build step (#4792)
Fixes the test screenshot upload build step:

- Added missing positional parameters
- Double quoting the command statements to allow variable expansion (also fixes broken Python call)
- Escaping the double quotes within the command statement

Signed-off-by: Mike Chang <changml@amazon.com>
4 years ago
AMZN-Phil 7ce376b5b2 Add some log output if the cached files cannot be found
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
AMZN-Phil c48d957485 Slightly better variable naming
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
AMZN-Phil e5a73fe8ff Add helper function for UI to return paths to all cached gem jsons for a given repo
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
Esteban Papp 5df85534b7 Merge branch 'development' into o3de_sdk/installer_configs
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

# Conflicts:
#	cmake/Packaging.cmake
#	cmake/Projects.cmake
#	scripts/build/Platform/Windows/build_config.json
4 years ago
AMZN-Phil 6f86f0ea5c
Merge pull request #4736 from aws-lumberyard-dev/CLIDownloadGems
Additional fixes for downloading O3DE objects
4 years ago
AMZN-nggieber 4899309b6a
Connect Adding and Removal of Gem Repo UI to CLI (#4729)
* Connect Adding and Removal of Gem Repo UI to CLI

Signed-off-by: nggieber <nggieber@amazon.com>

* Addressed PR feedback

Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
AMZN-Phil 08643f8232 Use a more generic project name in download.py dest-path help
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
AMZN-Phil 1d220c3eaa Updates based on feedback, flip return value of a function to return a more expected value whether validation passes
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
AMZN-Phil 6efeaa405d Additional fixes for downloading O3DE objects, and using the cache folder
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
brianherrera b792ff3d33
Fix command formatting
Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
brianherrera 4625e6d315
Fix indentation
Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
brianherrera 3c7357da47
Update existing unmount step to use new offline function
Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
brianherrera ac8201a2fa
Update disk setup step to use context manager
Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
brianherrera 56f0ea68a6
Add step to verify disk is in an offline state
Some Windows configs will automatically set new drives as online causing diskpart setup script to fail.

Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
AMZN-Phil c355c162ab
Merge pull request #4486 from aws-lumberyard-dev/Prism/ShowRepoList
repo.jsons can be listed and and access through CLI and is Integrated with Project Manager
4 years ago
nggieber 56de6064d2 Hopefully final pass on PR comments
Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
Junbo Liang c759bc9cd0
Update the Jenkins script to disable access log during AWSI deployment (#4579)
Signed-off-by: Junbo Liang <junbo@amazon.com>
4 years ago
nggieber 3698c3c3a7 Merge branch 'development' into Prism/ShowRepoList 4 years ago
nggieber 70053f055a More python PR feedback
Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
Brian Herrera 0d576004fe
Merge pull request #4577 from aws-lumberyard-dev/pipelines/add-retry-inc-build-util
Add retry config to boto3 clients
4 years ago
Junbo Liang a5cd1b55e4
Make the access log bucket optional in the AWSCore CDK application (#4553)
* Make the access log bucket optional in the AWSCore CDK application

Signed-off-by: Junbo Liang <junbo@amazon.com>
4 years ago
brianherrera e256df3f05
Add retry config to boto3 clients
This change makes the inc_build_util script more resilient against transient network issues and issues encountered during node boot-up.

Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
nggieber 06953bb81f More PR changes
Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
Chris Galvan 437a4d2d63 Updated engine template unit tests.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
Chris Galvan e1c02cc146 Updated cpp file extension check per PR feedback.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
Chris Galvan a138094027 Updated create-template logic to prefer the SanitizedCppName when running replacements on cpp file contents.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
AMZN-Phil 8d5d5f8bfb
Merge pull request #4505 from aws-lumberyard-dev/CLIAddExternalGemRepo
Fix issues with adding a repo
4 years ago
jonawals ae96d1ab86
Merge pull request #4525 from aws-lumberyard-dev/tiaf_disable
Disable TIAF Jenkins job
4 years ago
Shirang Jia 8f1a589fcf
Send message to post_ar_build SNS topic after build finished (#4442)
* Send message to post_ar_build SNS topic after build finished

Signed-off-by: shiranj <shiranj@amazon.com>

* Add build number to SNS message

Signed-off-by: shiranj <shiranj@amazon.com>
4 years ago
John 79bd15aabd Disable TIAF job
Signed-off-by: John <jonawals@amazon.com>
4 years ago
nggieber acfbed8a58 Fix minor PR feedback
Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
AMZN-Phil 64fa92c56d Constrain exceptions to the type we are handling.
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
AMZN-Phil 09342694c9 Fix issues with adding a repo and make each object type optional in repo.json
Signed-off-by: AMZN-Phil <pconroy@amazon.com>
4 years ago
nggieber 0f5a347205 Merge branch 'development' into Prism/ShowRepoList 4 years ago
nggieber 118834efde Added ability to list gem repos using CLI and integrated support into Project Manager
Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
Esteban Papp f3e6adce7f
LYN-6882 release builds are executing code in asserts (#4305)
* adding Windows/release to PR-validation builds

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* changing trace back to expand to nothing for release

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* typo

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* more fixes

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* fixing some more unused variable cases

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* renaming file in ScriptCanvas that causes a msbuild warning

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* reverting a previous change

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Mike Chang 0b718d435c
Installer, bootstrapper, and executable signing for Windows (#4406)
* Added a codesigning script to pre and post build steps in the Windows installer CD job

* Changed `windows_installer` job name to `installer_vs2019`

* Added `installer-nightly` tag for `installer_vs2019`

* Updated `build_config.json` to use an envvar for the installer url and bucket

Signed-off-by: Mike Chang <changml@amazon.com>
4 years ago
Esteban Papp f273664e1d Merge branch 'development' into o3de_sdk/installer_configs
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
dialweb6 4627361b4e
Merge pull request #4245 from AMZN-scspaldi/scspaldi_update_Jenkins_test_results
Updated Linux and Mac configs to show test results.
4 years ago
Esteban Papp 3106e88155 Merge branch 'development' into o3de_sdk/installer_configs
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Esteban Papp 642f2b37ee able to put release monolithic into an installer
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Brian Herrera aaf292f8cd
Merge pull request #4250 from aws-lumberyard-dev/pipelines/retry-pipeline
Retry pipeline stages when a node goes offline
4 years ago
brianherrera eaddd4eee7
Throw exception for build stage
This is required so Jenkins will properly mark the build failed on valid script errors.

Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
Mike Chang 39a4e05492
Change installer domain to o3debinaries.org (#4244)
* Change installer domain to o3debinaries.org

* Removed 3p license parameter

Signed-off-by: Mike Chang <changml@amazon.com>
4 years ago
brianherrera d44cc844d1
Retry pipeline stages when a node goes offline
This change is required to support spot instances in the AR pipeline.

- When a node goes offline the job is retried on a new node
- Valid script errors are handled properly and the status is updated in Jenkins

Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
scspaldi d11dbd3732 Updated Linux and Mac configs to show test results.
Signed-off-by: scspaldi <scspaldi@amazon.com>
4 years ago
Chris Burel 3812e60ab6
Add support for keyboard events from xcb (#3971)
Signed-off-by: Chris Burel <burelc@amazon.com>
4 years ago
brianherrera 0d0b6c8033
Add support for nvme volumes
Signed-off-by: brianherrera <briher@amazon.com>
4 years ago
Junbo Liang 3535d6b6b3
[AWS Automation] Stack tear down mechanism for the deployment on Jenkins (#3656)
Make awsi_test_profile_vs2019 a nonblocking step to make sure that resources are cleaned up even though automation tests failed

Signed-off-by: Junbo Liang <junbo@amazon.com>
4 years ago
jromnoa 5c4ce8cd06
Upload test screenshots to s3 on test failure. (#3815)
* add s3 upload on screenshot test failure, should only apply to the test_gpu_profile_vs2019 job on nightly runs

* adds support for zipping screenshot files up prior to uploading to the s3 bucket and also adds the ACL extra arg to the upload_to_s3.py execution

* remove unused json import

* remove regex to use .endswith() instead and rename variables to be more clear (PR feedback)

* rename create_zip_archive to create_screenshots_archive

Signed-off-by: jromnoa <jromnoa@amazon.com>
4 years ago
Junbo Liang 8edcc504bd
Add AWSI automation tests to a new test suite and pipe (#3913)
Signed-off-by: junbo <junbo@amazon.com>
4 years ago
bosnichd 82659f24e9
Add support for a custom path root separator using a trait. (#3678)
* - Add support for a custom path root separator using a trait.
- Ensure to set both FilePathKey_ProjectUserPath and FilePathKey_ProjectLogPath to a writeable storage location on non-host platforms.

Signed-off-by: bosnichd <bosnichd@amazon.com>

* Remove az_trait_validator.py, as our PAL guidance no longer dictates that all traits must be defined for all platforms.

Signed-off-by: bosnichd <bosnichd@amazon.com>

* Updated based on review feedback.

Signed-off-by: bosnichd <bosnichd@amazon.com>
4 years ago
Shirang Jia c1c6fa3615
Auto-cut Github issue on build failures from main/development (#3730)
* Send SNS topic on build failure

* send build failure root cause data to SNS topic that triggeres lambda function to automatically create Github issues

* Change SNS topic subject to Build failure and add build url to the message

* Change SNS topic name
4 years ago
Sean Sweeney 873654a9fc
Merge pull request #3116 from aws-lumberyard-dev/benchmark_metrics_jenkins
Adds the MARS benchmark script in AR
4 years ago
Mike Chang 2bf83ad9fb
Ubuntu 20.04 package update for clang-12 (#3727)
* Updated CMake, added libxkbcommon-dev and clang-12

* Add Corretto JDK 11 to packages for Ubuntu 20 and 18

* Add mountpoint creation command

Signed-off-by: Mike Chang <changml@amazon.com>
4 years ago
Mike Chang 6e3b03dfaa
Install and configure Ubuntu nodes with Nvidia drivers and NiceDCV (#3728)
These scripts will configure build and test Ubuntu 18/20 nodes with:

Nvidia gaming drivers for GPU instances (g4dn)
lightdm for Ubuntu 18 and gdm3 for Ubuntu 20 (wayland is disabled here, but should have xcb capabilities)
NiceDCV for remote console access

Note: Does not include user setup and firewall configurations

Signed-off-by: Mike Chang <changml@amazon.com>
4 years ago
evanchia 5e4ec8c421 adding missing param to Jenkins benchmark step
Signed-off-by: evanchia <evanchia@amazon.com>
4 years ago
lumberyard-employee-dm b53182fdb6
Fixed invalid variable access in register.py (#3433)
* Fixed invalid variable access in register.py

The register_restricted_path and register_template_path methods did not specify the parameter for project path

Updated register_o3de_object_path function to pass in absolute paths to the engine or project in order to make relative paths to the registered o3de object to it.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed register_*_path resolve calls in register.py

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
4 years ago
Esteban Papp f94043dc83
Adds functions so targets can define install directories/files 4 years ago
Chris Burel 86770deefa
Add linux equivalents for all the places that enumerate platforms (#3325)
* Add linux equivalents for all the places that enumerate platforms

Signed-off-by: Chris Burel <burelc@amazon.com>

* Fix the AssetFileInfoListComparisonTest fixture to not hardcode the PC platform

Signed-off-by: Chris Burel <burelc@amazon.com>
4 years ago
Shirang Jia ce85bb3b7d
Add awsi deployment as a non-blocking step in nightly pipeline (#3397)
* Add awsi deployment as a non-blocking step in nightly pipeline

* Make step fail if it's not a non-blocking step
4 years ago
Esteban Papp dc8b48601c PR comments/improvements
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Esteban Papp b609d5e2cb supporting multiple directories
automatically excluding obvious things
cleanup and verification

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Mike Chang a92684c0b8
Jenkinsfile LFS fix for iOS/Mac pulls (#3396)
* Add git lfs install and pull into Jenkins Git pull stage

Signed-off-by: Mike Chang <changml@amazon.com>
4 years ago
Esteban Papp 8bc9ed3d01
removing some rad leftovers (#3366)
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
hultonha a529256274
Add extra logging for CHANGE_ID and COMMIT_ID - TIAF
Merge pull request #3328 from aws-lumberyard-dev/jenkins_commit_hash_log
4 years ago
Junbo Liang c93a18ab82
Update the AWS automation tests to use existing CloudFormation stacks (#3092) 4 years ago
John dcde3f3e36 Add extra logging for CHANGE_ID and COMMIT_ID.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
Esteban Papp a1b59e5e5e install reorg/cleanup
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Shirang Jia e48798fdbf
Add missing {} for string interpolation (#3291) 4 years ago
Shirang Jia 3e51240a05
Allow customized Jenkins parameters for different pipelines and add aws integration deployment pipeline (#3248)
Allow customized Jenkins parameters for different pipelines so that we can define different Jenkins parameters for a new pipeline and doesn't affect AR build parameters.

Add aws integration deployment pipeline.
4 years ago
jonawals 252c268ffd
Fix corrupt merge. (#3142)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
4 years ago
jonawals 5925bd22f6
Further subdivide TIAF data by suite. (#3128)
* Further subdivide TIAF data by suite.
* Key fix typo.

Signed-off-by: John <jonawals@amazon.com>
4 years ago
evanchia 78d34e6960 Adds the MARS benchmark script in AR
Signed-off-by: evanchia <evanchia@amazon.com>
4 years ago
John a1d855a920 Invert corner case flag for readability.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
John 4dd39c3929 Address commit re-run corner case.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
jonawals ff659fbbb6
Tiaf bucket top level fix (#3085)
* Revert to regular run when invalid commits used.

* Cirrect s3 logging of last commit hash storage

* Add s3 top level url and build number script params.

* Use existing REPOSITORY_NAME env var.

Signed-off-by: John <jonawals@amazon.com>
4 years ago
John 199273b880 Remove positional argument from change list generaiton.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
John 9601db55aa Add sanity checking to last commit hash.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
John 268a7f547e Add additional logging to s3 storage.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
jonawals f4e8ecd35e
TIAF script fixes (#3044)
* Fix typo in MARS document key.

Signed-off-by: John <jonawals@amazon.com>

* Add diff support for commits on different branches.

Signed-off-by: John <jonawals@amazon.com>

* Restore archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Correctly handle commit diffs for PR builds.

Signed-off-by: John <jonawals@amazon.com>

* Comment out archiving of historic data objects.

Signed-off-by: John <jonawals@amazon.com>

* Fix python typo

Signed-off-by: John <jonawals@amazon.com>

* Remove comment pertaining to s3 bucket.

Signed-off-by: John <jonawals@amazon.com>

* Fix test selection efficiency for MARS.

Signed-off-by: John <jonawals@amazon.com>
4 years ago
Chris Galvan 0f8ea17235
Merge pull request #3051 from aws-lumberyard-dev/cgalvan/gitflow_210811_dev
Merged stabilization/2106 to development
4 years ago
Junbo Liang c6714595b1
Define a new Jenkins build script to deploy AWS resources (#2983)
* Define a new Jenkins job to deploy AWS resources

Signed-off-by: junbo <junbo@amazon.com>

* Address PR comments

Signed-off-by: junbo <junbo@amazon.com>

* Add error checking for all the calls

Signed-off-by: junbo <junbo@amazon.com>

* Remove the weekly-build-metrics tag

Signed-off-by: junbo <junbo@amazon.com>

* Parameterize ARN/Region and move it to a Jenkins envvar.

Signed-off-by: junbo <junbo@amazon.com>

* Revert the changes for build_config.json

Signed-off-by: junbo <junbo@amazon.com>
4 years ago
Chris Galvan 5b0046ce78 Resolved conflicts in merge from stabilization/2106 -> development.
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
4 years ago
jonawals a7a0e8d2ef
Fix for PathLib unlink wrong version parameter. (#3020)
* Fix for PathLib unlink wrong version parameter.

* Fix dst target for TIAF.

* Remove TIAF historic data arhciving on s3 buckets

* Change permissions for TIAF bucket upload

Signed-off-by: John <jonawals@amazon.com>
4 years ago
John 5dbbb93a06 Fix path for subprocess
Signed-off-by: John <jonawals@amazon.com>
4 years ago
John e3221224be Add traceback to tiaf exception handler.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
John 2be043ca96 Fix None comparisons in TIAF scripts.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
John c2a2d1c5c7 Add changes from TIF/Feature branch.
Signed-off-by: John <jonawals@amazon.com>
4 years ago
lumberyard-employee-dm 9afbd07a88
Added new feature to the register command to auto detect manifest file based on input path (#2967)
* Added new feature to the register command to auto detect the correct
o3de manifest file to write to based on the supplied registration path

Updated the create-gem command to register the gem on creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added new feature to the register command to auto detect the correct
o3de manifest file to write to based on the supplied registration path

Updated the create-gem command to register the gem on creation.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix incorrect variable reference in register_project_path

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed o3de python package unit test
The enable_gem, register and engine template test have been updated to
account for the logic to register a gem after creation

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Added a --no-register arg to engine-template.py
This allows registration of gems/projects using the create-project and
create-gem commands to be skipped

Prevented registration of a gems and projects in the engine_template
command test

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Wrapped first parameter to find_ancestor_dir_containing_file with
pathlib.PurePath

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
4 years ago
Sean Sweeney 54b93d7c25
Merge pull request #2885 from aws-lumberyard-dev/test_metrics_env_var_fix
Escape character in Jenkins for test metrics
4 years ago
Scott Romero 0999a7f6e4
[development] s3 upload script now replicates directory structure relative to search root (#2947)
Previously the script was using the full local path as the artifact key when uploading to s3. This was causing the installer artifacts to be uploaded incorrectly and non-functional for normal use.

Signed-off-by: AMZN-ScottR 24445312+AMZN-ScottR@users.noreply.github.com
4 years ago
Shirang Jia 4f9382e8c6
Include build failure root cause in email notification (#2491) (#2888)
Signed-off-by: shiranj <shiranj@amazon.com>
4 years ago
AMZN-nggieber 2c7f6f9742
Adds Links to Gem Directory and Documentation for Gems (#2922)
* Changed blue text to white that is not meant as a link, made 'View in Director' link work for gems in the inspector, added parsing for gem documentation link

Signed-off-by: nggieber <nggieber@amazon.com>

* Added documentation links for gems, changed markup for urls in summaries and requirements so they are clickable

Signed-off-by: nggieber <nggieber@amazon.com>

* Fixed a couple of the documentation links

Signed-off-by: nggieber <nggieber@amazon.com>

* Added documentation url to edit gem properties script and updated unit tests

Signed-off-by: nggieber <nggieber@amazon.com>
4 years ago
evanchia 3a76f2e6ce Fixing incorrect string replacement in Jenkinsfile
Signed-off-by: evanchia <evanchia@amazon.com>
4 years ago