From 49e8f358581dad7ca0e8127905a9d5e864c41cee Mon Sep 17 00:00:00 2001 From: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> Date: Thu, 18 Nov 2021 16:26:49 -0800 Subject: [PATCH] Creates a deb package (#5567) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * configure/generation working Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Generates deb package forLinux Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes to EngineFinder Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * changes the owner of the installed folder so it is writable by the user Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * jenkins scripts Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes execution permission for scripts Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes wrong parameter to build for linux Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * removal of clang parameters in build_config.json Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Simplifies and fixes build configs Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * fixes to EngineFinder Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Adds linux deb package upload and small fixes/organization cleanup Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * fixes checksum generation Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Some more small fixes for reviewing Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Test if this space is necessary for cpack Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * More fixes for installer jobs Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Adding eval to variables that need to double-expand Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Removing unused variables being passed Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Do not remove symlink Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Use cmake_path for concatenating paths Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * removing quotes so the variable expands Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * cleaned up some variables that have way too many ways to be passed Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Multiple fixes for Jenkins Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Cleanup/re-organization Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * fixes some temp variables so we can run cpack jenkins jobs locally Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Delays cmake download to install step Cleanup on installer_windows.cmd, we already have the path to cpack (same as cmake, which is added to the path) Some simplification in the windows installer generation, we now generate the same even if we dont upload Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes mac install Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * CPack can generate hashes for us Defiine CPACK_OUTPUT_FILE_PREFIX to prevent the exe/msi from ending in the bin dir Remove unnecessary generation of folder from 3rdPartyPacakges which fails when including from cpack Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * multiple fixes Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * fixes typo Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * more fixes Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes for linux/mac/windows (fixing discrepancies with Jenkins) Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * uploads to S3 from the upload directory and fixes the latest upload Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Adds ERROR_VARIABLE to S3 upload to find a Jenkins issue Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * printing command invocation to try to figure out the errors Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Changes the hashing algorithm to sha256 and fixes the cmd used to call to pythojn Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * wrong path being passed in mac Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * removes compiler parameters from a config job (after rebase) Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * more fixes fro deb packaging job Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * small fix for fingerprint patching for latest Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Changing extension of public key and leaving better TODO places for code signing Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Removes sdl as a dependency, leaving it commented since it is needed for wwise Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * fixes runtime copy during install, makes it more consistent with regular runtime dependencies and optimizes copy when there is already an install Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * $ENV{DESTDIR} and CMAKE_INSTALL_PREFIX cannot be used in cmake_path because the CMAKE_INSTALL_PREFIX is absolute Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixing some runtime_dependencies optimizations, we cannot take size of a bundle Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes warning in mac for an unused variable Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fixes a variable that was being passed to fix qt frameworks Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * More fixes for Jenkins Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Updates email to use for maintainer to match certificate Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * make directory because in some cases the dependency is copied before the actual library and the directory has to be created Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * Fix for a new warning/error in linux Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * The sha file is created after the upload happens, so creating it Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> * remove the size comparison for install since it stomps over the copy from the target and doesnt trigger stripping Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com> --- AutomatedTesting/CMakeLists.txt | 3 +- AutomatedTesting/cmake/CompilerSettings.cmake | 13 ++ .../{ => cmake}/EngineFinder.cmake | 42 ++++- .../Linux/CompilerSettings_linux.cmake | 0 .../AzCore/Serialization/AZStdContainers.inl | 2 +- .../AzToolsFramework/API/PythonLoader.h | 2 +- .../Template/cmake/CompilerSettings.cmake | 2 +- .../Linux/CompilerSettings_linux.cmake} | 0 Templates/DefaultProject/template.json | 4 +- .../Template/cmake/CompilerSettings.cmake | 2 +- .../Linux/CompilerSettings_linux.cmake | 0 Templates/MinimalProject/template.json | 4 +- cmake/3rdPartyPackages.cmake | 11 +- cmake/CompilerSettings.cmake | 2 +- cmake/Packaging.cmake | 177 +++++------------- cmake/Packaging/CMakeDownload.cmake.in | 54 ++++++ cmake/Platform/Common/Install_common.cmake | 17 +- .../Common/PackagingPostBuild_common.cmake | 114 +++++++++++ .../Common/PackagingPreBuild_common.cmake | 5 + .../runtime_dependencies_common.cmake.in | 1 + .../Linux/CompilerSettings_linux.cmake | 34 ++++ cmake/Platform/Linux/Install_linux.cmake | 29 ++- cmake/Platform/Linux/PAL_linux.cmake | 2 +- cmake/Platform/Linux/Packaging/postinst.in | 21 +++ cmake/Platform/Linux/Packaging/postrm.in | 15 ++ cmake/Platform/Linux/Packaging/prerm.in | 23 +++ .../Linux/PackagingPostBuild_linux.cmake | 62 ++++++ .../Linux/PackagingPreBuild_linux.cmake | 16 ++ cmake/Platform/Linux/Packaging_linux.cmake | 56 ++++++ .../Platform/Linux/platform_linux_files.cmake | 8 + .../Linux/runtime_dependencies_linux.cmake.in | 1 + cmake/Platform/Mac/InstallUtils_mac.cmake.in | 37 ++-- .../Platform/Mac/PackagingPostBuild_mac.cmake | 10 + .../Platform/Mac/PackagingPreBuild_mac.cmake | 10 + .../Mac/runtime_dependencies_mac.cmake.in | 19 +- ...cmake => PackagingPostBuild_windows.cmake} | 139 +++----------- ....cmake => PackagingPreBuild_windows.cmake} | 3 + .../Platform/Windows/Packaging_windows.cmake | 65 +++---- .../Windows/platform_windows_files.cmake | 5 +- .../build/Platform/Linux/build_config.json | 43 +++-- .../Platform/Linux/build_installer_linux.sh | 15 ++ scripts/build/Platform/Linux/build_linux.sh | 8 +- .../build/Platform/Linux/installer_linux.sh | 30 +++ .../build/Platform/Windows/build_config.json | 4 +- .../build/Platform/Windows/build_windows.cmd | 7 - .../build/Platform/Windows/env_windows.cmd | 11 ++ .../Platform/Windows/installer_windows.cmd | 14 +- scripts/build/Platform/iOS/build_config.json | 4 +- .../Linux/package-list.ubuntu-bionic.txt | 2 +- .../Linux/package-list.ubuntu-focal.txt | 2 +- 50 files changed, 771 insertions(+), 379 deletions(-) create mode 100644 AutomatedTesting/cmake/CompilerSettings.cmake rename AutomatedTesting/{ => cmake}/EngineFinder.cmake (63%) rename Templates/DefaultProject/Template/cmake/Platform/Linux/CompilerSettings.cmake => AutomatedTesting/cmake/Platform/Linux/CompilerSettings_linux.cmake (100%) rename Templates/{MinimalProject/Template/cmake/Platform/Linux/CompilerSettings.cmake => DefaultProject/Template/cmake/Platform/Linux/CompilerSettings_linux.cmake} (100%) rename cmake/Platform/Linux/CompilerSettings.cmake => Templates/MinimalProject/Template/cmake/Platform/Linux/CompilerSettings_linux.cmake (100%) create mode 100644 cmake/Packaging/CMakeDownload.cmake.in create mode 100644 cmake/Platform/Common/PackagingPostBuild_common.cmake create mode 100644 cmake/Platform/Common/PackagingPreBuild_common.cmake create mode 100644 cmake/Platform/Linux/CompilerSettings_linux.cmake create mode 100644 cmake/Platform/Linux/Packaging/postinst.in create mode 100644 cmake/Platform/Linux/Packaging/postrm.in create mode 100644 cmake/Platform/Linux/Packaging/prerm.in create mode 100644 cmake/Platform/Linux/PackagingPostBuild_linux.cmake create mode 100644 cmake/Platform/Linux/PackagingPreBuild_linux.cmake create mode 100644 cmake/Platform/Linux/Packaging_linux.cmake create mode 100644 cmake/Platform/Mac/PackagingPostBuild_mac.cmake create mode 100644 cmake/Platform/Mac/PackagingPreBuild_mac.cmake rename cmake/Platform/Windows/{PackagingPostBuild.cmake => PackagingPostBuild_windows.cmake} (52%) rename cmake/Platform/Windows/{PackagingPreBuild.cmake => PackagingPreBuild_windows.cmake} (93%) create mode 100755 scripts/build/Platform/Linux/build_installer_linux.sh create mode 100755 scripts/build/Platform/Linux/installer_linux.sh diff --git a/AutomatedTesting/CMakeLists.txt b/AutomatedTesting/CMakeLists.txt index dee9d73aea..1c5382ba4b 100644 --- a/AutomatedTesting/CMakeLists.txt +++ b/AutomatedTesting/CMakeLists.txt @@ -8,11 +8,12 @@ if(NOT PROJECT_NAME) cmake_minimum_required(VERSION 3.20) + include(cmake/CompilerSettings.cmake) project(AutomatedTesting LANGUAGES C CXX VERSION 1.0.0.0 ) - include(EngineFinder.cmake OPTIONAL) + include(cmake/EngineFinder.cmake OPTIONAL) find_package(o3de REQUIRED) o3de_initialize() else() diff --git a/AutomatedTesting/cmake/CompilerSettings.cmake b/AutomatedTesting/cmake/CompilerSettings.cmake new file mode 100644 index 0000000000..60bda1d45b --- /dev/null +++ b/AutomatedTesting/cmake/CompilerSettings.cmake @@ -0,0 +1,13 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +# File to tweak compiler settings before compiler detection happens (before project() is called) +# We dont have PAL enabled at this point, so we can only use pure-CMake variables +if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Linux") + include(cmake/Platform/Linux/CompilerSettings_linux.cmake) +endif() diff --git a/AutomatedTesting/EngineFinder.cmake b/AutomatedTesting/cmake/EngineFinder.cmake similarity index 63% rename from AutomatedTesting/EngineFinder.cmake rename to AutomatedTesting/cmake/EngineFinder.cmake index 0a34a43b77..15b96eb8a9 100644 --- a/AutomatedTesting/EngineFinder.cmake +++ b/AutomatedTesting/cmake/EngineFinder.cmake @@ -1,3 +1,4 @@ +# {BEGIN_LICENSE} # # Copyright (c) Contributors to the Open 3D Engine Project. # For complete copyright and license terms please see the LICENSE at the root of this distribution. @@ -5,18 +6,34 @@ # SPDX-License-Identifier: Apache-2.0 OR MIT # # +# {END_LICENSE} # This file is copied during engine registration. Edits to this file will be lost next # time a registration happens. include_guard() # Read the engine name from the project_json file -file(READ ${CMAKE_CURRENT_LIST_DIR}/project.json project_json) -set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${CMAKE_CURRENT_LIST_DIR}/project.json) +file(READ ${CMAKE_CURRENT_SOURCE_DIR}/project.json project_json) +set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/project.json) string(JSON LY_ENGINE_NAME_TO_USE ERROR_VARIABLE json_error GET ${project_json} engine) if(json_error) - message(FATAL_ERROR "Unable to read key 'engine' from 'project.json', error: ${json_error}") + message(FATAL_ERROR "Unable to read key 'engine' from 'project.json'\nError: ${json_error}") +endif() + +if(CMAKE_MODULE_PATH) + foreach(module_path ${CMAKE_MODULE_PATH}) + if(EXISTS ${module_path}/Findo3de.cmake) + file(READ ${module_path}/../engine.json engine_json) + string(JSON engine_name ERROR_VARIABLE json_error GET ${engine_json} engine_name) + if(json_error) + message(FATAL_ERROR "Unable to read key 'engine_name' from 'engine.json'\nError: ${json_error}") + endif() + if(LY_ENGINE_NAME_TO_USE STREQUAL engine_name) + return() # Engine being forced through CMAKE_MODULE_PATH + endif() + endif() + endforeach() endif() if(DEFINED ENV{USERPROFILE} AND EXISTS $ENV{USERPROFILE}) @@ -25,6 +42,11 @@ else() set(manifest_path $ENV{HOME}/.o3de/o3de_manifest.json) # Unix endif() +set(registration_error [=[ +Engine registration is required before configuring a project. +Run 'scripts/o3de register --this-engine' from the engine root. +]=]) + # Read the ~/.o3de/o3de_manifest.json file and look through the 'engines_path' object. # Find a key that matches LY_ENGINE_NAME_TO_USE and use that as the engine path. if(EXISTS ${manifest_path}) @@ -33,36 +55,38 @@ if(EXISTS ${manifest_path}) string(JSON engines_path_count ERROR_VARIABLE json_error LENGTH ${manifest_json} engines_path) if(json_error) - message(FATAL_ERROR "Unable to read key 'engines_path' from '${manifest_path}', error: ${json_error}") + message(FATAL_ERROR "Unable to read key 'engines_path' from '${manifest_path}'\nError: ${json_error}\n${registration_error}") endif() string(JSON engines_path_type ERROR_VARIABLE json_error TYPE ${manifest_json} engines_path) if(json_error OR NOT ${engines_path_type} STREQUAL "OBJECT") - message(FATAL_ERROR "Type of 'engines_path' in '${manifest_path}' is not a JSON Object, error: ${json_error}") + message(FATAL_ERROR "Type of 'engines_path' in '${manifest_path}' is not a JSON Object\nError: ${json_error}") endif() math(EXPR engines_path_count "${engines_path_count}-1") foreach(engine_path_index RANGE ${engines_path_count}) string(JSON engine_name ERROR_VARIABLE json_error MEMBER ${manifest_json} engines_path ${engine_path_index}) if(json_error) - message(FATAL_ERROR "Unable to read 'engines_path/${engine_path_index}' from '${manifest_path}', error: ${json_error}") + message(FATAL_ERROR "Unable to read 'engines_path/${engine_path_index}' from '${manifest_path}'\nError: ${json_error}") endif() if(LY_ENGINE_NAME_TO_USE STREQUAL engine_name) string(JSON engine_path ERROR_VARIABLE json_error GET ${manifest_json} engines_path ${engine_name}) if(json_error) - message(FATAL_ERROR "Unable to read value from 'engines_path/${engine_name}', error: ${json_error}") + message(FATAL_ERROR "Unable to read value from 'engines_path/${engine_name}'\nError: ${json_error}") endif() if(engine_path) list(APPEND CMAKE_MODULE_PATH "${engine_path}/cmake") - break() + return() endif() endif() endforeach() + + message(FATAL_ERROR "The project.json uses engine name '${LY_ENGINE_NAME_TO_USE}' but no engine with that name has been registered.\n${registration_error}") else() # If the user is passing CMAKE_MODULE_PATH we assume thats where we will find the engine if(NOT CMAKE_MODULE_PATH) - message(FATAL_ERROR "Engine registration is required before configuring a project. Please register an engine by running 'scripts/o3de register --this-engine'") + message(FATAL_ERROR "O3DE Manifest file not found.\n${registration_error}") endif() endif() diff --git a/Templates/DefaultProject/Template/cmake/Platform/Linux/CompilerSettings.cmake b/AutomatedTesting/cmake/Platform/Linux/CompilerSettings_linux.cmake similarity index 100% rename from Templates/DefaultProject/Template/cmake/Platform/Linux/CompilerSettings.cmake rename to AutomatedTesting/cmake/Platform/Linux/CompilerSettings_linux.cmake diff --git a/Code/Framework/AzCore/AzCore/Serialization/AZStdContainers.inl b/Code/Framework/AzCore/AzCore/Serialization/AZStdContainers.inl index 92d295cf08..a633af22da 100644 --- a/Code/Framework/AzCore/AzCore/Serialization/AZStdContainers.inl +++ b/Code/Framework/AzCore/AzCore/Serialization/AZStdContainers.inl @@ -89,7 +89,7 @@ namespace AZ template AZStd::enable_if_t::value> InitializeDefaultIfPodType(T& t) { - t = {}; + t = T{}; } template diff --git a/Code/Framework/AzToolsFramework/AzToolsFramework/API/PythonLoader.h b/Code/Framework/AzToolsFramework/AzToolsFramework/API/PythonLoader.h index 29125667d6..6acc160ddc 100644 --- a/Code/Framework/AzToolsFramework/AzToolsFramework/API/PythonLoader.h +++ b/Code/Framework/AzToolsFramework/AzToolsFramework/API/PythonLoader.h @@ -19,7 +19,7 @@ namespace AzToolsFramework::EmbeddedPython ~PythonLoader(); private: - void* m_embeddedLibPythonHandle{ nullptr }; + [[maybe_unused]] void* m_embeddedLibPythonHandle{ nullptr }; }; } // namespace AzToolsFramework::EmbeddedPython diff --git a/Templates/DefaultProject/Template/cmake/CompilerSettings.cmake b/Templates/DefaultProject/Template/cmake/CompilerSettings.cmake index cf6614e4a5..60bda1d45b 100644 --- a/Templates/DefaultProject/Template/cmake/CompilerSettings.cmake +++ b/Templates/DefaultProject/Template/cmake/CompilerSettings.cmake @@ -9,5 +9,5 @@ # File to tweak compiler settings before compiler detection happens (before project() is called) # We dont have PAL enabled at this point, so we can only use pure-CMake variables if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Linux") - include(cmake/Platform/${CMAKE_HOST_SYSTEM_NAME}/CompilerSettings.cmake) + include(cmake/Platform/Linux/CompilerSettings_linux.cmake) endif() diff --git a/Templates/MinimalProject/Template/cmake/Platform/Linux/CompilerSettings.cmake b/Templates/DefaultProject/Template/cmake/Platform/Linux/CompilerSettings_linux.cmake similarity index 100% rename from Templates/MinimalProject/Template/cmake/Platform/Linux/CompilerSettings.cmake rename to Templates/DefaultProject/Template/cmake/Platform/Linux/CompilerSettings_linux.cmake diff --git a/Templates/DefaultProject/template.json b/Templates/DefaultProject/template.json index a36926f632..fcffafcb34 100644 --- a/Templates/DefaultProject/template.json +++ b/Templates/DefaultProject/template.json @@ -193,8 +193,8 @@ "isOptional": false }, { - "file": "cmake/Platform/Linux/CompilerSettings.cmake", - "origin": "cmake/Platform/Linux/CompilerSettings.cmake", + "file": "cmake/Platform/Linux/CompilerSettings_linux.cmake", + "origin": "cmake/Platform/Linux/CompilerSettings_linux.cmake", "isTemplated": false, "isOptional": false }, diff --git a/Templates/MinimalProject/Template/cmake/CompilerSettings.cmake b/Templates/MinimalProject/Template/cmake/CompilerSettings.cmake index cf6614e4a5..60bda1d45b 100644 --- a/Templates/MinimalProject/Template/cmake/CompilerSettings.cmake +++ b/Templates/MinimalProject/Template/cmake/CompilerSettings.cmake @@ -9,5 +9,5 @@ # File to tweak compiler settings before compiler detection happens (before project() is called) # We dont have PAL enabled at this point, so we can only use pure-CMake variables if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Linux") - include(cmake/Platform/${CMAKE_HOST_SYSTEM_NAME}/CompilerSettings.cmake) + include(cmake/Platform/Linux/CompilerSettings_linux.cmake) endif() diff --git a/cmake/Platform/Linux/CompilerSettings.cmake b/Templates/MinimalProject/Template/cmake/Platform/Linux/CompilerSettings_linux.cmake similarity index 100% rename from cmake/Platform/Linux/CompilerSettings.cmake rename to Templates/MinimalProject/Template/cmake/Platform/Linux/CompilerSettings_linux.cmake diff --git a/Templates/MinimalProject/template.json b/Templates/MinimalProject/template.json index 4260e71527..7d6a4f9b94 100644 --- a/Templates/MinimalProject/template.json +++ b/Templates/MinimalProject/template.json @@ -185,8 +185,8 @@ "isOptional": false }, { - "file": "cmake/Platform/Linux/CompilerSettings.cmake", - "origin": "cmake/Platform/Linux/CompilerSettings.cmake", + "file": "cmake/Platform/Linux/CompilerSettings_linux.cmake", + "origin": "cmake/Platform/Linux/CompilerSettings_linux.cmake", "isTemplated": false, "isOptional": false }, diff --git a/cmake/3rdPartyPackages.cmake b/cmake/3rdPartyPackages.cmake index efe67b4d24..a3f15bdb22 100644 --- a/cmake/3rdPartyPackages.cmake +++ b/cmake/3rdPartyPackages.cmake @@ -7,7 +7,7 @@ include_guard() -include(cmake/LySet.cmake) +include(${LY_ROOT_FOLDER}/cmake/LySet.cmake) # OVERVIEW: # this is the Open 3D Engine Package system. @@ -80,10 +80,7 @@ macro(ly_package_message) endif() endmacro() -file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/packages) - -include(cmake/LYPackage_S3Downloader.cmake) - +include(${LY_ROOT_FOLDER}/cmake/LYPackage_S3Downloader.cmake) # Attempts one time to download a file. # sets should_retry to true if the caller should retry due to an intermittent problem @@ -711,11 +708,11 @@ if (NOT CMAKE_SCRIPT_MODE_FILE) # include the built in 3rd party packages that are for every platform. # you can put your package associations anywhere, but this provides # a good starting point. - include(cmake/3rdParty/BuiltInPackages.cmake) + include(${LY_ROOT_FOLDER}/cmake/3rdParty/BuiltInPackages.cmake) endif() if(PAL_TRAIT_BUILD_HOST_TOOLS) - include(cmake/LYWrappers.cmake) + include(${LY_ROOT_FOLDER}/cmake/LYWrappers.cmake) # Importing this globally to handle AUTOMOC, AUTOUIC, AUTORCC ly_parse_third_party_dependencies(3rdParty::Qt) endif() diff --git a/cmake/CompilerSettings.cmake b/cmake/CompilerSettings.cmake index cf6614e4a5..60bda1d45b 100644 --- a/cmake/CompilerSettings.cmake +++ b/cmake/CompilerSettings.cmake @@ -9,5 +9,5 @@ # File to tweak compiler settings before compiler detection happens (before project() is called) # We dont have PAL enabled at this point, so we can only use pure-CMake variables if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Linux") - include(cmake/Platform/${CMAKE_HOST_SYSTEM_NAME}/CompilerSettings.cmake) + include(cmake/Platform/Linux/CompilerSettings_linux.cmake) endif() diff --git a/cmake/Packaging.cmake b/cmake/Packaging.cmake index 6a1e2d8cef..d716efb225 100644 --- a/cmake/Packaging.cmake +++ b/cmake/Packaging.cmake @@ -24,18 +24,23 @@ number will automatically appended as '/'. If LY_INSTALLER_AUTO_ full URL format will be: //" ) -set(LY_INSTALLER_UPLOAD_URL "" CACHE STRING -"Base URL used to upload the installer artifacts after generation, the host target and version number \ -will automatically appended as '/'. If LY_INSTALLER_AUTO_GEN_TAG is set, the full URL \ -format will be: //. Can also be set via LY_INSTALLER_UPLOAD_URL environment \ -variable. Currently only accepts S3 URLs e.g. s3:///" +set(CPACK_UPLOAD_URL "" CACHE STRING +"URL used to upload the installer artifacts after generation, the host target and version number \ +will automatically appended as '/'. If LY_INSTALLER_AUTO_GEN_TAG is set, the full URL \ +format will be: //. Currently only accepts S3 URLs e.g. s3:///" ) -set(LY_INSTALLER_AWS_PROFILE "" CACHE STRING -"AWS CLI profile for uploading artifacts. Can also be set via LY_INSTALLER_AWS_PROFILE environment variable." +set(CPACK_AWS_PROFILE "" CACHE STRING +"AWS CLI profile for uploading artifacts." ) +set(CPACK_THREADS 0) set(CPACK_DESIRED_CMAKE_VERSION 3.20.2) +if(${CPACK_DESIRED_CMAKE_VERSION} VERSION_LESS ${CMAKE_MINIMUM_REQUIRED_VERSION}) + message(FATAL_ERROR + "The desired version of CMake to be included in the package is " + "below the minimum required version of CMake to run") +endif() # set all common cpack variable overrides first so they can be accessible via configure_file # when the platform specific settings are applied below. additionally, any variable with @@ -44,15 +49,16 @@ set(CPACK_DESIRED_CMAKE_VERSION 3.20.2) set(CPACK_PACKAGE_NAME "${PROJECT_NAME}") set(CPACK_PACKAGE_FULL_NAME "Open3D Engine") set(CPACK_PACKAGE_VENDOR "O3DE Binary Project a Series of LF Projects, LLC") +set(CPACK_PACKAGE_CONTACT "info@o3debinaries.org") set(CPACK_PACKAGE_VERSION "${LY_VERSION_STRING}") set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "Installation Tool") string(TOLOWER "${CPACK_PACKAGE_NAME}_${CPACK_PACKAGE_VERSION}" CPACK_PACKAGE_FILE_NAME) set(DEFAULT_LICENSE_NAME "Apache-2.0") -set(DEFAULT_LICENSE_FILE "${CMAKE_CURRENT_SOURCE_DIR}/LICENSE.txt") -set(CPACK_RESOURCE_FILE_LICENSE ${DEFAULT_LICENSE_FILE}) +set(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_CURRENT_SOURCE_DIR}/LICENSE.txt") +set(CPACK_RESOURCE_FILE_README "${CMAKE_CURRENT_SOURCE_DIR}/README.md") set(CPACK_LICENSE_URL ${LY_INSTALLER_LICENSE_URL}) set(CPACK_PACKAGE_INSTALL_DIRECTORY "${CPACK_PACKAGE_NAME}/${CPACK_PACKAGE_VERSION}") @@ -60,6 +66,7 @@ set(CPACK_PACKAGE_INSTALL_DIRECTORY "${CPACK_PACKAGE_NAME}/${CPACK_PACKAGE_VERSI # neither of the SOURCE_DIR variables equate to anything during execution of pre/post build scripts set(CPACK_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/cmake) set(CPACK_BINARY_DIR ${CMAKE_BINARY_DIR}/_CPack) # to match other CPack out dirs +set(CPACK_OUTPUT_FILE_PREFIX CPackUploads) # this config file allows the dynamic setting of cpack variables at cpack-time instead of cmake configure set(CPACK_PROJECT_CONFIG_FILE ${CPACK_SOURCE_DIR}/PackagingConfig.cmake) @@ -74,132 +81,37 @@ if(NOT CPACK_GENERATOR) return() endif() -if(${CPACK_DESIRED_CMAKE_VERSION} VERSION_LESS ${CMAKE_MINIMUM_REQUIRED_VERSION}) - message(FATAL_ERROR - "The desired version of CMake to be included in the package is " - "below the minimum required version of CMake to run") -endif() - -# pull down the desired copy of CMake so it can be included in the package +# We will download the desired copy of CMake so it can be included in the package, we defer the downloading +# to the install process, to do so we generate a script that will perform the download and execute such script +# during the install process (before packaging) if(NOT (CPACK_CMAKE_PACKAGE_FILE AND CPACK_CMAKE_PACKAGE_HASH)) message(FATAL_ERROR "Packaging is missing one or more following properties required to include CMake: " " CPACK_CMAKE_PACKAGE_FILE, CPACK_CMAKE_PACKAGE_HASH") endif() -set(_cmake_package_dest ${CPACK_BINARY_DIR}/${CPACK_CMAKE_PACKAGE_FILE}) +# We download it to a different location because CPACK_PACKAGING_INSTALL_PREFIX will be removed during +# cpack generation. CPACK_BINARY_DIR persists across cpack invocations +set(LY_CMAKE_PACKAGE_DOWNLOAD_PATH ${CPACK_BINARY_DIR}/${CPACK_CMAKE_PACKAGE_FILE}) -if(EXISTS ${_cmake_package_dest}) - file(SHA256 ${_cmake_package_dest} hash_of_downloaded_file) - if (NOT "${hash_of_downloaded_file}" STREQUAL "${CPACK_CMAKE_PACKAGE_HASH}") - message(STATUS "CMake ${CPACK_DESIRED_CMAKE_VERSION} found at ${_cmake_package_dest} but expected hash missmatches, re-downloading...") - file(REMOVE ${_cmake_package_dest}) - else() - message(STATUS "CMake ${CPACK_DESIRED_CMAKE_VERSION} found") - endif() -endif() -if(NOT EXISTS ${_cmake_package_dest}) - # download it - string(REPLACE "." ";" _version_componets "${CPACK_DESIRED_CMAKE_VERSION}") - list(GET _version_componets 0 _major_version) - list(GET _version_componets 1 _minor_version) - - set(_url_version_tag "v${_major_version}.${_minor_version}") - set(_package_url "https://cmake.org/files/${_url_version_tag}/${CPACK_CMAKE_PACKAGE_FILE}") - - message(STATUS "Downloading CMake ${CPACK_DESIRED_CMAKE_VERSION} for packaging...") - download_file( - URL ${_package_url} - TARGET_FILE ${_cmake_package_dest} - EXPECTED_HASH ${CPACK_CMAKE_PACKAGE_HASH} - RESULTS _results - ) - list(GET _results 0 _status_code) - - if (${_status_code} EQUAL 0 AND EXISTS ${_cmake_package_dest}) - message(STATUS "CMake ${CPACK_DESIRED_CMAKE_VERSION} found") - else() - file(REMOVE ${_cmake_package_dest}) - list(REMOVE_AT _results 0) - - set(_error_message "An error occurred, code ${_status_code}. URL ${_package_url} - ${_results}") - - if(${_status_code} EQUAL 1) - string(APPEND _error_message - " Please double check the CPACK_CMAKE_PACKAGE_FILE and " - "CPACK_CMAKE_PACKAGE_HASH properties before trying again.") - endif() - - message(FATAL_ERROR ${_error_message}) - endif() -endif() - -ly_install(FILES ${_cmake_package_dest} - DESTINATION ./Tools/Redistributables/CMake +configure_file(${LY_ROOT_FOLDER}/cmake/Packaging/CMakeDownload.cmake.in + ${CPACK_BINARY_DIR}/CMakeDownload.cmake + @ONLY +) +ly_install(SCRIPT ${CPACK_BINARY_DIR}/CMakeDownload.cmake + COMPONENT ${CMAKE_INSTALL_DEFAULT_COMPONENT_NAME} +) +ly_install(FILES ${LY_CMAKE_PACKAGE_DOWNLOAD_PATH} + DESTINATION Tools/Redistributables/CMake COMPONENT ${CMAKE_INSTALL_DEFAULT_COMPONENT_NAME} ) -# the version string and git tags are intended to be synchronized so it should be safe to use that instead -# of directly calling into git which could get messy in certain scenarios -if(${CPACK_PACKAGE_VERSION} VERSION_GREATER "0.0.0.0") - set(_3rd_party_license_filename NOTICES.txt) - - set(_3rd_party_license_url "https://raw.githubusercontent.com/o3de/3p-package-source/${CPACK_PACKAGE_VERSION}/${_3rd_party_license_filename}") - set(_3rd_party_license_dest ${CPACK_BINARY_DIR}/${_3rd_party_license_filename}) - - # use the plain file downloader as we don't have the file hash available and using a dummy will - # delete the file once it fails hash verification - file(DOWNLOAD - ${_3rd_party_license_url} - ${_3rd_party_license_dest} - STATUS _status - TLS_VERIFY ON - ) - list(POP_FRONT _status _status_code) - - if (${_status_code} EQUAL 0 AND EXISTS ${_3rd_party_license_dest}) - ly_install(FILES ${_3rd_party_license_dest} - DESTINATION . - COMPONENT ${CMAKE_INSTALL_DEFAULT_COMPONENT_NAME} - ) - else() - file(REMOVE ${_3rd_party_license_dest}) - message(FATAL_ERROR "Failed to acquire the 3rd Party license manifest file at ${_3rd_party_license_url}. Error: ${_status}") - endif() -endif() - -# checks for and removes trailing slash -function(strip_trailing_slash in_url out_url) - string(LENGTH ${in_url} _url_length) - MATH(EXPR _url_length "${_url_length}-1") - - string(SUBSTRING ${in_url} 0 ${_url_length} _clean_url) - if("${in_url}" STREQUAL "${_clean_url}/") - set(${out_url} ${_clean_url} PARENT_SCOPE) - else() - set(${out_url} ${in_url} PARENT_SCOPE) - endif() -endfunction() - -if(NOT LY_INSTALLER_UPLOAD_URL AND DEFINED ENV{LY_INSTALLER_UPLOAD_URL}) - set(LY_INSTALLER_UPLOAD_URL $ENV{LY_INSTALLER_UPLOAD_URL}) -endif() - -if(LY_INSTALLER_UPLOAD_URL) - ly_is_s3_url(${LY_INSTALLER_UPLOAD_URL} _is_s3_bucket) - if(NOT _is_s3_bucket) - message(FATAL_ERROR "Only S3 installer uploading is supported at this time") - endif() - - if (LY_INSTALLER_AWS_PROFILE) - set(CPACK_AWS_PROFILE ${LY_INSTALLER_AWS_PROFILE}) - elseif (DEFINED ENV{LY_INSTALLER_AWS_PROFILE}) - set(CPACK_AWS_PROFILE $ENV{LY_INSTALLER_AWS_PROFILE}) - endif() - - strip_trailing_slash(${LY_INSTALLER_UPLOAD_URL} LY_INSTALLER_UPLOAD_URL) - set(CPACK_UPLOAD_URL ${LY_INSTALLER_UPLOAD_URL}) -endif() +# Set common CPACK variables to all platforms/generators +set(CPACK_STRIP_FILES TRUE) # always strip symbols on packaging +set(CPACK_PACKAGE_CHECKSUM SHA256) # Generate checksum file +set(CPACK_PRE_BUILD_SCRIPTS ${pal_dir}/PackagingPreBuild_${PAL_HOST_PLATFORM_NAME_LOWERCASE}.cmake) +set(CPACK_POST_BUILD_SCRIPTS ${pal_dir}/PackagingPostBuild_${PAL_HOST_PLATFORM_NAME_LOWERCASE}.cmake) +set(CPACK_LY_PYTHON_CMD ${LY_PYTHON_CMD}) # IMPORTANT: required to be included AFTER setting all property overrides include(CPack REQUIRED) @@ -268,13 +180,26 @@ foreach(external_dir ${LY_INSTALL_EXTERNAL_BUILD_DIRS}) ) endforeach() +# checks for and removes trailing slash +function(strip_trailing_slash in_url out_url) + string(LENGTH ${in_url} _url_length) + MATH(EXPR _url_length "${_url_length}-1") + + string(SUBSTRING ${in_url} 0 ${_url_length} _clean_url) + if("${in_url}" STREQUAL "${_clean_url}/") + set(${out_url} ${_clean_url} PARENT_SCOPE) + else() + set(${out_url} ${in_url} PARENT_SCOPE) + endif() +endfunction() + if(LY_INSTALLER_DOWNLOAD_URL) strip_trailing_slash(${LY_INSTALLER_DOWNLOAD_URL} LY_INSTALLER_DOWNLOAD_URL) # this will set the following variables: CPACK_DOWNLOAD_SITE, CPACK_DOWNLOAD_ALL, and CPACK_UPLOAD_DIRECTORY (local) cpack_configure_downloads( ${LY_INSTALLER_DOWNLOAD_URL} - UPLOAD_DIRECTORY ${CMAKE_BINARY_DIR}/_CPack_Uploads # to match the _CPack_Packages directory + UPLOAD_DIRECTORY ${CMAKE_BINARY_DIR}/CPackUploads # to match the _CPack_Packages directory ALL ) endif() diff --git a/cmake/Packaging/CMakeDownload.cmake.in b/cmake/Packaging/CMakeDownload.cmake.in new file mode 100644 index 0000000000..e84611b354 --- /dev/null +++ b/cmake/Packaging/CMakeDownload.cmake.in @@ -0,0 +1,54 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set(LY_ROOT_FOLDER "@LY_ROOT_FOLDER@") +set(CMAKE_SCRIPT_MODE_FILE TRUE) +include(@LY_ROOT_FOLDER@/cmake/3rdPartyPackages.cmake) + +if(EXISTS "@LY_CMAKE_PACKAGE_DOWNLOAD_PATH@") + file(SHA256 "@LY_CMAKE_PACKAGE_DOWNLOAD_PATH@" hash_of_downloaded_file) + if (NOT "${hash_of_downloaded_file}" STREQUAL "@CPACK_CMAKE_PACKAGE_HASH@") + message(STATUS "CMake @CPACK_DESIRED_CMAKE_VERSION@ found at @LY_CMAKE_PACKAGE_DOWNLOAD_PATH@ but expected hash missmatches, re-downloading...") + file(REMOVE "@LY_CMAKE_PACKAGE_DOWNLOAD_PATH@") + else() + message(STATUS "CMake @CPACK_DESIRED_CMAKE_VERSION@ found") + endif() +endif() +if(NOT EXISTS "@LY_CMAKE_PACKAGE_DOWNLOAD_PATH@") + # download it + string(REPLACE "." ";" _version_components "@CPACK_DESIRED_CMAKE_VERSION@") + list(GET _version_components 0 _major_version) + list(GET _version_components 1 _minor_version) + + set(_url_version_tag "v${_major_version}.${_minor_version}") + set(_package_url "https://cmake.org/files/${_url_version_tag}/@CPACK_CMAKE_PACKAGE_FILE@") + + message(STATUS "Downloading CMake @CPACK_DESIRED_CMAKE_VERSION@ for packaging...") + download_file( + URL ${_package_url} + TARGET_FILE @LY_CMAKE_PACKAGE_DOWNLOAD_PATH@ + EXPECTED_HASH @CPACK_CMAKE_PACKAGE_HASH@ + RESULTS _results + ) + list(GET _results 0 _status_code) + + if (${_status_code} EQUAL 0 AND EXISTS "@LY_CMAKE_PACKAGE_DOWNLOAD_PATH@") + message(STATUS "CMake @CPACK_DESIRED_CMAKE_VERSION@ found") + else() + file(REMOVE "@LY_CMAKE_PACKAGE_DOWNLOAD_PATH@") + list(REMOVE_AT _results 0) + + set(_error_message "An error occurred, code ${_status_code}. URL ${_package_url} - ${_results}") + if(${_status_code} EQUAL 1) + string(APPEND _error_message + " Please double check the CPACK_CMAKE_PACKAGE_FILE and " + "CPACK_CMAKE_PACKAGE_HASH properties before trying again.") + endif() + message(FATAL_ERROR ${_error_message}) + endif() +endif() diff --git a/cmake/Platform/Common/Install_common.cmake b/cmake/Platform/Common/Install_common.cmake index 7904bbe611..1b41036311 100644 --- a/cmake/Platform/Common/Install_common.cmake +++ b/cmake/Platform/Common/Install_common.cmake @@ -569,9 +569,13 @@ function(ly_setup_runtime_dependencies) string(TOUPPER ${conf} UCONF) ly_install(CODE "function(ly_copy source_file target_directory) - cmake_path(GET source_file FILENAME file_name) - if(NOT EXISTS \${target_directory}/\${file_name}) - file(COPY \"\${source_file}\" DESTINATION \"\${target_directory}\" FILE_PERMISSIONS ${LY_COPY_PERMISSIONS}) + cmake_path(GET source_file FILENAME target_filename) + cmake_path(APPEND full_target_directory \"\$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}\" \"\${target_directory}\") + cmake_path(APPEND target_file \"\${full_target_directory}\" \"\${target_filename}\") + if(\"\${source_file}\" IS_NEWER_THAN \"\${target_file}\") + message(STATUS \"Copying \${source_file} to \${full_target_directory}...\") + file(COPY \"\${source_file}\" DESTINATION \"\${full_target_directory}\" FILE_PERMISSIONS ${LY_COPY_PERMISSIONS} FOLLOW_SYMLINK_CHAIN) + file(TOUCH_NOCREATE \"${target_file}\") endif() endfunction()" COMPONENT ${LY_INSTALL_PERMUTATION_COMPONENT}_${UCONF} @@ -596,12 +600,7 @@ endfunction()" endif() # runtime dependencies that need to be copied to the output - # Anywhere CMAKE_INSTALL_PREFIX is used, it has to be escaped so it is baked into the cmake_install.cmake script instead - # of baking the path. This is needed so `cmake --install --prefix ` works regardless of the CMAKE_INSTALL_PREFIX - # used to generate the solution. - # CMAKE_INSTALL_PREFIX is still used when building the INSTALL target - set(install_output_folder "\${CMAKE_INSTALL_PREFIX}/${runtime_output_directory}") - set(target_file_dir "${install_output_folder}/${target_runtime_output_subdirectory}") + set(target_file_dir "${runtime_output_directory}/${target_runtime_output_subdirectory}") ly_get_runtime_dependencies(runtime_dependencies ${target}) foreach(runtime_dependency ${runtime_dependencies}) unset(runtime_command) diff --git a/cmake/Platform/Common/PackagingPostBuild_common.cmake b/cmake/Platform/Common/PackagingPostBuild_common.cmake new file mode 100644 index 0000000000..6e3c7ddf0b --- /dev/null +++ b/cmake/Platform/Common/PackagingPostBuild_common.cmake @@ -0,0 +1,114 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +message(STATUS "Executing packaging postbuild...") + +# ly_is_s3_url +# if the given URL is a s3 url of thr form "s3://(stuff)" then sets +# the output_variable_name to TRUE otherwise unsets it. +function (ly_is_s3_url download_url output_variable_name) + if ("${download_url}" MATCHES "s3://.*") + set(${output_variable_name} TRUE PARENT_SCOPE) + else() + unset(${output_variable_name} PARENT_SCOPE) + endif() +endfunction() + +function(ly_upload_to_url in_url in_local_path in_file_regex) + + message(STATUS "Uploading ${in_local_path}/${in_file_regex} artifacts to ${CPACK_UPLOAD_URL}") + ly_is_s3_url(${in_url} _is_s3_bucket) + if(NOT _is_s3_bucket) + message(FATAL_ERROR "Only S3 installer uploading is supported at this time") + endif() + + # strip the scheme and extract the bucket/key prefix from the URL + string(REPLACE "s3://" "" _stripped_url ${in_url}) + string(REPLACE "/" ";" _tokens ${_stripped_url}) + + list(POP_FRONT _tokens _bucket) + string(JOIN "/" _prefix ${_tokens}) + + set(_extra_args [[{"ACL":"bucket-owner-full-control"}]]) + + file(TO_NATIVE_PATH "${LY_ROOT_FOLDER}/scripts/build/tools/upload_to_s3.py" _upload_script) + + set(_upload_command + ${CPACK_LY_PYTHON_CMD} -s + -u ${_upload_script} + --base_dir ${in_local_path} + --file_regex="${in_file_regex}" + --bucket ${_bucket} + --key_prefix ${_prefix} + --extra_args ${_extra_args} + ) + + if(CPACK_AWS_PROFILE) + list(APPEND _upload_command --profile ${CPACK_AWS_PROFILE}) + endif() + + execute_process( + COMMAND ${_upload_command} + RESULT_VARIABLE _upload_result + OUTPUT_VARIABLE _upload_output + ERROR_VARIABLE _upload_error + OUTPUT_STRIP_TRAILING_WHITESPACE + ) + + if (${_upload_result} EQUAL 0) + message(STATUS "Artifact uploading complete!") + else() + message(FATAL_ERROR "An error occurred uploading to s3.\n Output: ${_upload_output}\n\ Error: ${_upload_error}") + endif() +endfunction() + +function(ly_upload_to_latest in_url in_path) + + message(STATUS "Updating latest tagged build") + + # make sure we can extra the commit info from the URL first + string(REGEX MATCH "([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9a-zA-Z]+)" + commit_info ${in_url} + ) + if(NOT commit_info) + message(FATAL_ERROR "Failed to extract the build tag") + endif() + + # Create a temp directory where we are going to rename the file to take out the version + # and then upload it + set(temp_dir ${CPACK_BINARY_DIR}/temp) + if(NOT EXISTS ${temp_dir}) + file(MAKE_DIRECTORY ${temp_dir}) + endif() + file(COPY ${in_path} DESTINATION ${temp_dir}) + + cmake_path(GET in_path FILENAME in_path_filename) + string(REPLACE "_${CPACK_PACKAGE_VERSION}" "" non_versioned_in_path_filename ${in_path_filename}) + file(RENAME "${temp_dir}/${in_path_filename}" "${temp_dir}/${non_versioned_in_path_filename}") + + # include the commit info in a text file that will live next to the exe + set(_temp_info_file ${temp_dir}/build_tag.txt) + file(WRITE ${_temp_info_file} ${commit_info}) + + # update the URL and upload + string(REPLACE + ${commit_info} "Latest" + latest_upload_url ${in_url} + ) + + ly_upload_to_url( + ${latest_upload_url} + ${temp_dir} + ".*(${non_versioned_in_path_filename}|build_tag.txt)$" + ) + + # cleanup the temp files + file(REMOVE_RECURSE ${temp_dir}) + message(STATUS "Latest build update complete!") + +endfunction() \ No newline at end of file diff --git a/cmake/Platform/Common/PackagingPreBuild_common.cmake b/cmake/Platform/Common/PackagingPreBuild_common.cmake new file mode 100644 index 0000000000..e6b8a7796e --- /dev/null +++ b/cmake/Platform/Common/PackagingPreBuild_common.cmake @@ -0,0 +1,5 @@ +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# diff --git a/cmake/Platform/Common/runtime_dependencies_common.cmake.in b/cmake/Platform/Common/runtime_dependencies_common.cmake.in index 263111903e..8717710a3b 100644 --- a/cmake/Platform/Common/runtime_dependencies_common.cmake.in +++ b/cmake/Platform/Common/runtime_dependencies_common.cmake.in @@ -22,6 +22,7 @@ function(ly_copy source_file target_directory) endif() if((NOT source_file_size EQUAL target_file_size) OR "${source_file}" IS_NEWER_THAN "${target_file}") message(STATUS "Copying \"${source_file}\" to \"${target_directory}\"...") + file(MAKE_DIRECTORY "${full_target_directory}") file(COPY "${source_file}" DESTINATION "${target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) file(TOUCH_NOCREATE ${target_file}) endif() diff --git a/cmake/Platform/Linux/CompilerSettings_linux.cmake b/cmake/Platform/Linux/CompilerSettings_linux.cmake new file mode 100644 index 0000000000..9bb629c53b --- /dev/null +++ b/cmake/Platform/Linux/CompilerSettings_linux.cmake @@ -0,0 +1,34 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +if(NOT CMAKE_C_COMPILER AND NOT CMAKE_CXX_COMPILER AND NOT "$ENV{CC}" AND NOT "$ENV{CXX}") + set(path_search + /bin + /usr/bin + /usr/local/bin + /sbin + /usr/sbin + /usr/local/sbin + ) + list(TRANSFORM path_search APPEND "/clang-[0-9]*") + file(GLOB clang_versions ${path_search}) + if(clang_versions) + # Find and pick the highest installed version + list(SORT clang_versions COMPARE NATURAL) + list(GET clang_versions 0 clang_higher_version_path) + string(REGEX MATCH "clang-([0-9.]*)" clang_higher_version ${clang_higher_version_path}) + if(CMAKE_MATCH_1) + set(CMAKE_C_COMPILER clang-${CMAKE_MATCH_1}) + set(CMAKE_CXX_COMPILER clang++-${CMAKE_MATCH_1}) + else() + message(FATAL_ERROR "Clang not found, please install clang") + endif() + else() + message(FATAL_ERROR "Clang not found, please install clang") + endif() +endif() diff --git a/cmake/Platform/Linux/Install_linux.cmake b/cmake/Platform/Linux/Install_linux.cmake index 02baa6e61e..0f5494131a 100644 --- a/cmake/Platform/Linux/Install_linux.cmake +++ b/cmake/Platform/Linux/Install_linux.cmake @@ -9,14 +9,27 @@ #! ly_setup_runtime_dependencies_copy_function_override: Linux-specific copy function to handle RPATH fixes set(ly_copy_template [[ function(ly_copy source_file target_directory) - file(COPY "${source_file}" DESTINATION "${target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) - get_filename_component(target_filename_ext "${source_file}" LAST_EXT) - if("${source_file}" MATCHES "qt/plugins" AND "${target_filename_ext}" STREQUAL ".so") - get_filename_component(target_filename "${source_file}" NAME) - file(RPATH_CHANGE FILE "${target_directory}/${target_filename}" OLD_RPATH "\$ORIGIN/../../lib" NEW_RPATH "\$ORIGIN/..") - elseif("${source_file}" MATCHES "lrelease") - get_filename_component(target_filename "${source_file}" NAME) - file(RPATH_CHANGE FILE "${target_directory}/${target_filename}" OLD_RPATH "\$ORIGIN/../lib" NEW_RPATH "\$ORIGIN") + cmake_path(GET source_file FILENAME target_filename) + cmake_path(APPEND full_target_directory "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}" "${target_directory}") + cmake_path(APPEND target_file "${full_target_directory}" "${target_filename}") + if("${source_file}" IS_NEWER_THAN "${target_file}") + message(STATUS "Copying ${source_file} to ${full_target_directory}...") + file(MAKE_DIRECTORY "${full_target_directory}") + file(COPY "${source_file}" DESTINATION "${full_target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) + file(TOUCH_NOCREATE "${target_file}") + + # Special case for install + cmake_PATH(GET source_file EXTENSION target_filename_ext) + if("${target_filename_ext}" STREQUAL ".so") + if("${source_file}" MATCHES "qt/plugins") + file(RPATH_CHANGE FILE "${target_file}" OLD_RPATH "\$ORIGIN/../../lib" NEW_RPATH "\$ORIGIN/..") + endif() + if(CMAKE_INSTALL_DO_STRIP) + execute_process(COMMAND @CMAKE_STRIP@ "${target_file}") + endif() + elseif("${source_file}" MATCHES "lrelease") + file(RPATH_CHANGE FILE "${target_file}" OLD_RPATH "\$ORIGIN/../lib" NEW_RPATH "\$ORIGIN") + endif() endif() endfunction()]]) diff --git a/cmake/Platform/Linux/PAL_linux.cmake b/cmake/Platform/Linux/PAL_linux.cmake index 4941f143a6..b7fd062c8c 100644 --- a/cmake/Platform/Linux/PAL_linux.cmake +++ b/cmake/Platform/Linux/PAL_linux.cmake @@ -16,7 +16,7 @@ ly_set(PAL_TRAIT_BUILD_TESTS_SUPPORTED TRUE) ly_set(PAL_TRAIT_BUILD_UNITY_SUPPORTED TRUE) ly_set(PAL_TRAIT_BUILD_UNITY_EXCLUDE_EXTENSIONS) ly_set(PAL_TRAIT_BUILD_EXCLUDE_ALL_TEST_RUNS_FROM_IDE FALSE) -ly_set(PAL_TRAIT_BUILD_CPACK_SUPPORTED FALSE) +ly_set(PAL_TRAIT_BUILD_CPACK_SUPPORTED TRUE) ly_set(PAL_TRAIT_PROF_PIX_SUPPORTED FALSE) diff --git a/cmake/Platform/Linux/Packaging/postinst.in b/cmake/Platform/Linux/Packaging/postinst.in new file mode 100644 index 0000000000..c6c0ba228d --- /dev/null +++ b/cmake/Platform/Linux/Packaging/postinst.in @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set -o errexit # exit on the first failure encountered + +{ + if [[ ! -f "/usr/lib/x86_64-linux-gnu/libffi.so.6" ]]; then + sudo ln -s /usr/lib/x86_64-linux-gnu/libffi.so.7 /usr/lib/x86_64-linux-gnu/libffi.so.6 + fi + + pushd @CPACK_PACKAGING_INSTALL_PREFIX@ + python/get_python.sh + chown -R $SUDO_USER . + popd +} &> /dev/null # hide output diff --git a/cmake/Platform/Linux/Packaging/postrm.in b/cmake/Platform/Linux/Packaging/postrm.in new file mode 100644 index 0000000000..acda38bf1e --- /dev/null +++ b/cmake/Platform/Linux/Packaging/postrm.in @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set -o errexit # exit on the first failure encountered + +{ + pushd @CPACK_PACKAGING_INSTALL_PREFIX@ + popd +} &> /dev/null # hide output diff --git a/cmake/Platform/Linux/Packaging/prerm.in b/cmake/Platform/Linux/Packaging/prerm.in new file mode 100644 index 0000000000..5595d7010f --- /dev/null +++ b/cmake/Platform/Linux/Packaging/prerm.in @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set -o errexit # exit on the first failure encountered + +{ + # We dont remove this symlink that we potentially created because the user could have + # installed themselves. + #if [[ -L "/usr/lib/x86_64-linux-gnu/libffi.so.6" ]]; then + # sudo rm /usr/lib/x86_64-linux-gnu/libffi.so.6 + #fi + + pushd @CPACK_PACKAGING_INSTALL_PREFIX@ + # delete python downloads + rm -rf python/downloaded_packages python/runtime + popd +} &> /dev/null # hide output diff --git a/cmake/Platform/Linux/PackagingPostBuild_linux.cmake b/cmake/Platform/Linux/PackagingPostBuild_linux.cmake new file mode 100644 index 0000000000..d92ee908fd --- /dev/null +++ b/cmake/Platform/Linux/PackagingPostBuild_linux.cmake @@ -0,0 +1,62 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +file(REAL_PATH "${CPACK_SOURCE_DIR}/.." LY_ROOT_FOLDER) +include(${LY_ROOT_FOLDER}/cmake/Platform/Common/PackagingPostBuild_common.cmake) + +file(${CPACK_PACKAGE_CHECKSUM} ${CPACK_TOPLEVEL_DIRECTORY}/${CPACK_PACKAGE_FILE_NAME}.deb file_checksum) +file(WRITE ${CPACK_TOPLEVEL_DIRECTORY}/${CPACK_PACKAGE_FILE_NAME}.deb.sha256 "${file_checksum} ${CPACK_PACKAGE_FILE_NAME}.deb") + +if(CPACK_UPLOAD_URL) + + # use the internal default path if somehow not specified from cpack_configure_downloads + if(NOT CPACK_UPLOAD_DIRECTORY) + set(CPACK_UPLOAD_DIRECTORY ${CPACK_PACKAGE_DIRECTORY}/CPackUploads) + endif() + + # Copy the artifacts intended to be uploaded to a remote server into the folder specified + # through CPACK_UPLOAD_DIRECTORY. This mimics the same process cpack does natively for + # some other frameworks that have built-in online installer support. + message(STATUS "Copying packaging artifacts to upload directory...") + file(REMOVE_RECURSE ${CPACK_UPLOAD_DIRECTORY}) + file(GLOB _artifacts + "${CPACK_TOPLEVEL_DIRECTORY}/*.deb" + "${CPACK_TOPLEVEL_DIRECTORY}/*.sha256" + ) + file(COPY ${_artifacts} + DESTINATION ${CPACK_UPLOAD_DIRECTORY} + ) + message(STATUS "Artifacts copied to ${CPACK_UPLOAD_DIRECTORY}") + + # TODO: copy gpg file to CPACK_UPLOAD_DIRECTORY + + ly_upload_to_url( + ${CPACK_UPLOAD_URL} + ${CPACK_UPLOAD_DIRECTORY} + ".*(.deb|.gpg|.sha256)$" + ) + + # for auto tagged builds, we will also upload a second copy of just the boostrapper + # to a special "Latest" folder under the branch in place of the commit date/hash + if(CPACK_AUTO_GEN_TAG) + + set(latest_deb_package "${CPACK_UPLOAD_DIRECTORY}/${CPACK_PACKAGE_NAME}_latest.deb") + file(COPY_FILE + ${CPACK_UPLOAD_DIRECTORY}/${CPACK_PACKAGE_FILE_NAME}.deb + ${latest_deb_package} + ) + ly_upload_to_latest(${CPACK_UPLOAD_URL} ${latest_deb_package}) + + # TODO: upload gpg file to latest + + # Generate a checksum file for latest and upload it + set(latest_hash_file "${CPACK_UPLOAD_DIRECTORY}/${CPACK_PACKAGE_NAME}_latest.deb.sha256") + file(WRITE "${latest_hash_file}" "${file_checksum} ${CPACK_PACKAGE_NAME}_latest.deb") + ly_upload_to_latest(${CPACK_UPLOAD_URL} "${latest_hash_file}") + endif() +endif() diff --git a/cmake/Platform/Linux/PackagingPreBuild_linux.cmake b/cmake/Platform/Linux/PackagingPreBuild_linux.cmake new file mode 100644 index 0000000000..31dc393307 --- /dev/null +++ b/cmake/Platform/Linux/PackagingPreBuild_linux.cmake @@ -0,0 +1,16 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +file(REAL_PATH "${CPACK_SOURCE_DIR}/.." LY_ROOT_FOLDER) +include(${LY_ROOT_FOLDER}/cmake/Platform/Common/PackagingPreBuild_common.cmake) + +if(NOT CPACK_UPLOAD_URL) # Skip signing if we are not uploading the package + return() +endif() + +# TODO: do signing diff --git a/cmake/Platform/Linux/Packaging_linux.cmake b/cmake/Platform/Linux/Packaging_linux.cmake new file mode 100644 index 0000000000..2e178429c3 --- /dev/null +++ b/cmake/Platform/Linux/Packaging_linux.cmake @@ -0,0 +1,56 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set(CPACK_GENERATOR DEB) + +set(CPACK_PACKAGING_INSTALL_PREFIX "/opt/${CPACK_PACKAGE_NAME}/${LY_VERSION_STRING}") + +set(_cmake_package_name "cmake-${CPACK_DESIRED_CMAKE_VERSION}-linux-x86_64") +set(CPACK_CMAKE_PACKAGE_FILE "${_cmake_package_name}.tar.gz") +set(CPACK_CMAKE_PACKAGE_HASH "3f827544f9c82e74ddf5016461fdfcfea4ede58a26f82612f473bf6bfad8bfc2") + +# get all the package dependencies, extracted from scripts\build\build_node\Platform\Linux\package-list.ubuntu-focal.txt +set(package_dependencies + libffi7 + clang-12 + ninja-build + # Build Libraries + libglu1-mesa-dev # For Qt (GL dependency) + libxcb-xinerama0 # For Qt plugins at runtime + libxcb-xinput0 # For Qt plugins at runtime + libfontconfig1-dev # For Qt plugins at runtime + libcurl4-openssl-dev # For HttpRequestor + # libsdl2-dev # for WWise/Audio + libxcb-xkb-dev # For xcb keyboard input + libxkbcommon-x11-dev # For xcb keyboard input + libxkbcommon-dev # For xcb keyboard input + libxcb-xfixes0-dev # For mouse input + libxcb-xinput-dev # For mouse input + zlib1g-dev + mesa-common-dev +) +list(JOIN package_dependencies "," CPACK_DEBIAN_PACKAGE_DEPENDS) + +# Post-installation and pre/post removal scripts +configure_file("${LY_ROOT_FOLDER}/cmake/Platform/${PAL_PLATFORM_NAME}/Packaging/postinst.in" + "${CMAKE_BINARY_DIR}/cmake/Platform/${PAL_PLATFORM_NAME}/Packaging/postinst" + @ONLY +) +configure_file("${LY_ROOT_FOLDER}/cmake/Platform/${PAL_PLATFORM_NAME}/Packaging/prerm.in" + "${CMAKE_BINARY_DIR}/cmake/Platform/${PAL_PLATFORM_NAME}/Packaging/prerm" + @ONLY +) +configure_file("${LY_ROOT_FOLDER}/cmake/Platform/${PAL_PLATFORM_NAME}/Packaging/postrm.in" + "${CMAKE_BINARY_DIR}/cmake/Platform/${PAL_PLATFORM_NAME}/Packaging/postrm" + @ONLY +) +set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA + ${CMAKE_BINARY_DIR}/cmake/Platform/Linux/Packaging/postinst + ${CMAKE_BINARY_DIR}/cmake/Platform/Linux/Packaging/prerm + ${CMAKE_BINARY_DIR}/cmake/Platform/Linux/Packaging/postrm +) diff --git a/cmake/Platform/Linux/platform_linux_files.cmake b/cmake/Platform/Linux/platform_linux_files.cmake index fa5545cd26..d30959b8d2 100644 --- a/cmake/Platform/Linux/platform_linux_files.cmake +++ b/cmake/Platform/Linux/platform_linux_files.cmake @@ -10,11 +10,19 @@ set(FILES ../Common/Configurations_common.cmake ../Common/Clang/Configurations_clang.cmake ../Common/Install_common.cmake + ../Common/PackagingPostBuild_common.cmake + ../Common/PackagingPreBuild_common.cmake + CompilerSettings_linux.cmake Configurations_linux.cmake Install_linux.cmake LYTestWrappers_linux.cmake LYWrappers_linux.cmake + Packaging_linux.cmake + PackagingPostBuild_linux.cmake + PackagingPreBuild_linux.cmake PAL_linux.cmake PALDetection_linux.cmake RPathChange.cmake + runtime_dependencies_linux.cmake.in + RuntimeDependencies_linux.cmake ) diff --git a/cmake/Platform/Linux/runtime_dependencies_linux.cmake.in b/cmake/Platform/Linux/runtime_dependencies_linux.cmake.in index b7ecc57737..4ccf123e27 100644 --- a/cmake/Platform/Linux/runtime_dependencies_linux.cmake.in +++ b/cmake/Platform/Linux/runtime_dependencies_linux.cmake.in @@ -22,6 +22,7 @@ function(ly_copy source_file target_directory) endif() if((NOT source_file_size EQUAL target_file_size) OR "${source_file}" IS_NEWER_THAN "${target_file}") message(STATUS "Copying \"${source_file}\" to \"${target_directory}\"...") + file(MAKE_DIRECTORY "${full_target_directory}") file(COPY "${source_file}" DESTINATION "${target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) file(TOUCH_NOCREATE "${target_file}") diff --git a/cmake/Platform/Mac/InstallUtils_mac.cmake.in b/cmake/Platform/Mac/InstallUtils_mac.cmake.in index d73c4db459..89ce4a59f2 100644 --- a/cmake/Platform/Mac/InstallUtils_mac.cmake.in +++ b/cmake/Platform/Mac/InstallUtils_mac.cmake.in @@ -130,27 +130,36 @@ endfunction() function(ly_copy source_file target_directory) - if("${source_file}" MATCHES "\\.[Ff]ramework[^\\.]") + if("${source_file}" MATCHES "\\.[Ff]ramework") # fixup origin to copy the whole Framework folder string(REGEX REPLACE "(.*\\.[Ff]ramework).*" "\\1" source_file "${source_file}") endif() - get_filename_component(target_filename "${source_file}" NAME) - file(COPY "${source_file}" DESTINATION "${target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) - # Our Qt and Python frameworks aren't in the correct bundle format to be codesigned. - if("${target_filename}" MATCHES "(Qt[^.]+)\\.[Ff]ramework") - fixup_qt_framework(${CMAKE_MATCH_1} "${target_directory}/${target_filename}") - # For some Qt frameworks(QtCore), signing the bundle doesn't work because of bundle - # format issues(despite the fixes above). But once we've patched the framework above, there's - # only one executable that we need to sign so we can do it directly. - set(target_filename "${target_filename}/Versions/5/${CMAKE_MATCH_1}") - elseif("${target_filename}" MATCHES "Python.framework") - fixup_python_framework("${target_directory}/${target_filename}") - codesign_python_framework_binaries("${target_directory}/${target_filename}") + cmake_path(GET source_file FILENAME target_filename) + cmake_path(APPEND full_target_directory "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}" "${target_directory}") + cmake_path(APPEND target_file "${full_target_directory}" "${target_filename}") + + if("${source_file}" IS_NEWER_THAN "${target_file}") + message(STATUS "Copying ${source_file} to ${full_target_directory}...") + file(MAKE_DIRECTORY "${full_target_directory}") + file(COPY "${source_file}" DESTINATION "${full_target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) + file(TOUCH_NOCREATE "${target_file}") + + # Our Qt and Python frameworks aren't in the correct bundle format to be codesigned. + if("${target_filename}" MATCHES "(Qt[^.]+)\\.[Ff]ramework") + fixup_qt_framework(${CMAKE_MATCH_1} "${target_file}") + # For some Qt frameworks(QtCore), signing the bundle doesn't work because of bundle + # format issues(despite the fixes above). But once we've patched the framework above, there's + # only one executable that we need to sign so we can do it directly. + set(target_filename "${target_filename}/Versions/5/${CMAKE_MATCH_1}") + elseif("${target_filename}" MATCHES "Python.framework") + fixup_python_framework("${target_file}") + codesign_python_framework_binaries("${target_file}") + endif() + codesign_file("${target_file}" "none") endif() - codesign_file("${target_directory}/${target_filename}" "none") endfunction() diff --git a/cmake/Platform/Mac/PackagingPostBuild_mac.cmake b/cmake/Platform/Mac/PackagingPostBuild_mac.cmake new file mode 100644 index 0000000000..5fa3787c21 --- /dev/null +++ b/cmake/Platform/Mac/PackagingPostBuild_mac.cmake @@ -0,0 +1,10 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +file(REAL_PATH "${CPACK_SOURCE_DIR}/.." LY_ROOT_FOLDER) +include(${LY_ROOT_FOLDER}/cmake/Platform/Common/PackagingPostBuild_common.cmake) diff --git a/cmake/Platform/Mac/PackagingPreBuild_mac.cmake b/cmake/Platform/Mac/PackagingPreBuild_mac.cmake new file mode 100644 index 0000000000..1d30e21767 --- /dev/null +++ b/cmake/Platform/Mac/PackagingPreBuild_mac.cmake @@ -0,0 +1,10 @@ +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +file(REAL_PATH "${CPACK_SOURCE_DIR}/.." LY_ROOT_FOLDER) +include(${LY_ROOT_FOLDER}/cmake/Platform/Common/PackagingPreBuild_common.cmake) diff --git a/cmake/Platform/Mac/runtime_dependencies_mac.cmake.in b/cmake/Platform/Mac/runtime_dependencies_mac.cmake.in index 32bf777634..892a90640f 100644 --- a/cmake/Platform/Mac/runtime_dependencies_mac.cmake.in +++ b/cmake/Platform/Mac/runtime_dependencies_mac.cmake.in @@ -120,19 +120,28 @@ function(ly_copy source_file target_directory) file(MAKE_DIRECTORY "${target_directory}") endif() - if(NOT target_is_bundle) + set(is_framework FALSE) + if("${source_file}" MATCHES "\\.[Ff]ramework") + set(is_framework TRUE) + endif() + if(NOT is_framework) # if it is a bundle, there is no contention about the files in the destination, each bundle target will copy everything # we dont want these files to invalidate the bundle and cause a new signature file(LOCK ${target_file}.lock GUARD FUNCTION TIMEOUT 300) - endif() - file(SIZE "${source_file}" source_file_size) - if(EXISTS "${target_file}") - file(SIZE "${target_file}" target_file_size) + file(SIZE "${source_file}" source_file_size) + if(EXISTS "${target_file}") + file(SIZE "${target_file}" target_file_size) + else() + set(target_file_size 0) + endif() else() + set(source_file_size 0) set(target_file_size 0) endif() + if((NOT source_file_size EQUAL target_file_size) OR "${source_file}" IS_NEWER_THAN "${target_file}") message(STATUS "Copying \"${source_file}\" to \"${target_directory}\"...") + file(MAKE_DIRECTORY "${target_directory}") file(COPY "${source_file}" DESTINATION "${target_directory}" FILE_PERMISSIONS @LY_COPY_PERMISSIONS@ FOLLOW_SYMLINK_CHAIN) file(TOUCH_NOCREATE "${target_file}") set(anything_new TRUE PARENT_SCOPE) diff --git a/cmake/Platform/Windows/PackagingPostBuild.cmake b/cmake/Platform/Windows/PackagingPostBuild_windows.cmake similarity index 52% rename from cmake/Platform/Windows/PackagingPostBuild.cmake rename to cmake/Platform/Windows/PackagingPostBuild_windows.cmake index ac457bea87..0993135c23 100644 --- a/cmake/Platform/Windows/PackagingPostBuild.cmake +++ b/cmake/Platform/Windows/PackagingPostBuild_windows.cmake @@ -6,6 +6,9 @@ # # +file(REAL_PATH "${CPACK_SOURCE_DIR}/.." LY_ROOT_FOLDER) +include(${LY_ROOT_FOLDER}/cmake/Platform/Common/PackagingPostBuild_common.cmake) + # convert the path to a windows style path using string replace because TO_NATIVE_PATH # only works on real paths string(REPLACE "/" "\\" _fixed_package_install_dir ${CPACK_PACKAGE_INSTALL_DIRECTORY}) @@ -56,8 +59,7 @@ set(_light_command ) if(CPACK_UPLOAD_URL) # Skip signing if we are not uploading the package - file(REAL_PATH "${CPACK_SOURCE_DIR}/.." _root_path) - file(TO_NATIVE_PATH "${_root_path}/scripts/signer/Platform/Windows/signer.ps1" _sign_script) + file(TO_NATIVE_PATH "${LY_ROOT_FOLDER}/scripts/signer/Platform/Windows/signer.ps1" _sign_script) unset(_signing_command) find_program(_psiexec_path psexec.exe) @@ -111,16 +113,12 @@ if(NOT ${_light_result} EQUAL 0) message(FATAL_ERROR "An error occurred invoking light.exe. ${_light_errors}") endif() -file(COPY ${_bootstrap_output_file} - DESTINATION ${CPACK_PACKAGE_DIRECTORY} -) - -message(STATUS "Bootstrap installer generated to ${CPACK_PACKAGE_DIRECTORY}/${_bootstrap_filename}") +message(STATUS "Bootstrap installer generated to ${_bootstrap_output_file}") if(CPACK_UPLOAD_URL) # Skip signing if we are not uploading the package - message(STATUS "Signing bootstrap installer in ${CPACK_PACKAGE_DIRECTORY}") + message(STATUS "Signing bootstrap installer in ${_bootstrap_output_file}") execute_process( - COMMAND ${_signing_command} -bootstrapPath ${CPACK_PACKAGE_DIRECTORY}/${_bootstrap_filename} + COMMAND ${_signing_command} -bootstrapPath ${_bootstrap_output_file} RESULT_VARIABLE _signing_result ERROR_VARIABLE _signing_errors OUTPUT_VARIABLE _signing_output @@ -137,113 +135,32 @@ if(NOT CPACK_UPLOAD_DIRECTORY) set(CPACK_UPLOAD_DIRECTORY ${CPACK_PACKAGE_DIRECTORY}/CPackUploads) endif() -# copy the artifacts intended to be uploaded to a remote server into the folder specified -# through cpack_configure_downloads. this mimics the same process cpack does natively for +# Copy the artifacts intended to be uploaded to a remote server into the folder specified +# through CPACK_UPLOAD_DIRECTORY. This mimics the same process cpack does natively for # some other frameworks that have built-in online installer support. -message(STATUS "Copying installer artifacts to upload directory...") +message(STATUS "Copying packaging artifacts to upload directory...") file(REMOVE_RECURSE ${CPACK_UPLOAD_DIRECTORY}) -file(GLOB _artifacts "${_cpack_wix_out_dir}/*.msi" "${_cpack_wix_out_dir}/*.cab") +file(GLOB _artifacts + "${_cpack_wix_out_dir}/*.msi" + "${_cpack_wix_out_dir}/*.cab" + "${_cpack_wix_out_dir}/*.exe" +) file(COPY ${_artifacts} DESTINATION ${CPACK_UPLOAD_DIRECTORY} ) message(STATUS "Artifacts copied to ${CPACK_UPLOAD_DIRECTORY}") -if(NOT CPACK_UPLOAD_URL) - return() -endif() - -file(TO_NATIVE_PATH "${_cpack_wix_out_dir}" _cpack_wix_out_dir) -file(TO_NATIVE_PATH "${_root_path}/python/python.cmd" _python_cmd) -file(TO_NATIVE_PATH "${_root_path}/scripts/build/tools/upload_to_s3.py" _upload_script) - -function(upload_to_s3 in_url in_local_path in_file_regex) - - # strip the scheme and extract the bucket/key prefix from the URL - string(REPLACE "s3://" "" _stripped_url ${in_url}) - string(REPLACE "/" ";" _tokens ${_stripped_url}) - - list(POP_FRONT _tokens _bucket) - string(JOIN "/" _prefix ${_tokens}) - - set(_extra_args [[{"ACL":"bucket-owner-full-control"}]]) - - set(_upload_command - ${_python_cmd} -s - -u ${_upload_script} - --base_dir ${in_local_path} - --file_regex="${in_file_regex}" - --bucket ${_bucket} - --key_prefix ${_prefix} - --extra_args ${_extra_args} - ) - - if(CPACK_AWS_PROFILE) - list(APPEND _upload_command --profile ${CPACK_AWS_PROFILE}) - endif() - - execute_process( - COMMAND ${_upload_command} - RESULT_VARIABLE _upload_result - OUTPUT_VARIABLE _upload_output - OUTPUT_STRIP_TRAILING_WHITESPACE - ) - - if (NOT ${_upload_result} EQUAL 0) - message(FATAL_ERROR "An error occurred uploading to s3.\nOutput:\n${_upload_output}") - endif() -endfunction() - -message(STATUS "Uploading artifacts to ${CPACK_UPLOAD_URL}") -upload_to_s3( - ${CPACK_UPLOAD_URL} - ${_cpack_wix_out_dir} - ".*(cab|exe|msi)$" -) -message(STATUS "Artifact uploading complete!") - -# for auto tagged builds, we will also upload a second copy of just the boostrapper -# to a special "Latest" folder under the branch in place of the commit date/hash -if(CPACK_AUTO_GEN_TAG) - message(STATUS "Updating latest tagged build") - - # make sure we can extra the commit info from the URL first - string(REGEX MATCH "([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9a-zA-Z]+)" - _commit_info ${CPACK_UPLOAD_URL} - ) - if(NOT _commit_info) - message(FATAL_ERROR "Failed to extract the build tag") - endif() - - set(_temp_dir ${_cpack_wix_out_dir}/temp) - if(NOT EXISTS ${_temp_dir}) - file(MAKE_DIRECTORY ${_temp_dir}) - endif() - - # strip the version number form the exe name in the one uploaded to latest - string(TOLOWER "${CPACK_PACKAGE_NAME}_installer.exe" _non_versioned_exe) - set(_temp_exe_copy ${_temp_dir}/${_non_versioned_exe}) - - file(COPY ${_bootstrap_output_file} DESTINATION ${_temp_dir}) - file(RENAME "${_temp_dir}/${_bootstrap_filename}" ${_temp_exe_copy}) - - # include the commit info in a text file that will live next to the exe - set(_temp_info_file ${_temp_dir}/build_tag.txt) - file(WRITE ${_temp_info_file} ${_commit_info}) - - # update the URL and upload - string(REPLACE - ${_commit_info} "Latest" - _latest_upload_url ${CPACK_UPLOAD_URL} - ) - - upload_to_s3( - ${_latest_upload_url} - ${_temp_dir} - ".*(${_non_versioned_exe}|build_tag.txt)$" - ) - - # cleanup the temp files - file(REMOVE_RECURSE ${_temp_dir}) - - message(STATUS "Latest build update complete!") +if(CPACK_UPLOAD_URL) + file(TO_NATIVE_PATH "${_cpack_wix_out_dir}" _cpack_wix_out_dir) + ly_upload_to_url( + ${CPACK_UPLOAD_URL} + ${_cpack_wix_out_dir} + ".*(cab|exe|msi)$" + ) + + # for auto tagged builds, we will also upload a second copy of just the boostrapper + # to a special "Latest" folder under the branch in place of the commit date/hash + if(CPACK_AUTO_GEN_TAG) + ly_upload_to_latest(${CPACK_UPLOAD_URL} ${_bootstrap_output_file}) + endif() endif() diff --git a/cmake/Platform/Windows/PackagingPreBuild.cmake b/cmake/Platform/Windows/PackagingPreBuild_windows.cmake similarity index 93% rename from cmake/Platform/Windows/PackagingPreBuild.cmake rename to cmake/Platform/Windows/PackagingPreBuild_windows.cmake index 7f2eedf352..29995518da 100644 --- a/cmake/Platform/Windows/PackagingPreBuild.cmake +++ b/cmake/Platform/Windows/PackagingPreBuild_windows.cmake @@ -6,6 +6,9 @@ # # +file(REAL_PATH "${CPACK_SOURCE_DIR}/.." LY_ROOT_FOLDER) +include(${LY_ROOT_FOLDER}/cmake/Platform/Common/PackagingPreBuild_common.cmake) + if(NOT CPACK_UPLOAD_URL) # Skip signing if we are not uploading the package return() endif() diff --git a/cmake/Platform/Windows/Packaging_windows.cmake b/cmake/Platform/Windows/Packaging_windows.cmake index 7b8f5a6c19..f24e9dee1c 100644 --- a/cmake/Platform/Windows/Packaging_windows.cmake +++ b/cmake/Platform/Windows/Packaging_windows.cmake @@ -23,7 +23,6 @@ set(CPACK_WIX_ROOT ${LY_INSTALLER_WIX_ROOT}) set(CPACK_GENERATOR WIX) -set(CPACK_THREADS 0) set(_cmake_package_name "cmake-${CPACK_DESIRED_CMAKE_VERSION}-windows-x86_64") set(CPACK_CMAKE_PACKAGE_FILE "${_cmake_package_name}.zip") set(CPACK_CMAKE_PACKAGE_HASH "15a49e2ab81c1822d75b1b1a92f7863f58e31f6d6aac1c4103eef2b071be3112") @@ -105,47 +104,35 @@ set(_raw_text_license [[ #(loc.InstallEulaAcceptance) ]]) -# The offline installer generation will be a single monolithic MSI. The WIX burn tool for the bootstrapper EXE has a size limitation. -# So we will exclude the generation of the boostrapper EXE in the offline case. -if(LY_INSTALLER_DOWNLOAD_URL) - set(WIX_THEME_WARNING_IMAGE ${CPACK_SOURCE_DIR}/Platform/Windows/Packaging/warning.png) +set(WIX_THEME_WARNING_IMAGE ${CPACK_SOURCE_DIR}/Platform/Windows/Packaging/warning.png) - if(LY_INSTALLER_LICENSE_URL) - set(WIX_THEME_INSTALL_LICENSE_ELEMENTS ${_hyperlink_license}) - set(WIX_THEME_EULA_ACCEPTANCE_TEXT "<a href=\"#\">Terms of Use</a>") - else() - set(WIX_THEME_INSTALL_LICENSE_ELEMENTS ${_raw_text_license}) - set(WIX_THEME_EULA_ACCEPTANCE_TEXT "Terms of Use above") - endif() - - # theme ux file - configure_file( - "${CPACK_SOURCE_DIR}/Platform/Windows/Packaging/BootstrapperTheme.xml.in" - "${CPACK_BINARY_DIR}/BootstrapperTheme.xml" - @ONLY - ) - - # theme localization file - configure_file( - "${CPACK_SOURCE_DIR}/Platform/Windows/Packaging/BootstrapperTheme.wxl.in" - "${CPACK_BINARY_DIR}/BootstrapperTheme.wxl" - @ONLY - ) - - set(_embed_artifacts "no") - - # the bootstrapper will at the very least need a different upgrade guid - generate_wix_guid(CPACK_WIX_BOOTSTRAP_UPGRADE_GUID "${_guid_seed_base}_Bootstrap_UpgradeCode") - - set(CPACK_PRE_BUILD_SCRIPTS - ${CPACK_SOURCE_DIR}/Platform/Windows/PackagingPreBuild.cmake - ) - - set(CPACK_POST_BUILD_SCRIPTS - ${CPACK_SOURCE_DIR}/Platform/Windows/PackagingPostBuild.cmake - ) +if(LY_INSTALLER_LICENSE_URL) + set(WIX_THEME_INSTALL_LICENSE_ELEMENTS ${_hyperlink_license}) + set(WIX_THEME_EULA_ACCEPTANCE_TEXT "<a href=\"#\">Terms of Use</a>") +else() + set(WIX_THEME_INSTALL_LICENSE_ELEMENTS ${_raw_text_license}) + set(WIX_THEME_EULA_ACCEPTANCE_TEXT "Terms of Use above") endif() +# theme ux file +configure_file( + "${CPACK_SOURCE_DIR}/Platform/Windows/Packaging/BootstrapperTheme.xml.in" + "${CPACK_BINARY_DIR}/BootstrapperTheme.xml" + @ONLY +) + +# theme localization file +configure_file( + "${CPACK_SOURCE_DIR}/Platform/Windows/Packaging/BootstrapperTheme.wxl.in" + "${CPACK_BINARY_DIR}/BootstrapperTheme.wxl" + @ONLY +) + +set(_embed_artifacts "no") + +# the bootstrapper will at the very least need a different upgrade guid +generate_wix_guid(CPACK_WIX_BOOTSTRAP_UPGRADE_GUID "${_guid_seed_base}_Bootstrap_UpgradeCode") + set(CPACK_WIX_CANDLE_EXTRA_FLAGS -dCPACK_EMBED_ARTIFACTS=${_embed_artifacts} -dCPACK_CMAKE_PACKAGE_NAME=${_cmake_package_name} diff --git a/cmake/Platform/Windows/platform_windows_files.cmake b/cmake/Platform/Windows/platform_windows_files.cmake index fcc47ab6eb..984d985380 100644 --- a/cmake/Platform/Windows/platform_windows_files.cmake +++ b/cmake/Platform/Windows/platform_windows_files.cmake @@ -15,6 +15,8 @@ set(FILES ../Common/MSVC/VisualStudio_common.cmake ../Common/Install_common.cmake ../Common/LYWrappers_default.cmake + ../Common/PackagingPostBuild_common.cmake + ../Common/PackagingPreBuild_common.cmake ../Common/TargetIncludeSystemDirectories_unsupported.cmake Configurations_windows.cmake LYTestWrappers_windows.cmake @@ -23,7 +25,8 @@ set(FILES PALDetection_windows.cmake Install_windows.cmake Packaging_windows.cmake - PackagingPostBuild.cmake + PackagingPostBuild_windows.cmake + PackagingPreBuild_windows.cmake Packaging/Bootstrapper.wxs Packaging/BootstrapperTheme.wxl.in Packaging/BootstrapperTheme.xml.in diff --git a/scripts/build/Platform/Linux/build_config.json b/scripts/build/Platform/Linux/build_config.json index f485f43315..84d1726d47 100644 --- a/scripts/build/Platform/Linux/build_config.json +++ b/scripts/build/Platform/Linux/build_config.json @@ -37,7 +37,7 @@ "PARAMETERS": { "CONFIGURATION": "debug", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all" } @@ -53,7 +53,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all" } @@ -66,7 +66,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_UNITY_BUILD=FALSE -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_UNITY_BUILD=FALSE -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all" } @@ -80,7 +80,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all", "CTEST_OPTIONS": "-E (AutomatedTesting::Atom_TestSuite_Main|AutomatedTesting::PrefabTests|AutomatedTesting::TerrainTests_Main|Gem::EMotionFX.Editor.Tests) -L (SUITE_smoke|SUITE_main) -LE (REQUIRES_gpu) --no-tests=error", @@ -93,7 +93,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_UNITY_BUILD=FALSE -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_UNITY_BUILD=FALSE -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all", "CTEST_OPTIONS": "-E (AutomatedTesting::Atom_TestSuite_Main|AutomatedTesting::PrefabTests|AutomatedTesting::TerrainTests_Main|Gem::EMotionFX.Editor.Tests) -L (SUITE_smoke|SUITE_main) -LE (REQUIRES_gpu) --no-tests=error", @@ -110,7 +110,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "AssetProcessorBatch", "ASSET_PROCESSOR_BINARY": "bin/profile/AssetProcessorBatch", @@ -124,7 +124,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_UNITY_BUILD=FALSE -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_UNITY_BUILD=FALSE -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "AssetProcessorBatch", "ASSET_PROCESSOR_BINARY": "bin/profile/AssetProcessorBatch", @@ -142,7 +142,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "TEST_SUITE_periodic", "CTEST_OPTIONS": "-L (SUITE_periodic) --no-tests=error", @@ -162,7 +162,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4 -DO3DE_HOME_PATH=\"${WORKSPACE}/home\" -DO3DE_REGISTER_ENGINE_PATH=\"${WORKSPACE}/o3de\" -DO3DE_REGISTER_THIS_ENGINE=TRUE", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all", "CTEST_OPTIONS": "-L (SUITE_sandbox) --no-tests=error" @@ -178,7 +178,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "TEST_SUITE_benchmark", "CTEST_OPTIONS": "-L (SUITE_benchmark) --no-tests=error", @@ -195,7 +195,7 @@ "PARAMETERS": { "CONFIGURATION": "release", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all" } @@ -210,7 +210,7 @@ "PARAMETERS": { "CONFIGURATION": "release", "OUTPUT_DIRECTORY": "build/mono_linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_MONOLITHIC_GAME=TRUE -DLY_PARALLEL_LINK_JOBS=4", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_MONOLITHIC_GAME=TRUE -DLY_PARALLEL_LINK_JOBS=4", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "all" } @@ -221,10 +221,25 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4 -DLY_DISABLE_TEST_MODULES=TRUE", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4 -DLY_DISABLE_TEST_MODULES=TRUE", "CMAKE_TARGET": "install" } }, + "installer": { + "TAGS": [ + "nightly-clean", + "nightly-installer" + ], + "COMMAND": "build_installer_linux.sh", + "PARAMETERS": { + "CONFIGURATION": "profile", + "OUTPUT_DIRECTORY": "build/linux", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4 -DLY_DISABLE_TEST_MODULES=TRUE -DLY_VERSION_ENGINE_NAME=o3de-sdk", + "EXTRA_CMAKE_OPTIONS": "-DLY_INSTALLER_AUTO_GEN_TAG=TRUE -DLY_INSTALLER_DOWNLOAD_URL=${INSTALLER_DOWNLOAD_URL} -DLY_INSTALLER_LICENSE_URL=${INSTALLER_DOWNLOAD_URL}/license", + "CPACK_OPTIONS": "-D CPACK_UPLOAD_URL=${CPACK_UPLOAD_URL}", + "CMAKE_TARGET": "all" + } + }, "install_profile_pipe": { "TAGS": [ "nightly-incremental", @@ -254,7 +269,7 @@ "COMMAND_CWD": "${WORKSPACE}/${PROJECT_REPOSITORY_NAME}", "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/linux", - "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DCMAKE_C_COMPILER=clang-12 -DCMAKE_CXX_COMPILER=clang++-12 -DLY_PARALLEL_LINK_JOBS=4 -DCMAKE_MODULE_PATH=${WORKSPACE}/o3de/install/cmake", + "CMAKE_OPTIONS": "-G 'Ninja Multi-Config' -DLY_PARALLEL_LINK_JOBS=4 -DCMAKE_MODULE_PATH=${WORKSPACE}/o3de/install/cmake", "CMAKE_TARGET": "all" } } diff --git a/scripts/build/Platform/Linux/build_installer_linux.sh b/scripts/build/Platform/Linux/build_installer_linux.sh new file mode 100755 index 0000000000..301eb5f16d --- /dev/null +++ b/scripts/build/Platform/Linux/build_installer_linux.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set -o errexit # exit on the first failure encountered + +BASEDIR=$(dirname "$0") +source $BASEDIR/build_linux.sh + +source $BASEDIR/installer_linux.sh diff --git a/scripts/build/Platform/Linux/build_linux.sh b/scripts/build/Platform/Linux/build_linux.sh index ab51913550..c14d4b5073 100755 --- a/scripts/build/Platform/Linux/build_linux.sh +++ b/scripts/build/Platform/Linux/build_linux.sh @@ -37,13 +37,13 @@ else fi if [[ ! -z "$RUN_CONFIGURE" ]]; then # have to use eval since $CMAKE_OPTIONS (${EXTRA_CMAKE_OPTIONS}) contains quotes that need to be processed - echo [ci_build] ${CONFIGURE_CMD} + eval echo [ci_build] ${CONFIGURE_CMD} eval ${CONFIGURE_CMD} # Save the run only if success - echo "${CONFIGURE_CMD}" > ${LAST_CONFIGURE_CMD_FILE} + eval echo "${CONFIGURE_CMD}" > ${LAST_CONFIGURE_CMD_FILE} fi -echo [ci_build] cmake --build . --target ${CMAKE_TARGET} --config ${CONFIGURATION} -j $(grep -c processor /proc/cpuinfo) -- ${CMAKE_NATIVE_BUILD_ARGS} -cmake --build . --target ${CMAKE_TARGET} --config ${CONFIGURATION} -j $(grep -c processor /proc/cpuinfo) -- ${CMAKE_NATIVE_BUILD_ARGS} +eval echo [ci_build] cmake --build . --target ${CMAKE_TARGET} --config ${CONFIGURATION} -j $(grep -c processor /proc/cpuinfo) -- ${CMAKE_NATIVE_BUILD_ARGS} +eval cmake --build . --target ${CMAKE_TARGET} --config ${CONFIGURATION} -j $(grep -c processor /proc/cpuinfo) -- ${CMAKE_NATIVE_BUILD_ARGS} popd diff --git a/scripts/build/Platform/Linux/installer_linux.sh b/scripts/build/Platform/Linux/installer_linux.sh new file mode 100755 index 0000000000..3ded242522 --- /dev/null +++ b/scripts/build/Platform/Linux/installer_linux.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# +# Copyright (c) Contributors to the Open 3D Engine Project. +# For complete copyright and license terms please see the LICENSE at the root of this distribution. +# +# SPDX-License-Identifier: Apache-2.0 OR MIT +# +# + +set -o errexit # exit on the first failure encountered + +BASEDIR=$(dirname "$0") +source $BASEDIR/env_linux.sh + +mkdir -p ${OUTPUT_DIRECTORY} +SOURCE_DIRECTORY=${PWD} +pushd $OUTPUT_DIRECTORY + +if ! command -v cpack &> /dev/null; then + echo "[ci_build] CPack not found" + exit 1 +fi + +echo [ci_build] cpack --version +cpack --version + +eval echo [ci_build] cpack -C ${CONFIGURATION} ${CPACK_OPTIONS} +eval cpack -C ${CONFIGURATION} ${CPACK_OPTIONS} + +popd diff --git a/scripts/build/Platform/Windows/build_config.json b/scripts/build/Platform/Windows/build_config.json index 3260c9af79..6f1aaa1570 100644 --- a/scripts/build/Platform/Windows/build_config.json +++ b/scripts/build/Platform/Windows/build_config.json @@ -360,9 +360,9 @@ "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build\\windows_vs2019", "CMAKE_OPTIONS": "-G \"Visual Studio 16 2019\" -DCMAKE_SYSTEM_VERSION=10.0 -DLY_DISABLE_TEST_MODULES=TRUE -DLY_VERSION_ENGINE_NAME=o3de-sdk -DLY_INSTALLER_WIX_ROOT=\"!WIX! \"", - "EXTRA_CMAKE_OPTIONS": "-DLY_INSTALLER_AUTO_GEN_TAG=ON -DLY_INSTALLER_DOWNLOAD_URL=!INSTALLER_DOWNLOAD_URL! -DLY_INSTALLER_LICENSE_URL=!INSTALLER_DOWNLOAD_URL!/license", - "CPACK_BUCKET": "%INSTALLER_BUCKET%", + "EXTRA_CMAKE_OPTIONS": "-DLY_INSTALLER_AUTO_GEN_TAG=TRUE -DLY_INSTALLER_DOWNLOAD_URL=!INSTALLER_DOWNLOAD_URL! -DLY_INSTALLER_LICENSE_URL=!INSTALLER_DOWNLOAD_URL!/license", "CMAKE_TARGET": "ALL_BUILD", + "CPACK_OPTIONS": "-D CPACK_UPLOAD_URL=\"!CPACK_UPLOAD_URL!\"", "CMAKE_NATIVE_BUILD_ARGS": "/m /nologo" } }, diff --git a/scripts/build/Platform/Windows/build_windows.cmd b/scripts/build/Platform/Windows/build_windows.cmd index b9e862e04f..6c3ce91397 100644 --- a/scripts/build/Platform/Windows/build_windows.cmd +++ b/scripts/build/Platform/Windows/build_windows.cmd @@ -9,13 +9,6 @@ REM SETLOCAL EnableDelayedExpansion -REM Jenkins reports MSB8029 when TMP/TEMP is not defined, define a dummy folder -SET TMP=%cd%/temp -SET TEMP=%cd%/temp -IF NOT EXIST %TMP% ( - MKDIR temp -) - CALL %~dp0env_windows.cmd IF NOT EXIST "%OUTPUT_DIRECTORY%" ( diff --git a/scripts/build/Platform/Windows/env_windows.cmd b/scripts/build/Platform/Windows/env_windows.cmd index f11d394519..73dc781f44 100644 --- a/scripts/build/Platform/Windows/env_windows.cmd +++ b/scripts/build/Platform/Windows/env_windows.cmd @@ -18,6 +18,17 @@ IF NOT "%COMMAND_CWD%"=="" ( CD %COMMAND_CWD% ) +REM Jenkins reports MSB8029 when TMP/TEMP is not defined, define a dummy folder +IF NOT "%TMP%"=="" ( + IF NOT "%WORKSPACE_TMP%"=="" ( + SET TMP=%WORKSPACE_TMP% + SET TEMP=%WORKSPACE_TMP% + ) ELSE ( + SET TMP=%cd%/temp + SET TEMP=%cd%/temp + ) +) + EXIT /b 0 :error diff --git a/scripts/build/Platform/Windows/installer_windows.cmd b/scripts/build/Platform/Windows/installer_windows.cmd index 87f53adc7f..8af8e7b625 100644 --- a/scripts/build/Platform/Windows/installer_windows.cmd +++ b/scripts/build/Platform/Windows/installer_windows.cmd @@ -17,10 +17,12 @@ IF NOT EXIST %OUTPUT_DIRECTORY% ( ) PUSHD %OUTPUT_DIRECTORY% -REM Override the temporary directory used by wix to the workspace -SET "WIX_TEMP=!WORKSPACE_TMP!/wix" -IF NOT EXIST "%WIX_TEMP%" ( - MKDIR "%WIX_TEMP%" +REM Override the temporary directory used by wix to the workspace (if we have a WORKSPACE_TMP) +IF NOT "%WORKSPACE_TMP%"=="" ( + SET "WIX_TEMP=!WORKSPACE_TMP!/wix" + IF NOT EXIST "%WIX_TEMP%" ( + MKDIR "%WIX_TEMP%" + ) ) REM Make sure we are using the CMake version of CPack and not the one that comes with chocolatey @@ -47,10 +49,6 @@ IF ERRORLEVEL 1 ( GOTO :popd_error ) -IF NOT "%CPACK_BUCKET%"=="" ( - SET "CPACK_OPTIONS=-D CPACK_UPLOAD_URL=s3://%CPACK_BUCKET% %CPACK_OPTIONS%" -) - ECHO [ci_build] "!CPACK_PATH!" -C %CONFIGURATION% %CPACK_OPTIONS% "!CPACK_PATH!" -C %CONFIGURATION% %CPACK_OPTIONS% IF NOT %ERRORLEVEL%==0 ( diff --git a/scripts/build/Platform/iOS/build_config.json b/scripts/build/Platform/iOS/build_config.json index 895f74daae..ec2b763dda 100644 --- a/scripts/build/Platform/iOS/build_config.json +++ b/scripts/build/Platform/iOS/build_config.json @@ -76,7 +76,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/mac", - "CMAKE_OPTIONS": "-G Xcode -DO3DE_HOME_PATH=\"${WORKSPACE}/home\" -DO3DE_REGISTER_ENGINE_PATH=\"${WORKSPACE}/o3de\" -DO3DE_REGISTER_THIS_ENGINE=TRUE", + "CMAKE_OPTIONS": "-G Xcode", "CMAKE_LY_PROJECTS": "AutomatedTesting", "CMAKE_TARGET": "AssetProcessorBatch", "ASSET_PROCESSOR_BINARY": "bin/profile/AssetProcessorBatch", @@ -112,7 +112,7 @@ "PARAMETERS": { "CONFIGURATION": "profile", "OUTPUT_DIRECTORY": "build/ios_test", - "CMAKE_OPTIONS": "-G Xcode -DCMAKE_TOOLCHAIN_FILE=cmake/Platform/iOS/Toolchain_ios.cmake -DLY_MONOLITHIC_GAME=FALSE -DCMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_REQUIRED=TRUE -DLY_IOS_CODE_SIGNING_IDENTITY=\"\" -DCMAKE_XCODE_ATTRIBUTE_CODE_SIGN_ENTITLEMENTS=\"\" -DCMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_ALLOWED=TRUE -DO3DE_HOME_PATH=\"${WORKSPACE}/home\" -DO3DE_REGISTER_ENGINE_PATH=\"${WORKSPACE}/o3de\" -DO3DE_REGISTER_THIS_ENGINE=TRUE", + "CMAKE_OPTIONS": "-G Xcode -DCMAKE_TOOLCHAIN_FILE=cmake/Platform/iOS/Toolchain_ios.cmake -DLY_MONOLITHIC_GAME=FALSE -DCMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_REQUIRED=TRUE -DLY_IOS_CODE_SIGNING_IDENTITY=\"\" -DCMAKE_XCODE_ATTRIBUTE_CODE_SIGN_ENTITLEMENTS=\"\" -DCMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_ALLOWED=TRUE", "CMAKE_TARGET": "ALL_BUILD", "CMAKE_NATIVE_BUILD_ARGS": "", "TARGET_DEVICE_NAME": "Lumberyard", diff --git a/scripts/build/build_node/Platform/Linux/package-list.ubuntu-bionic.txt b/scripts/build/build_node/Platform/Linux/package-list.ubuntu-bionic.txt index 2af55ab180..30259a6dc7 100644 --- a/scripts/build/build_node/Platform/Linux/package-list.ubuntu-bionic.txt +++ b/scripts/build/build_node/Platform/Linux/package-list.ubuntu-bionic.txt @@ -12,7 +12,7 @@ libxcb-xinerama0 # For Qt plugins at runtime libxcb-xinput0 # For Qt plugins at runtime libfontconfig1-dev # For Qt plugins at runtime libcurl4-openssl-dev # For HttpRequestor -libsdl2-dev # For WWise/Audio +# libsdl2-dev # For WWise/Audio libxcb-xkb-dev # For xcb keyboard input libxkbcommon-x11-dev # For xcb keyboard input libxkbcommon-dev # For xcb keyboard input diff --git a/scripts/build/build_node/Platform/Linux/package-list.ubuntu-focal.txt b/scripts/build/build_node/Platform/Linux/package-list.ubuntu-focal.txt index e0e03cda90..71958d74bd 100644 --- a/scripts/build/build_node/Platform/Linux/package-list.ubuntu-focal.txt +++ b/scripts/build/build_node/Platform/Linux/package-list.ubuntu-focal.txt @@ -12,7 +12,7 @@ libxcb-xinerama0 # For Qt plugins at runtime libxcb-xinput0 # For Qt plugins at runtime libfontconfig1-dev # For Qt plugins at runtime libcurl4-openssl-dev # For HttpRequestor -libsdl2-dev # for WWise/Audio +# libsdl2-dev # for WWise/Audio libxcb-xkb-dev # For xcb keyboard input libxkbcommon-x11-dev # For xcb keyboard input libxkbcommon-dev # For xcb keyboard input