Steve Pham
b2bafc44ab
* Fix Delete_Real_Readonly_Fails to mark parent directory as read-only as well Read-only files in Windows cannot be deleted. The previous version of this code relied on that fact, and would attempt to delete a file even when `skipReadOnly = true`, relying on the OS to refuse to delete the file if it is read only. On Linux, it is the writable state of the *directory* that determines if a file can be deleted or not. This fixes the test to set up the correct situation where a file deletion would fail. * Remove excluded items from a vector before iterating over it Removing items from the `pathMatches` `QStringList` while iterating over it was causing a segfault on Linux. This change separates out the item removal from the item iteration, which allows the item iteration loop to use a range-for loop instead of directly manipulating iterators. * Remove invalid test that asserts a file's metadata file can have differing file casing This test is asserting that a given source file and its accompanying metadata file can have the the same name but differing case. This is really testing whether or not the underlying filesystem that those files live on is case sensitive or not. The 99% chance is that users are using the default filesystem that their host OS gives them, NTFS on Windows, EXT* on Linux, and APFS on Mac. Even though NTFS is case-insensitive by default, it [can be configured per-directory](https://devblogs.microsoft.com/commandline/improved-per-directory-case-sensitivity-support-in-wsl/). APFS as well can be configured to be case-sensitive. For users with case sensitive filesystems, this test makes no sense. We could extend this test to inspect the case-sensitivity of the underlying filesystem, but then it is just testing the filesystem's behavior, which seems out of scope of this test. * Use a non-priviliged port for the Asset Processor tests From https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html: > The TCP/IP port numbers below 1024 are special in that normal users are not > allowed to run servers on them. This is a security feaure, in that if you > connect to a service on one of these ports you can be fairly sure that you > have the real thing, and not a fake which some hacker has put up for you. > > When you run a server as a test from a non-priviliged account, you will > normally test it on other ports, such as 2784, 5000, 8001 or 8080. * Fix for `QDir::rmdir(".")` not working in Linux Qt uses `::rmdir` to remove directories on Linux. This comes from [unistd.h](https://pubs.opengroup.org/onlinepubs/007904875/functions/rmdir.html) The documentation for that function states: > If the path argument refers to a path whose final component is either dot > or dot-dot, rmdir() shall fail. So calling `dir.rmdir(".")` will never work on Linux. Instead, get the parent directory, and remove the child directory by name. * Avoid lowercasing source asset paths when resolving dependencies Source asset paths may be case sensitive, so their case must be preserved when doing operations that hit the underlying filesystem. This method was always lowercasing them, which would cause dependencies to not be found. * Correct test to expect product filenames to be lowercase The modtime tests were failing in Linux due to something unrelated to file modtime checking. The Asset Processor Manager does this during AnalyzeJob: ``` if (foundInDatabase && jobs[0].m_fingerprint == jobDetails.m_jobEntry.m_computedFingerprint) { // If the fingerprint hasn't changed, we won't process it.. unless...is it missing a product. ``` In this case, the test was setting up a product whose file case was the same as the source asset, and would write it to the cache dir using mixed case, but use the normal asset processor API to write the product file path to the database, which recorded the path in lowercase. When the manager then went to check if the source asset's products all exist, it checked the lowercase path, which didn't exist. This fixes that test failure, by updating the test to write the product file to the cache using the proper lowercased path. * Update test to define a "not current platform" for Linux This test was failing because it was setting some "not current platform" variable to be set to "pc" on Linux, when `AssetSystem::GetHostAssetPlatform()` is defined to: ```cpp inline const char* GetHostAssetPlatform() { return "mac"; return "pc"; // set this to pc because that's what bootstrap.cfg currently defines the platform to "pc", even on Linux return "pc"; #error Unimplemented Host Asset Platform } ``` The test would go on to assert that "pc" was simultaneously in a list and not in the same list. This fixes the test by updating the code to set the "not the current platform" variable appropriately on Linux. The expectations were also updated to improve the output on test failure. Instead of this: ``` Value of: recogs["rend"].m_platformSpecs.contains(platformWhichIsNotCurrentPlatform) Actual: true Expected: false ``` You now get this: ``` Value of: recogs["rend"].m_platformSpecs.keys() Expected: (has 3 elements and there exists some permutation of elements such that: - element #0 is equal to pc, and - element #1 is equal to es3, and - element #2 is equal to server) and (doesn't contain any element that is equal to pc) Actual: { pc, server, es3 } (of type QList<QString>), whose element #0 matches ``` * Prevent windows supported path separators to be included in the test paths for UpdateToCorrectCase_ExistingFile_ReturnsTrue_CorrectsCase * Fix failing linux unit test "PlatformConfigurationUnitTests.TestFailReadConfigFile_RegularScanfolder" caused by static variable not being reset from a different test run when using AssetUtilities::ComputeProjectPath * Fix AZ_RTTI declaration for RequestEscalateAsset Message * Implement FileWatcher for Linux to fix AssetProcessorMessages.All test (RequestAssetStatus) * Split AssetProcessorMessages into 2 tests, one with RequestAssetStatus/ResponseAssetStatus and one without Add The RequestAssetStatus/ResponseAssetStatus as a sandbox test because it relies on FileWatcher thread and seems to be timing related * Remove FileWatcher_win.cpp from the Linux specific folder for FileWatcher * - Fix build error related to non-unity builds - Fixed failed linux test 'Test/LegacyTestAdapter.AllTests/UtilitiesUnitTest' caused by misplaced windows only EXPECT - Remove test trait AZ_TRAIT_DISABLE_FAILED_ASSET_PROCESSOR_TESTS for linux to expose remaining failed tests * Fixed failed linux test 'Test/LegacyTestAdapter.AllTests/RCcontrollerUnitTests' caused by misplaced windows only EXPECT * - Fix FileWatcher unit test, disable incompatible subtests for Linux - Fix errors in FileWatcher_linux from results of the FileWatcher Unit Test * Remove AZ::AssetProcessor.Tests.Sandbox tests from definition and restore the original AssetProcessorMessages.All tests now that Filewatcher_linux was fixed * Fixes for failed unit tests: AssetProcessorManagerUnitTests and AssetProcessorManagerUnitTests_JobDependencies_Fingerprint - Caused by differences between between case-sensitive files (Linux) and non-case-sensitive Filesystems (Windows) * Update consts in FileWatcher_linux.cpp to constexpr * Fixes related to PR comment suggestions * - Removed std::bind and replaced with lambda in FileWatcher_linux - Replaced String replace functions for path separators to use AZ::IO::Path::LexicallyNormal() instead * Restoring string replace function in PathDependencyManager::ResolveDependencies due to unit test failure Co-authored-by: Chris Burel <burelc@amazon.com>
Project Spectra Private Preview
Confidentiality; Pre-Release Access
Welcome to the Project Spectra Private Preview. This is a confidential pre-release project; your use is subject to the nondisclosure agreement between you (or your organization) and Amazon. Do not disclose the existence of this project, your participation in it, or any of the materials provided, to any unauthorized third party. To request access for a third party, please contact Royal O'Brien, obriroya@amazon.com.
Full instructions can be found here:
https://docs.o3de.org/docs/welcome-guide/setup/setup-from-github/
(Note: Contact Royal or Doug Erickson, dougeric@amazon.com for access)
Updates to this readme
May 14, 2021
- Removed instructions for the 3rdParty zip file and downloader URL. This is no longer a requirement.
- Updated instructions for dependencies
- Links to full documentation
April 7-13, 2021
- Updates to the 3rdParty zip file
March 25, 2021
- Initial commit for instructions
Download and Install
This repository uses Git LFS for storing large binary files. You will need to create a Github personal access token to authenticate with the LFS service.
To install Git LFS, download the binary here: https://git-lfs.github.com/.
After installation, you will need to install the necessary git hooks with this command
git lfs install
Create a Git Personal Access Token
You will need your personal access token credentials to authenticate when you clone the repository and when downloading objects from Git LFS
Create a personal access token with the 'repo' scope.
During the clone operation, you will be prompted to enter a password. Your token will be used as the password. You will also be prompted a second time for Git LFS.
(Recommended) Verify you have a credential manager installed to store your credentials
Recent versions of Git install a credential manager to store your credentials so you don't have to put in the credentials for every request.
It is highly recommended you check that you have a credential manager installed and configured
For Linux and Mac, use the following commands to store credentials
Linux:
git config --global credential.helper cache
Mac:
git config --global credential.helper osxkeychain
Clone the repository
> git clone https://github.com/aws-lumberyard/o3de.git
Cloning into 'o3de'...
# initial prompt for credentials to download the repository code
# enter your Github username and personal access token
remote: Counting objects: 29619, done.
Receiving objects: 100% (29619/29619), 40.50 MiB | 881.00 KiB/s, done.
Resolving deltas: 100% (8829/8829), done.
Updating files: 100% (27037/27037), done.
# second prompt for credentials when downloading LFS files
# enter your Github username and personal access token
Filtering content: 100% (3853/3853), 621.43 MiB | 881.00 KiB/s, done.
If you have the Git credential manager core or other credential helpers installed, you should not be prompted for your credentials anymore.
Building the Engine
Build Requirements and redistributables
Windows
- Visual Studio 2019 16.9.2 minimum (All versions supported, including Community): https://visualstudio.microsoft.com/downloads/
- Install the following workloads:
- Game Development with C++
- MSVC v142 - VS 2019 C++ x64/x86
- C++ 2019 redistributable update
- Install the following workloads:
- CMake 3.19.1 minimum: https://cmake.org/files/LatestRelease/cmake-3.19.1-win64-x64.msi
Optional
- WWise - 2019.2.8.7432 minimum: https://www.audiokinetic.com/download/
- Note: This requires registration and installation of a client to download
- You will also need to set a environment variable:
set LY_WWISE_INSTALL_PATH=<path to WWise version> - For example:
set LY_WWISE_INSTALL_PATH="C:\Program Files (x86)\Audiokinetic\Wwise 2019.2.8.7432"
Quick Start Build Steps
-
Create a writable folder to cache 3rd Party dependencies. You can also use this to store other redistributable SDKs.
For the 0.5 branch - Create an empty text file named
3rdParty.txtin this folder, to allow a legacy CMake validator to pass -
Install the following redistributables to the following:
- Visual Studio and VC++ redistributable can be installed to any location
- CMake can be installed to any location, as long as it's available in the system path, otherwise it can be installed to:
<3rdParty Path>\CMake\3.19.1 - WWise can be installed anywhere, but you will need to set an environment variable for CMake to detect it:
set LY_WWISE_INSTALL_PATH=<path to WWise>
-
Navigate into the repo folder, then download the python runtime with this command
For the 0.5 branch - Set this environment variable prior to the
get_pythoncommand below:set LY_PACKAGE_SERVER_URLS=https://d2c171ws20a1rv.cloudfront.netpython\get_python.bat -
Configure the source into a solution using this command line, replacing and <3rdParty cache path> to a path you've created:
cmake -B <your build path> -S <your source path> -G "Visual Studio 16" -DLY_3RDPARTY_PATH=<3rdParty cache path> -DLY_UNITY_BUILD=ON -DLY_PROJECTS=AutomatedTestingNote: Do not use trailing slashes for the <3rdParty cache path>
-
Alternatively, you can do this through the CMake GUI:
- Start
cmake-gui.exe - Select the local path of the repo under "Where is the source code"
- Select a path where to build binaries under "Where to build the binaries"
- Click "Configure"
- Wait for the key values to populate. Fill in the fields that are relevant, including
LY_3RDPARTY_PATHandLY_PROJECTS - Click "Generate"
- Start
-
The configuration of the solution is complete. To build the Editor and AssetProcessor to binaries, run this command inside your repo:
cmake --build <your build path> --target AutomatedTesting.GameLauncher AssetProcessor Editor --config profile -- /m -
This will compile after some time and binaries will be available in the build path you've specified
Setting up new projects
- While still within the repo folder, register the engine with this command:
scripts\o3de.bat register --this-engine - Setup new projects using the
o3de create-projectcommand. In the 0.5 branch, the project directory must be a subdirectory in the repo folder.<Repo path>\scripts\o3de.bat create-project --project-path <your new project path> - Register the engine to the project
<Repo path>\scripts\o3de.bat register --project-path <New project path> - Once you're ready to build the project, run the same set of commands to configure and build:
cmake -B <your project build path> -S <your new project source path> -G "Visual Studio 16" -DLY_3RDPARTY_PATH=<3rdParty cache path> // For the 0.5 branch, you must build a new Editor for each project: cmake --build <your project build path> --target <New Project Name>.GameLauncher Editor --config profile -- /m // For all other branches, just build the project: cmake --build <your project build path> --target <New Project Name>.GameLauncher --config profile -- /m
For a tutorial on project configuration, see Creating Projects Using the Command Line in the documentation.
License
For terms please see the LICENSE*.TXT file at the root of this distribution.