Commit Graph

14 Commits (4721ef829835e1d15e413c41cc410ccfa5200ed8)

Author SHA1 Message Date
Pip Potter d692cde146
LYN-7473: Ensure Requestor background thread is shutdown correctly (#4744)
Signed-off-by: rppotter <rppotter@amazon.com>
4 years ago
amzn-phist fec77632ef
Archive Component - Rewrite and additional work on Archive and Asset Bundler (#4332)
* Fix issues with seedlist for AutomatedTesting

Fixes error reporting so it will show the file hint in the tool.
Removes any missing assets from the .seed file.
Remove an unnecessary dependency from AutomatedTesting dependencies
file.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Make ArchiveComponent use AZ::IO::IArchive

Initial changes that will get the sychronous calls in ArchiveComponent
to use IArchive interface rather than external zip/7z tools.

Some of the asynchronous api are still in place, anything that wasn't
being used has been removed for now.  This may change later if we move
towards all the api being asynchronous.  Until then, we can't remove the
reliance upon the external archive tools completely.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Updates AZStd::thread constructors

Adds a variadic constructor which forwards args to the functor.
Because of our thread_desc extension, there was confusion on the
arugments, so the args were reordered to take the thread_desc first,
before the functor and args.
Also the thread_desc is taken as reference rather than by pointer.
Update callsites to account for this change.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Async operation of ArchiveComponent api

This sets up the ArchiveComponent to operate asynchronously.  It uses
promise/future to transfer results to caller.

This is still broken, there's a few things that need to get fixed up,
but this is a good checkpoint for the work as it solidifies the api,
cleans up a bunch of unused code, and compiles.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Removes the platform-specific ArchiveComponen

These are no longer needed, as they control the direct interaction with
host OS tools like 7za.exe or /bin/zip.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Removes the platform-specific files from cmake

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Removes the 7za.exe (and legal notice)

This tool is no longer needed in the repo.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fixes usage of IArchive::GetFullPath()

This changed to return a PathView, updated to reflect that.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix promises and threads

Make sure promises are only set exactly once.  This meant reworking some
of the initial error checking.
Detach threads when created.  Adds [[nodiscard]] to the functions that
return a future.  Since threads are detached, the future is the main way
to get communication from the thread.
Clean up interface, add comments.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* More edits to thread creation

Changes to thread construction to account for parameter change.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix some remaining issues with ArchiveComponent

Put created threads inside a container, then join them at Deactivate.
Fix asset bundler case when injecting a file with no working directory.
Fix thread constructor that applies args to a function.
Fix lambdas to take string args by value rather than reference.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fixes some remaining bugs in ArchiveComponent

Open archive as read-only during extract & list operations.
Fixes paths issues.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix initialize of opaque thread handle in thread_UnixLike.h

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed unused variable in AssetBundleComponent.cpp to fix compiler warning

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fix some issues with archives

File paths in the CDR and the local headers need to match, but there
were issues with path separators and case that made it possible to get
invalid headers errors in some archives.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Adds some new ArchiveComponent unit tests

Adds new tests for extraction of archive and adding files from a file
list to an archive.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Fix file data offset issues when opening archives

When opening an INestedArchive it would run through the CDR headers to
create file entries in the zip cache.  The offsets to the compressed
data were being calculated incorrectly because they were using the CDR
headers rather than jumping to the local file headers and getting
offsets from those sizes.

Removed and refactored some archive validation flow and zip cache
factory init methods to either init default or init w/ additional
validation checks.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Addresses PR feedback

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address more points of feedback in PR

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address additional PR feedback

 Fixes up some error checks and uses of strings vs paths.
 Enable archive component tests on Linux so we can see if they will
 pass.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address PR feedback

Change the INestedArchive interface to list files as AZ::IO::Path.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Disabling the ArchiveComponent tests on Linux

They failed so we will revisit them to attempt a fix.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Rename a member variable to be more accurate

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Address feedback on PR

Bump version of Archive Components for serialize context.
Improve error messages during archive open and validation.

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

* Revert recent changes

Signed-off-by: amzn-phist <52085794+amzn-phist@users.noreply.github.com>

Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
4 years ago
Guthrie Adams 9bd23dc5d8 reverting call to AWSNativeSDKInit::InitializationManager::Shutdown()
Signed-off-by: Guthrie Adams <guthadam@amazon.com>
4 years ago
Guthrie Adams 45c0430f96 Merge branch 'Atom/guthadam/atomtools_updating_cmake_config' into Atom/guthadam/atomtools_updating_cmake_config_plus_bootstrap 4 years ago
Chris Burel b53bf52e0d
Perform global deinitialization when exiting the game launcher (#4131)
* Fix code that deregisters the Atom Scene subsystem from the AzFramework Scene

The AzFramework Scene subsystem API is a generic container based on the
type of argument that is passed to it. It maintains a vector of typeids,
and only one object of any type is stored at a time. The Bootstrap system
component registers the Atom scene as a `ScenePtr` (aka
`AZStd::shared_ptr<RPI:Scene>`) with the AzFramework Scene's generic
subsystem. However, the component was previously deregistering the type by
value, `RPI::Scene`. Since no subsystem for the type `RPI::Scene` was set,
unsetting this type did nothing. The result was that the `RPI::Scene`
object would still be around by the time that all the Atom
`InstanceDatabse`s were being destroyed, resulting in a large number of
errors reported about leaked instances during global shutdown.

This fixes the above issue by passing the `m_defaultScene` as a parameter
to `AzFramework::Scene::UnsetSubsystem`, the same value that is passed to
`SetSubsystem`. This is better, because instead of providing explicit
template arguments (which were specifying the incorrect type), this now
allows the compiler to deduce the correct type, and the syntax is symmetric
with the call to `SetSubsystem`.

Signed-off-by: Chris Burel <burelc@amazon.com>

* Correctly release the AWS API from the `HttpRequestManager` module

This code was incorrectly assuming that
`AWSNativeSDKInit::InitializationManager::Shutdown()` would be called
automatically by the `InitializationManager` itself. However, all that
`InitAwsApi()` does is create an `AZ::EnvironmentVariable`, which is a
ref-counted type, and stores it in a global static. That global static is
defined in a static library (namely `AWSNativeSDKInit`), which is linked
in to the `HttpRequestManager` dynamic lib. Because it is a global static,
it has to be explicitly cleared with the call to `Shutdown()`. Otherwise
the destructor of the EnvironmentVariable doesn't happen until global
destruction, by which time the allocator that is supplied to the AWS SDK
has already been destroyed, and the shutdown of the AWS SDK attempts to use
the already-destroyed allocator.

Signed-off-by: Chris Burel <burelc@amazon.com>

* Avoid blocking the remote console server thread if there are no connections

The Remote console server runs in a separate thread. Previously, it would
directly call `AzSock::Accept()` and block the server thread until some
client connected to it. However, if no client connected, the thread would
continue to be blocked, even if the game launcher tried to exit.

This adds a check to see if there's a client on the socket before calling
`Accept()`, to avoid the deadlock on launcher exit.

Signed-off-by: Chris Burel <burelc@amazon.com>

* Fix a log message to print one message per line

Signed-off-by: Chris Burel <burelc@amazon.com>

* Allow pumping the event loop to close the launcher window

Events from the OS are handled in the game's main loop. The general loop
looks like this:

 * Read events from the OS
 * Tick the game application

One of the events that can come from the OS is that the window hosting the
game is closed. When this event happens, many resources provided by the
renderer are freed, and the game application's `shouldExit` bit is set.
However, when the game's `Tick()` is called, there is lots of code that
assumes the renderer is still there. To avoid crashing in the `Tick()`
call, check if the game should exit after pumping the system events.

Signed-off-by: Chris Burel <burelc@amazon.com>

* Unload the level when exiting the launcher

This ensures that any resources held onto by the level are freed before the
launcher exits.

Signed-off-by: Chris Burel <burelc@amazon.com>

* Add an explicit bus `Disconnect()` call to `AZCoreLogSink`

This is necessary because this bus has virtual functions and can be called
from multiple threads.

Signed-off-by: Chris Burel <burelc@amazon.com>

* Allow normal cleanup to take place when exiting the game launcher

Previously, global cleanup was side-stepped by calling `TerminateProcess`
or `exit`, when quitting the game launcher. This is in contrast to the call
to `_exit` on Linux and Mac when exiting the Editor. That leading `_` makes
a big difference: the former runs object destruction, the latter does not.
Instead of making the launcher exit with `_exit` on Linux, instead, remove
that call and actually run all the atexit code.

This does not modify the Editor's behavior however. It still uses `_exit`
and `TerminateProcess`.

Signed-off-by: Chris Burel <burelc@amazon.com>
4 years ago
Guthrie Adams c2b2cc060a WIP
Signed-off-by: Guthrie Adams <guthadam@amazon.com>
4 years ago
Esteban Papp 1f9b284de2 Merge branch 'development' into cmake/SPEC-7179
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

# Conflicts:
#	Code/Editor/Plugins/ComponentEntityEditorPlugin/ComponentEntityEditorPlugin_precompiled.h
#	Code/Editor/Plugins/EditorCommon/EditorCommon_precompiled.h
#	Code/Editor/Plugins/EditorCommon/stdafx.cpp
#	Code/Editor/Plugins/FFMPEGPlugin/FFMPEGPlugin_precompiled.h
#	Code/Editor/Plugins/PerforcePlugin/PerforcePlugin_precompiled.h
#	Code/Editor/Plugins/ProjectSettingsTool/ProjectSettingsTool_precompiled.h
#	Code/Framework/AzToolsFramework/AzToolsFramework/AzToolsFramework_precompiled.h
#	Code/Tools/AssetProcessor/native/precompiled.h
#	Code/Tools/Standalone/StandaloneTools_precompiled.h
#	Gems/AssetMemoryAnalyzer/Code/Source/AssetMemoryAnalyzer_precompiled.h
#	Gems/Atom/Asset/ImageProcessingAtom/Code/Source/ImageProcessing_precompiled.h
#	Gems/Atom/RHI/DX12/Code/Source/RHI/Atom_RHI_DX12_precompiled.h
#	Gems/Atom/RHI/Metal/Code/Include/Platform/Mac/Atom_RHI_Metal_precompiled_Platform.h
#	Gems/Atom/RHI/Metal/Code/Include/Platform/iOS/Atom_RHI_Metal_precompiled_Platform.h
#	Gems/Atom/RHI/Metal/Code/Source/Atom_RHI_Metal_precompiled.h
#	Gems/Atom/RHI/Metal/Code/atom_rhi_metal_common_files.cmake
#	Gems/Atom/RHI/Null/Code/Source/Atom_RHI_Null_precompiled.h
#	Gems/Atom/RHI/Null/Code/atom_rhi_null_common_files.cmake
#	Gems/Atom/RHI/Vulkan/Code/Include/Platform/Android/Atom_RHI_Vulkan_precompiled_Platform.h
#	Gems/Atom/RHI/Vulkan/Code/Include/Platform/Linux/Atom_RHI_Vulkan_precompiled_Platform.h
#	Gems/Atom/RHI/Vulkan/Code/Include/Platform/Mac/Atom_RHI_Vulkan_precompiled_Platform.h
#	Gems/Atom/RHI/Vulkan/Code/Include/Platform/Windows/Atom_RHI_Vulkan_precompiled_Platform.h
#	Gems/Atom/RHI/Vulkan/Code/Source/Atom_RHI_Vulkan_precompiled.h
#	Gems/Atom/RHI/Vulkan/Code/Source/RHI/SwapChain.cpp
#	Gems/Atom/RHI/Vulkan/Code/atom_rhi_vulkan_common_files.cmake
#	Gems/AtomLyIntegration/AtomFont/Code/Include/AtomLyIntegration/AtomFont/AtomFont_precompiled.h
#	Gems/Blast/Code/Source/StdAfx.cpp
#	Gems/Camera/Code/Source/Camera_precompiled.h
#	Gems/EMotionFX/Code/Source/EMotionFX_precompiled.h
#	Gems/FastNoise/Code/Source/FastNoise_precompiled.h
#	Gems/Gestures/Code/Source/Gestures_precompiled.h
#	Gems/GradientSignal/Code/Source/GradientSignal_precompiled.h
#	Gems/GraphCanvas/Code/precompiled.h
#	Gems/ImGui/Code/Source/ImGui_precompiled.h
#	Gems/InAppPurchases/Code/Source/InAppPurchases_precompiled.h
#	Gems/LmbrCentral/Code/Source/LmbrCentral_precompiled.h
#	Gems/LmbrCentral/Code/Tests/ShapeGeometryUtilTest.cpp
#	Gems/LyShine/Code/Editor/UiCanvasEditor_precompiled.h
#	Gems/LyShine/Code/Source/Animation/LyShine_precompiled.h
#	Gems/LyShine/Code/Source/LyShine_precompiled.h
#	Gems/LyShineExamples/Code/Source/LyShineExamples_precompiled.h
#	Gems/Maestro/Code/Source/Cinematics/Maestro_precompiled.h
#	Gems/Maestro/Code/Source/Maestro_precompiled.h
#	Gems/MessagePopup/Code/Source/MessagePopup_precompiled.h
#	Gems/Metastream/Code/Source/Metastream_precompiled.h
#	Gems/Microphone/Code/Source/Microphone_precompiled.h
#	Gems/Multiplayer/Code/Source/Multiplayer_precompiled.h
#	Gems/PhysX/Code/NumericalMethods/Source/NumericalMethods_precompiled.h
#	Gems/PhysX/Code/Source/PhysXUnsupported_precompiled.h
#	Gems/PhysX/Code/Source/PhysX_precompiled.h
#	Gems/PhysX/Code/physx_unsupported_files.cmake
#	Gems/PhysXDebug/Code/Source/PhysXDebugUnsupported_precompiled.h
#	Gems/PhysXDebug/Code/Source/PhysXDebug_precompiled.h
#	Gems/ScriptCanvas/Code/Editor/precompiled.h
#	Gems/ScriptCanvas/Code/Source/precompiled.h
#	Gems/ScriptCanvasDeveloper/Code/Source/precompiled.h
#	Gems/ScriptCanvasPhysics/Code/Source/ScriptCanvasPhysics_precompiled.h
#	Gems/ScriptEvents/Code/Source/precompiled.h
#	Gems/ScriptEvents/Code/Tests/Editor/EditorTests.cpp
#	Gems/ScriptedEntityTweener/Code/Source/ScriptedEntityTweener_precompiled.h
#	Gems/SliceFavorites/Code/Source/SliceFavorites_precompiled.h
#	Gems/StartingPointCamera/Code/Source/StartingPointCamera_precompiled.h
#	Gems/StartingPointInput/Code/Source/StartingPointInput_precompiled.h
#	Gems/StartingPointMovement/Code/Source/StartingPointMovement_precompiled.h
#	Gems/SurfaceData/Code/Source/SurfaceData_precompiled.h
#	Gems/TextureAtlas/Code/Source/TextureAtlas_precompiled.h
#	Gems/TickBusOrderViewer/Code/Source/TickBusOrderViewer_precompiled.h
#	Gems/Twitch/Code/Source/Twitch_precompiled.h
#	Gems/VirtualGamepad/Code/Source/VirtualGamepad_precompiled.h
#	Gems/WhiteBox/Code/Source/WhiteBoxUnsupported_precompiled.h
#	Gems/WhiteBox/Code/Source/WhiteBox_precompiled.h
4 years ago
Steve Pham 38261d0800
Shorten copyright headers by splitting into 2 lines (#2213)
* Updated all copyright headers to split the longer original copyright line into 2 shorter lines

Signed-off-by: Steve Pham <spham@amazon.com>
4 years ago
Esteban Papp 5eed3234ad Gems/HttpRequestor
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
4 years ago
Steve Pham b4a2edec6a
Final update copyright headers to reference license files at the repo root (#1693)
* Final update copyright headers to reference license files at the repo root

Signed-off-by: spham <spham@amazon.com>

* Fix copyright validator unit tests to support the stale O3DE header scenario

Signed-off-by: spham <spham@amazon.com>
5 years ago
Steve Pham 70042fcdcd
O3DE Copyright Updates for Linux Foundation (#1504) 5 years ago
Esteban Papp 36cb0f6d40
SPEC-7178 Removal of precompiled cpp files (#1171)
* SPEC-7178  Removal of precompiled cpp files

* Missing files...
5 years ago
Chris Burel 28170ffe41 Add newlines to the end of all files 5 years ago
alexpete a10351f38d Initial commit 5 years ago