Repository: alvr-org/ALVR Branch: master Commit: 1a4194ff1379 Files: 495 Total size: 3.6 MB Directory structure: gitextract_unk25vpb/ ├── .cargo/ │ └── config.toml ├── .clang-format ├── .editorconfig ├── .gitattributes ├── .github/ │ ├── FUNDING.yml │ ├── ISSUE_TEMPLATE/ │ │ └── bug_report.md │ ├── dependabot.yml │ └── workflows/ │ ├── prepare-release.yml │ ├── queue.yml │ ├── rust.yml │ ├── stale.yml │ └── wiki.yml ├── .gitignore ├── .gitmodules ├── CHANGELOG.md ├── CONTRIBUTING.md ├── Cargo.toml ├── LICENSE ├── README.md ├── about.toml ├── alvr/ │ ├── adb/ │ │ ├── Cargo.toml │ │ └── src/ │ │ ├── commands.rs │ │ ├── lib.rs │ │ └── parse.rs │ ├── audio/ │ │ ├── Cargo.toml │ │ └── src/ │ │ ├── lib.rs │ │ ├── linux.rs │ │ └── windows.rs │ ├── client_core/ │ │ ├── Cargo.toml │ │ ├── LICENSE │ │ ├── README.md │ │ ├── build.rs │ │ ├── cbindgen.toml │ │ └── src/ │ │ ├── audio.rs │ │ ├── c_api.rs │ │ ├── connection.rs │ │ ├── lib.rs │ │ ├── logging_backend.rs │ │ ├── sockets.rs │ │ ├── statistics.rs │ │ ├── storage.rs │ │ └── video_decoder/ │ │ ├── android.rs │ │ └── mod.rs │ ├── client_mock/ │ │ ├── Cargo.toml │ │ └── src/ │ │ └── main.rs │ ├── client_openxr/ │ │ ├── Cargo.toml │ │ ├── cbindgen.toml │ │ ├── resources/ │ │ │ ├── drawable/ │ │ │ │ └── ic_launcher_foreground.xml │ │ │ ├── mipmap-anydpi-v26/ │ │ │ │ └── ic_launcher.xml │ │ │ └── values/ │ │ │ └── ic_launcher_background.xml │ │ └── src/ │ │ ├── c_api.rs │ │ ├── extra_extensions/ │ │ │ ├── body_tracking_bd.rs │ │ │ ├── body_tracking_fb.rs │ │ │ ├── eye_gaze_interaction.rs │ │ │ ├── eye_tracking_social.rs │ │ │ ├── face_tracking2_fb.rs │ │ │ ├── face_tracking_pico.rs │ │ │ ├── facial_tracking_htc.rs │ │ │ ├── mod.rs │ │ │ ├── motion_tracking_bd.rs │ │ │ ├── multimodal_input.rs │ │ │ ├── passthrough_fb.rs │ │ │ └── passthrough_htc.rs │ │ ├── graphics.rs │ │ ├── interaction.rs │ │ ├── lib.rs │ │ ├── lobby.rs │ │ ├── passthrough.rs │ │ └── stream.rs │ ├── common/ │ │ ├── Cargo.toml │ │ ├── LICENSE │ │ ├── README.md │ │ └── src/ │ │ ├── average.rs │ │ ├── c_api.rs │ │ ├── connection_result.rs │ │ ├── inputs.rs │ │ ├── lib.rs │ │ ├── logging.rs │ │ ├── primitives.rs │ │ └── version.rs │ ├── dashboard/ │ │ ├── Cargo.toml │ │ ├── README.md │ │ ├── build.rs │ │ └── src/ │ │ ├── dashboard/ │ │ │ ├── components/ │ │ │ │ ├── about.rs │ │ │ │ ├── debug.rs │ │ │ │ ├── devices.rs │ │ │ │ ├── installation.rs │ │ │ │ ├── logs.rs │ │ │ │ ├── mod.rs │ │ │ │ ├── new_version_popup.rs │ │ │ │ ├── notifications.rs │ │ │ │ ├── settings.rs │ │ │ │ ├── settings_controls/ │ │ │ │ │ ├── array.rs │ │ │ │ │ ├── boolean.rs │ │ │ │ │ ├── choice.rs │ │ │ │ │ ├── collapsible.rs │ │ │ │ │ ├── dictionary.rs │ │ │ │ │ ├── mod.rs │ │ │ │ │ ├── notice.rs │ │ │ │ │ ├── number.rs │ │ │ │ │ ├── optional.rs │ │ │ │ │ ├── presets/ │ │ │ │ │ │ ├── builtin_schema.rs │ │ │ │ │ │ ├── higher_order_choice.rs │ │ │ │ │ │ ├── mirror.rs │ │ │ │ │ │ ├── mod.rs │ │ │ │ │ │ └── schema.rs │ │ │ │ │ ├── reset.rs │ │ │ │ │ ├── section.rs │ │ │ │ │ ├── switch.rs │ │ │ │ │ ├── text.rs │ │ │ │ │ ├── up_down.rs │ │ │ │ │ └── vector.rs │ │ │ │ ├── setup_wizard.rs │ │ │ │ └── statistics.rs │ │ │ └── mod.rs │ │ ├── data_sources.rs │ │ ├── data_sources_wasm.rs │ │ ├── linux_checks.rs │ │ ├── logging_backend.rs │ │ ├── main.rs │ │ └── steamvr_launcher/ │ │ ├── linux_steamvr.rs │ │ ├── mod.rs │ │ └── windows_steamvr.rs │ ├── events/ │ │ ├── Cargo.toml │ │ └── src/ │ │ └── lib.rs │ ├── filesystem/ │ │ ├── Cargo.toml │ │ ├── build.rs │ │ └── src/ │ │ └── lib.rs │ ├── graphics/ │ │ ├── Cargo.toml │ │ ├── resources/ │ │ │ ├── lobby_line.wgsl │ │ │ ├── lobby_quad.wgsl │ │ │ ├── staging_fragment.glsl │ │ │ ├── staging_vertex.glsl │ │ │ └── stream.wgsl │ │ └── src/ │ │ ├── lib.rs │ │ ├── lobby.rs │ │ ├── staging.rs │ │ └── stream.rs │ ├── gui_common/ │ │ ├── Cargo.toml │ │ └── src/ │ │ ├── basic_components/ │ │ │ ├── button_group.rs │ │ │ ├── mod.rs │ │ │ ├── modal.rs │ │ │ └── switch.rs │ │ ├── lib.rs │ │ └── theme.rs │ ├── launcher/ │ │ ├── Cargo.toml │ │ ├── build.rs │ │ └── src/ │ │ ├── actions.rs │ │ ├── main.rs │ │ └── ui.rs │ ├── packets/ │ │ ├── Cargo.toml │ │ └── src/ │ │ └── lib.rs │ ├── server_core/ │ │ ├── Cargo.toml │ │ ├── cbindgen.toml │ │ └── src/ │ │ ├── bitrate.rs │ │ ├── c_api.rs │ │ ├── connection.rs │ │ ├── hand_gestures.rs │ │ ├── haptics.rs │ │ ├── input_mapping.rs │ │ ├── lib.rs │ │ ├── logging_backend.rs │ │ ├── sockets.rs │ │ ├── statistics.rs │ │ ├── tracking/ │ │ │ ├── body.rs │ │ │ ├── face.rs │ │ │ ├── mod.rs │ │ │ └── vmc.rs │ │ └── web_server.rs │ ├── server_io/ │ │ ├── Cargo.toml │ │ ├── README.md │ │ └── src/ │ │ ├── firewall.rs │ │ ├── lib.rs │ │ ├── openvr_drivers.rs │ │ └── openvrpaths.rs │ ├── server_openvr/ │ │ ├── Cargo.toml │ │ ├── LICENSE │ │ ├── LICENSE-Valve │ │ ├── build.rs │ │ ├── cpp/ │ │ │ ├── ALVR-common/ │ │ │ │ ├── common-utils.cpp │ │ │ │ ├── common-utils.h │ │ │ │ ├── exception.cpp │ │ │ │ ├── exception.h │ │ │ │ └── packet_types.h │ │ │ ├── alvr_server/ │ │ │ │ ├── ChaperoneUpdater.cpp │ │ │ │ ├── Controller.cpp │ │ │ │ ├── Controller.h │ │ │ │ ├── FakeViveTracker.cpp │ │ │ │ ├── FakeViveTracker.h │ │ │ │ ├── HMD.cpp │ │ │ │ ├── HMD.h │ │ │ │ ├── IDRScheduler.cpp │ │ │ │ ├── IDRScheduler.h │ │ │ │ ├── Logger.cpp │ │ │ │ ├── Logger.h │ │ │ │ ├── NalParsing.cpp │ │ │ │ ├── Paths.cpp │ │ │ │ ├── Paths.h │ │ │ │ ├── PoseHistory.cpp │ │ │ │ ├── PoseHistory.h │ │ │ │ ├── Settings.cpp │ │ │ │ ├── Settings.h │ │ │ │ ├── TrackedDevice.cpp │ │ │ │ ├── TrackedDevice.h │ │ │ │ ├── Utils.h │ │ │ │ ├── ViveTrackerProxy.cpp │ │ │ │ ├── ViveTrackerProxy.h │ │ │ │ ├── alvr_server.cpp │ │ │ │ ├── bindings.h │ │ │ │ ├── driverlog.cpp │ │ │ │ ├── driverlog.h │ │ │ │ ├── include/ │ │ │ │ │ ├── openvr_math.h │ │ │ │ │ └── picojson.h │ │ │ │ ├── nvEncodeAPI.h │ │ │ │ ├── openvr_driver_wrap.h │ │ │ │ └── shader/ │ │ │ │ ├── ColorCorrectionPixelShader.hlsl │ │ │ │ ├── CompressAxisAlignedPixelShader.hlsl │ │ │ │ ├── FoveatedRendering.hlsli │ │ │ │ ├── FrameRender.fx │ │ │ │ ├── FrameRenderPS.hlsl │ │ │ │ ├── FrameRenderVS.hlsl │ │ │ │ └── rgbtoyuv420.hlsl │ │ │ ├── platform/ │ │ │ │ ├── linux/ │ │ │ │ │ ├── CEncoder.cpp │ │ │ │ │ ├── CEncoder.h │ │ │ │ │ ├── CrashHandler.cpp │ │ │ │ │ ├── EncodePipeline.cpp │ │ │ │ │ ├── EncodePipeline.h │ │ │ │ │ ├── EncodePipelineNvEnc.cpp │ │ │ │ │ ├── EncodePipelineNvEnc.h │ │ │ │ │ ├── EncodePipelineSW.cpp │ │ │ │ │ ├── EncodePipelineSW.h │ │ │ │ │ ├── EncodePipelineVAAPI.cpp │ │ │ │ │ ├── EncodePipelineVAAPI.h │ │ │ │ │ ├── FormatConverter.cpp │ │ │ │ │ ├── FormatConverter.h │ │ │ │ │ ├── FrameRender.cpp │ │ │ │ │ ├── FrameRender.h │ │ │ │ │ ├── Renderer.cpp │ │ │ │ │ ├── Renderer.h │ │ │ │ │ ├── ffmpeg_helper.cpp │ │ │ │ │ ├── ffmpeg_helper.h │ │ │ │ │ ├── protocol.h │ │ │ │ │ └── shader/ │ │ │ │ │ ├── color.comp │ │ │ │ │ ├── color.comp.spv │ │ │ │ │ ├── ffr.comp │ │ │ │ │ ├── ffr.comp.spv │ │ │ │ │ ├── quad.comp │ │ │ │ │ ├── quad.comp.spv │ │ │ │ │ ├── rgbtoyuv420.comp │ │ │ │ │ └── rgbtoyuv420.comp.spv │ │ │ │ ├── macos/ │ │ │ │ │ ├── CEncoder.h │ │ │ │ │ └── CrashHandler.cpp │ │ │ │ └── win32/ │ │ │ │ ├── CEncoder.cpp │ │ │ │ ├── CEncoder.h │ │ │ │ ├── ColorCorrectionPixelShader.cso │ │ │ │ ├── CompressAxisAlignedPixelShader.cso │ │ │ │ ├── CrashHandler.cpp │ │ │ │ ├── FFR.cpp │ │ │ │ ├── FFR.h │ │ │ │ ├── FrameRender.cpp │ │ │ │ ├── FrameRender.h │ │ │ │ ├── FrameRenderPS.cso │ │ │ │ ├── FrameRenderVS.cso │ │ │ │ ├── NvCodecUtils.h │ │ │ │ ├── NvEncoder.cpp │ │ │ │ ├── NvEncoder.h │ │ │ │ ├── NvEncoderD3D11.cpp │ │ │ │ ├── NvEncoderD3D11.h │ │ │ │ ├── OvrDirectModeComponent.cpp │ │ │ │ ├── OvrDirectModeComponent.h │ │ │ │ ├── QuadVertexShader.cso │ │ │ │ ├── VideoEncoder.cpp │ │ │ │ ├── VideoEncoder.h │ │ │ │ ├── VideoEncoderAMF.cpp │ │ │ │ ├── VideoEncoderAMF.h │ │ │ │ ├── VideoEncoderNVENC.cpp │ │ │ │ ├── VideoEncoderNVENC.h │ │ │ │ ├── VideoEncoderSW.cpp │ │ │ │ ├── VideoEncoderSW.h │ │ │ │ ├── VideoEncoderVPL.cpp │ │ │ │ ├── VideoEncoderVPL.h │ │ │ │ ├── d3d-render-utils/ │ │ │ │ │ ├── QuadVertexShader.hlsl │ │ │ │ │ ├── RenderPipeline.cpp │ │ │ │ │ ├── RenderPipeline.h │ │ │ │ │ ├── RenderPipelineYUV.cpp │ │ │ │ │ ├── RenderPipelineYUV.h │ │ │ │ │ ├── RenderUtils.cpp │ │ │ │ │ └── RenderUtils.h │ │ │ │ ├── rgbtoyuv420.cso │ │ │ │ └── shared/ │ │ │ │ ├── d3drender.cpp │ │ │ │ └── d3drender.h │ │ │ └── shared/ │ │ │ ├── amf/ │ │ │ │ └── public/ │ │ │ │ ├── common/ │ │ │ │ │ ├── AMFFactory.cpp │ │ │ │ │ ├── AMFFactory.h │ │ │ │ │ ├── AMFMath.h │ │ │ │ │ ├── AMFSTL.cpp │ │ │ │ │ ├── AMFSTL.h │ │ │ │ │ ├── ByteArray.h │ │ │ │ │ ├── CPUCaps.h │ │ │ │ │ ├── CurrentTimeImpl.cpp │ │ │ │ │ ├── CurrentTimeImpl.h │ │ │ │ │ ├── DataStream.h │ │ │ │ │ ├── DataStreamFactory.cpp │ │ │ │ │ ├── DataStreamFile.cpp │ │ │ │ │ ├── DataStreamFile.h │ │ │ │ │ ├── DataStreamMemory.cpp │ │ │ │ │ ├── DataStreamMemory.h │ │ │ │ │ ├── IOCapsImpl.cpp │ │ │ │ │ ├── IOCapsImpl.h │ │ │ │ │ ├── InterfaceImpl.h │ │ │ │ │ ├── ObservableImpl.h │ │ │ │ │ ├── PropertyStorageExImpl.cpp │ │ │ │ │ ├── PropertyStorageExImpl.h │ │ │ │ │ ├── PropertyStorageImpl.h │ │ │ │ │ ├── Thread.cpp │ │ │ │ │ ├── Thread.h │ │ │ │ │ ├── TraceAdapter.cpp │ │ │ │ │ ├── TraceAdapter.h │ │ │ │ │ └── Windows/ │ │ │ │ │ └── ThreadWindows.cpp │ │ │ │ └── include/ │ │ │ │ ├── components/ │ │ │ │ │ ├── Ambisonic2SRenderer.h │ │ │ │ │ ├── AudioCapture.h │ │ │ │ │ ├── Capture.h │ │ │ │ │ ├── ChromaKey.h │ │ │ │ │ ├── ColorSpace.h │ │ │ │ │ ├── Component.h │ │ │ │ │ ├── ComponentCaps.h │ │ │ │ │ ├── CursorCapture.h │ │ │ │ │ ├── DisplayCapture.h │ │ │ │ │ ├── FFMPEGAudioConverter.h │ │ │ │ │ ├── FFMPEGAudioDecoder.h │ │ │ │ │ ├── FFMPEGAudioEncoder.h │ │ │ │ │ ├── FFMPEGComponents.h │ │ │ │ │ ├── FFMPEGEncoderAV1.h │ │ │ │ │ ├── FFMPEGEncoderH264.h │ │ │ │ │ ├── FFMPEGEncoderHEVC.h │ │ │ │ │ ├── FFMPEGFileDemuxer.h │ │ │ │ │ ├── FFMPEGFileMuxer.h │ │ │ │ │ ├── FFMPEGVideoDecoder.h │ │ │ │ │ ├── FRC.h │ │ │ │ │ ├── HQScaler.h │ │ │ │ │ ├── MediaSource.h │ │ │ │ │ ├── PreAnalysis.h │ │ │ │ │ ├── PreProcessing.h │ │ │ │ │ ├── SupportedCodecs.h │ │ │ │ │ ├── VQEnhancer.h │ │ │ │ │ ├── VideoCapture.h │ │ │ │ │ ├── VideoConverter.h │ │ │ │ │ ├── VideoDecoderUVD.h │ │ │ │ │ ├── VideoEncoderAV1.h │ │ │ │ │ ├── VideoEncoderHEVC.h │ │ │ │ │ ├── VideoEncoderVCE.h │ │ │ │ │ ├── VideoStitch.h │ │ │ │ │ └── ZCamLiveStream.h │ │ │ │ └── core/ │ │ │ │ ├── AudioBuffer.h │ │ │ │ ├── Buffer.h │ │ │ │ ├── Compute.h │ │ │ │ ├── ComputeFactory.h │ │ │ │ ├── Context.h │ │ │ │ ├── CurrentTime.h │ │ │ │ ├── D3D12AMF.h │ │ │ │ ├── Data.h │ │ │ │ ├── Debug.h │ │ │ │ ├── Dump.h │ │ │ │ ├── Factory.h │ │ │ │ ├── Interface.h │ │ │ │ ├── Plane.h │ │ │ │ ├── Platform.h │ │ │ │ ├── PropertyStorage.h │ │ │ │ ├── PropertyStorageEx.h │ │ │ │ ├── Result.h │ │ │ │ ├── Surface.h │ │ │ │ ├── Trace.h │ │ │ │ ├── Variant.h │ │ │ │ ├── Version.h │ │ │ │ └── VulkanAMF.h │ │ │ ├── backward.cpp │ │ │ ├── backward.hpp │ │ │ ├── threadtools.cpp │ │ │ └── threadtools.h │ │ └── src/ │ │ ├── graphics.rs │ │ ├── lib.rs │ │ ├── props.rs │ │ └── tracking.rs │ ├── session/ │ │ ├── Cargo.toml │ │ ├── build.rs │ │ └── src/ │ │ ├── lib.rs │ │ └── settings.rs │ ├── sockets/ │ │ ├── Cargo.toml │ │ └── src/ │ │ ├── control_socket.rs │ │ ├── lib.rs │ │ └── stream_socket/ │ │ ├── mod.rs │ │ ├── tcp.rs │ │ └── udp.rs │ ├── system_info/ │ │ ├── Cargo.toml │ │ └── src/ │ │ ├── android.rs │ │ └── lib.rs │ ├── vrcompositor_wrapper/ │ │ ├── Cargo.toml │ │ ├── build.rs │ │ ├── drm-lease-shim.cpp │ │ └── src/ │ │ └── main.rs │ ├── vulkan_layer/ │ │ ├── .gitignore │ │ ├── Cargo.toml │ │ ├── LICENSE │ │ ├── README.md │ │ ├── build.rs │ │ ├── layer/ │ │ │ ├── alvr_x86_64.json │ │ │ ├── device_api.cpp │ │ │ ├── device_api.hpp │ │ │ ├── layer.cpp │ │ │ ├── layer.h │ │ │ ├── private_data.cpp │ │ │ ├── private_data.hpp │ │ │ ├── settings.cpp │ │ │ ├── settings.h │ │ │ ├── surface_api.cpp │ │ │ ├── surface_api.hpp │ │ │ ├── swapchain_api.cpp │ │ │ └── swapchain_api.hpp │ │ ├── src/ │ │ │ └── lib.rs │ │ ├── util/ │ │ │ ├── custom_allocator.cpp │ │ │ ├── custom_allocator.hpp │ │ │ ├── extension_list.cpp │ │ │ ├── extension_list.hpp │ │ │ ├── logger.cpp │ │ │ ├── logger.h │ │ │ ├── platform_set.hpp │ │ │ ├── pose.cpp │ │ │ ├── pose.hpp │ │ │ ├── timed_semaphore.cpp │ │ │ └── timed_semaphore.hpp │ │ └── wsi/ │ │ ├── display.cpp │ │ ├── display.hpp │ │ ├── headless/ │ │ │ ├── surface_properties.cpp │ │ │ ├── surface_properties.hpp │ │ │ ├── swapchain.cpp │ │ │ └── swapchain.hpp │ │ ├── surface_properties.hpp │ │ ├── swapchain_base.cpp │ │ ├── swapchain_base.hpp │ │ ├── wsi_factory.cpp │ │ └── wsi_factory.hpp │ └── xtask/ │ ├── Cargo.toml │ ├── LICENSE │ ├── README.md │ ├── build.rs │ ├── firewall/ │ │ ├── alvr-firewalld.xml │ │ ├── alvr_fw_config.sh │ │ └── ufw-alvr │ ├── flatpak/ │ │ ├── README.md │ │ ├── build_and_install.sh │ │ ├── com.valvesoftware.Steam.Utility.alvr.desktop │ │ ├── com.valvesoftware.Steam.Utility.alvr.json │ │ ├── run_with_adb_keys.sh │ │ └── setup_xdg_shortcut.sh │ ├── licenses_template.hbs │ ├── patches/ │ │ ├── 0001-Add-AV_VAAPI_DRIVER_QUIRK_HEVC_ENCODER_ALIGN_64_16-f.patch │ │ ├── 0001-av1-encode-backport.patch │ │ ├── 0001-clip-constants-used-with-shift-instr.patch │ │ ├── 0001-guid-conftest.patch │ │ ├── 0001-lavu-hwcontext_vulkan-Fix-importing-RGBx-frames-to-C.patch │ │ ├── 0001-update-rc-modes.patch │ │ ├── 0001-vaapi_encode-Add-filler_data-option.patch │ │ ├── 0001-vaapi_encode-Allow-to-dynamically-change-bitrate-and.patch │ │ ├── 0001-vaapi_encode-Enable-global-header.patch │ │ └── 0001-vaapi_encode_h265-Set-vui_parameters_present_flag.patch │ ├── resources/ │ │ ├── alvr.desktop │ │ └── driver.vrdrivermanifest │ └── src/ │ ├── build.rs │ ├── ci.rs │ ├── command.rs │ ├── dependencies.rs │ ├── format.rs │ ├── main.rs │ ├── packaging.rs │ └── version.rs └── wiki/ ├── ALVR-Checklist.md ├── ALVR-wired-setup-(ALVR-over-USB).md ├── Building-From-Source.md ├── Controller-latency.md ├── FFmpeg-Hardware-Encoding-Testing.md ├── Fixed-Foveated-Rendering-(FFR).md ├── Hand-tracking-controller-bindings.md ├── Headset-and-ALVR-streamer-on-separate-networks.md ├── Home.md ├── How-ALVR-works.md ├── Information-and-Recommendations.md ├── Installation-guide.md ├── Installing-ALVR-and-using-SteamVR-on-Linux-through-Flatpak.md ├── Linux-Troubleshooting.md ├── My-game-is-not-working-properly!-Help!.md ├── Other-resources.md ├── Real-time-video-upscaling-experiments.md ├── Roadmap.md ├── Settings-tutorial.md ├── Troubleshooting.md └── _Sidebar.md ================================================ FILE CONTENTS ================================================ ================================================ FILE: .cargo/config.toml ================================================ [alias] xtask = "run -p alvr_xtask --" [target.x86_64-pc-windows-msvc] rustflags = ["-C", "target-feature=+crt-static"] ================================================ FILE: .clang-format ================================================ BasedOnStyle: WebKit IndentWidth: 4 ColumnLimit: 100 BinPackArguments: false BinPackParameters: false AlignAfterOpenBracket: BlockIndent BreakBeforeBraces: Attach InsertNewlineAtEOF: true ================================================ FILE: .editorconfig ================================================ # Editor configuration, see https://editorconfig.org [*.{c,cpp,h,hpp}] indent_style = space indent_size = 4 tab_width = 4 max_line_length = 100 end_of_line = lf insert_final_newline = true [.editorconfig] indent_style = space indent_size = 4 tab_width = 4 end_of_line = lf insert_final_newline = true ================================================ FILE: .gitattributes ================================================ * text=auto eol=lf alvr/server_openvr/cpp/alvr_server/include/** linguist-vendored alvr/server_openvr/cpp/alvr_server/nvEncodeAPI.h linguist-vendored alvr/server_openvr/cpp/platform/win32/NvCodecUtils.h linguist-vendored alvr/server_openvr/cpp/platform/win32/NvEncoder.cpp linguist-vendored alvr/server_openvr/cpp/platform/win32/NvEncoder.h linguist-vendored alvr/server_openvr/cpp/shared/** linguist-vendored # note: some of these folders contain ALVR code, but only a minor amount alvr/vulkan_layer/layer/** linguist-vendored alvr/vulkan_layer/util/** linguist-vendored alvr/vulkan_layer/wsi/** linguist-vendored ================================================ FILE: .github/FUNDING.yml ================================================ # These are supported funding model platforms github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] patreon: # Replace with a single Patreon username open_collective: alvr ko_fi: # Replace with a single Ko-fi username tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry liberapay: # Replace with a single Liberapay username issuehunt: # Replace with a single IssueHunt username otechie: # Replace with a single Otechie username custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] ================================================ FILE: .github/ISSUE_TEMPLATE/bug_report.md ================================================ --- name: Bug Report about: Create a report to help us improve title: '' labels: '' assignees: '' --- ## Description ## General Troubleshooting - [ ] I carefully followed the instructions in the [README](https://github.com/alvr-org/ALVR/blob/master/README.md) and successfully completed the setup wizard - [ ] I read the [ALVR GitHub Wiki](https://github.com/alvr-org/ALVR/wiki) ## Environment ### Hardware **Note**: for Linux, an upload to the [`hw-probe`](https://linux-hardware.org/) database is preferred: `hw-probe -all -upload` **CPU**: **GPU**: **GPU Driver Version**: **Audio**: ### Installation **ALVR Version**: **ALVR Settings File**: **SteamVR Version**: **Install Type**: - [ ] Packaged (`exe`, `deb`, `rpm`, etc) - [ ] Portable (`zip`) - [ ] Source **OS Name and Version** (`winver` on Windows or `grep PRETTY_NAME /etc/os-release` on most Linux distributions): ================================================ FILE: .github/dependabot.yml ================================================ # To get started with Dependabot version updates, you'll need to specify which # package ecosystems to update and where the package manifests are located. # Please see the documentation for all configuration options: # https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates version: 2 updates: - package-ecosystem: "cargo" # See documentation for possible values directory: "/" # Location of package manifests schedule: interval: "monthly" ignore: - dependency-name: "*" update-types: [ "version-update:semver-minor","version-update:semver-patch" ] ================================================ FILE: .github/workflows/prepare-release.yml ================================================ name: Create release env: CARGO_TERM_COLOR: always on: workflow_dispatch: inputs: version: description: "Version" required: false default: "" jobs: prepare_release: runs-on: windows-2022 outputs: release_ref: ${{ steps.output_ref.outputs.release_ref }} upload_url: ${{ steps.create_release.outputs.upload_url }} release_id: ${{ steps.create_release.outputs.id }} steps: - name: Configure git run: git config --global core.autocrlf false - uses: actions/checkout@v2 - uses: actions-rs/toolchain@v1 with: toolchain: stable - name: Bump version id: bump_version env: RUST_BACKTRACE: 1 run: | $versionarg = "${{ github.event.inputs.version }}" $versionarg = If ($versionarg.Length -gt 0) { "--version $versionarg" } else { "" } $out = cargo xtask bump $versionarg.split() echo $out cargo update -p alvr_common echo "::set-output name=version_tag::$(echo $out | sls -CaseSensitive -Pattern '^v.*$')" - name: Push changes uses: stefanzweifel/git-auto-commit-action@v4 with: commit_message: "[Auto] Bump version" - name: Output ref for later checkouts id: output_ref run: echo "::set-output name=release_ref::$(git rev-parse HEAD)" - name: Create Release id: create_release uses: actions/create-release@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: tag_name: ${{ steps.bump_version.outputs.version_tag }} release_name: ALVR ${{ steps.bump_version.outputs.version_tag }} draft: true prerelease: false commitish: ${{ steps.output_ref.outputs.release_ref }} build_windows_streamer: runs-on: windows-2022 needs: [prepare_release] steps: - uses: actions/checkout@v2 with: ref: ${{ needs.prepare_release.outputs.release_ref }} submodules: true - uses: actions-rs/toolchain@v1 with: toolchain: stable override: true - uses: crazy-max/ghaction-chocolatey@v1 with: args: install zip unzip pkgconfiglite wixtoolset - name: Build and package ALVR id: build env: RUST_BACKTRACE: 1 run: | cargo xtask package-streamer --gpl --ci cargo xtask package-launcher --ci - name: Upload streamer uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ needs.prepare_release.outputs.upload_url }} asset_path: ./build/alvr_streamer_windows.zip asset_name: alvr_streamer_windows.zip asset_content_type: application/zip - name: Upload launcher uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ needs.prepare_release.outputs.upload_url }} asset_path: ./build/alvr_launcher_windows.zip asset_name: alvr_launcher_windows.zip asset_content_type: application/zip build_linux_streamer: runs-on: ubuntu-22.04 needs: [prepare_release] steps: - uses: actions/checkout@v2 with: ref: ${{ needs.prepare_release.outputs.release_ref }} submodules: true - uses: actions-rs/toolchain@v1 with: toolchain: stable override: true - name: Build and install dependencies env: RUST_BACKTRACE: 1 run: | sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 25088A0359807596 echo "deb http://ppa.launchpad.net/pipewire-debian/pipewire-upstream/ubuntu $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/pipewire-upstream.list sudo apt-get update sudo apt-get install libfuse2 build-essential pkg-config nasm libva-dev libdrm-dev libvulkan-dev libx264-dev libx265-dev cmake libasound2-dev libjack-jackd2-dev libxrandr-dev libunwind-dev libgtk-3-dev libpipewire-0.3-dev libspa-0.2-dev - name: Build and package ALVR (.tar.gz) id: build env: RUST_BACKTRACE: 1 run: | cargo xtask package-streamer --gpl --ci cargo xtask package-launcher --ci - name: Upload streamer (tar.gz) uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ needs.prepare_release.outputs.upload_url }} asset_path: ./build/alvr_streamer_linux.tar.gz asset_name: alvr_streamer_linux.tar.gz asset_content_type: application/gzip - name: Upload launcher uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ needs.prepare_release.outputs.upload_url }} asset_path: ./build/alvr_launcher_linux.tar.gz asset_name: alvr_launcher_linux.tar.gz asset_content_type: application/gzip build_flatpak_bundle: runs-on: ubuntu-latest needs: [prepare_release] steps: - uses: actions/checkout@v2 with: ref: ${{ needs.prepare_release.outputs.release_ref }} submodules: true - name: Build and install dependencies env: RUST_BACKTRACE: 1 run: | sudo apt-get update sudo apt-get install flatpak flatpak-builder sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo - name: Build and package ALVR flatpak (.flatpak) id: build_flatpak run: | sudo flatpak-builder --repo=.flatpak-repo --install-deps-from=flathub --force-clean --default-branch=stable --arch=x86_64 .flatpak-build-dir alvr/xtask/flatpak/com.valvesoftware.Steam.Utility.alvr.json flatpak build-bundle .flatpak-repo com.valvesoftware.Steam.Utility.alvr.flatpak com.valvesoftware.Steam.Utility.alvr stable --runtime - name: Upload flatpak streamer for Linux uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ needs.prepare_release.outputs.upload_url }} asset_path: com.valvesoftware.Steam.Utility.alvr.flatpak asset_name: com.valvesoftware.Steam.Utility.alvr.flatpak asset_content_type: application/octet-stream build_android_client: runs-on: ubuntu-latest needs: [prepare_release] steps: - uses: actions/checkout@v2 with: ref: ${{ needs.prepare_release.outputs.release_ref }} submodules: true - uses: actions-rs/toolchain@v1 with: toolchain: stable target: aarch64-linux-android override: true - uses: actions/setup-java@v2 with: distribution: "temurin" java-version: "17" - uses: android-actions/setup-android@v3 with: packages: "platforms;android-32" - uses: nttld/setup-ndk@v1 id: setup-ndk with: ndk-version: r26b - name: Build and package ALVR id: build env: RUST_BACKTRACE: 1 ANDROID_NDK_HOME: ${{ steps.setup-ndk.outputs.ndk-path }} run: cargo xtask package-client --ci - name: Sign APK uses: ilharp/sign-android-release@v1 id: sign_apk with: releaseDir: build/alvr_client_android signingKey: ${{ secrets.SIGNING_KEY }} keyAlias: ${{ secrets.KEY_ALIAS }} keyStorePassword: ${{ secrets.KEY_STORE_PASSWORD }} keyPassword: ${{ secrets.KEY_PASSWORD }} buildToolsVersion: 34.0.0 - name: Upload APK uses: actions/upload-release-asset@v1 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: upload_url: ${{ needs.prepare_release.outputs.upload_url }} asset_path: ${{ steps.sign_apk.outputs.signedFile }} asset_name: alvr_client_android.apk asset_content_type: application/vnd.android.package-archive ================================================ FILE: .github/workflows/queue.yml ================================================ name: Merge queue only checks # Merge queue checks need to be selected as required for prs to actually run, so simply skip them on prs on: pull_request: merge_group: env: CARGO_TERM_COLOR: always jobs: check-linux-old: if: github.event_name == 'merge_group' runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable with: components: clippy - uses: Swatinem/rust-cache@v2 - name: Install dependencies run: | sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 25088A0359807596 echo "deb http://ppa.launchpad.net/pipewire-debian/pipewire-upstream/ubuntu $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/pipewire-upstream.list sudo add-apt-repository universe sudo apt-get update sudo apt-get install libfuse2 build-essential pkg-config nasm libva-dev libdrm-dev libvulkan-dev libx264-dev libx265-dev cmake libasound2-dev libjack-jackd2-dev libxrandr-dev libunwind-dev libgtk-3-dev libpipewire-0.3-dev libspa-0.2-dev - name: Prepare deps env: RUST_BACKTRACE: 1 run: cargo xtask prepare-deps --platform linux --no-nvidia - run: cargo clippy check-msrv-windows: if: github.event_name == 'merge_group' runs-on: windows-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 # Fix actions rust install and msrv both being stupid - run: rustup update - name: Prepare deps env: RUST_BACKTRACE: 1 run: cargo xtask prepare-deps --platform windows - run: cargo xtask check-msrv check-msrv-linux: if: github.event_name == 'merge_group' runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 - name: Install dependencies run: | sudo apt update sudo apt install build-essential pkg-config nasm libva-dev libdrm-dev libvulkan-dev libx264-dev libasound2-dev libxrandr-dev libunwind-dev libgtk-3-dev libpipewire-0.3-dev libspa-0.2-dev - name: Prepare deps env: RUST_BACKTRACE: 1 run: cargo xtask prepare-deps --platform linux --no-nvidia - run: cargo xtask check-msrv check-licenses: if: github.event_name == 'merge_group' runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 - run: cargo xtask check-licenses ================================================ FILE: .github/workflows/rust.yml ================================================ name: Rust on: pull_request: branches: [master] merge_group: env: CARGO_TERM_COLOR: always jobs: check-windows: runs-on: windows-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable with: components: clippy - uses: Swatinem/rust-cache@v2 - name: Prepare deps env: RUST_BACKTRACE: 1 run: cargo xtask prepare-deps --platform windows - run: cargo xtask clippy --ci check-linux: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable with: components: clippy - uses: Swatinem/rust-cache@v2 - name: Install dependencies run: | sudo apt update sudo apt install build-essential pkg-config nasm libva-dev libdrm-dev libvulkan-dev libx264-dev libasound2-dev libxrandr-dev libunwind-dev libgtk-3-dev libpipewire-0.3-dev libspa-0.2-dev - name: Prepare deps env: RUST_BACKTRACE: 1 run: cargo xtask prepare-deps --platform linux --no-nvidia - run: cargo xtask clippy --ci check-macos: runs-on: macos-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable with: components: clippy - uses: Swatinem/rust-cache@v2 # This step currently does nothing on macos, but might in the future # - run: cargo xtask prepare-deps --platform macos - run: cargo clippy build-android: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable with: targets: aarch64-linux-android - uses: Swatinem/rust-cache@v2 - uses: actions/setup-java@v2 with: distribution: 'temurin' java-version: '17' - uses: android-actions/setup-android@v3 with: packages: 'platforms;android-32' - uses: nttld/setup-ndk@v1 id: setup-ndk with: ndk-version: r26b - name: Install deps run: cargo install cargo-apk # Create folder without content to make the build succeed - run: mkdir -p deps/android_openxr/arm64-v8a - name: Build client env: ANDROID_NDK_HOME: ${{ steps.setup-ndk.outputs.ndk-path }} working-directory: ./alvr/client_openxr run: cargo xtask build-client tests: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: submodules: true - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 - name: Run tests run: cargo test -p alvr_session check-format: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable with: components: rustfmt - run: cargo xtask check-format ================================================ FILE: .github/workflows/stale.yml ================================================ name: 'Close stale issues' on: schedule: - cron: '0 0 * * *' jobs: stale: runs-on: ubuntu-latest steps: - uses: actions/stale@v9 with: stale-issue-message: 'This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.' exempt-issue-labels: 'bug,documentation,enhancement,good-first-issue,hep-wanted,needs-testing,release' exempt-draft-pr: true ================================================ FILE: .github/workflows/wiki.yml ================================================ name: Publish to GitHub Wiki on: push: branches: [master] workflow_dispatch: jobs: wiki: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Publish to wiki uses: cmbrose/github-docs-to-wiki@v0.24 with: githubToken: ${{ secrets.GH_PAT }} rootDocsFolder: "wiki" ================================================ FILE: .gitignore ================================================ Debug Release *.opensdf *.sdf *.suo *.filters *.user *.dll *.exe *.mexw64 *.aps # editor .vs/ .idea/ # NuGet Packages *.nupkg # The packages folder can be ignored because of Package Restore **/[Pp]ackages/* # except build/, which is used as an MSBuild target. !**/[Pp]ackages/build/ # Uncomment if necessary however generally it will be regenerated when needed #!**/[Pp]ackages/repositories.config # NuGet v3's project.json files produces more ignorable files *.nuget.props *.nuget.targets release-files/ # Ignore generated file with harvested file paths for WiX wix/harvested.wxs # Other WiX files *.wixobj *.wixpdb *.msi # Generated by Cargo # will have compiled files and executables /target/ # Remove Cargo.lock from gitignore if creating an executable, leave it for libraries # More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html #Cargo.lock # These are backup files generated by rustfmt **/*.rs.bk # build result /build/ dist .vscode/ .DS_Store .ccls-cache # dependencies installed by xtask deps # dependencies installed by node node_modules # The .gitignore for android is project folder. # flatpak .flatpak* ALVR-Launcher #allows adding CMake files for some IDEs to parse C++ files correctly CMakeLists.txt cmake-build-debug/ *.jks ================================================ FILE: .gitmodules ================================================ [submodule "openvr"] path = openvr url = https://github.com/ValveSoftware/openvr.git shallow = true ================================================ FILE: CHANGELOG.md ================================================ # Changelog ## v20.11.0 * Add flatpak launcher (by @failboat78 #2207) * Fix Linux/Nvenc error popups (by @failboat78 #2338) * Add debug groups (by @zarik5 #2332) * Fix black screen on Focus 3 (by @zarik5 #2346) * Fix launcher installation popup (by @zarik5 #2356) * Fix crash on invalid haptics (by @zarik5 #2355) * Rework logging (by @zarik5 #2351) * Disable "Adapt to framerate" by default (by @zarik5) * Show body skeleton in lobby (by @zarik5 #2366) * Add multimodal input (by @zarik5 #2367) * Fix high CPU usage on Linux (by @The-personified-devil #2372 #2375) * Fix stream being half width resolution (by @zarik5 #2378) ## v20.10.0 * Internal refactors of the client graphics (wgpu) and server code architecture (by @zarik5) * Fix Quest 3 not supporting 120Hz (by @The-personified-devil) * Tracking timing jitter (by @zarik5 #2285) * Support VRChat hand tracking (by @ReinaS-64892 @AdalynBlack @zarik5 #2295, @zarik5 #2313 #2323) * Fix crash with some games (by @szneqz #2309) * Fix crash when shutting down launcher (by @zarik5 #2320) * Fix crash with vibration on Focus 3 (by @zarik5 #2324) ### v20.9.1 (2024-07-06) * Fix performance issues on lobby room. ## v20.9.0 (2024-07-06) * Display license inside the dashboard (by @zarik5 #2117) * Add GPU checks for Linux (by @Meister1593 #2110) * Reorder settings (by @zarik5 #2119) * Shallow rename "client" -> "device" (by @zarik5 #2120) * Update manifest for AppLab (by @zarik5 #2146) * Allow recentering in the lobby (by @zarik5 #2155) * Tweak graph labels and colors (by @zarik5 #2176) * Fix controller detection on Pico Neo 3 Link (by @HoLo85 #2192) * Loosen restrictions on device selection with VoiceMeeter (by @xuan25 #2209) * Fix HEVC black screen on Linux/VAAPI (by @Nibor62 #2203) * Show hands and controllers in the lobby (by @zarik5 #2218) * Add PipeWire support on Linux (by @Meister1593 #1973) ### v20.8.1 (2024-05-08) * Fix crash on Linux (by @SniperJoe #2108) ## v20.8.0 (2024-05-04) * Bring back settings tabs (by @Meister1593 #2076) * Make FFE shader much lighter (by @yoyobuae #2083) * Improve firewall rules on Linux (by @Meister1593 #2078) * Fix error message not displaying correctly on Linux sometimes (by @SniperJoe #2088) * Fix Nvidia encoder on some Linux systems (by @Xaphiosis @nowrep #2074) * Fix client warmstart crash (by @ShootingKing-AM #2084) * Fix segfault on Linux (by @SniperJoe #2090) * Fix protocol break in v20.7.0 (by @zarik5 2098) ### v20.7.1 (2024-04-11) * Fix colors on Pico (by @shinyquagsire23 and @zarik5) * Fix joystick gesture offset on right hand (by @jarettmillard #2065) ## v20.7.0 (2024-04-06) * Add AV1 support on Windows/AMD (by @barnabwhy #1967) * Add AV1 support on Linux/Nvidia (by @wsippel #1975, @Vixea #1994) * Add HDR support on Windows (by @shinyquagsire23 #2030) * Add full range encoding support (by @shinyquagsire23 #2011, @Vixea #1971) * Fix VAAPI encoder on Intel GPUs (by @nowrep #1981) * Add Pre-Analysis support on Windows/AMD (by @barnabwhy #1985) * Add linux hardware encoding checks (by @Meister1593 #2042 #2055) * Add full body tracking support on Quest (by @barnabwhy #1979, @galister #1984) * Fix aux-bones in SteamVR (by @shinyquagsire23 #2009) * Make USB connection work when WiFi is disabled (by @Meister1593 #1962) * Add mDNS discovery on the server (by @zarik5 #1978) * Fix audio when disconnecting headphones in headset (by @Okabintaro #2025 #2040) * Add compatibility layer for surround audio devices (by @barnabwhy #2026) * Fix black screen on Vive Focus 3, XR Elite (by @zarik5) * Fix stuttering on Linux after recentering (by @galister #2017) * Fixes for Flatpak (by @jkcdarunday #1980, @Meister1593 #2039 #2044) * Change colors of lobby room (by @barnabwhy #1968) * Remove WIX installer (by @zarik5) * Remove AppImage (by @Meister1593 #2056) ### v20.6.1 (2024-01-26) * Add AV1 support, only for Linux/VAAPI, with 10bits support (by @wsippel #1955 #1964) * Fix image corruption on h264/VAAPI (by @galister / @nowrep #1956) ## v20.6.0 (2024-01-10) * Add tongue tracking for Quest Pro (by @zarik5) * This is a breaking change in the protocol, but only affects Quest Pro users. * Only VRCFT ALVR module v1.2.0 and up is supported * Add Quest 3 emulation mode + icons for SteamVR HUD (by @Goodguy140 #1926) * Add Type of Service (ToS) socket settings, tested only on Linux (by @Vixea #1946) * Add software decoding option and fallback (by @20kdc #1933) * Add Baseline encoding option for h264 (by @20kdc #1932) * Fix ADB connection (by @The-personified-devil #1942) * Fix rare bug preventing reconnections on wifi (by @zarik5) ## v20.5.0 (2023-12-05) * Fix Vulkan layer GPU loading (by @nairaner #1847) * Fix dynamic bitrate for VAAPI (by @nowrep #1863) * Add notification tips (by @zarik5 #1865) * Fix hand tracking for Lynx R1 (by @technobaboo #1874) * Various wiki updates * Fix battery update during streming (by @zarik5) * Fix playspace recentering delay (by @zarik5) * Support eye tracking for Pico 4 Pro (by @zarik5 @Meister1593 #1897) * Add desktop file for Flatpak (by @Vixea #1906) * Install audio dependencies from the setup wizard (by @Meister1593 #1893, @zarik5) * Significantly reduce latency with NVENC on Linux (by @nowrep @Xaphiosis #1911) * Fix SteamVR hanging when restarting on Linux (by @Vixea @zarik5) * Other dashboard updates ### v20.4.3 (2023-10-06) * Fix dashboard crash on Windows * Fix settings reset bug when upgrading (session extrapolation failed) ### v20.4.2 (2023-09-22) * Fix YVR crash because of invalid controller bindings ### v20.4.1 (2023-09-19) * Fix inverted `Enable skeleton` switch * Add `Only touch` gestures option ## v20.4.0 (2023-09-15) * Full hand tracking gestures support, with joystick (by @barnabwhy #1794) * Fully remappable controller buttons (by @zarik5 #1817) * Custom controller profile (by @zarik5) * Fix Nvidia support on Linux (by @Killrmemz #1830) ### v20.3.1 (2023-09-11) * Fix some controller buttons not working * Fix changing controller emulation profile not triggering a SteamVR restart * Add back Rift S controller emulation profile ## v20.3.0 (2023-09-08) * Add Lynx R1 headset support (by @zarik5 #1823) * Currently there is an issue with hand tracking which is being investigated * Make settings sections collapsible (by @zarik5) * Other UI tweaks (by @zarik5) * *Actually* fix controller freeze (by @zarik5) * Fix Pico controller buttons (by @zarik5 @galister @Meister1593 #1820) * Fix bitrate hikes when "Adapt to framerate" is enabled (by @zarik5) * Fix Nvenc encoder on Linux (by @Killrmemz #1824) * Timeout connection if lingering (by @zarik5) * Fix warmstart crash on client (by @ShootingKing-AM #1813) ### v20.2.1 (2023-08-27) * Fix VRCFaceTracking mode panicing. * (Potential) Fix for dashboard crash on Wayland. ## v20.2.0 (2023-08-26) * Add Flatpak build (by @CharlieQLe #1683 #1724 #1735 #1742, @Meister1593 #1769) * Finish VRCFaceTracking support (by @zarik5) * You can download the ALVR Module from the VRCFaceTracking app itself. * Only supports the Quest Pro at the moment. * New more performant sockets implementation (by @zarik5) * Zero copy + zero allocations, and provides better packet prioritization. * Avoid controller freezing during high latency (by @zarik5) * Add message popups on Linux (disabled on the appimage build) (by @zarik5 #1711) * Show backtrace on unhandled exceptions (Windows only) (by @zarik5) * Previously these would make SteamVR hard crash without any useful log * Optionally show full backtraces for logs (by @zarik5) * Add option to select client log level (by zarik5) * Make Log tab stick to bottom (by @zarik5) * Encoder fixes on Linux (by @nowrep #1751 #1753 #1767 #1768 #1796, @Vixea #1805) * Use Constant bitrate mode by default * Support rolling video recording (by @zarik5) * Fix OpenGL crash on the client (by @ShootingKing-AM #1801) * Fix white dashboard bug on Linux (by @zarik5) ## v20.1.0 (2023-06-20) * Fix firewall rules on Windows (by @zarik5) * Fix firewall rules on linux for the tar.gz (by @Vixea #1675) * Add bitrate graph (by @zarik5 #1689) * Add encoder latency limiter (by @zarik5 #1678) * Fix network latency limiter (by @zarik5) * Fix image corruption on AMD (by @zarik5 #1681) * Fix dashboard audio dropdowns on Linux (by @zarik5) * Add connection status for clients (by @zarik5 #1688) * Fix HMD plugged status (by @zarik5) * Fix crash on some Unreal Engine 5 games (by @deiteris #1685) * Add option to disable game render optimization (by @zarik5) * Add separate history size for bitrate (by @zarik5) # v20.0.0 (2023-06-02) * New OpenXR-based client, add support for Vive Focus 3/XR Elite, Pico 4/Neo 3 and YVR 1/2. Worked on by: * @zarik5 #1321 * @galister #1321, #1442 * @deiteris #1434, #1439, #1445 * New egui (OpenGL) dashboard * The launcher is replaced by the new dashboard executable. * by @Kirottu #1247 #1274, @zarik5, @m00nwtchr #1292, @TheComputerNerd88 #1574 #1575 #1576 1582 * Add position and rotation recentering modes (by @zarik5 #1321) * Defaults to local floor and local yaw. * Add support for eye and face tracking (by @zarik5 #1577) * Currently supporting VRChat Eye OSC, VRCFaceTracking support coming soon * Reduce game rendering latency (by @zarik5) * Apply some settings in real-time (by @zarik5 #1635) * New more consistent controller prediction algorithm (by @zarik5 #1561) * Controller input fixes (by @zarik5 #1560) * Soft-toggle controllers at runtime (by @galister #1600) * New wiki hosted in the main git tree (by @m00nwtchr #1309) * Send client log to streamer (by @zarik5) * Encoder improvements (by @nowrep #1562 #1565 #1568, @deiteris #1403 #1422 #1400 #1402, @zarik5 #1564) * Remove Forward Error Correction (by @zarik5: #1384, #1389; @deiteris: #1386, #1387, #1390) * Unified code for NAL parsing (by @deiteris #1403, #1422, #1400, #1402) * Some tweaks for alvr_client_core compatibility (by @ShootingKing-AM #1580 #1578 #1586 #1624 #1621) * Fix server build with clang (by @nowrep) ### v19.1.1 (2023-03-03) * Relax discovery protocol for future ALVR versions ## v19.1.0 (2023-02-14) * Encoder improvements and new Linux server compositor: * @deiteris #1227, #1252, #1281, #1287, #1302, #1304, #1318, #1331, #1336, #1368, #1393, #1367, #1397 * @Vixea #1227, #1254, #1412 * @nowrep #1251, #1261, #1253, #1267, #1264, #1266, #1273, #1272, #1277, #1280, #1279, #1278, #1282, #1265, #1294, #1295, #1312, #1314, #1316, #1325, #1328, #1326, #1330, #1334, #1338, #1329, #1346, #1350, #1357, #1352, #1348, #1365, #1349, #1361, #1370, #1372, #1393 * @m00nwtchr #1347 * @galister #1428, #1429 * Controller fixes (by @Timocop #1236 #1241) * Vulkan layer fixes (by @nowrep #1291, #1293, #1324, #1339, #1376) * Show client IP in the headset and dashboard (by @zarik5) * Disable Wi-Fi scans (by @zarik5) * Reduce lag after loading screens (by @zarik5) * Fix server debug builds (by @nowrep #1288) * Add trigger/grip threshold (by @sctanf) * Don't spam stdout on Linux (by @nowrep #1317) * Fix recentering on Linux (by @nowrep #1353) ================================================ FILE: CONTRIBUTING.md ================================================ # Style Checklist for code style. This is on top of common Rust styling rules. These rules are not mandatory but I might point them out if not respected in PRs :) -zarik ## Naming - Respect Rust naming conventions (not respecting this will cause a warning). - Add useful information in the name, with the exception of indices for iterating. - Do not put type or scope information in the name. - Avoid abbreviations. - Avoid prefixes and suffixes. - if necessary prefer suffixes rather than prefixes. - `_ref` or `_mut` suffixes are accepted when you want to put emphasis on that the variable is a mutable reference, and so assigning values has side effects, even if not consumed later. Suffixes are not needed if the variable is only mutable or instead a immutable reference. - Use `maybe_` prefix if the variable is an `Option` or `Result`. Omit if it's clear from the context. Never use it for parameter or field definitions. - Shadowing is encouraged. - If shadowing cannot be used and two variables have similar meaning but different types, suffix the variable with the least useful or least specialized type with its type. Example: ```rust let myfile_string = "./file.txt"; let myfile = Path::new(my_file_string); ``` - If both directory and file paths are used in the same context, suffix directories with `_dir` and files with `_path`. Suffix file names with `_fname`. ## Top level definitions - For each file, define in order: private imports, public imports, ffi bindings import, private constants, public constants, private structs, public structs, private top level functions, public top level functions. ### Imports - Do not leave spaces between imports, only between the private and public import blocks. - Define imports in alphabetical order (use cargo fmt). - Group imports using braces when there are common parts of the path. ### Structs - Prefer adding trait bounds to the impl type parameters instead of struct type parameters - Define in order: struct, Default impl, custom impl, Drop impl, all in the same module. Do not split the custom impl. ## Spacing Smartly use empty newlines between blocks of code to improve legibility - Rust recommends not to specify the return keyword when returning at the end of a function. To put emphasis on the return expression, isolate it with a empty new line just before. - Inside each block (function, if/else/while/for etc or just braces) make so that there are roughly 2 to 6 blocks of code separated by empty lines for the amount of code that fits in a single screen (long functions that span multiple screens can have way more than 6 block). Check existing codebase for an example. - Spaces between groups of statements should be done so the groups are similar is size and that each achieves a specific purpose. You should be able to easily describe what the group does in few words, even if you don't comment it (because the meaning should be self evident). - Do not define variables at the start of the function/block, but do define them at the start of the functional group delimited by spaces. ## Comments - It's important to use comments when the meaning or inner workings of a piece of code is not clear from the context. Well-named symbols (variables and functions) are often enough. In doubt do use comments. - Do not add comments about how certain parts of the language/std library work, unless it's about quirks of features. ## Panicking and error handling - Use of `panic!()` is discouraged - When matching exhaustively, prefer `unreachable!()` over `panic!()` for certain branches. - `unwrap()` is discouraged, but prefer `unwrap()` over `expect()` (bubble up the error instead). - Prefer `.get()` to index a collection rather than `[]`, unless extremely certain it will never index out of bounds. - Add a `// # Safety` comment before a statement that contains a `unwrap()` or raw indexing, explaining why it should never crash. - Use `todo!()` to mark unfinished code (it returns `!` and so it helps with the missing return statement). ## Code repetition and maintainability - Lean towards the DRY rule, without overdoing it. - Extract a piece of code (in a function or lambda) only when it is used two or more times and it doesn't depend on many parameters. - Always extract constants for "arbitrary" values (literals) which are chosen with no absolute rule and makes sense to change in the future. Example: time interval between resending discovery packet. Opposite example: number of eyes on a human head (it's always going to be 2, no need to use a constant to change its value in the future :) ). - Prefer defining constants at the start of the file, even if used locally in a single function. - Prefer using "complex" types for constants, if the std library allows it. Example: prefer using `Duration` instead of an integer type if the constant represents a time duration. Same with `Path` vs string. ## Structural soundness - Try to avoid invalid states in the data, using Rust enums. Example: do not use `resumed` + `streaming` boolean variables if the state `resumed == false` + `streaming == true` is invalid. Instead use `enum State { Paused, Resumed, Streaming }`. - Make full use of pattern matching with `if let` and `while let`, this reduces the use of `unwrap()`. ================================================ FILE: Cargo.toml ================================================ [workspace] resolver = "2" members = ["alvr/*"] [workspace.package] version = "21.0.0-dev12" edition = "2024" rust-version = "1.88" authors = ["alvr-org"] license = "MIT" [workspace.dependencies] alvr_adb = { path = "alvr/adb" } alvr_audio = { path = "alvr/audio" } alvr_client_core = { path = "alvr/client_core" } alvr_common = { path = "alvr/common" } alvr_events = { path = "alvr/events" } alvr_filesystem = { path = "alvr/filesystem" } alvr_graphics = { path = "alvr/graphics" } alvr_gui_common = { path = "alvr/gui_common" } alvr_packets = { path = "alvr/packets" } alvr_server_core = { path = "alvr/server_core"} alvr_server_io = { path = "alvr/server_io" } alvr_session = { path = "alvr/session" } alvr_sockets = { path = "alvr/sockets" } alvr_system_info = { path = "alvr/system_info" } [profile.release] debug = "limited" strip = false [profile.distribution] inherits = "release" lto = true ================================================ FILE: LICENSE ================================================ Copyright (c) 2018-2019 polygraphene Copyright (c) 2020-2024 alvr-org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================

# ALVR - Air Light VR [![badge-discord][]][link-discord] [![badge-matrix][]][link-matrix] [![badge-opencollective][]][link-opencollective] Stream VR games from your PC to your headset over Wi-Fi. This is a fork of [ALVR](https://github.com/polygraphene/ALVR). ### Direct download (latest version): ### [Windows Launcher](https://github.com/alvr-org/ALVR/releases/latest/download/alvr_launcher_windows.zip) | [Linux Launcher](https://github.com/alvr-org/ALVR/releases/latest/download/alvr_launcher_linux.tar.gz) ## Compatibility | VR Headset | Support | | :--------------------------: | :------------------------------------------------------------------------------------: | | Apple Vision Pro | :heavy_check_mark: ([store link](https://apps.apple.com/app/alvr/id6479728026)) | | Quest 1/2/3/3S/Pro | :heavy_check_mark: ([store link](https://www.meta.com/experiences/7674846229245715) *) | | Pico Neo 3/4/4 Ultra | :heavy_check_mark: | | Play For Dream YVR 1/2/MR | :heavy_check_mark: | | Vive Focus 3/Vision/XR Elite | :heavy_check_mark: | | Lynx R1 | :heavy_check_mark: | | PhoneVR (smartphone) | :heavy_check_mark: ** ([repo](https://github.com/PhoneVR-Developers/PhoneVR)) | | Android/Monado | :warning: ** | | Oculus Go | :x: ([old repo](https://github.com/polygraphene/ALVR)) | \* ALVR for Quest 1 is not available through the Meta store. \** Works on some smartphones, but has not been extensively tested. | PC OS | Support | | :------------: | :---------------------------------------------------------------------------: | | Windows 10/11 | :heavy_check_mark: ([store link](https://store.steampowered.com/app/3312710)) | | Windows XP/7/8 | :x: | | Linux | :heavy_check_mark:*** | | macOS | :x: | \*** Please check the wiki for detailed compatibility information. ### Requirements - A supported standalone VR headset (see compatibility table above). - SteamVR. - A high-end gaming PC: - See the OS compatibility table above. - NVIDIA GPU with NVENC support (GTX 1000 series or newer), an AMD GPU with AMF VCE support, or an INTEL GPU with VPL support (Arc, Tiger Lake or newer), with the latest drivers. - On laptops with both an integrated GPU (Intel HD, AMD iGPU) and a dedicated GPU (NVIDIA GTX/RTX, AMD HD/R5/R7), make sure to assign the dedicated GPU (or "high performance graphics adapter") to ALVR and SteamVR for the best performance and compatibility. (NVIDIA: Nvidia Control Panel → 3D Settings → Application Settings; AMD: similar method) - Network: - 802.11ac 5 GHz Wi-Fi for the headset, and wired Ethernet for the PC is recommended. - The PC and the headset must be connected to the same router (or use a routed connection as described [here](https://github.com/alvr-org/ALVR/wiki/ALVR-v14-and-Above)). ## Installation Follow the [installation guide](https://github.com/alvr-org/ALVR/wiki/Installation-guide). ## Troubleshooting - See the [Troubleshooting](https://github.com/alvr-org/ALVR/wiki/Troubleshooting) page, and [Linux Troubleshooting](https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting) if applicable. - Configuration recommendations and additional information can be found [here](https://github.com/alvr-org/ALVR/wiki/Information-and-Recommendations). ## Uninstallation Open `ALVR Dashboard.exe`, go to the `Installation` tab, then press `Remove firewall rules`. Close the ALVR window and delete the ALVR folder. ## Build from Source Follow the [build guide](https://github.com/alvr-org/ALVR/wiki/Building-From-Source). ## License ALVR is licensed under the [MIT License](LICENSE). ## Privacy Policy ALVR apps do not directly collect any personal data. ## Donate If you would like to support this project, you can donate through our [Open Source Collective account](https://opencollective.com/alvr). [badge-discord]: https://img.shields.io/discord/720612397580025886?style=for-the-badge&logo=discord&color=5865F2 "Join us on Discord" [link-discord]: https://discord.gg/ALVR [badge-matrix]: https://img.shields.io/static/v1?label=chat&message=%23alvr&style=for-the-badge&logo=matrix&color=blueviolet "Join us on Matrix" [link-matrix]: https://matrix.to/#/#alvr:ckie.dev?via=ckie.dev [badge-opencollective]: https://img.shields.io/opencollective/all/alvr?style=for-the-badge&logo=opencollective&color=79a3e6 "Donate" [link-opencollective]: https://opencollective.com/alvr ================================================ FILE: about.toml ================================================ accepted = [ "MIT", "Apache-2.0", "Apache-2.0 WITH LLVM-exception", "BSD-2-Clause", "BSD-3-Clause", "BSL-1.0", "bzip2-1.0.6", "CC0-1.0", "CDLA-Permissive-2.0", "ISC", "LicenseRef-UFL-1.0", "MPL-2.0", "OFL-1.1", "OpenSSL", "Ubuntu-font-1.0", "Unicode-DFS-2016", "Unicode-3.0", "Unlicense", "Zlib", "zlib-acknowledgement", ] targets = [ "x86_64-pc-windows-msvc", "x86_64-unknown-linux-gnu", "aarch64-linux-android", "wasm32-unknown-unknown", ] workarounds = ["ring"] filter-noassertion = true ================================================ FILE: alvr/adb/Cargo.toml ================================================ [package] name = "alvr_adb" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_filesystem.workspace = true alvr_system_info.workspace = true alvr_session.workspace = true anyhow = "1" ureq = "3" zip = "4" ================================================ FILE: alvr/adb/src/commands.rs ================================================ // https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/docs/user/adb.1.md use crate::parse::{self, Device, ForwardedPorts}; use alvr_filesystem as afs; use anyhow::{Context, Result, anyhow}; use std::{ collections::HashSet, io::{Cursor, Read}, process::Command, str::FromStr, time::Duration, }; use zip::ZipArchive; #[cfg(windows)] use std::os::windows::process::CommandExt; // https://developer.android.com/tools/releases/platform-tools#revisions // NOTE: At the time of writing this comment, the revisions section above // shows the latest version as 35.0.2, but the latest that can be downloaded // by specifying a version is 35.0.0 const PLATFORM_TOOLS_VERSION: &str = "-latest"; // E.g. "_r35.0.0" #[cfg(target_os = "linux")] const PLATFORM_TOOLS_OS: &str = "linux"; #[cfg(target_os = "macos")] const PLATFORM_TOOLS_OS: &str = "darwin"; #[cfg(windows)] const PLATFORM_TOOLS_OS: &str = "windows"; const REQUEST_TIMEOUT: Duration = Duration::from_secs(5); fn get_command(adb_path: &str, args: &[&str]) -> Command { let mut command = Command::new(adb_path); command.args(args); #[cfg(windows)] command.creation_flags(0x08000000); // CREATE_NO_WINDOW command } pub fn download(url: &str, progress_callback: impl Fn(usize, Option)) -> Result> { let agent: ureq::Agent = ureq::Agent::config_builder() .timeout_global(Some(REQUEST_TIMEOUT)) .build() .into(); let response = agent.get(url).call()?; let maybe_expected_size = response .headers() .get("Content-Length") .and_then(|v| v.to_str().ok()?.parse::().ok()); let mut result = maybe_expected_size .map(Vec::with_capacity) .unwrap_or_default(); let mut reader = response.into_body().into_reader(); let mut buffer = vec![0; 65535]; loop { let read_count: usize = reader.read(&mut buffer)?; if read_count == 0 { break; } result.extend_from_slice(&buffer[..read_count]); let current_size = result.len(); (progress_callback)(current_size, maybe_expected_size); } Ok(result) } /////////// // Activity pub fn get_process_id( adb_path: &str, device_serial: &str, process_name: &str, ) -> Result> { let output = get_command( adb_path, &["-s", device_serial, "shell", "pidof", process_name], ) .output() .context(format!("Failed to get ID of process {process_name}"))?; let text = String::from_utf8_lossy(&output.stdout).trim().to_owned(); if text.is_empty() { return Ok(None); } let process_id = text .parse::() .context("Failed to parse process ID")?; Ok(Some(process_id)) } pub fn is_activity_resumed( adb_path: &str, device_serial: &str, activity_name: &str, ) -> Result { let output = get_command( adb_path, &[ "-s", device_serial, "shell", "dumpsys", "activity", activity_name, ], ) .output() .context(format!("Failed to get state of activity {activity_name}"))?; let text = String::from_utf8_lossy(&output.stdout); if let Some(line) = text .lines() .map(|l| l.trim()) .find(|l| l.contains("mResumed")) { let (entry, _) = line .split_once(' ') .ok_or(anyhow!("Failed to split resumed state line"))?; let (_, value) = entry .split_once('=') .ok_or(anyhow!("Failed to split resumed state entry"))?; match value { "true" => Ok(true), "false" => Ok(false), _ => Err(anyhow!("Failed to parse resumed state value"))?, } } else { Err(anyhow!("Failed to find resumed state line")) } } /////////////////// // ADB Installation pub fn require_adb( layout: &afs::Layout, progress_callback: impl Fn(usize, Option), ) -> Result { if let Some(path) = get_adb_path(layout) { Ok(path) } else { install_adb(layout, progress_callback).context("Failed to install ADB")?; Ok(get_adb_path(layout).context("Failed to get ADB path after installation")?) } } fn install_adb( layout: &afs::Layout, progress_callback: impl Fn(usize, Option), ) -> Result<()> { let mut reader = Cursor::new(download_adb(progress_callback)?); ZipArchive::new(&mut reader)?.extract(layout.executables_dir.clone())?; Ok(()) } fn download_adb(progress_callback: impl Fn(usize, Option)) -> Result> { let url = get_platform_tools_url(); download(&url, progress_callback).context(format!("Failed to download ADB from {url}")) } fn get_platform_tools_url() -> String { format!( "https://dl.google.com/android/repository/platform-tools{PLATFORM_TOOLS_VERSION}-{PLATFORM_TOOLS_OS}.zip" ) } /////////////// // Applications pub fn start_application(adb_path: &str, device_serial: &str, application_id: &str) -> Result<()> { get_command( adb_path, &[ "-s", device_serial, "shell", "monkey", "-p", application_id, "1", ], ) .output() .context(format!("Failed to start {application_id}"))?; Ok(()) } ////////// // Devices pub fn list_devices(adb_path: &str) -> Result> { let output = get_command(adb_path, &["devices", "-l"]) .output() .context("Failed to list ADB devices")?; let text = String::from_utf8_lossy(&output.stdout); let devices = text .lines() .skip(1) .filter_map(parse::parse_device) .collect(); Ok(devices) } /////////// // Packages pub fn install_package(adb_path: &str, device_serial: &str, apk_path: &str) -> Result<()> { get_command(adb_path, &["-s", device_serial, "install", "-r", apk_path]) .output() .context(format!("Failed to install {apk_path}"))?; Ok(()) } pub fn is_package_installed( adb_path: &str, device_serial: &str, application_id: &str, ) -> Result { let found = list_installed_packages(adb_path, device_serial) .context(format!( "Failed to check if package {application_id} is installed" ))? .contains(application_id); Ok(found) } pub fn uninstall_package(adb_path: &str, device_serial: &str, application_id: &str) -> Result<()> { get_command( adb_path, &["-s", device_serial, "uninstall", application_id], ) .output() .context(format!("Failed to uninstall {application_id}"))?; Ok(()) } pub fn list_installed_packages(adb_path: &str, device_serial: &str) -> Result> { let output = get_command( adb_path, &["-s", device_serial, "shell", "pm", "list", "package"], ) .output() .context("Failed to list installed packages")?; let text = String::from_utf8_lossy(&output.stdout); let packages = text.lines().map(|l| l.replace("package:", "")).collect(); Ok(packages) } //////// // Paths /// Returns the path of a local (i.e. installed by ALVR) or OS version of `adb` if found, `None` otherwise. pub fn get_adb_path(layout: &afs::Layout) -> Option { let exe_name = afs::exec_fname("adb").to_owned(); let adb_path = get_command(&exe_name, &[]) .output() .is_ok() .then_some(exe_name); adb_path.or_else(|| { let path = layout.local_adb_exe(); path.try_exists() .unwrap_or(false) .then(|| path.to_string_lossy().to_string()) }) } //////// // Utility pub fn get_uptime(adb_path: &str, device_serial: &str) -> Result { let output = get_command( adb_path, &["-s", device_serial, "shell", "cat", "/proc/uptime"], ) .output() .context("Failed to get system uptime")?; let output_str = String::from_utf8_lossy(&output.stdout); let uptime_string = output_str .split_ascii_whitespace() .next() .context("Empty result from /proc/uptime")?; let uptime = f64::from_str(uptime_string).context("Cannot parse uptime into an f64")?; Duration::try_from_secs_f64(uptime).context("Invalid f64 value for a duration ") } ////////////////// // Port forwarding pub fn list_forwarded_ports(adb_path: &str, device_serial: &str) -> Result> { let output = get_command(adb_path, &["-s", device_serial, "forward", "--list"]) .output() .context(format!( "Failed to list forwarded ports of device {device_serial:?}" ))?; let text = String::from_utf8_lossy(&output.stdout); let forwarded_ports = text .lines() .filter_map(parse::parse_forwarded_ports) .collect(); Ok(forwarded_ports) } pub fn forward_port(adb_path: &str, device_serial: &str, port: u16) -> Result<()> { get_command( adb_path, &[ "-s", device_serial, "forward", &format!("tcp:{port}"), &format!("tcp:{port}"), ], ) .output() .context(format!( "Failed to forward port {port:?} of device {device_serial:?}" ))?; Ok(()) } ///////// // Server pub fn kill_server(adb_path: &str) -> Result<()> { get_command(adb_path, &["kill-server"]) .output() .context("Failed to kill ADB server")?; Ok(()) } ================================================ FILE: alvr/adb/src/lib.rs ================================================ pub mod commands; mod parse; use alvr_common::anyhow::Result; use alvr_common::{dbg_connection, error, warn}; use alvr_session::WiredClientAutoLaunchConfig; use alvr_system_info::{ ClientFlavor, PACKAGE_NAME_GITHUB_DEV, PACKAGE_NAME_GITHUB_STABLE, PACKAGE_NAME_STORE, }; use std::collections::HashSet; use std::time::Duration; pub enum WiredConnectionStatus { Ready, NotReady(String), } pub struct WiredConnection { adb_path: String, } impl WiredConnection { pub fn new( layout: &alvr_filesystem::Layout, download_progress_callback: impl Fn(usize, Option), ) -> Result { let adb_path = commands::require_adb(layout, download_progress_callback)?; Ok(Self { adb_path }) } pub fn setup( &self, control_port: u16, stream_port: u16, client_type: &ClientFlavor, client_autolaunch: Option, ) -> Result { let Some(device_serial) = commands::list_devices(&self.adb_path)? .into_iter() .filter_map(|d| d.serial) .find(|s| !s.starts_with("127.0.0.1")) else { return Ok(WiredConnectionStatus::NotReady( "No wired devices found".to_owned(), )); }; let ports = HashSet::from([control_port, stream_port]); let forwarded_ports: HashSet = commands::list_forwarded_ports(&self.adb_path, &device_serial)? .into_iter() .map(|f| f.local) .collect(); let missing_ports = ports.difference(&forwarded_ports); for port in missing_ports { commands::forward_port(&self.adb_path, &device_serial, *port)?; dbg_connection!( "setup_wired_connection: Forwarded port {port} of device {device_serial}" ); } let Some(process_name) = get_process_name(&self.adb_path, &device_serial, client_type) else { return Ok(WiredConnectionStatus::NotReady( "No suitable ALVR client is installed".to_owned(), )); }; if commands::get_process_id(&self.adb_path, &device_serial, &process_name)?.is_none() { if let Some(client_autolaunch) = client_autolaunch { if client_autolaunch.boot_delay > 0 { match commands::get_uptime(&self.adb_path, &device_serial) { Ok(uptime) => { if uptime < Duration::from_secs(client_autolaunch.boot_delay.into()) { return Ok(WiredConnectionStatus::NotReady( "Waiting for device boot".to_owned(), )); } } Err(failure) => { warn!("wired_connection: get_uptime failed with {}", failure); } } } commands::start_application(&self.adb_path, &device_serial, &process_name)?; Ok(WiredConnectionStatus::NotReady( "Starting ALVR client".to_owned(), )) } else { Ok(WiredConnectionStatus::NotReady( "ALVR client is not running".to_owned(), )) } } else if !commands::is_activity_resumed(&self.adb_path, &device_serial, &process_name)? { Ok(WiredConnectionStatus::NotReady( "ALVR client is paused".to_owned(), )) } else { Ok(WiredConnectionStatus::Ready) } } } impl Drop for WiredConnection { fn drop(&mut self) { dbg_connection!("wired_connection: Killing ADB server"); if let Err(e) = commands::kill_server(&self.adb_path) { error!("{e:?}"); } } } pub fn get_process_name( adb_path: &str, device_serial: &str, flavor: &ClientFlavor, ) -> Option { let fallbacks = match flavor { ClientFlavor::Store => { if alvr_common::is_stable() { vec![PACKAGE_NAME_STORE, PACKAGE_NAME_GITHUB_STABLE] } else { vec![PACKAGE_NAME_GITHUB_DEV] } } ClientFlavor::Github => { if alvr_common::is_stable() { vec![PACKAGE_NAME_GITHUB_STABLE, PACKAGE_NAME_STORE] } else { vec![PACKAGE_NAME_GITHUB_DEV] } } ClientFlavor::Custom(name) => { if alvr_common::is_stable() { vec![name, PACKAGE_NAME_STORE, PACKAGE_NAME_GITHUB_STABLE] } else { vec![name, PACKAGE_NAME_GITHUB_DEV] } } }; fallbacks .iter() .find(|name| { commands::is_package_installed(adb_path, device_serial, name) .is_ok_and(|installed| installed) }) .map(|name| (*name).to_string()) } ================================================ FILE: alvr/adb/src/parse.rs ================================================ // https://cs.android.com/android/platform/superproject/main/+/7dbe542b9a93fb3cee6c528e16e2d02a26da7cc0:packages/modules/adb/transport.cpp;l=1409 // The serial number is printed with a "%-22s" format, meaning that it's a left-aligned space-padded string of 22 characters. const SERIAL_NUMBER_COLUMN_LENGTH: usize = 22; // https://cs.android.com/android/platform/superproject/main/+/7dbe542b9a93fb3cee6c528e16e2d02a26da7cc0:packages/modules/adb/adb.h;l=104-122 #[derive(Debug)] pub enum ConnectionState { Authorizing, Bootloader, Connecting, Detached, Device, Host, NoPermissions, // https://cs.android.com/android/platform/superproject/main/+/main:system/core/diagnose_usb/diagnose_usb.cpp;l=83-90 Offline, Recovery, Rescue, Sideload, Unauthorized, } pub fn parse_connection_state(value: &str) -> Option { match value { "authorizing" => Some(ConnectionState::Authorizing), "bootloader" => Some(ConnectionState::Bootloader), "connecting" => Some(ConnectionState::Connecting), "detached" => Some(ConnectionState::Detached), "device" => Some(ConnectionState::Device), "host" => Some(ConnectionState::Host), "offline" => Some(ConnectionState::Offline), "recovery" => Some(ConnectionState::Recovery), "rescue" => Some(ConnectionState::Rescue), "sideload" => Some(ConnectionState::Sideload), "unauthorized" => Some(ConnectionState::Unauthorized), _ => None, } } fn parse_pair(pair: &str) -> Option { let mut slice = pair.split(':'); let _key = slice.next(); slice.next().map(|value| value.to_string()) } // https://cs.android.com/android/platform/superproject/main/+/7dbe542b9a93fb3cee6c528e16e2d02a26da7cc0:packages/modules/adb/adb.h;l=95-100 #[derive(Debug)] pub enum TransportType { Usb, Local, Any, Host, } pub fn parse_transport_type(pair: &str) -> Option { let mut slice = pair.split(':'); let _key = slice.next(); if let Ok(value) = slice.next()?.parse::() { match value { 0 => Some(TransportType::Usb), 1 => Some(TransportType::Local), 2 => Some(TransportType::Any), 3 => Some(TransportType::Host), _ => None, } } else { None } } // https://cs.android.com/android/platform/superproject/main/+/7dbe542b9a93fb3cee6c528e16e2d02a26da7cc0:packages/modules/adb/transport.cpp;l=1398 #[derive(Debug)] pub struct Device { pub connection_state: Option, pub device: Option, pub model: Option, pub product: Option, pub serial: Option, pub transport_type: Option, } pub fn parse_device(line: &str) -> Option { if line.len() < SERIAL_NUMBER_COLUMN_LENGTH { return None; } let (left, right) = line.split_at(SERIAL_NUMBER_COLUMN_LENGTH); let serial = (!left.contains("(no serial number)")).then(|| left.trim().to_owned()); let mut remaining = right.trim(); let connection_state = if remaining.starts_with("no permissions") { // Since the current user's name can be printed in the error message, // we are gambling that there's not a "]" in it. if let Some((_, right)) = remaining.split_once(']') { remaining = right; Some(ConnectionState::NoPermissions) } else { None } } else if let Some((left, right)) = remaining.split_once(' ') { remaining = right; parse_connection_state(left) } else { None }; let mut slices = remaining.split_whitespace(); let product = slices.next().and_then(parse_pair); let model = slices.next().and_then(parse_pair); let device = slices.next().and_then(parse_pair); let transport_type = slices.next().and_then(parse_transport_type); Some(Device { connection_state, device, model, product, serial, transport_type, }) } #[derive(Debug)] pub struct ForwardedPorts { pub local: u16, pub remote: u16, pub serial: String, } pub fn parse_forwarded_ports(line: &str) -> Option { let mut slices = line.split_whitespace(); let serial = slices.next(); let local = parse_port(slices.next()?); let remote = parse_port(slices.next()?); if let Some(serial) = serial && let Some(local) = local && let Some(remote) = remote { Some(ForwardedPorts { local, remote, serial: serial.to_owned(), }) } else { None } } fn parse_port(value: &str) -> Option { let mut slices = value.split(':'); let _protocol = slices.next(); let maybe_port = slices.next(); maybe_port.and_then(|p| p.parse::().ok()) } ================================================ FILE: alvr/audio/Cargo.toml ================================================ [package] name = "alvr_audio" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_session.workspace = true alvr_sockets.workspace = true cpal = "0.16" rodio = "0.21" serde = "1" [target.'cfg(windows)'.dependencies] widestring = "1" windows = { version = "0.61", features = [ "Win32_Devices_FunctionDiscovery", "Win32_Foundation", "Win32_Media_Audio_Endpoints", "Win32_System_Com_StructuredStorage", "Win32_System_Variant", "Win32_UI_Shell_PropertiesSystem", ] } [target.'cfg(target_os = "linux")'.dependencies] pipewire = { version = "0.8.0", features = ["v0_3_49"] } libspa-sys = "0.8.0" ================================================ FILE: alvr/audio/src/lib.rs ================================================ #[cfg(windows)] mod windows; #[cfg(target_os = "linux")] pub mod linux; #[cfg(windows)] pub use crate::windows::*; use alvr_common::{ ConnectionError, ToAny, anyhow::{self, Context, Result, bail}, info, parking_lot::Mutex, }; use alvr_session::{AudioBufferingConfig, CustomAudioDeviceConfig, MicrophoneDevicesConfig}; use alvr_sockets::{StreamReceiver, StreamSender}; use cpal::{ BufferSize, Host, Sample, SampleFormat, StreamConfig, traits::{DeviceTrait, HostTrait, StreamTrait}, }; use rodio::{OutputStreamBuilder, Source}; use std::{collections::VecDeque, sync::Arc, thread, time::Duration}; pub use cpal::Device; fn device_from_custom_config( host: &Host, config: &CustomAudioDeviceConfig, is_output: bool, ) -> Result { let mut devices = if is_output { host.output_devices()? } else { host.input_devices()? }; Ok(match config { CustomAudioDeviceConfig::NameSubstring(name_substring) => devices .find(|d| { d.name() .map(|name| name.to_lowercase().contains(&name_substring.to_lowercase())) .unwrap_or(false) }) .with_context(|| { format!("Cannot find audio device which name contains \"{name_substring}\"") })?, CustomAudioDeviceConfig::Index(index) => devices .nth(*index) .with_context(|| format!("Cannot find audio device at index {index}"))?, }) } pub fn new_output(config: Option<&CustomAudioDeviceConfig>) -> Result { let host = cpal::default_host(); let device = match config { None => host .default_output_device() .context("No output audio device found")?, Some(config) => device_from_custom_config(&host, config, true)?, }; Ok(device) } pub fn new_input(config: Option) -> Result { let host = cpal::default_host(); let device = match config { None => host .default_input_device() .context("No input audio device found")?, Some(config) => device_from_custom_config(&host, &config, false)?, }; Ok(device) } // returns (sink, source) pub fn new_virtual_microphone_pair(config: MicrophoneDevicesConfig) -> Result<(Device, Device)> { // No-op on Windows (this is windows specific code) let host = cpal::default_host(); let (sink_name, source_name) = match config { MicrophoneDevicesConfig::Automatic => { // NOTE: This will iterate over all devices for every option it tries, if the audio // code is slow, change that first return [ MicrophoneDevicesConfig::VAC, MicrophoneDevicesConfig::VBCable, MicrophoneDevicesConfig::VoiceMeeter, MicrophoneDevicesConfig::VoiceMeeterAux, MicrophoneDevicesConfig::VoiceMeeterVaio3, ] .into_iter() .find_map(|cable_type| new_virtual_microphone_pair(cable_type).ok()) .context("No microphones found"); } MicrophoneDevicesConfig::VAC => ("Line 1", "Line 1"), MicrophoneDevicesConfig::VBCable => ("CABLE Input", "CABLE Output"), MicrophoneDevicesConfig::VoiceMeeter => ("VoiceMeeter Input", "VoiceMeeter Output"), MicrophoneDevicesConfig::VoiceMeeterAux => { ("VoiceMeeter Aux Input", "VoiceMeeter Aux Output") } MicrophoneDevicesConfig::VoiceMeeterVaio3 => { ("VoiceMeeter VAIO3 Input", "VoiceMeeter VAIO3 Output") } MicrophoneDevicesConfig::Custom { sink, source } => { return Ok(( device_from_custom_config(&host, &sink, true)?, device_from_custom_config(&host, &source, false)?, )); } }; let sink = host .output_devices()? .find(|d| d.name().unwrap_or_default().contains(sink_name)) .context("Virtual Audio Cable, VB-CABLE or VoiceMeeter not found. Please install or reinstall one")?; let source = host .input_devices()? .find(|d| d.name().unwrap_or_default().contains(source_name)) .context("Matching output microphone not found. Did you rename it?")?; Ok((sink, source)) } pub fn input_sample_rate(dev: &Device) -> Result { let config = dev .default_input_config() // On Windows, loopback devices are not recognized as input devices. Use output config. .or_else(|_| dev.default_output_config())?; Ok(config.sample_rate().0) } pub enum AudioRecordState { Recording, ShouldStop, Err(Option), } pub enum AudioChannel { FrontLeft, FrontRight, Center, SurroundLeft, SurroundRight, BackLeft, BackRight, Top, HighFrontLeft, HighFrontRight, HighFrontCenter, HighBackLeft, HighBackRight, LowFrequency, } fn downmix_channels(channels: &[AudioChannel], data: &[u8], out_channels: u16) -> Vec { let mut left = 0.0; let mut right = 0.0; for i in 0..channels.len() { let [l, r] = match &channels[i] { AudioChannel::FrontLeft => [1.0, 0.0], AudioChannel::FrontRight => [0.0, 1.0], AudioChannel::Center => [0.707, 0.707], AudioChannel::SurroundLeft => [0.707, 0.0], AudioChannel::SurroundRight => [0.0, 0.707], AudioChannel::BackLeft => [0.707, 0.0], AudioChannel::BackRight => [0.0, 0.707], AudioChannel::Top => [0.577, 0.577], AudioChannel::HighFrontLeft => [0.707, 0.0], AudioChannel::HighFrontRight => [0.0, 0.707], AudioChannel::HighFrontCenter => [0.5, 0.5], AudioChannel::HighBackLeft => [0.5, 0.0], AudioChannel::HighBackRight => [0.0, 0.5], _ => [0.0, 0.0], }; let val = i16::from_ne_bytes([data[i * 2], data[i * 2 + 1]]).to_sample::(); left += val * l; right += val * r; } if out_channels == 1 { let bytes = ((left + right) / 2.0).to_sample::().to_ne_bytes(); vec![bytes[0], bytes[1]] } else { let left_bytes = left.to_sample::().to_ne_bytes(); let right_bytes = right.to_sample::().to_ne_bytes(); vec![left_bytes[0], left_bytes[1], right_bytes[0], right_bytes[1]] } } fn downmix_audio(data: Vec, in_channels: u16, out_channels: u16) -> Vec { if in_channels == out_channels { data } else if in_channels == 1 && out_channels == 2 { data.chunks_exact(2) .flat_map(|c| vec![c[0], c[1], c[0], c[1]]) .collect() } else { let channels = match in_channels { 2 => vec![AudioChannel::FrontLeft, AudioChannel::FrontRight], 3 => vec![ AudioChannel::FrontLeft, AudioChannel::FrontRight, AudioChannel::LowFrequency, ], 4 => vec![ AudioChannel::FrontLeft, AudioChannel::FrontRight, AudioChannel::BackLeft, AudioChannel::BackRight, ], 6 => vec![ AudioChannel::FrontRight, AudioChannel::Center, AudioChannel::LowFrequency, AudioChannel::SurroundLeft, // Sometimes actually BackLeft, has same level so it's okay AudioChannel::SurroundRight, // Sometimes actually BackRight, has same level so it's okay ], 8 => vec![ AudioChannel::FrontLeft, AudioChannel::FrontRight, AudioChannel::Center, AudioChannel::LowFrequency, AudioChannel::BackLeft, AudioChannel::BackRight, AudioChannel::SurroundLeft, AudioChannel::SurroundRight, ], _ => unreachable!("Invalid input channel count"), }; data.chunks_exact(in_channels as usize * 2) .flat_map(|c| downmix_channels(&channels, c, out_channels)) .collect() } } #[allow(unused_variables)] pub fn record_audio_blocking( is_running: Arc bool + Send + Sync>, mut sender: StreamSender<()>, device: &Device, channels_count: u16, mute: bool, ) -> Result<()> { let config = device .default_input_config() // On Windows, loopback devices are not recognized as input devices. Use output config. .or_else(|_| device.default_output_config())?; if config.channels() > 8 { bail!( "Audio devices with more than 8 channels are not supported. {}", "Please turn off surround audio." ); } else if config.channels() == 5 || config.channels() == 7 { bail!( "Audio devices with {} channels are not supported.", config.channels() ); } let stream_config = StreamConfig { channels: config.channels(), sample_rate: config.sample_rate(), buffer_size: BufferSize::Default, }; let state = Arc::new(Mutex::new(AudioRecordState::Recording)); let stream = device.build_input_stream_raw( &stream_config, config.sample_format(), { let state = Arc::clone(&state); let is_running = Arc::clone(&is_running); move |data, _| { let data = if config.sample_format() == SampleFormat::F32 { data.bytes() .chunks_exact(4) .flat_map(|b| { f32::from_ne_bytes([b[0], b[1], b[2], b[3]]) .to_sample::() .to_ne_bytes() .to_vec() }) .collect() } else { data.bytes().to_vec() }; let data = downmix_audio(data, config.channels(), channels_count); if is_running() { sender.send_header_with_payload(&(), &data).ok(); } else { *state.lock() = AudioRecordState::ShouldStop; } } }, { let state = Arc::clone(&state); move |e| *state.lock() = AudioRecordState::Err(Some(e.into())) }, None, )?; #[cfg(windows)] if mute && device.supports_output() { crate::windows::set_mute_windows_device(device, true).ok(); } let mut res = stream.play().to_any(); if res.is_ok() { while matches!(*state.lock(), AudioRecordState::Recording) && is_running() { thread::sleep(Duration::from_millis(500)) } if let AudioRecordState::Err(e) = &mut *state.lock() { res = Err(e.take().unwrap()); } } #[cfg(windows)] if mute && device.supports_output() { set_mute_windows_device(device, false).ok(); } res } // Audio callback. This is designed to be as less complex as possible. Still, when needed, this // callback can render a fade-out autonomously. #[inline] pub fn get_next_frame_batch( sample_buffer: &mut VecDeque, channels_count: usize, batch_frames_count: usize, ) -> Vec { if sample_buffer.len() / channels_count >= batch_frames_count { let mut batch = sample_buffer .drain(0..batch_frames_count * channels_count) .collect::>(); if sample_buffer.len() / channels_count < batch_frames_count { // Render fade-out. It is completely contained in the current batch for f in 0..batch_frames_count { let volume = 1. - f as f32 / batch_frames_count as f32; for c in 0..channels_count { batch[f * channels_count + c] *= volume; } } } // fade-ins and cross-fades are rendered in the receive loop directly inside sample_buffer. batch } else { vec![0.; batch_frames_count * channels_count] } } // The receive loop is resposible for ensuring smooth transitions in case of disruptions (buffer // underflow, overflow, packet loss). In case the computation takes too much time, the audio // callback will gracefully handle an interruption, and the callback timing and sound wave // continuity will not be affected. pub fn receive_samples_loop( is_running: impl Fn() -> bool, receiver: &mut StreamReceiver<()>, sample_buffer: Arc>>, channels_count: usize, batch_frames_count: usize, average_buffer_frames_count: usize, ) -> Result<()> { let mut recovery_sample_buffer = vec![]; while is_running() { let data = match receiver.recv(Duration::from_millis(500)) { Ok(data) => data, Err(ConnectionError::TryAgain(_)) => continue, Err(ConnectionError::Other(e)) => return Err(e), }; let (_, packet) = data.get()?; let new_samples = packet .chunks_exact(2) .map(|c| i16::from_ne_bytes([c[0], c[1]]).to_sample::()) .collect::>(); let mut sample_buffer_ref = sample_buffer.lock(); if data.had_packet_loss() { info!("Audio packet loss!"); if sample_buffer_ref.len() / channels_count < batch_frames_count { sample_buffer_ref.clear(); } else { // clear remaining samples sample_buffer_ref.drain(batch_frames_count * channels_count..); } recovery_sample_buffer.clear(); } if sample_buffer_ref.len() / channels_count < batch_frames_count { recovery_sample_buffer.extend(sample_buffer_ref.drain(..)); } if sample_buffer_ref.is_empty() || data.had_packet_loss() { recovery_sample_buffer.extend(&new_samples); if recovery_sample_buffer.len() / channels_count > average_buffer_frames_count + batch_frames_count { // Fade-in for f in 0..batch_frames_count { let volume = f as f32 / batch_frames_count as f32; for c in 0..channels_count { recovery_sample_buffer[f * channels_count + c] *= volume; } } if data.had_packet_loss() && sample_buffer_ref.len() / channels_count == batch_frames_count { // Add a fade-out to make a cross-fade. for f in 0..batch_frames_count { let volume = 1. - f as f32 / batch_frames_count as f32; for c in 0..channels_count { recovery_sample_buffer[f * channels_count + c] += sample_buffer_ref[f * channels_count + c] * volume; } } sample_buffer_ref.clear(); } sample_buffer_ref.extend(recovery_sample_buffer.drain(..)); info!("Audio recovered"); } } else { sample_buffer_ref.extend(&new_samples); } // todo: use smarter policy with EventTiming let buffer_frames_size = sample_buffer_ref.len() / channels_count; if buffer_frames_size > 2 * average_buffer_frames_count + batch_frames_count { info!("Audio buffer overflow! size: {buffer_frames_size}"); let drained_samples = sample_buffer_ref .drain(0..(buffer_frames_size - average_buffer_frames_count) * channels_count) .collect::>(); // Render a cross-fade. for f in 0..batch_frames_count { let volume = f as f32 / batch_frames_count as f32; for c in 0..channels_count { let index = f * channels_count + c; sample_buffer_ref[index] = sample_buffer_ref[index] .mul_add(volume, drained_samples[index] * (1. - volume)); } } } } Ok(()) } struct StreamingSource { sample_buffer: Arc>>, current_batch: Vec, current_batch_cursor: usize, channels_count: usize, sample_rate: u32, batch_frames_count: usize, } impl Source for StreamingSource { fn current_span_len(&self) -> Option { None } fn channels(&self) -> u16 { self.channels_count as _ } fn sample_rate(&self) -> u32 { self.sample_rate } fn total_duration(&self) -> Option { None } } impl Iterator for StreamingSource { type Item = f32; #[inline] fn next(&mut self) -> Option { if self.current_batch_cursor == 0 { self.current_batch = get_next_frame_batch( &mut self.sample_buffer.lock(), self.channels_count, self.batch_frames_count, ); } let sample = self.current_batch[self.current_batch_cursor]; self.current_batch_cursor = (self.current_batch_cursor + 1) % (self.batch_frames_count * self.channels_count); Some(sample) } } pub fn play_audio_loop( is_running: impl Fn() -> bool, device: &Device, channels_count: u16, sample_rate: u32, config: AudioBufferingConfig, receiver: &mut StreamReceiver<()>, ) -> Result<()> { // Size of a chunk of frames. It corresponds to the duration if a fade-in/out in frames. let batch_frames_count = sample_rate as usize * config.batch_ms as usize / 1000; // Average buffer size in frames let average_buffer_frames_count = sample_rate as usize * config.average_buffering_ms as usize / 1000; let sample_buffer = Arc::new(Mutex::new(VecDeque::new())); let stream = OutputStreamBuilder::from_device(device.clone())?.open_stream()?; stream.mixer().add(StreamingSource { sample_buffer: Arc::clone(&sample_buffer), current_batch: vec![], current_batch_cursor: 0, channels_count: channels_count as _, sample_rate, batch_frames_count, }); receive_samples_loop( is_running, receiver, sample_buffer, channels_count as _, batch_frames_count, average_buffer_frames_count, ) .ok(); Ok(()) } ================================================ FILE: alvr/audio/src/linux.rs ================================================ use alvr_common::{ConnectionError, anyhow::Result, debug, error, parking_lot::Mutex}; use alvr_session::AudioBufferingConfig; use alvr_sockets::{StreamReceiver, StreamSender}; use std::os::unix::fs::FileTypeExt; use std::{ collections::VecDeque, fs, io, path::Path, sync::{ Arc, atomic::{AtomicBool, Ordering}, }, thread::{self, sleep}, time::Duration, }; use pipewire::{ channel::Receiver, context::Context, core::Core, keys, main_loop::MainLoop, properties, spa::{ param::audio::{AudioFormat, AudioInfoRaw}, pod::{self, Pod, Value, serialize::PodSerializer}, utils::Direction, }, stream::{Stream, StreamFlags, StreamListener, StreamState}, }; pub fn try_load_pipewire() -> Result<()> { if let Err(e) = probe_pipewire() { if !matches!(e, pipewire::Error::CreationFailed) { return Err(e.into()); } error!("Could not initialize PipeWire."); let is_under_flatpak = std::env::var("FLATPAK_ID").is_ok(); let is_pw_socket_available = std::env::var("XDG_RUNTIME_DIR").is_ok_and(|xdg_runtime_dir| { let pw_socket_path = Path::new(&xdg_runtime_dir).join("pipewire-0"); fs::metadata(&pw_socket_path).is_ok_and(|m| m.file_type().is_socket()) }); if is_under_flatpak && !is_pw_socket_available { error!( "Please visit the following page to find help on how to fix broken audio on flatpak." ); error!( "https://github.com/alvr-org/ALVR/wiki/Installing-ALVR-and-using-SteamVR-on-Linux-through-Flatpak#failed-to-create-pipewire-errors" ); } error!("Make sure PipeWire is installed on your system, running and it's version is at least 0.3.49. To retry, please restart SteamVR with ALVR."); } Ok(()) } fn probe_pipewire() -> Result<(), pipewire::Error> { let mainloop = MainLoop::new(None)?; let context = Context::new(&mainloop)?; context.connect(None)?; Ok(()) } #[derive(Clone, Copy)] pub struct AudioInfo { pub sample_rate: u32, pub channel_count: u32, } struct Terminate; // fixme: Opening pavucontrol while audio is actively streaming // will cause audio cut out for short time, // possibly related to fast state changes caused by pavucontrol static MIC_STREAMING: AtomicBool = AtomicBool::new(false); pub fn audio_loop( is_running: impl Fn() -> bool, sender: StreamSender<()>, speaker_info: Option, receiver: &mut StreamReceiver<()>, mic_info: Option<(AudioInfo, AudioBufferingConfig)>, ) { let sample_queue = Arc::new(Mutex::new(VecDeque::new())); MIC_STREAMING.store(false, Ordering::Relaxed); let (pw_sender, pw_receiver) = pipewire::channel::channel(); // Stall pipewire startup until we're actually streaming to not cause latency by packet buildup if !is_running() { return; } let pw_thread = thread::spawn({ let sample_queue = sample_queue.clone(); let mic_info = mic_info.as_ref().map(|(info, _)| *info); move || { if let Err(e) = pw_main_loop(pw_receiver, sender, speaker_info, sample_queue, mic_info) { error!("Unhandled pipewire audio device error, please report it on GitHub: {e}"); } debug!("Pipewire audio loop exiting"); } }); while is_running() { if let Some((mic_info, buffering)) = &mic_info { let rate = mic_info.sample_rate as usize; let batch_frames_count = rate * buffering.batch_ms as usize / 1000; let average_buffer_frames_count = rate * buffering.average_buffering_ms as usize / 1000; if let Err(e) = crate::receive_samples_loop( || is_running() && MIC_STREAMING.load(Ordering::Relaxed), receiver, sample_queue.clone(), mic_info.channel_count as usize, batch_frames_count, average_buffer_frames_count, ) { error!("Receive samples loop encountered error {e:?}"); } // if we end up here then no consumer is currently connected to the output // so discard audio packets to not cause a buildup if matches!( receiver.recv(Duration::from_millis(500)), Err(ConnectionError::Other(_)) ) { break; } } else { sleep(Duration::from_millis(500)); } } if pw_sender.send(Terminate).is_err() { error!( "Couldn't send pipewire termination signal, deinitializing forcefully. Restart the VR app to reinitialize the audio device." ); unsafe { pipewire::deinit() }; } pw_thread.join().ok(); } fn pw_main_loop( pw_receiver: Receiver, audio_sender: StreamSender<()>, speaker_info: Option, sample_queue: Arc>>, mic_info: Option, ) -> Result<(), pipewire::Error> { debug!("Starting pipewire thread"); let mainloop = MainLoop::new(None)?; let _receiver = pw_receiver.attach(mainloop.as_ref(), { let mainloop = mainloop.clone(); move |_| mainloop.quit() }); let context = Context::new(&mainloop)?; let pw_core = context.connect(None)?; let _speaker = if let Some(info) = speaker_info { debug!("Creating pw output audio stream"); Some(create_speaker_stream( &pw_core, audio_sender, info.sample_rate, info.channel_count, )?) } else { None }; let _mic = if let Some(info) = mic_info { debug!("Creating pw microphone stream"); Some(create_mic_stream( &pw_core, sample_queue, info.sample_rate, info.channel_count, )?) } else { None }; debug!("Running pipewire thread"); mainloop.run(); Ok(()) } fn audio_info_to_vec(audio_info: AudioInfoRaw) -> Vec { PodSerializer::serialize( io::Cursor::new(Vec::new()), &Value::Object(pod::Object { type_: libspa_sys::SPA_TYPE_OBJECT_Format, id: libspa_sys::SPA_PARAM_EnumFormat, properties: audio_info.into(), }), ) .unwrap() .0 .into_inner() } fn create_speaker_stream( pw_core: &Core, mut sender: StreamSender<()>, sample_rate: u32, channel_count: u32, ) -> Result<(Stream, StreamListener), pipewire::Error> { let stream = Stream::new( pw_core, "alvr-audio", properties::properties! { *keys::NODE_NAME => "ALVR Audio", *keys::MEDIA_NAME => "alvr-audio", *keys::MEDIA_TYPE => "Audio", *keys::MEDIA_CATEGORY => "Capture", *keys::MEDIA_CLASS => "Audio/Sink", *keys::MEDIA_ROLE => "Game", }, )?; let listener: StreamListener = stream .add_local_listener() .process(move |stream, _| { if let Some(mut pw_buf) = stream.dequeue_buffer() && let Some(pw_buf) = pw_buf.datas_mut().first_mut() { let size = pw_buf.chunk_mut().size() as usize; if let Some(data) = pw_buf.data() { // Data is given as s16le in the correct layout by pipewire already, // no need to do conversions sender.send_header_with_payload(&(), &data[0..size]).ok(); } } }) .register()?; let mut audio_info = AudioInfoRaw::new(); audio_info.set_format(AudioFormat::S16LE); audio_info.set_rate(sample_rate); audio_info.set_channels(channel_count); stream.connect( Direction::Input, None, StreamFlags::AUTOCONNECT | StreamFlags::MAP_BUFFERS, &mut [Pod::from_bytes(&audio_info_to_vec(audio_info)).unwrap()], )?; Ok((stream, listener)) } fn create_mic_stream( pw_core: &Core, sample_queue: Arc>>, sample_rate: u32, channel_count: u32, ) -> Result<(Stream, StreamListener), pipewire::Error> { let stream = Stream::new( pw_core, "alvr-mic", properties::properties! { *keys::NODE_NAME => "ALVR Microphone", *keys::MEDIA_NAME => "alvr-mic", *keys::MEDIA_TYPE => "Audio", *keys::MEDIA_CATEGORY => "Playback", *keys::MEDIA_CLASS => "Audio/Source", *keys::MEDIA_ROLE => "Communication", }, )?; let chan_size = std::mem::size_of::(); let listener: StreamListener = stream .add_local_listener() .state_changed(move |_, _, _, new_state| { MIC_STREAMING.store(new_state == StreamState::Streaming, Ordering::Relaxed); }) .process(move |stream, _| { fill_pw_buf( sample_queue.clone(), chan_size, channel_count as usize, stream, ); }) .register()?; let mut audio_info = AudioInfoRaw::new(); audio_info.set_format(AudioFormat::F32LE); audio_info.set_rate(sample_rate); audio_info.set_channels(channel_count); stream.connect( Direction::Output, None, StreamFlags::AUTOCONNECT | StreamFlags::MAP_BUFFERS, &mut [Pod::from_bytes(&audio_info_to_vec(audio_info)).unwrap()], )?; Ok((stream, listener)) } fn fill_pw_buf( sample_queue: Arc>>, chan_size: usize, chan_count: usize, stream: &pipewire::stream::StreamRef, ) { if let Some(mut pw_buf) = stream.dequeue_buffer() { let requested = pw_buf.requested() as usize; if let Some(pw_data) = pw_buf.datas_mut().first_mut() && let Some(slice) = pw_data.data() && let Some(mut samples) = sample_queue.try_lock() { // TODO: Would it be more correct to try to split it up over multiple datas? // Or is the requested size already right for the first data chunk? let mut it = slice .chunks_exact_mut(chan_size) .take(requested * chan_count); let pw_sample_count = it.len(); let (front, back) = samples.as_slices(); let copy_sample = |(chunk, sample): (&mut [u8], &f32)| chunk.copy_from_slice(&sample.to_le_bytes()); // Split up so the compiler actually optimizes this properly it.by_ref().zip(front).for_each(copy_sample); it.zip(back).for_each(copy_sample); let sample_count = pw_sample_count.min(samples.len()); drop(samples.drain(..sample_count)); let chunk = pw_data.chunk_mut(); *chunk.offset_mut() = 0; *chunk.stride_mut() = (chan_size * chan_count) as _; *chunk.size_mut() = (sample_count * chan_size) as _; } } } ================================================ FILE: alvr/audio/src/windows.rs ================================================ use alvr_common::anyhow::{Result, bail}; use cpal::{Device, platform::DeviceInner}; use rodio::DeviceTrait; use windows::{ Win32::{ Devices::FunctionDiscovery::PKEY_Device_FriendlyName, Media::Audio::{ DEVICE_STATE_ACTIVE, Endpoints::IAudioEndpointVolume, IMMDevice, IMMDeviceEnumerator, MMDeviceEnumerator, eCapture, eRender, }, System::Com::{self, CLSCTX_ALL, COINIT_MULTITHREADED, STGM_READ}, }, core::GUID, }; fn get_windows_device(device: &Device) -> Result { let device_name = device.name()?; unsafe { // This will fail the second time is called, ignore the error Com::CoInitializeEx(None, COINIT_MULTITHREADED).ok().ok(); let imm_device_enumerator: IMMDeviceEnumerator = Com::CoCreateInstance(&MMDeviceEnumerator, None, CLSCTX_ALL)?; let direction = if device.supports_output() { eRender } else { eCapture }; let imm_device_collection = imm_device_enumerator.EnumAudioEndpoints(direction, DEVICE_STATE_ACTIVE)?; for i in 0..imm_device_collection.GetCount()? { let imm_device = imm_device_collection.Item(i)?; let imm_device_name = imm_device .OpenPropertyStore(STGM_READ)? .GetValue(&PKEY_Device_FriendlyName)? .to_string(); if imm_device_name == device_name { return Ok(imm_device); } } bail!("No device found with specified name") } } pub fn get_windows_device_id(device: &Device) -> Result { unsafe { let imm_device = get_windows_device(device)?; let id_str_ptr = imm_device.GetId()?; let id_str = id_str_ptr.to_string()?; Com::CoTaskMemFree(Some(id_str_ptr.0 as _)); Ok(id_str) } } // device must be an output device pub fn set_mute_windows_device(device: &Device, mute: bool) -> Result<()> { unsafe { let imm_device = get_windows_device(device)?; let endpoint_volume = imm_device.Activate::(CLSCTX_ALL, None)?; endpoint_volume.SetMute(mute, &GUID::zeroed())?; } Ok(()) } pub fn is_same_device(device1: &Device, device2: &Device) -> bool { let DeviceInner::Wasapi(dev1) = device1.as_inner(); let DeviceInner::Wasapi(dev2) = device2.as_inner(); dev1 == dev2 } ================================================ FILE: alvr/client_core/Cargo.toml ================================================ [package] name = "alvr_client_core" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [lib] crate-type = ["rlib", "staticlib", "cdylib"] [features] link-stdcpp-shared = [] default = ["link-stdcpp-shared"] [dependencies] alvr_audio.workspace = true alvr_common.workspace = true alvr_graphics.workspace = true alvr_packets.workspace = true alvr_session.workspace = true alvr_sockets.workspace = true alvr_system_info.workspace = true app_dirs2 = "2" mdns-sd = "0.14" rand = "0.9" serde = "1" serde_json = "1" [target.'cfg(target_os = "android")'.dependencies] android_logger = "0.15" ndk = { version = "0.9", features = ["api-level-28", "audio", "media"] } ndk-context = "0.1" [target.'cfg(not(target_os = "android"))'.dependencies] env_logger = "0.11" ================================================ FILE: alvr/client_core/LICENSE ================================================ Copyright (c) 2020-2024 alvr-org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: alvr/client_core/README.md ================================================ # alvr_client_core Rust crate containing all major components for an ALVR client except the XR-API-related code. ================================================ FILE: alvr/client_core/build.rs ================================================ fn main() { let platform_name = std::env::var("CARGO_CFG_TARGET_OS").unwrap(); if platform_name == "android" { println!("cargo:rustc-link-lib=log"); println!("cargo:rustc-link-lib=EGL"); println!("cargo:rustc-link-lib=GLESv3"); println!("cargo:rustc-link-lib=android"); #[cfg(feature = "link-stdcpp-shared")] println!("cargo:rustc-link-lib=c++_shared"); } } ================================================ FILE: alvr/client_core/cbindgen.toml ================================================ language = "C" header = "/* ALVR is licensed under the MIT license. https://github.com/alvr-org/ALVR/blob/master/LICENSE */" pragma_once = true autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */" cpp_compat = true tab_width = 4 documentation_style = "c99" [enum] rename_variants = "QualifiedScreamingSnakeCase" [parse] parse_deps = true include = ["alvr_common"] ================================================ FILE: alvr/client_core/src/audio.rs ================================================ use alvr_audio::Device; use alvr_common::{ anyhow::{Result, bail}, parking_lot::Mutex, }; use alvr_session::AudioBufferingConfig; use alvr_sockets::{StreamReceiver, StreamSender}; use ndk::audio::{ AudioCallbackResult, AudioDirection, AudioError, AudioFormat, AudioInputPreset, AudioPerformanceMode, AudioSharingMode, AudioStreamBuilder, }; use std::{ collections::VecDeque, mem, slice, sync::{Arc, mpsc}, time::Duration, }; const INPUT_SAMPLES_MAX_BUFFER_COUNT: usize = 20; const INPUT_RECV_TIMEOUT: Duration = Duration::from_millis(20); #[allow(unused_variables)] pub fn record_audio_blocking( is_running: Arc bool + Send + Sync>, mut sender: StreamSender<()>, device: &Device, channels_count: u16, mute: bool, ) -> Result<()> { assert_eq!( channels_count, 1, "This code only supports mono microphone input" ); let sample_rate = alvr_audio::input_sample_rate(device)?; let error = Arc::new(Mutex::new(None::)); let (samples_sender, samples_receiver) = mpsc::sync_channel::>(INPUT_SAMPLES_MAX_BUFFER_COUNT); let stream = AudioStreamBuilder::new()? .direction(AudioDirection::Input) .channel_count(1) .sample_rate(sample_rate as _) .format(AudioFormat::PCM_I16) .input_preset(AudioInputPreset::VoiceCommunication) .performance_mode(AudioPerformanceMode::LowLatency) .sharing_mode(AudioSharingMode::Shared) .data_callback(Box::new(move |_, data_ptr, frames_count| { let buffer_size = frames_count as usize * mem::size_of::(); let sample_buffer = unsafe { slice::from_raw_parts(data_ptr as *mut u8, buffer_size) }.to_vec(); // it will block only when the channel is full samples_sender.send(sample_buffer).ok(); AudioCallbackResult::Continue })) .error_callback({ let error = Arc::clone(&error); Box::new(move |_, e| *error.lock() = Some(e.into())) }) .open_stream()?; // If configuration changed, the stream must be restarted if stream.channel_count() != 1 || stream.sample_rate() != sample_rate as i32 || stream.format() != AudioFormat::PCM_I16 { bail!("Invalid audio configuration"); } stream.request_start()?; while is_running() && error.lock().is_none() { while let Ok(sample_buffer) = samples_receiver.recv_timeout(INPUT_RECV_TIMEOUT) { sender.send_header_with_payload(&(), &sample_buffer).ok(); } } // Note: request_stop() is asynchronous, and must be called always, even on error stream.request_stop()?; if let Some(e) = *error.lock() { return Err(e.into()); } Ok(()) } #[allow(unused_variables)] pub fn play_audio_loop( is_running: impl Fn() -> bool, device: &Device, channels_count: u16, sample_rate: u32, config: AudioBufferingConfig, receiver: &mut StreamReceiver<()>, ) -> Result<()> { assert_eq!(channels_count, 2, "This code only supports stereo output"); // the client sends invalid sample rates sometimes, and we crash if we try and use one // (batch_frames_count ends up zero and the audio callback gets confused) if sample_rate < 8000 { bail!("Invalid audio sample rate"); } let batch_frames_count = sample_rate as usize * config.batch_ms as usize / 1000; let average_buffer_frames_count = sample_rate as usize * config.average_buffering_ms as usize / 1000; let sample_buffer = Arc::new(Mutex::new(VecDeque::new())); let error = Arc::new(Mutex::new(None)); let stream = AudioStreamBuilder::new()? .direction(AudioDirection::Output) .channel_count(2) .sample_rate(sample_rate as _) .format(AudioFormat::PCM_Float) .frames_per_data_callback(batch_frames_count as _) .performance_mode(AudioPerformanceMode::LowLatency) .sharing_mode(AudioSharingMode::Shared) .data_callback({ let sample_buffer = Arc::clone(&sample_buffer); Box::new(move |_, data_ptr, frames_count| { assert!(frames_count == batch_frames_count as i32); let samples = alvr_audio::get_next_frame_batch( &mut *sample_buffer.lock(), 2, batch_frames_count as _, ); let out_frames = unsafe { // 2 channels slice::from_raw_parts_mut(data_ptr as *mut f32, frames_count as usize * 2) }; out_frames.copy_from_slice(&samples); AudioCallbackResult::Continue }) }) .error_callback({ let error = Arc::clone(&error); Box::new(move |_, e| *error.lock() = Some(e)) }) .open_stream()?; // If configuration changed, the stream must be restarted if stream.channel_count() != 2 || stream.sample_rate() != sample_rate as i32 || stream.format() != AudioFormat::PCM_Float || stream.frames_per_data_callback() != Some(batch_frames_count as _) { bail!("Invalid audio configuration"); } stream.request_start()?; alvr_audio::receive_samples_loop( || is_running() && error.lock().is_none(), receiver, sample_buffer, 2, batch_frames_count, average_buffer_frames_count, ) .ok(); // Note: request_stop() is asynchronous, and must be called always, even on error stream.request_stop()?; if let Some(e) = *error.lock() { return Err(e.into()); } Ok(()) } ================================================ FILE: alvr/client_core/src/c_api.rs ================================================ #![expect(dead_code)] use crate::{ ClientCapabilities, ClientCoreContext, ClientCoreEvent, storage, video_decoder::{self, VideoDecoderConfig, VideoDecoderSource}, }; use alvr_common::{ AlvrCodecType, AlvrFov, AlvrPose, AlvrQuat, AlvrViewParams, DeviceMotion, Pose, ViewParams, anyhow::Result, debug, error, glam::{UVec2, Vec2, Vec3}, info, parking_lot::Mutex, warn, }; use alvr_graphics::{ GraphicsContext, HandData, LobbyRenderer, LobbyViewParams, SDR_FORMAT_GL, StreamRenderer, StreamViewParams, }; use alvr_packets::{ButtonEntry, ButtonValue, FaceData, TrackingData}; use alvr_session::{ CodecType, FoveatedEncodingConfig, MediacodecPropType, MediacodecProperty, UpscalingConfig, }; use std::{ cell::RefCell, ffi::{CStr, CString, c_char, c_void}, ptr, rc::Rc, slice, time::{Duration, Instant}, }; static CLIENT_CORE_CONTEXT: Mutex> = Mutex::new(None); static HUD_MESSAGE: Mutex = Mutex::new(String::new()); static SETTINGS: Mutex = Mutex::new(String::new()); static SERVER_VERSION: Mutex = Mutex::new(String::new()); static DECODER_CONFIG_BUFFER: Mutex> = Mutex::new(vec![]); // Core interface: #[repr(C)] pub struct AlvrClientCapabilities { default_view_width: u32, default_view_height: u32, max_view_width: u32, max_view_height: u32, refresh_rates: *const f32, refresh_rates_count: u64, foveated_encoding: bool, encoder_high_profile: bool, encoder_10_bits: bool, encoder_av1: bool, prefer_10bit: bool, preferred_encoding_gamma: f32, prefer_hdr: bool, } #[repr(u8)] pub enum AlvrEvent { HudMessageUpdated, StreamingStarted { view_width: u32, view_height: u32, refresh_rate_hint: f32, encoding_gamma: f32, enable_foveated_encoding: bool, enable_hdr: bool, }, StreamingStopped, Haptics { device_id: u64, duration_s: f32, frequency: f32, amplitude: f32, }, /// Note: All subsequent DecoderConfig events should be ignored until reconnection DecoderConfig { codec: AlvrCodecType, }, // Unimplemented RealTimeConfig {}, } #[repr(C)] pub struct AlvrVideoFrameData { callback_context: *mut c_void, timestamp_ns: u64, buffer_ptr: *const u8, buffer_size: u64, } #[repr(C)] #[derive(Clone, Default)] pub struct AlvrDeviceMotion { device_id: u64, pose: AlvrPose, linear_velocity: [f32; 3], angular_velocity: [f32; 3], } #[allow(dead_code)] #[repr(C)] pub enum AlvrButtonValue { Binary(bool), Scalar(f32), } #[allow(dead_code)] #[repr(u8)] pub enum AlvrLogLevel { Error, Warn, Info, Debug, } #[unsafe(no_mangle)] pub extern "C" fn alvr_initialize_logging() { crate::init_logging(); } #[unsafe(no_mangle)] pub extern "C" fn alvr_path_string_to_id(path: *const c_char) -> u64 { alvr_common::hash_string(unsafe { CStr::from_ptr(path) }.to_str().unwrap()) } #[unsafe(no_mangle)] pub extern "C" fn alvr_log(level: AlvrLogLevel, message: *const c_char) { let message = unsafe { CStr::from_ptr(message) }.to_str().unwrap(); match level { AlvrLogLevel::Error => error!("[ALVR NATIVE] {message}"), AlvrLogLevel::Warn => warn!("[ALVR NATIVE] {message}"), AlvrLogLevel::Info => info!("[ALVR NATIVE] {message}"), AlvrLogLevel::Debug => debug!("[ALVR NATIVE] {message}"), } } #[unsafe(no_mangle)] #[cfg_attr(not(debug_assertions), expect(unused_variables))] pub extern "C" fn alvr_dbg_client_impl(message: *const c_char) { alvr_common::dbg_client_impl!("{}", unsafe { CStr::from_ptr(message) }.to_str().unwrap()) } #[unsafe(no_mangle)] #[cfg_attr(not(debug_assertions), expect(unused_variables))] pub extern "C" fn alvr_dbg_decoder(message: *const c_char) { alvr_common::dbg_decoder!("{}", unsafe { CStr::from_ptr(message) }.to_str().unwrap()) } #[unsafe(no_mangle)] pub extern "C" fn alvr_log_time(tag: *const c_char) { let tag = unsafe { CStr::from_ptr(tag) }.to_str().unwrap(); error!("[ALVR NATIVE] {tag}: {:?}", Instant::now()); } fn string_to_c_str(buffer: *mut c_char, value: &str) -> u64 { let cstring = CString::new(value).unwrap(); if !buffer.is_null() { unsafe { ptr::copy_nonoverlapping(cstring.as_ptr(), buffer, cstring.as_bytes_with_nul().len()); } } cstring.as_bytes_with_nul().len() as u64 } #[unsafe(no_mangle)] pub extern "C" fn alvr_mdns_service(service_buffer: *mut c_char) -> u64 { string_to_c_str(service_buffer, alvr_sockets::MDNS_SERVICE_TYPE) } /// To make sure the value is correct, call after alvr_initialize() #[unsafe(no_mangle)] pub extern "C" fn alvr_hostname(hostname_buffer: *mut c_char) -> u64 { string_to_c_str(hostname_buffer, &storage::Config::load().hostname) } /// To make sure the value is correct, call after alvr_initialize() #[unsafe(no_mangle)] pub extern "C" fn alvr_protocol_id(protocol_buffer: *mut c_char) -> u64 { string_to_c_str(protocol_buffer, &storage::Config::load().protocol_id) } #[cfg(target_os = "android")] #[unsafe(no_mangle)] pub extern "C" fn alvr_try_get_permission(permission: *const c_char) { alvr_system_info::try_get_permission(unsafe { CStr::from_ptr(permission) }.to_str().unwrap()); } /// NB: for android, `context` must be thread safe. #[cfg(target_os = "android")] #[unsafe(no_mangle)] pub extern "C" fn alvr_initialize_android_context(java_vm: *mut c_void, context: *mut c_void) { unsafe { ndk_context::initialize_android_context(java_vm, context) }; } /// On android, alvr_initialize_android_context() must be called first, then alvr_initialize(). #[unsafe(no_mangle)] pub extern "C" fn alvr_initialize(capabilities: AlvrClientCapabilities) { let default_view_resolution = UVec2::new( capabilities.default_view_width, capabilities.default_view_height, ); let max_view_resolution = UVec2::new(capabilities.max_view_width, capabilities.max_view_height); let refresh_rates = unsafe { slice::from_raw_parts( capabilities.refresh_rates, capabilities.refresh_rates_count as usize, ) } .to_vec(); let capabilities = ClientCapabilities { platform: alvr_system_info::platform(None, None), default_view_resolution, max_view_resolution, refresh_rates, foveated_encoding: capabilities.foveated_encoding, encoder_high_profile: capabilities.encoder_high_profile, encoder_10_bits: capabilities.encoder_10_bits, encoder_av1: capabilities.encoder_av1, prefer_10bit: capabilities.prefer_10bit, preferred_encoding_gamma: capabilities.preferred_encoding_gamma, prefer_hdr: capabilities.prefer_hdr, }; *CLIENT_CORE_CONTEXT.lock() = Some(ClientCoreContext::new(capabilities)); } #[unsafe(no_mangle)] pub extern "C" fn alvr_destroy() { *CLIENT_CORE_CONTEXT.lock() = None; #[cfg(target_os = "android")] unsafe { ndk_context::release_android_context() }; } #[unsafe(no_mangle)] pub extern "C" fn alvr_resume() { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.resume(); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_pause() { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.pause(); } } /// Returns true if there was a new event #[unsafe(no_mangle)] pub extern "C" fn alvr_poll_event(out_event: *mut AlvrEvent) -> bool { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() && let Some(event) = context.poll_event() { let event = match event { ClientCoreEvent::UpdateHudMessage(message) => { *HUD_MESSAGE.lock() = message; AlvrEvent::HudMessageUpdated } ClientCoreEvent::StreamingStarted(stream_config) => { *SETTINGS.lock() = serde_json::to_string(&stream_config.settings).unwrap(); *SERVER_VERSION.lock() = stream_config.server_version.to_string(); AlvrEvent::StreamingStarted { view_width: stream_config.negotiated_config.view_resolution.x, view_height: stream_config.negotiated_config.view_resolution.y, refresh_rate_hint: stream_config.negotiated_config.refresh_rate_hint, encoding_gamma: stream_config.negotiated_config.encoding_gamma, enable_foveated_encoding: stream_config .negotiated_config .enable_foveated_encoding, enable_hdr: stream_config.negotiated_config.enable_hdr, } } ClientCoreEvent::StreamingStopped => AlvrEvent::StreamingStopped, ClientCoreEvent::Haptics { device_id, duration, frequency, amplitude, } => AlvrEvent::Haptics { device_id, duration_s: duration.as_secs_f32(), frequency, amplitude, }, ClientCoreEvent::DecoderConfig { codec, config_nal } => { *DECODER_CONFIG_BUFFER.lock() = config_nal; AlvrEvent::DecoderConfig { codec: match codec { CodecType::H264 => AlvrCodecType::H264, CodecType::Hevc => AlvrCodecType::Hevc, CodecType::AV1 => AlvrCodecType::AV1, }, } } ClientCoreEvent::RealTimeConfig(_) => AlvrEvent::RealTimeConfig {}, }; unsafe { *out_event = event }; true } else { false } } // Returns the length of the message. message_buffer can be null. #[unsafe(no_mangle)] pub extern "C" fn alvr_hud_message(message_buffer: *mut c_char) -> u64 { let cstring = CString::new(HUD_MESSAGE.lock().clone()).unwrap(); if !message_buffer.is_null() { unsafe { ptr::copy_nonoverlapping( cstring.as_ptr(), message_buffer, cstring.as_bytes_with_nul().len(), ); } } cstring.as_bytes_with_nul().len() as u64 } /// Settings will be updated after receiving StreamingStarted event #[unsafe(no_mangle)] pub extern "C" fn alvr_get_settings_json(out_buffer: *mut c_char) -> u64 { string_to_c_str(out_buffer, &SETTINGS.lock()) } /// Will be updated after receiving StreamingStarted event #[unsafe(no_mangle)] pub extern "C" fn alvr_get_server_version(out_buffer: *mut c_char) -> u64 { string_to_c_str(out_buffer, &SERVER_VERSION.lock()) } /// Returns the number of bytes of the decoder_buffer #[unsafe(no_mangle)] pub extern "C" fn alvr_get_decoder_config(out_buffer: *mut c_char) -> u64 { let buffer = DECODER_CONFIG_BUFFER.lock(); let size = buffer.len(); if !out_buffer.is_null() { unsafe { ptr::copy_nonoverlapping(buffer.as_ptr(), out_buffer.cast(), size) } } size as u64 } #[unsafe(no_mangle)] pub extern "C" fn alvr_send_battery(device_id: u64, gauge_value: f32, is_plugged: bool) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.send_battery(device_id, gauge_value, is_plugged); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_send_playspace(width: f32, height: f32) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.send_playspace(Some(Vec2::new(width, height))); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_send_active_interaction_profile( device_id: u64, profile_id: u64, input_ids_ptr: *const u64, input_ids_count: u64, ) { let input_ids = if !input_ids_ptr.is_null() { unsafe { slice::from_raw_parts(input_ids_ptr, input_ids_count as usize) } } else { &[] }; if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.send_active_interaction_profile( device_id, profile_id, input_ids.iter().cloned().collect(), ); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_send_button(path_id: u64, value: AlvrButtonValue) { let value = match value { AlvrButtonValue::Binary(value) => ButtonValue::Binary(value), AlvrButtonValue::Scalar(value) => ButtonValue::Scalar(value), }; // crate::send_buttons(vec![ButtonEntry { path_id, value }]); if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.send_buttons(vec![ButtonEntry { path_id, value }]); } } /// The view poses need to be in local space, as if the head is at the origin. /// view_params: array of 2 #[unsafe(no_mangle)] pub extern "C" fn alvr_send_view_params(view_params: *const AlvrViewParams) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.send_view_params(unsafe { [ alvr_common::from_capi_view_params(&(*view_params)), alvr_common::from_capi_view_params(&(*view_params.offset(1))), ] }); } } /// hand_skeleton: /// * outer ptr: array of 2 (can be null); /// * inner ptr: array of 26 (can be null if hand is absent) /// /// combined_eye_gaze: can be null if eye gaze is absent #[unsafe(no_mangle)] pub extern "C" fn alvr_send_tracking( poll_timestamp_ns: u64, device_motions: *const AlvrDeviceMotion, device_motions_count: u64, hand_skeletons: *const *const AlvrPose, combined_eye_gaze: *const AlvrQuat, ) { let mut raw_motions = vec![AlvrDeviceMotion::default(); device_motions_count as _]; unsafe { ptr::copy_nonoverlapping( device_motions, raw_motions.as_mut_ptr(), device_motions_count as usize, ); } let device_motions = raw_motions .into_iter() .map(|motion| { ( motion.device_id, DeviceMotion { pose: alvr_common::from_capi_pose(&motion.pose), linear_velocity: Vec3::from_slice(&motion.linear_velocity), angular_velocity: Vec3::from_slice(&motion.angular_velocity), }, ) }) .collect::>(); let hand_skeletons = if !hand_skeletons.is_null() { let hand_skeletons = unsafe { slice::from_raw_parts(hand_skeletons, 2) }; let hand_skeletons = hand_skeletons .iter() .map(|&hand_skeleton| { (!hand_skeleton.is_null()).then(|| { let hand_skeleton = unsafe { slice::from_raw_parts(hand_skeleton, 26) }; let mut array = [Pose::IDENTITY; 26]; for (pose, capi_pose) in array.iter_mut().zip(hand_skeleton.iter()) { *pose = Pose { orientation: alvr_common::from_capi_quat(&capi_pose.orientation), position: Vec3::from_slice(&capi_pose.position), }; } array }) }) .collect::>(); [hand_skeletons[0], hand_skeletons[1]] } else { [None, None] }; let eyes_combined = if !combined_eye_gaze.is_null() { Some(alvr_common::from_capi_quat(unsafe { &*combined_eye_gaze })) } else { None }; if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.send_tracking(TrackingData { poll_timestamp: Duration::from_nanos(poll_timestamp_ns), device_motions, hand_skeletons, face: FaceData { eyes_combined, ..Default::default() }, body: None, }); } } /// Safety: `context` must be thread safe and valid until the StreamingStopped event. #[unsafe(no_mangle)] pub extern "C" fn alvr_set_decoder_input_callback( callback_context: *mut c_void, callback: extern "C" fn(AlvrVideoFrameData) -> bool, ) { struct CallbackContext(*mut c_void); unsafe impl Send for CallbackContext {} let callback_context = CallbackContext(callback_context); if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.set_decoder_input_callback(Box::new(move |timestamp, buffer| { // Make sure to capture the struct itself instead of just the pointer to make the // borrow checker happy let callback_context = &callback_context; callback(AlvrVideoFrameData { callback_context: callback_context.0, timestamp_ns: timestamp.as_nanos() as u64, buffer_ptr: buffer.as_ptr(), buffer_size: buffer.len() as u64, }) })); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_report_frame_decoded(target_timestamp_ns: u64) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.report_frame_decoded(Duration::from_nanos(target_timestamp_ns)); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_report_fatal_decoder_error(message: *const c_char) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.report_fatal_decoder_error(unsafe { CStr::from_ptr(message).to_str().unwrap() }); } } /// out_view_params must be a vector of 2 elements /// out_view_params is populated only if the core context is valid #[unsafe(no_mangle)] pub extern "C" fn alvr_report_compositor_start( target_timestamp_ns: u64, out_view_params: *mut AlvrViewParams, ) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { let view_params = context.report_compositor_start(Duration::from_nanos(target_timestamp_ns)); unsafe { *out_view_params = alvr_common::to_capi_view_params(&view_params[0]); *out_view_params.offset(1) = alvr_common::to_capi_view_params(&view_params[1]); } } } #[unsafe(no_mangle)] pub extern "C" fn alvr_report_submit(target_timestamp_ns: u64, vsync_queue_ns: u64) { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.report_submit( Duration::from_nanos(target_timestamp_ns), Duration::from_nanos(vsync_queue_ns), ); } } // OpenGL-related interface thread_local! { static GRAPHICS_CONTEXT: RefCell>> = const { RefCell::new(None) }; static LOBBY_RENDERER: RefCell> = const { RefCell::new(None) }; static STREAM_RENDERER: RefCell> = const { RefCell::new(None) }; } #[repr(C)] pub struct AlvrLobbyViewParams { swapchain_index: u32, view_params: AlvrViewParams, } #[repr(C)] pub struct AlvrStreamViewParams { swapchain_index: u32, reprojection_rotation: AlvrQuat, fov: AlvrFov, } #[repr(C)] pub struct AlvrStreamConfig { view_resolution_width: u32, view_resolution_height: u32, swapchain_textures: *mut *const u32, swapchain_length: u32, enable_foveation: bool, foveation_center_size_x: f32, foveation_center_size_y: f32, foveation_center_shift_x: f32, foveation_center_shift_y: f32, foveation_edge_ratio_x: f32, foveation_edge_ratio_y: f32, enable_upscaling: bool, upscaling_edge_direction: bool, upscaling_edge_threshold: f32, upscaling_edge_sharpness: f32, upscale_factor: f32, } #[unsafe(no_mangle)] pub extern "C" fn alvr_initialize_opengl() { GRAPHICS_CONTEXT.set(Some(Rc::new(GraphicsContext::new_gl()))); } #[unsafe(no_mangle)] pub extern "C" fn alvr_destroy_opengl() { GRAPHICS_CONTEXT.set(None); } fn convert_swapchain_array( swapchain_textures: *mut *const u32, swapchain_length: u32, ) -> [Vec; 2] { let swapchain_length = swapchain_length as usize; let mut left_swapchain = vec![0; swapchain_length]; unsafe { ptr::copy_nonoverlapping( *swapchain_textures, left_swapchain.as_mut_ptr(), swapchain_length, ) }; let mut right_swapchain = vec![0; swapchain_length]; unsafe { ptr::copy_nonoverlapping( *swapchain_textures.offset(1), right_swapchain.as_mut_ptr(), swapchain_length, ) }; [left_swapchain, right_swapchain] } #[unsafe(no_mangle)] pub extern "C" fn alvr_resume_opengl( preferred_view_width: u32, preferred_view_height: u32, swapchain_textures: *mut *const u32, swapchain_length: u32, ) { LOBBY_RENDERER.set(Some(LobbyRenderer::new( GRAPHICS_CONTEXT.with_borrow(|c| c.as_ref().unwrap().clone()), UVec2::new(preferred_view_width, preferred_view_height), convert_swapchain_array(swapchain_textures, swapchain_length), "", ))); } #[unsafe(no_mangle)] pub extern "C" fn alvr_pause_opengl() { STREAM_RENDERER.set(None); LOBBY_RENDERER.set(None) } #[unsafe(no_mangle)] pub extern "C" fn alvr_update_hud_message_opengl(message: *const c_char) { LOBBY_RENDERER.with_borrow(|renderer| { if let Some(renderer) = renderer { renderer.update_hud_message(unsafe { CStr::from_ptr(message) }.to_str().unwrap()); } }); } #[unsafe(no_mangle)] pub extern "C" fn alvr_start_stream_opengl(config: AlvrStreamConfig) { let view_resolution = UVec2::new(config.view_resolution_width, config.view_resolution_height); let swapchain_textures = convert_swapchain_array(config.swapchain_textures, config.swapchain_length); let foveated_encoding = config.enable_foveation.then_some(FoveatedEncodingConfig { force_enable: true, center_size_x: config.foveation_center_size_x, center_size_y: config.foveation_center_size_y, center_shift_x: config.foveation_center_shift_x, center_shift_y: config.foveation_center_shift_y, edge_ratio_x: config.foveation_edge_ratio_x, edge_ratio_y: config.foveation_edge_ratio_y, }); let upscaling = config.enable_upscaling.then_some(UpscalingConfig { edge_direction: config.upscaling_edge_direction, edge_sharpness: config.upscaling_edge_sharpness, edge_threshold: config.upscaling_edge_threshold, upscale_factor: config.upscale_factor, }); STREAM_RENDERER.set(Some(StreamRenderer::new( GRAPHICS_CONTEXT.with_borrow(|c| c.as_ref().unwrap().clone()), view_resolution, alvr_graphics::compute_target_view_resolution(view_resolution, &upscaling), swapchain_textures, SDR_FORMAT_GL, foveated_encoding, true, false, // TODO: limited range fix config 1.0, // TODO: encoding gamma config upscaling, ))); } // todo: support hands #[unsafe(no_mangle)] pub extern "C" fn alvr_render_lobby_opengl( view_inputs: *const AlvrLobbyViewParams, render_background: bool, ) { let view_inputs = unsafe { [ LobbyViewParams { swapchain_index: (*view_inputs).swapchain_index, view_params: alvr_common::from_capi_view_params(&(*view_inputs).view_params), }, LobbyViewParams { swapchain_index: (*view_inputs.offset(1)).swapchain_index, view_params: alvr_common::from_capi_view_params( &(*view_inputs.offset(1)).view_params, ), }, ] }; LOBBY_RENDERER.with_borrow(|renderer| { if let Some(renderer) = renderer { renderer.render( view_inputs, [ HandData { grip_motion: None, detached_grip_motion: None, skeleton_joints: None, }, HandData { grip_motion: None, detached_grip_motion: None, skeleton_joints: None, }, ], None, None, render_background, false, ); } }); } /// view_params: array of 2 #[unsafe(no_mangle)] pub extern "C" fn alvr_render_stream_opengl( hardware_buffer: *mut c_void, view_params: *const AlvrStreamViewParams, ) { STREAM_RENDERER.with_borrow(|renderer| { if let Some(renderer) = renderer { let left_params = unsafe { &*view_params }; let right_params = unsafe { &*view_params.offset(1) }; renderer.render( hardware_buffer, [ StreamViewParams { swapchain_index: left_params.swapchain_index, input_view_params: ViewParams { pose: Pose::IDENTITY, fov: alvr_common::from_capi_fov(&left_params.fov), }, output_view_params: ViewParams { pose: Pose { orientation: alvr_common::from_capi_quat( &left_params.reprojection_rotation, ), position: Vec3::ZERO, }, fov: alvr_common::from_capi_fov(&left_params.fov), }, }, StreamViewParams { swapchain_index: right_params.swapchain_index, input_view_params: ViewParams { pose: Pose::IDENTITY, fov: alvr_common::from_capi_fov(&right_params.fov), }, output_view_params: ViewParams { pose: Pose { orientation: alvr_common::from_capi_quat( &right_params.reprojection_rotation, ), position: Vec3::ZERO, }, fov: alvr_common::from_capi_fov(&right_params.fov), }, }, ], None, ); } }); } // Decoder-related interface static DECODER_SOURCE: Mutex> = Mutex::new(None); #[repr(u8)] pub enum AlvrMediacodecPropType { Float, Int32, Int64, String, } #[repr(C)] pub union AlvrMediacodecPropValue { float_: f32, int32: i32, int64: i64, string: *const c_char, } #[repr(C)] pub struct AlvrMediacodecOption { key: *const c_char, ty: AlvrMediacodecPropType, value: AlvrMediacodecPropValue, } #[repr(C)] pub struct AlvrDecoderConfig { codec: AlvrCodecType, force_software_decoder: bool, max_buffering_frames: f32, buffering_history_weight: f32, options: *const AlvrMediacodecOption, options_count: u64, config_buffer: *const u8, config_buffer_size: u64, } /// alvr_initialize() must be called before alvr_create_decoder #[unsafe(no_mangle)] pub extern "C" fn alvr_create_decoder(config: AlvrDecoderConfig) { let config = VideoDecoderConfig { codec: match config.codec { AlvrCodecType::H264 => CodecType::H264, AlvrCodecType::Hevc => CodecType::Hevc, AlvrCodecType::AV1 => CodecType::AV1, }, force_software_decoder: config.force_software_decoder, max_buffering_frames: config.max_buffering_frames, buffering_history_weight: config.buffering_history_weight, options: if !config.options.is_null() { let options = unsafe { slice::from_raw_parts(config.options, config.options_count as usize) }; options .iter() .map(|option| unsafe { let key = CStr::from_ptr(option.key).to_str().unwrap(); let prop = match option.ty { AlvrMediacodecPropType::Float => MediacodecProperty { ty: MediacodecPropType::Float, value: option.value.float_.to_string(), }, AlvrMediacodecPropType::Int32 => MediacodecProperty { ty: MediacodecPropType::Int32, value: option.value.int32.to_string(), }, AlvrMediacodecPropType::Int64 => MediacodecProperty { ty: MediacodecPropType::Int64, value: option.value.int64.to_string(), }, AlvrMediacodecPropType::String => MediacodecProperty { ty: MediacodecPropType::String, value: CStr::from_ptr(option.value.string) .to_str() .unwrap() .to_owned(), }, }; (key.to_owned(), prop) }) .collect() } else { vec![] }, config_buffer: unsafe { slice::from_raw_parts(config.config_buffer, config.config_buffer_size as usize).to_vec() }, }; let (mut sink, source) = video_decoder::create_decoder(config, |maybe_timestamp: Result| { if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { match maybe_timestamp { Ok(timestamp) => context.report_frame_decoded(timestamp), Err(e) => context.report_fatal_decoder_error(&e.to_string()), } } }); *DECODER_SOURCE.lock() = Some(source); if let Some(context) = &*CLIENT_CORE_CONTEXT.lock() { context.set_decoder_input_callback(Box::new(move |timestamp, buffer| { sink.push_nal(timestamp, buffer) })); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_destroy_decoder() { *DECODER_SOURCE.lock() = None; } // Returns true if the timestamp and buffer has been written to #[unsafe(no_mangle)] pub extern "C" fn alvr_get_frame( out_timestamp_ns: *mut u64, out_buffer_ptr: *mut *mut c_void, ) -> bool { if let Some(source) = &mut *DECODER_SOURCE.lock() && let Some((timestamp, buffer_ptr)) = source.get_frame() { unsafe { *out_timestamp_ns = timestamp.as_nanos() as u64; *out_buffer_ptr = buffer_ptr; } true } else { false } } #[unsafe(no_mangle)] pub extern "C" fn alvr_rotation_delta(source: AlvrQuat, destination: AlvrQuat) -> AlvrQuat { alvr_common::to_capi_quat( &(alvr_common::from_capi_quat(&source).inverse() * alvr_common::from_capi_quat(&destination)), ) } ================================================ FILE: alvr/client_core/src/connection.rs ================================================ #![allow(clippy::if_same_then_else)] use crate::{ ClientCapabilities, ClientCoreEvent, logging_backend::{LOG_CHANNEL_SENDER, LogMirrorData}, sockets::AnnouncerSocket, statistics::StatisticsManager, storage::Config, }; use alvr_common::{ ALVR_VERSION, AnyhowToCon, ConResult, ConnectionError, ConnectionState, LifecycleState, ViewParams, dbg_connection, debug, error, info, parking_lot::{Condvar, Mutex, RwLock}, wait_rwlock, warn, }; use alvr_packets::{ AUDIO, ClientConnectionResult, ClientControlPacket, ClientStatistics, ConnectionAcceptedInfo, HAPTICS, Haptics, STATISTICS, ServerControlPacket, StreamConfigPacket, TRACKING, TrackingData, VIDEO, VideoPacketHeader, VideoStreamingCapabilities, VideoStreamingCapabilitiesExt, }; use alvr_session::{SocketProtocol, settings_schema::Switch}; use alvr_sockets::{ ControlSocketSender, KEEPALIVE_INTERVAL, KEEPALIVE_TIMEOUT, PeerType, ProtoControlSocket, StreamSender, StreamSocketBuilder, }; use std::{ collections::VecDeque, sync::{Arc, mpsc}, thread, time::{Duration, Instant}, }; #[cfg(target_os = "android")] use crate::audio; #[cfg(not(target_os = "android"))] use alvr_audio as audio; const INITIAL_MESSAGE: &str = concat!( "Searching for streamer...\n", "Open ALVR on your PC then click \"Trust\"\n", "next to the device entry", ); const SUCCESS_CONNECT_MESSAGE: &str = "Successful connection!\nPlease wait..."; const STREAM_STARTING_MESSAGE: &str = "The stream will begin soon\nPlease wait..."; const SERVER_RESTART_MESSAGE: &str = "The streamer is restarting\nPlease wait..."; const SERVER_DISCONNECTED_MESSAGE: &str = "The streamer has disconnected."; const CONNECTION_TIMEOUT_MESSAGE: &str = "Connection timeout."; const SOCKET_INIT_RETRY_INTERVAL: Duration = Duration::from_millis(500); const CONNECTION_RETRY_INTERVAL: Duration = Duration::from_secs(1); const HANDSHAKE_ACTION_TIMEOUT: Duration = Duration::from_secs(2); const STREAMING_RECV_TIMEOUT: Duration = Duration::from_millis(500); const MAX_UNREAD_PACKETS: usize = 10; // Applies per stream pub type DecoderCallback = dyn FnMut(Duration, &[u8]) -> bool + Send; #[derive(Default)] pub struct ConnectionContext { pub state: RwLock, pub disconnected_notif: Condvar, pub control_sender: Mutex>>, pub tracking_sender: Mutex>>, pub statistics_sender: Mutex>>, pub statistics_manager: Mutex>, pub decoder_callback: Mutex>>, pub global_view_params_queue: Mutex>, pub max_prediction: RwLock, } fn set_hud_message(event_queue: &Mutex>, message: &str) { let message = format!( "ALVR v{}\nhostname: {}\nIP: {}\n\n{message}", *ALVR_VERSION, Config::load().hostname, alvr_system_info::local_ip(), ); event_queue .lock() .push_back(ClientCoreEvent::UpdateHudMessage(message)); } fn is_streaming(ctx: &ConnectionContext) -> bool { *ctx.state.read() == ConnectionState::Streaming } pub fn connection_lifecycle_loop( capabilities: ClientCapabilities, ctx: Arc, lifecycle_state: Arc>, event_queue: Arc>>, ) { dbg_connection!("connection_lifecycle_loop: Begin"); set_hud_message(&event_queue, INITIAL_MESSAGE); while *lifecycle_state.read() != LifecycleState::ShuttingDown { if *lifecycle_state.read() == LifecycleState::Resumed { if let Err(e) = connection_pipeline( capabilities.clone(), Arc::clone(&ctx), Arc::clone(&lifecycle_state), Arc::clone(&event_queue), ) { let message = format!("Connection error:\n{e}\nCheck the PC for more details"); set_hud_message(&event_queue, &message); error!("Connection error: {e}"); } } else { debug!("Skip try connection because the device is sleeping"); } *ctx.state.write() = ConnectionState::Disconnected; ctx.disconnected_notif.notify_all(); thread::sleep(CONNECTION_RETRY_INTERVAL); } dbg_connection!("connection_lifecycle_loop: End"); } fn connection_pipeline( capabilities: ClientCapabilities, ctx: Arc, lifecycle_state: Arc>, event_queue: Arc>>, ) -> ConResult { dbg_connection!("connection_pipeline: Begin"); let (mut proto_control_socket, server_ip) = { let config = Config::load(); let announcer_socket = AnnouncerSocket::new(&config.hostname).to_con()?; let listener_socket = alvr_sockets::get_server_listener(HANDSHAKE_ACTION_TIMEOUT).to_con()?; loop { if *lifecycle_state.write() != LifecycleState::Resumed { return Ok(()); } announcer_socket.announce().ok(); if let Ok(pair) = ProtoControlSocket::connect_to( SOCKET_INIT_RETRY_INTERVAL, PeerType::Server(&listener_socket), ) { set_hud_message(&event_queue, SUCCESS_CONNECT_MESSAGE); break pair; } } }; let mut connection_state_lock = ctx.state.write(); let disconnect_notif = Arc::new(Condvar::new()); *connection_state_lock = ConnectionState::Connecting; // TODO: Don't fetch cpal sample rate, get directly from AAudio let microphone_sample_rate = alvr_audio::input_sample_rate(&alvr_audio::new_input(None).to_con()?).to_con()?; dbg_connection!("connection_pipeline: Send stream capabilities"); proto_control_socket .send(&ClientConnectionResult::ConnectionAccepted(Box::new( ConnectionAcceptedInfo { client_protocol_id: alvr_common::protocol_id_u64(), platform_string: capabilities.platform.to_string(), server_ip, streaming_capabilities: Some( VideoStreamingCapabilities { default_view_resolution: capabilities.default_view_resolution, max_view_resolution: capabilities.max_view_resolution, refresh_rates: capabilities.refresh_rates, microphone_sample_rate, foveated_encoding: capabilities.foveated_encoding, encoder_high_profile: capabilities.encoder_high_profile, encoder_10_bits: capabilities.encoder_10_bits, encoder_av1: capabilities.encoder_av1, prefer_10bit: capabilities.prefer_10bit, preferred_encoding_gamma: capabilities.preferred_encoding_gamma, prefer_hdr: capabilities.prefer_hdr, ext_str: String::new(), } .with_ext(VideoStreamingCapabilitiesExt {}), ), }, ))) .to_con()?; let config_packet = proto_control_socket.recv::(HANDSHAKE_ACTION_TIMEOUT)?; dbg_connection!("connection_pipeline: stream config received"); let stream_config = config_packet.to_stream_config().to_con()?; let streaming_start_event = ClientCoreEvent::StreamingStarted(Box::new(stream_config.clone())); let settings = stream_config.settings; let negotiated_config = stream_config.negotiated_config; *ctx.max_prediction.write() = Duration::from_millis(settings.headset.max_prediction_ms); *ctx.statistics_manager.lock() = Some(StatisticsManager::new( settings.connection.statistics_history_size, )); let (mut control_sender, mut control_receiver) = proto_control_socket .split(STREAMING_RECV_TIMEOUT) .to_con()?; match control_receiver.recv(HANDSHAKE_ACTION_TIMEOUT) { Ok(ServerControlPacket::StartStream) => { info!("Stream starting"); set_hud_message(&event_queue, STREAM_STARTING_MESSAGE); } Ok(ServerControlPacket::Restarting) => { info!("Server restarting"); set_hud_message(&event_queue, SERVER_RESTART_MESSAGE); return Ok(()); } Err(e) => { info!("Server disconnected. Cause: {e}"); set_hud_message(&event_queue, SERVER_DISCONNECTED_MESSAGE); return Ok(()); } _ => { info!("Unexpected packet"); set_hud_message(&event_queue, "Unexpected packet"); return Ok(()); } } let stream_protocol = if negotiated_config.wired { SocketProtocol::Tcp } else { settings.connection.stream_protocol }; dbg_connection!("connection_pipeline: create StreamSocket"); let stream_socket_builder = StreamSocketBuilder::listen_for_server( Duration::from_secs(1), settings.connection.stream_port, stream_protocol, settings.connection.dscp, settings.connection.client_buffer_config, ) .to_con()?; dbg_connection!("connection_pipeline: Send StreamReady"); if let Err(e) = control_sender.send(&ClientControlPacket::StreamReady) { info!("Server disconnected. Cause: {e:?}"); set_hud_message(&event_queue, SERVER_DISCONNECTED_MESSAGE); return Ok(()); } dbg_connection!("connection_pipeline: accept connection"); let mut stream_socket = stream_socket_builder.accept_from_server( server_ip, settings.connection.stream_port, settings.connection.packet_size as _, HANDSHAKE_ACTION_TIMEOUT, )?; info!("Connected to server"); let mut video_receiver = stream_socket.subscribe_to_stream::(VIDEO, MAX_UNREAD_PACKETS); let mut game_audio_receiver = stream_socket.subscribe_to_stream(AUDIO, MAX_UNREAD_PACKETS); let tracking_sender = stream_socket.request_stream(TRACKING); let mut haptics_receiver = stream_socket.subscribe_to_stream::(HAPTICS, MAX_UNREAD_PACKETS); let statistics_sender = stream_socket.request_stream(STATISTICS); let video_receive_thread = thread::spawn({ let ctx = Arc::clone(&ctx); move || { let mut stream_corrupted = true; while is_streaming(&ctx) { let data = match video_receiver.recv(STREAMING_RECV_TIMEOUT) { Ok(data) => data, Err(ConnectionError::TryAgain(_)) => continue, Err(ConnectionError::Other(_)) => return, }; let Ok((header, nal)) = data.get() else { return; }; if let Some(stats) = &mut *ctx.statistics_manager.lock() { stats.report_video_packet_received(header.timestamp); } if header.is_idr { stream_corrupted = false; } else if data.had_packet_loss() { stream_corrupted = true; if let Some(sender) = &mut *ctx.control_sender.lock() { sender.send(&ClientControlPacket::RequestIdr).ok(); } warn!("Network dropped video packet"); } if !stream_corrupted || !settings.connection.avoid_video_glitching { // The view params must be enqueued before calling the decoder callback, there // is no problem if the callback fails { let global_view_params_queue_lock = &mut ctx.global_view_params_queue.lock(); global_view_params_queue_lock .push_back((header.timestamp, header.global_view_params)); while global_view_params_queue_lock.len() > 128 { global_view_params_queue_lock.pop_front(); } } let submitted = ctx .decoder_callback .lock() .as_mut() .is_some_and(|callback| callback(header.timestamp, nal)); if !submitted { stream_corrupted = true; if let Some(sender) = &mut *ctx.control_sender.lock() { sender.send(&ClientControlPacket::RequestIdr).ok(); } warn!("Dropped video packet. Reason: Decoder saturation") } } else { if let Some(sender) = &mut *ctx.control_sender.lock() { sender.send(&ClientControlPacket::RequestIdr).ok(); } warn!("Dropped video packet. Reason: Waiting for IDR frame") } } } }); let game_audio_thread = if let Switch::Enabled(config) = settings.audio.game_audio { let device = alvr_audio::new_output(None).to_con()?; thread::spawn({ let ctx = Arc::clone(&ctx); move || { while is_streaming(&ctx) { alvr_common::show_err(audio::play_audio_loop( || is_streaming(&ctx), &device, 2, negotiated_config.game_audio_sample_rate, config.buffering.clone(), &mut game_audio_receiver, )); } } }) } else { thread::spawn(|| ()) }; let microphone_thread = if matches!(settings.audio.microphone, Switch::Enabled(_)) { let device = alvr_audio::new_input(None).to_con()?; let microphone_sender = stream_socket.request_stream(AUDIO); thread::spawn({ let ctx = Arc::clone(&ctx); move || { while is_streaming(&ctx) { let ctx = Arc::clone(&ctx); match audio::record_audio_blocking( Arc::new(move || is_streaming(&ctx)), microphone_sender.clone(), &device, 1, false, ) { Ok(()) => break, Err(e) => { error!("Audio record error: {e}"); continue; } } } } }) } else { thread::spawn(|| ()) }; let haptics_receive_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let event_queue = Arc::clone(&event_queue); move || { while is_streaming(&ctx) { let data = match haptics_receiver.recv(STREAMING_RECV_TIMEOUT) { Ok(packet) => packet, Err(ConnectionError::TryAgain(_)) => continue, Err(ConnectionError::Other(_)) => return, }; let Ok(haptics) = data.get_header() else { return; }; event_queue.lock().push_back(ClientCoreEvent::Haptics { device_id: haptics.device_id, duration: haptics.duration, frequency: haptics.frequency, amplitude: haptics.amplitude, }); } } }); let (log_channel_sender, log_channel_receiver) = mpsc::channel(); let control_send_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let event_queue = Arc::clone(&event_queue); let disconnect_notif = Arc::clone(&disconnect_notif); move || { let mut keepalive_deadline = Instant::now(); #[cfg(target_os = "android")] let mut battery_deadline = Instant::now(); while is_streaming(&ctx) && *lifecycle_state.read() == LifecycleState::Resumed { if let Ok(packet) = log_channel_receiver.recv_timeout(STREAMING_RECV_TIMEOUT) && let Some(sender) = &mut *ctx.control_sender.lock() && let Err(e) = sender.send(&packet) { info!("Server disconnected. Cause: {e:?}"); set_hud_message(&event_queue, SERVER_DISCONNECTED_MESSAGE); break; } if Instant::now() > keepalive_deadline && let Some(sender) = &mut *ctx.control_sender.lock() { sender.send(&ClientControlPacket::KeepAlive).ok(); keepalive_deadline = Instant::now() + KEEPALIVE_INTERVAL; } #[cfg(target_os = "android")] if Instant::now() > battery_deadline { let (gauge_value, is_plugged) = alvr_system_info::get_battery_status(); if let Some(sender) = &mut *ctx.control_sender.lock() { sender .send(&ClientControlPacket::Battery(crate::BatteryInfo { device_id: *alvr_common::HEAD_ID, gauge_value, is_plugged, })) .ok(); } battery_deadline = Instant::now() + Duration::from_secs(5); } } disconnect_notif.notify_one(); } }); let control_receive_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let event_queue = Arc::clone(&event_queue); let disconnect_notif = Arc::clone(&disconnect_notif); move || { let mut disconnection_deadline = Instant::now() + KEEPALIVE_TIMEOUT; while is_streaming(&ctx) { let maybe_packet = control_receiver.recv(STREAMING_RECV_TIMEOUT); match maybe_packet { Ok(ServerControlPacket::DecoderConfig(config)) => { event_queue .lock() .push_back(ClientCoreEvent::DecoderConfig { codec: config.codec, config_nal: config.config_buffer, }); } Ok(ServerControlPacket::Restarting) => { info!("{SERVER_RESTART_MESSAGE}"); set_hud_message(&event_queue, SERVER_RESTART_MESSAGE); disconnect_notif.notify_one(); } Ok(ServerControlPacket::RealTimeConfig(config)) => { event_queue .lock() .push_back(ClientCoreEvent::RealTimeConfig(config)); } Ok(ServerControlPacket::StartStream) => { error!("Unexpected StartStream paceket"); } Ok(ServerControlPacket::KeepAlive) => (), Ok( ServerControlPacket::Reserved(_) | ServerControlPacket::ReservedBuffer(_), ) => {} Err(ConnectionError::TryAgain(_)) => { if Instant::now() > disconnection_deadline { info!("{CONNECTION_TIMEOUT_MESSAGE}"); set_hud_message(&event_queue, CONNECTION_TIMEOUT_MESSAGE); disconnect_notif.notify_one(); } else { continue; } } Err(e) => { info!("{SERVER_DISCONNECTED_MESSAGE} Cause: {e}"); set_hud_message(&event_queue, SERVER_DISCONNECTED_MESSAGE); disconnect_notif.notify_one(); } } disconnection_deadline = Instant::now() + KEEPALIVE_TIMEOUT; } } }); let stream_receive_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let event_queue = Arc::clone(&event_queue); let disconnect_notif = Arc::clone(&disconnect_notif); move || { while is_streaming(&ctx) { match stream_socket.recv() { Ok(()) => (), Err(ConnectionError::TryAgain(_)) => continue, Err(e) => { info!("Client disconnected. Cause: {e}"); set_hud_message(&event_queue, SERVER_DISCONNECTED_MESSAGE); disconnect_notif.notify_one(); } } } } }); *ctx.control_sender.lock() = Some(control_sender); *ctx.tracking_sender.lock() = Some(tracking_sender); *ctx.statistics_sender.lock() = Some(statistics_sender); if let Switch::Enabled(filter_level) = settings.extra.logging.client_log_report_level { *LOG_CHANNEL_SENDER.lock() = Some(LogMirrorData { sender: log_channel_sender, filter_level, debug_groups_config: settings.extra.logging.debug_groups, }); } event_queue.lock().push_back(streaming_start_event); *connection_state_lock = ConnectionState::Streaming; dbg_connection!("connection_pipeline: Unlock streams"); // Unlock CONNECTION_STATE and block thread wait_rwlock(&disconnect_notif, &mut connection_state_lock); *connection_state_lock = ConnectionState::Disconnecting; *ctx.control_sender.lock() = None; *ctx.tracking_sender.lock() = None; *ctx.statistics_sender.lock() = None; *LOG_CHANNEL_SENDER.lock() = None; event_queue .lock() .push_back(ClientCoreEvent::StreamingStopped); // Remove lock to allow threads to properly exit: drop(connection_state_lock); dbg_connection!("connection_pipeline: Destroying streams"); video_receive_thread.join().ok(); game_audio_thread.join().ok(); microphone_thread.join().ok(); haptics_receive_thread.join().ok(); control_send_thread.join().ok(); control_receive_thread.join().ok(); stream_receive_thread.join().ok(); dbg_connection!("connection_pipeline: End"); Ok(()) } ================================================ FILE: alvr/client_core/src/lib.rs ================================================ #![allow( non_upper_case_globals, non_snake_case, clippy::missing_safety_doc, clippy::unseparated_literal_suffix )] mod c_api; mod connection; mod logging_backend; mod sockets; mod statistics; mod storage; #[cfg(target_os = "android")] mod audio; pub mod video_decoder; use alvr_common::{ ConnectionState, LifecycleState, ViewParams, dbg_client_core, error, glam::{UVec2, Vec2}, parking_lot::{Mutex, RwLock}, warn, }; use alvr_packets::{ BatteryInfo, ButtonEntry, ClientControlPacket, RealTimeConfig, StreamConfig, TrackingData, }; use alvr_session::CodecType; use alvr_system_info::Platform; use connection::{ConnectionContext, DecoderCallback}; use std::{ collections::{HashSet, VecDeque}, sync::Arc, thread::{self, JoinHandle}, time::Duration, }; use storage::Config; pub use logging_backend::init_logging; pub enum ClientCoreEvent { UpdateHudMessage(String), StreamingStarted(Box), StreamingStopped, Haptics { device_id: u64, duration: Duration, frequency: f32, amplitude: f32, }, // Note: All subsequent DecoderConfig events should be ignored until reconnection DecoderConfig { codec: CodecType, config_nal: Vec, }, RealTimeConfig(RealTimeConfig), } // Note: this struct may change without breaking network protocol changes #[derive(Clone)] pub struct ClientCapabilities { pub platform: Platform, pub default_view_resolution: UVec2, pub max_view_resolution: UVec2, pub refresh_rates: Vec, pub foveated_encoding: bool, pub encoder_high_profile: bool, pub encoder_10_bits: bool, pub encoder_av1: bool, pub prefer_10bit: bool, pub preferred_encoding_gamma: f32, pub prefer_hdr: bool, } pub struct ClientCoreContext { platform: Platform, lifecycle_state: Arc>, event_queue: Arc>>, connection_context: Arc, connection_thread: Arc>>>, last_good_global_view_params: Mutex<[ViewParams; 2]>, } impl ClientCoreContext { pub fn new(capabilities: ClientCapabilities) -> Self { dbg_client_core!("Create"); // Make sure to reset config in case of version compat mismatch. if Config::load().protocol_id != alvr_common::protocol_id() { // NB: Config::default() sets the current protocol ID Config::default().store(); } #[cfg(target_os = "android")] { dbg_client_core!("Getting permissions"); alvr_system_info::try_get_permission(alvr_system_info::MICROPHONE_PERMISSION); alvr_system_info::set_wifi_lock(true); } let platform = capabilities.platform; let lifecycle_state = Arc::new(RwLock::new(LifecycleState::Idle)); let event_queue = Arc::new(Mutex::new(VecDeque::new())); let connection_context = Arc::new(ConnectionContext::default()); let connection_thread = thread::spawn({ let lifecycle_state = Arc::clone(&lifecycle_state); let connection_context = Arc::clone(&connection_context); let event_queue = Arc::clone(&event_queue); move || { connection::connection_lifecycle_loop( capabilities, connection_context, lifecycle_state, event_queue, ) } }); Self { platform, lifecycle_state, event_queue, connection_context, connection_thread: Arc::new(Mutex::new(Some(connection_thread))), last_good_global_view_params: Mutex::new([ViewParams::DUMMY; 2]), } } pub fn resume(&self) { dbg_client_core!("resume"); *self.lifecycle_state.write() = LifecycleState::Resumed; } pub fn pause(&self) { dbg_client_core!("pause"); let mut connection_state_lock = self.connection_context.state.write(); *self.lifecycle_state.write() = LifecycleState::Idle; // We want to shutdown streaming when pausing. if *connection_state_lock != ConnectionState::Disconnected { alvr_common::wait_rwlock( &self.connection_context.disconnected_notif, &mut connection_state_lock, ); } } pub fn poll_event(&self) -> Option { dbg_client_core!("poll_event"); self.event_queue.lock().pop_front() } pub fn send_battery(&self, device_id: u64, gauge_value: f32, is_plugged: bool) { dbg_client_core!("send_battery"); if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender .send(&ClientControlPacket::Battery(BatteryInfo { device_id, gauge_value, is_plugged, })) .ok(); } } pub fn send_playspace(&self, area: Option) { dbg_client_core!("send_playspace"); if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender.send(&ClientControlPacket::PlayspaceSync(area)).ok(); } } pub fn send_active_interaction_profile( &self, device_id: u64, profile_id: u64, input_ids: HashSet, ) { dbg_client_core!("send_active_interaction_profile"); if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender .send(&ClientControlPacket::ActiveInteractionProfile { device_id, profile_id, input_ids, }) .ok(); } } pub fn send_buttons(&self, entries: Vec) { dbg_client_core!("send_buttons"); if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender.send(&ClientControlPacket::Buttons(entries)).ok(); } } // These must be in its local space, as if the head pose is in the origin. pub fn send_view_params(&self, views: [ViewParams; 2]) { dbg_client_core!("send_view_params"); if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender .send(&ClientControlPacket::LocalViewParams(views)) .ok(); } } pub fn send_tracking(&self, data: TrackingData) { dbg_client_core!("send_tracking"); if let Some(sender) = &mut *self.connection_context.tracking_sender.lock() { sender.send_header(&data).ok(); if let Some(stats) = &mut *self.connection_context.statistics_manager.lock() { stats.report_input_acquired(data.poll_timestamp); } } } pub fn send_proximity_state(&self, headset_is_worn: bool) { if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender .send(&ClientControlPacket::ProximityState(headset_is_worn)) .ok(); } } pub fn get_total_prediction_offset(&self) -> Duration { dbg_client_core!("get_total_prediction_offset"); if let Some(stats) = &*self.connection_context.statistics_manager.lock() { Duration::min( stats.average_total_pipeline_latency(), *self.connection_context.max_prediction.read(), ) } else { Duration::ZERO } } /// The callback should return true if the frame was successfully submitted to the decoder pub fn set_decoder_input_callback(&self, callback: Box) { dbg_client_core!("set_decoder_input_callback"); *self.connection_context.decoder_callback.lock() = Some(callback); if let Some(sender) = &mut *self.connection_context.control_sender.lock() { sender.send(&ClientControlPacket::RequestIdr).ok(); } } pub fn report_frame_decoded(&self, timestamp: Duration) { dbg_client_core!("report_frame_decoded"); if let Some(stats) = &mut *self.connection_context.statistics_manager.lock() { stats.report_frame_decoded(timestamp); } } pub fn report_fatal_decoder_error(&self, error: &str) { error!("Fatal decoder error, restarting connection: {error}"); // The connection loop observes changes on this value *self.connection_context.state.write() = ConnectionState::Disconnecting; } pub fn report_compositor_start(&self, timestamp: Duration) -> [ViewParams; 2] { dbg_client_core!("report_compositor_start"); if let Some(stats) = &mut *self.connection_context.statistics_manager.lock() { stats.report_compositor_start(timestamp); } let global_view_params_lock = &mut *self.last_good_global_view_params.lock(); for (ts, params) in &*self.connection_context.global_view_params_queue.lock() { if *ts == timestamp { *global_view_params_lock = *params; break; } } *global_view_params_lock } pub fn report_submit(&self, timestamp: Duration, vsync_queue: Duration) { dbg_client_core!("report_submit"); if let Some(stats) = &mut *self.connection_context.statistics_manager.lock() { stats.report_submit(timestamp, vsync_queue); if let Some(sender) = &mut *self.connection_context.statistics_sender.lock() { if let Some(stats) = stats.summary(timestamp) { sender.send_header(&stats).ok(); } else { warn!("Statistics summary not ready!"); } } } } pub fn platform(&self) -> Platform { self.platform } } impl Drop for ClientCoreContext { fn drop(&mut self) { dbg_client_core!("Drop"); *self.lifecycle_state.write() = LifecycleState::ShuttingDown; if let Some(thread) = self.connection_thread.lock().take() { thread.join().ok(); } #[cfg(target_os = "android")] alvr_system_info::set_wifi_lock(false); } } ================================================ FILE: alvr/client_core/src/logging_backend.rs ================================================ use alvr_common::{ DebugGroupsConfig, LogSeverity, log::{Level, Record}, parking_lot::Mutex, }; use alvr_packets::ClientControlPacket; use std::{ sync::{LazyLock, mpsc}, time::{Duration, Instant}, }; const LOG_REPEAT_TIMEOUT: Duration = Duration::from_secs(1); pub struct LogMirrorData { pub sender: mpsc::Sender, pub filter_level: LogSeverity, pub debug_groups_config: DebugGroupsConfig, } pub static LOG_CHANNEL_SENDER: Mutex> = Mutex::new(None); struct RepeatedLogEvent { message: String, repetition_times: usize, initial_timestamp: Instant, } static LAST_LOG_EVENT: LazyLock> = LazyLock::new(|| { Mutex::new(RepeatedLogEvent { message: "".into(), repetition_times: 0, initial_timestamp: Instant::now(), }) }); pub fn init_logging() { fn send_log(record: &Record) -> bool { let Some(data) = &*LOG_CHANNEL_SENDER.lock() else { // if channel has not been setup, always print everything to stdout // todo: the client debug groups settings should be moved client side when feasible return true; }; let level = match record.level() { Level::Error => LogSeverity::Error, Level::Warn => LogSeverity::Warning, Level::Info => LogSeverity::Info, Level::Debug | Level::Trace => LogSeverity::Debug, }; if level < data.filter_level { return false; } let message = format!("{}", record.args()); if !alvr_common::filter_debug_groups(&message, &data.debug_groups_config) { return false; } let mut last_log_event_lock = LAST_LOG_EVENT.lock(); if last_log_event_lock.message == message && last_log_event_lock.initial_timestamp + LOG_REPEAT_TIMEOUT > Instant::now() { last_log_event_lock.repetition_times += 1; } else { if last_log_event_lock.repetition_times > 1 { data.sender .send(ClientControlPacket::Log { level: LogSeverity::Info, message: format!( "Last log line repeated {} times", last_log_event_lock.repetition_times ), }) .ok(); } *last_log_event_lock = RepeatedLogEvent { message: message.clone(), repetition_times: 1, initial_timestamp: Instant::now(), }; data.sender .send(ClientControlPacket::Log { level, message }) .ok(); } true } #[cfg(target_os = "android")] { android_logger::init_once( android_logger::Config::default() .with_tag("[ALVR NATIVE-RUST]") .format(|f, record| { if send_log(record) { writeln!(f, "{}", record.args()) } else { Ok(()) } }) .with_max_level(alvr_common::log::LevelFilter::Info), ); } #[cfg(not(target_os = "android"))] { use std::io::Write; env_logger::builder() .format(|f, record| { if send_log(record) { writeln!(f, "{}", record.args()) } else { Ok(()) } }) .try_init() .ok(); } alvr_common::set_panic_hook(); } ================================================ FILE: alvr/client_core/src/sockets.rs ================================================ use alvr_common::anyhow::{Result, bail}; use mdns_sd::{ServiceDaemon, ServiceInfo}; pub struct AnnouncerSocket { hostname: String, daemon: ServiceDaemon, } impl AnnouncerSocket { pub fn new(hostname: &str) -> Result { let daemon = ServiceDaemon::new()?; Ok(Self { daemon, hostname: hostname.to_owned(), }) } pub fn announce(&self) -> Result<()> { let local_ip = alvr_system_info::local_ip(); if local_ip.is_unspecified() { bail!("IP is unspecified"); } self.daemon.register(ServiceInfo::new( alvr_sockets::MDNS_SERVICE_TYPE, &format!("alvr{}", rand::random::()), &self.hostname, local_ip, 5353, &[( alvr_sockets::MDNS_PROTOCOL_KEY, alvr_common::protocol_id().as_str(), )][..], )?)?; Ok(()) } } ================================================ FILE: alvr/client_core/src/statistics.rs ================================================ use alvr_common::SlidingWindowAverage; use alvr_packets::ClientStatistics; use std::{ collections::VecDeque, time::{Duration, Instant}, }; struct HistoryFrame { input_acquired: Instant, video_packet_received: Instant, client_stats: ClientStatistics, } pub struct StatisticsManager { history_buffer: VecDeque, max_history_size: usize, prev_vsync: Instant, total_pipeline_latency_average: SlidingWindowAverage, } impl StatisticsManager { pub fn new(max_history_size: usize) -> Self { Self { max_history_size, history_buffer: VecDeque::new(), prev_vsync: Instant::now(), total_pipeline_latency_average: SlidingWindowAverage::new( Duration::ZERO, max_history_size, ), } } pub fn report_input_acquired(&mut self, target_timestamp: Duration) { if !self .history_buffer .iter() .any(|frame| frame.client_stats.target_timestamp == target_timestamp) { self.history_buffer.push_front(HistoryFrame { input_acquired: Instant::now(), // this is just a placeholder because Instant does not have a default value video_packet_received: Instant::now(), client_stats: ClientStatistics { target_timestamp, ..Default::default() }, }); } if self.history_buffer.len() > self.max_history_size { self.history_buffer.pop_back(); } } pub fn report_video_packet_received(&mut self, target_timestamp: Duration) { if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.client_stats.target_timestamp == target_timestamp) { frame.video_packet_received = Instant::now(); } } pub fn report_frame_decoded(&mut self, target_timestamp: Duration) { if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.client_stats.target_timestamp == target_timestamp) { frame.client_stats.video_decode = Instant::now().saturating_duration_since(frame.video_packet_received); } } pub fn report_compositor_start(&mut self, target_timestamp: Duration) { if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.client_stats.target_timestamp == target_timestamp) { frame.client_stats.video_decoder_queue = Instant::now().saturating_duration_since( frame.video_packet_received + frame.client_stats.video_decode, ); } } // vsync_queue is the latency between this call and the vsync. it cannot be measured by ALVR and // should be reported by the VR runtime pub fn report_submit(&mut self, target_timestamp: Duration, vsync_queue: Duration) { let now = Instant::now(); if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.client_stats.target_timestamp == target_timestamp) { frame.client_stats.rendering = now.saturating_duration_since( frame.video_packet_received + frame.client_stats.video_decode + frame.client_stats.video_decoder_queue, ); frame.client_stats.vsync_queue = vsync_queue; frame.client_stats.total_pipeline_latency = now.saturating_duration_since(frame.input_acquired) + vsync_queue; self.total_pipeline_latency_average .submit_sample(frame.client_stats.total_pipeline_latency); let vsync = now + vsync_queue; frame.client_stats.frame_interval = vsync.saturating_duration_since(self.prev_vsync); self.prev_vsync = vsync; } } pub fn summary(&self, target_timestamp: Duration) -> Option { self.history_buffer .iter() .find(|frame| frame.client_stats.target_timestamp == target_timestamp) .map(|frame| frame.client_stats.clone()) } // latency used for head prediction pub fn average_total_pipeline_latency(&self) -> Duration { self.total_pipeline_latency_average.get_average() } } ================================================ FILE: alvr/client_core/src/storage.rs ================================================ use alvr_common::{error, info}; use app_dirs2::{AppDataType, AppInfo}; use rand::Rng; use serde::{Deserialize, Serialize}; use std::{fs, path::PathBuf}; fn config_path() -> PathBuf { app_dirs2::app_root( AppDataType::UserConfig, &AppInfo { name: "ALVR Client", author: "ALVR", }, ) .unwrap() .join("session.json") } #[derive(Serialize, Deserialize)] pub struct Config { pub hostname: String, pub protocol_id: String, } impl Default for Config { fn default() -> Self { let mut rng = rand::rng(); Self { hostname: format!( "{}{}{}{}.client.local.", rng.random_range(0..10), rng.random_range(0..10), rng.random_range(0..10), rng.random_range(0..10), ), protocol_id: alvr_common::protocol_id(), } } } impl Config { pub fn load() -> Self { if let Ok(config_string) = fs::read_to_string(config_path()) { // Failure happens if the Config signature changed between versions. // todo: recover data from mismatched Config signature. low priority if let Ok(config) = serde_json::from_str(&config_string) { return config; } else { info!("Error parsing ALVR config. Using default"); } } else { info!("Error reading ALVR config. Using default"); } let config = Config::default(); config.store(); config } pub fn store(&self) { let config_string = serde_json::to_string(self).unwrap(); if let Err(e) = fs::write(config_path(), config_string) { error!("Error writing ALVR config: {e}") } } } ================================================ FILE: alvr/client_core/src/video_decoder/android.rs ================================================ use super::VideoDecoderConfig; use alvr_common::{ RelaxedAtomic, ToAny, anyhow::{Context, Result, anyhow, bail}, error, info, parking_lot::{Condvar, Mutex}, warn, }; use alvr_session::{CodecType, MediacodecPropType}; use ndk::{ hardware_buffer::HardwareBufferUsage, media::{ image_reader::{AcquireResult, Image, ImageFormat, ImageReader}, media_codec::{ DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection, MediaFormat, }, }, }; use std::{ collections::VecDeque, ffi::c_void, ops::Deref, ptr, sync::{Arc, Weak}, thread::{self, JoinHandle}, time::Duration, }; struct FakeThreadSafe(T); unsafe impl Send for FakeThreadSafe {} unsafe impl Sync for FakeThreadSafe {} impl Deref for FakeThreadSafe { type Target = T; fn deref(&self) -> &T { &self.0 } } type SharedMediaCodec = Arc>; pub struct VideoDecoderSink { inner: Arc>>, } unsafe impl Send for VideoDecoderSink {} impl VideoDecoderSink { // Block until the buffer has been written or timeout is reached. Returns false if timeout. pub fn push_frame_nal(&mut self, timestamp: Duration, data: &[u8]) -> Result { let Some(decoder) = &*self.inner.lock() else { // This might happen only during destruction return Ok(false); }; match decoder.dequeue_input_buffer(Duration::ZERO) { Ok(DequeuedInputBufferResult::Buffer(mut buffer)) => { unsafe { ptr::copy_nonoverlapping( data.as_ptr(), buffer.buffer_mut().as_mut_ptr().cast(), data.len(), ) }; // NB: the function expects the timestamp in micros, but nanos is used to have // complete precision, so when converted back to Duration it can compare correctly // to other Durations decoder.queue_input_buffer(buffer, 0, data.len(), timestamp.as_nanos() as _, 0)?; Ok(true) } Ok(DequeuedInputBufferResult::TryAgainLater) => Ok(false), Err(e) => bail!("{e}"), } } } struct QueuedImage { timestamp: Duration, image: Image, in_use: bool, } unsafe impl Send for QueuedImage {} // Access the image queue synchronously. pub struct VideoDecoderSource { running: Arc, dequeue_thread: Option>, image_queue: Arc>>, config: VideoDecoderConfig, buffering_running_average: f32, } unsafe impl Send for VideoDecoderSource {} impl VideoDecoderSource { // The application MUST finish using the returned buffer before calling this function again pub fn dequeue_frame(&mut self) -> Option<(Duration, *mut c_void)> { let mut image_queue_lock = self.image_queue.lock(); if let Some(queued_image) = image_queue_lock.front() && queued_image.in_use { // image is released and ready to be reused by the decoder image_queue_lock.pop_front(); } // use running average to give more weight to recent samples self.buffering_running_average = self.buffering_running_average * self.config.buffering_history_weight + image_queue_lock.len() as f32 * (1. - self.config.buffering_history_weight); if self.buffering_running_average > self.config.max_buffering_frames as f32 { image_queue_lock.pop_front(); } if let Some(queued_image) = image_queue_lock.front_mut() { queued_image.in_use = true; Some(( queued_image.timestamp, queued_image .image .hardware_buffer() .unwrap() .as_ptr() .cast(), )) } else { // TODO: add back when implementing proper phase sync //warn!("Video frame queue underflow!"); None } } } impl Drop for VideoDecoderSource { fn drop(&mut self) { self.running.set(false); // Destruction of decoder, buffered images and ImageReader self.dequeue_thread.take().map(|t| t.join()); } } fn mime_for_codec(codec: CodecType) -> &'static str { match codec { CodecType::H264 => "video/avc", CodecType::Hevc => "video/hevc", CodecType::AV1 => "video/av01", } } // Attempts to create a MediaCodec, and then configure and start it. fn decoder_attempt_setup( codec_type: CodecType, is_software: bool, format: &MediaFormat, image_reader: &ImageReader, ) -> Result { let decoder = if is_software { let sw_codec_name = match codec_type { CodecType::H264 => "OMX.google.h264.decoder", CodecType::Hevc => "OMX.google.hevc.decoder", CodecType::AV1 => bail!("AV1 is not supported for software decoding"), }; MediaCodec::from_codec_name(&sw_codec_name) .ok_or(anyhow!("no such codec: {}", &sw_codec_name))? } else { let mime = mime_for_codec(codec_type); MediaCodec::from_decoder_type(&mime) .ok_or(anyhow!("unable to find decoder for mime type: {}", &mime))? }; decoder .configure( &format, Some(&image_reader.window()?), MediaCodecDirection::Decoder, ) .with_context(|| format!("failed to configure decoder"))?; decoder .start() .with_context(|| format!("failed to start decoder"))?; Ok(decoder) } // Since we leak the ImageReader, and we pass frame_result_callback to it which contains a reference // to ClientCoreContext, to avoid circular references we need to use a Weak reference. fn decoder_lifecycle( config: VideoDecoderConfig, csd_0: Vec, frame_result_callback: Weak) + Send + Sync + 'static>, running: Arc, decoder_sink: Arc>>, decoder_ready_notifier: Arc, image_queue: Arc>>, image_reader: &mut ImageReader, ) -> Result<()> { // 2x: keep the target buffering in the middle of the max amount of queuable frames let available_buffering_frames = (2. * config.max_buffering_frames).ceil() as usize; image_reader.set_image_listener(Box::new({ let image_queue = Arc::clone(&image_queue); move |image_reader| { let mut image_queue_lock = image_queue.lock(); if image_queue_lock.len() > available_buffering_frames { warn!("Video frame queue overflow!"); image_queue_lock.pop_front(); } match image_reader.acquire_next_image() { Ok(AcquireResult::Image(image)) => { let timestamp = Duration::from_nanos(image.timestamp().unwrap() as u64); if let Some(callback) = frame_result_callback.upgrade() { callback(Ok(timestamp)); } image_queue_lock.push_back(QueuedImage { timestamp, image, in_use: false, }); } Ok(e) => { error!("ImageReader error: {e:?}"); image_queue_lock.pop_front(); } Err(e) => { error!("ImageReader error: {e}"); image_queue_lock.clear(); } } } }))?; // Documentation says that this call is necessary to properly dispose acquired buffers. // todo: find out how to use it and avoid leaking the ImageReader image_reader.set_buffer_removed_listener(Box::new(|_, _| ()))?; let mime = mime_for_codec(config.codec); let mut format = MediaFormat::new(); format.set_str("mime", mime); // Given https://github.com/alvr-org/ALVR/pull/1933#discussion_r1431902906 - change at own risk. // It might be harmless, it might not be, but it's definitely a risk. format.set_i32("width", 512); format.set_i32("height", 1024); format.set_buffer("csd-0", &csd_0); for (key, prop) in &config.options { let maybe_error = match prop.ty { MediacodecPropType::Float => prop .value .parse() .map(|value| format.set_f32(key, value)) .to_any(), MediacodecPropType::Int32 => prop .value .parse() .map(|value| format.set_i32(key, value)) .to_any(), MediacodecPropType::Int64 => prop .value .parse() .map(|value| format.set_i64(key, value)) .to_any(), MediacodecPropType::String => Ok(format.set_str(key, &prop.value)), }; if let Err(e) = maybe_error { error!("Failed to set property {key} to {}: {e}", prop.value); } } info!("Using AMediaCodec format:{} ", format); let decoder = if config.force_software_decoder { decoder_attempt_setup(config.codec, true, &format, &image_reader)? } else { // Hardware decoders sometimes fail at the CSD-0. // May as well fall back if this occurs. match decoder_attempt_setup(config.codec, false, &format, &image_reader) { Ok(d) => d, Err(e) => { // would be "warn!" but this is a severe caveat and a pretty major error. error!("Attempting software fallback due to error in default decoder: {e:#}"); decoder_attempt_setup(config.codec, true, &format, &image_reader)? } } }; let decoder = Arc::new(FakeThreadSafe(decoder)); { let mut decoder_lock = decoder_sink.lock(); *decoder_lock = Some(Arc::clone(&decoder)); decoder_ready_notifier.notify_one(); } let mut error_counter = 0; while running.value() { match decoder.dequeue_output_buffer(Duration::from_millis(1)) { Ok(DequeuedOutputBufferInfoResult::Buffer(buffer)) => { // The buffer timestamp is actually nanoseconds let presentation_time_ns = buffer.info().presentation_time_us(); if let Err(e) = decoder.release_output_buffer_at_time(buffer, presentation_time_ns) { error!("Decoder dequeue error: {e}"); } } Ok(DequeuedOutputBufferInfoResult::TryAgainLater) => continue, Ok(i) => info!("Decoder dequeue event: {i:?}"), Err(e) => { error!("Decoder dequeue error: {e}"); error_counter += 1; if error_counter > 10 { bail!("Too many decoder errors: {e}"); } // lessen logcat flood (just in case) thread::sleep(Duration::from_millis(50)); continue; } } error_counter = 0; } // Destroy all resources decoder_sink.lock().take(); // Make sure the shared ref is deleted first decoder.stop()?; drop(decoder); Ok(()) } // Create a sink/source pair pub fn video_decoder_split( config: VideoDecoderConfig, csd_0: Vec, frame_result_callback: impl Fn(Result) + Send + Sync + 'static, ) -> Result<(VideoDecoderSink, VideoDecoderSource)> { let running = Arc::new(RelaxedAtomic::new(true)); let decoder_sink = Arc::new(Mutex::new(None::)); let decoder_ready_notifier = Arc::new(Condvar::new()); let image_queue = Arc::new(Mutex::new(VecDeque::::new())); let dequeue_thread = thread::spawn({ let config = config.clone(); let running = Arc::clone(&running); let decoder_sink = Arc::clone(&decoder_sink); let decoder_ready_notifier = Arc::clone(&decoder_ready_notifier); let image_queue = Arc::clone(&image_queue); move || { const MAX_BUFFERING_FRAMES: usize = 10; let mut image_reader = match ImageReader::new_with_usage( 1, 1, ImageFormat::PRIVATE, HardwareBufferUsage::GPU_SAMPLED_IMAGE, MAX_BUFFERING_FRAMES as i32, ) { Ok(reader) => reader, Err(e) => { frame_result_callback(Err(anyhow!("{e}"))); return; } }; let frame_result_callback = Arc::new(frame_result_callback); if let Err(e) = decoder_lifecycle( config, csd_0, Arc::downgrade(&frame_result_callback), running, decoder_sink, decoder_ready_notifier, Arc::clone(&image_queue), &mut image_reader, ) { frame_result_callback(Err(e)); } image_queue.lock().clear(); error!("FIXME: Leaking Imagereader!"); Box::leak(Box::new(image_reader)); } }); // Make sure the decoder is ready: we don't want to try to enqueue frame and lose them, to avoid // image corruption. { let mut decoder_lock = decoder_sink.lock(); if decoder_lock.is_none() { // No spurious wakeups decoder_ready_notifier.wait(&mut decoder_lock); } } let sink = VideoDecoderSink { inner: decoder_sink, }; let source = VideoDecoderSource { running, dequeue_thread: Some(dequeue_thread), image_queue, config, buffering_running_average: 0.0, }; Ok((sink, source)) } ================================================ FILE: alvr/client_core/src/video_decoder/mod.rs ================================================ #[cfg(target_os = "android")] mod android; use alvr_common::anyhow::Result; use alvr_session::{CodecType, MediacodecProperty}; use std::time::Duration; #[derive(Clone, Default, PartialEq)] pub struct VideoDecoderConfig { pub codec: CodecType, pub force_software_decoder: bool, pub max_buffering_frames: f32, pub buffering_history_weight: f32, pub options: Vec<(String, MediacodecProperty)>, pub config_buffer: Vec, } pub struct VideoDecoderSink { #[cfg(target_os = "android")] inner: android::VideoDecoderSink, } impl VideoDecoderSink { // returns true if frame has been successfully enqueued #[allow(unused_variables)] pub fn push_nal(&mut self, timestamp: Duration, nal: &[u8]) -> bool { #[cfg(target_os = "android")] { alvr_common::show_err(self.inner.push_frame_nal(timestamp, nal)).unwrap_or(false) } #[cfg(not(target_os = "android"))] false } } pub struct VideoDecoderSource { #[cfg(target_os = "android")] inner: android::VideoDecoderSource, } impl VideoDecoderSource { /// If a frame is available, return the timestamp and the AHardwareBuffer. pub fn get_frame(&mut self) -> Option<(Duration, *mut std::ffi::c_void)> { #[cfg(target_os = "android")] { self.inner.dequeue_frame() } #[cfg(not(target_os = "android"))] None } } // report_frame_decoded: (target_timestamp: Duration) -> () #[allow(unused_variables)] pub fn create_decoder( config: VideoDecoderConfig, report_frame_decoded: impl Fn(Result) + Send + Sync + 'static, ) -> (VideoDecoderSink, VideoDecoderSource) { #[cfg(target_os = "android")] { let (sink, source) = android::video_decoder_split( config.clone(), config.config_buffer, report_frame_decoded, ) .unwrap(); ( VideoDecoderSink { inner: sink }, VideoDecoderSource { inner: source }, ) } #[cfg(not(target_os = "android"))] (VideoDecoderSink {}, VideoDecoderSource {}) } ================================================ FILE: alvr/client_mock/Cargo.toml ================================================ [package] name = "alvr_client_mock" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_client_core.workspace = true alvr_packets.workspace = true alvr_session.workspace = true alvr_system_info.workspace = true eframe = "0.32" env_logger = "0.11" rand = "0.9" ================================================ FILE: alvr/client_mock/src/main.rs ================================================ use alvr_client_core::{ClientCapabilities, ClientCoreContext, ClientCoreEvent}; use alvr_common::{ DeviceMotion, HEAD_ID, Pose, RelaxedAtomic, ViewParams, glam::{Quat, UVec2, Vec3}, parking_lot::RwLock, }; use alvr_packets::{FaceData, TrackingData}; use alvr_session::CodecType; use eframe::{ Frame, NativeOptions, egui::{CentralPanel, Context, RichText, Slider, ViewportBuilder}, }; use std::{ f32::consts::{FRAC_PI_2, PI}, sync::{ Arc, mpsc::{self, TryRecvError}, }, thread, time::{Duration, Instant}, }; #[derive(Clone, PartialEq)] struct WindowInput { height: f32, yaw: f32, pitch: f32, use_random_orientation: bool, emulated_decode_ms: u64, emulated_compositor_ms: u64, emulated_vsync_ms: u64, } impl Default for WindowInput { fn default() -> Self { Self { height: 1.5, yaw: 0.0, pitch: 0.0, use_random_orientation: true, emulated_decode_ms: 5, emulated_compositor_ms: 1, emulated_vsync_ms: 25, } } } #[derive(Clone)] struct WindowOutput { hud_message: String, connected: bool, fps: f32, resolution: UVec2, decoder_codec: Option, current_frame_timestamp: Duration, } impl Default for WindowOutput { fn default() -> Self { Self { hud_message: "".into(), connected: false, fps: 1.0, resolution: UVec2::ZERO, decoder_codec: None, current_frame_timestamp: Duration::ZERO, } } } pub struct Window { input: WindowInput, input_sender: mpsc::Sender, output: WindowOutput, output_receiver: mpsc::Receiver, } impl Window { fn new( input_sender: mpsc::Sender, output_receiver: mpsc::Receiver, ) -> Self { Self { input: WindowInput::default(), input_sender, output: WindowOutput::default(), output_receiver, } } } impl eframe::App for Window { fn update(&mut self, context: &Context, _: &mut Frame) { while let Ok(output) = self.output_receiver.try_recv() { self.output = output; } let mut input = self.input.clone(); CentralPanel::default().show(context, |ui| { ui.vertical_centered(|ui| { ui.heading(RichText::new(&self.output.hud_message)); }); ui.label(format!("Connected: {}", self.output.connected)); ui.label(format!("FPS: {}", self.output.fps)); ui.label(format!("View resolution: {}", self.output.resolution)); ui.label(format!("Codec: {:?}", self.output.decoder_codec)); ui.label(format!( "Current frame: {:?}", self.output.current_frame_timestamp )); ui.add_space(10.0); ui.horizontal(|ui| { ui.label("Height:"); ui.add(Slider::new(&mut input.height, 0.0..=2.0)); }); ui.horizontal(|ui| { ui.label("Yaw:"); ui.add(Slider::new(&mut input.yaw, -PI..=PI)); }); ui.horizontal(|ui| { ui.label("Pitch:"); ui.add(Slider::new(&mut input.pitch, -FRAC_PI_2..=FRAC_PI_2)); }); ui.checkbox( &mut input.use_random_orientation, "Use randomized orientation offset", ); }); if input != self.input { self.input = input; self.input_sender.send(self.input.clone()).ok(); } context.request_repaint(); } } fn tracking_thread( context: Arc, streaming: Arc, fps: f32, input: Arc>, ) { let timestamp_origin = Instant::now(); context.send_view_params([ViewParams::DUMMY; 2]); let mut loop_deadline = Instant::now(); while streaming.value() { let input_lock = input.read(); let mut orientation = Quat::from_rotation_y(input_lock.yaw) * Quat::from_rotation_x(input_lock.pitch); if input_lock.use_random_orientation { orientation *= Quat::from_rotation_z(rand::random::() * 0.001); } let position = Vec3::new(0.0, input_lock.height, 0.0); context.send_tracking(TrackingData { poll_timestamp: timestamp_origin.elapsed(), device_motions: vec![( *HEAD_ID, DeviceMotion { pose: Pose { orientation, position, }, linear_velocity: Vec3::ZERO, angular_velocity: Vec3::ZERO, }, )], hand_skeletons: [None, None], face: FaceData::default(), body: None, }); drop(input_lock); loop_deadline += Duration::from_secs_f32(1.0 / fps / 3.0); thread::sleep(loop_deadline.saturating_duration_since(Instant::now())) } } fn client_thread( output_sender: mpsc::Sender, input_receiver: mpsc::Receiver, ) { let capabilities = ClientCapabilities { platform: alvr_system_info::platform(None, None), default_view_resolution: UVec2::new(1920, 1832), max_view_resolution: UVec2::new(1920, 1832), refresh_rates: vec![60.0, 72.0, 80.0, 90.0, 120.0], foveated_encoding: false, encoder_high_profile: false, encoder_10_bits: false, encoder_av1: false, prefer_10bit: false, preferred_encoding_gamma: 1.0, prefer_hdr: false, }; let client_core_context = Arc::new(ClientCoreContext::new(capabilities)); client_core_context.resume(); let streaming = Arc::new(RelaxedAtomic::new(false)); let got_decoder_config = Arc::new(RelaxedAtomic::new(false)); let mut maybe_tracking_thread = None; let mut window_output = WindowOutput::default(); let window_input = Arc::new(RwLock::new(WindowInput::default())); let mut deadline = Instant::now(); 'main_loop: loop { let input_lock = window_input.read(); while let Some(event) = client_core_context.poll_event() { match event { ClientCoreEvent::UpdateHudMessage(message) => { window_output.hud_message = message; } ClientCoreEvent::StreamingStarted(config) => { window_output.fps = config.negotiated_config.refresh_rate_hint; window_output.connected = true; window_output.resolution = config.negotiated_config.view_resolution; streaming.set(true); let context = Arc::clone(&client_core_context); let streaming = Arc::clone(&streaming); let input = Arc::clone(&window_input); maybe_tracking_thread = Some(thread::spawn(move || { tracking_thread( context, streaming, config.negotiated_config.refresh_rate_hint, input, ) })); } ClientCoreEvent::StreamingStopped => { streaming.set(false); got_decoder_config.set(false); if let Some(thread) = maybe_tracking_thread.take() { thread.join().ok(); } window_output.fps = 1.0; window_output.connected = false; window_output.resolution = UVec2::ZERO; window_output.decoder_codec = None; } ClientCoreEvent::DecoderConfig { codec, .. } => { got_decoder_config.set(true); window_output.decoder_codec = Some(codec); } ClientCoreEvent::Haptics { .. } | ClientCoreEvent::RealTimeConfig(_) => (), } output_sender.send(window_output.clone()).ok(); } thread::sleep(Duration::from_millis(3)); client_core_context.report_compositor_start(window_output.current_frame_timestamp); thread::sleep(Duration::from_millis(input_lock.emulated_compositor_ms)); client_core_context.report_submit( window_output.current_frame_timestamp, Duration::from_millis(input_lock.emulated_vsync_ms), ); drop(input_lock); match input_receiver.try_recv() { Ok(input) => *window_input.write() = input, Err(TryRecvError::Disconnected) => break 'main_loop, Err(TryRecvError::Empty) => (), } deadline += Duration::from_secs_f32(1.0 / window_output.fps); thread::sleep(deadline.saturating_duration_since(Instant::now())); } streaming.set(false); if let Some(thread) = maybe_tracking_thread { thread.join().unwrap(); } client_core_context.pause() // client_core_context destroy is called here on drop } fn main() { env_logger::init(); let (input_sender, input_receiver) = mpsc::channel::(); let (output_sender, output_receiver) = mpsc::channel::(); let client_thread = thread::spawn(|| { client_thread(output_sender, input_receiver); }); eframe::run_native( "Mock client", NativeOptions { viewport: ViewportBuilder::default().with_inner_size((400.0, 400.0)), ..Default::default() }, Box::new(|_| Ok(Box::new(Window::new(input_sender, output_receiver)))), ) .ok(); client_thread.join().unwrap(); } ================================================ FILE: alvr/client_openxr/Cargo.toml ================================================ [package] name = "alvr_client_openxr" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [lib] crate-type = ["cdylib"] [dependencies] alvr_common.workspace = true alvr_client_core.workspace = true alvr_graphics.workspace = true alvr_packets.workspace = true alvr_session.workspace = true alvr_system_info.workspace = true openxr = { git = "https://github.com/zmerp/openxrs", rev = "e7c1b155e79ff8b58c2f6558d28e1398ebe08d2d" } [target.'cfg(target_os = "android")'.dependencies] android-activity = { version = "0.6", features = ["native-activity"] } jni = "0.21" libc = "0.2" ndk-context = "0.1" [package.metadata.android] package = "alvr.client.dev" # Changed for Meta Store install_location = "auto" build_targets = ["aarch64-linux-android"] runtime_libs = "../../deps/android_openxr" resources = "resources" [package.metadata.android.signing.release] path = "../../build/alvr_client_android/debug.keystore" keystore_password = "alvrclient" [package.metadata.android.signing.distribution] path = "../../build/alvr_client_android/debug.keystore" keystore_password = "alvrclient" [package.metadata.android.sdk] min_sdk_version = 28 target_sdk_version = 32 [[package.metadata.android.uses_feature]] name = "android.hardware.microphone" required = true [[package.metadata.android.uses_feature]] name = "android.hardware.vr.headtracking" required = true version = 1 [[package.metadata.android.uses_feature]] opengles_version = [3, 2] required = true [[package.metadata.android.uses_permission]] name = "android.permission.ACCESS_WIFI_STATE" [[package.metadata.android.uses_permission]] name = "android.permission.ACCESS_NETWORK_STATE" [[package.metadata.android.uses_permission]] name = "android.permission.CHANGE_WIFI_MULTICAST_STATE" [[package.metadata.android.uses_permission]] name = "android.permission.INTERNET" [[package.metadata.android.uses_permission]] name = "android.permission.RECORD_AUDIO" # WAKE_LOCK is needed for proper wifi locking # https://developer.android.com/reference/android/net/wifi/WifiManager.WifiLock [[package.metadata.android.uses_permission]] name = "android.permission.WAKE_LOCK" [[package.metadata.android.uses_permission]] name = "org.khronos.openxr.permission.OPENXR" [[package.metadata.android.uses_permission]] name = "org.khronos.openxr.permission.OPENXR_SYSTEM" [[package.metadata.android.queries.intent]] actions = ["org.khronos.openxr.OpenXRRuntimeService"] [[package.metadata.android.queries.provider]] name = "org.khronos.openxr" authorities = "org.khronos.openxr.runtime_broker;org.khronos.openxr.system_runtime_broker" [package.metadata.android.application] debuggable = false theme = "@android:style/Theme.Black.NoTitleBar.Fullscreen" icon = "@mipmap/ic_launcher" label = "ALVR" [package.metadata.android.application.activity] config_changes = "density|keyboard|keyboardHidden|navigation|orientation|screenLayout|screenSize|uiMode" launch_mode = "singleTask" orientation = "landscape" [[package.metadata.android.application.activity.intent_filter]] actions = ["android.intent.action.MAIN"] categories = [ "android.intent.category.LAUNCHER", "com.oculus.intent.category.VR", "com.yvr.intent.category.VR", "org.khronos.openxr.intent.category.IMMERSIVE_HMD", ] # Quest entries [[package.metadata.android.uses_feature]] name = "oculus.software.eye_tracking" required = false [[package.metadata.android.uses_feature]] name = "oculus.software.face_tracking" required = false [[package.metadata.android.uses_feature]] name = "oculus.software.handtracking" required = false [[package.metadata.android.uses_feature]] name = "com.oculus.feature.PASSTHROUGH" required = false [[package.metadata.android.uses_feature]] name = "com.oculus.software.body_tracking" required = false [[package.metadata.android.uses_permission]] name = "com.oculus.permission.BODY_TRACKING" [[package.metadata.android.uses_permission]] name = "com.oculus.permission.EYE_TRACKING" [[package.metadata.android.uses_permission]] name = "com.oculus.permission.FACE_TRACKING" [[package.metadata.android.uses_permission]] name = "com.oculus.permission.HAND_TRACKING" [[package.metadata.android.uses_permission]] name = "com.oculus.permission.WIFI_LOCK" [[package.metadata.android.application.meta_data]] name = "com.oculus.intent.category.VR" value = "vr_only" [[package.metadata.android.application.meta_data]] name = "com.oculus.supportedDevices" # Note: value is changed for the Meta store, which requires an explicit list of platforms. # "all" is required to support Quest 1 which doesn't have newer platform names registered. value = "all" [[package.metadata.android.application.meta_data]] name = "com.oculus.vr.focusaware" value = "true" [[package.metadata.android.application.meta_data]] name = "com.oculus.handtracking.frequency" value = "HIGH" [[package.metadata.android.application.meta_data]] name = "com.oculus.handtracking.version" value = "V2.0" # Vive entries [[package.metadata.android.uses_feature]] name = "wave.feature.eyetracking" required = false [[package.metadata.android.uses_feature]] name = "wave.feature.handtracking" required = true [[package.metadata.android.uses_feature]] name = "wave.feature.lipexpression" required = false [[package.metadata.android.application.meta_data]] name = "minWaveSDKVersion" value = "1" [[package.metadata.android.application.meta_data]] name = "com.htc.vr.content.NumController" value = "1,2" [[package.metadata.android.application.meta_data]] name = "com.htc.vr.content.NumDoFController" value = "3,6DoF" [[package.metadata.android.application.meta_data]] name = "com.htc.vr.content.NumDoFHmd" value = "3,6DoF" # Pico entries [[package.metadata.android.uses_permission]] name = "com.picovr.permission.EYE_TRACKING" [[package.metadata.android.uses_permission]] name = "com.picovr.permission.FACE_TRACKING" [[package.metadata.android.application.meta_data]] name = "eyetracking_calibration" value = "true" [[package.metadata.android.application.meta_data]] name = "handtracking" value = "1" [[package.metadata.android.application.meta_data]] name = "picovr.software.eye_tracking" value = "1" [[package.metadata.android.application.meta_data]] name = "picovr.software.face_tracking" value = "true" [[package.metadata.android.application.meta_data]] name = "pvr.app.type" value = "vr" [[package.metadata.android.application.meta_data]] name = "pvr.display.orientation" value = "180" [[package.metadata.android.application.meta_data]] name = "pvr.sdk.version" value = "OpenXR" [[package.metadata.android.application.meta_data]] name = "pxr.sdk.version_code" value = "5900" # Yvr entries [[package.metadata.android.application.meta_data]] name = "com.yvr.intent.category.VR" value = "vr_only" # Lynx entries [[package.metadata.android.queries.package]] name = "com.ultraleap.tracking.service" [[package.metadata.android.queries.package]] name = "com.ultraleap.openxr.api_layer" # Android XR [[package.metadata.android.uses_feature]] name = "android.software.xr.api.spatial" required = true [[package.metadata.android.uses_feature]] name = "android.software.xr.api.openxr" required = true [[package.metadata.android.uses_feature]] name = "android.software.xr.input.controller" required = false [[package.metadata.android.uses_feature]] name = "android.software.xr.input.hand_tracking" required = false [[package.metadata.android.uses_feature]] name = "android.software.xr.input.eye_tracking" required = false [[package.metadata.android.application.uses_native_library]] name = "libopenxr.google.so" required = false [[package.metadata.android.application.property]] name = "android.window.PROPERTY_XR_ACTIVITY_START_MODE" value = "XR_ACTIVITY_START_MODE_FULL_SPACE_UNMANAGED" [[package.metadata.android.application.property]] name = "android.window.PROPERTY_XR_BOUNDARY_TYPE_RECOMMENDED" value = "XR_BOUNDARY_TYPE_LARGE" ================================================ FILE: alvr/client_openxr/cbindgen.toml ================================================ language = "C" header = "/* ALVR is licensed under the MIT license. https://github.com/alvr-org/ALVR/blob/master/LICENSE */" pragma_once = true autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */" cpp_compat = true tab_width = 4 documentation_style = "c99" [enum] rename_variants = "QualifiedScreamingSnakeCase" ================================================ FILE: alvr/client_openxr/resources/drawable/ic_launcher_foreground.xml ================================================ ================================================ FILE: alvr/client_openxr/resources/mipmap-anydpi-v26/ic_launcher.xml ================================================ ================================================ FILE: alvr/client_openxr/resources/values/ic_launcher_background.xml ================================================ #FFFFFF ================================================ FILE: alvr/client_openxr/src/c_api.rs ================================================ #[cfg(target_os = "android")] #[unsafe(no_mangle)] pub extern "C" fn alvr_entry_point(java_vm: *mut std::ffi::c_void, context: *mut std::ffi::c_void) { unsafe { ndk_context::initialize_android_context(java_vm, context) }; crate::entry_point(); } ================================================ FILE: alvr/client_openxr/src/extra_extensions/body_tracking_bd.rs ================================================ use crate::extra_extensions::get_instance_proc; use openxr::{self as xr, AnyGraphics, sys}; use std::{ ffi::{CString, c_char, c_void}, ptr, sync::LazyLock, }; pub const BD_BODY_TRACKING_EXTENSION_NAME: &str = "XR_BD_body_tracking"; static TYPE_BODY_TRACKER_CREATE_INFO_BD: LazyLock = LazyLock::new(|| xr::StructureType::from_raw(1000385001)); static TYPE_BODY_JOINTS_LOCATE_INFO_BD: LazyLock = LazyLock::new(|| xr::StructureType::from_raw(1000385002)); static TYPE_BODY_JOINT_LOCATIONS_BD: LazyLock = LazyLock::new(|| xr::StructureType::from_raw(1000385003)); static TYPE_SYSTEM_BODY_TRACKING_PROPERTIES_BD: LazyLock = LazyLock::new(|| xr::StructureType::from_raw(1000385004)); pub const BODY_JOINT_COUNT_BD: usize = 24; #[repr(transparent)] #[derive(Debug, Copy, Clone, Eq, PartialEq, Hash)] pub struct XrBodyTrackerBD(u64); #[repr(transparent)] #[derive(Copy, Clone, Eq, PartialEq)] pub struct BodyJointSetBD(i32); impl BodyJointSetBD { pub const BODY_WITHOUT_ARM: BodyJointSetBD = Self(1i32); pub const FULL_BODY_JOINTS: BodyJointSetBD = Self(2i32); } #[repr(transparent)] #[derive(Copy, Clone, Eq, PartialEq)] pub struct BodyTrackingStatusCodeBD(i32); impl BodyTrackingStatusCodeBD { pub const INVALID: BodyTrackingStatusCodeBD = Self(0i32); } #[repr(transparent)] #[derive(Copy, Clone, Eq, PartialEq)] pub struct BodyTrackingErrorCodeBD(i32); impl BodyTrackingErrorCodeBD { pub const INNER_EXCEPTION: BodyTrackingErrorCodeBD = Self(0i32); pub const TRACKER_NOT_CALIBRATED: BodyTrackingErrorCodeBD = Self(1i32); } #[repr(transparent)] #[derive(Copy, Clone, Eq, PartialEq)] pub struct CalibAppFlagBD(i32); impl CalibAppFlagBD { pub const MOTION_TRACKER_2: CalibAppFlagBD = Self(1i32); } #[repr(C)] struct BodyTrackerCreateInfoBD { ty: xr::StructureType, next: *const c_void, joint_set: BodyJointSetBD, } #[repr(C)] struct BodyJointsLocateInfoBD { ty: xr::StructureType, next: *const c_void, base_space: sys::Space, time: sys::Time, } #[repr(C)] pub struct BodyJointLocationBD { pub location_flags: sys::SpaceLocationFlags, pub pose: sys::Posef, pub radius: f32, } #[repr(C)] struct BodyJointLocationsBD { ty: xr::StructureType, next: *const c_void, all_joint_poses_tracked: sys::Bool32, joint_count: u32, joint_locations: *mut BodyJointLocationBD, } #[repr(C)] struct SystemBodyTrackingPropertiesBD { ty: xr::StructureType, next: *const c_void, supports_body_tracking: sys::Bool32, } type CreateBodyTrackerBD = unsafe extern "system" fn( sys::Session, *const BodyTrackerCreateInfoBD, *mut XrBodyTrackerBD, ) -> sys::Result; type DestroyBodyTrackerBD = unsafe extern "system" fn(XrBodyTrackerBD) -> sys::Result; type LocateBodyJointsBD = unsafe extern "system" fn( XrBodyTrackerBD, *const BodyJointsLocateInfoBD, *mut BodyJointLocationsBD, ) -> sys::Result; type StartBodyTrackingCalibAppBD = unsafe extern "system" fn(sys::Instance, *const c_char, CalibAppFlagBD) -> sys::Result; type GetBodyTrackingStateBD = unsafe extern "system" fn( sys::Instance, *mut BodyTrackingStatusCodeBD, *mut BodyTrackingErrorCodeBD, ) -> sys::Result; pub struct BodyTrackerBD { handle: XrBodyTrackerBD, session: xr::Session, destroy_body_tracker: DestroyBodyTrackerBD, locate_body_joints: LocateBodyJointsBD, get_body_tracking_state: GetBodyTrackingStateBD, } impl BodyTrackerBD { pub fn new( session: xr::Session, joint_set: BodyJointSetBD, extra_extensions: &[String], system: xr::SystemId, prompt_calibration: bool, ) -> xr::Result { if !extra_extensions.contains(&BD_BODY_TRACKING_EXTENSION_NAME.to_owned()) { return Err(sys::Result::ERROR_EXTENSION_NOT_PRESENT); } let create_body_tracker: CreateBodyTrackerBD = get_instance_proc(&session, "xrCreateBodyTrackerBD")?; let start_body_tracking_calib_app: StartBodyTrackingCalibAppBD = get_instance_proc(&session, "xrStartBodyTrackingCalibAppBD")?; let get_body_tracking_state: GetBodyTrackingStateBD = get_instance_proc(&session, "xrGetBodyTrackingStateBD")?; let destroy_body_tracker = get_instance_proc(&session, "xrDestroyBodyTrackerBD")?; let locate_body_joints = get_instance_proc(&session, "xrLocateBodyJointsBD")?; let props = super::get_props( &session, system, SystemBodyTrackingPropertiesBD { ty: *TYPE_SYSTEM_BODY_TRACKING_PROPERTIES_BD, next: ptr::null(), supports_body_tracking: sys::FALSE, }, )?; if props.supports_body_tracking == sys::FALSE { return Err(sys::Result::ERROR_FEATURE_UNSUPPORTED); } let mut handle = XrBodyTrackerBD(0); let info = BodyTrackerCreateInfoBD { ty: *TYPE_BODY_TRACKER_CREATE_INFO_BD, next: ptr::null(), joint_set, }; unsafe { super::xr_res(create_body_tracker(session.as_raw(), &info, &mut handle))?; }; let mut status_code = BodyTrackingStatusCodeBD::INVALID; let mut error_code = BodyTrackingErrorCodeBD::INNER_EXCEPTION; if prompt_calibration { unsafe { super::xr_res(get_body_tracking_state( session.instance().as_raw(), &mut status_code, &mut error_code, ))?; // todo: include actual Android package name let package_name = CString::new("").unwrap(); if status_code == BodyTrackingStatusCodeBD::INVALID || error_code == BodyTrackingErrorCodeBD::TRACKER_NOT_CALIBRATED { super::xr_res(start_body_tracking_calib_app( session.instance().as_raw(), package_name.as_ptr(), CalibAppFlagBD::MOTION_TRACKER_2, ))?; } } } Ok(Self { handle, session: session.into_any_graphics(), destroy_body_tracker, locate_body_joints, get_body_tracking_state, }) } pub fn locate_body_joints( &self, time: xr::Time, reference_space: &xr::Space, ) -> xr::Result>> { let mut status_code = BodyTrackingStatusCodeBD::INVALID; let mut error_code = BodyTrackingErrorCodeBD::INNER_EXCEPTION; unsafe { super::xr_res((self.get_body_tracking_state)( self.session.instance().as_raw(), &mut status_code, &mut error_code, ))?; } if status_code == BodyTrackingStatusCodeBD::INVALID { return Ok(None); } let locate_info = BodyJointsLocateInfoBD { ty: *TYPE_BODY_JOINTS_LOCATE_INFO_BD, next: ptr::null(), base_space: reference_space.as_raw(), time, }; let joint_count = BODY_JOINT_COUNT_BD; let mut locations: Vec = Vec::with_capacity(joint_count); let mut location_info = BodyJointLocationsBD { ty: *TYPE_BODY_JOINT_LOCATIONS_BD, next: ptr::null(), all_joint_poses_tracked: sys::FALSE, joint_count: joint_count as u32, joint_locations: locations.as_mut_ptr(), }; unsafe { super::xr_res((self.locate_body_joints)( self.handle, &locate_info, &mut location_info, ))?; Ok(if location_info.all_joint_poses_tracked.into() { locations.set_len(joint_count); Some(locations) } else { None }) } } } impl Drop for BodyTrackerBD { fn drop(&mut self) { unsafe { (self.destroy_body_tracker)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/body_tracking_fb.rs ================================================ #![allow(dead_code)] use crate::extra_extensions::get_instance_proc; use openxr::{ self as xr, raw, sys::{self, Handle}, }; use std::{ptr, sync::LazyLock}; pub const META_BODY_TRACKING_FULL_BODY_EXTENSION_NAME: &str = "XR_META_body_tracking_full_body"; pub static BODY_JOINT_SET_FULL_BODY_META: LazyLock = LazyLock::new(|| xr::BodyJointSetFB::from_raw(1000274000)); pub const META_BODY_TRACKING_FIDELITY_EXTENSION_NAME: &str = "XR_META_body_tracking_fidelity"; pub static SYSTEM_PROPERTIES_BODY_TRACKING_FIDELITY_META: LazyLock = LazyLock::new(|| xr::StructureType::from_raw(1000284001)); pub const FULL_BODY_JOINT_COUNT_META: usize = 84; #[repr(C)] struct SystemPropertiesBodyTrackingFullBodyMETA { ty: xr::StructureType, next: *mut std::ffi::c_void, supports_full_body_tracking: sys::Bool32, } pub struct BodyTrackerFB { handle: sys::BodyTrackerFB, ext_fns: raw::BodyTrackingFB, } #[repr(C)] struct SystemPropertiesBodyTrackingFidelityMETA { ty: xr::StructureType, next: *mut std::ffi::c_void, supports_body_tracking_fidelity: sys::Bool32, } #[repr(C)] enum BodyTrackingFidelityMode { Low = 1, High = 2, } type RequestBodyTrackingFidelityMETA = unsafe extern "system" fn(sys::BodyTrackerFB, BodyTrackingFidelityMode) -> sys::Result; impl BodyTrackerFB { pub fn new( session: &xr::Session, system: xr::SystemId, body_joint_set: xr::BodyJointSetFB, prefer_high_fidelity: bool, ) -> xr::Result { let ext_fns = session .instance() .exts() .fb_body_tracking .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let mut handle = sys::BodyTrackerFB::NULL; let info = sys::BodyTrackerCreateInfoFB { ty: sys::BodyTrackerCreateInfoFB::TYPE, next: ptr::null(), body_joint_set, }; let body_tracking_fidelity_props = super::get_props( session, system, SystemPropertiesBodyTrackingFidelityMETA { ty: *SYSTEM_PROPERTIES_BODY_TRACKING_FIDELITY_META, next: ptr::null_mut(), supports_body_tracking_fidelity: sys::FALSE, }, )?; let preferred_fidelity_mode: BodyTrackingFidelityMode = if prefer_high_fidelity { BodyTrackingFidelityMode::High } else { BodyTrackingFidelityMode::Low }; unsafe { super::xr_res((ext_fns.create_body_tracker)( session.as_raw(), &info, &mut handle, ))?; if body_tracking_fidelity_props.supports_body_tracking_fidelity == sys::TRUE { let request_body_tracking_fidelity: RequestBodyTrackingFidelityMETA = get_instance_proc(session, "xrRequestBodyTrackingFidelityMETA")?; super::xr_res(request_body_tracking_fidelity( handle, preferred_fidelity_mode, )) .ok(); // This is very unlikely to fail as the void falls back to Low on an invalid call. } }; Ok(Self { handle, ext_fns }) } pub fn locate_body_joints( &self, time: xr::Time, reference_space: &xr::Space, joint_count: usize, ) -> xr::Result>> { let locate_info = sys::BodyJointsLocateInfoFB { ty: sys::BodyJointsLocateInfoFB::TYPE, next: ptr::null(), base_space: reference_space.as_raw(), time, }; let mut locations: Vec = Vec::with_capacity(joint_count); let mut location_info = sys::BodyJointLocationsFB { ty: sys::BodyJointLocationsFB::TYPE, next: ptr::null_mut(), is_active: sys::FALSE, confidence: 0.0, joint_count: joint_count as u32, joint_locations: locations.as_mut_ptr(), skeleton_changed_count: 0, time: xr::Time::from_nanos(0), }; unsafe { super::xr_res((self.ext_fns.locate_body_joints)( self.handle, &locate_info, &mut location_info, ))?; Ok(if location_info.is_active.into() { locations.set_len(joint_count); Some(locations) } else { None }) } } } impl Drop for BodyTrackerFB { fn drop(&mut self) { unsafe { (self.ext_fns.destroy_body_tracker)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/eye_gaze_interaction.rs ================================================ use openxr::{self as xr, sys}; use std::ptr; pub fn supports_eye_gaze_interaction(session: &xr::Session, system: xr::SystemId) -> bool { if session.instance().exts().ext_eye_gaze_interaction.is_none() { return false; } super::get_props( session, system, sys::SystemEyeGazeInteractionPropertiesEXT { ty: sys::SystemEyeGazeInteractionPropertiesEXT::TYPE, next: ptr::null_mut(), supports_eye_gaze_interaction: sys::FALSE, }, ) .map(|props| props.supports_eye_gaze_interaction.into()) .unwrap_or(false) } ================================================ FILE: alvr/client_openxr/src/extra_extensions/eye_tracking_social.rs ================================================ use openxr::{ self as xr, raw, sys::{self, Handle}, }; use std::ptr; pub struct EyeTrackerSocial { handle: sys::EyeTrackerFB, ext_fns: raw::EyeTrackingSocialFB, } impl EyeTrackerSocial { pub fn new(session: &xr::Session) -> xr::Result { let ext_fns = session .instance() .exts() .fb_eye_tracking_social .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let mut handle = sys::EyeTrackerFB::NULL; let info = sys::EyeTrackerCreateInfoFB { ty: sys::EyeTrackerCreateInfoFB::TYPE, next: ptr::null(), }; unsafe { super::xr_res((ext_fns.create_eye_tracker)( session.as_raw(), &info, &mut handle, ))? }; Ok(Self { handle, ext_fns }) } pub fn get_eye_gazes( &self, base: &xr::Space, time: xr::Time, ) -> xr::Result<[Option; 2]> { let gaze_info = sys::EyeGazesInfoFB { ty: sys::EyeGazesInfoFB::TYPE, next: ptr::null(), base_space: base.as_raw(), time, }; let mut eye_gazes = sys::EyeGazesFB::out(ptr::null_mut()); let eye_gazes = unsafe { super::xr_res((self.ext_fns.get_eye_gazes)( self.handle, &gaze_info, eye_gazes.as_mut_ptr(), ))?; eye_gazes.assume_init() }; let left_valid: bool = eye_gazes.gaze[0].is_valid.into(); let right_valid: bool = eye_gazes.gaze[1].is_valid.into(); Ok([ left_valid.then(|| eye_gazes.gaze[0].gaze_pose), right_valid.then(|| eye_gazes.gaze[1].gaze_pose), ]) } } impl Drop for EyeTrackerSocial { fn drop(&mut self) { unsafe { (self.ext_fns.destroy_eye_tracker)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/face_tracking2_fb.rs ================================================ use openxr::{ self as xr, raw, sys::{self, Handle}, }; use std::ptr; pub struct FaceTracker2FB { // Keeping a reference to the session to ensure that the tracker handle remains valid _session: xr::Session, handle: sys::FaceTracker2FB, ext_fns: raw::FaceTracking2FB, } impl FaceTracker2FB { pub fn new(session: xr::Session, visual: bool, audio: bool) -> xr::Result { let ext_fns = session .instance() .exts() .fb_face_tracking2 .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let mut requested_data_sources = vec![]; if visual { requested_data_sources.push(sys::FaceTrackingDataSource2FB::VISUAL); } if audio { requested_data_sources.push(sys::FaceTrackingDataSource2FB::AUDIO); } let mut handle = sys::FaceTracker2FB::NULL; let info = sys::FaceTrackerCreateInfo2FB { ty: sys::FaceTrackerCreateInfo2FB::TYPE, next: ptr::null(), face_expression_set: xr::FaceExpressionSet2FB::DEFAULT, requested_data_source_count: requested_data_sources.len() as u32, requested_data_sources: requested_data_sources.as_mut_ptr(), }; unsafe { super::xr_res((ext_fns.create_face_tracker2)( session.as_raw(), &info, &mut handle, ))? }; Ok(Self { _session: session.into_any_graphics(), handle, ext_fns, }) } pub fn get_face_expression_weights(&self, time: xr::Time) -> xr::Result>> { let expression_info = sys::FaceExpressionInfo2FB { ty: sys::FaceExpressionInfo2FB::TYPE, next: ptr::null(), time, }; let weights_count = xr::FaceExpression2FB::COUNT.into_raw() as usize; let confidence_count = xr::FaceConfidence2FB::COUNT.into_raw() as usize; let mut weights: Vec = Vec::with_capacity(weights_count); let mut confidences: Vec = vec![0.0; confidence_count]; let mut expression_weights = sys::FaceExpressionWeights2FB { ty: sys::FaceExpressionWeights2FB::TYPE, next: ptr::null_mut(), weight_count: weights_count as u32, weights: weights.as_mut_ptr(), confidence_count: confidence_count as u32, confidences: confidences.as_mut_ptr(), is_valid: sys::FALSE, is_eye_following_blendshapes_valid: sys::FALSE, data_source: sys::FaceTrackingDataSource2FB::from_raw(0), time: xr::Time::from_nanos(0), }; unsafe { super::xr_res((self.ext_fns.get_face_expression_weights2)( self.handle, &expression_info, &mut expression_weights, ))?; if expression_weights.is_valid.into() { weights.set_len(weights_count); Ok(Some(weights)) } else { Ok(None) } } } } impl Drop for FaceTracker2FB { fn drop(&mut self) { unsafe { (self.ext_fns.destroy_face_tracker2)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/face_tracking_pico.rs ================================================ use crate::extra_extensions::get_instance_proc; use openxr::{self as xr, sys}; const TRACKING_MODE_FACE_BIT: u64 = 0x00000008; const PICO_FACE_EXPRESSION_COUNT: usize = 52; #[repr(C)] struct FaceTrackingDataPICO { time: sys::Time, blend_shape_weight: [f32; 72], is_video_input_valid: [f32; 10], laughing_probability: f32, emotion_probability: [f32; 10], reserved: [f32; 128], } type StartEyeTrackingPICO = unsafe extern "system" fn(sys::Session) -> sys::Result; type StopEyeTrackingPICO = unsafe extern "system" fn(sys::Session, u64) -> sys::Result; type SetTrackingModePICO = unsafe extern "system" fn(sys::Session, u64) -> sys::Result; type GetFaceTrackingDataPICO = unsafe extern "system" fn( sys::Session, sys::Time, i32, *mut FaceTrackingDataPICO, ) -> sys::Result; pub struct FaceTrackerPico { session: xr::Session, start_eye_tracking: StartEyeTrackingPICO, stop_eye_tracking: StopEyeTrackingPICO, set_tracking_mode: SetTrackingModePICO, get_face_tracking_data: GetFaceTrackingDataPICO, } impl FaceTrackerPico { pub fn new(session: xr::Session) -> xr::Result { session .instance() .exts() .ext_eye_gaze_interaction .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let start_eye_tracking = get_instance_proc(&session, "xrStartEyeTrackingPICO")?; let stop_eye_tracking = get_instance_proc(&session, "xrStopEyeTrackingPICO")?; let set_tracking_mode = get_instance_proc(&session, "xrSetTrackingModePICO")?; let get_face_tracking_data = get_instance_proc(&session, "xrGetFaceTrackingDataPICO")?; Ok(Self { session: session.into_any_graphics(), start_eye_tracking, stop_eye_tracking, set_tracking_mode, get_face_tracking_data, }) } pub fn get_face_tracking_data(&self, time: xr::Time) -> xr::Result>> { let mut face_tracking_data = FaceTrackingDataPICO { time: xr::Time::from_nanos(0), blend_shape_weight: [0.0; 72], is_video_input_valid: [0.0; 10], laughing_probability: 0.0, emotion_probability: [0.0; 10], reserved: [0.0; 128], }; unsafe { super::xr_res((self.get_face_tracking_data)( self.session.as_raw(), time, 0, &mut face_tracking_data, ))?; if face_tracking_data.time.as_nanos() != 0 { let blend_shape_slice = face_tracking_data.blend_shape_weight[..PICO_FACE_EXPRESSION_COUNT].to_vec(); Ok(Some(blend_shape_slice)) } else { Ok(None) } } } pub fn start_face_tracking(&self) -> xr::Result<()> { unsafe { super::xr_res((self.start_eye_tracking)(self.session.as_raw()))?; super::xr_res((self.set_tracking_mode)( self.session.as_raw(), TRACKING_MODE_FACE_BIT, )) } } pub fn stop_face_tracking(&self) -> xr::Result<()> { unsafe { super::xr_res((self.stop_eye_tracking)( self.session.as_raw(), TRACKING_MODE_FACE_BIT, )) } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/facial_tracking_htc.rs ================================================ use openxr::{ self as xr, raw, sys::{self, Handle}, }; use std::ptr; pub struct FacialTrackerHTC { // Keeping a reference to the session to ensure that the tracker handle remains valid _session: xr::Session, handle: sys::FacialTrackerHTC, ext_fns: raw::FacialTrackingHTC, expression_count: usize, } impl FacialTrackerHTC { pub fn new( session: xr::Session, system: xr::SystemId, facial_tracking_type: xr::FacialTrackingTypeHTC, ) -> xr::Result { let ext_fns = session .instance() .exts() .htc_facial_tracking .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let props = super::get_props( &session, system, sys::SystemFacialTrackingPropertiesHTC { ty: sys::SystemFacialTrackingPropertiesHTC::TYPE, next: ptr::null_mut(), support_eye_facial_tracking: sys::FALSE, support_lip_facial_tracking: sys::FALSE, }, )?; let expression_count = if facial_tracking_type == sys::FacialTrackingTypeHTC::EYE_DEFAULT && props.support_eye_facial_tracking.into() { sys::FACIAL_EXPRESSION_EYE_COUNT_HTC } else if facial_tracking_type == sys::FacialTrackingTypeHTC::LIP_DEFAULT && props.support_lip_facial_tracking.into() { sys::FACIAL_EXPRESSION_LIP_COUNT_HTC } else { return Err(sys::Result::ERROR_FEATURE_UNSUPPORTED); }; let mut handle = sys::FacialTrackerHTC::NULL; let info = sys::FacialTrackerCreateInfoHTC { ty: sys::FacialTrackerCreateInfoHTC::TYPE, next: ptr::null(), facial_tracking_type, }; unsafe { super::xr_res((ext_fns.create_facial_tracker)( session.as_raw(), &info, &mut handle, ))? }; Ok(Self { _session: session.into_any_graphics(), handle, ext_fns, expression_count, }) } pub fn get_facial_expressions(&self, time: xr::Time) -> xr::Result>> { let mut weights = Vec::with_capacity(self.expression_count); let mut facial_expressions = sys::FacialExpressionsHTC { ty: sys::FacialExpressionsHTC::TYPE, next: ptr::null_mut(), is_active: sys::FALSE, sample_time: time, expression_count: self.expression_count as u32, expression_weightings: weights.as_mut_ptr(), }; unsafe { super::xr_res((self.ext_fns.get_facial_expressions)( self.handle, &mut facial_expressions, ))?; if facial_expressions.is_active.into() { weights.set_len(self.expression_count); Ok(Some(weights)) } else { Ok(None) } } } } impl Drop for FacialTrackerHTC { fn drop(&mut self) { unsafe { (self.ext_fns.destroy_facial_tracker)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/mod.rs ================================================ mod body_tracking_bd; mod body_tracking_fb; mod eye_gaze_interaction; mod eye_tracking_social; mod face_tracking2_fb; mod face_tracking_pico; mod facial_tracking_htc; mod motion_tracking_bd; mod multimodal_input; mod passthrough_fb; mod passthrough_htc; pub use body_tracking_bd::*; pub use body_tracking_fb::*; pub use eye_gaze_interaction::*; pub use eye_tracking_social::*; pub use face_tracking_pico::*; pub use face_tracking2_fb::*; pub use facial_tracking_htc::*; pub use motion_tracking_bd::*; pub use multimodal_input::*; pub use passthrough_fb::*; pub use passthrough_htc::*; use std::ffi::CString; use std::mem; use openxr::{self as xr, sys}; fn xr_res(result: sys::Result) -> xr::Result<()> { if result.into_raw() >= 0 { Ok(()) } else { Err(result) } } fn get_props( session: &xr::Session, system: xr::SystemId, default_struct: T, ) -> xr::Result { let instance = session.instance(); let mut props = default_struct; let mut system_properties = sys::SystemProperties::out((&raw mut props).cast()); let result = unsafe { (instance.fp().get_system_properties)( instance.as_raw(), system, system_properties.as_mut_ptr(), ) }; xr_res(result).map(|_| props) } fn get_instance_proc(session: &xr::Session, method_name: &str) -> xr::Result { unsafe { let method_name = CString::new(method_name).unwrap(); let mut function_handle = None; xr_res((session.instance().fp().get_instance_proc_addr)( session.instance().as_raw(), method_name.as_ptr(), &mut function_handle, ))?; function_handle .map(|pfn| mem::transmute_copy(&pfn)) .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT) } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/motion_tracking_bd.rs ================================================ use crate::extra_extensions::get_instance_proc; use openxr::{self as xr, AnyGraphics, sys}; use std::ffi::{CString, c_char}; pub const BD_MOTION_TRACKING_EXTENSION_NAME: &str = "XR_BD_motion_tracking"; pub const PICO_CONFIGURATION_EXTENSION_NAME: &str = "XR_PICO_configuration"; #[repr(C)] #[derive(Default)] struct MotionTrackerConnectStateBD { tracker_count: i32, serials: [MotionTrackerSerialBD; 6], } #[repr(C)] #[derive(Ord, Eq, PartialEq, PartialOrd, Default)] pub struct MotionTrackerSerialBD { pub serial: [u8; 24], } #[repr(transparent)] #[derive(Copy, Clone, Eq, PartialEq)] pub struct MotionTrackerConfidenceBD(i32); #[repr(C)] pub struct MotionTrackerLocationBD { pub pose: sys::Posef, pub angular_velocity: sys::Vector3f, pub linear_velocity: sys::Vector3f, pub angular_acceleration: sys::Vector3f, pub linear_acceleration: sys::Vector3f, } #[repr(C)] pub struct MotionTrackerLocationsBD { pub serial: MotionTrackerSerialBD, pub local_pose: MotionTrackerLocationBD, pub confidence: MotionTrackerConfidenceBD, pub global_pose: MotionTrackerLocationBD, } type GetMotionTrackerConnectStateBD = unsafe extern "system" fn(sys::Instance, *mut MotionTrackerConnectStateBD) -> sys::Result; type GetMotionTrackerLocationsBD = unsafe extern "system" fn( sys::Instance, sys::Time, *const MotionTrackerSerialBD, *mut MotionTrackerLocationsBD, ) -> sys::Result; type SetConfigPICO = unsafe extern "system" fn(sys::Session, i32, *const c_char) -> sys::Result; pub struct MotionTrackerBD { session: xr::Session, get_motion_tracker_connect_state: GetMotionTrackerConnectStateBD, get_motion_tracker_locations: GetMotionTrackerLocationsBD, } impl MotionTrackerBD { pub fn new(session: xr::Session, extra_extensions: &[String]) -> xr::Result { if !extra_extensions.contains(&BD_MOTION_TRACKING_EXTENSION_NAME.to_owned()) || !extra_extensions.contains(&PICO_CONFIGURATION_EXTENSION_NAME.to_owned()) { return Err(sys::Result::ERROR_EXTENSION_NOT_PRESENT); } let get_motion_tracker_connect_state = get_instance_proc(&session, "xrGetMotionTrackerConnectStateBD")?; let get_motion_tracker_locations = get_instance_proc(&session, "xrGetMotionTrackerLocationsBD")?; let set_config: SetConfigPICO = get_instance_proc(&session, "xrSetConfigPICO")?; unsafe { //Floor height tracking origin let str = CString::new("1").unwrap(); //Set config property for tracking origin super::xr_res(set_config(session.as_raw(), 1, str.as_ptr()))?; }; Ok(Self { session: session.into_any_graphics(), get_motion_tracker_connect_state, get_motion_tracker_locations, }) } pub fn locate_motion_trackers( &self, time: xr::Time, ) -> xr::Result>> { let mut locations = Vec::with_capacity(3); let mut connect_state = MotionTrackerConnectStateBD::default(); unsafe { super::xr_res((self.get_motion_tracker_connect_state)( self.session.instance().as_raw(), &mut connect_state, ))?; // Pico doesn't provide a way to connect more than three trackers now if connect_state.tracker_count > 3 { connect_state.tracker_count = 3 } for i in 0..connect_state.tracker_count as usize { let mut location = MotionTrackerLocationsBD { serial: MotionTrackerSerialBD { serial: [0; 24] }, local_pose: MotionTrackerLocationBD { pose: xr::Posef::IDENTITY, angular_velocity: Default::default(), linear_velocity: Default::default(), angular_acceleration: Default::default(), linear_acceleration: Default::default(), }, confidence: MotionTrackerConfidenceBD(0), global_pose: MotionTrackerLocationBD { pose: xr::Posef::IDENTITY, angular_velocity: Default::default(), linear_velocity: Default::default(), angular_acceleration: Default::default(), linear_acceleration: Default::default(), }, }; super::xr_res((self.get_motion_tracker_locations)( self.session.instance().as_raw(), time, &connect_state.serials[i], &mut location, ))?; locations.push(location); } } Ok(Some(locations)) } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/multimodal_input.rs ================================================ // Code taken from: // https://github.com/meta-quest/Meta-OpenXR-SDK/blob/main/OpenXR/meta_openxr_preview/meta_simultaneous_hands_and_controllers.h use crate::extra_extensions::get_instance_proc; use openxr::{ self as xr, sys::{self}, }; use std::{ffi::c_void, ptr, sync::LazyLock}; pub const META_SIMULTANEOUS_HANDS_AND_CONTROLLERS_EXTENSION_NAME: &str = "XR_META_simultaneous_hands_and_controllers"; pub const META_DETACHED_CONTROLLERS_EXTENSION_NAME: &str = "XR_META_detached_controllers"; static TYPE_SYSTEM_SIMULTANEOUS_HANDS_AND_CONTROLLERS_PROPERTIES_META: LazyLock = LazyLock::new(|| xr::StructureType::from_raw(1000532001)); static TYPE_SIMULTANEOUS_HANDS_AND_CONTROLLERS_TRACKING_RESUME_INFO_META: LazyLock< xr::StructureType, > = LazyLock::new(|| xr::StructureType::from_raw(1000532002)); static TYPE_SIMULTANEOUS_HANDS_AND_CONTROLLERS_TRACKING_PAUSE_INFO_META: LazyLock< xr::StructureType, > = LazyLock::new(|| xr::StructureType::from_raw(1000532003)); #[repr(C)] struct SystemSymultaneousHandsAndControllersPropertiesMETA { ty: xr::StructureType, next: *const c_void, supports_simultaneous_hands_and_controllers: sys::Bool32, } #[repr(C)] struct SimultaneousHandsAndControllersTrackingResumeInfoMETA { ty: xr::StructureType, next: *const c_void, } #[repr(C)] struct SimultaneousHandsAndControllersTrackingPauseInfoMETA { ty: xr::StructureType, next: *const c_void, } type ResumeSimultaneousHandsAndControllersTrackingMETA = unsafe extern "system" fn( sys::Session, *const SimultaneousHandsAndControllersTrackingResumeInfoMETA, ) -> sys::Result; type PauseSimultaneousHandsAndControllersTrackingMETA = unsafe extern "system" fn( sys::Session, *const SimultaneousHandsAndControllersTrackingPauseInfoMETA, ) -> sys::Result; pub struct MultimodalMeta { session: xr::Session, resume_simultaneous_hands_and_controllers_tracking_meta: ResumeSimultaneousHandsAndControllersTrackingMETA, pause_simultaneous_hands_and_controllers_tracking_meta: PauseSimultaneousHandsAndControllersTrackingMETA, } impl MultimodalMeta { pub fn new( session: xr::Session, extra_extensions: &[String], system: xr::SystemId, ) -> xr::Result { if !extra_extensions .contains(&META_SIMULTANEOUS_HANDS_AND_CONTROLLERS_EXTENSION_NAME.to_owned()) || !extra_extensions.contains(&META_DETACHED_CONTROLLERS_EXTENSION_NAME.to_owned()) { return Err(sys::Result::ERROR_EXTENSION_NOT_PRESENT); } let resume_simultaneous_hands_and_controllers_tracking_meta = get_instance_proc( &session, "xrResumeSimultaneousHandsAndControllersTrackingMETA", )?; let pause_simultaneous_hands_and_controllers_tracking_meta = get_instance_proc( &session, "xrPauseSimultaneousHandsAndControllersTrackingMETA", )?; let props = super::get_props( &session, system, SystemSymultaneousHandsAndControllersPropertiesMETA { ty: *TYPE_SYSTEM_SIMULTANEOUS_HANDS_AND_CONTROLLERS_PROPERTIES_META, next: ptr::null(), supports_simultaneous_hands_and_controllers: xr::sys::FALSE, }, )?; if props.supports_simultaneous_hands_and_controllers.into() { Ok(Self { session: session.into_any_graphics(), resume_simultaneous_hands_and_controllers_tracking_meta, pause_simultaneous_hands_and_controllers_tracking_meta, }) } else { Err(sys::Result::ERROR_FEATURE_UNSUPPORTED) } } pub fn resume(&self) -> xr::Result<()> { let resume_info = SimultaneousHandsAndControllersTrackingResumeInfoMETA { ty: *TYPE_SIMULTANEOUS_HANDS_AND_CONTROLLERS_TRACKING_RESUME_INFO_META, next: ptr::null(), }; unsafe { super::xr_res((self .resume_simultaneous_hands_and_controllers_tracking_meta)( self.session.as_raw(), &resume_info, )) } } pub fn pause(&self) -> xr::Result<()> { let pause_info = SimultaneousHandsAndControllersTrackingPauseInfoMETA { ty: *TYPE_SIMULTANEOUS_HANDS_AND_CONTROLLERS_TRACKING_PAUSE_INFO_META, next: ptr::null(), }; unsafe { super::xr_res((self .pause_simultaneous_hands_and_controllers_tracking_meta)( self.session.as_raw(), &pause_info, )) } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/passthrough_fb.rs ================================================ use alvr_system_info::Platform; use openxr::{ self as xr, raw, sys::{self, Handle}, }; use std::ptr; pub struct PassthroughFB { handle: sys::PassthroughFB, layer_handle: sys::PassthroughLayerFB, layer: sys::CompositionLayerPassthroughFB, ext_fns: raw::PassthroughFB, } impl PassthroughFB { pub fn new(session: &xr::Session, platform: Platform) -> xr::Result { let ext_fns = session .instance() .exts() .fb_passthrough .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let mut handle = sys::PassthroughFB::NULL; let info = sys::PassthroughCreateInfoFB { ty: sys::PassthroughCreateInfoFB::TYPE, next: ptr::null(), flags: sys::PassthroughFlagsFB::IS_RUNNING_AT_CREATION, }; unsafe { super::xr_res((ext_fns.create_passthrough)( session.as_raw(), &info, &mut handle, ))? }; let mut layer_handle = sys::PassthroughLayerFB::NULL; let info = sys::PassthroughLayerCreateInfoFB { ty: sys::PassthroughLayerCreateInfoFB::TYPE, next: ptr::null(), passthrough: handle, flags: sys::PassthroughFlagsFB::IS_RUNNING_AT_CREATION, purpose: sys::PassthroughLayerPurposeFB::RECONSTRUCTION, }; unsafe { super::xr_res((ext_fns.create_passthrough_layer)( session.as_raw(), &info, &mut layer_handle, ))? }; let layer = sys::CompositionLayerPassthroughFB { ty: sys::CompositionLayerPassthroughFB::TYPE, next: ptr::null(), flags: xr::CompositionLayerFlags::BLEND_TEXTURE_SOURCE_ALPHA, space: sys::Space::NULL, layer_handle, }; // HACK: YVR runtime seems to ignore IS_RUNNING_AT_CREATION on versions <= 3.0.1 if platform.is_yvr() { unsafe { super::xr_res((ext_fns.passthrough_start)(handle))? }; } Ok(Self { handle, layer_handle, layer, ext_fns, }) } // return reference to make sure the passthrough handle is not dropped while the layer is in use pub fn layer(&self) -> &sys::CompositionLayerPassthroughFB { &self.layer } } impl Drop for PassthroughFB { fn drop(&mut self) { unsafe { (self.ext_fns.destroy_passthrough_layer)(self.layer_handle); (self.ext_fns.destroy_passthrough)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/extra_extensions/passthrough_htc.rs ================================================ use openxr::{ self as xr, raw, sys::{self, Handle}, }; use std::ptr; pub struct PassthroughHTC { handle: sys::PassthroughHTC, layer: sys::CompositionLayerPassthroughHTC, ext_fns: raw::PassthroughHTC, } impl PassthroughHTC { pub fn new(session: &xr::Session) -> xr::Result { let ext_fns = session .instance() .exts() .htc_passthrough .ok_or(sys::Result::ERROR_EXTENSION_NOT_PRESENT)?; let mut handle = sys::PassthroughHTC::NULL; let info = sys::PassthroughCreateInfoHTC { ty: sys::PassthroughCreateInfoHTC::TYPE, next: ptr::null(), form: sys::PassthroughFormHTC::PLANAR, }; unsafe { super::xr_res((ext_fns.create_passthrough)( session.as_raw(), &info, &mut handle, ))? }; let layer = sys::CompositionLayerPassthroughHTC { ty: sys::CompositionLayerPassthroughHTC::TYPE, next: ptr::null(), layer_flags: xr::CompositionLayerFlags::EMPTY, space: sys::Space::NULL, passthrough: handle, color: sys::PassthroughColorHTC { ty: sys::PassthroughColorHTC::TYPE, next: ptr::null(), alpha: 1.0, }, }; Ok(Self { handle, layer, ext_fns, }) } // return reference to make sure the passthrough handle is not dropped while the layer is in use pub fn layer(&self) -> &sys::CompositionLayerPassthroughHTC { &self.layer } } impl Drop for PassthroughHTC { fn drop(&mut self) { unsafe { (self.ext_fns.destroy_passthrough)(self.handle); } } } ================================================ FILE: alvr/client_openxr/src/graphics.rs ================================================ use alvr_common::glam::UVec2; use alvr_graphics::GraphicsContext; use alvr_session::ClientsidePostProcessingConfig; use openxr as xr; use std::ptr; #[allow(unused)] pub fn session_create_info(ctx: &GraphicsContext) -> xr::opengles::SessionCreateInfo { #[cfg(target_os = "android")] { xr::opengles::SessionCreateInfo::Android { display: ctx.egl_display.as_ptr(), config: ctx.egl_config.as_ptr(), context: ctx.egl_context.as_ptr(), } } #[cfg(not(target_os = "android"))] unimplemented!() } pub fn swapchain_format( gfx_ctx: &GraphicsContext, session: &xr::Session, enable_hdr: bool, ) -> u32 { gfx_ctx.make_current(); let formats = session.enumerate_swapchain_formats().unwrap(); alvr_graphics::choose_swapchain_format(&formats, enable_hdr) } #[allow(unused_variables)] pub fn create_swapchain( session: &xr::Session, gfx_ctx: &GraphicsContext, resolution: UVec2, format: u32, foveation: Option<&xr::FoveationProfileFB>, ) -> xr::Swapchain { gfx_ctx.make_current(); let swapchain_info = xr::SwapchainCreateInfo { create_flags: xr::SwapchainCreateFlags::EMPTY, usage_flags: xr::SwapchainUsageFlags::COLOR_ATTACHMENT | xr::SwapchainUsageFlags::SAMPLED, format, sample_count: 1, width: resolution.x, height: resolution.y, face_count: 1, array_size: 1, mip_count: 1, }; session.create_swapchain(&swapchain_info).unwrap() } pub struct ProjectionLayerAlphaConfig { pub premultiplied: bool, } // This is needed to work around lifetime limitations. Deref cannot be implemented because there are // nested references, and in a way or the other I would get `cannot return reference to temporary // value` pub struct ProjectionLayerBuilder<'a> { reference_space: &'a xr::Space, layers: [xr::CompositionLayerProjectionView<'a, xr::OpenGlEs>; 2], alpha: Option, composition_layer_settings: Option, } impl<'a> ProjectionLayerBuilder<'a> { pub fn new( reference_space: &'a xr::Space, layers: [xr::CompositionLayerProjectionView<'a, xr::OpenGlEs>; 2], alpha: Option, clientside_post_processing_config: Option, ) -> Self { let composition_layer_settings = clientside_post_processing_config .map(|post_processing| { xr::CompositionLayerSettingsFlagsFB::from_raw( (post_processing.sharpening as u64) | (post_processing.super_sampling as u64), ) }) .filter(|&flags| flags != xr::CompositionLayerSettingsFlagsFB::EMPTY) .map(|flags| xr::sys::CompositionLayerSettingsFB { ty: xr::StructureType::COMPOSITION_LAYER_SETTINGS_FB, next: std::ptr::null(), layer_flags: flags, }); Self { reference_space, layers, alpha, composition_layer_settings, } } pub fn build(&self) -> xr::CompositionLayerProjection<'_, xr::OpenGlEs> { let mut flags = xr::CompositionLayerFlags::EMPTY; if let Some(alpha) = &self.alpha { flags |= xr::CompositionLayerFlags::BLEND_TEXTURE_SOURCE_ALPHA; if !alpha.premultiplied { flags |= xr::CompositionLayerFlags::UNPREMULTIPLIED_ALPHA; } } let layer = xr::CompositionLayerProjection::new() .layer_flags(flags) .space(self.reference_space) .views(&self.layers); if let Some(composition_layer_settings) = self.composition_layer_settings.as_ref() { unsafe { xr::CompositionLayerProjection::from_raw(xr::sys::CompositionLayerProjection { next: ptr::from_ref(composition_layer_settings).cast(), ..layer.into_raw() }) } } else { layer } } } ================================================ FILE: alvr/client_openxr/src/interaction.rs ================================================ use crate::{ Platform, extra_extensions::{ self, BODY_JOINT_SET_FULL_BODY_META, BodyJointSetBD, BodyTrackerBD, BodyTrackerFB, EyeTrackerSocial, FULL_BODY_JOINT_COUNT_META, FaceTracker2FB, FaceTrackerPico, FacialTrackerHTC, MotionTrackerBD, MultimodalMeta, }, }; use alvr_common::{ glam::{Quat, Vec3}, *, }; use alvr_graphics::HandData; use alvr_packets::{ButtonEntry, ButtonValue, FaceData, FaceExpressions, StreamConfig}; use alvr_session::{BodyTrackingBDConfig, BodyTrackingSourcesConfig, FaceTrackingSourcesConfig}; use openxr as xr; use std::{ collections::{HashMap, HashSet}, time::Duration, }; use xr::SpaceLocationFlags; const IPD_CHANGE_EPS: f32 = 0.001; // Most OpenXR runtime, including Meta's one, do not follow perfectly the specification regarding // controller pose. The Z axis should point down through the center of the controller grip, the X // axis should go out perpendicular from the palm, and the position should be aligned roughtly with // the center of the palm. // https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html // Note: right controller offsets are calculated from left controller offsets by mirroring along the // Y-Z plane. fn get_controller_offset(platform: Platform, is_right_hand: bool) -> Pose { const DEG_TO_RAD: f32 = std::f32::consts::PI / 180.0; let left_offset = match platform { Platform::Quest1 => Pose { position: Vec3::new(-0.013, -0.005, 0.0), orientation: Quat::from_rotation_x(-20.0 * DEG_TO_RAD), }, // todo: check Quest 2 p if p.is_quest() => Pose { position: Vec3::new(-0.005, -0.005, 0.00), orientation: Quat::from_rotation_x(-15.0 * DEG_TO_RAD), }, Platform::PicoNeo3 => Pose { position: Vec3::new(-0.013, -0.035, 0.0), orientation: Quat::IDENTITY, }, // todo: check (base) Pico 4 p if p.is_pico() => Pose { position: Vec3::new(-0.01, -0.035, 0.0), orientation: Quat::from_rotation_y(6.0 * DEG_TO_RAD) * Quat::from_rotation_x(-6.0 * DEG_TO_RAD), }, p if p.is_vive() => Pose { position: Vec3::new(0.0, 0.0, -0.02), orientation: Quat::IDENTITY, }, Platform::SamsungGalaxyXR => Pose { position: Vec3::new(0.0, 0.0, 0.055), orientation: Quat::IDENTITY, }, _ => Pose::IDENTITY, }; if is_right_hand { let p = left_offset.position; let q = left_offset.orientation; Pose { position: Vec3::new(-p[0], p[1], p[2]), orientation: Quat::from_xyzw(-q.x, q.y, q.z, -q.w), } } else { left_offset } } fn check_ext_object(name: &str, result: xr::Result) -> Option { match result { Ok(obj) => Some(obj), Err(xr::sys::Result::ERROR_FEATURE_UNSUPPORTED) => { warn!("Cannot create unsupported {name}"); None } Err(xr::sys::Result::ERROR_EXTENSION_NOT_PRESENT) => None, Err(e) => { warn!("Failed to create {name}: {e}"); None } } } pub enum ButtonAction { Binary(xr::Action), Scalar(xr::Action), } pub struct HandInteraction { pub controllers_profile_id: u64, pub input_ids: HashSet, pub pose_offset: Pose, pub grip_action: xr::Action, pub grip_space: xr::Space, #[expect(dead_code)] pub aim_action: xr::Action, #[expect(dead_code)] pub aim_space: xr::Space, pub detached_grip_action: Option>, pub detached_grip_space: Option, pub vibration_action: xr::Action, pub skeleton_tracker: Option, } pub enum FaceExpressionsTracker { Fb(FaceTracker2FB), Pico(FaceTrackerPico), Htc { eye: Option, lip: Option, }, } pub struct FaceSources { eyes_combined: Option<(xr::Action, xr::Space)>, eyes_social: Option, face_expressions_tracker: Option, } pub enum BodyTracker { Fb { tracker: BodyTrackerFB, joint_count: usize, }, BodyBD(BodyTrackerBD), MotionBD(MotionTrackerBD), } #[derive(Clone)] pub struct InteractionSourcesConfig { pub face_tracking: Option, pub body_tracking: Option, pub prefers_multimodal_input: bool, } impl InteractionSourcesConfig { pub fn new(config: &StreamConfig) -> Self { Self { face_tracking: config .settings .headset .face_tracking .as_option() .map(|c| c.sources.clone()), body_tracking: config .settings .headset .body_tracking .as_option() .map(|c| c.sources.clone()), prefers_multimodal_input: config .settings .headset .multimodal_tracking .as_option() .is_some_and(|c| c.enabled), } } } pub struct InteractionContext { xr_session: xr::Session, xr_system: xr::SystemId, extra_extensions: Vec, platform: Platform, pub action_set: xr::ActionSet, pub button_actions: HashMap, pub hands_interaction: [HandInteraction; 2], multimodal_handle: Option, pub multimodal_hands_enabled: bool, pub face_sources: FaceSources, pub body_source: Option, } impl InteractionContext { pub fn new( xr_session: xr::Session, extra_extensions: Vec, xr_system: xr::SystemId, platform: Platform, ) -> Self { let xr_instance = xr_session.instance(); let action_set = xr_instance .create_action_set("alvr_interaction", "ALVR interaction", 0) .unwrap(); let mut bindings = vec![]; fn binding<'a, T: xr::ActionTy>(action: &'a xr::Action, path: &str) -> xr::Binding<'a> { xr::Binding::new(action, action.instance().string_to_path(path).unwrap()) } let controllers_profile_path = match platform { p if p.is_quest() => QUEST_CONTROLLER_PROFILE_PATH, // todo: create new controller profile for quest pro and 3 Platform::PicoG3 => PICO_G3_CONTROLLER_PROFILE_PATH, Platform::PicoNeo3 => PICO_NEO3_CONTROLLER_PROFILE_PATH, Platform::Pico4Ultra => PICO4S_CONTROLLER_PROFILE_PATH, Platform::Pico4 | Platform::Pico4Pro | Platform::Pico4Enterprise => { PICO4_CONTROLLER_PROFILE_PATH } p if p.is_pico() => PICO4S_CONTROLLER_PROFILE_PATH, p if p.is_vive() => FOCUS3_CONTROLLER_PROFILE_PATH, p if p.is_yvr() => YVR_CONTROLLER_PROFILE_PATH, _ => QUEST_CONTROLLER_PROFILE_PATH, }; let controllers_profile_id = alvr_common::hash_string(controllers_profile_path); // Create actions: let mut button_actions = HashMap::new(); let button_set = CONTROLLER_PROFILE_INFO .get(&controllers_profile_id) .unwrap() .button_set .clone(); for button_id in &button_set { let info = BUTTON_INFO.get(button_id).unwrap(); let name = info.path[1..].replace('/', "_"); let display_name = format!( "{}{}", name[0..1].to_uppercase(), name[1..].replace('_', " ") ); let action = match info.button_type { ButtonType::Binary => ButtonAction::Binary( action_set.create_action(&name, &display_name, &[]).unwrap(), ), ButtonType::Scalar => ButtonAction::Scalar( action_set.create_action(&name, &display_name, &[]).unwrap(), ), }; button_actions.insert(*button_id, action); } let left_grip_action = action_set .create_action("left_grip_pose", "Left grip pose", &[]) .unwrap(); let right_grip_action = action_set .create_action("right_grip_pose", "Right grip pose", &[]) .unwrap(); let left_aim_action = action_set .create_action("left_aim_pose", "Left aim pose", &[]) .unwrap(); let right_aim_action = action_set .create_action("right_aim_pose", "Right aim pose", &[]) .unwrap(); let left_vibration_action = action_set .create_action("left_hand_vibration", "Left hand vibration", &[]) .unwrap(); let right_vibration_action = action_set .create_action("right_hand_vibration", "Right hand vibration", &[]) .unwrap(); // Create action bindings: for (id, action) in &button_actions { let path = &BUTTON_INFO.get(id).unwrap().path; match action { ButtonAction::Binary(action) => { bindings.push(binding(action, path)); } ButtonAction::Scalar(action) => { bindings.push(binding(action, path)); } } } bindings.push(binding( &left_grip_action, "/user/hand/left/input/grip/pose", )); bindings.push(binding( &right_grip_action, "/user/hand/right/input/grip/pose", )); bindings.push(binding(&left_aim_action, "/user/hand/left/input/aim/pose")); bindings.push(binding( &right_aim_action, "/user/hand/right/input/aim/pose", )); bindings.push(binding( &left_vibration_action, "/user/hand/left/output/haptic", )); bindings.push(binding( &right_vibration_action, "/user/hand/right/output/haptic", )); let multimodal_handle = check_ext_object( "MultimodalMeta", MultimodalMeta::new(xr_session.clone(), &extra_extensions, xr_system), ); let mut left_detached_grip_action = None; let mut right_detached_grip_action = None; if multimodal_handle.is_some() { // Note: when multimodal input is enabled, both controllers and hands will always be // active. Held controllers and detached controllers are sent to the server as separate // devices. let left_detached_grip_action = left_detached_grip_action.insert( action_set .create_action::( "left_detached_grip_pose", "Left detached grip pose", &[], ) .unwrap(), ); let right_detached_grip_action = right_detached_grip_action.insert( action_set .create_action::( "right_detached_grip_pose", "Right detached grip pose", &[], ) .unwrap(), ); bindings.push(binding( left_detached_grip_action, "/user/detached_controller_meta/left/input/grip/pose", )); bindings.push(binding( right_detached_grip_action, "/user/detached_controller_meta/right/input/grip/pose", )); } // Apply bindings: xr_instance .suggest_interaction_profile_bindings( xr_instance .string_to_path(controllers_profile_path) .unwrap(), &bindings, ) .unwrap(); let left_grip_space = left_grip_action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap(); let right_grip_space = right_grip_action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap(); let left_aim_space = left_aim_action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap(); let right_aim_space = right_aim_action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap(); let left_detached_grip_space = left_detached_grip_action.as_ref().map(|action| { action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap() }); let right_detached_grip_space = right_detached_grip_action.as_ref().map(|action| { action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap() }); let left_hand_tracker = check_ext_object( "HandTracker (left)", xr_session.create_hand_tracker(xr::Hand::LEFT), ); let right_hand_tracker = check_ext_object( "HandTracker (right)", xr_session.create_hand_tracker(xr::Hand::RIGHT), ); let eyes_combined = if extra_extensions::supports_eye_gaze_interaction(&xr_session, xr_system) { if matches!(platform, Platform::QuestPro) { #[cfg(target_os = "android")] alvr_system_info::try_get_permission("com.oculus.permission.EYE_TRACKING"); } else if matches!( platform, Platform::PicoNeo3 | Platform::Pico4Pro | Platform::Pico4Enterprise ) { #[cfg(target_os = "android")] alvr_system_info::try_get_permission("com.picovr.permission.EYE_TRACKING"); } let action = action_set .create_action("combined_eye_gaze", "Combined eye gaze", &[]) .unwrap(); let res = xr_instance.suggest_interaction_profile_bindings( xr_instance .string_to_path("/interaction_profiles/ext/eye_gaze_interaction") .unwrap(), &[binding(&action, "/user/eyes_ext/input/gaze_ext/pose")], ); if res.is_err() { warn!("Failed to register combined eye gaze input: {res:?}"); } let space = action .create_space(&xr_session, xr::Path::NULL, xr::Posef::IDENTITY) .unwrap(); Some((action, space)) } else { None }; // Note: HTC facial tracking can only be created at startup before xrBeginSession. We don't // know the reason. let face_expressions_tracker = if platform.is_vive() { let eye = check_ext_object( "FacialTrackerHTC (eyes)", FacialTrackerHTC::new( xr_session.clone(), xr_system, xr::FacialTrackingTypeHTC::EYE_DEFAULT, ), ); let lip = check_ext_object( "FacialTrackerHTC (lips)", FacialTrackerHTC::new( xr_session.clone(), xr_system, xr::FacialTrackingTypeHTC::LIP_DEFAULT, ), ); Some(FaceExpressionsTracker::Htc { eye, lip }) } else { None }; xr_session.attach_action_sets(&[&action_set]).unwrap(); Self { xr_session, xr_system, extra_extensions, platform, action_set, button_actions, hands_interaction: [ HandInteraction { controllers_profile_id, input_ids: button_set.clone(), pose_offset: get_controller_offset(platform, false), grip_action: left_grip_action, grip_space: left_grip_space, aim_action: left_aim_action, aim_space: left_aim_space, detached_grip_action: left_detached_grip_action, detached_grip_space: left_detached_grip_space, vibration_action: left_vibration_action, skeleton_tracker: left_hand_tracker, }, HandInteraction { controllers_profile_id, input_ids: button_set, pose_offset: get_controller_offset(platform, true), grip_action: right_grip_action, grip_space: right_grip_space, aim_action: right_aim_action, aim_space: right_aim_space, detached_grip_action: right_detached_grip_action, detached_grip_space: right_detached_grip_space, vibration_action: right_vibration_action, skeleton_tracker: right_hand_tracker, }, ], multimodal_handle, multimodal_hands_enabled: false, face_sources: FaceSources { eyes_combined, eyes_social: None, face_expressions_tracker, }, body_source: None, } } pub fn select_sources(&mut self, config: &InteractionSourcesConfig) { // First of all, disable/delete all sources. This ensures there are no conflicts if let Some(handle) = &mut self.multimodal_handle { handle.pause().ok(); } if let Some(FaceExpressionsTracker::Pico(tracker)) = &self.face_sources.face_expressions_tracker { tracker.stop_face_tracking().ok(); } self.multimodal_hands_enabled = false; self.face_sources.eyes_social = None; // HTC trackers must not be destroyed or the app will crash if !matches!( self.face_sources.face_expressions_tracker, Some(FaceExpressionsTracker::Htc { .. }) ) { self.face_sources.face_expressions_tracker = None; } self.body_source = None; if let Some(config) = &config.face_tracking { if matches!(self.platform, Platform::QuestPro) && matches!(config, FaceTrackingSourcesConfig::PreferFullFaceTracking) { #[cfg(target_os = "android")] { alvr_system_info::try_get_permission("android.permission.RECORD_AUDIO"); alvr_system_info::try_get_permission("com.oculus.permission.FACE_TRACKING") } } if matches!( self.platform, Platform::PicoNeo3 | Platform::Pico4Pro | Platform::Pico4Enterprise ) && matches!(config, FaceTrackingSourcesConfig::PreferFullFaceTracking) && extra_extensions::supports_eye_gaze_interaction(&self.xr_session, self.xr_system) { #[cfg(target_os = "android")] { alvr_system_info::try_get_permission("android.permission.RECORD_AUDIO"); alvr_system_info::try_get_permission("com.picovr.permission.FACE_TRACKING") } } } if config.body_tracking.is_some() && self.platform.is_quest() && self.platform != Platform::Quest1 { #[cfg(target_os = "android")] alvr_system_info::try_get_permission("com.oculus.permission.BODY_TRACKING") } // Note: We cannot enable multimodal if fb body tracking is active. It would result in a // ERROR_RUNTIME_FAILURE crash. if config.prefers_multimodal_input && config.body_tracking.is_none() && let Some(handle) = &mut self.multimodal_handle && handle.resume().is_ok() { self.multimodal_hands_enabled = true; } if let Some(config) = &config.face_tracking { // Note: this is actually used by multiple vendors self.face_sources.eyes_social = check_ext_object("EyeTrackerSocial", EyeTrackerSocial::new(&self.xr_session)); if matches!(config, FaceTrackingSourcesConfig::PreferFullFaceTracking) { if let Some(tracker) = check_ext_object( "FaceTracker2FB", FaceTracker2FB::new(self.xr_session.clone(), true, true), ) { self.face_sources.face_expressions_tracker = Some(FaceExpressionsTracker::Fb(tracker)) } else if let Some(tracker) = check_ext_object( "FaceTrackerPico", FaceTrackerPico::new(self.xr_session.clone()), ) { tracker.start_face_tracking().ok(); self.face_sources.face_expressions_tracker = Some(FaceExpressionsTracker::Pico(tracker)); } // For vive, face trackers are always created at startup regardless of settings, and // also cannot be destroyed early. } } if let Some(config) = &config.body_tracking { if config.meta.prefer_full_body { self.body_source = check_ext_object( "BodyTrackerFB (full set)", BodyTrackerFB::new( &self.xr_session, self.xr_system, *BODY_JOINT_SET_FULL_BODY_META, config.meta.prefer_high_fidelity, ), ) .map(|tracker| BodyTracker::Fb { tracker, joint_count: FULL_BODY_JOINT_COUNT_META, }); } if self.body_source.is_none() { self.body_source = check_ext_object( "BodyTrackerFB (default set)", BodyTrackerFB::new( &self.xr_session, self.xr_system, xr::BodyJointSetFB::DEFAULT, config.meta.prefer_high_fidelity, ), ) .map(|tracker| BodyTracker::Fb { tracker, joint_count: xr::BodyJointFB::COUNT.into_raw() as usize, }); } if self.body_source.is_none() { match config.bd { BodyTrackingBDConfig::BodyTracking { high_accuracy, prompt_calibration_on_start, } => { if high_accuracy { self.body_source = check_ext_object( "BodyTrackerBD (high accuracy)", BodyTrackerBD::new( self.xr_session.clone(), BodyJointSetBD::FULL_BODY_JOINTS, &self.extra_extensions, self.xr_system, prompt_calibration_on_start, ), ) .map(BodyTracker::BodyBD); } if self.body_source.is_none() { self.body_source = check_ext_object( "BodyTrackerBD (low accuracy)", BodyTrackerBD::new( self.xr_session.clone(), BodyJointSetBD::BODY_WITHOUT_ARM, &self.extra_extensions, self.xr_system, prompt_calibration_on_start, ), ) .map(BodyTracker::BodyBD); } } BodyTrackingBDConfig::ObjectTracking => { self.body_source = check_ext_object( "MotionTrackerBD (object tracking)", MotionTrackerBD::new(self.xr_session.clone(), &self.extra_extensions), ) .map(BodyTracker::MotionBD); } } } } } } pub fn get_reference_space( xr_session: &xr::Session, ty: xr::ReferenceSpaceType, ) -> xr::Space { xr_session .create_reference_space(ty, xr::Posef::IDENTITY) .unwrap() } pub fn get_head_data( xr_session: &xr::Session, platform: Platform, stage_reference_space: &xr::Space, view_reference_space: &xr::Space, time: Duration, future_time: Duration, last_view_params: &[ViewParams; 2], ) -> Option<(DeviceMotion, Option<[ViewParams; 2]>)> { let xr_time = crate::to_xr_time(time); let (head_location, head_velocity) = view_reference_space .relate(stage_reference_space, xr_time) .ok()?; if !head_location .location_flags .contains(xr::SpaceLocationFlags::ORIENTATION_VALID) { return None; } let (view_flags, views) = xr_session .locate_views( xr::ViewConfigurationType::PRIMARY_STEREO, xr_time, stage_reference_space, ) .ok()?; if !view_flags.contains(xr::ViewStateFlags::POSITION_VALID) || !view_flags.contains(xr::ViewStateFlags::ORIENTATION_VALID) { return None; } let mut motion = DeviceMotion { pose: crate::from_xr_pose(head_location.pose), linear_velocity: if head_velocity .velocity_flags .contains(xr::SpaceVelocityFlags::LINEAR_VALID) { crate::from_xr_vec3(head_velocity.linear_velocity) } else { Vec3::ZERO }, angular_velocity: if head_velocity .velocity_flags .contains(xr::SpaceVelocityFlags::ANGULAR_VALID) { crate::from_xr_vec3(head_velocity.angular_velocity) } else { Vec3::ZERO }, }; // Some headsets use wrong frame of reference for linear and angular velocities. if platform.is_pico() || platform.is_vive() || platform.is_yvr() { let xr_future_time = crate::to_xr_time(future_time); let predicted_location = view_reference_space .locate(stage_reference_space, xr_future_time) .ok()?; if !predicted_location .location_flags .contains(xr::SpaceLocationFlags::ORIENTATION_VALID) { return None; } let time_offset = future_time.saturating_sub(time); if !time_offset.is_zero() { let time_offset_s = time_offset.as_secs_f32(); motion.linear_velocity = (crate::from_xr_vec3(predicted_location.pose.position) - motion.pose.position) / time_offset_s; motion.angular_velocity = (crate::from_xr_quat(predicted_location.pose.orientation) * motion.pose.orientation.inverse()) .to_scaled_axis() / time_offset_s; } } let last_ipd_m = last_view_params[0] .pose .position .distance(last_view_params[1].pose.position); let current_ipd_m = crate::from_xr_vec3(views[1].pose.position) .distance(crate::from_xr_vec3(views[0].pose.position)); let view_params = if f32::abs(current_ipd_m - last_ipd_m) > IPD_CHANGE_EPS { Some([ ViewParams { pose: motion.pose.inverse() * crate::from_xr_pose(views[0].pose), fov: crate::from_xr_fov(views[0].fov), }, ViewParams { pose: motion.pose.inverse() * crate::from_xr_pose(views[1].pose), fov: crate::from_xr_fov(views[1].fov), }, ]) } else { None }; Some((motion, view_params)) } #[expect(clippy::too_many_arguments)] pub fn get_hand_data( xr_session: &xr::Session, platform: Platform, reference_space: &xr::Space, time: Duration, future_time: Duration, hand_source: &HandInteraction, last_controller_pose: &mut Pose, last_palm_pose: &mut Pose, ) -> HandData { let xr_time = crate::to_xr_time(time); let xr_now = crate::xr_runtime_now(xr_session.instance()).unwrap_or(xr_time); let grip_motion = if hand_source .grip_action .is_active(xr_session, xr::Path::NULL) .unwrap_or(false) && let Ok((location, velocity)) = hand_source.grip_space.relate(reference_space, xr_time) { let orientation_valid = location .location_flags .contains(xr::SpaceLocationFlags::ORIENTATION_VALID); let position_valid = location .location_flags .contains(xr::SpaceLocationFlags::POSITION_VALID); if orientation_valid { last_controller_pose.orientation = crate::from_xr_quat(location.pose.orientation); } if position_valid { last_controller_pose.position = crate::from_xr_vec3(location.pose.position); } let pose = *last_controller_pose * hand_source.pose_offset; let mut linear_velocity = crate::from_xr_vec3(velocity.linear_velocity); let mut angular_velocity = crate::from_xr_vec3(velocity.angular_velocity); let time_offset = future_time.saturating_sub(time); // Some headsets use wrong frame of reference for linear and angular velocities. if (platform.is_pico() || platform.is_vive()) && !time_offset.is_zero() && let Ok(future_location) = hand_source .grip_space .locate(reference_space, crate::to_xr_time(future_time)) && future_location.location_flags.contains( xr::SpaceLocationFlags::ORIENTATION_VALID | xr::SpaceLocationFlags::POSITION_VALID, ) { let time_offset_s = time_offset.as_secs_f32(); linear_velocity = (crate::from_xr_vec3(future_location.pose.position) - last_controller_pose.position) / time_offset_s; angular_velocity = (crate::from_xr_quat(future_location.pose.orientation) * last_controller_pose.orientation.inverse()) .to_scaled_axis() / time_offset_s; } Some(DeviceMotion { pose, linear_velocity, angular_velocity, }) } else { None }; let detached_grip_motion = if let Some(detached_grip_action) = &hand_source.detached_grip_action && detached_grip_action .is_active(xr_session, xr::Path::NULL) .unwrap_or(false) && let Ok((location, velocity)) = hand_source .detached_grip_space .as_ref() .unwrap() .relate(reference_space, xr_time) { if location .location_flags .contains(xr::SpaceLocationFlags::ORIENTATION_VALID) { last_controller_pose.orientation = crate::from_xr_quat(location.pose.orientation); } if location .location_flags .contains(xr::SpaceLocationFlags::POSITION_VALID) { last_controller_pose.position = crate::from_xr_vec3(location.pose.position); } Some(DeviceMotion { pose: *last_controller_pose, linear_velocity: crate::from_xr_vec3(velocity.linear_velocity), angular_velocity: crate::from_xr_vec3(velocity.angular_velocity), }) } else { None }; let skeleton_joints = if let Some(tracker) = &hand_source.skeleton_tracker && let Some(joint_locations) = reference_space .locate_hand_joints(tracker, xr_now) .ok() .flatten() { if joint_locations[0] .location_flags .contains(xr::SpaceLocationFlags::ORIENTATION_VALID) { last_palm_pose.orientation = crate::from_xr_quat(joint_locations[0].pose.orientation); } if joint_locations[0] .location_flags .contains(xr::SpaceLocationFlags::POSITION_VALID) { last_palm_pose.position = crate::from_xr_vec3(joint_locations[0].pose.position); } let mut joints: [_; 26] = joint_locations .iter() .map(|j| crate::from_xr_pose(j.pose)) .collect::>() .try_into() .unwrap(); joints[0] = *last_palm_pose; Some(joints) } else { None }; HandData { grip_motion, detached_grip_motion, skeleton_joints, } } pub fn update_buttons( xr_session: &xr::Session, button_actions: &HashMap, ) -> Vec { let mut button_entries = Vec::with_capacity(2); for (id, action) in button_actions { match action { ButtonAction::Binary(action) => { let Ok(state) = action.state(xr_session, xr::Path::NULL) else { continue; }; if state.changed_since_last_sync { button_entries.push(ButtonEntry { path_id: *id, value: ButtonValue::Binary(state.current_state), }); } } ButtonAction::Scalar(action) => { let Ok(state) = action.state(xr_session, xr::Path::NULL) else { continue; }; if state.changed_since_last_sync { button_entries.push(ButtonEntry { path_id: *id, value: ButtonValue::Scalar(state.current_state), }); } } } } button_entries } // Note: Using the headset view space in order to get heading-independent eye gazes pub fn get_face_data( xr_session: &xr::Session, sources: &FaceSources, view_reference_space: &xr::Space, time: Duration, ) -> FaceData { let xr_time = crate::to_xr_time(time); let eyes_combined = if let Some((action, space)) = &sources.eyes_combined && action .is_active(xr_session, xr::Path::NULL) .unwrap_or(false) && let Ok(location) = space.locate(view_reference_space, xr_time) && location .location_flags .contains(xr::SpaceLocationFlags::ORIENTATION_VALID) { Some(crate::from_xr_quat(location.pose.orientation)) } else { None }; let eyes_social = if let Some(tracker) = &sources.eyes_social && let Ok(gazes) = tracker.get_eye_gazes(view_reference_space, xr_time) { [ gazes[0].map(|p| crate::from_xr_quat(p.orientation)), gazes[1].map(|p| crate::from_xr_quat(p.orientation)), ] } else { [None, None] }; let face_expressions = if let Some(tracker) = &sources.face_expressions_tracker { match tracker { FaceExpressionsTracker::Fb(tracker) => tracker .get_face_expression_weights(xr_time) .ok() .flatten() .map(|weights| FaceExpressions::Fb(weights.into_iter().collect())), FaceExpressionsTracker::Pico(face_tracker_pico) => face_tracker_pico .get_face_tracking_data(xr_time) .ok() .flatten() .map(|weights| FaceExpressions::Pico(weights.into_iter().collect())), FaceExpressionsTracker::Htc { eye, lip } => { let eye = eye .as_ref() .and_then(|tracker| tracker.get_facial_expressions(xr_time).ok().flatten()); let lip = lip .as_ref() .and_then(|tracker| tracker.get_facial_expressions(xr_time).ok().flatten()); Some(FaceExpressions::Htc { eye, lip }) } } } else { None }; FaceData { eyes_combined, eyes_social, face_expressions, } } pub fn get_body_skeleton( source: &BodyTracker, reference_space: &xr::Space, time: Duration, ) -> Option { let xr_time = crate::to_xr_time(time); let check_and_convert_pose = |pose, location_flags: &xr::SpaceLocationFlags| { if location_flags .contains(SpaceLocationFlags::ORIENTATION_VALID | SpaceLocationFlags::POSITION_VALID) { Some(crate::from_xr_pose(pose)) } else { None } }; match source { BodyTracker::Fb { tracker, joint_count, } => { if let Some(joints) = tracker .locate_body_joints(xr_time, reference_space, *joint_count) .ok() .flatten() { let joints = joints .iter() .map(|joint| check_and_convert_pose(joint.pose, &joint.location_flags)) .collect::>(); Some(BodySkeleton::Fb(Box::new(BodySkeletonFb { upper_body: joints[..18].try_into().unwrap(), lower_body: (joints.len() >= 84).then(|| joints[70..84].try_into().unwrap()), }))) } else { None } } BodyTracker::BodyBD(tracker) => { if let Some(joints) = tracker .locate_body_joints(xr_time, reference_space) .ok() .flatten() { let joints = joints .iter() .map(|joint| check_and_convert_pose(joint.pose, &joint.location_flags)) .collect::>(); Some(BodySkeleton::Bd(Box::new(BodySkeletonBd( joints.try_into().unwrap(), )))) } else { None } } // Motion trackers are polled separately BodyTracker::MotionBD(_) => None, } } pub fn get_bd_motion_trackers(source: &BodyTracker, time: Duration) -> Vec<(u64, DeviceMotion)> { let xr_time = crate::to_xr_time(time); if let BodyTracker::MotionBD(tracker) = source && let Some(mut trackers) = tracker.locate_motion_trackers(xr_time).ok().flatten() { let mut joints = Vec::<(u64, DeviceMotion)>::with_capacity(3); let joints_ids = [ *GENERIC_TRACKER_1_ID, *GENERIC_TRACKER_2_ID, *GENERIC_TRACKER_3_ID, ]; trackers.sort_by(|a, b| a.serial.cmp(&b.serial)); for (i, item) in trackers.iter().enumerate() { joints.push(( joints_ids[i], DeviceMotion { pose: crate::from_xr_pose(item.local_pose.pose), linear_velocity: crate::from_xr_vec3(item.local_pose.linear_velocity), angular_velocity: crate::from_xr_vec3(item.local_pose.angular_velocity), }, )) } return joints; } Vec::new() } ================================================ FILE: alvr/client_openxr/src/lib.rs ================================================ mod c_api; mod extra_extensions; mod graphics; mod interaction; mod lobby; mod passthrough; mod stream; use crate::stream::ParsedStreamConfig; use alvr_client_core::{ClientCapabilities, ClientCoreContext, ClientCoreEvent}; use alvr_common::{ Fov, HAND_LEFT_ID, Pose, debug, error, glam::{Quat, UVec2, Vec3}, info, parking_lot::RwLock, }; use alvr_graphics::GraphicsContext; use alvr_session::{BodyTrackingBDConfig, BodyTrackingSourcesConfig, PerformanceLevel}; use alvr_system_info::Platform; use extra_extensions::{ BD_BODY_TRACKING_EXTENSION_NAME, BD_MOTION_TRACKING_EXTENSION_NAME, META_BODY_TRACKING_FIDELITY_EXTENSION_NAME, META_BODY_TRACKING_FULL_BODY_EXTENSION_NAME, META_DETACHED_CONTROLLERS_EXTENSION_NAME, META_SIMULTANEOUS_HANDS_AND_CONTROLLERS_EXTENSION_NAME, PICO_CONFIGURATION_EXTENSION_NAME, }; use interaction::{InteractionContext, InteractionSourcesConfig}; use lobby::Lobby; use openxr as xr; use passthrough::PassthroughLayer; use std::{ffi::CStr, path::Path, rc::Rc, sync::Arc, thread, time::Duration}; use stream::StreamContext; fn from_xr_vec3(v: xr::Vector3f) -> Vec3 { Vec3::new(v.x, v.y, v.z) } fn to_xr_vec3(v: Vec3) -> xr::Vector3f { xr::Vector3f { x: v.x, y: v.y, z: v.z, } } fn from_xr_quat(q: xr::Quaternionf) -> Quat { Quat::from_xyzw(q.x, q.y, q.z, q.w) } fn to_xr_quat(q: Quat) -> xr::Quaternionf { xr::Quaternionf { x: q.x, y: q.y, z: q.z, w: q.w, } } fn from_xr_pose(p: xr::Posef) -> Pose { Pose { orientation: from_xr_quat(p.orientation), position: from_xr_vec3(p.position), } } fn to_xr_pose(p: Pose) -> xr::Posef { xr::Posef { orientation: to_xr_quat(p.orientation), position: to_xr_vec3(p.position), } } fn from_xr_fov(f: xr::Fovf) -> Fov { Fov { left: f.angle_left, right: f.angle_right, up: f.angle_up, down: f.angle_down, } } fn to_xr_fov(f: Fov) -> xr::Fovf { xr::Fovf { angle_left: f.left, angle_right: f.right, angle_up: f.up, angle_down: f.down, } } fn from_xr_time(timestamp: xr::Time) -> Duration { Duration::from_nanos(timestamp.as_nanos() as _) } fn to_xr_time(timestamp: Duration) -> xr::Time { xr::Time::from_nanos(timestamp.as_nanos() as _) } fn to_perf_settings_level(level: PerformanceLevel) -> xr::PerfSettingsLevelEXT { match level { PerformanceLevel::PowerSavings => xr::PerfSettingsLevelEXT::POWER_SAVINGS, PerformanceLevel::SustainedLow => xr::PerfSettingsLevelEXT::SUSTAINED_LOW, PerformanceLevel::SustainedHigh => xr::PerfSettingsLevelEXT::SUSTAINED_HIGH, PerformanceLevel::Boost => xr::PerfSettingsLevelEXT::BOOST, } } fn set_performance_level( xr_instance: &xr::Instance, xr_session: &xr::Session, domain: xr::PerfSettingsDomainEXT, level: PerformanceLevel, ) { if let Some(performance_settings) = xr_instance.exts().ext_performance_settings { unsafe { (performance_settings.perf_settings_set_performance_level)( xr_session.as_raw(), domain, to_perf_settings_level(level), ); } } } fn default_view() -> xr::View { xr::View { pose: xr::Posef { orientation: xr::Quaternionf { x: 0.0, y: 0.0, z: 0.0, w: 1.0, }, position: xr::Vector3f::default(), }, fov: xr::Fovf { angle_left: -1.0, angle_right: 1.0, angle_up: 1.0, angle_down: -1.0, }, } } // This exists to circumvent dead-code analysis fn create_session( xr_instance: &xr::Instance, xr_system: xr::SystemId, graphics_context: &GraphicsContext, ) -> ( xr::Session, xr::FrameWaiter, xr::FrameStream, ) { #[allow(unreachable_code)] unsafe { xr_instance .create_session(xr_system, &graphics::session_create_info(graphics_context)) .unwrap() } } pub fn entry_point() { alvr_client_core::init_logging(); const LEGACY_OPENXR_VERSION: xr::Version = xr::Version::new(1, 0, 34); const CURRENT_OPENXR_VERSION: xr::Version = xr::Version::new(1, 1, 36); // Using a provisional platform, before we can get the runtime info let (loader_suffix, openxr_version) = match alvr_system_info::platform(None, None) { Platform::Quest1 => ("_quest1", LEGACY_OPENXR_VERSION), Platform::PicoNeo3 | Platform::PicoG3 | Platform::Pico4 | Platform::Pico4Pro | Platform::Pico4Enterprise => ("_pico_old", LEGACY_OPENXR_VERSION), p if p.is_vive() => ("", LEGACY_OPENXR_VERSION), p if p.is_yvr() => ("_yvr", LEGACY_OPENXR_VERSION), Platform::Lynx => ("_lynx", LEGACY_OPENXR_VERSION), _ => ("", CURRENT_OPENXR_VERSION), }; let xr_entry = unsafe { xr::Entry::load_from(Path::new(&format!("libopenxr_loader{loader_suffix}.so"))).unwrap() }; #[cfg(target_os = "android")] xr_entry.initialize_android_loader().unwrap(); let available_extensions = xr_entry.enumerate_extensions().unwrap(); info!("OpenXR available extensions: {available_extensions:#?}"); info!( "Extra available extensions: {:#?}", available_extensions .other .iter() .map(|vec| CStr::from_bytes_with_nul(vec) .unwrap() .to_str() .unwrap() .to_owned()) .collect::>() ); // todo: switch to vulkan assert!(available_extensions.khr_opengl_es_enable); let mut exts = xr::ExtensionSet::default(); exts.bd_controller_interaction = available_extensions.bd_controller_interaction; exts.ext_eye_gaze_interaction = available_extensions.ext_eye_gaze_interaction; exts.ext_hand_tracking = available_extensions.ext_hand_tracking; exts.ext_local_floor = available_extensions.ext_local_floor; exts.ext_performance_settings = available_extensions.ext_performance_settings; exts.ext_user_presence = available_extensions.ext_user_presence; exts.fb_body_tracking = available_extensions.fb_body_tracking; exts.fb_color_space = available_extensions.fb_color_space; exts.fb_composition_layer_settings = available_extensions.fb_composition_layer_settings; exts.fb_display_refresh_rate = available_extensions.fb_display_refresh_rate; exts.fb_eye_tracking_social = available_extensions.fb_eye_tracking_social; exts.fb_face_tracking2 = available_extensions.fb_face_tracking2; exts.fb_foveation = available_extensions.fb_foveation; exts.fb_foveation_configuration = available_extensions.fb_foveation_configuration; exts.fb_passthrough = available_extensions.fb_passthrough; exts.fb_swapchain_update_state = available_extensions.fb_swapchain_update_state; exts.htc_facial_tracking = available_extensions.htc_facial_tracking; exts.htc_passthrough = available_extensions.htc_passthrough; exts.htc_vive_focus3_controller_interaction = available_extensions.htc_vive_focus3_controller_interaction; #[cfg(target_os = "android")] { exts.khr_android_create_instance = true; } exts.khr_convert_timespec_time = true; exts.khr_opengl_es_enable = true; exts.other = available_extensions .other .into_iter() .filter(|ext| { [ META_BODY_TRACKING_FULL_BODY_EXTENSION_NAME, META_BODY_TRACKING_FIDELITY_EXTENSION_NAME, META_SIMULTANEOUS_HANDS_AND_CONTROLLERS_EXTENSION_NAME, META_DETACHED_CONTROLLERS_EXTENSION_NAME, BD_BODY_TRACKING_EXTENSION_NAME, BD_MOTION_TRACKING_EXTENSION_NAME, PICO_CONFIGURATION_EXTENSION_NAME, ] .contains(&CStr::from_bytes_with_nul(ext).unwrap().to_str().unwrap()) }) .collect(); let available_layers = xr_entry.enumerate_layers().unwrap(); info!("OpenXR available layers: {available_layers:#?}"); let other_exts = exts .other .iter() .map(|vec| { CStr::from_bytes_with_nul(vec) .unwrap() .to_str() .unwrap() .to_owned() }) .collect::>(); let xr_instance = xr_entry .create_instance( &xr::ApplicationInfo { application_name: "ALVR Client", application_version: 0, engine_name: "ALVR", engine_version: 0, api_version: openxr_version, }, &exts, &[], ) .unwrap(); let platform = alvr_system_info::platform( xr_instance .properties() .ok() .map(|s| s.runtime_name.to_owned()), xr_instance .properties() .ok() .map(|s| s.runtime_version.into_raw()), ); let graphics_context = Rc::new(GraphicsContext::new_gl()); let mut last_lobby_message = String::new(); 'session_loop: loop { let xr_system = xr_instance .system(xr::FormFactor::HEAD_MOUNTED_DISPLAY) .unwrap(); // mandatory call let _ = xr_instance .graphics_requirements::(xr_system) .unwrap(); let (xr_session, mut xr_frame_waiter, mut xr_frame_stream) = create_session(&xr_instance, xr_system, &graphics_context); let views_config = xr_instance .enumerate_view_configuration_views( xr_system, xr::ViewConfigurationType::PRIMARY_STEREO, ) .unwrap(); assert_eq!(views_config.len(), 2); let default_view_resolution = UVec2::new( views_config[0].recommended_image_rect_width, views_config[0].recommended_image_rect_height, ); let max_view_resolution = UVec2::new( views_config[0].max_image_rect_width, views_config[0].max_image_rect_height, ); let refresh_rates = if exts.fb_display_refresh_rate { xr_session.enumerate_display_refresh_rates().unwrap() } else { vec![90.0] }; if exts.fb_color_space { xr_session .set_color_space(xr::ColorSpaceFB::REC709) .unwrap(); } let capabilities = ClientCapabilities { platform, default_view_resolution, max_view_resolution, refresh_rates, foveated_encoding: platform != Platform::Unknown, encoder_high_profile: platform != Platform::Unknown, encoder_10_bits: platform != Platform::Unknown, encoder_av1: matches!( platform, Platform::Quest3 | Platform::Quest3S | Platform::Pico4Ultra ), prefer_10bit: false, preferred_encoding_gamma: 1.0, prefer_hdr: false, }; let core_context = Arc::new(ClientCoreContext::new(capabilities)); let interaction_context = Arc::new(RwLock::new(InteractionContext::new( xr_session.clone(), other_exts.clone(), xr_system, platform, ))); let mut lobby = Lobby::new( xr_session.clone(), Rc::clone(&graphics_context), Arc::clone(&interaction_context), platform, UVec2::min(default_view_resolution * 2, max_view_resolution), &last_lobby_message, ); // For Meta/Quest enabling body tracking would disable multimodal input let lobby_body_tracking_config = if platform.is_pico() { Some(BodyTrackingSourcesConfig { bd: BodyTrackingBDConfig::BodyTracking { high_accuracy: true, prompt_calibration_on_start: false, }, meta: Default::default(), }) } else { None }; let lobby_interaction_sources = InteractionSourcesConfig { face_tracking: None, body_tracking: lobby_body_tracking_config, prefers_multimodal_input: true, }; interaction_context .write() .select_sources(&lobby_interaction_sources); let mut session_running = false; let mut stream_context = None::; let mut passthrough_layer = None; let mut event_storage = xr::EventDataBuffer::new(); let mut headset_is_worn = true; 'render_loop: loop { while let Some(event) = xr_instance.poll_event(&mut event_storage).unwrap() { match event { xr::Event::EventsLost(event) => { error!("OpenXR: lost {} events!", event.lost_event_count()); } xr::Event::InstanceLossPending(_) => break 'session_loop, xr::Event::SessionStateChanged(event) => match event.state() { xr::SessionState::READY => { xr_session .begin(xr::ViewConfigurationType::PRIMARY_STEREO) .unwrap(); core_context.resume(); passthrough_layer = PassthroughLayer::new(&xr_session, platform).ok(); session_running = true; } xr::SessionState::STOPPING => { session_running = false; passthrough_layer = None; core_context.pause(); xr_session.end().unwrap(); } xr::SessionState::EXITING | xr::SessionState::LOSS_PENDING => { break 'render_loop; } _ => (), }, xr::Event::ReferenceSpaceChangePending(event) => { info!( "ReferenceSpaceChangePending type: {:?}", event.reference_space_type() ); lobby.update_reference_space(); if let Some(stream) = &mut stream_context { stream.update_reference_space(); } } xr::Event::PerfSettingsEXT(event) => { info!( "Perf: from {:?} to {:?}, domain: {:?}/{:?}", event.from_level(), event.to_level(), event.domain(), event.sub_domain(), ); } xr::Event::InteractionProfileChanged(_) | xr::Event::PassthroughStateChangedFB(_) => { // todo } xr::Event::UserPresenceChangedEXT(event) => { debug!("user present: {:?}", event.is_user_present()); headset_is_worn = event.is_user_present(); core_context.send_proximity_state(event.is_user_present()); } xr::Event::Unknown => { // use event_storage.as_raw(), reinterpret as sys::BaseInStructure, get type // and then reinterpret as the event struct } _ => (), } } if !session_running { thread::sleep(Duration::from_millis(100)); continue; } while let Some(event) = core_context.poll_event() { match event { ClientCoreEvent::UpdateHudMessage(message) => { last_lobby_message.clone_from(&message); lobby.update_hud_message(&message); } ClientCoreEvent::StreamingStarted(config) => { let config = ParsedStreamConfig::new(&config); let context = StreamContext::new( Arc::clone(&core_context), xr_session.clone(), Rc::clone(&graphics_context), Arc::clone(&interaction_context), config, ); if !context.uses_passthrough() { passthrough_layer = None; } stream_context = Some(context); core_context.send_proximity_state(headset_is_worn); } ClientCoreEvent::StreamingStopped => { if passthrough_layer.is_none() { passthrough_layer = PassthroughLayer::new(&xr_session, platform).ok(); } interaction_context .write() .select_sources(&lobby_interaction_sources); stream_context = None; } ClientCoreEvent::Haptics { device_id, duration, frequency, amplitude, } => { let idx = if device_id == *HAND_LEFT_ID { 0 } else { 1 }; let action = &interaction_context.read().hands_interaction[idx].vibration_action; action .apply_feedback( &xr_session, xr::Path::NULL, &xr::HapticVibration::new() .amplitude(amplitude.clamp(0.0, 1.0)) .frequency(frequency.max(0.0)) .duration(xr::Duration::from_nanos(duration.as_nanos() as _)), ) .unwrap(); } ClientCoreEvent::DecoderConfig { codec, config_nal } => { if let Some(stream) = &mut stream_context { stream.maybe_initialize_decoder(codec, config_nal); } } ClientCoreEvent::RealTimeConfig(config) => { if config.passthrough.is_some() && passthrough_layer.is_none() { passthrough_layer = PassthroughLayer::new(&xr_session, platform).ok(); } else if config.passthrough.is_none() && passthrough_layer.is_some() { passthrough_layer = None; } if let Some(cpu_performance_level) = &config.cpu_performance_level { set_performance_level( &xr_instance, &xr_session, xr::PerfSettingsDomainEXT::CPU, cpu_performance_level.clone(), ); } if let Some(gpu_performance_level) = &config.gpu_performance_level { set_performance_level( &xr_instance, &xr_session, xr::PerfSettingsDomainEXT::GPU, gpu_performance_level.clone(), ); } if let Some(stream) = &mut stream_context { stream.update_real_time_config(&config); } } } } let frame_state = match xr_frame_waiter.wait() { Ok(state) => state, Err(e) => { error!("{e}"); panic!(); } }; let frame_interval = Duration::from_nanos(frame_state.predicted_display_period.as_nanos() as _); let vsync_time = Duration::from_nanos(frame_state.predicted_display_time.as_nanos() as _); xr_frame_stream.begin().unwrap(); if !frame_state.should_render { xr_frame_stream .end( frame_state.predicted_display_time, xr::EnvironmentBlendMode::OPAQUE, &[], ) .unwrap(); continue; } // todo: allow rendering lobby and stream layers at the same time and add cross fade let (layer, display_time) = if let Some(stream) = &mut stream_context { stream.render(frame_interval, vsync_time) } else { (lobby.render(vsync_time), vsync_time) }; let layers: &[&xr::CompositionLayerBase<_>] = if let Some(passthrough_layer) = &passthrough_layer { &[passthrough_layer, &layer.build()] } else { &[&layer.build()] }; graphics_context.make_current(); let res = xr_frame_stream.end( to_xr_time(display_time), xr::EnvironmentBlendMode::OPAQUE, layers, ); if let Err(e) = res { let time = to_xr_time(display_time); error!("End frame failed! {e}, timestamp: {display_time:?}, time: {time:?}"); if !platform.is_vive() { xr_frame_stream .end( frame_state.predicted_display_time, xr::EnvironmentBlendMode::OPAQUE, &[], ) .unwrap(); } } } } // grapics_context is dropped here } #[allow(unused)] fn xr_runtime_now(xr_instance: &xr::Instance) -> Option { xr_instance .now() .ok() .filter(|&time_nanos| time_nanos.as_nanos() > 0) } #[cfg(target_os = "android")] #[unsafe(no_mangle)] fn android_main(app: android_activity::AndroidApp) { use android_activity::{InputStatus, MainEvent, PollEvent}; let rendering_thread = thread::spawn(|| { // workaround for the Pico runtime let context = ndk_context::android_context(); let vm = unsafe { jni::JavaVM::from_raw(context.vm().cast()) }.unwrap(); let _env = vm.attach_current_thread().unwrap(); entry_point(); }); let mut should_quit = false; while !should_quit { app.poll_events(Some(Duration::from_millis(100)), |event| match event { PollEvent::Main(MainEvent::Destroy) => { should_quit = true; } PollEvent::Main(MainEvent::InputAvailable) => { if let Ok(mut iter) = app.input_events_iter() { while iter.next(|_| InputStatus::Unhandled) {} } } _ => (), }); } // Note: the quit event is sent from OpenXR too, this will return rather quicly. rendering_thread.join().unwrap(); } ================================================ FILE: alvr/client_openxr/src/lobby.rs ================================================ use crate::{ graphics::{self, ProjectionLayerAlphaConfig, ProjectionLayerBuilder}, interaction::{self, InteractionContext}, }; use alvr_common::{Pose, ViewParams, glam::UVec2, parking_lot::RwLock}; use alvr_graphics::{GraphicsContext, LobbyRenderer, LobbyViewParams, SDR_FORMAT_GL}; use alvr_system_info::Platform; use openxr as xr; use std::{rc::Rc, sync::Arc, time::Duration}; // todo: add interaction? pub struct Lobby { xr_session: xr::Session, interaction_ctx: Arc>, platform: Platform, reference_space: xr::Space, swapchains: [xr::Swapchain; 2], view_resolution: UVec2, reference_space_type: xr::ReferenceSpaceType, renderer: LobbyRenderer, } impl Lobby { pub fn new( xr_session: xr::Session, gfx_ctx: Rc, interaction_ctx: Arc>, platform: Platform, view_resolution: UVec2, initial_hud_message: &str, ) -> Self { let reference_space_type = if xr_session.instance().exts().ext_local_floor.is_some() { xr::ReferenceSpaceType::LOCAL_FLOOR_EXT } else { // The Quest 1 doesn't support LOCAL_FLOOR_EXT, recentering is required for AppLab, but // the Quest 1 is excluded from AppLab anyway. xr::ReferenceSpaceType::STAGE }; let reference_space = interaction::get_reference_space(&xr_session, reference_space_type); let swapchains = [ graphics::create_swapchain(&xr_session, &gfx_ctx, view_resolution, SDR_FORMAT_GL, None), graphics::create_swapchain(&xr_session, &gfx_ctx, view_resolution, SDR_FORMAT_GL, None), ]; let renderer = LobbyRenderer::new( gfx_ctx, view_resolution, [ swapchains[0] .enumerate_images() .unwrap() .iter() .map(|i| *i as _) .collect(), swapchains[1] .enumerate_images() .unwrap() .iter() .map(|i| *i as _) .collect(), ], initial_hud_message, ); Self { xr_session, interaction_ctx, platform, reference_space, swapchains, view_resolution, reference_space_type, renderer, } } pub fn update_reference_space(&mut self) { self.reference_space = interaction::get_reference_space(&self.xr_session, self.reference_space_type); } pub fn update_hud_message(&self, message: &str) { self.renderer.update_hud_message(message); } pub fn render(&mut self, vsync_time: Duration) -> ProjectionLayerBuilder<'_> { let xr_vsync_time = crate::to_xr_time(vsync_time); let (flags, maybe_views) = self .xr_session .locate_views( xr::ViewConfigurationType::PRIMARY_STEREO, xr_vsync_time, &self.reference_space, ) .unwrap(); let views = if flags.contains(xr::ViewStateFlags::ORIENTATION_VALID) { maybe_views } else { vec![crate::default_view(), crate::default_view()] }; self.xr_session .sync_actions(&[(&self.interaction_ctx.read().action_set).into()]) .ok(); // future_time doesn't have to be any particular value, just something after vsync_time let future_time = vsync_time + Duration::from_millis(80); let left_hand_data = interaction::get_hand_data( &self.xr_session, self.platform, &self.reference_space, vsync_time, future_time, &self.interaction_ctx.read().hands_interaction[0], &mut Pose::default(), &mut Pose::default(), ); let right_hand_data = interaction::get_hand_data( &self.xr_session, self.platform, &self.reference_space, vsync_time, future_time, &self.interaction_ctx.read().hands_interaction[1], &mut Pose::default(), &mut Pose::default(), ); let additional_motions = self .interaction_ctx .read() .body_source .as_ref() .map(|source| { interaction::get_bd_motion_trackers(source, vsync_time) .iter() .map(|(_, motion)| *motion) .collect() }); let body_skeleton = self .interaction_ctx .read() .body_source .as_ref() .and_then(|source| { interaction::get_body_skeleton(source, &self.reference_space, vsync_time) }); let left_swapchain_idx = self.swapchains[0].acquire_image().unwrap(); let right_swapchain_idx = self.swapchains[1].acquire_image().unwrap(); self.swapchains[0] .wait_image(xr::Duration::INFINITE) .unwrap(); self.swapchains[1] .wait_image(xr::Duration::INFINITE) .unwrap(); self.renderer.render( [ LobbyViewParams { view_params: ViewParams { pose: crate::from_xr_pose(views[0].pose), fov: crate::from_xr_fov(views[0].fov), }, swapchain_index: left_swapchain_idx, }, LobbyViewParams { view_params: ViewParams { pose: crate::from_xr_pose(views[1].pose), fov: crate::from_xr_fov(views[1].fov), }, swapchain_index: right_swapchain_idx, }, ], [left_hand_data, right_hand_data], body_skeleton, additional_motions, false, cfg!(debug_assertions), ); self.swapchains[0].release_image().unwrap(); self.swapchains[1].release_image().unwrap(); let rect = xr::Rect2Di { offset: xr::Offset2Di { x: 0, y: 0 }, extent: xr::Extent2Di { width: self.view_resolution.x as _, height: self.view_resolution.y as _, }, }; ProjectionLayerBuilder::new( &self.reference_space, [ xr::CompositionLayerProjectionView::new() .pose(views[0].pose) .fov(views[0].fov) .sub_image( xr::SwapchainSubImage::new() .swapchain(&self.swapchains[0]) .image_array_index(0) .image_rect(rect), ), xr::CompositionLayerProjectionView::new() .pose(views[1].pose) .fov(views[1].fov) .sub_image( xr::SwapchainSubImage::new() .swapchain(&self.swapchains[1]) .image_array_index(0) .image_rect(rect), ), ], Some(ProjectionLayerAlphaConfig { premultiplied: true, }), None, ) } } ================================================ FILE: alvr/client_openxr/src/passthrough.rs ================================================ use crate::extra_extensions::{PassthroughFB, PassthroughHTC}; use alvr_common::anyhow::{Result, bail}; use alvr_system_info::Platform; use openxr::{self as xr}; use std::{marker::PhantomData, ops::Deref, ptr}; pub struct PassthroughLayer<'a> { handle_fb: Option, handle_htc: Option, _marker: PhantomData<&'a ()>, } impl PassthroughLayer<'_> { pub fn new(session: &xr::Session, platform: Platform) -> Result { let mut handle_fb = None; let mut handle_htc = None; let exts = session.instance().exts(); if exts.fb_passthrough.is_some() { handle_fb = Some(PassthroughFB::new(session, platform)?); } else if exts.htc_passthrough.is_some() { handle_htc = Some(PassthroughHTC::new(session)?); } else { bail!("No passthrough extension available"); }; Ok(Self { handle_fb, handle_htc, _marker: PhantomData, }) } } impl<'a> Deref for PassthroughLayer<'a> { type Target = xr::CompositionLayerBase<'a, xr::OpenGlEs>; fn deref(&self) -> &Self::Target { if let Some(handle) = &self.handle_fb { unsafe { &*ptr::from_ref(handle.layer()).cast() } } else if let Some(handle) = &self.handle_htc { unsafe { &*ptr::from_ref(handle.layer()).cast() } } else { panic!("No passthrough extension available"); } } } ================================================ FILE: alvr/client_openxr/src/stream.rs ================================================ use crate::{ graphics::{self, ProjectionLayerAlphaConfig, ProjectionLayerBuilder}, interaction::{self, InteractionContext, InteractionSourcesConfig}, }; use alvr_client_core::{ ClientCoreContext, video_decoder::{self, VideoDecoderConfig, VideoDecoderSource}, }; use alvr_common::{ DETACHED_CONTROLLER_LEFT_ID, DETACHED_CONTROLLER_RIGHT_ID, HAND_LEFT_ID, HAND_RIGHT_ID, HEAD_ID, Pose, RelaxedAtomic, ViewParams, anyhow::Result, error, glam::{UVec2, Vec2}, parking_lot::RwLock, }; use alvr_graphics::{GraphicsContext, StreamRenderer, StreamViewParams}; use alvr_packets::{RealTimeConfig, StreamConfig, TrackingData}; use alvr_session::{ ClientsideFoveationConfig, ClientsideFoveationMode, ClientsidePostProcessingConfig, CodecType, FoveatedEncodingConfig, MediacodecProperty, PassthroughMode, UpscalingConfig, }; use alvr_system_info::Platform; use openxr as xr; use std::{ ptr, rc::Rc, sync::Arc, thread::{self, JoinHandle}, time::{Duration, Instant}, }; const DECODER_MAX_TIMEOUT_MULTIPLIER: f32 = 0.8; pub struct ParsedStreamConfig { pub view_resolution: UVec2, pub refresh_rate_hint: f32, pub encoding_gamma: f32, pub enable_hdr: bool, pub passthrough: Option, pub foveated_encoding_config: Option, pub clientside_foveation_config: Option, pub clientside_post_processing: Option, pub upscaling: Option, pub force_software_decoder: bool, pub max_buffering_frames: f32, pub buffering_history_weight: f32, pub decoder_options: Vec<(String, MediacodecProperty)>, pub interaction_sources: InteractionSourcesConfig, } impl ParsedStreamConfig { pub fn new(config: &StreamConfig) -> Self { Self { view_resolution: config.negotiated_config.view_resolution, refresh_rate_hint: config.negotiated_config.refresh_rate_hint, encoding_gamma: config.negotiated_config.encoding_gamma, enable_hdr: config.negotiated_config.enable_hdr, passthrough: config.settings.video.passthrough.as_option().cloned(), foveated_encoding_config: config .negotiated_config .enable_foveated_encoding .then(|| config.settings.video.foveated_encoding.as_option().cloned()) .flatten(), clientside_foveation_config: config .settings .video .clientside_foveation .as_option() .cloned(), clientside_post_processing: config .settings .video .clientside_post_processing .as_option() .cloned(), upscaling: config.settings.video.upscaling.as_option().cloned(), force_software_decoder: config.settings.video.force_software_decoder, max_buffering_frames: config.settings.video.max_buffering_frames, buffering_history_weight: config.settings.video.buffering_history_weight, decoder_options: config.settings.video.mediacodec_extra_options.clone(), interaction_sources: InteractionSourcesConfig::new(config), } } } pub struct StreamContext { core_context: Arc, xr_session: xr::Session, interaction_context: Arc>, stage_reference_space: Arc, view_reference_space: Arc, swapchains: [xr::Swapchain; 2], last_good_view_params: [ViewParams; 2], input_thread: Option>, input_thread_running: Arc, config: ParsedStreamConfig, target_view_resolution: UVec2, renderer: StreamRenderer, decoder: Option<(VideoDecoderConfig, VideoDecoderSource)>, use_custom_reprojection: bool, } impl StreamContext { pub fn new( core_ctx: Arc, xr_session: xr::Session, gfx_ctx: Rc, interaction_ctx: Arc>, config: ParsedStreamConfig, ) -> StreamContext { interaction_ctx .write() .select_sources(&config.interaction_sources); let xr_exts = xr_session.instance().exts(); if xr_exts.fb_display_refresh_rate.is_some() { xr_session .request_display_refresh_rate(config.refresh_rate_hint) .unwrap(); } let foveation_profile = if let Some(config) = &config.clientside_foveation_config && xr_exts.fb_swapchain_update_state.is_some() && xr_exts.fb_foveation.is_some() && xr_exts.fb_foveation_configuration.is_some() { let level; let dynamic; match config.mode { ClientsideFoveationMode::Static { level: lvl } => { level = lvl; dynamic = false; } ClientsideFoveationMode::Dynamic { max_level } => { level = max_level; dynamic = true; } }; xr_session .create_foveation_profile(Some(xr::FoveationLevelProfile { level: xr::FoveationLevelFB::from_raw(level as i32), vertical_offset: config.vertical_offset_deg, dynamic: xr::FoveationDynamicFB::from_raw(dynamic as i32), })) .ok() } else { None }; let target_view_resolution = alvr_graphics::compute_target_view_resolution( config.view_resolution, &config.upscaling, ); let format = graphics::swapchain_format(&gfx_ctx, &xr_session, config.enable_hdr); let swapchains = [ graphics::create_swapchain( &xr_session, &gfx_ctx, target_view_resolution, format, foveation_profile.as_ref(), ), graphics::create_swapchain( &xr_session, &gfx_ctx, target_view_resolution, format, foveation_profile.as_ref(), ), ]; let renderer = StreamRenderer::new( gfx_ctx, config.view_resolution, target_view_resolution, [ swapchains[0] .enumerate_images() .unwrap() .iter() .map(|i| *i as _) .collect(), swapchains[1] .enumerate_images() .unwrap() .iter() .map(|i| *i as _) .collect(), ], format, config.foveated_encoding_config.clone(), core_ctx.platform() != Platform::Lynx && !((core_ctx.platform().is_pico() || (core_ctx.platform() == Platform::SamsungGalaxyXR)) && config.enable_hdr), // TODO: Find a driver heuristic for the limited range bug instead? core_ctx.platform() != Platform::SamsungGalaxyXR && !config.enable_hdr, config.encoding_gamma, config.upscaling.clone(), ); { let int_ctx = interaction_ctx.read(); core_ctx.send_active_interaction_profile( *HAND_LEFT_ID, int_ctx.hands_interaction[0].controllers_profile_id, int_ctx.hands_interaction[0].input_ids.clone(), ); core_ctx.send_active_interaction_profile( *HAND_RIGHT_ID, int_ctx.hands_interaction[1].controllers_profile_id, int_ctx.hands_interaction[1].input_ids.clone(), ); } let input_thread_running = Arc::new(RelaxedAtomic::new(false)); let stage_reference_space = Arc::new(interaction::get_reference_space( &xr_session, xr::ReferenceSpaceType::STAGE, )); let view_reference_space = Arc::new(interaction::get_reference_space( &xr_session, xr::ReferenceSpaceType::VIEW, )); let mut this = StreamContext { use_custom_reprojection: core_ctx.platform().is_yvr(), core_context: core_ctx, xr_session, interaction_context: interaction_ctx, stage_reference_space, view_reference_space, swapchains, last_good_view_params: [ViewParams::DUMMY; 2], input_thread: None, input_thread_running, config, target_view_resolution, renderer, decoder: None, }; this.update_reference_space(); this } pub fn uses_passthrough(&self) -> bool { self.config.passthrough.is_some() } pub fn update_reference_space(&mut self) { self.input_thread_running.set(false); self.stage_reference_space = Arc::new(interaction::get_reference_space( &self.xr_session, xr::ReferenceSpaceType::STAGE, )); self.view_reference_space = Arc::new(interaction::get_reference_space( &self.xr_session, xr::ReferenceSpaceType::VIEW, )); self.core_context.send_playspace( self.xr_session .reference_space_bounds_rect(xr::ReferenceSpaceType::STAGE) .unwrap() .map(|a| Vec2::new(a.width, a.height)), ); if let Some(running) = self.input_thread.take() { running.join().ok(); } self.input_thread_running.set(true); self.input_thread = Some(thread::spawn({ let core_ctx = Arc::clone(&self.core_context); let xr_session = self.xr_session.clone(); let interaction_ctx = Arc::clone(&self.interaction_context); let stage_reference_space = Arc::clone(&self.stage_reference_space); let view_reference_space = Arc::clone(&self.view_reference_space); let refresh_rate = self.config.refresh_rate_hint; let running = Arc::clone(&self.input_thread_running); move || { stream_input_loop( &core_ctx, xr_session, &interaction_ctx, &stage_reference_space, &view_reference_space, refresh_rate, running, ) } })); } pub fn maybe_initialize_decoder(&mut self, codec: CodecType, config_nal: Vec) { let new_config = VideoDecoderConfig { codec, force_software_decoder: self.config.force_software_decoder, max_buffering_frames: self.config.max_buffering_frames, buffering_history_weight: self.config.buffering_history_weight, options: self.config.decoder_options.clone(), config_buffer: config_nal, }; let maybe_config = if let Some((config, _)) = &self.decoder { (new_config != *config).then_some(new_config) } else { Some(new_config) }; if let Some(config) = maybe_config { let (mut sink, source) = video_decoder::create_decoder(config.clone(), { let ctx = Arc::clone(&self.core_context); move |maybe_timestamp: Result| match maybe_timestamp { Ok(timestamp) => ctx.report_frame_decoded(timestamp), Err(e) => ctx.report_fatal_decoder_error(&e.to_string()), } }); self.decoder = Some((config, source)); self.core_context.set_decoder_input_callback(Box::new( move |timestamp, buffer| -> bool { sink.push_nal(timestamp, buffer) }, )); } } pub fn update_real_time_config(&mut self, config: &RealTimeConfig) { self.config.passthrough = config.passthrough.clone(); self.config.clientside_post_processing = config.clientside_post_processing.clone(); } pub fn render( &mut self, frame_interval: Duration, vsync_time: Duration, ) -> (ProjectionLayerBuilder<'_>, Duration) { let xr_vsync_time = xr::Time::from_nanos(vsync_time.as_nanos() as _); let frame_poll_deadline = Instant::now() + Duration::from_secs_f32( frame_interval.as_secs_f32() * DECODER_MAX_TIMEOUT_MULTIPLIER, ); let mut frame_result = None; if let Some((_, source)) = &mut self.decoder { while frame_result.is_none() && Instant::now() < frame_poll_deadline { frame_result = source.get_frame(); thread::sleep(Duration::from_micros(500)); } } let (timestamp, view_params, buffer_ptr) = if let Some((timestamp, buffer_ptr)) = frame_result { let view_params = self.core_context.report_compositor_start(timestamp); self.last_good_view_params = view_params; (timestamp, view_params, buffer_ptr) } else { (vsync_time, self.last_good_view_params, ptr::null_mut()) }; let left_swapchain_idx = self.swapchains[0].acquire_image().unwrap(); let right_swapchain_idx = self.swapchains[1].acquire_image().unwrap(); self.swapchains[0] .wait_image(xr::Duration::INFINITE) .unwrap(); self.swapchains[1] .wait_image(xr::Duration::INFINITE) .unwrap(); let (flags, maybe_views) = self .xr_session .locate_views( xr::ViewConfigurationType::PRIMARY_STEREO, xr_vsync_time, &self.stage_reference_space, ) .unwrap(); let current_headset_views = if flags.contains(xr::ViewStateFlags::ORIENTATION_VALID) { maybe_views } else { vec![crate::default_view(), crate::default_view()] }; // The poses and FoVs we received from the PC runtime, which may differ and/or include // altered FoVs based on settings and view conversions done for canting. let input_view_params = view_params; let mut output_view_params = input_view_params; // Avoid passing invalid timestamp to runtime. // `timestamp` is generally a current vsync time, but may be repeated if frames are // dropped. Some runtimes dislike it if the timestamp is repeated for too long, so after // one second we begin presenting a lagged vsync time instead. let mut openxr_display_time = Duration::max(timestamp, vsync_time.saturating_sub(Duration::from_secs(1))); // (shinyquagsire23) I don't entirely trust runtimes to implement CompositionLayerProjectionView // correctly, but if we do trust them, avoid doing rotation ourselves. Otherwise, rerender. // Ex: YVR/PFDMR has issues with aspect ratio mismatches and passthrough compositing. if self.use_custom_reprojection { output_view_params = [ ViewParams { pose: crate::from_xr_pose(current_headset_views[0].pose), fov: crate::from_xr_fov(current_headset_views[0].fov), }, ViewParams { pose: crate::from_xr_pose(current_headset_views[1].pose), fov: crate::from_xr_fov(current_headset_views[1].fov), }, ]; openxr_display_time = vsync_time; } self.renderer.render( buffer_ptr, [ StreamViewParams { swapchain_index: left_swapchain_idx, input_view_params: input_view_params[0], output_view_params: output_view_params[0], }, StreamViewParams { swapchain_index: right_swapchain_idx, input_view_params: input_view_params[1], output_view_params: output_view_params[1], }, ], self.config.passthrough.as_ref(), ); self.swapchains[0].release_image().unwrap(); self.swapchains[1].release_image().unwrap(); if !buffer_ptr.is_null() && let Some(xr_now) = crate::xr_runtime_now(self.xr_session.instance()) { self.core_context.report_submit( timestamp, vsync_time.saturating_sub(Duration::from_nanos(xr_now.as_nanos() as u64)), ); } let rect = xr::Rect2Di { offset: xr::Offset2Di { x: 0, y: 0 }, extent: xr::Extent2Di { width: self.target_view_resolution.x as _, height: self.target_view_resolution.y as _, }, }; let clientside_post_processing = self .xr_session .instance() .exts() .fb_composition_layer_settings .and(self.config.clientside_post_processing.clone()); let layer = ProjectionLayerBuilder::new( &self.stage_reference_space, [ xr::CompositionLayerProjectionView::new() .pose(crate::to_xr_pose(output_view_params[0].pose)) .fov(crate::to_xr_fov(output_view_params[0].fov)) .sub_image( xr::SwapchainSubImage::new() .swapchain(&self.swapchains[0]) .image_array_index(0) .image_rect(rect), ), xr::CompositionLayerProjectionView::new() .pose(crate::to_xr_pose(output_view_params[1].pose)) .fov(crate::to_xr_fov(output_view_params[1].fov)) .sub_image( xr::SwapchainSubImage::new() .swapchain(&self.swapchains[1]) .image_array_index(0) .image_rect(rect), ), ], self.config .passthrough .clone() .map(|mode| ProjectionLayerAlphaConfig { premultiplied: matches!( mode, PassthroughMode::Blend { premultiplied_alpha: true, .. } | PassthroughMode::RgbChromaKey(_) | PassthroughMode::HsvChromaKey(_) ), }), clientside_post_processing, ); (layer, openxr_display_time) } } impl Drop for StreamContext { fn drop(&mut self) { self.input_thread_running.set(false); self.input_thread.take().unwrap().join().ok(); } } fn stream_input_loop( core_ctx: &ClientCoreContext, xr_session: xr::Session, interaction_ctx: &RwLock, stage_reference_space: &xr::Space, view_reference_space: &xr::Space, refresh_rate: f32, running: Arc, ) { let mut last_controller_poses = [Pose::IDENTITY; 2]; let mut last_palm_poses = [Pose::IDENTITY; 2]; let mut last_view_params = [ViewParams::DUMMY; 2]; let mut deadline = Instant::now(); let frame_interval = Duration::from_secs_f32(1.0 / refresh_rate); while running.value() { let int_ctx = &*interaction_ctx.read(); // Streaming related inputs are updated here. Make sure every input poll is done in this // thread if let Err(e) = xr_session.sync_actions(&[(&int_ctx.action_set).into()]) { error!("{e}"); return; } let Some(now) = crate::xr_runtime_now(xr_session.instance()).map(crate::from_xr_time) else { error!("Cannot poll tracking: invalid time"); return; }; let target_time = now + core_ctx.get_total_prediction_offset(); let Some((head_motion, local_views)) = interaction::get_head_data( &xr_session, core_ctx.platform(), stage_reference_space, view_reference_space, now, target_time, &last_view_params, ) else { continue; }; if let Some(views) = local_views { core_ctx.send_view_params(views); last_view_params = views; } let mut device_motions = Vec::with_capacity(3); device_motions.push((*HEAD_ID, head_motion)); let left_hand_data = crate::interaction::get_hand_data( &xr_session, core_ctx.platform(), stage_reference_space, now, target_time, &int_ctx.hands_interaction[0], &mut last_controller_poses[0], &mut last_palm_poses[0], ); let right_hand_data = crate::interaction::get_hand_data( &xr_session, core_ctx.platform(), stage_reference_space, now, target_time, &int_ctx.hands_interaction[1], &mut last_controller_poses[1], &mut last_palm_poses[1], ); // Note: When multimodal input is enabled, we are sure that when free hands are used // (not holding controllers) the controller data is None. if (int_ctx.multimodal_hands_enabled || left_hand_data.skeleton_joints.is_none()) && let Some(motion) = left_hand_data.grip_motion { device_motions.push((*HAND_LEFT_ID, motion)); } if (int_ctx.multimodal_hands_enabled || right_hand_data.skeleton_joints.is_none()) && let Some(motion) = right_hand_data.grip_motion { device_motions.push((*HAND_RIGHT_ID, motion)); } if int_ctx.multimodal_hands_enabled && let Some(detached_controller) = left_hand_data.detached_grip_motion { device_motions.push((*DETACHED_CONTROLLER_LEFT_ID, detached_controller)); } if int_ctx.multimodal_hands_enabled && let Some(detached_controller) = right_hand_data.detached_grip_motion { device_motions.push((*DETACHED_CONTROLLER_RIGHT_ID, detached_controller)); } let face = interaction::get_face_data( &xr_session, &int_ctx.face_sources, view_reference_space, now, ); let body = int_ctx .body_source .as_ref() .and_then(|source| interaction::get_body_skeleton(source, stage_reference_space, now)); if let Some(source) = &int_ctx.body_source { device_motions.append(&mut interaction::get_bd_motion_trackers(source, now)); } // Even though the server is already adding the motion-to-photon latency, here we use // target_time as the poll_timestamp to compensate for the fact that video frames are sent // with the poll timestamp instead of the vsync time. This is to ensure correctness when // submitting frames to OpenXR. This won't cause any desync with the server because no time // sync step is performed between client and server. core_ctx.send_tracking(TrackingData { poll_timestamp: target_time, device_motions, hand_skeletons: [ left_hand_data.skeleton_joints, right_hand_data.skeleton_joints, ], face, body, }); let button_entries = interaction::update_buttons(&xr_session, &int_ctx.button_actions); if !button_entries.is_empty() { core_ctx.send_buttons(button_entries); } deadline += frame_interval / 3; thread::sleep(deadline.saturating_duration_since(Instant::now())); } } ================================================ FILE: alvr/common/Cargo.toml ================================================ [package] name = "alvr_common" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] anyhow = { version = "1", features = ["backtrace"] } backtrace = "0.3" glam = { version = "0.30", features = ["serde"] } log = "0.4" parking_lot = "0.12" paste = "1" semver = { version = "1", features = ["serde"] } serde = { version = "1", features = ["derive"] } settings-schema = { git = "https://github.com/alvr-org/settings-schema-rs", rev = "676185f" } # settings-schema = { path = "../../../../settings-schema-rs/settings-schema" } ================================================ FILE: alvr/common/LICENSE ================================================ Copyright (c) 2020-2021 alvr-org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: alvr/common/README.md ================================================ # alvr_common This crate contains some basic functionality shared across all crates in the workspace. In particular it contains constants, logging front-end, and re-exports frequently-used external dependencies. ================================================ FILE: alvr/common/src/average.rs ================================================ use std::{collections::VecDeque, time::Duration}; pub struct SlidingWindowAverage { history_buffer: VecDeque, max_history_size: usize, } impl SlidingWindowAverage { pub fn new(initial_value: T, max_history_size: usize) -> Self { Self { history_buffer: [initial_value].into_iter().collect(), max_history_size, } } pub fn submit_sample(&mut self, sample: T) { if self.history_buffer.len() >= self.max_history_size { self.history_buffer.pop_front(); } self.history_buffer.push_back(sample); } pub fn retain(&mut self, count: usize) { self.history_buffer .drain(0..self.history_buffer.len().saturating_sub(count)); } } impl SlidingWindowAverage { pub fn get_average(&self) -> f32 { self.history_buffer.iter().sum::() / self.history_buffer.len() as f32 } } impl SlidingWindowAverage { pub fn get_average(&self) -> Duration { self.history_buffer.iter().sum::() / self.history_buffer.len() as u32 } } ================================================ FILE: alvr/common/src/c_api.rs ================================================ use glam::{Quat, Vec3}; use crate::{Fov, Pose, ViewParams}; #[repr(C)] pub struct AlvrFov { /// Negative, radians pub left: f32, /// Positive, radians pub right: f32, /// Positive, radians pub up: f32, /// Negative, radians pub down: f32, } #[repr(C)] #[derive(Clone, Default)] pub struct AlvrQuat { pub x: f32, pub y: f32, pub z: f32, pub w: f32, } #[repr(C)] #[derive(Clone, Default)] pub struct AlvrPose { pub orientation: AlvrQuat, pub position: [f32; 3], } #[repr(C)] pub struct AlvrViewParams { pub pose: AlvrPose, pub fov: AlvrFov, } #[repr(u8)] pub enum AlvrCodecType { H264 = 0, Hevc = 1, AV1 = 2, } pub fn to_capi_fov(fov: &Fov) -> AlvrFov { AlvrFov { left: fov.left, right: fov.right, up: fov.up, down: fov.down, } } pub fn from_capi_fov(fov: &AlvrFov) -> Fov { Fov { left: fov.left, right: fov.right, up: fov.up, down: fov.down, } } pub fn from_capi_quat(quat: &AlvrQuat) -> Quat { Quat::from_xyzw(quat.x, quat.y, quat.z, quat.w) } pub fn to_capi_quat(quat: &Quat) -> AlvrQuat { AlvrQuat { x: quat.x, y: quat.y, z: quat.z, w: quat.w, } } pub fn to_capi_pose(pose: &Pose) -> AlvrPose { AlvrPose { orientation: to_capi_quat(&pose.orientation), position: pose.position.to_array(), } } pub fn from_capi_pose(pose: &AlvrPose) -> Pose { Pose { orientation: from_capi_quat(&pose.orientation), position: Vec3::from_slice(&pose.position), } } pub fn to_capi_view_params(view_params: &ViewParams) -> AlvrViewParams { AlvrViewParams { pose: to_capi_pose(&view_params.pose), fov: to_capi_fov(&view_params.fov), } } pub fn from_capi_view_params(view_params: &AlvrViewParams) -> ViewParams { ViewParams { pose: from_capi_pose(&view_params.pose), fov: from_capi_fov(&view_params.fov), } } ================================================ FILE: alvr/common/src/connection_result.rs ================================================ use anyhow::{Result, anyhow}; use std::{ error::Error, fmt::Display, io, sync::mpsc::{RecvTimeoutError, TryRecvError}, }; pub enum ConnectionError { TryAgain(anyhow::Error), Other(anyhow::Error), } impl Display for ConnectionError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { let (ConnectionError::TryAgain(e) | ConnectionError::Other(e)) = self; write!(f, "{e:?}") } } pub type ConResult = Result; pub fn try_again() -> ConResult { Err(ConnectionError::TryAgain(anyhow!("Try again"))) } #[macro_export] macro_rules! con_bail { ($($args:tt)+) => { return Err(alvr_common::ConnectionError::Other(alvr_common::anyhow::anyhow!($($args)+))) }; } pub trait ToCon { /// Convert result to ConResult. The error is always mapped to `Other()` fn to_con(self) -> ConResult; } impl ToCon for Option { fn to_con(self) -> ConResult { self.ok_or_else(|| ConnectionError::Other(anyhow!("Unexpected None"))) } } impl ToCon for Result { fn to_con(self) -> ConResult { self.map_err(|e| ConnectionError::Other(e.into())) } } pub trait AnyhowToCon { fn to_con(self) -> ConResult; } impl AnyhowToCon for Result { fn to_con(self) -> ConResult { self.map_err(ConnectionError::Other) } } pub trait HandleTryAgain { fn handle_try_again(self) -> ConResult; } impl HandleTryAgain for io::Result { fn handle_try_again(self) -> ConResult { self.map_err(|e| { // Ignore ERROR_IO_PENDING on Windows (code 997) if e.kind() == io::ErrorKind::TimedOut || e.kind() == io::ErrorKind::WouldBlock || (cfg!(windows) && e.raw_os_error() == Some(997)) { ConnectionError::TryAgain(e.into()) } else { ConnectionError::Other(e.into()) } }) } } impl HandleTryAgain for std::result::Result { fn handle_try_again(self) -> ConResult { self.map_err(|e| match e { RecvTimeoutError::Timeout => ConnectionError::TryAgain(e.into()), RecvTimeoutError::Disconnected => ConnectionError::Other(e.into()), }) } } impl HandleTryAgain for std::result::Result { fn handle_try_again(self) -> ConResult { self.map_err(|e| match e { TryRecvError::Empty => ConnectionError::TryAgain(e.into()), TryRecvError::Disconnected => ConnectionError::Other(e.into()), }) } } ================================================ FILE: alvr/common/src/inputs.rs ================================================ use crate::hash_string; use std::{ collections::{HashMap, HashSet}, sync::LazyLock, }; macro_rules! interaction_profile { ($ty:ident, $path:expr) => { paste::paste! { pub const [<$ty _CONTROLLER_PROFILE_PATH>]: &str = concat!("/interaction_profiles/", $path, "_controller"); pub static [<$ty _CONTROLLER_PROFILE_ID>]: LazyLock = LazyLock::new(|| hash_string([<$ty _CONTROLLER_PROFILE_PATH>])); } }; } interaction_profile!(QUEST, "oculus/touch"); interaction_profile!(VIVE, "htc/vive"); interaction_profile!(INDEX, "valve/index"); interaction_profile!(PICO_NEO3, "bytedance/pico_neo3"); interaction_profile!(PICO4, "bytedance/pico4"); interaction_profile!(PICO4S, "bytedance/pico4s"); interaction_profile!(PICO_G3, "bytedance/pico_g3"); interaction_profile!(PSVR2, "sony/playstation_vr2_sense"); interaction_profile!(FOCUS3, "htc/vive_focus3"); interaction_profile!(YVR, "yvr/touch"); macro_rules! devices { ($(($name:ident, $path:expr),)*) => { paste::paste! { $( pub const [<$name _PATH>]: &str = $path; pub static [<$name _ID>]: LazyLock = LazyLock::new(|| hash_string([<$name _PATH>])); )* pub static DEVICE_ID_TO_PATH: LazyLock> = LazyLock::new(|| { [ $((*[<$name _ID>], [<$name _PATH>]),)* ] .into_iter() .collect() }); } }; } devices! { (HEAD, "/user/head"), (HAND_LEFT, "/user/hand/left"), (HAND_RIGHT, "/user/hand/right"), (HAND_TRACKER_LEFT,"/user/hand_tracker/left"), (HAND_TRACKER_RIGHT, "/user/hand_tracker/right"), (BODY_CHEST, "/user/body/chest"), (BODY_HIPS, "/user/body/waist"), (BODY_LEFT_ELBOW, "/user/body/left_elbow"), (BODY_RIGHT_ELBOW, "/user/body/right_elbow"), (BODY_LEFT_KNEE, "/user/body/left_knee"), (BODY_LEFT_FOOT, "/user/body/left_foot"), (BODY_RIGHT_KNEE, "/user/body/right_knee"), (BODY_RIGHT_FOOT, "/user/body/right_foot"), (DETACHED_CONTROLLER_LEFT, "/user/detached_controller_meta/left"), (DETACHED_CONTROLLER_RIGHT, "/user/detached_controller_meta/right"), (GENERIC_TRACKER_1, "/user/generic_tracker/1"), (GENERIC_TRACKER_2, "/user/generic_tracker/2"), (GENERIC_TRACKER_3, "/user/generic_tracker/3"), } pub enum ButtonType { Binary, Scalar, } pub struct ButtonInfo { pub path: &'static str, pub button_type: ButtonType, pub device_id: u64, } macro_rules! controller_inputs { ($(($inputs:ident, $paths:literal, $ty:ident),)*) => { paste::paste! { $( pub const []: &str = concat!("/user/hand/left/input/", $paths); pub static []: LazyLock = LazyLock::new(|| hash_string([])); pub const []: &str = concat!("/user/hand/right/input/", $paths); pub static []: LazyLock = LazyLock::new(|| hash_string([])); )* pub static BUTTON_INFO: LazyLock> = LazyLock::new(|| { [ $(( *[], ButtonInfo { path: [], button_type: ButtonType::$ty, device_id: *HAND_LEFT_ID, }, ), ( *[], ButtonInfo { path: [], button_type: ButtonType::$ty, device_id: *HAND_RIGHT_ID, }, ),)* ] .into_iter() .collect() }); } }; } controller_inputs! { (SYSTEM_CLICK, "system/click", Binary), (SYSTEM_TOUCH, "system/touch", Binary), (MENU_CLICK, "menu/click", Binary), (MENU_TOUCH, "menu/touch", Binary), (BACK_CLICK, "back/click", Binary), (A_CLICK, "a/click", Binary), (A_TOUCH, "a/touch", Binary), (B_CLICK, "b/click", Binary), (B_TOUCH, "b/touch", Binary), (X_CLICK, "x/click", Binary), (X_TOUCH, "x/touch", Binary), (Y_CLICK, "y/click", Binary), (Y_TOUCH, "y/touch", Binary), (SQUEEZE_CLICK, "squeeze/click", Binary), (SQUEEZE_TOUCH, "squeeze/touch", Binary), (SQUEEZE_VALUE, "squeeze/value", Scalar), (SQUEEZE_FORCE, "squeeze/force", Scalar), (TRIGGER_CLICK, "trigger/click", Binary), (TRIGGER_VALUE, "trigger/value", Scalar), (TRIGGER_TOUCH, "trigger/touch", Binary), (THUMBSTICK_X, "thumbstick/x", Scalar), (THUMBSTICK_Y, "thumbstick/y", Scalar), (THUMBSTICK_CLICK, "thumbstick/click", Binary), (THUMBSTICK_TOUCH, "thumbstick/touch", Binary), (TRACKPAD_X, "trackpad/x", Scalar), (TRACKPAD_Y, "trackpad/y", Scalar), (TRACKPAD_CLICK, "trackpad/click", Binary), (TRACKPAD_FORCE, "trackpad/force", Scalar), (TRACKPAD_TOUCH, "trackpad/touch", Binary), (THUMBREST_TOUCH, "thumbrest/touch", Binary), } pub struct InteractionProfileInfo { pub path: &'static str, pub button_set: HashSet, } pub static CONTROLLER_PROFILE_INFO: LazyLock> = LazyLock::new(|| { [ ( *QUEST_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: QUEST_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_THUMBREST_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_THUMBREST_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *VIVE_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: VIVE_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_SYSTEM_CLICK_ID, *LEFT_SQUEEZE_CLICK_ID, *LEFT_MENU_CLICK_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRACKPAD_X_ID, *LEFT_TRACKPAD_Y_ID, *LEFT_TRACKPAD_CLICK_ID, *LEFT_TRACKPAD_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_MENU_CLICK_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRACKPAD_X_ID, *RIGHT_TRACKPAD_Y_ID, *RIGHT_TRACKPAD_CLICK_ID, *RIGHT_TRACKPAD_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *INDEX_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: INDEX_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_SYSTEM_CLICK_ID, *LEFT_SYSTEM_TOUCH_ID, *LEFT_A_CLICK_ID, *LEFT_A_TOUCH_ID, *LEFT_B_CLICK_ID, *LEFT_B_TOUCH_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_SQUEEZE_FORCE_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_TRACKPAD_X_ID, *LEFT_TRACKPAD_Y_ID, *LEFT_TRACKPAD_FORCE_ID, *LEFT_TRACKPAD_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SYSTEM_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_SQUEEZE_FORCE_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_TRACKPAD_X_ID, *RIGHT_TRACKPAD_Y_ID, *RIGHT_TRACKPAD_FORCE_ID, *RIGHT_TRACKPAD_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *PICO_G3_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: PICO_G3_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_MENU_CLICK_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRACKPAD_Y_ID, *LEFT_TRACKPAD_X_ID, *LEFT_TRACKPAD_CLICK_ID, *LEFT_TRACKPAD_TOUCH_ID, *RIGHT_MENU_CLICK_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRACKPAD_Y_ID, *RIGHT_TRACKPAD_X_ID, *RIGHT_TRACKPAD_CLICK_ID, *RIGHT_TRACKPAD_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *PICO_NEO3_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: PICO_NEO3_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_SYSTEM_CLICK_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_SQUEEZE_CLICK_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_THUMBREST_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_MENU_CLICK_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_THUMBREST_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *PICO4_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: PICO4_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_SYSTEM_CLICK_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_SQUEEZE_CLICK_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_THUMBREST_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_THUMBREST_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *PICO4S_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: PICO4S_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_SYSTEM_CLICK_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_SQUEEZE_CLICK_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_THUMBREST_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_THUMBREST_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *PSVR2_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: PSVR2_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_MENU_TOUCH_ID, *LEFT_SYSTEM_CLICK_ID, *LEFT_SYSTEM_TOUCH_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_SQUEEZE_CLICK_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_SQUEEZE_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SYSTEM_TOUCH_ID, *RIGHT_MENU_CLICK_ID, *RIGHT_MENU_TOUCH_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_SQUEEZE_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *FOCUS3_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: FOCUS3_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_Y_CLICK_ID, *LEFT_MENU_CLICK_ID, *LEFT_SQUEEZE_CLICK_ID, // *LEFT_SQUEEZE_TOUCH_ID, // not actually working *LEFT_SQUEEZE_VALUE_ID, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_THUMBREST_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_B_CLICK_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SQUEEZE_CLICK_ID, // *RIGHT_SQUEEZE_TOUCH_ID, // not actually working *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_THUMBREST_TOUCH_ID, ] .into_iter() .collect(), }, ), ( *YVR_CONTROLLER_PROFILE_ID, InteractionProfileInfo { path: YVR_CONTROLLER_PROFILE_PATH, button_set: [ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_SQUEEZE_CLICK_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *LEFT_THUMBREST_TOUCH_ID, // might not actually be present? *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, *RIGHT_THUMBREST_TOUCH_ID, // might not actually be present? ] .into_iter() .collect(), }, ), ] .into_iter() .collect() }); ================================================ FILE: alvr/common/src/lib.rs ================================================ mod average; mod c_api; mod connection_result; mod inputs; mod logging; mod primitives; mod version; use parking_lot::{Condvar, Mutex, RwLockWriteGuard}; use serde::{Deserialize, Serialize}; use std::sync::atomic::{AtomicBool, Ordering}; pub use anyhow; pub use glam; pub use log; pub use parking_lot; pub use semver; pub use settings_schema; pub use average::*; pub use c_api::*; pub use connection_result::*; pub use inputs::*; pub use log::{debug, error, info, warn}; pub use logging::*; pub use primitives::*; pub use version::*; pub const ALVR_NAME: &str = "ALVR"; // Simple wrapper for AtomicBool when using Ordering::Relaxed. Deref cannot be implemented (cannot // return local reference) #[derive(Default)] pub struct RelaxedAtomic(AtomicBool); impl RelaxedAtomic { pub const fn new(initial_value: bool) -> Self { Self(AtomicBool::new(initial_value)) } pub fn value(&self) -> bool { self.0.load(Ordering::Relaxed) } pub fn set(&self, value: bool) { self.0.store(value, Ordering::Relaxed); } } #[derive(PartialEq, Eq, Debug)] pub enum LifecycleState { StartingUp, Idle, Resumed, ShuttingDown, } #[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq, Default)] pub enum ConnectionState { #[default] Disconnected, Connecting, Connected, Streaming, Disconnecting, } pub fn wait_rwlock(condvar: &Condvar, guard: &mut RwLockWriteGuard<'_, T>) { let staging_mutex = Mutex::<()>::new(()); let mut inner_guard = staging_mutex.lock(); RwLockWriteGuard::unlocked(guard, move || { condvar.wait(&mut inner_guard); }); } ================================================ FILE: alvr/common/src/logging.rs ================================================ use anyhow::Result; use backtrace::Backtrace; use parking_lot::Mutex; use serde::{Deserialize, Serialize}; use settings_schema::SettingsSchema; use std::{error::Error, fmt::Display, sync::OnceLock}; pub const SERVER_IMPL_DBG_LABEL: &str = "SERVER IMPL"; pub const CLIENT_IMPL_DBG_LABEL: &str = "CLIENT IMPL"; pub const SERVER_CORE_DBG_LABEL: &str = "SERVER CORE"; pub const CLIENT_CORE_DBG_LABEL: &str = "CLIENT CORE"; pub const CONNECTION_DBG_LABEL: &str = "CONNECTION"; pub const SOCKETS_DBG_LABEL: &str = "SOCKETS"; pub const SERVER_GFX_DBG_LABEL: &str = "SERVER GFX"; pub const CLIENT_GFX_DBG_LABEL: &str = "CLIENT GFX"; pub const ENCODER_DBG_LABEL: &str = "ENCODER"; pub const DECODER_DBG_LABEL: &str = "DECODER"; static POPUP_CALLBACK: OnceLock = OnceLock::new(); #[macro_export] macro_rules! dbg_server_impl { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::SERVER_IMPL_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_client_impl { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::CLIENT_IMPL_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_server_core { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::SERVER_CORE_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_client_core { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::CLIENT_CORE_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_connection { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::CONNECTION_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_sockets { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::SOCKETS_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_server_gfx { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::SERVER_GFX_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_client_gfx { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::CLIENT_GFX_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_encoder { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::ENCODER_DBG_LABEL, $($args)*); }; } #[macro_export] macro_rules! dbg_decoder { ($($args:tt)*) => { #[cfg(debug_assertions)] $crate::log::debug!(target: $crate::DECODER_DBG_LABEL, $($args)*); }; } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct DebugGroupsConfig { #[schema(flag = "steamvr-restart")] pub server_impl: bool, #[schema(flag = "steamvr-restart")] pub client_impl: bool, #[schema(flag = "steamvr-restart")] pub server_core: bool, #[schema(flag = "steamvr-restart")] pub client_core: bool, #[schema(flag = "steamvr-restart")] pub connection: bool, #[schema(flag = "steamvr-restart")] pub sockets: bool, #[schema(flag = "steamvr-restart")] pub server_gfx: bool, #[schema(flag = "steamvr-restart")] pub client_gfx: bool, #[schema(flag = "steamvr-restart")] pub encoder: bool, #[schema(flag = "steamvr-restart")] pub decoder: bool, } pub fn filter_debug_groups(target: &str, config: &DebugGroupsConfig) -> bool { match target { SERVER_IMPL_DBG_LABEL => config.server_impl, CLIENT_IMPL_DBG_LABEL => config.client_impl, SERVER_CORE_DBG_LABEL => config.server_core, CLIENT_CORE_DBG_LABEL => config.client_core, CONNECTION_DBG_LABEL => config.connection, SOCKETS_DBG_LABEL => config.sockets, SERVER_GFX_DBG_LABEL => config.server_gfx, CLIENT_GFX_DBG_LABEL => config.client_gfx, ENCODER_DBG_LABEL => config.encoder, DECODER_DBG_LABEL => config.decoder, _ => true, } } pub fn is_enabled_debug_group(target: &str, config: &DebugGroupsConfig) -> bool { (config.server_impl && target == SERVER_IMPL_DBG_LABEL) || (config.client_impl && target == CLIENT_IMPL_DBG_LABEL) || (config.server_core && target == SERVER_CORE_DBG_LABEL) || (config.client_core && target == CLIENT_CORE_DBG_LABEL) || (config.connection && target == CONNECTION_DBG_LABEL) || (config.sockets && target == SOCKETS_DBG_LABEL) || (config.server_gfx && target == SERVER_GFX_DBG_LABEL) || (config.client_gfx && target == CLIENT_GFX_DBG_LABEL) || (config.encoder && target == ENCODER_DBG_LABEL) || (config.decoder && target == DECODER_DBG_LABEL) } #[derive( SettingsSchema, Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, )] pub enum LogSeverity { Error = 3, Warning = 2, Info = 1, Debug = 0, } impl LogSeverity { pub fn from_log_level(level: log::Level) -> Self { match level { log::Level::Error => LogSeverity::Error, log::Level::Warn => LogSeverity::Warning, log::Level::Info => LogSeverity::Info, log::Level::Debug | log::Level::Trace => LogSeverity::Debug, } } pub fn into_log_level(self) -> log::Level { match self { LogSeverity::Error => log::Level::Error, LogSeverity::Warning => log::Level::Warn, LogSeverity::Info => log::Level::Info, LogSeverity::Debug => log::Level::Debug, } } } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct LogEntry { pub severity: LogSeverity, pub content: String, } // The callback has parameters in order: title, message, severity pub fn set_popup_callback(callback: fn(&str, &str, LogSeverity)) { POPUP_CALLBACK.set(callback).ok(); } pub fn set_panic_hook() { std::panic::set_hook(Box::new(|panic_info| { let err_str = format!( "What happened:\n{panic_info}\n\nBacktrace:\n{:?}", Backtrace::new() ); log::error!("ALVR panicked: {err_str}"); std::thread::spawn({ let panic_str = panic_info.to_string(); move || { if let Some(callback) = POPUP_CALLBACK.get() { callback("ALVR panicked", &panic_str, LogSeverity::Error); } } }); })) } pub fn show_w(w: W) { log::warn!("{w}"); std::thread::spawn(move || { if let Some(callback) = POPUP_CALLBACK.get() { callback("ALVR warning", &w.to_string(), LogSeverity::Warning); } }); } pub fn show_warn(res: Result) -> Option { res.map_err(show_w).ok() } fn show_e_block(e: E, blocking: bool) { log::error!("{e}"); // Store the last error shown in a message box. Do not open a new message box if the content // of the error has not changed static LAST_MESSAGEBOX_ERROR: Mutex = Mutex::new(String::new()); let err_string = e.to_string(); let last_messagebox_error_ref = &mut *LAST_MESSAGEBOX_ERROR.lock(); if *last_messagebox_error_ref != err_string { let show_msgbox = { let err_string = err_string.clone(); move || { if let Some(callback) = POPUP_CALLBACK.get() { callback("ALVR error", &err_string, LogSeverity::Error); } } }; if blocking { show_msgbox(); } else { std::thread::spawn(show_msgbox); } *last_messagebox_error_ref = err_string; } } pub fn show_e(e: E) { show_e_block(e, false); } pub fn show_e_dbg(e: E) { show_e_block(format!("{e:?}"), false); } pub fn show_e_blocking(e: E) { show_e_block(e, true); } pub fn show_err(res: Result) -> Option { res.map_err(|e| show_e_block(e, false)).ok() } pub fn show_err_blocking(res: Result) -> Option { res.map_err(|e| show_e_block(e, true)).ok() } pub trait ToAny { fn to_any(self) -> Result; } impl ToAny for Option { fn to_any(self) -> Result { match self { Some(value) => Ok(value), None => Err(anyhow::anyhow!("Unexpected None")), } } } impl ToAny for Result { fn to_any(self) -> Result { match self { Ok(value) => Ok(value), Err(e) => Err(e.into()), } } } ================================================ FILE: alvr/common/src/primitives.rs ================================================ use glam::{Quat, Vec3}; use serde::{Deserialize, Serialize}; use std::{ops::Mul, time::Duration}; // Field of view in radians #[derive(Serialize, Deserialize, PartialEq, Debug, Clone, Copy)] pub struct Fov { pub left: f32, pub right: f32, pub up: f32, pub down: f32, } impl Fov { pub const DUMMY: Self = Fov { left: -1.0, right: 1.0, up: 1.0, down: -1.0, }; } #[derive(Serialize, Deserialize, Clone, Copy, Default, Debug)] pub struct Pose { pub orientation: Quat, pub position: Vec3, } impl Pose { pub const IDENTITY: Self = Pose { orientation: Quat::IDENTITY, position: Vec3::ZERO, }; pub fn inverse(&self) -> Pose { let inverse_orientation = self.orientation.conjugate(); Pose { orientation: inverse_orientation, position: inverse_orientation * -self.position, } } } impl Mul for Pose { type Output = Pose; fn mul(self, rhs: Pose) -> Pose { Pose { orientation: self.orientation * rhs.orientation, position: self.position + self.orientation * rhs.position, } } } #[derive(Serialize, Deserialize, Clone, Copy, Default, Debug)] pub struct DeviceMotion { pub pose: Pose, pub linear_velocity: Vec3, pub angular_velocity: Vec3, } impl Mul for Pose { type Output = DeviceMotion; fn mul(self, rhs: DeviceMotion) -> DeviceMotion { DeviceMotion { pose: self * rhs.pose, linear_velocity: self.orientation * rhs.linear_velocity, angular_velocity: self.orientation * rhs.angular_velocity, } } } // Calculate difference ensuring maximum precision is preserved fn difference_seconds(from: Duration, to: Duration) -> f32 { to.saturating_sub(from).as_secs_f32() - from.saturating_sub(to).as_secs_f32() } impl DeviceMotion { pub const IDENTITY: Self = DeviceMotion { pose: Pose::IDENTITY, linear_velocity: Vec3::ZERO, angular_velocity: Vec3::ZERO, }; pub fn predict(&self, from_timestamp: Duration, to_timestamp: Duration) -> Self { let delta_time_s = difference_seconds(from_timestamp, to_timestamp); let delta_position = self.linear_velocity * delta_time_s; let delta_orientation = Quat::from_scaled_axis(self.angular_velocity * delta_time_s); DeviceMotion { pose: Pose { orientation: delta_orientation * self.pose.orientation, position: self.pose.position + delta_position, }, linear_velocity: self.linear_velocity, angular_velocity: self.angular_velocity, } } } // Per eye view parameters #[derive(Serialize, Deserialize, Clone, Copy)] pub struct ViewParams { pub pose: Pose, pub fov: Fov, } impl ViewParams { pub const DUMMY: Self = ViewParams { pose: Pose::IDENTITY, fov: Fov::DUMMY, }; } #[derive(Serialize, Deserialize, Clone)] pub struct BodySkeletonFb { pub upper_body: [Option; 18], pub lower_body: Option<[Option; 14]>, } #[derive(Serialize, Deserialize, Clone)] pub struct BodySkeletonBd(pub [Option; 24]); #[derive(Serialize, Deserialize, Clone)] pub enum BodySkeleton { Fb(Box), Bd(Box), } ================================================ FILE: alvr/common/src/version.rs ================================================ use semver::Version; use std::{ collections::hash_map::DefaultHasher, hash::{Hash, Hasher}, sync::LazyLock, }; pub static ALVR_VERSION: LazyLock = LazyLock::new(|| Version::parse(env!("CARGO_PKG_VERSION")).unwrap()); // Consistent across architectures, might not be consistent across different compiler versions. pub fn hash_string(string: &str) -> u64 { let mut hasher = DefaultHasher::new(); string.hash(&mut hasher); hasher.finish() } pub fn is_nightly() -> bool { ALVR_VERSION.build.contains("nightly") } pub fn is_stable() -> bool { ALVR_VERSION.pre.is_empty() && !is_nightly() } // Semver compatible versions will produce the same protocol ID. Protocol IDs are not ordered // As a convention, encode/decode the protocol ID bytes as little endian. // Only makor and pub fn protocol_id() -> String { if ALVR_VERSION.pre.is_empty() { ALVR_VERSION.major.to_string() } else { format!("{}-{}", ALVR_VERSION.major, ALVR_VERSION.pre) } } pub fn protocol_id_u64() -> u64 { hash_string(&protocol_id()) } // deprecated pub fn is_version_compatible(other_version: &Version) -> bool { let protocol_string = if other_version.pre.is_empty() { other_version.major.to_string() } else { format!("{}-{}", other_version.major, other_version.pre) }; protocol_id_u64() == hash_string(&protocol_string) } ================================================ FILE: alvr/dashboard/Cargo.toml ================================================ [package] name = "alvr_dashboard" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_adb.workspace = true alvr_common.workspace = true alvr_events.workspace = true alvr_filesystem.workspace = true alvr_packets.workspace = true alvr_session.workspace = true alvr_sockets.workspace = true alvr_gui_common.workspace = true alvr_audio.workspace = true chrono = "0.4" eframe = "0.32" env_logger = "0.11" ico = "0.4" rand = "0.9" serde = { version = "1", features = ["derive"] } serde_json = "1" settings-schema = { git = "https://github.com/alvr-org/settings-schema-rs", rev = "676185f" } statrs = "0.18" [target.'cfg(not(target_arch = "wasm32"))'.dependencies] alvr_server_io.workspace = true sysinfo = "0.37" tungstenite = "0.27" ureq = { version = "3", features = ["json"] } [target.'cfg(target_os = "linux")'.dependencies] wgpu = "25" libva = { package = "cros-libva", version = "0.0.7" } # Latest that works on Ubuntu 22.04 nvml-wrapper = "0.11.0" [target.'cfg(windows)'.build-dependencies] winres = "0.1" ================================================ FILE: alvr/dashboard/README.md ================================================ # alvr_gui Crate for GUI-related code. It needs a backend to display the UI. ================================================ FILE: alvr/dashboard/build.rs ================================================ #[cfg(windows)] fn main() { let mut resource = winres::WindowsResource::new(); resource.set_icon("resources/dashboard.ico"); resource.compile().unwrap(); } #[cfg(not(windows))] fn main() {} ================================================ FILE: alvr/dashboard/src/dashboard/components/about.rs ================================================ use alvr_common::ALVR_VERSION; use alvr_gui_common::theme; use eframe::egui::{Frame, RichText, ScrollArea, Ui}; pub fn about_tab_ui(ui: &mut Ui) { ui.label(RichText::new(format!("ALVR streamer v{}", *ALVR_VERSION)).size(30.0)); ui.add_space(10.0); ui.hyperlink_to("Visit us on GitHub", "https://github.com/alvr-org/ALVR"); ui.hyperlink_to("Join us on Discord", "https://discord.gg/ALVR"); ui.hyperlink_to( "Latest release", "https://github.com/alvr-org/ALVR/releases/latest", ); ui.hyperlink_to( "Donate to ALVR on Open Collective", "https://opencollective.com/alvr", ); ui.add_space(10.0); ui.label("License:"); Frame::group(ui.style()) .fill(theme::DARKER_BG) .inner_margin(theme::FRAME_PADDING) .show(ui, |ui| { ScrollArea::new([false, true]) .id_salt("license_scroll") .show(ui, |ui| ui.label(include_str!("../../../../../LICENSE"))) }); } ================================================ FILE: alvr/dashboard/src/dashboard/components/debug.rs ================================================ use crate::dashboard::ServerRequest; use eframe::egui::Ui; pub fn debug_tab_ui(ui: &mut Ui) -> Option { let mut request = None; ui.label( "Recording from ALVR using the buttons below is not suitable for capturing gameplay. For that, use other means of recording, for example through headset or desktop VR output.", ); ui.columns(4, |ui| { if ui[0].button("Capture frame").clicked() { request = Some(ServerRequest::CaptureFrame); } if ui[1].button("Insert IDR").clicked() { request = Some(ServerRequest::InsertIdr); } if ui[2].button("Start recording").clicked() { request = Some(ServerRequest::StartRecording); } if ui[3].button("Stop recording").clicked() { request = Some(ServerRequest::StopRecording); } }); request } ================================================ FILE: alvr/dashboard/src/dashboard/components/devices.rs ================================================ use crate::dashboard::ServerRequest; use alvr_common::ConnectionState; use alvr_gui_common::theme::{self, log_colors}; use alvr_packets::ClientConnectionsAction; use alvr_session::{ClientConnectionConfig, SessionConfig}; use alvr_sockets::WIRED_CLIENT_HOSTNAME; use eframe::{ egui::{self, Frame, Grid, Layout, ProgressBar, RichText, TextEdit, Ui, Window}, emath::{Align, Align2}, epaint::Color32, }; struct EditPopupState { new_devices: bool, hostname: String, ips: Vec, } pub struct DevicesTab { new_devices: Option>, trusted_devices: Option>, edit_popup_state: Option, adb_download_progress: Option, } impl DevicesTab { pub fn new() -> Self { Self { new_devices: None, trusted_devices: None, edit_popup_state: None, adb_download_progress: None, } } pub fn update_client_list(&mut self, session: &SessionConfig) { let (trusted_clients, untrusted_clients) = session .client_connections .clone() .into_iter() .partition::, _>(|(_, data)| data.trusted); self.trusted_devices = Some(trusted_clients); self.new_devices = Some(untrusted_clients); } pub fn update_adb_download_progress(&mut self, progress: f32) { self.adb_download_progress = Some(progress); } pub fn ui(&mut self, ui: &mut Ui, connected_to_server: bool) -> Vec { let mut requests = vec![]; if self.new_devices.is_none() { requests.push(ServerRequest::GetSession); } if !connected_to_server { Frame::group(ui.style()) .inner_margin(theme::FRAME_PADDING) .fill(log_colors::WARNING_LIGHT) .show(ui, |ui| { Grid::new(0).num_columns(2).show(ui, |ui| { ui.horizontal(|ui| { ui.add_space(theme::FRAME_TEXT_SPACING); ui.heading( RichText::new( "ALVR requires running SteamVR! \ Devices will not be discovered or connected.", ) .color(Color32::BLACK) .size(16.0), ); }); #[cfg(not(target_arch = "wasm32"))] ui.with_layout(Layout::right_to_left(eframe::emath::Align::Center), |ui| { if ui.button("Launch SteamVR").clicked() { crate::steamvr_launcher::LAUNCHER.lock().launch_steamvr(); } }); }) }); } ui.vertical_centered_justified(|ui| { if let Some(clients) = &mut self.trusted_devices && let Some(request) = wired_client_section( ui, clients .iter() .find(|(hostname, _)| hostname == WIRED_CLIENT_HOSTNAME), self.adb_download_progress, ) { requests.push(request); } ui.add_space(theme::FRAME_PADDING); if let Some(clients) = &self.new_devices && let Some(request) = new_clients_section(ui, clients) { requests.push(request); } ui.add_space(theme::FRAME_PADDING); if let Some(clients) = &mut self.trusted_devices && let Some(request) = trusted_clients_section( ui, clients .iter() .filter(|(hostname, _)| hostname != WIRED_CLIENT_HOSTNAME) .collect::>() .as_slice(), &mut self.edit_popup_state, ) { requests.push(request); } }); if let Some(mut state) = self.edit_popup_state.take() { Window::new("Edit connection") .anchor(Align2::CENTER_CENTER, (0.0, 0.0)) .resizable(false) .collapsible(false) .show(ui.ctx(), |ui| { ui.add_space(5.0); ui.columns(2, |ui| { ui[0].horizontal(|ui| { ui.add_space(5.0); ui.label("Hostname:"); }); ui[1].add_enabled( state.new_devices, TextEdit::singleline(&mut state.hostname), ); ui[0].horizontal(|ui| { ui.add_space(5.0); ui.label("IP Addresses:"); }); for address in &mut state.ips { ui[1].text_edit_singleline(address); } if ui[1].button("Add new").clicked() { state.ips.push("192.168.X.X".into()); } }); ui.columns(2, |ui| { if ui[0].button("Cancel").clicked() { return; } if ui[1].button("Save").clicked() { let manual_ips = state.ips.iter().filter_map(|s| s.parse().ok()).collect(); if state.new_devices { requests.push(ServerRequest::UpdateClientList { hostname: state.hostname, action: ClientConnectionsAction::AddIfMissing { trusted: true, manual_ips, }, }); } else { requests.push(ServerRequest::UpdateClientList { hostname: state.hostname, action: ClientConnectionsAction::SetManualIps(manual_ips), }); } } else { self.edit_popup_state = Some(state); } }) }); } requests } } fn wired_client_section( ui: &mut Ui, maybe_client: Option<&(String, ClientConnectionConfig)>, adb_download_progress: Option, ) -> Option { let mut request = None; Frame::group(ui.style()) .fill(theme::SECTION_BG) .inner_margin(egui::vec2( theme::FRAME_PADDING + theme::FRAME_TEXT_SPACING, theme::FRAME_PADDING, )) .show(ui, |ui| { ui.horizontal(|ui| { Grid::new("wired-client") .num_columns(2) .spacing(egui::vec2(8.0, 8.0)) .show(ui, |ui| { ui.heading("Wired Connection"); ui.with_layout(Layout::right_to_left(Align::Center), |ui| { let mut wired = maybe_client.is_some(); if alvr_gui_common::switch(ui, &mut wired).changed() { if wired { request = Some(ServerRequest::UpdateClientList { hostname: WIRED_CLIENT_HOSTNAME.to_owned(), action: ClientConnectionsAction::AddIfMissing { trusted: true, manual_ips: Vec::new(), }, }); } else { request = Some(ServerRequest::UpdateClientList { hostname: WIRED_CLIENT_HOSTNAME.to_owned(), action: ClientConnectionsAction::RemoveEntry, }); } } ui.horizontal(|ui| { ui.add_space(theme::FRAME_TEXT_SPACING); }); }); ui.end_row(); if let Some(progress) = adb_download_progress.filter(|p| *p < 1.0) { ui.horizontal(|ui| { ui.label("ADB download progress"); }); ui.horizontal(|ui| { ui.add(ProgressBar::new(progress).animate(true).show_percentage()); }); ui.end_row(); } else if let Some((_, data)) = maybe_client { ui.horizontal(|ui| { ui.label(&data.display_name); }); ui.with_layout(Layout::right_to_left(Align::Center), |ui| { connection_label(ui, &data.connection_state); }); ui.end_row(); } }); }); }); request } fn new_clients_section( ui: &mut Ui, clients: &[(String, ClientConnectionConfig)], ) -> Option { let mut request = None; Frame::group(ui.style()) .inner_margin(theme::FRAME_PADDING) .fill(theme::SECTION_BG) .show(ui, |ui| { ui.vertical_centered_justified(|ui| { ui.horizontal(|ui| { ui.add_space(theme::FRAME_TEXT_SPACING); ui.heading("New Wireless Devices"); // Extend to the right ui.with_layout(Layout::right_to_left(Align::Center), |_| ()); }); }); for (hostname, _) in clients { Frame::group(ui.style()) .fill(theme::DARKER_BG) .inner_margin(egui::vec2(15.0, 12.0)) .show(ui, |ui| { Grid::new(format!("{hostname}-new-clients")) .num_columns(2) .spacing(egui::vec2(8.0, 8.0)) .show(ui, |ui| { ui.horizontal(|ui| { ui.label(hostname); }); ui.with_layout(Layout::right_to_left(Align::Center), |ui| { if ui.button("Trust").clicked() { request = Some(ServerRequest::UpdateClientList { hostname: hostname.clone(), action: ClientConnectionsAction::Trust, }); }; }); ui.end_row(); }); }); } }); request } fn trusted_clients_section( ui: &mut Ui, clients: &[&(String, ClientConnectionConfig)], edit_popup_state: &mut Option, ) -> Option { let mut request = None; Frame::group(ui.style()) .fill(theme::SECTION_BG) .inner_margin(theme::FRAME_PADDING) .show(ui, |ui| { Grid::new(0).num_columns(2).show(ui, |ui| { ui.horizontal(|ui| { ui.add_space(theme::FRAME_TEXT_SPACING); ui.heading("Trusted Wireless Devices"); }); ui.with_layout(Layout::right_to_left(eframe::emath::Align::Center), |ui| { if ui.button("Add device manually").clicked() { *edit_popup_state = Some(EditPopupState { hostname: "XXXX.client.local.".into(), new_devices: true, ips: Vec::new(), }); } }); }); for (hostname, data) in clients { Frame::group(ui.style()) .fill(theme::DARKER_BG) .inner_margin(egui::vec2(15.0, 12.0)) .show(ui, |ui| { Grid::new(format!("{hostname}-clients")) .num_columns(2) .spacing(egui::vec2(8.0, 8.0)) .show(ui, |ui| { ui.label(&data.display_name); ui.horizontal(|ui| { ui.with_layout(Layout::right_to_left(Align::Center), |ui| { connection_label(ui, &data.connection_state) }); }); ui.end_row(); ui.label(format!( "{hostname}: {}", data.current_ip .map_or_else(|| "Unknown IP".into(), |ip| ip.to_string()), )); ui.with_layout(Layout::right_to_left(Align::Center), |ui| { if ui.button("Remove").clicked() { request = Some(ServerRequest::UpdateClientList { hostname: hostname.clone(), action: ClientConnectionsAction::RemoveEntry, }); } if ui.button("Edit").clicked() { *edit_popup_state = Some(EditPopupState { new_devices: false, hostname: hostname.to_owned(), ips: data .manual_ips .iter() .map(|addr| addr.to_string()) .collect::>(), }); } }); }); }); } }); request } fn connection_label(ui: &mut Ui, connection_state: &ConnectionState) { match connection_state { ConnectionState::Disconnected => ui.colored_label(Color32::GRAY, "Disconnected"), ConnectionState::Connecting => ui.colored_label(log_colors::WARNING_LIGHT, "Connecting"), ConnectionState::Connected => ui.colored_label(theme::OK_GREEN, "Connected"), ConnectionState::Streaming => ui.colored_label(theme::OK_GREEN, "Streaming"), ConnectionState::Disconnecting => { ui.colored_label(log_colors::WARNING_LIGHT, "Disconnecting") } }; } ================================================ FILE: alvr/dashboard/src/dashboard/components/installation.rs ================================================ use crate::dashboard::ServerRequest; use alvr_gui_common::theme; use eframe::egui::{Frame, Grid, RichText, ScrollArea, Ui}; use std::{ path::PathBuf, time::{Duration, Instant}, }; const DRIVER_UPDATE_INTERVAL: Duration = Duration::from_secs(1); pub enum InstallationTabRequest { OpenSetupWizard, ServerRequest(ServerRequest), } pub struct InstallationTab { drivers: Vec, last_update_instant: Instant, } impl InstallationTab { pub fn new() -> Self { Self { drivers: vec![], last_update_instant: Instant::now(), } } pub fn update_drivers(&mut self, list: Vec) { self.drivers = list; } pub fn ui(&mut self, ui: &mut Ui) -> Vec { let mut requests = vec![]; let now = Instant::now(); if now > self.last_update_instant + DRIVER_UPDATE_INTERVAL { requests.push(InstallationTabRequest::ServerRequest( ServerRequest::GetDriverList, )); self.last_update_instant = now; } ui.vertical_centered_justified(|ui| { if ui.button("Run setup wizard").clicked() { requests.push(InstallationTabRequest::OpenSetupWizard); } ui.columns(2, |ui| { if ui[0].button("Add firewall rules").clicked() { requests.push(InstallationTabRequest::ServerRequest( ServerRequest::AddFirewallRules, )); } if ui[1].button("Remove firewall rules").clicked() { requests.push(InstallationTabRequest::ServerRequest( ServerRequest::RemoveFirewallRules, )); } }); Frame::group(ui.style()) .fill(theme::SECTION_BG) .inner_margin(theme::FRAME_PADDING) .show(ui, |ui| { ui.vertical_centered_justified(|ui| { ui.label(RichText::new("Registered drivers").size(18.0)); }); Grid::new(0).num_columns(2).show(ui, |ui| { for driver_path in &self.drivers { if ui.button("Remove").clicked() { requests.push(InstallationTabRequest::ServerRequest( ServerRequest::UnregisterDriver(driver_path.clone()), )); } ScrollArea::new([true, false]) .auto_shrink([false, false]) .id_salt(driver_path) .show(ui, |ui| { ui.label(driver_path.to_string_lossy()); }); ui.end_row(); } }); if ui.button("Register ALVR driver").clicked() { requests.push(InstallationTabRequest::ServerRequest( ServerRequest::RegisterAlvrDriver, )); } }); }); requests } } ================================================ FILE: alvr/dashboard/src/dashboard/components/logs.rs ================================================ use alvr_common::LogSeverity; use alvr_events::{Event, EventType}; use alvr_gui_common::theme::log_colors; use alvr_session::{RawEventsConfig, Settings}; use eframe::{ egui::{Grid, OpenUrl, OutputCommand, RichText, ScrollArea, Ui}, epaint::Color32, }; use settings_schema::Switch; use std::collections::VecDeque; struct Entry { color: Color32, timestamp: String, ty: String, message: String, } pub struct LogsTab { raw_events_config: Switch, entries: VecDeque, log_limit: usize, } impl LogsTab { pub fn new() -> Self { Self { raw_events_config: Switch::Enabled(RawEventsConfig { hide_spammy_events: false, }), entries: VecDeque::new(), log_limit: 1000, } } pub fn update_settings(&mut self, settings: &Settings) { self.raw_events_config = settings.extra.logging.show_raw_events.clone(); } pub fn push_event(&mut self, event: Event) { let color = if let EventType::Log(entry) = &event.event_type { Some(match entry.severity { LogSeverity::Error => log_colors::ERROR_LIGHT, LogSeverity::Warning => log_colors::WARNING_LIGHT, LogSeverity::Info => log_colors::INFO_LIGHT, LogSeverity::Debug => log_colors::DEBUG_LIGHT, }) } else if let Switch::Enabled(config) = &self.raw_events_config { (!config.hide_spammy_events || !matches!( event.event_type, EventType::StatisticsSummary(_) | EventType::GraphStatistics(_) | EventType::Tracking(_) )) .then_some(log_colors::EVENT_LIGHT) } else { None }; if let Some(color) = color { self.entries.push_back(Entry { color, timestamp: event.timestamp.clone(), ty: event.event_type_string(), message: event.message(), }); if self.entries.len() > self.log_limit { self.entries.pop_front(); } } } pub fn ui(&mut self, ui: &mut Ui) { ui.horizontal(|ui| { if ui.button("Copy all").clicked() { ui.output_mut(|out| { out.commands .push(OutputCommand::CopyText(self.entries.iter().fold( String::new(), |acc, entry| { format!( "{}{} [{}] {}\n", acc, entry.timestamp, entry.ty, entry.message ) }, ))); }) } if ui.button("Open logs directory").clicked() { let log_dir = crate::get_filesystem_layout().log_dir; ui.ctx().open_url(OpenUrl::same_tab(format!( "file://{}", log_dir.to_string_lossy() ))); } if ui.button("Clear all").clicked() { self.entries.clear(); } }); ScrollArea::both() .stick_to_bottom(true) .auto_shrink([false, false]) .show(ui, |ui| { Grid::new(0) .spacing((10.0, 2.0)) .num_columns(3) .striped(true) .show(ui, |ui| { for entry in &self.entries { ui.colored_label( entry.color, RichText::new(&entry.timestamp).size(12.0), ); ui.colored_label(entry.color, RichText::new(&entry.ty).size(12.0)); ui.colored_label(entry.color, RichText::new(&entry.message).size(12.0)); ui.end_row(); } }); }); } } ================================================ FILE: alvr/dashboard/src/dashboard/components/mod.rs ================================================ mod about; mod debug; mod devices; mod logs; mod new_version_popup; mod notifications; mod settings; mod settings_controls; mod setup_wizard; mod statistics; #[cfg(not(target_arch = "wasm32"))] mod installation; pub use about::*; pub use debug::*; pub use devices::*; pub use logs::*; pub use new_version_popup::*; pub use notifications::*; pub use settings::*; pub use settings_controls::*; pub use setup_wizard::*; pub use statistics::*; #[cfg(not(target_arch = "wasm32"))] pub use installation::*; ================================================ FILE: alvr/dashboard/src/dashboard/components/new_version_popup.rs ================================================ use crate::dashboard::ServerRequest; use alvr_gui_common::ModalButton; use alvr_packets::PathValuePair; use eframe::egui::{self, Context, OpenUrl, Ui}; use std::{path::PathBuf, process::Command}; pub enum CloseAction { Close, CloseWithRequest(ServerRequest), } pub struct NewVersionPopup { version: String, message: String, launcher_path: Option, } impl NewVersionPopup { pub fn new(version: String, message: String) -> Self { let mut launcher_path = None; let layout = crate::get_filesystem_layout(); if let Some(path) = layout.launcher_exe() && path.exists() { launcher_path = Some(path); } Self { version, message, launcher_path, } } pub fn ui(&self, context: &Context, shutdown_alvr_cb: impl Fn()) -> Option { let no_remind_button = ModalButton::Custom("Don't remind me again for this version".to_string()); let result = alvr_gui_common::modal( context, "New ALVR version available", Some(|ui: &mut Ui| { ui.horizontal(|ui| { ui.add_space(10.0); ui.vertical(|ui| { ui.heading(format!("ALVR v{}", self.version)); ui.horizontal(|ui| { ui.spacing_mut().item_spacing.x = 5.0; ui.style_mut().spacing.button_padding = egui::vec2(10.0, 4.0); ui.heading("You can download this version using the launcher:"); if let Some(path) = &self.launcher_path { if ui.button("Open Launcher").clicked() && Command::new(path).spawn().is_ok() { shutdown_alvr_cb(); } } else if ui.button("Download Launcher").clicked() { let base_url = "https://github.com/alvr-org/ALVR/releases/latest/download/"; let file = if cfg!(windows) { "alvr_launcher_windows.zip" } else { "alvr_launcher_linux.tar.gz" }; context.open_url(OpenUrl::new_tab(format!("{base_url}{file}"))); } }); ui.add_space(10.0); ui.label(&self.message); ui.hyperlink_to( "Releases page", "https://github.com/alvr-org/ALVR/releases", ); }); ui.add_space(10.0); }); }), &[no_remind_button.clone(), ModalButton::Close], Some(490.0), ); if let Some(button) = result { if button == no_remind_button { Some(CloseAction::CloseWithRequest( ServerRequest::SetSessionValues(vec![PathValuePair { path: alvr_packets::parse_path( "session_settings.extra.new_version_popup.content.hide_while_version", ), value: serde_json::Value::String(self.version.clone()), }]), )) } else { Some(CloseAction::Close) } } else { None } } } ================================================ FILE: alvr/dashboard/src/dashboard/components/notifications.rs ================================================ use alvr_common::{LogEntry, LogSeverity}; use alvr_gui_common::theme::{self, log_colors}; use alvr_session::Settings; use eframe::{ egui::{self, Frame, Label, Layout, RichText, TextWrapMode, TopBottomPanel}, emath::Align, epaint::Color32, }; use rand::seq::IndexedRandom; use std::time::Duration; #[cfg(target_arch = "wasm32")] use instant::Instant; #[cfg(not(target_arch = "wasm32"))] use std::time::Instant; const TIMEOUT: Duration = Duration::from_secs(5); const NO_NOTIFICATIONS_MESSAGE: &str = "No new notifications"; const NOTIFICATION_TIPS: &[&str] = &[ // The following tips are ordered roughtly in the order settings appear r#"If you started having crashes after changing some settings, reset ALVR by re-running "Run setup wizard" from the "Installation" tab and clicking "Reset settings"."#, r#"Some settings are hidden by default. Click the "Expand" button next to some settings to expand the submenus."#, r#"It's highly advisable to keep audio settings as default in ALVR and modify the default audio device in the taskbar tray."#, r#"Increasing "Video"->"Maximum buffering" may reduce stutters at the cost of more latency."#, r#"Sometimes switching between h264 and HEVC codecs is necessary on certain GPUs to fix crashing or fallback to software encoding."#, r#"If you're using an NVIDIA GPU, it's best to use high-bitrate H264; if you're using an AMD GPU, HEVC might look better."#, r#"If you experience "white snow" flickering, set "Presets"->"Resolution" to "Low" and disable "Video"->"Foveated encoding"."#, r#"Increasing "Video"->"Color correction"->"Sharpness" may improve the perceived image quality."#, r#"If you have problems syncing external controllers or trackers to ALVR tracking space, add one element to "Headset"->"Extra OpenVR properties", then set a custom "Tracking system name string"."#, r#"To change the visual appearance of controllers, set "Headset"->"Controllers"->"Emulation mode"."#, r#"ALVR supports custom button bindings! If you need help, please ask us on our Discord server."#, r#"ALVR supports hand tracking gestures ("Presets"->"Hand tracking interaction"->"ALVR bindings"). Check out wiki how to use them properly: https://github.com/alvr-org/ALVR/wiki/Hand-tracking-controller-bindings."#, r#"If hand tracking gestures are annoying, you can disable them in "Headset"->"Controllers"->"Hand tracking interaction". Alternatively, you can enable "Hand tracking interaction"->"Only touch"."#, r#"You can fine-tune the controllers' responsiveness with "Headset"->"Controllers"->"Prediction"."#, r#"If the visual controller/hand models do not match the physical controller's position, you can tweak the offset in "Headset"->"Controllers"->"Left controller position/rotation offset" (affects both controllers)."#, r#"When using external trackers or controllers, you should set both "Headset"->"Position/Rotation recentering mode" to "Disabled"."#, r#"You can enable tilt mode. Set "Headset"->"Position recentering mode" to "Local" and "Headset"->"Rotation recentering mode" to "Tilted"."#, r#"If you often experience image glitching, you can trade that with stutter frames using "Connection"->"Avoid video glitching"."#, r#"You can run custom commands/programs at headset connection/disconnection using "Connection"->"Enable on connect/disconnect script"."#, r#"In case you want to report a bug, to get a log file, enable "Extra"->"Logging"->"Log to disk". The log will be inside "session_log.txt"."#, r#"For hacking purposes, you can enable "Extra"->"Logging"->"Log tracking", "Log button presses" and "Log haptics". You can get the data using a websocket at ws://localhost:8082/api/events."#, r#"In case you want to report a bug and share your log, you should enable "Extra"->"Logging"->"Prefer backtrace"."#, r#"You can quickly cycle through tips like this one by toggling "Extra"->"Logging"->"Show notification tip"."#, r#"It's handy to enable "Extra"->"SteamVR Launcher"->"Open and close SteamVR automatically"."#, r#"If you want to share a video recording for reporting a bug, you can enable "Extra"->"Capture"->"Rolling video files" to limit the file size of the upload."#, // Miscellaneous r#"If your headset does not appear in the device list, it might be in a different subnet. Try "Add device manually" with IP shown from inside device."#, ]; pub struct NotificationBar { message: String, current_level: LogSeverity, receive_instant: Instant, min_notification_level: LogSeverity, tip_message: Option, expanded: bool, } impl NotificationBar { pub fn new() -> Self { Self { message: NO_NOTIFICATIONS_MESSAGE.into(), current_level: LogSeverity::Debug, receive_instant: Instant::now(), min_notification_level: LogSeverity::Debug, tip_message: None, expanded: false, } } pub fn update_settings(&mut self, settings: &Settings) { self.min_notification_level = settings.extra.logging.notification_level; if settings.extra.logging.show_notification_tip { if self.tip_message.is_none() { self.tip_message = NOTIFICATION_TIPS .choose(&mut rand::rng()) .map(|s| format!("Tip: {s}")); } } else { self.tip_message = None; } } pub fn push_notification(&mut self, event: LogEntry, from_dashboard: bool) { let now = Instant::now(); let min_severity = if from_dashboard { if cfg!(debug_assertions) { LogSeverity::Debug } else { LogSeverity::Info } } else { self.min_notification_level }; if event.severity >= min_severity && (now > self.receive_instant + TIMEOUT || event.severity >= self.current_level) { self.message = event.content; self.current_level = event.severity; self.receive_instant = now; } } pub fn ui(&mut self, context: &egui::Context) { let now = Instant::now(); if now > self.receive_instant + TIMEOUT { self.message = self .tip_message .clone() .unwrap_or_else(|| NO_NOTIFICATIONS_MESSAGE.into()); self.current_level = LogSeverity::Debug; } let (fg, bg) = match self.current_level { LogSeverity::Error => (Color32::BLACK, log_colors::ERROR_LIGHT), LogSeverity::Warning => (Color32::BLACK, log_colors::WARNING_LIGHT), LogSeverity::Info => (Color32::BLACK, log_colors::INFO_LIGHT), LogSeverity::Debug => (theme::FG, theme::LIGHTER_BG), }; let mut bottom_bar = TopBottomPanel::bottom("bottom_panel").frame( Frame::default() .inner_margin(egui::vec2(10.0, 5.0)) .fill(bg), ); let alignment = if !self.expanded { bottom_bar = bottom_bar.max_height(26.0); Align::TOP } else { Align::Center }; let wrapping = if !self.expanded { TextWrapMode::Truncate } else { TextWrapMode::Wrap }; bottom_bar.show(context, |ui| { ui.with_layout(Layout::right_to_left(alignment), |ui| { if !self.expanded { if ui.small_button("Expand").clicked() { self.expanded = true; } } else if ui.button("Reduce").clicked() { self.expanded = false; } ui.with_layout(Layout::left_to_right(alignment), |ui| { //A LayoutJob that has its TextWrapping updated to fill the available space would probably be a more elegant solution. ui.add( Label::new(RichText::new(&self.message).color(fg).size(12.0)) .wrap_mode(wrapping), ); }) }) }); } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings.rs ================================================ use super::{ NestingInfo, SettingControl, presets::{PresetControl, builtin_schema}, }; use crate::dashboard::ServerRequest; use alvr_gui_common::{DisplayString, theme}; use alvr_session::{SessionSettings, Settings}; use eframe::egui::{Align, Frame, Grid, Layout, RichText, ScrollArea, Ui}; #[cfg(target_arch = "wasm32")] use instant::Instant; use serde_json as json; use settings_schema::SchemaNode; use std::time::Duration; #[cfg(not(target_arch = "wasm32"))] use std::time::Instant; const DATA_UPDATE_INTERVAL: Duration = Duration::from_secs(1); const MIN_COLUMN_SIZE: f32 = 300.0; struct TopLevelEntry { id: DisplayString, control: SettingControl, } pub struct SettingsTab { selected_top_tab_id: String, resolution_preset: PresetControl, framerate_preset: PresetControl, encoder_preset: PresetControl, foveation_preset: PresetControl, codec_preset: PresetControl, game_audio_preset: PresetControl, microphone_preset: PresetControl, hand_tracking_interaction_preset: PresetControl, eye_face_tracking_preset: PresetControl, top_level_entries: Vec, session_settings_json: Option, last_update_instant: Instant, } impl SettingsTab { pub fn new() -> Self { let nesting_info = NestingInfo { path: vec!["session_settings".into()], indentation_level: 0, }; let schema = Settings::schema(alvr_session::session_settings_default()); // Top level node must be a section let SchemaNode::Section { entries, .. } = schema else { unreachable!(); }; let top_level_entries = entries .into_iter() .map(|entry| { let id = entry.name; let display = super::get_display_name(&id, &entry.strings); let mut nesting_info = nesting_info.clone(); nesting_info.path.push(id.clone().into()); TopLevelEntry { id: DisplayString { id, display }, control: SettingControl::new(nesting_info, entry.content), } }) .collect(); Self { selected_top_tab_id: "presets".into(), resolution_preset: PresetControl::new(builtin_schema::resolution_schema()), framerate_preset: PresetControl::new(builtin_schema::framerate_schema()), encoder_preset: PresetControl::new(builtin_schema::encoder_preset_schema()), foveation_preset: PresetControl::new(builtin_schema::foveation_preset_schema()), codec_preset: PresetControl::new(builtin_schema::codec_preset_schema()), game_audio_preset: PresetControl::new(builtin_schema::game_audio_schema()), microphone_preset: PresetControl::new(builtin_schema::microphone_schema()), hand_tracking_interaction_preset: PresetControl::new( builtin_schema::hand_tracking_interaction_schema(), ), eye_face_tracking_preset: PresetControl::new(builtin_schema::eye_face_tracking_schema()), top_level_entries, session_settings_json: None, last_update_instant: Instant::now(), } } pub fn update_session(&mut self, session_settings: &SessionSettings) { let settings_json = json::to_value(session_settings).unwrap(); self.resolution_preset .update_session_settings(&settings_json); self.framerate_preset .update_session_settings(&settings_json); self.encoder_preset.update_session_settings(&settings_json); self.foveation_preset .update_session_settings(&settings_json); self.codec_preset.update_session_settings(&settings_json); self.game_audio_preset .update_session_settings(&settings_json); self.microphone_preset .update_session_settings(&settings_json); self.hand_tracking_interaction_preset .update_session_settings(&settings_json); self.eye_face_tracking_preset .update_session_settings(&settings_json); self.session_settings_json = Some(settings_json); } pub fn ui(&mut self, ui: &mut Ui) -> Vec { let mut requests = vec![]; let now = Instant::now(); if now > self.last_update_instant + DATA_UPDATE_INTERVAL { if self.session_settings_json.is_none() { requests.push(ServerRequest::GetSession); } self.last_update_instant = now; } let mut path_value_pairs = vec![]; ui.with_layout(Layout::left_to_right(Align::Min), |ui| { Frame::group(ui.style()) .fill(theme::DARKER_BG) .inner_margin(theme::FRAME_PADDING) .show(ui, |ui| { ui.horizontal_wrapped(|ui| { ui.selectable_value( &mut self.selected_top_tab_id, "presets".into(), RichText::new("Presets").raised().size(15.0), ); for entry in &mut self.top_level_entries { ui.selectable_value( &mut self.selected_top_tab_id, entry.id.id.clone(), RichText::new(entry.id.display.clone()).raised().size(15.0), ); } }) }) }); if self.selected_top_tab_id == "presets" { ScrollArea::new([false, true]) .id_salt("presets_scroll") .show(ui, |ui| { Grid::new("presets_grid") .striped(true) .num_columns(2) .min_col_width(MIN_COLUMN_SIZE) .show(ui, |ui| { path_value_pairs.extend(self.resolution_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.framerate_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.encoder_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.foveation_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.codec_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.game_audio_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.microphone_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.hand_tracking_interaction_preset.ui(ui)); ui.end_row(); path_value_pairs.extend(self.eye_face_tracking_preset.ui(ui)); ui.end_row(); }) }); } else { ScrollArea::new([false, true]) .id_salt(format!("{}_scroll", self.selected_top_tab_id)) .show(ui, |ui| { Grid::new(format!("{}_grid", self.selected_top_tab_id)) .striped(true) .num_columns(2) .min_col_width(MIN_COLUMN_SIZE) .show(ui, |ui| { if let Some(session_fragment) = &mut self.session_settings_json { let session_fragments_mut = session_fragment.as_object_mut().unwrap(); let entry = self .top_level_entries .iter_mut() .find(|entry: &&mut TopLevelEntry| { entry.id.id == self.selected_top_tab_id }) .unwrap(); let response = entry.control.ui( ui, &mut session_fragments_mut[&entry.id.id], false, ); if let Some(response) = response { path_value_pairs.push(response); } ui.end_row(); } }) }); } if !path_value_pairs.is_empty() { requests.push(ServerRequest::SetSessionValues(path_value_pairs)); } requests } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/array.rs ================================================ use super::{NestingInfo, SettingControl, collapsible}; use alvr_packets::PathValuePair; use alvr_session::settings_schema::SchemaNode; use eframe::egui::Ui; use serde_json as json; pub struct Control { nesting_info: NestingInfo, controls: Vec, } impl Control { pub fn new(nesting_info: NestingInfo, schema_array: Vec) -> Self { let controls = schema_array .into_iter() .enumerate() .map(|(idx, schema)| { let mut nesting_info = nesting_info.clone(); nesting_info.path.push("content".into()); nesting_info.path.push(idx.into()); SettingControl::new(nesting_info, schema) }) .collect(); Self { nesting_info, controls, } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let mut request = None; let collapsed = collapsible::collapsible_button(ui, &self.nesting_info, session_fragment, &mut request); if !collapsed { let session_array_mut = session_fragment["content"].as_array_mut().unwrap(); for (idx, control) in self.controls.iter_mut().enumerate() { ui.end_row(); request = control .ui(ui, &mut session_array_mut[idx], false) .or(request); } } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/boolean.rs ================================================ use super::{NestingInfo, reset}; use alvr_packets::PathValuePair; use eframe::{ egui::{Layout, Ui}, emath::Align, }; use serde_json as json; pub struct Control { nesting_info: NestingInfo, default: bool, default_string: String, } impl Control { pub fn new(nesting_info: NestingInfo, default: bool) -> Self { let default_string = if default { "ON".into() } else { "OFF".into() }; Self { nesting_info, default, default_string, } } pub fn ui( &self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let json::Value::Bool(enabled_mut) = session_fragment else { unreachable!() }; let mut request = None; fn get_request(nesting_info: &NestingInfo, enabled: bool) -> Option { Some(PathValuePair { path: nesting_info.path.clone(), value: json::Value::Bool(enabled), }) } ui.with_layout(Layout::left_to_right(Align::Center), |ui| { if alvr_gui_common::switch(ui, enabled_mut).clicked() { request = get_request(&self.nesting_info, *enabled_mut); } if reset::reset_button(ui, *enabled_mut != self.default, &self.default_string).clicked() { request = get_request(&self.nesting_info, self.default); } }); request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/choice.rs ================================================ use super::{NestingInfo, SettingControl, reset}; use alvr_gui_common::DisplayString; use alvr_packets::PathValuePair; use alvr_session::settings_schema::{ChoiceControlType, SchemaEntry, SchemaNode}; use eframe::{ egui::{ComboBox, Layout, Ui}, emath::Align, }; use serde_json as json; use std::collections::HashMap; fn get_display_name(id: &str, strings: &HashMap) -> String { strings.get("display_name").cloned().unwrap_or_else(|| { let mut chars = id.chars(); let mut new_chars = vec![chars.next().unwrap()]; for c in chars { let new_c = c.to_ascii_lowercase(); if new_c != c { new_chars.push(' '); } new_chars.push(new_c); } new_chars.into_iter().collect::() }) } pub struct Control { nesting_info: NestingInfo, default_variant: String, default_string: String, variant_labels: Vec, variant_indices: HashMap, variant_controls: HashMap, gui: ChoiceControlType, combobox_id: usize, } impl Control { pub fn new( nesting_info: NestingInfo, default: String, schema_variants: Vec>>, gui: Option, ) -> Self { let variant_labels = schema_variants .iter() .map(|entry| DisplayString { id: entry.name.clone(), display: get_display_name(&entry.name, &entry.strings), }) .collect::>(); let default_string = format!( "\"{}\"", variant_labels .iter() .find(|d| d.id == default) .cloned() .unwrap() .display ); let variant_indices = schema_variants .iter() .enumerate() .map(|(idx, entry)| (entry.name.clone(), idx)) .collect(); let variant_controls = schema_variants .into_iter() .map(|entry| { let mut nesting_info = nesting_info.clone(); nesting_info.path.push(entry.name.clone().into()); let control = if let Some(schema) = entry.content { SettingControl::new(nesting_info, schema) } else { SettingControl::None }; (entry.name, control) }) .collect(); Self { nesting_info, default_variant: default, default_string, variant_labels, variant_indices, variant_controls, gui: gui.unwrap_or(ChoiceControlType::Dropdown), combobox_id: alvr_gui_common::get_id(), } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let session_variants_mut = session_fragment.as_object_mut().unwrap(); let json::Value::String(variant_mut) = &mut session_variants_mut["variant"] else { unreachable!() }; fn get_request(nesting_info: &NestingInfo, variant: &str) -> Option { super::get_single_value( nesting_info, "variant".into(), json::Value::String(variant.to_owned()), ) } let mut request = None; ui.with_layout(Layout::left_to_right(Align::Center), |ui| { if matches!(&self.gui, ChoiceControlType::ButtonGroup) { if alvr_gui_common::button_group_clicked(ui, &self.variant_labels, variant_mut) { request = get_request(&self.nesting_info, variant_mut); } } else if let Some(mut index) = self.variant_indices.get(variant_mut).cloned() { let response = ComboBox::from_id_salt(self.combobox_id).show_index( ui, &mut index, self.variant_labels.len(), |idx| self.variant_labels[idx].display.clone(), ); if response.changed() { variant_mut.clone_from(&self.variant_labels[index].id); request = get_request(&self.nesting_info, variant_mut); } if cfg!(debug_assertions) { response.on_hover_text(&self.variant_labels[index].id); } } else { let mut index = 0; let response = ComboBox::from_id_salt(self.combobox_id).show_index( ui, &mut index, self.variant_labels.len() + 1, |idx| { if idx == 0 { "Preset not applied".into() } else { self.variant_labels[idx - 1].display.clone() } }, ); if response.changed() { variant_mut.clone_from(&self.variant_labels[index].id); request = get_request(&self.nesting_info, variant_mut); } } if reset::reset_button( ui, *variant_mut != self.default_variant, &self.default_string, ) .clicked() { request = get_request(&self.nesting_info, &self.default_variant); } }); if let Some(control) = self.variant_controls.get_mut(&*variant_mut) && !matches!(control, SettingControl::None) { ui.end_row(); //fixes "cannot borrow `*session_variants` as mutable more than once at a time" let variant = variant_mut.clone(); request = control .ui(ui, &mut session_variants_mut[&variant], false) .or(request); } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/collapsible.rs ================================================ use super::NestingInfo; use alvr_packets::PathValuePair; use eframe::egui::Ui; use serde_json as json; pub fn collapsible_button( ui: &mut Ui, nesting_info: &NestingInfo, session_fragment: &mut json::Value, request: &mut Option, ) -> bool { let json::Value::Bool(state_mut) = &mut session_fragment["gui_collapsed"] else { unreachable!() }; if (*state_mut && ui.small_button("Expand").clicked()) || (!*state_mut && ui.small_button("Collapse").clicked()) { *state_mut = !*state_mut; *request = super::get_single_value( nesting_info, "gui_collapsed".into(), json::Value::Bool(*state_mut), ); } *state_mut } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/dictionary.rs ================================================ use super::{INDENTATION_STEP, NestingInfo, SettingControl, reset}; use crate::dashboard::components::{ collapsible, up_down::{self, UpDownResult}, }; use alvr_packets::PathValuePair; use alvr_session::settings_schema::SchemaNode; use eframe::{ egui::{Layout, TextEdit, Ui}, emath::Align, }; use serde_json as json; struct Entry { editing_key: Option, control: SettingControl, } pub struct Control { nesting_info: NestingInfo, default_key: String, default_value: SchemaNode, default: Vec, controls: Vec, } impl Control { pub fn new( nesting_info: NestingInfo, default_key: String, default_value: SchemaNode, default: Vec<(String, json::Value)>, ) -> Self { Self { nesting_info, default_key, default_value, default: default .into_iter() .map(|pair| json::to_value(pair).unwrap()) .collect(), controls: vec![], } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); fn get_content_request( nesting_info: &NestingInfo, entries: Vec, ) -> Option { super::get_single_value( nesting_info, "content".into(), json::to_value(entries).unwrap(), ) } let mut request = None; let collapsed = ui .with_layout(Layout::left_to_right(Align::Center), |ui| { let collapsed = collapsible::collapsible_button( ui, &self.nesting_info, session_fragment, &mut request, ); if reset::reset_button(ui, true, "default list").clicked() { request = get_content_request(&self.nesting_info, self.default.clone()) } collapsed }) .inner; let session_content = session_fragment["content"].as_array_mut().unwrap(); while session_content.len() > self.controls.len() { let mut nesting_info = self.nesting_info.clone(); nesting_info.path.extend_from_slice(&[ "content".into(), self.controls.len().into(), 1.into(), ]); self.controls.push(Entry { editing_key: None, control: SettingControl::new(nesting_info, self.default_value.clone()), }); } while session_content.len() < self.controls.len() { self.controls.pop(); } if !collapsed { ui.end_row(); let mut idx = 0; while idx < self.controls.len() { let delete_entry = ui .horizontal(|ui| { ui.add_space(INDENTATION_STEP * self.nesting_info.indentation_level as f32); let delete_entry = ui.button("❌").clicked(); let up_down_result = up_down::up_down_buttons(ui, idx, self.controls.len()); if up_down_result != UpDownResult::None { if up_down_result == UpDownResult::Up { session_content.swap(idx, idx - 1); } else { session_content.swap(idx, idx + 1); } request = get_content_request(&self.nesting_info, session_content.clone()); } let json::Value::String(text_mut) = &mut session_content[idx][0] else { unreachable!() }; let editing_key_mut = &mut self.controls[idx].editing_key; let textbox = if let Some(editing_key_mut) = editing_key_mut { TextEdit::singleline(editing_key_mut) } else { TextEdit::singleline(text_mut) }; let response = ui.add(textbox.desired_width(f32::INFINITY)); if response.lost_focus() { if let Some(editing_key_mut) = editing_key_mut { let mut nesting_info = self.nesting_info.clone(); nesting_info .path .extend_from_slice(&["content".into(), idx.into()]); request = super::get_single_value( &nesting_info, 0.into(), json::Value::String(editing_key_mut.clone()), ); text_mut.clone_from(editing_key_mut); } *editing_key_mut = None; } if response.gained_focus() { *editing_key_mut = Some(text_mut.clone()); }; delete_entry }) .inner; if delete_entry { session_content.remove(idx); self.controls.remove(idx); request = get_content_request(&self.nesting_info, session_content.clone()); } else { request = self.controls[idx] .control .ui(ui, &mut session_content[idx][1], true) .or(request); } ui.end_row(); idx += 1; } ui.label(" "); if ui.button("Add entry").clicked() { let mut session_content = session_fragment["content"].as_array_mut().unwrap().clone(); session_content.push(json::Value::Array(vec![ json::Value::String(self.default_key.clone()), session_fragment["value"].clone(), ])); request = get_content_request(&self.nesting_info, session_content); } } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/mod.rs ================================================ pub mod array; pub mod boolean; pub mod choice; pub mod collapsible; pub mod dictionary; pub mod notice; pub mod number; pub mod optional; pub mod presets; pub mod reset; pub mod section; pub mod switch; pub mod text; pub mod up_down; pub mod vector; use alvr_packets::{PathSegment, PathValuePair}; use alvr_session::settings_schema::SchemaNode; use eframe::egui::Ui; use serde_json as json; use std::collections::HashMap; pub const INDENTATION_STEP: f32 = 20.0; fn get_single_value( nesting_info: &NestingInfo, leaf: PathSegment, new_value: json::Value, ) -> Option { let mut path = nesting_info.path.clone(); path.push(leaf); Some(PathValuePair { path, value: new_value, }) } fn grid_flow_inline(ui: &mut Ui, allow_inline: bool) { if !allow_inline { // Note: ui.add_space() does not work ui.label(" "); } } pub fn get_display_name(id: &str, strings: &HashMap) -> String { strings.get("display_name").cloned().unwrap_or_else(|| { let mut chars = id.chars(); chars.next().unwrap().to_uppercase().collect::() + chars.as_str().replace('_', " ").as_str() }) } pub fn f64_eq(f1: f64, f2: f64) -> bool { f64::abs(f1 - f2) < f32::EPSILON as f64 // Alternative solution: // format!("{:.6}", f1) == format!("{:.6}", f2) } pub fn json_values_eq(a: &serde_json::Value, b: &serde_json::Value) -> bool { // Note: is_f64() will exclude integers, while just as_f64() will silently do the conversion if let serde_json::Value::Number(n1) = a && let serde_json::Value::Number(n2) = b && (n1.is_f64() || n2.is_f64()) && let Some(f1) = n1.as_f64() && let Some(f2) = n2.as_f64() { f64_eq(f1, f2) } else { a == b } } #[derive(Clone)] pub struct NestingInfo { pub path: Vec, pub indentation_level: usize, } pub enum SettingControl { Section(section::Control), Choice(choice::Control), Optional(optional::Control), Switch(switch::Control), Boolean(boolean::Control), Text(text::Control), Numeric(number::Control), Array(array::Control), Vector(vector::Control), Dictionary(dictionary::Control), None, } impl SettingControl { pub fn new(nesting_info: NestingInfo, schema: SchemaNode) -> Self { match schema { SchemaNode::Section { entries, gui_collapsible, } => Self::Section(section::Control::new( nesting_info, entries, gui_collapsible, )), SchemaNode::Choice { default, variants, gui, } => Self::Choice(choice::Control::new(nesting_info, default, variants, gui)), SchemaNode::Optional { default_set, content, } => Self::Optional(optional::Control::new(nesting_info, default_set, *content)), SchemaNode::Switch { default_enabled, content, } => Self::Switch(switch::Control::new( nesting_info, default_enabled, *content, )), SchemaNode::Boolean { default } => { Self::Boolean(boolean::Control::new(nesting_info, default)) } SchemaNode::Number { default, ty, gui, suffix, } => Self::Numeric(number::Control::new(nesting_info, default, ty, gui, suffix)), SchemaNode::Text { default } => Self::Text(text::Control::new(nesting_info, default)), SchemaNode::Array(schema_array) => { Self::Array(array::Control::new(nesting_info, schema_array)) } SchemaNode::Vector { default_element, default, } => Self::Vector(vector::Control::new( nesting_info, *default_element, default, )), SchemaNode::Dictionary { default_key, default_value, default, } => Self::Dictionary(dictionary::Control::new( nesting_info, default_key, *default_value, default, )), _ => Self::None, } } // inline: first field child, could be rendered beside the field label pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { match self { Self::Section(control) => control.ui(ui, session_fragment, allow_inline), Self::Choice(control) => control.ui(ui, session_fragment, allow_inline), Self::Optional(control) => control.ui(ui, session_fragment, allow_inline), Self::Switch(control) => control.ui(ui, session_fragment, allow_inline), Self::Boolean(control) => control.ui(ui, session_fragment, allow_inline), Self::Text(control) => control.ui(ui, session_fragment, allow_inline), Self::Numeric(control) => control.ui(ui, session_fragment, allow_inline), Self::Array(control) => control.ui(ui, session_fragment, allow_inline), Self::Vector(control) => control.ui(ui, session_fragment, allow_inline), Self::Dictionary(control) => control.ui(ui, session_fragment, allow_inline), Self::None => { grid_flow_inline(ui, allow_inline); ui.add_enabled_ui(false, |ui| ui.label("Unimplemented UI")); None } } } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/notice.rs ================================================ use alvr_gui_common::theme::{self, log_colors}; use eframe::{ egui::{Frame, Label, RichText, Ui}, epaint::Color32, }; pub fn notice(ui: &mut Ui, text: &str) { Frame::group(ui.style()) .fill(log_colors::WARNING_LIGHT) .corner_radius(theme::CORNER_RADIUS) .show(ui, |ui| { ui.add(Label::new(RichText::new(text).size(11.0).color(Color32::BLACK)).wrap()); }); } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/number.rs ================================================ use super::{NestingInfo, reset}; use crate::dashboard::components::f64_eq; use alvr_gui_common::theme::SCROLLBAR_DOT_DIAMETER; use alvr_packets::PathValuePair; use alvr_session::settings_schema::{NumberType, NumericGuiType}; use eframe::{ egui::{DragValue, Layout, Slider, Ui}, emath::Align, }; use json::Number; use serde_json as json; fn to_json_value(number: f64, ty: NumberType) -> json::Value { match ty { NumberType::UnsignedInteger => json::Value::from(number.abs() as u64), NumberType::SignedInteger => json::Value::from(number as i64), NumberType::Float => json::Value::Number(Number::from_f64(number).unwrap()), } } pub struct Control { nesting_info: NestingInfo, editing_value_f64: Option, default: f64, default_string: String, ty: NumberType, gui_type: NumericGuiType, suffix: Option, } impl Control { pub fn new( nesting_info: NestingInfo, default: f64, ty: NumberType, gui: NumericGuiType, suffix: Option, ) -> Self { let default_string = format!("{default}{}", suffix.clone().unwrap_or_default()); Self { nesting_info, editing_value_f64: None, default, default_string, ty, gui_type: gui, suffix, } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let mut session_value = session_fragment.as_f64().unwrap(); let mut request = None; fn get_request( nesting_info: &NestingInfo, number: f64, ty: NumberType, ) -> Option { Some(PathValuePair { path: nesting_info.path.clone(), value: to_json_value(number, ty), }) } ui.with_layout(Layout::left_to_right(Align::Center), |ui| { let editing_value_mut = if let Some(editing_value_mut) = &mut self.editing_value_f64 { editing_value_mut } else { &mut session_value }; let response = match &self.gui_type { NumericGuiType::Slider { range, step, logarithmic, } => { let mut slider = Slider::new(editing_value_mut, range.clone()) .logarithmic(*logarithmic) .show_value(false); if let Some(step) = step { slider = slider.step_by(*step); } if !matches!(self.ty, NumberType::Float) { slider = slider.integer(); } let slider_response = { ui.style_mut().spacing.interact_size.y = SCROLLBAR_DOT_DIAMETER; ui.add(slider) }; let mut drag_value = DragValue::new(editing_value_mut); // Note: the following ifs cannot be merged with the ones above to avoid double // mutable borrow of editing_value_mut. if let Some(step) = step { drag_value = drag_value.speed(*step); } if !matches!(self.ty, NumberType::Float) { drag_value = drag_value.fixed_decimals(0); } if let Some(suffix) = &self.suffix { drag_value = drag_value.suffix(suffix); } let textbox_response = ui.add(drag_value); slider_response.union(textbox_response) } NumericGuiType::TextBox => { let mut drag_value = DragValue::new(editing_value_mut); if !matches!(self.ty, NumberType::Float) { drag_value = drag_value.fixed_decimals(0); } if let Some(suffix) = &self.suffix { drag_value = drag_value.suffix(suffix); } ui.add(drag_value) } }; if response.drag_started() || response.gained_focus() || response.clicked() { self.editing_value_f64 = Some(session_value) } else if response.drag_stopped() || response.lost_focus() { request = get_request(&self.nesting_info, *editing_value_mut, self.ty); *session_fragment = to_json_value(*editing_value_mut, self.ty); self.editing_value_f64 = None; } if reset::reset_button( ui, !f64_eq(session_value, self.default), &self.default_string, ) .clicked() { request = get_request(&self.nesting_info, self.default, self.ty); } }); request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/optional.rs ================================================ use super::{NestingInfo, SettingControl, reset}; use alvr_packets::PathValuePair; use alvr_session::settings_schema::SchemaNode; use eframe::{ egui::{Layout, Ui}, emath::Align, }; use serde_json as json; pub struct Control { nesting_info: NestingInfo, default_set: bool, default_string: String, content_control: Box, } impl Control { pub fn new(nesting_info: NestingInfo, default_set: bool, schema_content: SchemaNode) -> Self { let default_string = if default_set { "Set".into() } else { "Default".into() }; let control = { let mut nesting_info = nesting_info.clone(); nesting_info.path.push("content".into()); SettingControl::new(nesting_info, schema_content) }; Self { nesting_info, default_set, default_string, content_control: Box::new(control), } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let session_switch_mut = session_fragment.as_object_mut().unwrap(); let json::Value::Bool(set_mut) = &mut session_switch_mut["set"] else { unreachable!() }; let mut request = None; fn get_request(nesting_info: &NestingInfo, enabled: bool) -> Option { super::get_single_value(nesting_info, "set".into(), json::Value::Bool(enabled)) } ui.with_layout(Layout::left_to_right(Align::Center), |ui| { if ui.selectable_value(set_mut, false, "Default").clicked() || ui.selectable_value(set_mut, true, "Set").clicked() { request = get_request(&self.nesting_info, *set_mut); } if reset::reset_button(ui, *set_mut != self.default_set, &self.default_string).clicked() { request = get_request(&self.nesting_info, self.default_set); } }); if *set_mut { ui.end_row(); request = self .content_control .ui(ui, &mut session_switch_mut["content"], false) .or(request); } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/presets/builtin_schema.rs ================================================ use super::schema::{ HigherOrderChoiceOption, HigherOrderChoiceSchema, PresetModifier, PresetSchemaNode, }; use crate::dashboard::components::presets::schema::PresetModifierOperation; use settings_schema::ChoiceControlType; use std::{ collections::{HashMap, HashSet}, str::FromStr, }; fn string_modifier(target_path: &str, value: &str) -> PresetModifier { PresetModifier { target_path: target_path.into(), operation: PresetModifierOperation::Assign(serde_json::Value::String(value.into())), } } fn num_modifier(target_path: &str, value: &str) -> PresetModifier { PresetModifier { target_path: target_path.into(), operation: PresetModifierOperation::Assign(serde_json::Value::Number( serde_json::Number::from_str(value).unwrap(), )), } } fn bool_modifier(target_path: &str, value: bool) -> PresetModifier { PresetModifier { target_path: target_path.into(), operation: PresetModifierOperation::Assign(serde_json::Value::Bool(value)), } } pub fn resolution_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Resolution".into(), strings: [( "help".into(), "Choosing too high resolution (commonly 'High (width: 5184)') may result in high latency or black screen.".into(), )] .into_iter() .collect(), flags: ["steamvr-restart".into()].into_iter().collect(), options: [ ("Very Low (width: 3072)", "1536"), ("Low (width: 3712)", "1856"), ("Medium (width: 4288)", "2144"), ("High (width: 5184)", "2592"), ("Ultra (width: 5632)", "2816"), ("Extreme (width: 6080)", "3040"), ] .into_iter() .map(|(key, value)| HigherOrderChoiceOption { display_name: key.into(), modifiers: [ string_modifier( "session_settings.video.transcoding_view_resolution.variant", "Absolute", ), num_modifier( "session_settings.video.transcoding_view_resolution.Absolute.width", value, ), bool_modifier( "session_settings.video.transcoding_view_resolution.Absolute.height.set", false, ), string_modifier( "session_settings.video.emulated_headset_view_resolution.variant", "Absolute", ), num_modifier( "session_settings.video.emulated_headset_view_resolution.Absolute.width", value, ), bool_modifier( "session_settings.video.emulated_headset_view_resolution.Absolute.height.set", false, ), ] .into_iter() .collect(), content: None, }) .collect(), default_option_display_name: "Medium (width: 4288)".into(), gui: ChoiceControlType::Dropdown, }) } pub fn framerate_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Preferred framerate".into(), strings: HashMap::new(), flags: ["steamvr-restart".into()].into_iter().collect(), options: [60, 72, 80, 90, 120] .into_iter() .map(|framerate| HigherOrderChoiceOption { display_name: format!("{framerate}Hz"), modifiers: [num_modifier( "session_settings.video.preferred_fps", &format!("{:?}", framerate as f32), )] .into_iter() .collect(), content: None, }) .collect(), default_option_display_name: "72Hz".into(), gui: ChoiceControlType::ButtonGroup, }) } pub fn codec_preset_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Codec preset".into(), strings: [ ( "notice".into(), "AV1 encoding is only supported on RDNA3, Ada Lovelace, Intel ARC or newer GPUs (AMD RX 7xxx+ , NVIDIA RTX 40xx+, Intel ARC) and on headsets that have XR2 Gen 2 onboard (Quest 3, Pico 4 Ultra).\n H264 encoding is currently NOT supported on Intel ARC GPUs on Windows." .into(), ), ] .into_iter() .collect(), flags: ["steamvr-restart".into()].into_iter().collect(), options: [("H264", "H264"), ("HEVC", "Hevc"), ("AV1", "AV1")] .into_iter() .map(|(key, val_codec)| HigherOrderChoiceOption { display_name: key.into(), modifiers: [string_modifier( "session_settings.video.preferred_codec.variant", val_codec, )] .into_iter() .collect(), content: None, }) .collect(), default_option_display_name: "H264".into(), gui: ChoiceControlType::ButtonGroup, }) } pub fn encoder_preset_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Encoder preset".into(), strings: [( "help".into(), "Selecting a quality too high may result in stuttering or still image!".into(), )] .into_iter() .collect(), flags: ["steamvr-restart".into()].into_iter().collect(), options: [ ("Speed", "Speed", "P1"), ("Balanced", "Balanced", "P3"), ("Quality", "Quality", "P5"), ] .into_iter() .map(|(key, val_amd, val_nv)| HigherOrderChoiceOption { display_name: key.into(), modifiers: [ string_modifier( "session_settings.video.encoder_config.nvenc.quality_preset.variant", val_nv, ), string_modifier( "session_settings.video.encoder_config.quality_preset.variant", val_amd, ), ] .into_iter() .collect(), content: None, }) .collect(), default_option_display_name: "Speed".into(), gui: ChoiceControlType::ButtonGroup, }) } pub fn foveation_preset_schema() -> PresetSchemaNode { const PREFIX: &str = "session_settings.video.foveated_encoding"; PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Foveation preset".into(), strings: [( "help".into(), "Foveation affects pixelation on the edges of \ the screen and significantly reduces codec latency. It is not recommended to fully disable it, as it may cause \ shutterring and high encode/decode latency!" .into(), )] .into_iter() .collect(), flags: ["steamvr-restart".into()].into_iter().collect(), options: [ ("Light", 0.80, 0.80, 8.0, 8.0), ("Medium", 0.66, 0.60, 6.0, 6.0), ("High", 0.45, 0.40, 4.0, 5.0), ] .into_iter() .map( |(key, val_size_x, val_size_y, val_edge_x, val_edge_y)| HigherOrderChoiceOption { display_name: key.into(), modifiers: [ bool_modifier(&format!("{PREFIX}.enabled"), true), num_modifier( &format!("{PREFIX}.content.center_size_x"), &val_size_x.to_string(), ), num_modifier( &format!("{PREFIX}.content.center_size_y"), &val_size_y.to_string(), ), num_modifier( &format!("{PREFIX}.content.edge_ratio_x"), &val_edge_x.to_string(), ), num_modifier( &format!("{PREFIX}.content.edge_ratio_y"), &val_edge_y.to_string(), ), ] .into_iter() .collect(), content: None, }, ) .collect(), default_option_display_name: "High".into(), gui: ChoiceControlType::ButtonGroup, }) } #[cfg(target_os = "linux")] pub fn game_audio_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Headset speaker".into(), strings: HashMap::new(), flags: HashSet::new(), options: [ HigherOrderChoiceOption { display_name: "Disabled".into(), modifiers: vec![bool_modifier( "session_settings.audio.game_audio.enabled", false, )], content: None, }, HigherOrderChoiceOption { display_name: "Enabled".into(), modifiers: vec![bool_modifier( "session_settings.audio.game_audio.enabled", true, )], content: None, }, ] .into_iter() .collect(), default_option_display_name: "Enabled".into(), gui: ChoiceControlType::ButtonGroup, }) } #[cfg(target_os = "linux")] pub fn microphone_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Headset microphone".into(), strings: HashMap::new(), flags: HashSet::new(), options: [ HigherOrderChoiceOption { display_name: "Disabled".into(), modifiers: vec![bool_modifier( "session_settings.audio.microphone.enabled", false, )], content: None, }, HigherOrderChoiceOption { display_name: "Enabled".into(), modifiers: vec![bool_modifier( "session_settings.audio.microphone.enabled", true, )], content: None, }, ] .into_iter() .collect(), default_option_display_name: "Enabled".into(), gui: ChoiceControlType::ButtonGroup, }) } #[cfg(not(target_os = "linux"))] pub fn game_audio_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Headset speaker".into(), strings: [( "notice".into(), "You can change the default audio device from the system taskbar tray (bottom right)" .into(), )] .into_iter() .collect(), flags: HashSet::new(), options: vec![ HigherOrderChoiceOption { display_name: "Disabled".into(), modifiers: vec![bool_modifier( "session_settings.audio.game_audio.enabled", false, )], content: None, }, HigherOrderChoiceOption { display_name: "System Default".to_owned(), modifiers: vec![ bool_modifier("session_settings.audio.game_audio.enabled", true), bool_modifier( "session_settings.audio.game_audio.content.device.set", false, ), ], content: None, }, ] .into_iter() .collect(), default_option_display_name: "System Default".into(), gui: ChoiceControlType::ButtonGroup, }) } #[cfg(not(target_os = "linux"))] pub fn microphone_schema() -> PresetSchemaNode { let mut microhone_options = vec![HigherOrderChoiceOption { display_name: "Disabled".to_owned(), modifiers: vec![bool_modifier( "session_settings.audio.microphone.enabled", false, )], content: None, }]; if cfg!(windows) { for (key, display_name) in [ ("Automatic", "Automatic"), ("VAC", "Virtual Audio Cable"), ("VBCable", "VB Cable"), ("VoiceMeeter", "VoiceMeeter"), ("VoiceMeeterAux", "VoiceMeeter Aux"), ("VoiceMeeterVaio3", "VoiceMeeter VAIO3"), ] { microhone_options.push(HigherOrderChoiceOption { display_name: display_name.into(), modifiers: vec![ bool_modifier("session_settings.audio.microphone.enabled", true), string_modifier( "session_settings.audio.microphone.content.devices.variant", key, ), ], content: None, }) } } PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Headset microphone".into(), strings: HashMap::new(), flags: HashSet::new(), options: microhone_options.into_iter().collect(), default_option_display_name: "Disabled".into(), gui: ChoiceControlType::Dropdown, }) } pub fn hand_tracking_interaction_schema() -> PresetSchemaNode { const HELP: &str = r"Disabled: hands cannot emulate buttons. Useful for using Joy-Cons or other non-native controllers. SteamVR Input 2.0: create separate SteamVR devices for hand tracking. ALVR bindings: use ALVR hand tracking button bindings. Check the wiki for help. "; const PREFIX: &str = "session_settings.headset.controllers.content"; PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Hand tracking interaction".into(), strings: [("help".into(), HELP.into())].into_iter().collect(), flags: ["steamvr-restart".into()].into_iter().collect(), options: [ HigherOrderChoiceOption { display_name: "Disabled".into(), modifiers: vec![ bool_modifier("session_settings.headset.controllers.enabled", true), bool_modifier( &format!("{PREFIX}.hand_skeleton.content.steamvr_input_2_0"), false, ), bool_modifier( &format!("{PREFIX}.hand_tracking_interaction.enabled"), false, ), ], content: None, }, HigherOrderChoiceOption { display_name: "SteamVR Input 2.0".into(), modifiers: vec![ bool_modifier("session_settings.headset.controllers.enabled", true), bool_modifier(&format!("{PREFIX}.hand_skeleton.enabled"), true), bool_modifier( &format!("{PREFIX}.hand_skeleton.content.steamvr_input_2_0"), true, ), bool_modifier( &format!("{PREFIX}.hand_tracking_interaction.enabled"), false, ), ], content: None, }, HigherOrderChoiceOption { display_name: "ALVR bindings".into(), modifiers: vec![ bool_modifier("session_settings.headset.controllers.enabled", true), bool_modifier( &format!("{PREFIX}.hand_skeleton.content.steamvr_input_2_0"), false, ), bool_modifier(&format!("{PREFIX}.hand_tracking_interaction.enabled"), true), ], content: None, }, ] .into_iter() .collect(), default_option_display_name: "SteamVR Input 2.0".into(), gui: ChoiceControlType::ButtonGroup, }) } pub fn eye_face_tracking_schema() -> PresetSchemaNode { PresetSchemaNode::HigherOrderChoice(HigherOrderChoiceSchema { name: "Eye and face tracking".into(), strings: HashMap::new(), flags: HashSet::new(), options: [ HigherOrderChoiceOption { display_name: "Disabled".into(), modifiers: vec![bool_modifier( "session_settings.headset.face_tracking.enabled", false, )], content: None, }, HigherOrderChoiceOption { display_name: "VRChat Eye OSC".into(), modifiers: vec![ bool_modifier("session_settings.headset.face_tracking.enabled", true), string_modifier( "session_settings.headset.face_tracking.content.sink.variant", "VrchatEyeOsc", ), ], content: None, }, HigherOrderChoiceOption { display_name: "VRCFaceTracking".into(), modifiers: vec![ bool_modifier("session_settings.headset.face_tracking.enabled", true), string_modifier( "session_settings.headset.face_tracking.content.sink.variant", "VrcFaceTracking", ), ], content: None, }, ] .into_iter() .collect(), default_option_display_name: "Disabled".into(), gui: ChoiceControlType::ButtonGroup, }) } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/presets/higher_order_choice.rs ================================================ use super::schema::{HigherOrderChoiceSchema, PresetModifierOperation}; use crate::dashboard::components::{self, NestingInfo, SettingControl}; use alvr_packets::{PathSegment, PathValuePair}; use eframe::egui::Ui; use serde_json as json; use settings_schema::{SchemaEntry, SchemaNode}; use std::collections::{HashMap, HashSet}; pub struct Control { name: String, modifiers: HashMap>, control: SettingControl, preset_json: json::Value, } impl Control { pub fn new(schema: HigherOrderChoiceSchema) -> Self { let modifiers = schema .options .iter() .map(|option| { ( option.display_name.clone(), option .modifiers .iter() .map(|modifier| match &modifier.operation { PresetModifierOperation::Assign(value) => PathValuePair { path: alvr_packets::parse_path(&modifier.target_path), value: value.clone(), }, }) .collect(), ) }) .collect(); let mut strings = schema.strings; strings.insert("display_name".into(), schema.name.clone()); let control_schema = SchemaNode::Section { entries: vec![SchemaEntry { name: schema.name.clone(), strings, flags: schema.flags, content: SchemaNode::Choice { default: schema .options .iter() .find(|option| option.display_name == schema.default_option_display_name) .unwrap() .display_name .clone(), variants: schema .options .into_iter() .map(|option| SchemaEntry { name: option.display_name.clone(), strings: [("display_name".into(), option.display_name)] .into_iter() .collect(), flags: HashSet::new(), content: None, }) .collect(), gui: Some(schema.gui), }, }], gui_collapsible: false, }; let control = SettingControl::new( NestingInfo { path: vec![], indentation_level: 0, }, control_schema, ); let preset_json = json::json!({ {&schema.name}: { "variant": "" } }); Self { name: schema.name, modifiers, control, preset_json, } } pub fn update_session_settings(&mut self, session_setting_json: &json::Value) { let mut selected_option = String::new(); 'outer: for (key, descs) in &self.modifiers { for desc in descs { let mut session_ref = session_setting_json; // Note: the first path segment is always "settings_schema". Skip that. for segment in &desc.path[1..] { session_ref = match segment { PathSegment::Name(name) => { if let Some(name) = session_ref.get(name) { name } else { continue 'outer; } } PathSegment::Index(index) => { if let Some(index) = session_ref.get(index) { index } else { continue 'outer; } } }; } if !components::json_values_eq(session_ref, &desc.value) { continue 'outer; } } // At this point the session matches all modifiers selected_option.clone_from(key); break; } // Note: if no modifier matched, the control will unselect all options self.preset_json[&self.name]["variant"] = json::Value::String(selected_option); } pub fn ui(&mut self, ui: &mut Ui) -> Vec { if let Some(desc) = self.control.ui(ui, &mut self.preset_json, false) { // todo: handle children requests self.modifiers[desc.value.as_str().unwrap()].clone() } else { vec![] } } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/presets/mirror.rs ================================================ ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/presets/mod.rs ================================================ mod higher_order_choice; mod mirror; pub mod builtin_schema; pub mod schema; use self::schema::PresetSchemaNode; use alvr_packets::PathValuePair; use eframe::egui::Ui; use serde_json as json; pub enum PresetControl { HigherOrderChoice(higher_order_choice::Control), // Mirror(...) } impl PresetControl { pub fn new(schema: PresetSchemaNode) -> Self { match schema { PresetSchemaNode::HigherOrderChoice(schema) => { Self::HigherOrderChoice(higher_order_choice::Control::new(schema)) } PresetSchemaNode::Mirror(_) => unimplemented!(), } } pub fn update_session_settings(&mut self, session_settings_json: &json::Value) { match self { Self::HigherOrderChoice(control) => { control.update_session_settings(session_settings_json) } } } pub fn ui(&mut self, ui: &mut Ui) -> Vec { match self { Self::HigherOrderChoice(control) => control.ui(ui), } } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/presets/schema.rs ================================================ use serde::{Deserialize, Serialize}; use serde_json as json; use settings_schema::ChoiceControlType; use std::collections::{HashMap, HashSet}; #[derive(Serialize, Deserialize, Clone)] pub enum PresetModifierOperation { Assign(json::Value), } #[derive(Serialize, Deserialize)] pub struct PresetModifier { // session-style path pub target_path: String, pub operation: PresetModifierOperation, } #[derive(Serialize, Deserialize)] pub struct HigherOrderChoiceOption { pub display_name: String, pub modifiers: Vec, pub content: Option, } #[derive(Serialize, Deserialize)] pub struct HigherOrderChoiceSchema { pub name: String, pub strings: HashMap, pub flags: HashSet, pub options: Vec, pub default_option_display_name: String, pub gui: ChoiceControlType, } #[derive(Serialize, Deserialize)] pub enum PresetSchemaNode { HigherOrderChoice(HigherOrderChoiceSchema), // session-style path Mirror(String), } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/reset.rs ================================================ use eframe::{ egui::{self, Button, Layout, Response, Ui}, emath::Align, }; pub fn reset_button(ui: &mut Ui, enabled: bool, default_str: &str) -> Response { ui.with_layout(Layout::right_to_left(Align::Center), |ui| { ui.add_space(5.0); let height = ui.spacing().interact_size.y; ui.add_enabled( enabled, Button::new("⟲").min_size(egui::vec2(height, height)), ) .on_hover_text(format!("Reset to {default_str}")) }) .inner } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/section.rs ================================================ use super::{INDENTATION_STEP, NestingInfo, SettingControl, collapsible, notice}; use alvr_gui_common::{ DisplayString, theme::{ OK_GREEN, log_colors::{INFO_LIGHT, WARNING_LIGHT}, }, }; use alvr_packets::PathValuePair; use alvr_session::settings_schema::{SchemaEntry, SchemaNode}; use eframe::egui::Ui; use serde_json as json; struct Entry { id: DisplayString, help: Option, notice: Option, hidden: bool, steamvr_restart_flag: bool, real_time_flag: bool, control: SettingControl, } pub struct Control { nesting_info: NestingInfo, entries: Vec, gui_collapsible: bool, } impl Control { pub fn new( mut nesting_info: NestingInfo, schema_entries: Vec>, gui_collapsible: bool, ) -> Self { nesting_info.indentation_level += 1; let entries = schema_entries .into_iter() .map(|entry| { let id = entry.name; let display = super::get_display_name(&id, &entry.strings); let help = entry.strings.get("help").cloned(); let notice = entry.strings.get("notice").cloned(); let hidden = entry.flags.contains("hidden"); let steamvr_restart_flag = entry.flags.contains("steamvr-restart"); let real_time_flag = entry.flags.contains("real-time"); let mut nesting_info = nesting_info.clone(); nesting_info.path.push(id.clone().into()); Entry { id: DisplayString { id, display }, help, notice, hidden, steamvr_restart_flag, real_time_flag, control: SettingControl::new(nesting_info, entry.content), } }) .collect(); Self { nesting_info, entries, gui_collapsible, } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { let entries_count = self.entries.len(); let mut request = None; let collapsed = if self.gui_collapsible { super::grid_flow_inline(ui, allow_inline); let collapsed = collapsible::collapsible_button( ui, &self.nesting_info, session_fragment, &mut request, ); if !collapsed { ui.end_row(); } collapsed } else { if allow_inline { ui.end_row(); } false }; if !collapsed { for (i, entry) in self.entries.iter_mut().enumerate() { if entry.hidden { continue; } ui.horizontal(|ui| { ui.add_space(INDENTATION_STEP * self.nesting_info.indentation_level as f32); let label_res = ui.label(&entry.id.display); if cfg!(debug_assertions) { label_res.on_hover_text_at_pointer(&*entry.id); } if let Some(string) = &entry.help { ui.colored_label(INFO_LIGHT, "❓") .on_hover_text_at_pointer(string); } if entry.steamvr_restart_flag { ui.colored_label(WARNING_LIGHT, "⚠") .on_hover_text_at_pointer( "Changing this setting will make SteamVR restart!\n\ Please save your in-game progress first", ); } if entry.real_time_flag { // The emoji is blue but it will be green in the UI ui.colored_label(OK_GREEN, "🔵").on_hover_text_at_pointer( "This setting can be changed in real-time during streaming!", ); } }); if let Some(string) = &entry.notice { notice::notice(ui, string); ui.end_row(); ui.label(" "); } request = entry .control .ui(ui, &mut session_fragment[&entry.id.id], true) .or(request); if i != entries_count - 1 { ui.end_row(); } } } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/switch.rs ================================================ use super::{NestingInfo, SettingControl, reset}; use alvr_packets::PathValuePair; use alvr_session::settings_schema::SchemaNode; use eframe::{ egui::{Layout, Ui}, emath::Align, }; use serde_json as json; pub struct Control { nesting_info: NestingInfo, default_enabled: bool, default_string: String, content_control: Box, } impl Control { pub fn new( nesting_info: NestingInfo, default_enabled: bool, schema_content: SchemaNode, ) -> Self { let default_string = if default_enabled { "ON".into() } else { "OFF".into() }; let control = { let mut nesting_info = nesting_info.clone(); nesting_info.path.push("content".into()); SettingControl::new(nesting_info, schema_content) }; Self { nesting_info, default_enabled, default_string, content_control: Box::new(control), } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let session_switch_mut = session_fragment.as_object_mut().unwrap(); let json::Value::Bool(enabled_mut) = &mut session_switch_mut["enabled"] else { unreachable!() }; let mut request = None; fn get_request(nesting_info: &NestingInfo, enabled: bool) -> Option { super::get_single_value(nesting_info, "enabled".into(), json::Value::Bool(enabled)) } ui.with_layout(Layout::left_to_right(Align::Center), |ui| { if alvr_gui_common::switch(ui, enabled_mut).clicked() { request = get_request(&self.nesting_info, *enabled_mut); } if reset::reset_button( ui, *enabled_mut != self.default_enabled, &self.default_string, ) .clicked() { request = get_request(&self.nesting_info, self.default_enabled); } }); if *enabled_mut { ui.end_row(); request = self .content_control .ui(ui, &mut session_switch_mut["content"], false) .or(request); } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/text.rs ================================================ use super::{NestingInfo, reset}; use alvr_packets::PathValuePair; use eframe::{ egui::{Layout, TextEdit, Ui}, emath::Align, }; use serde_json as json; pub struct Control { nesting_info: NestingInfo, editing_value: Option, default: String, default_string: String, } impl Control { pub fn new(nesting_info: NestingInfo, default: String) -> Self { let default_string = format!("\"{default}\""); Self { nesting_info, editing_value: None, default, default_string, } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); let json::Value::String(text_mut) = session_fragment else { unreachable!() }; let mut request = None; fn get_request(nesting_info: &NestingInfo, text: &str) -> Option { Some(PathValuePair { path: nesting_info.path.clone(), value: json::Value::String(text.to_owned()), }) } ui.with_layout(Layout::left_to_right(Align::Center), |ui| { let textbox = if let Some(editing_value_mut) = &mut self.editing_value { TextEdit::singleline(editing_value_mut) } else { TextEdit::singleline(text_mut) }; let response = ui.add(textbox.desired_width(250.)); if response.lost_focus() { if let Some(editing_value_mut) = &mut self.editing_value { request = get_request(&self.nesting_info, editing_value_mut); text_mut.clone_from(editing_value_mut); } self.editing_value = None; } if response.gained_focus() { self.editing_value = Some(text_mut.clone()); }; if reset::reset_button(ui, *text_mut != self.default, &self.default_string).clicked() { request = get_request(&self.nesting_info, &self.default); } }); request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/up_down.rs ================================================ use eframe::{ egui::{self, Button, Layout, Ui}, emath::Align, }; #[derive(PartialEq, Eq)] pub enum UpDownResult { Up, Down, None, } pub fn up_down_buttons(ui: &mut Ui, index: usize, count: usize) -> UpDownResult { ui.with_layout(Layout::top_down(Align::LEFT), |ui| { ui.spacing_mut().item_spacing = egui::vec2(0.0, 0.0); let up_clicked = ui .add_visible(index > 0, Button::new("⬆").small()) .clicked(); let down_clicked = ui .add_visible(index < count - 1, Button::new("⬇").small()) .clicked(); if up_clicked { UpDownResult::Up } else if down_clicked { UpDownResult::Down } else { UpDownResult::None } }) .inner } ================================================ FILE: alvr/dashboard/src/dashboard/components/settings_controls/vector.rs ================================================ use super::{INDENTATION_STEP, NestingInfo, SettingControl, reset}; use crate::dashboard::components::{ collapsible, up_down::{self, UpDownResult}, }; use alvr_packets::PathValuePair; use alvr_session::settings_schema::SchemaNode; use eframe::{ egui::{Layout, Ui}, emath::Align, }; use serde_json as json; pub struct Control { nesting_info: NestingInfo, default_element: SchemaNode, default: Vec, controls: Vec, } impl Control { pub fn new( nesting_info: NestingInfo, default_element: SchemaNode, default: Vec, ) -> Self { Self { nesting_info, default_element, default, controls: vec![], } } pub fn ui( &mut self, ui: &mut Ui, session_fragment: &mut json::Value, allow_inline: bool, ) -> Option { super::grid_flow_inline(ui, allow_inline); fn get_content_request( nesting_info: &NestingInfo, elements: Vec, ) -> Option { super::get_single_value(nesting_info, "content".into(), json::Value::Array(elements)) } let mut request = None; let collapsed = ui .with_layout(Layout::left_to_right(Align::Center), |ui| { let collapsed = collapsible::collapsible_button( ui, &self.nesting_info, session_fragment, &mut request, ); if reset::reset_button(ui, true, "default list").clicked() { request = get_content_request(&self.nesting_info, self.default.clone()) } collapsed }) .inner; let session_content = session_fragment["content"].as_array_mut().unwrap(); while session_content.len() > self.controls.len() { let mut nesting_info = self.nesting_info.clone(); nesting_info.path.push("content".into()); nesting_info.path.push(self.controls.len().into()); self.controls.push(SettingControl::new( nesting_info, self.default_element.clone(), )) } while session_content.len() < self.controls.len() { self.controls.pop(); } if !collapsed { ui.end_row(); let mut idx = 0; while idx < self.controls.len() { let delete_element = ui .horizontal(|ui| { ui.add_space(INDENTATION_STEP * self.nesting_info.indentation_level as f32); let delete_element = ui.button("❌").clicked(); let up_down_result = up_down::up_down_buttons(ui, idx, self.controls.len()); if up_down_result != UpDownResult::None { if up_down_result == UpDownResult::Up { session_content.swap(idx, idx - 1); } else { session_content.swap(idx, idx + 1); } request = get_content_request(&self.nesting_info, session_content.clone()); } delete_element }) .inner; if delete_element { session_content.remove(idx); self.controls.remove(idx); request = get_content_request(&self.nesting_info, session_content.clone()); } else { request = self.controls[idx] .ui(ui, &mut session_content[idx], true) .or(request); } ui.end_row(); idx += 1; } ui.label(" "); if ui.button("Add element").clicked() { let mut session_content = session_fragment["content"].as_array_mut().unwrap().clone(); session_content.push(session_fragment["element"].clone()); request = get_content_request(&self.nesting_info, session_content); } } request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/setup_wizard.rs ================================================ use crate::dashboard::ServerRequest; use eframe::{ egui::{Button, Label, Layout, RichText, Ui}, emath::Align, }; pub enum SetupWizardRequest { ServerRequest(ServerRequest), Close { finished: bool }, } #[derive(Clone, Copy, PartialEq, Eq)] enum Page { Welcome = 0, ResetSettings = 1, HardwareRequirements = 2, SoftwareRequirements = 3, Firewall = 4, Recommendations = 5, Finished = 6, } fn index_to_page(index: usize) -> Page { match index { 0 => Page::Welcome, 1 => Page::ResetSettings, 2 => Page::HardwareRequirements, 3 => Page::SoftwareRequirements, 4 => Page::Firewall, 5 => Page::Recommendations, 6 => Page::Finished, _ => panic!("Invalid page index"), } } fn page_content( ui: &mut Ui, subtitle: &str, paragraph: &str, interactible_content: impl FnMut(&mut Ui), ) { ui.with_layout(Layout::right_to_left(Align::Min), |ui| { ui.add_space(60.0); ui.with_layout(Layout::left_to_right(Align::Min), |ui| { ui.add_space(60.0); ui.with_layout(Layout::top_down(Align::LEFT), |ui| { ui.add_space(15.0); ui.heading(RichText::new(subtitle).size(20.0)); ui.add(Label::new(RichText::new(paragraph).size(14.0)).wrap()); ui.add_space(30.0); ui.vertical_centered(interactible_content); }); }) }); } pub struct SetupWizard { page: Page, } impl SetupWizard { pub fn new() -> Self { Self { page: Page::Welcome, } } pub fn ui(&mut self, ui: &mut Ui) -> Option { let mut request = None; ui.horizontal(|ui| { ui.add_space(60.0); ui.vertical(|ui| { ui.add_space(30.0); ui.heading(RichText::new("Welcome to ALVR").size(30.0)); ui.add_space(5.0); }); ui.with_layout(Layout::right_to_left(Align::Center), |ui| { ui.add_space(15.0); if ui.button("❌").clicked() { request = Some(SetupWizardRequest::Close { finished: false }); } }) }); ui.separator(); match &self.page { Page::Welcome => page_content( ui, "This setup wizard will help you setup ALVR.", "", |_| (), ), Page::ResetSettings => page_content( ui, "Reset settings", "It is recommended to reset your settings everytime you update ALVR.", |ui| { if ui.button("Reset settings").clicked() { request = Some(SetupWizardRequest::ServerRequest( ServerRequest::UpdateSession(Box::default()), )); } }, ), Page::HardwareRequirements => page_content( ui, "Hardware requirements", r"ALVR requires a dedicated and recent graphics card. Low-end Intel integrated graphics may fail to work. Make sure you have at least one output audio device.", |_| (), ), Page::SoftwareRequirements => page_content( ui, "Software requirements", if cfg!(windows) { r"To stream the headset microphone on Windows you need to install Virtual Audio Cable, VB-Cable, Voicemeeter" } else if cfg!(target_os = "linux") { r"You need the PipeWire (0.3.49+ version) audio system to be able to stream audio and use microphone." } else { r"Unsupported OS" }, #[allow(unused_variables)] |ui| { #[cfg(windows)] if ui.button("Download Virtual Audio Cable (Lite)").clicked() { ui.ctx().open_url(eframe::egui::OpenUrl::same_tab( "https://software.muzychenko.net/freeware/vac470lite.zip", )); } }, ), Page::Firewall => page_content( ui, "Firewall", r"To communicate with the headset, some firewall rules need to be set. This requires administrator rights!", |ui| { if ui.button("Add firewall rules").clicked() { request = Some(SetupWizardRequest::ServerRequest( ServerRequest::AddFirewallRules, )); } }, ), Page::Recommendations => page_content( ui, "Recommendations", r"ALVR supports multiple types of PC hardware and headsets but not all might work correctly with default settings. Please try tweaking different settings like resolution, bitrate, encoder and others if your ALVR experience is not optimal.", |_| (), ), Page::Finished => page_content( ui, "Finished", r#"You can always restart this setup wizard from the "Installation" tab on the left."#, |_| (), ), }; ui.with_layout(Layout::bottom_up(Align::RIGHT), |ui| { ui.add_space(30.0); ui.horizontal(|ui| { ui.add_space(15.0); if self.page == Page::Finished { if ui.button("Finish").clicked() { request = Some(SetupWizardRequest::Close { finished: true }); } } else if ui.button("Next").clicked() { self.page = index_to_page(self.page as usize + 1); } if ui .add_visible(self.page != Page::Welcome, Button::new("Back")) .clicked() { self.page = index_to_page(self.page as usize - 1); } }); ui.separator(); }); request } } ================================================ FILE: alvr/dashboard/src/dashboard/components/statistics.rs ================================================ use crate::dashboard::{ServerRequest, theme::graph_colors}; use alvr_events::{GraphStatistics, StatisticsSummary}; use alvr_gui_common::theme; use eframe::{ egui::{ Align2, Color32, CornerRadius, FontId, Frame, Grid, Painter, Rect, RichText, ScrollArea, Shape, Stroke, Ui, pos2, vec2, }, emath::RectTransform, epaint::Pos2, }; use statrs::statistics::{self, OrderStatistics}; use std::{collections::VecDeque, ops::RangeInclusive}; const GRAPH_HISTORY_SIZE: usize = 1000; const UPPER_QUANTILE: f64 = 0.90; fn draw_lines(painter: &Painter, points: Vec, color: Color32) { painter.add(Shape::line(points, Stroke::new(1.0, color))); } pub struct StatisticsTab { history: VecDeque, last_statistics_summary: Option, } impl StatisticsTab { pub fn new() -> Self { Self { history: vec![GraphStatistics::default(); GRAPH_HISTORY_SIZE] .into_iter() .collect(), last_statistics_summary: None, } } pub fn update_statistics(&mut self, statistics: StatisticsSummary) { self.last_statistics_summary = Some(statistics); } pub fn update_graph_statistics(&mut self, statistics: GraphStatistics) { self.history.pop_front(); self.history.push_back(statistics); } pub fn ui(&self, ui: &mut Ui) -> Option { if let Some(stats) = &self.last_statistics_summary { ScrollArea::new([false, true]).show(ui, |ui| { let available_width = ui.available_width(); self.draw_latency_graph(ui, available_width); self.draw_fps_graph(ui, available_width); self.draw_bitrate_graph(ui, available_width); self.draw_statistics_overview(ui, stats); }); } else { ui.heading( "No statistics available. Start SteamVR and connect to a device to gather statistics.", ); } None } fn draw_graph( &self, ui: &mut Ui, available_width: f32, title: &str, data_range: RangeInclusive, graph_content: impl FnOnce(&Painter, RectTransform), tooltip_content: impl FnOnce(&mut Ui, &GraphStatistics), ) { ui.add_space(10.0); ui.label(RichText::new(title).size(20.0)); let canvas_response = Frame::canvas(ui.style()).show(ui, |ui| { ui.ctx().request_repaint(); let size = available_width * vec2(1.0, 0.2); let (_id, canvas_rect) = ui.allocate_space(size); let max = *data_range.end(); let min = *data_range.start(); let data_rect = Rect::from_x_y_ranges(0.0..=GRAPH_HISTORY_SIZE as f32, max..=min); let to_screen = RectTransform::from_to(data_rect, canvas_rect); let painter = ui.painter().with_clip_rect(canvas_rect); if max == min { // Drawing using a 0 sized rectangle causes a crash return data_rect; } graph_content(&painter, to_screen); ui.painter().text( to_screen * pos2(0.0, min), Align2::LEFT_BOTTOM, format!("{min:.0}"), FontId::monospace(20.0), Color32::GRAY, ); ui.painter().text( to_screen * pos2(0.0, max), Align2::LEFT_TOP, format!("{max:.0}"), FontId::monospace(20.0), Color32::GRAY, ); data_rect }); if let Some(pos) = canvas_response.response.hover_pos() { let graph_pos = RectTransform::from_to(canvas_response.response.rect, canvas_response.inner) * pos; let history_index = (graph_pos.x as usize).clamp(0, GRAPH_HISTORY_SIZE - 1); canvas_response .response .on_hover_ui_at_pointer(|ui| tooltip_content(ui, &self.history[history_index])); } } fn draw_latency_graph(&self, ui: &mut Ui, available_width: f32) { let mut data = statistics::Data::new( self.history .iter() .map(|stats| stats.total_pipeline_latency_s as f64) .collect::>(), ); self.draw_graph( ui, available_width, "Latency", 0.0..=(data.quantile(UPPER_QUANTILE) * 1.2) as f32 * 1000.0, |painter, to_screen_trans| { for i in 0..GRAPH_HISTORY_SIZE { let stats = &self.history[i]; let mut offset = 0.0; for (value, color) in &[ (stats.game_time_s, graph_colors::RENDER_EXTERNAL), (stats.server_compositor_s, graph_colors::RENDER), (stats.encoder_s, graph_colors::TRANSCODE), (stats.network_s, graph_colors::NETWORK), (stats.decoder_s, graph_colors::TRANSCODE), (stats.decoder_queue_s, graph_colors::IDLE), (stats.client_compositor_s, graph_colors::RENDER), (stats.vsync_queue_s, graph_colors::RENDER_EXTERNAL), ] { painter.rect_filled( Rect { min: to_screen_trans * pos2(i as f32, offset + value * 1000.0), max: to_screen_trans * pos2(i as f32 + 2.0, offset), }, CornerRadius::ZERO, *color, ); offset += value * 1000.0; } } }, |ui, stats| { use graph_colors::*; Grid::new("latency_tooltip").num_columns(2).show(ui, |ui| { fn label(ui: &mut Ui, text: &str, value_s: f32, color: Color32) { ui.colored_label(color, text); ui.colored_label(color, format!("{:.2}ms", value_s * 1000.0)); ui.end_row(); } let transmission_total_latency_s = stats.server_compositor_s + stats.encoder_s + stats.network_s + stats.decoder_s + stats.decoder_queue_s + stats.client_compositor_s; label( ui, "Motion to Photon Latency", stats.total_pipeline_latency_s, theme::FG, ); label(ui, "ALVR Latency", transmission_total_latency_s, theme::FG); label( ui, "Client System (not ALVR latency)", stats.vsync_queue_s, RENDER_EXTERNAL_LABEL, ); label( ui, "Client App Compositor", stats.client_compositor_s, RENDER, ); label(ui, "Frame Buffering", stats.decoder_queue_s, IDLE); label(ui, "Decode", stats.decoder_s, TRANSCODE); label(ui, "Network", stats.network_s, NETWORK); label(ui, "Encode", stats.encoder_s, TRANSCODE); label(ui, "Streamer Compositor", stats.server_compositor_s, RENDER); label( ui, "Game Render (not ALVR latency)", stats.game_time_s, RENDER_EXTERNAL_LABEL, ); }); }, ); } fn draw_fps_graph(&self, ui: &mut Ui, available_width: f32) { let mut data = statistics::Data::new( self.history .iter() .map(|stats| stats.client_fps) .chain(self.history.iter().map(|stats| stats.server_fps)) .map(|v| v as f64) .collect::>(), ); let upper_quantile = data.quantile(UPPER_QUANTILE); let lower_quantile = data.quantile(1.0 - UPPER_QUANTILE); let max = upper_quantile + (upper_quantile - lower_quantile); let min = f64::max(0.0, lower_quantile - (upper_quantile - lower_quantile)); self.draw_graph( ui, available_width, "Framerate", min as f32..=max as f32, |painter, to_screen_trans| { let (server_fps_points, client_fps_points) = (0..GRAPH_HISTORY_SIZE) .map(|i| { ( to_screen_trans * pos2(i as f32, self.history[i].server_fps), to_screen_trans * pos2(i as f32, self.history[i].client_fps), ) }) .unzip(); draw_lines(painter, server_fps_points, graph_colors::SERVER_FPS); draw_lines(painter, client_fps_points, graph_colors::CLIENT_FPS); }, |ui, stats| { Grid::new("fps_tooltip").num_columns(2).show(ui, |ui| { fn label(ui: &mut Ui, text: &str, value: f32, color: Color32) { ui.colored_label(color, text); ui.colored_label(color, format!("{value:.2}Hz")); ui.end_row(); } label(ui, "Server FPS", stats.server_fps, graph_colors::SERVER_FPS); label(ui, "Client FPS", stats.client_fps, graph_colors::CLIENT_FPS); }); }, ); } fn draw_bitrate_graph(&self, ui: &mut Ui, available_width: f32) { let mut data = statistics::Data::new( self.history .iter() .map(|stats| stats.throughput_bps as f64) .collect::>(), ); self.draw_graph( ui, available_width, "Bitrate and Throughput", 0.0..=(data.quantile(UPPER_QUANTILE) * 1.2) as f32 / 1e6, |painter, to_screen_trans| { let mut scaled_calculated = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut decoder_latency_limiter = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut network_latency_limiter = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut encoder_latency_limiter = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut max_throughput = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut min_throughput = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut requested_bitrate = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut recorded_throughput = Vec::with_capacity(GRAPH_HISTORY_SIZE); let mut recorded_bitrate = Vec::with_capacity(GRAPH_HISTORY_SIZE); for i in 0..GRAPH_HISTORY_SIZE { let d = &self.history[i].bitrate_directives; if let Some(value) = d.scaled_calculated_throughput_bps { scaled_calculated.push(to_screen_trans * pos2(i as f32, value / 1e6)) } if let Some(value) = d.decoder_latency_limiter_bps { decoder_latency_limiter.push(to_screen_trans * pos2(i as f32, value / 1e6)) } if let Some(value) = d.network_latency_limiter_bps { network_latency_limiter.push(to_screen_trans * pos2(i as f32, value / 1e6)) } if let Some(value) = d.encoder_latency_limiter_bps { encoder_latency_limiter.push(to_screen_trans * pos2(i as f32, value / 1e6)) } if let Some(value) = d.manual_max_throughput_bps { max_throughput.push(to_screen_trans * pos2(i as f32, value / 1e6)) } if let Some(value) = d.manual_min_throughput_bps { min_throughput.push(to_screen_trans * pos2(i as f32, value / 1e6)) } requested_bitrate .push(to_screen_trans * pos2(i as f32, d.requested_bitrate_bps / 1e6)); recorded_throughput.push( to_screen_trans * pos2(i as f32, self.history[i].throughput_bps / 1e6), ); recorded_bitrate .push(to_screen_trans * pos2(i as f32, self.history[i].bitrate_bps / 1e6)); } draw_lines( painter, scaled_calculated, graph_colors::INITIAL_CALCULATED_THROUGHPUT, ); draw_lines( painter, encoder_latency_limiter, graph_colors::ENCODER_DECODER_LATENCY_LIMITER, ); draw_lines( painter, network_latency_limiter, graph_colors::NETWORK_LATENCY_LIMITER, ); draw_lines( painter, decoder_latency_limiter, graph_colors::ENCODER_DECODER_LATENCY_LIMITER, ); draw_lines( painter, max_throughput, graph_colors::MIN_MAX_LATENCY_THROUGHPUT, ); draw_lines( painter, min_throughput, graph_colors::MIN_MAX_LATENCY_THROUGHPUT, ); draw_lines(painter, requested_bitrate, graph_colors::REQUESTED_BITRATE); draw_lines( painter, recorded_throughput, graph_colors::RECORDED_THROUGHPUT, ); draw_lines(painter, recorded_bitrate, theme::FG); }, |ui, stats| { Grid::new("bitrate_tooltip").num_columns(2).show(ui, |ui| { fn maybe_label( ui: &mut Ui, text: &str, maybe_value_bps: Option, color: Color32, ) { if let Some(value) = maybe_value_bps { ui.colored_label(color, text); ui.colored_label(color, format!("{:.2} Mbps", value / 1e6)); ui.end_row(); } } let td = &stats.bitrate_directives; maybe_label( ui, "Initial calculated throughput", td.scaled_calculated_throughput_bps, graph_colors::INITIAL_CALCULATED_THROUGHPUT, ); maybe_label( ui, "Encoder latency limiter", td.encoder_latency_limiter_bps, graph_colors::ENCODER_DECODER_LATENCY_LIMITER, ); maybe_label( ui, "Network latency limiter", td.network_latency_limiter_bps, graph_colors::NETWORK_LATENCY_LIMITER, ); maybe_label( ui, "Decoder latency limiter", td.decoder_latency_limiter_bps .filter(|l| *l < stats.throughput_bps), graph_colors::ENCODER_DECODER_LATENCY_LIMITER, ); maybe_label( ui, "Manual max throughput", td.manual_max_throughput_bps, graph_colors::MIN_MAX_LATENCY_THROUGHPUT, ); maybe_label( ui, "Manual min throughput", td.manual_min_throughput_bps, graph_colors::MIN_MAX_LATENCY_THROUGHPUT, ); maybe_label( ui, "Requested bitrate", Some(td.requested_bitrate_bps), graph_colors::REQUESTED_BITRATE, ); maybe_label( ui, "Recorded throughput", Some(stats.throughput_bps), graph_colors::RECORDED_THROUGHPUT, ); maybe_label( ui, "Recorded bitrate", Some(stats.bitrate_bps), graph_colors::RECORDED_BITRATE, ); }); ui.small("Note: throughput is the peak bitrate, packet_size/network_latency."); }, ) } fn draw_statistics_overview(&self, ui: &mut Ui, statistics: &StatisticsSummary) { ui.add_space(10.0); ui.columns(2, |ui| { ui[0].label("Total packets:"); ui[1].label(format!( "{} packets ({} packets/s)", statistics.video_packets_total, statistics.video_packets_per_sec )); ui[0].label("Total sent:"); ui[1].label(format!("{} MB", statistics.video_mbytes_total)); ui[0].label("Bitrate:"); ui[1].label(format!("{:.1} Mbps", statistics.video_mbits_per_sec)); ui[0].label("Total latency:"); ui[1].label(format!("{:.0} ms", statistics.total_latency_ms)); ui[0].label("Encoder latency:"); ui[1].label(format!("{:.2} ms", statistics.encode_latency_ms)); ui[0].label("Transport latency:"); ui[1].label(format!("{:.2} ms", statistics.network_latency_ms)); ui[0].label("Decoder latency:"); ui[1].label(format!("{:.2} ms", statistics.decode_latency_ms)); ui[0].label("Client FPS:"); ui[1].label(format!("{} FPS", statistics.client_fps)); ui[0].label("Streamer FPS:"); ui[1].label(format!("{} FPS", statistics.server_fps)); ui[0].label("Headset battery"); ui[1].label(format!( "{}% ({})", statistics.battery_hmd, if statistics.hmd_plugged { "plugged" } else { "unplugged" } )); }); } } ================================================ FILE: alvr/dashboard/src/dashboard/mod.rs ================================================ mod components; use self::components::{ DevicesTab, LogsTab, NotificationBar, SettingsTab, SetupWizard, SetupWizardRequest, }; use crate::{ DataSources, dashboard::components::{CloseAction, NewVersionPopup, StatisticsTab}, }; use alvr_common::{ LogEntry, parking_lot::{Condvar, Mutex}, }; use alvr_events::EventType; use alvr_gui_common::theme; use alvr_packets::{ClientConnectionsAction, PathValuePair}; use alvr_session::SessionConfig; use eframe::egui::{ self, Align, CentralPanel, Direction, Frame, Layout, Margin, RichText, SidePanel, }; use serde::{Deserialize, Serialize}; use std::{collections::BTreeMap, path::PathBuf, sync::Arc}; #[derive(Serialize, Deserialize, Debug)] pub enum ServerRequest { Log(LogEntry), GetSession, UpdateSession(Box), SetSessionValues(Vec), UpdateClientList { hostname: String, action: ClientConnectionsAction, }, CaptureFrame, InsertIdr, StartRecording, StopRecording, AddFirewallRules, RemoveFirewallRules, GetDriverList, RegisterAlvrDriver, UnregisterDriver(PathBuf), RestartSteamvr, ShutdownSteamvr, } #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug)] enum Tab { Devices, Statistics, Settings, #[cfg(not(target_arch = "wasm32"))] Installation, Logs, Debug, About, } pub struct Dashboard { data_sources: DataSources, just_opened: bool, server_restarting: Arc>, server_restarting_condvar: Arc, selected_tab: Tab, tab_labels: BTreeMap, connections_tab: DevicesTab, statistics_tab: StatisticsTab, settings_tab: SettingsTab, #[cfg(not(target_arch = "wasm32"))] installation_tab: components::InstallationTab, logs_tab: LogsTab, notification_bar: NotificationBar, setup_wizard: SetupWizard, new_version_popup: Option, setup_wizard_open: bool, session: Option, } impl Dashboard { pub fn new(creation_context: &eframe::CreationContext<'_>, data_sources: DataSources) -> Self { alvr_gui_common::theme::set_theme(&creation_context.egui_ctx); data_sources.request(ServerRequest::GetSession); Self { data_sources, just_opened: true, server_restarting: Arc::new(Mutex::new(false)), server_restarting_condvar: Arc::new(Condvar::new()), selected_tab: Tab::Devices, tab_labels: [ (Tab::Devices, "🔌 Devices"), (Tab::Statistics, "📈 Statistics"), (Tab::Settings, "🔧 Settings"), #[cfg(not(target_arch = "wasm32"))] (Tab::Installation, "💾 Installation"), (Tab::Logs, "📝 Logs"), (Tab::Debug, "🐞 Debug"), (Tab::About, "ℹ About"), ] .into_iter() .collect(), connections_tab: DevicesTab::new(), statistics_tab: StatisticsTab::new(), settings_tab: SettingsTab::new(), #[cfg(not(target_arch = "wasm32"))] installation_tab: components::InstallationTab::new(), logs_tab: LogsTab::new(), notification_bar: NotificationBar::new(), setup_wizard: SetupWizard::new(), setup_wizard_open: false, session: None, new_version_popup: None, } } // This call may block fn restart_steamvr(&self, requests: &mut Vec) { requests.push(ServerRequest::RestartSteamvr); let mut server_restarting_lock = self.server_restarting.lock(); if *server_restarting_lock { self.server_restarting_condvar .wait(&mut server_restarting_lock); } *server_restarting_lock = true; #[cfg(not(target_arch = "wasm32"))] std::thread::spawn({ let server_restarting = Arc::clone(&self.server_restarting); let condvar = Arc::clone(&self.server_restarting_condvar); move || { crate::steamvr_launcher::LAUNCHER.lock().restart_steamvr(); *server_restarting.lock() = false; condvar.notify_one(); } }); } } impl eframe::App for Dashboard { fn update(&mut self, context: &egui::Context, _: &mut eframe::Frame) { let mut requests = vec![]; let connected_to_server = self.data_sources.server_connected(); while let Some(event) = self.data_sources.poll_event() { self.logs_tab.push_event(event.inner.clone()); match event.inner.event_type { EventType::Log(log_event) => { self.notification_bar .push_notification(log_event, event.from_dashboard); } EventType::GraphStatistics(graph_statistics) => self .statistics_tab .update_graph_statistics(graph_statistics), EventType::StatisticsSummary(statistics) => { self.statistics_tab.update_statistics(statistics) } EventType::Session(session) => { let settings = session.to_settings(); self.connections_tab.update_client_list(&session); self.settings_tab.update_session(&session.session_settings); self.logs_tab.update_settings(&settings); self.notification_bar.update_settings(&settings); if self.just_opened { if settings.extra.open_setup_wizard { self.setup_wizard_open = true; } self.just_opened = false; } self.session = Some(*session); } EventType::ServerRequestsSelfRestart => self.restart_steamvr(&mut requests), #[cfg(not(target_arch = "wasm32"))] EventType::DriversList(list) => self.installation_tab.update_drivers(list), EventType::Adb(adb_event) => self .connections_tab .update_adb_download_progress(adb_event.download_progress), EventType::NewVersionFound { version, message } => { self.new_version_popup = Some(NewVersionPopup::new(version, message)); } EventType::DebugGroup { .. } | EventType::Tracking(_) | EventType::Buttons(_) | EventType::Haptics(_) => (), } } if *self.server_restarting.lock() { CentralPanel::default().show(context, |ui| { // todo: find a way to center both vertically and horizontally ui.vertical_centered(|ui| { ui.add_space(100.0); ui.heading(RichText::new("SteamVR is restarting").size(30.0)); }); }); return; } self.notification_bar.ui(context); if self.setup_wizard_open { CentralPanel::default().show(context, |ui| { if let Some(request) = self.setup_wizard.ui(ui) { match request { SetupWizardRequest::ServerRequest(request) => { requests.push(request); } SetupWizardRequest::Close { finished } => { if finished { requests.push(ServerRequest::SetSessionValues(vec![ PathValuePair { path: alvr_packets::parse_path( "session_settings.extra.open_setup_wizard", ), value: serde_json::Value::Bool(false), }, ])) } self.setup_wizard_open = false; } } } }); } else { SidePanel::left("side_panel") .resizable(false) .frame( Frame::new() .fill(theme::LIGHTER_BG) .inner_margin(Margin::same(7)), ) .exact_width(160.0) .show(context, |ui| { ui.with_layout(Layout::top_down_justified(Align::Center), |ui| { ui.add_space(13.0); ui.heading(RichText::new("ALVR").size(25.0).strong()); egui::warn_if_debug_build(ui); }); ui.with_layout(Layout::top_down_justified(Align::Min), |ui| { for (tab, label) in &self.tab_labels { ui.selectable_value(&mut self.selected_tab, *tab, *label); } }); #[cfg(not(target_arch = "wasm32"))] ui.with_layout( Layout::bottom_up(Align::Center).with_cross_justify(true), |ui| { ui.add_space(5.0); if connected_to_server { if ui.button("Restart SteamVR").clicked() { self.restart_steamvr(&mut requests); } } else if ui.button("Launch SteamVR").clicked() { crate::steamvr_launcher::LAUNCHER.lock().launch_steamvr(); } ui.horizontal(|ui| { ui.add_space(4.0); ui.label(RichText::new("SteamVR:").size(13.0)); ui.add_space(-10.0); ui.with_layout( Layout::centered_and_justified(Direction::LeftToRight), |ui| { ui.label( if connected_to_server { RichText::new("Connected").color(theme::OK_GREEN) } else { RichText::new("Disconnected").color(theme::KO_RED) } .size(13.0), ) }, ); }) }, ) }); CentralPanel::default() .frame(Frame::new().inner_margin(Margin::same(20)).fill(theme::BG)) .show(context, |ui| { ui.with_layout(Layout::top_down_justified(Align::LEFT), |ui| { ui.heading(RichText::new(self.tab_labels[&self.selected_tab]).size(25.0)); match self.selected_tab { Tab::Devices => { requests.extend(self.connections_tab.ui(ui, connected_to_server)); } Tab::Statistics => { if let Some(request) = self.statistics_tab.ui(ui) { requests.push(request); } } Tab::Settings => { requests.extend(self.settings_tab.ui(ui)); } #[cfg(not(target_arch = "wasm32"))] Tab::Installation => { for request in self.installation_tab.ui(ui) { match request { components::InstallationTabRequest::OpenSetupWizard => { self.setup_wizard_open = true } components::InstallationTabRequest::ServerRequest( request, ) => { requests.push(request); } } } } Tab::Logs => self.logs_tab.ui(ui), Tab::Debug => { if let Some(request) = components::debug_tab_ui(ui) { requests.push(request); } } Tab::About => components::about_tab_ui(ui), } }) }); } let shutdown_alvr = || { self.data_sources.request(ServerRequest::ShutdownSteamvr); crate::steamvr_launcher::LAUNCHER .lock() .ensure_steamvr_shutdown(); }; if let Some(popup) = &self.new_version_popup && let Some(action) = popup.ui(context, shutdown_alvr) { if let CloseAction::CloseWithRequest(request) = action { requests.push(request); } self.new_version_popup = None; } for request in requests { self.data_sources.request(request); } if context.input(|state| state.viewport().close_requested()) && self.session.as_ref().is_some_and(|s| { s.to_settings() .extra .steamvr_launcher .open_close_steamvr_with_dashboard }) { shutdown_alvr(); } } } ================================================ FILE: alvr/dashboard/src/data_sources.rs ================================================ use crate::dashboard::ServerRequest; use alvr_common::{ ALVR_VERSION, RelaxedAtomic, debug, error, info, parking_lot::Mutex, semver::{Version, VersionReq}, warn, }; use alvr_events::{Event, EventType}; use alvr_packets::FirewallRulesAction; use alvr_server_io::ServerSessionManager; use eframe::egui; use serde::Serialize; use std::{ io::ErrorKind, net::{SocketAddr, TcpStream}, str::FromStr, sync::{Arc, mpsc}, thread::{self, JoinHandle}, time::{Duration, Instant}, }; use tungstenite::{ client::IntoClientRequest, http::{HeaderValue, Uri}, }; const LOCAL_REQUEST_TIMEOUT: Duration = Duration::from_millis(200); const REMOTE_REQUEST_TIMEOUT: Duration = Duration::from_secs(2); enum SessionSource { Local(Box), Remote, // Note: the remote (server) is probably living as a separate process in the same PC } fn get_local_session_source() -> ServerSessionManager { let session_file_path = crate::get_filesystem_layout().session(); ServerSessionManager::new(Some(session_file_path)) } pub fn clean_session() { let mut session_manager = get_local_session_source(); session_manager.clean_client_list(); #[cfg(target_os = "linux")] { let has_nvidia = wgpu::Instance::new(&wgpu::InstanceDescriptor { backends: wgpu::Backends::VULKAN, ..Default::default() }) .enumerate_adapters(wgpu::Backends::VULKAN) .iter() .any(|adapter| adapter.get_info().vendor == 0x10de); if has_nvidia { session_manager .session_mut() .session_settings .extra .patches .linux_async_reprojection = false; } } if session_manager.session().server_version != *ALVR_VERSION { let mut session_ref = session_manager.session_mut(); session_ref.server_version = ALVR_VERSION.clone(); session_ref.client_connections.clear(); session_ref.session_settings.extra.open_setup_wizard = true; session_ref .session_settings .extra .new_version_popup .content .hide_while_version = ALVR_VERSION.to_string(); } } // Disallows all methods for mutating (and overwriting to disk) the session pub fn get_read_only_local_session() -> Arc { Arc::new(get_local_session_source()) } fn report_event_local( context: &egui::Context, sender: &mpsc::Sender, event_type: EventType, ) { sender .send(PolledEvent { inner: Event { timestamp: "".into(), event_type, }, from_dashboard: false, }) .ok(); context.request_repaint(); } fn report_session_local( context: &egui::Context, sender: &mpsc::Sender, session_manager: &ServerSessionManager, ) { report_event_local( context, sender, EventType::Session(Box::new(session_manager.session().clone())), ) } pub struct PolledEvent { pub inner: Event, pub from_dashboard: bool, } pub struct DataSources { running: Arc, requests_sender: mpsc::Sender, events_receiver: mpsc::Receiver, server_connected: Arc, version_check_thread: Option>>, requests_thread: Option>, events_thread: Option>, ping_thread: Option>, } impl DataSources { pub fn new( context: egui::Context, events_sender: mpsc::Sender, events_receiver: mpsc::Receiver, ) -> Self { let filesystem_layout = crate::get_filesystem_layout(); let running = Arc::new(RelaxedAtomic::new(true)); let (requests_sender, requests_receiver) = mpsc::channel(); let server_connected = Arc::new(RelaxedAtomic::new(false)); let session_manager = get_local_session_source(); let port = session_manager.settings().connection.web_server_port; let session_source = Arc::new(Mutex::new(SessionSource::Local(Box::new(session_manager)))); let version_check_thread = thread::spawn({ let context = context.clone(); let session_source = Arc::clone(&session_source); let events_sender = events_sender.clone(); move || { let version_requirement = { // Best-effort: the check will succeed only when the server is not already running, // no retries let SessionSource::Local(session_manager_lock) = &mut *session_source.lock() else { return None; }; let version = &session_manager_lock .settings() .extra .new_version_popup .as_option()? .hide_while_version; VersionReq::parse(&format!(">{version}")).unwrap() }; let request_agent: ureq::Agent = ureq::Agent::config_builder() .timeout_global(Some(REMOTE_REQUEST_TIMEOUT)) .build() .into(); if let Ok(response) = request_agent .get("https://api.github.com/repos/alvr-org/ALVR/releases/latest") .call() { let version_data = response.into_body().read_json::().ok()?; let version_str = version_data .get("tag_name") .and_then(|v| Some(v.as_str()?.trim_start_matches("v")))?; let version = version_str.parse::().ok(); if version .map(|v| version_requirement.matches(&v)) .unwrap_or(false) { let message = version_data .get("body") .and_then(|v| v.as_str()) .unwrap_or("Error parsing release body"); report_event_local( &context, &events_sender, EventType::NewVersionFound { version: version_str.to_string(), message: message.to_string(), }, ); } } None } }); let requests_thread = thread::spawn({ let running = Arc::clone(&running); let context = context.clone(); let session_source = Arc::clone(&session_source); let events_sender = events_sender.clone(); move || { let base_uri = format!("http://127.0.0.1:{port}"); let rq: ureq::Agent = ureq::Agent::config_builder() .timeout_global(Some(LOCAL_REQUEST_TIMEOUT)) .build() .into(); while running.value() { while let Ok(request) = requests_receiver.try_recv() { debug!( "Dashboard request: {}", serde_json::to_string(&request).unwrap() ); if let SessionSource::Local(session_manager) = &mut *session_source.lock() { match request { ServerRequest::Log(_) => (), ServerRequest::GetSession => { report_session_local(&context, &events_sender, session_manager); } ServerRequest::UpdateSession(session) => { *session_manager.session_mut() = *session; report_session_local(&context, &events_sender, session_manager); } ServerRequest::SetSessionValues(values) => { if let Err(e) = session_manager.set_session_values(values) { error!("Failed to set session value: {e}") } report_session_local(&context, &events_sender, session_manager); } ServerRequest::UpdateClientList { hostname, action } => { session_manager.update_client_connections(hostname, action); report_session_local(&context, &events_sender, session_manager); } ServerRequest::AddFirewallRules => { if let Err(e) = alvr_server_io::firewall_rules( FirewallRulesAction::Add, &filesystem_layout, ) { error!("Failed to add firewall rules! code: {e}"); } else { info!("Successfully added firewall rules!"); } } ServerRequest::RemoveFirewallRules => { if let Err(e) = alvr_server_io::firewall_rules( FirewallRulesAction::Remove, &filesystem_layout, ) { error!("Failed to remove firewall rules! code: {e}"); } else { info!("Successfully removed firewall rules!"); } } ServerRequest::RegisterAlvrDriver => { let alvr_driver_dir = filesystem_layout.openvr_driver_root_dir.clone(); alvr_server_io::driver_registration(&[alvr_driver_dir], true) .ok(); if let Ok(list) = alvr_server_io::get_registered_drivers() { report_event_local( &context, &events_sender, EventType::DriversList(list), ) } } ServerRequest::UnregisterDriver(path) => { alvr_server_io::driver_registration(&[path], false).ok(); if let Ok(list) = alvr_server_io::get_registered_drivers() { report_event_local( &context, &events_sender, EventType::DriversList(list), ) } } ServerRequest::GetDriverList => { if let Ok(list) = alvr_server_io::get_registered_drivers() { report_event_local( &context, &events_sender, EventType::DriversList(list), ) } } ServerRequest::CaptureFrame | ServerRequest::InsertIdr | ServerRequest::StartRecording | ServerRequest::StopRecording => { warn!( "Cannot perform action, streamer (SteamVR) is not connected." ) } ServerRequest::RestartSteamvr | ServerRequest::ShutdownSteamvr => { warn!("Streamer not launched, can't signal SteamVR shutdown") } } } else { let get = |path: &str| { rq.get(format!("{base_uri}/api/{path}")) .header("X-ALVR", "true") .call() .ok(); }; fn post_body( rq: &ureq::Agent, base_uri: &str, path: &str, body: Option, ) { let builder = rq .post(format!("{base_uri}/api/{path}")) .header("X-ALVR", "true"); if let Some(body) = body { builder.send_json(body).ok(); } else { builder.send_empty().ok(); } } let post = |path: &str| post_body(&rq, &base_uri, path, None::<()>); match request { ServerRequest::Log(entry) => { post_body(&rq, &base_uri, "log", Some(entry)) } ServerRequest::GetSession => get("session"), ServerRequest::UpdateSession(session) => { post_body(&rq, &base_uri, "session", Some(&*session)) } ServerRequest::SetSessionValues(values) => { post_body(&rq, &base_uri, "session/values", Some(values)) } ServerRequest::UpdateClientList { hostname, action } => post_body( &rq, &base_uri, "session/client-connections", Some((hostname, action)), ), ServerRequest::AddFirewallRules => post("firewall-rules/add"), ServerRequest::RemoveFirewallRules => post("firewall-rules/remove"), ServerRequest::GetDriverList => get("drivers"), ServerRequest::RegisterAlvrDriver => post("drivers/register-alvr"), ServerRequest::UnregisterDriver(path) => { post_body(&rq, &base_uri, "drivers/unregister", Some(path)) } ServerRequest::CaptureFrame => post("capture-frame"), ServerRequest::InsertIdr => post("insert-idr"), ServerRequest::StartRecording => post("recording/start"), ServerRequest::StopRecording => post("recording/stop"), ServerRequest::RestartSteamvr => post("restart-steamvr"), ServerRequest::ShutdownSteamvr => post("shutdown-steamvr"), } } } thread::sleep(Duration::from_millis(100)); } } }); let events_thread = thread::spawn({ let running = Arc::clone(&running); let session_source = Arc::clone(&session_source); move || { while running.value() { if matches!(*session_source.lock(), SessionSource::Local(_)) { thread::sleep(Duration::from_millis(100)); continue; } let uri = Uri::from_str(&format!("ws://127.0.0.1:{port}/api/events")).unwrap(); let maybe_socket = TcpStream::connect_timeout( &SocketAddr::from_str(&format!("127.0.0.1:{port}")).unwrap(), Duration::from_millis(500), ); let Ok(socket) = maybe_socket else { thread::sleep(Duration::from_millis(500)); continue; }; let mut req = uri.into_client_request().unwrap(); req.headers_mut() .insert("X-ALVR", HeaderValue::from_str("true").unwrap()); let Ok((mut ws, _)) = tungstenite::client(req, socket) else { thread::sleep(Duration::from_millis(500)); continue; }; ws.get_mut().set_nonblocking(true).ok(); while running.value() { match ws.read() { Ok(tungstenite::Message::Text(json_string)) => { debug!("Server event: {json_string}"); if let Ok(event) = serde_json::from_str(&json_string) { events_sender .send(PolledEvent { inner: event, from_dashboard: false, }) .ok(); context.request_repaint(); } } Err(e) => { if let tungstenite::Error::Io(e) = e && e.kind() == ErrorKind::WouldBlock { thread::sleep(Duration::from_millis(50)); continue; } break; } _ => (), } } } } }); let ping_thread = thread::spawn({ let running = Arc::clone(&running); let session_source = Arc::clone(&session_source); let server_connected = Arc::clone(&server_connected); move || { const PING_INTERVAL: Duration = Duration::from_secs(1); let mut deadline = Instant::now(); let uri = format!("http://127.0.0.1:{port}/api/version"); let request_agent: ureq::Agent = ureq::Agent::config_builder() .timeout_global(Some(LOCAL_REQUEST_TIMEOUT)) .build() .into(); loop { let maybe_server_version = request_agent .get(&uri) .header("X-ALVR", "true") .call() .ok() .and_then(|r| { Version::from_str(&r.into_body().read_to_string().ok()?).ok() }); let connected = if let Some(version) = maybe_server_version { // We need exact match because we don't do session extrapolation at the // dashboard level. In the future we may relax the contraint and consider // protocol compatibility check for dashboard. let matches = version == *alvr_common::ALVR_VERSION; if !matches { error!( "Server version mismatch: found {version}. Please remove all previous ALVR installations" ); } matches } else { false }; { let mut session_source_lock = session_source.lock(); if connected && matches!(*session_source_lock, SessionSource::Local(_)) { info!("Server connected"); *session_source_lock = SessionSource::Remote; } else if !connected && matches!(*session_source_lock, SessionSource::Remote) { info!("Server disconnected"); *session_source_lock = SessionSource::Local(Box::new(get_local_session_source())); } } server_connected.set(connected); deadline += PING_INTERVAL; while Instant::now() < deadline { if !running.value() { return; } thread::sleep(Duration::from_millis(100)); } } } }); Self { requests_sender, events_receiver, server_connected, running, version_check_thread: Some(version_check_thread), requests_thread: Some(requests_thread), events_thread: Some(events_thread), ping_thread: Some(ping_thread), } } pub fn request(&self, request: ServerRequest) { self.requests_sender.send(request).ok(); } pub fn poll_event(&self) -> Option { self.events_receiver.try_recv().ok() } pub fn server_connected(&self) -> bool { self.server_connected.value() } } impl Drop for DataSources { fn drop(&mut self) { self.running.set(false); self.version_check_thread.take().unwrap().join().ok(); self.requests_thread.take().unwrap().join().ok(); self.events_thread.take().unwrap().join().ok(); self.ping_thread.take().unwrap().join().ok(); } } ================================================ FILE: alvr/dashboard/src/data_sources_wasm.rs ================================================ use alvr_events::Event; use alvr_packets::ServerRequest; use eframe::{egui, web_sys}; use ewebsock::{WsEvent, WsMessage, WsReceiver}; use gloo_net::http::Request; pub struct DataSources { context: egui::Context, ws_receiver: Option, } impl DataSources { pub fn new(context: egui::Context) -> Self { Self { context, ws_receiver: None, } } pub fn request(&self, request: ServerRequest) { let context = self.context.clone(); wasm_bindgen_futures::spawn_local(async move { Request::post("/api/dashboard-request") .header("X-ALVR", "true") .body(serde_json::to_string(&request).unwrap()) .send() .await .ok(); context.request_repaint(); }) } pub fn poll_event(&mut self) -> Option { if self.ws_receiver.is_none() { let host = web_sys::window().unwrap().location().host().unwrap(); // TODO: Set X-ALVR //let mut options = ewebsock::Options::default(); //options.additional_headers = vec!(("X-ALVR", "true")); let Ok((_, receiver)) = ewebsock::connect(format!("ws://{host}/api/events")) else { return None; }; self.ws_receiver = Some(receiver); } if let Some(event) = self.ws_receiver.as_ref().unwrap().try_recv() { match event { WsEvent::Message(WsMessage::Text(json_string)) => { serde_json::from_str(&json_string).ok() } WsEvent::Error(_) | WsEvent::Closed => { // recreate the ws connection next poll_event invocation self.ws_receiver = None; None } _ => None, } } else { None } } pub fn server_connected(&self) -> bool { true } } ================================================ FILE: alvr/dashboard/src/linux_checks.rs ================================================ pub fn audio_check() { // No check for result, just show errors in logs let _ = alvr_audio::linux::try_load_pipewire(); } ================================================ FILE: alvr/dashboard/src/logging_backend.rs ================================================ use crate::data_sources::PolledEvent; use alvr_common::{LogEntry, LogSeverity, log::LevelFilter, parking_lot::Mutex}; use alvr_events::{Event, EventType}; use std::{ io::Write, sync::{Arc, mpsc}, }; pub fn init_logging(event_sender: mpsc::Sender) { let event_sender = Arc::new(Mutex::new(event_sender)); env_logger::Builder::new() .filter(Some("alvr_events"), LevelFilter::Off) .filter(Some("naga"), LevelFilter::Off) .filter(Some("ureq"), LevelFilter::Off) .filter(Some("wgpu_core"), LevelFilter::Off) .filter(Some("wgpu_hal"), LevelFilter::Off) .filter_level(if cfg!(debug_assertions) { LevelFilter::Debug } else { LevelFilter::Info }) .format(move |f, record| { let timestamp = chrono::Local::now().format("%H:%M:%S.%3f").to_string(); event_sender .lock() .send(PolledEvent { inner: Event { timestamp: timestamp.clone(), event_type: EventType::Log(LogEntry { severity: LogSeverity::from_log_level(record.level()), content: format!("{}", record.args()), }), }, from_dashboard: true, }) .ok(); writeln!( f, "[{} {} {}] {}", timestamp, record.level(), record.module_path().unwrap_or_default(), record.args() ) }) .init(); } ================================================ FILE: alvr/dashboard/src/main.rs ================================================ // hide console window on Windows in release #![cfg_attr(not(debug_assertions), windows_subsystem = "windows")] mod dashboard; #[cfg(not(target_arch = "wasm32"))] mod data_sources; #[cfg(target_arch = "wasm32")] mod data_sources_wasm; #[cfg(not(target_arch = "wasm32"))] #[cfg(target_os = "linux")] mod linux_checks; #[cfg(not(target_arch = "wasm32"))] mod logging_backend; #[cfg(not(target_arch = "wasm32"))] mod steamvr_launcher; #[cfg(not(target_arch = "wasm32"))] use data_sources::DataSources; #[cfg(target_arch = "wasm32")] use data_sources_wasm::DataSources; use alvr_filesystem as afs; use dashboard::Dashboard; fn get_filesystem_layout() -> afs::Layout { afs::filesystem_layout_from_dashboard_exe(&std::env::current_exe().unwrap()).unwrap() } #[cfg(not(target_arch = "wasm32"))] fn main() { use alvr_common::ALVR_VERSION; use alvr_common::info; use alvr_filesystem as afs; use eframe::{ NativeOptions, egui::{IconData, ViewportBuilder}, }; use ico::IconDir; use std::{env, ffi::OsStr, fs}; use std::{io::Cursor, sync::mpsc}; let (server_events_sender, server_events_receiver) = mpsc::channel(); logging_backend::init_logging(server_events_sender.clone()); // Kill any other dashboard instance let self_path = std::env::current_exe().unwrap().canonicalize().unwrap(); for proc in sysinfo::System::new_all().processes_by_name(OsStr::new(&afs::dashboard_fname())) { // According to implementation notes, on linux the returned path can be empty due to // privileges, so canonicalize can fail if let Some(other_path) = proc.exe().and_then(|path| path.canonicalize().ok()) && other_path != self_path { info!( "Killing other dashboard process with path {}", other_path.display() ); proc.kill(); } } #[cfg(target_os = "linux")] linux_checks::audio_check(); data_sources::clean_session(); if data_sources::get_read_only_local_session() .settings() .extra .steamvr_launcher .open_close_steamvr_with_dashboard { steamvr_launcher::LAUNCHER.lock().launch_steamvr() } let ico = IconDir::read(Cursor::new(include_bytes!("../resources/dashboard.ico"))).unwrap(); let image = ico.entries().first().unwrap().decode().unwrap(); // Workaround for the steam deck if fs::read_to_string("/sys/devices/virtual/dmi/id/board_vendor") .map(|vendor| vendor.trim() == "Valve") .unwrap_or(false) { unsafe { env::set_var("WINIT_X11_SCALE_FACTOR", "1") }; } eframe::run_native( &format!("ALVR Dashboard (streamer v{})", *ALVR_VERSION), NativeOptions { viewport: ViewportBuilder::default() .with_app_id("alvr.dashboard") .with_inner_size((900.0, 600.0)) .with_icon(IconData { rgba: image.rgba_data().to_owned(), width: image.width(), height: image.height(), }), centered: true, ..Default::default() }, { Box::new(move |creation_context| { let data_source = DataSources::new( creation_context.egui_ctx.clone(), server_events_sender, server_events_receiver, ); Ok(Box::new(Dashboard::new(creation_context, data_source))) }) }, ) .unwrap(); } #[cfg(target_arch = "wasm32")] fn main() { console_error_panic_hook::set_once(); wasm_logger::init(wasm_logger::Config::default()); wasm_bindgen_futures::spawn_local(async { eframe::WebRunner::new() .start("dashboard_canvas", eframe::WebOptions::default(), { Box::new(move |creation_context| { let context = creation_context.egui_ctx.clone(); Box::new(Dashboard::new(creation_context, DataSources::new(context))) }) }) .await .ok(); }); } ================================================ FILE: alvr/dashboard/src/steamvr_launcher/linux_steamvr.rs ================================================ use std::fs; use std::path::Path; use std::process::Command; use alvr_common::anyhow::bail; use alvr_common::{debug, error, info, warn}; use sysinfo::Process; pub fn launch_steamvr_with_steam() { Command::new("steam") .args(["steam://rungameid/250820"]) .spawn() .ok(); } pub fn terminate_process(process: &Process) { process.kill_with(sysinfo::Signal::Term); } pub fn maybe_wrap_vrcompositor_launcher() -> alvr_common::anyhow::Result<()> { let steamvr_bin_dir = alvr_server_io::steamvr_root_dir()? .join("bin") .join("linux64"); let steamvr_vrserver_path = steamvr_bin_dir.join("vrserver"); debug!( "File path used to check for linux files: {}", steamvr_vrserver_path.display() ); match steamvr_vrserver_path.try_exists() { Ok(exists) => { if !exists { bail!( "SteamVR Linux files missing, aborting startup, please re-check compatibility tools for SteamVR, verify integrity of files for SteamVR and make sure you're not using Flatpak Steam with non-Flatpak ALVR." ); } } Err(e) => { return Err(e.into()); } }; let launcher_path = steamvr_bin_dir.join("vrcompositor"); // In case of SteamVR update, vrcompositor will be restored if fs::read_link(&launcher_path).is_ok() { fs::remove_file(&launcher_path)?; // recreate the link } else { fs::rename(&launcher_path, steamvr_bin_dir.join("vrcompositor.real"))?; } std::os::unix::fs::symlink( crate::get_filesystem_layout().vrcompositor_wrapper(), &launcher_path, )?; Ok(()) } #[derive(PartialEq)] enum DeviceInfo { Nvidia, Amd { device_type: wgpu::DeviceType }, Intel { device_type: wgpu::DeviceType }, Unknown, } pub fn linux_hardware_checks() { let wgpu_adapters = wgpu::Instance::new(&wgpu::InstanceDescriptor { backends: wgpu::Backends::VULKAN, ..Default::default() }) .enumerate_adapters(wgpu::Backends::VULKAN); let device_infos = wgpu_adapters .iter() .filter(|adapter| { adapter.get_info().device_type == wgpu::DeviceType::DiscreteGpu || adapter.get_info().device_type == wgpu::DeviceType::IntegratedGpu }) .map(|adapter| { let vendor = match adapter.get_info().vendor { 0x10de => DeviceInfo::Nvidia, 0x1002 => DeviceInfo::Amd { device_type: adapter.get_info().device_type, }, 0x8086 => DeviceInfo::Intel { device_type: adapter.get_info().device_type, }, _ => DeviceInfo::Unknown, }; (adapter, vendor) }) .collect::>(); linux_gpu_checks(&device_infos); linux_encoder_checks(&device_infos); } fn linux_gpu_checks(device_infos: &[(&wgpu::Adapter, DeviceInfo)]) { let have_intel_igpu = device_infos.iter().any(|gpu| { gpu.1 == DeviceInfo::Intel { device_type: wgpu::DeviceType::IntegratedGpu, } }); debug!("have_intel_igpu: {}", have_intel_igpu); let have_amd_igpu = device_infos.iter().any(|gpu| { gpu.1 == DeviceInfo::Amd { device_type: wgpu::DeviceType::IntegratedGpu, } }); debug!("have_amd_igpu: {}", have_amd_igpu); let have_igpu = have_intel_igpu || have_amd_igpu; debug!("have_igpu: {}", have_igpu); let have_nvidia_dgpu = device_infos.iter().any(|gpu| gpu.1 == DeviceInfo::Nvidia); debug!("have_nvidia_dgpu: {}", have_nvidia_dgpu); let have_amd_dgpu = device_infos.iter().any(|gpu| { gpu.1 == DeviceInfo::Amd { device_type: wgpu::DeviceType::DiscreteGpu, } }); debug!("have_amd_dgpu: {}", have_amd_dgpu); if have_amd_igpu || have_amd_dgpu { let is_any_amd_driver_invalid = device_infos.iter().any(|gpu| { info!("Driver name: {}", gpu.0.get_info().driver); match gpu.0.get_info().driver.as_str() { "AMD proprietary driver" | "AMD open-source driver" => true, // AMDGPU-Pro | AMDVLK _ => false, } }); if is_any_amd_driver_invalid { error!( "Amdvlk or amdgpu-pro vulkan drivers detected, SteamVR may not function properly. \ Please remove them or make them unavailable for SteamVR and games you're trying to launch.\n\ For more detailed info visit the wiki: \ https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting#artifacting-no-steamvr-overlay-or-graphical-glitches-in-streaming-view" ) } } let have_intel_dgpu = device_infos.iter().any(|gpu| { gpu.1 == DeviceInfo::Intel { device_type: wgpu::DeviceType::DiscreteGpu, } }); debug!("have_intel_dgpu: {}", have_intel_dgpu); let steamvr_root_dir = match alvr_server_io::steamvr_root_dir() { Ok(dir) => dir, Err(e) => { error!( "Couldn't find OpenVR or SteamVR files. \ Please make sure you have installed and ran SteamVR at least once. \ Or if you're using Flatpak Steam, make sure to use ALVR Dashboard from Flatpak ALVR. {e}" ); return; } }; let vrmonitor_path_string = steamvr_root_dir .join("bin") .join("vrmonitor.sh") .into_os_string() .into_string() .unwrap(); debug!("vrmonitor_path: {}", vrmonitor_path_string); let steamvr_opts = "For functioning VR you need to put the following line into SteamVR's launch options and restart it:"; let game_opts = "And this similar line to the launch options of ALL games that you're trying to launch from steam:"; let mut vrmonitor_path_written = false; if have_igpu { if have_nvidia_dgpu { let base_path = "/usr/share/vulkan/icd.d/nvidia_icd"; let nvidia_vk_override_path = if Path::new(&format!("{base_path}.json")).exists() { format!("VK_DRIVER_FILES={base_path}.json") } else if Path::new(&format!("{base_path}.x86_64.json")).exists() { format!("VK_DRIVER_FILES={base_path}.x86_64.json") } else { "__VK_LAYER_NV_optimus=NVIDIA_only".to_string() }; let nv_options = format!( "__GLX_VENDOR_LIBRARY_NAME=nvidia __NV_PRIME_RENDER_OFFLOAD=1 {nvidia_vk_override_path}" ); warn!("{steamvr_opts}\n{nv_options} {vrmonitor_path_string} %command%"); warn!("{game_opts}\n{nv_options} %command%"); vrmonitor_path_written = true; } else if have_intel_dgpu || have_amd_dgpu { warn!("{steamvr_opts}\nDRI_PRIME=1 {vrmonitor_path_string} %command%"); warn!("{game_opts}\nDRI_PRIME=1 %command%"); vrmonitor_path_written = true; } else { warn!( "Beware, using just integrated graphics might lead to very poor performance in SteamVR and VR games." ); warn!( "For more information, please refer to the wiki: https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting" ) } } if !vrmonitor_path_written { warn!( "Make sure you have put the following line in your SteamVR launch options and restart it:\n\ {vrmonitor_path_string} %command%" ) } } fn linux_encoder_checks(device_infos: &[(&wgpu::Adapter, DeviceInfo)]) { for device_info in device_infos { match device_info.1 { DeviceInfo::Nvidia => { match nvml_wrapper::Nvml::init() { Ok(nvml) => { let device_count = nvml.device_count().unwrap(); debug!("nvml device count: {}", device_count); // fixme: on multi-gpu nvidia system will do it twice, for index in 0..device_count { match nvml.device_by_index(index) { Ok(device) => { debug!("nvml device name: {}", device.name().unwrap()); probe_nvenc_encoder_profile( &device, nvml_wrapper::enum_wrappers::device::EncoderType::H264, "H264", ); probe_nvenc_encoder_profile( &device, nvml_wrapper::enum_wrappers::device::EncoderType::HEVC, "HEVC", ); // todo: probe for AV1 when will be available in nvml-wrapper } Err(e) => { error!("Failed to acquire NVML device with error: {}", e) } } } } Err(e) => { alvr_common::show_e(format!("Can't initialize NVML engine, error: {e}.")) } } } DeviceInfo::Amd { device_type: _ } | DeviceInfo::Intel { device_type: _ } => { let libva_display_open = libva::Display::open(); if let Some(libva_display) = libva_display_open { if let Ok(vendor_string) = libva_display.query_vendor_string() { info!("GPU Encoder vendor: {}", vendor_string); } probe_libva_encoder_profile( &libva_display, libva::VAProfile::VAProfileH264Main, "H264", true, ); probe_libva_encoder_profile( &libva_display, libva::VAProfile::VAProfileHEVCMain, "HEVC", true, ); probe_libva_encoder_profile( &libva_display, libva::VAProfile::VAProfileAV1Profile0, "AV1", false, ); } else { alvr_common::show_e( "Couldn't find VA-API runtime on system, \ you unlikely to have hardware encoding. \ Please install VA-API runtime for your distribution \ and make sure it works (Manjaro, Fedora affected). \ For detailed advice, check wiki: \ https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting#failed-to-create-vaapi-encoder", ); } } _ => alvr_common::show_e( "Couldn't determine gpu for hardware encoding. \ You will likely fallback to software encoding.", ), } } } fn probe_nvenc_encoder_profile( device: &nvml_wrapper::Device, encoder_type: nvml_wrapper::enum_wrappers::device::EncoderType, profile_name: &str, ) { match device.encoder_capacity(encoder_type) { Ok(_) => { info!("GPU supports {} profile.", profile_name); } Err(e) => { if matches!(e, nvml_wrapper::error::NvmlError::NotSupported) { alvr_common::show_e(format!( "Your NVIDIA gpu doesn't support {profile_name}. Please make sure CUDA is installed properly. Error: {e}" )) } else { error!("{}", e) } } } } fn probe_libva_encoder_profile( libva_display: &std::rc::Rc, profile_type: libva::VAProfile::Type, profile_name: &str, is_critical: bool, ) { let profile_probe = libva_display.query_config_entrypoints(profile_type); let mut message = String::new(); if profile_probe.is_err() { message = format!("Couldn't find {profile_name} encoder."); } else if let Ok(profile) = profile_probe { if profile.is_empty() { message = format!("{profile_name} profile entrypoint is empty."); } else if !profile.contains(&libva::VAEntrypoint::VAEntrypointEncSlice) { message = format!("{profile_name} profile does not contain encoding entrypoint."); } } if !message.is_empty() { if is_critical { error!("{} Your gpu may not suport encoding with this.", message); } else { info!( "{} Your gpu may not suport encoding with this. \ If you're not using this encoder, ignore this message.", message ); } } } ================================================ FILE: alvr/dashboard/src/steamvr_launcher/mod.rs ================================================ #[cfg(target_os = "linux")] mod linux_steamvr; #[cfg(windows)] mod windows_steamvr; use crate::data_sources; use alvr_adb::commands as adb; use alvr_common::{ anyhow::{Context, Result}, debug, error, glam::bool, parking_lot::Mutex, warn, }; use alvr_filesystem::{self as afs}; use serde_json::{self, json}; use std::{ ffi::OsStr, fs, marker::PhantomData, process::Command, thread, time::{Duration, Instant}, }; use sysinfo::{ProcessesToUpdate, System}; const SHUTDOWN_TIMEOUT: Duration = Duration::from_secs(10); const DRIVER_KEY: &str = "driver_alvr_server"; const BLOCKED_KEY: &str = "blocked_by_safe_mode"; pub fn is_steamvr_running() -> bool { System::new_all() .processes_by_name(OsStr::new(&afs::exec_fname("vrserver"))) .count() != 0 } pub fn maybe_kill_steamvr() { let mut system = System::new_all(); #[allow(unused_variables)] for process in system.processes_by_name(OsStr::new(&afs::exec_fname("vrmonitor"))) { debug!("Killing vrmonitor"); #[cfg(target_os = "linux")] linux_steamvr::terminate_process(process); #[cfg(windows)] windows_steamvr::kill_process(process.pid().as_u32()); thread::sleep(Duration::from_secs(1)); } system.refresh_processes(ProcessesToUpdate::All, true); #[allow(unused_variables)] for process in system.processes_by_name(OsStr::new(&afs::exec_fname("vrserver"))) { debug!("Killing vrserver"); #[cfg(target_os = "linux")] linux_steamvr::terminate_process(process); #[cfg(windows)] windows_steamvr::kill_process(process.pid().as_u32()); thread::sleep(Duration::from_secs(1)); } } fn unblock_alvr_driver() -> Result<()> { if !cfg!(target_os = "linux") { return Ok(()); } let path = alvr_server_io::steamvr_settings_file_path()?; let text = fs::read_to_string(&path).with_context(|| format!("Failed to read {path:?}"))?; let new_text = unblock_alvr_driver_within_vrsettings(text.as_str()) .with_context(|| "Failed to rewrite .vrsettings.")?; fs::write(&path, new_text) .with_context(|| "Failed to write .vrsettings back after changing it.")?; Ok(()) } // Reads and writes back steamvr.vrsettings in order to // ensure the ALVR driver is not blocked (safe mode). fn unblock_alvr_driver_within_vrsettings(text: &str) -> Result { let mut settings = serde_json::from_str::(text)?; let values = settings .as_object_mut() .with_context(|| "Failed to parse .vrsettings.")?; let blocked = values .get(DRIVER_KEY) .and_then(|driver| driver.get(BLOCKED_KEY)) .and_then(|blocked| blocked.as_bool()) .unwrap_or(false); if blocked { debug!("Unblocking ALVR driver in SteamVR."); if !values.contains_key(DRIVER_KEY) { values.insert(DRIVER_KEY.into(), json!({})); } let driver = settings[DRIVER_KEY] .as_object_mut() .with_context(|| "Did not find ALVR key in settings.")?; driver.insert(BLOCKED_KEY.into(), json!(false)); // overwrites if present } else { debug!("ALVR is not blocked in SteamVR."); } Ok(serde_json::to_string_pretty(&settings)?) } pub struct Launcher { _phantom: PhantomData<()>, } impl Launcher { pub fn launch_steamvr(&self) { // The ADB server might be left running because of an unclean termination of SteamVR. // Kill it unconditionally to ensure clean state, regardless of current connection mode. // Note: this will also kill a system-wide ADB server not started by ALVR. if let Some(path) = adb::get_adb_path(&crate::get_filesystem_layout()) { adb::kill_server(&path).ok(); } #[cfg(target_os = "linux")] linux_steamvr::linux_hardware_checks(); let alvr_driver_dir = crate::get_filesystem_layout().openvr_driver_root_dir; // Make sure to unregister any other ALVR driver because it would cause a socket conflict let other_alvr_dirs = alvr_server_io::get_registered_drivers() .unwrap_or_default() .into_iter() .filter(|path| { path.to_string_lossy().to_lowercase().contains("alvr") && *path != alvr_driver_dir }) .collect::>(); alvr_server_io::driver_registration(&other_alvr_dirs, false).ok(); alvr_server_io::driver_registration(&[alvr_driver_dir], true).ok(); if let Err(err) = unblock_alvr_driver() { warn!("Failed to unblock ALVR driver: {:?}", err); } #[cfg(target_os = "linux")] { let vrcompositor_wrap_result = linux_steamvr::maybe_wrap_vrcompositor_launcher(); alvr_common::show_err(linux_steamvr::maybe_wrap_vrcompositor_launcher()); if vrcompositor_wrap_result.is_err() { return; } } if is_steamvr_running() { return; } debug!("SteamVR is dead. Launching..."); if data_sources::get_read_only_local_session() .settings() .extra .steamvr_launcher .direct_launch { let start_script = afs::filesystem_layout_invalid().server_start_script(); if start_script.exists() { debug!("Running VR server start script: {}", start_script.display()); if let Err(e) = Command::new(&start_script).spawn() { error!("Failed to run VR server start script: {e}"); } } else if let Ok(steamvr_bin_dir) = alvr_server_io::steamvr_root_dir().map(|root| root.join("bin")) { let steamvr_path = if cfg!(windows) { steamvr_bin_dir.join("win64").join("vrstartup.exe") } else { steamvr_bin_dir.join("vrmonitor.sh") }; debug!("Launching SteamVR from path: {}", steamvr_path.display()); if let Err(e) = Command::new(&steamvr_path).spawn() { error!( "Failed to run SteamVR from automatically detected path {} with error: {e}", steamvr_path.display() ); } } else { error!("Failed to find SteamVR files to directly launch SteamVR"); } } else { #[cfg(windows)] windows_steamvr::launch_steamvr_with_steam(); #[cfg(target_os = "linux")] linux_steamvr::launch_steamvr_with_steam(); } } pub fn ensure_steamvr_shutdown(&self) { debug!("Waiting for SteamVR to shutdown..."); let start_time = Instant::now(); while start_time.elapsed() < SHUTDOWN_TIMEOUT && is_steamvr_running() { thread::sleep(Duration::from_millis(500)); } maybe_kill_steamvr(); } pub fn restart_steamvr(&self) { self.ensure_steamvr_shutdown(); self.launch_steamvr(); } } // Singleton with exclusive access pub static LAUNCHER: Mutex = Mutex::new(Launcher { _phantom: PhantomData, }); ================================================ FILE: alvr/dashboard/src/steamvr_launcher/windows_steamvr.rs ================================================ use std::os::windows::process::CommandExt; use std::process::Command; const CREATE_NO_WINDOW: u32 = 0x0800_0000; pub fn launch_steamvr_with_steam() { Command::new("cmd") .args(["/C", "start", "steam://rungameid/250820"]) .creation_flags(CREATE_NO_WINDOW) .spawn() .ok(); } pub fn kill_process(pid: u32) { Command::new("taskkill.exe") .args(["/PID", &pid.to_string(), "/F"]) .creation_flags(CREATE_NO_WINDOW) .output() .ok(); } ================================================ FILE: alvr/events/Cargo.toml ================================================ [package] name = "alvr_events" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_packets.workspace = true alvr_session.workspace = true serde = { version = "1", features = ["derive"] } serde_json = "1" ================================================ FILE: alvr/events/src/lib.rs ================================================ use alvr_common::{DeviceMotion, LogEntry, LogSeverity, Pose, info}; use alvr_packets::{ButtonValue, FaceData}; use alvr_session::SessionConfig; use serde::{Deserialize, Serialize}; use std::{path::PathBuf, time::Duration}; #[derive(Serialize, Deserialize, Clone, Debug, Default)] pub struct StatisticsSummary { pub video_packets_total: usize, pub video_packets_per_sec: usize, pub video_mbytes_total: usize, pub video_mbits_per_sec: f32, pub total_latency_ms: f32, pub network_latency_ms: f32, pub encode_latency_ms: f32, pub decode_latency_ms: f32, pub client_fps: u32, pub server_fps: u32, pub battery_hmd: u32, pub hmd_plugged: bool, } // Bitrate statistics minus the empirical output value #[derive(Serialize, Deserialize, Clone, Debug, Default)] pub struct BitrateDirectives { pub scaled_calculated_throughput_bps: Option, pub decoder_latency_limiter_bps: Option, pub network_latency_limiter_bps: Option, pub encoder_latency_limiter_bps: Option, pub manual_max_throughput_bps: Option, pub manual_min_throughput_bps: Option, pub requested_bitrate_bps: f32, } #[derive(Serialize, Deserialize, Clone, Debug, Default)] pub struct GraphStatistics { pub total_pipeline_latency_s: f32, pub game_time_s: f32, pub server_compositor_s: f32, pub encoder_s: f32, pub network_s: f32, pub decoder_s: f32, pub decoder_queue_s: f32, pub client_compositor_s: f32, pub vsync_queue_s: f32, pub client_fps: f32, pub server_fps: f32, pub bitrate_directives: BitrateDirectives, pub throughput_bps: f32, pub bitrate_bps: f32, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct TrackingEvent { pub device_motions: Vec<(String, DeviceMotion)>, pub hand_skeletons: [Option<[Pose; 26]>; 2], pub face: FaceData, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct ButtonEvent { pub path: String, pub value: ButtonValue, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct HapticsEvent { pub path: String, pub duration: Duration, pub frequency: f32, pub amplitude: f32, } #[derive(Serialize, Deserialize, Clone, Debug, Default)] pub struct AdbEvent { pub download_progress: f32, } #[derive(Serialize, Deserialize, Clone, Debug)] #[serde(tag = "id", content = "data")] pub enum EventType { Log(LogEntry), DebugGroup { group: String, message: String }, Session(Box), StatisticsSummary(StatisticsSummary), GraphStatistics(GraphStatistics), Tracking(Box), Buttons(Vec), Haptics(HapticsEvent), DriversList(Vec), ServerRequestsSelfRestart, Adb(AdbEvent), NewVersionFound { version: String, message: String }, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct Event { pub timestamp: String, pub event_type: EventType, } impl Event { pub fn event_type_string(&self) -> String { match &self.event_type { EventType::Log(entry) => match entry.severity { LogSeverity::Error => "ERROR".into(), LogSeverity::Warning => "WARNING".into(), LogSeverity::Info => "INFO".into(), LogSeverity::Debug => "DEBUG".into(), }, EventType::DebugGroup { group, .. } => group.clone(), EventType::Session(_) => "SESSION".to_string(), EventType::StatisticsSummary(_) => "STATS".to_string(), EventType::GraphStatistics(_) => "GRAPH".to_string(), EventType::Tracking(_) => "TRACKING".to_string(), EventType::Buttons(_) => "BUTTONS".to_string(), EventType::Haptics(_) => "HAPTICS".to_string(), EventType::DriversList(_) => "DRV LIST".to_string(), EventType::ServerRequestsSelfRestart => "RESTART".to_string(), EventType::Adb(_) => "ADB".to_string(), EventType::NewVersionFound { .. } => "NEW VER".to_string(), } } pub fn message(&self) -> String { match &self.event_type { EventType::Log(log_entry) => log_entry.content.clone(), EventType::DebugGroup { message, .. } => message.clone(), EventType::Session(_) => "Updated".into(), EventType::StatisticsSummary(_) | EventType::GraphStatistics(_) => "".into(), EventType::Tracking(tracking) => serde_json::to_string(tracking).unwrap(), EventType::Buttons(buttons) => serde_json::to_string(buttons).unwrap(), EventType::Haptics(haptics) => serde_json::to_string(haptics).unwrap(), EventType::DriversList(drivers) => serde_json::to_string(drivers).unwrap(), EventType::ServerRequestsSelfRestart => "Request for server restart".into(), EventType::Adb(adb) => serde_json::to_string(adb).unwrap(), EventType::NewVersionFound { version, .. } => version.clone(), } } } pub fn send_event(event_type: EventType) { info!("{}", serde_json::to_string(&event_type).unwrap()); } ================================================ FILE: alvr/filesystem/Cargo.toml ================================================ [package] name = "alvr_filesystem" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] dirs = "6" ================================================ FILE: alvr/filesystem/build.rs ================================================ // This is needed so that `OUT_DIR` is set and `afs::target_dir` works fn main() {} ================================================ FILE: alvr/filesystem/src/lib.rs ================================================ use std::{ env::{ self, consts::{DLL_EXTENSION, DLL_PREFIX, DLL_SUFFIX, EXE_SUFFIX, OS}, }, path::{Path, PathBuf}, }; pub fn exec_fname(name: &str) -> String { format!("{name}{EXE_SUFFIX}") } pub fn dynlib_fname(name: &str) -> String { format!("{DLL_PREFIX}{name}{DLL_SUFFIX}") } pub fn target_dir() -> PathBuf { // use `.parent().unwrap()` instead of `../` to maintain canonicalized form Path::new(env!("OUT_DIR")) .parent() .unwrap() .parent() .unwrap() .parent() .unwrap() .parent() .unwrap() .to_owned() } pub fn workspace_dir() -> PathBuf { Path::new(env!("CARGO_MANIFEST_DIR")) .parent() .unwrap() .parent() .unwrap() .to_owned() } pub fn crate_dir(name: &str) -> PathBuf { Path::new(env!("CARGO_MANIFEST_DIR")) .parent() .unwrap() .join(name) } pub fn deps_dir() -> PathBuf { workspace_dir().join("deps") } pub fn build_dir() -> PathBuf { workspace_dir().join("build") } pub fn streamer_build_dir() -> PathBuf { build_dir().join(format!("alvr_streamer_{OS}")) } pub fn launcher_fname() -> String { exec_fname("ALVR Launcher") } pub fn launcher_build_dir() -> PathBuf { build_dir().join(format!("alvr_launcher_{OS}")) } pub fn launcher_build_exe_path() -> PathBuf { launcher_build_dir().join(launcher_fname()) } pub fn installer_path() -> PathBuf { env::temp_dir().join(exec_fname("alvr_installer")) } pub fn dashboard_fname() -> &'static str { if cfg!(windows) { "ALVR Dashboard.exe" } else { "alvr_dashboard" } } // Layout of the ALVR installation. All paths are absolute #[derive(Clone, Default, Debug)] pub struct Layout { // directory containing the dashboard executable pub executables_dir: PathBuf, // (linux only) directory where libalvr_vulkan_layer.so is saved pub libraries_dir: PathBuf, // parent directory of resources like the dashboard and presets folders pub static_resources_dir: PathBuf, // directory for storing configuration files (session.json) pub config_dir: PathBuf, // directory for storing log pub log_dir: PathBuf, // directory to register in openVR driver path pub openvr_driver_root_dir: PathBuf, // (linux only) parent directory of the executable to wrap vrcompositor pub vrcompositor_wrapper_dir: PathBuf, // (linux only) parent directory of the firewall script pub firewall_script_dir: PathBuf, // (linux only) parent directory of the firewalld config pub firewalld_config_dir: PathBuf, // (linux only) parent directory of the ufw config pub ufw_config_dir: PathBuf, // (linux only) directory where the vulkan layer manifest is saved pub vulkan_layer_manifest_dir: PathBuf, pub launcher_root: Option, } impl Layout { pub fn new(root: &Path) -> Self { #[cfg(target_os = "linux")] { let or_path = |opt: Option<&'static str>, path| opt.map_or(root.join(path), PathBuf::from); // Get paths from environment or use FHS compliant paths let executables_dir = or_path(option_env!("ALVR_EXECUTABLES_DIR"), "bin"); let libraries_dir = or_path(option_env!("ALVR_LIBRARIES_DIR"), "lib64"); let static_resources_dir = or_path(option_env!("ALVR_STATIC_RESOURCES_DIR"), "share/alvr"); let openvr_driver_root_dir = or_path(option_env!("ALVR_OPENVR_DRIVER_ROOT_DIR"), "lib64/alvr"); let vrcompositor_wrapper_dir = or_path(option_env!("ALVR_VRCOMPOSITOR_WRAPPER_DIR"), "libexec/alvr"); let firewall_script_dir = or_path(option_env!("FIREWALL_SCRIPT_DIR"), "libexec/alvr"); let firewalld_config_dir = or_path(option_env!("FIREWALLD_CONFIG_DIR"), "libexec/alvr"); let ufw_config_dir = or_path(option_env!("UFW_CONFIG_DIR"), "libexec/alvr"); let vulkan_layer_manifest_dir = or_path( option_env!("ALVR_VULKAN_LAYER_MANIFEST_DIR"), "share/vulkan/explicit_layer.d", ); let config_dir = option_env!("ALVR_CONFIG_DIR") .map_or_else(|| dirs::config_dir().unwrap().join("alvr"), PathBuf::from); let log_dir = option_env!("ALVR_LOG_DIR") .map_or_else(|| dirs::home_dir().unwrap(), PathBuf::from); Self { executables_dir, libraries_dir, static_resources_dir, config_dir, log_dir, openvr_driver_root_dir, vrcompositor_wrapper_dir, firewall_script_dir, firewalld_config_dir, ufw_config_dir, vulkan_layer_manifest_dir, launcher_root: root .parent() .and_then(|p| p.parent()) .and_then(|p| p.parent()) .map(|p| p.to_owned()), } } #[cfg(not(target_os = "linux"))] Self { executables_dir: root.to_owned(), libraries_dir: root.to_owned(), static_resources_dir: root.to_owned(), config_dir: root.to_owned(), log_dir: root.to_owned(), openvr_driver_root_dir: root.to_owned(), vrcompositor_wrapper_dir: root.to_owned(), firewall_script_dir: root.to_owned(), firewalld_config_dir: root.to_owned(), ufw_config_dir: root.to_owned(), vulkan_layer_manifest_dir: root.to_owned(), launcher_root: root.parent().and_then(|p| p.parent()).map(|p| p.to_owned()), } } pub fn dashboard_exe(&self) -> PathBuf { self.executables_dir.join(dashboard_fname()) } pub fn local_adb_exe(&self) -> PathBuf { self.executables_dir .join("platform-tools") .join(exec_fname("adb")) } pub fn resources_dir(&self) -> PathBuf { self.openvr_driver_root_dir.join("resources") } pub fn dashboard_dir(&self) -> PathBuf { self.static_resources_dir.join("dashboard") } pub fn presets_dir(&self) -> PathBuf { self.static_resources_dir.join("presets") } pub fn session(&self) -> PathBuf { self.config_dir.join("session.json") } pub fn session_log(&self) -> PathBuf { if cfg!(target_os = "linux") { self.log_dir.join("alvr_session_log.txt") } else { self.log_dir.join("session_log.txt") } } pub fn server_start_script(&self) -> PathBuf { self.config_dir.join(if cfg!(windows) { "start_server.bat" } else { "start_server.sh" }) } pub fn connect_script(&self) -> PathBuf { self.config_dir.join(if cfg!(windows) { "on_connect.bat" } else { "on_connect.sh" }) } pub fn disconnect_script(&self) -> PathBuf { self.config_dir.join(if cfg!(windows) { "on_disconnect.bat" } else { "on_disconnect.sh" }) } pub fn crash_log(&self) -> PathBuf { self.log_dir.join("crash_log.txt") } pub fn openvr_driver_lib_dir(&self) -> PathBuf { let platform = if cfg!(windows) { "win64" } else if cfg!(target_os = "linux") { "linux64" } else if cfg!(target_os = "macos") { "macos" } else { unimplemented!() }; self.openvr_driver_root_dir.join("bin").join(platform) } // path to the shared library to be loaded by openVR pub fn openvr_driver_lib(&self) -> PathBuf { self.openvr_driver_lib_dir() .join(format!("driver_alvr_server.{DLL_EXTENSION}")) } // path to the manifest file for openVR pub fn openvr_driver_manifest(&self) -> PathBuf { self.openvr_driver_root_dir.join("driver.vrdrivermanifest") } pub fn vrcompositor_wrapper(&self) -> PathBuf { self.vrcompositor_wrapper_dir.join("vrcompositor-wrapper") } pub fn drm_lease_shim(&self) -> PathBuf { self.vrcompositor_wrapper_dir.join("alvr_drm_lease_shim.so") } pub fn vulkan_layer(&self) -> PathBuf { self.libraries_dir.join(dynlib_fname("alvr_vulkan_layer")) } pub fn firewall_script(&self) -> PathBuf { self.firewall_script_dir.join("alvr_fw_config.sh") } pub fn firewalld_config(&self) -> PathBuf { self.firewalld_config_dir.join("alvr-firewalld.xml") } pub fn ufw_config(&self) -> PathBuf { self.ufw_config_dir.join("ufw-alvr") } pub fn vulkan_layer_manifest(&self) -> PathBuf { self.vulkan_layer_manifest_dir.join("alvr_x86_64.json") } pub fn launcher_exe(&self) -> Option { self.launcher_root .as_ref() .map(|root| root.join(launcher_fname())) } } fn layout_from_env() -> Option { option_env!("ALVR_ROOT_DIR").map(|path| Layout::new(Path::new(path))) } // The path should include the executable file name // The path argument is used only if ALVR is built as portable pub fn filesystem_layout_from_dashboard_exe(path: &Path) -> Option { layout_from_env().or_else(|| { let root = if cfg!(target_os = "linux") { // FHS path is expected path.parent()?.parent()?.to_owned() } else { path.parent()?.to_owned() }; Some(Layout::new(&root)) }) } // The dir argument is used only if ALVR is built as portable pub fn filesystem_layout_from_openvr_driver_root_dir(dir: &Path) -> Option { layout_from_env().or_else(|| { let root = if cfg!(target_os = "linux") { // FHS path is expected dir.parent()?.parent()?.to_owned() } else { dir.to_owned() }; Some(Layout::new(&root)) }) } // Use this when there is no way of determining the current path. The resulting Layout paths will // be invalid, except for the ones that disregard the relative path (for example the config dir) and // the ones that have been overridden. pub fn filesystem_layout_invalid() -> Layout { layout_from_env().unwrap_or_else(|| Layout::new(Path::new("./"))) } ================================================ FILE: alvr/graphics/Cargo.toml ================================================ [package] name = "alvr_graphics" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_session.workspace = true glow = "0.16" glyph_brush_layout = "0.2" khronos-egl = { version = "6", features = ["dynamic"] } pollster = "0.4" wgpu = "25" ================================================ FILE: alvr/graphics/resources/lobby_line.wgsl ================================================ struct PushConstant { transform: mat4x4f, color: u32, } var pc: PushConstant; @vertex fn vertex_main(@builtin(vertex_index) vertex_index: u32) -> @builtin(position) vec4f { return pc.transform * vec4f(0.0, 0.0, -f32(vertex_index), 1.0); } @fragment fn fragment_main() -> @location(0) vec4f { return unpack4x8unorm(pc.color); } ================================================ FILE: alvr/graphics/resources/lobby_quad.wgsl ================================================ struct PushConstant { transform: mat4x4f, object_type: u32, floor_side: f32, } var pc: PushConstant; @group(0) @binding(0) var hud_texture: texture_2d; @group(0) @binding(1) var hud_sampler: sampler; struct VertexOutput { @builtin(position) position: vec4f, @location(0) uv: vec2f, } @vertex fn vertex_main(@builtin(vertex_index) vertex_index: u32) -> VertexOutput { var result: VertexOutput; result.uv = vec2f(f32(vertex_index & 1), f32(vertex_index >> 1)); result.position = pc.transform * vec4f(result.uv.x - 0.5, 0.5 - result.uv.y, 0.0, 1.0); return result; } @fragment fn fragment_main(@location(0) uv: vec2f) -> @location(0) vec4f { if pc.object_type == 0 { // Ground let world_xz = (uv - 0.5) * pc.floor_side; let ground_center = vec3f(0.0, 0.0, 0.0); let ground_horizon = vec3f(0.0, 0.0, 0.015); let grid_close = vec3f(0.114, 0.545, 0.804); let grid_far = vec3f(0.259, 0.863, 0.886); let line_fade_start = 10.0; let line_fade_end = 50.0; let line_fade_dist = line_fade_end - line_fade_start; let line_bloom = 10.0; let distance = length(world_xz); // Pick a coordinate to visualize in a grid let cell_size = 2.0; let coord = world_xz / cell_size; // Compute anti-aliased world-space grid lines let screen_space_line_width = 1.0 * fwidth(coord); // todo: make resolution agnostic? let grid = abs(fract(coord - 0.5) - 0.5) / screen_space_line_width; // Create mask for grid lines and fade over distance var line = clamp(1.0 - min(grid.x, grid.y), 0.0, 1.0); line *= clamp((line_fade_start - distance) / line_fade_dist, 0.0, 1.0); // Fill in normal ground colour var out_color = ground_center * (1.0 - line); // Add cheap and simple "bloom" to the grid lines line *= 1.0 + line_bloom; // Fill in grid line colour out_color += line * mix(grid_far, grid_close, clamp((line_fade_end - distance) / line_fade_end, 0.0, 1.0)); // Fade to the horizon colour over distance if distance > 10.0 { let coef = 1.0 - 10.0 / distance; out_color = (1.0 - coef) * out_color + coef * ground_horizon; } return vec4f(out_color, 1.0); } else { // HUD return textureSample(hud_texture, hud_sampler, uv); } } ================================================ FILE: alvr/graphics/resources/staging_fragment.glsl ================================================ #version 300 es #extension GL_OES_EGL_image_external_essl3 : enable precision mediump float; uniform samplerExternalOES tex; // Convert from limited colors to full const float LIMITED_MIN = 16.0 / 255.0; const float LIMITED_MAX = 235.0 / 255.0; in vec2 uv; out vec4 out_color; void main() { vec3 color = texture(tex, uv).rgb; #ifdef FIX_LIMITED_RANGE color = LIMITED_MIN + ((LIMITED_MAX - LIMITED_MIN) * color); #endif out_color = vec4(color, 1.0); } ================================================ FILE: alvr/graphics/resources/staging_vertex.glsl ================================================ #version 300 es uniform int view_idx; out vec2 uv; void main() { vec2 screen_uv = vec2(gl_VertexID & 1, gl_VertexID >> 1); gl_Position = vec4((screen_uv - 0.5f) * 2.f, 0, 1); uv = vec2((screen_uv.x + float(view_idx)) / 2.f, screen_uv.y); } ================================================ FILE: alvr/graphics/resources/stream.wgsl ================================================ const DIV12: f32 = 1.0 / 12.92; const DIV1: f32 = 1.0 / 1.055; const THRESHOLD: f32 = 0.04045; const GAMMA: vec3f = vec3f(2.4); override ENABLE_SRGB_CORRECTION: bool; override ENCODING_GAMMA: f32; override ENABLE_UPSCALING: bool = false; override UPSCALE_USE_EDGE_DIRECTION: bool = true; override UPSCALE_EDGE_THRESHOLD: f32 = 4.0/255.0; override UPSCALE_EDGE_SHARPNESS: f32 = 2.0; override ENABLE_FFE: bool = false; override VIEW_WIDTH_RATIO: f32 = 0.0; override VIEW_HEIGHT_RATIO: f32 = 0.0; override EDGE_X_RATIO: f32 = 0.0; override EDGE_Y_RATIO: f32 = 0.0; override C1_X: f32 = 0.0; override C1_Y: f32 = 0.0; override C2_X: f32 = 0.0; override C2_Y: f32 = 0.0; override LO_BOUND_X: f32 = 0.0; override LO_BOUND_Y: f32 = 0.0; override HI_BOUND_X: f32 = 0.0; override HI_BOUND_Y: f32 = 0.0; override A_LEFT_X: f32 = 0.0; override A_LEFT_Y: f32 = 0.0; override B_LEFT_X: f32 = 0.0; override B_LEFT_Y: f32 = 0.0; override A_RIGHT_X: f32 = 0.0; override A_RIGHT_Y: f32 = 0.0; override B_RIGHT_X: f32 = 0.0; override B_RIGHT_Y: f32 = 0.0; override C_RIGHT_X: f32 = 0.0; override C_RIGHT_Y: f32 = 0.0; struct PushConstant { reprojection_transform: mat4x4f, view_idx: u32, passthrough_mode: u32, // 0: Blend, 1: RGB chroma key, 2: HSV chroma key blend_alpha: f32, _align: u32, ck_channel0: vec4f, ck_channel1: vec4f, ck_channel2: vec4f, } var pc: PushConstant; @group(0) @binding(0) var stream_texture: texture_2d; @group(0) @binding(1) var stream_sampler: sampler; struct VertexOutput { @builtin(position) position: vec4f, @location(0) uv: vec2f, } @vertex fn vertex_main(@builtin(vertex_index) vertex_index: u32) -> VertexOutput { var result: VertexOutput; result.uv = vec2f(f32(vertex_index & 1), f32(vertex_index >> 1)); result.position = pc.reprojection_transform * vec4f(result.uv.x - 0.5, 0.5 - result.uv.y, 0.0, 1.0); return result; } @fragment fn fragment_main(@location(0) uv: vec2f) -> @location(0) vec4f { var corrected_uv = uv; // tell upscaler to target a lower resolution for the edges var upscale_source_resolution = 1.0; if ENABLE_FFE { let view_size_ratio = vec2f(VIEW_WIDTH_RATIO, VIEW_HEIGHT_RATIO); let edge_ratio = vec2f(EDGE_X_RATIO, EDGE_Y_RATIO); let c1 = vec2f(C1_X, C1_Y); let c2 = vec2f(C2_X, C2_Y); let lo_bound = vec2f(LO_BOUND_X, LO_BOUND_Y); let hi_bound = vec2f(HI_BOUND_X, HI_BOUND_Y); let a_left = vec2f(A_LEFT_X, A_LEFT_Y); let b_left = vec2f(B_LEFT_X, B_LEFT_Y); let a_right = vec2f(A_RIGHT_X, A_RIGHT_Y); let b_right = vec2f(B_RIGHT_X, B_RIGHT_Y); let c_right = vec2f(C_RIGHT_X, C_RIGHT_Y); if pc.view_idx == 1 { corrected_uv.x = 1.0 - corrected_uv.x; } let center = (corrected_uv - c1) * edge_ratio / c2; let left_edge = (-b_left + sqrt(b_left * b_left + 4.0 * a_left * corrected_uv)) / (2.0 * a_left); let right_edge = (-b_right + sqrt(b_right * b_right - 4.0 * (c_right - a_right * corrected_uv))) / (2.0 * a_right); if corrected_uv.x < lo_bound.x { corrected_uv.x = left_edge.x; upscale_source_resolution = upscale_source_resolution * edge_ratio.x; } else if corrected_uv.x > hi_bound.x { corrected_uv.x = right_edge.x; upscale_source_resolution = upscale_source_resolution * edge_ratio.x; } else { corrected_uv.x = center.x; } if corrected_uv.y < lo_bound.y { corrected_uv.y = left_edge.y; upscale_source_resolution = upscale_source_resolution * edge_ratio.y; } else if corrected_uv.y > hi_bound.y { corrected_uv.y = right_edge.y; upscale_source_resolution = upscale_source_resolution * edge_ratio.y; } else { corrected_uv.y = center.y; } corrected_uv = corrected_uv * view_size_ratio; if pc.view_idx == 1 { corrected_uv.x = 1.0 - corrected_uv.x; } } var color: vec3f; if ENABLE_UPSCALING { color = sgsr(vec4f(corrected_uv.x, corrected_uv.y, 0.0, 0.0), upscale_source_resolution).xyz; } else { color = textureSample(stream_texture, stream_sampler, corrected_uv).rgb; } if ENABLE_SRGB_CORRECTION { let condition = vec3f(f32(color.r < THRESHOLD), f32(color.g < THRESHOLD), f32(color.b < THRESHOLD)); let lowValues = color * DIV12; let highValues = pow((color + vec3f(0.055)) * DIV1, GAMMA); color = condition * lowValues + (1.0 - condition) * highValues; } if ENCODING_GAMMA != 0.0 { let enc_condition = vec3f(f32(color.r < 0.0), f32(color.g < 0.0), f32(color.b < 0.0)); let enc_lowValues = color; let enc_highValues = pow(color, vec3f(ENCODING_GAMMA)); color = enc_condition * enc_lowValues + (1.0 - enc_condition) * enc_highValues; } var alpha = pc.blend_alpha; // Default to Blend passthrough mode if pc.passthrough_mode != 0 { // Chroma key var current = color; if pc.passthrough_mode == 2 { // HSV mode current = rgb_to_hsv(color); } let mask = chroma_key_mask(current); // Note: because of this calculation, we require premultiplied alpha option in the XR layer color = max(color * mask, vec3f(0.0)); alpha = mask; } return vec4f(color, alpha); } fn chroma_key_mask(color: vec3f) -> f32 { let start_max = vec3f(pc.ck_channel0.x, pc.ck_channel1.x, pc.ck_channel2.x); let start_min = vec3f(pc.ck_channel0.y, pc.ck_channel1.y, pc.ck_channel2.y); let end_min = vec3f(pc.ck_channel0.z, pc.ck_channel1.z, pc.ck_channel2.z); let end_max = vec3f(pc.ck_channel0.w, pc.ck_channel1.w, pc.ck_channel2.w); let start_mask = smoothstep(start_min.yz, start_max.yz, color.yz); let end_mask = smoothstep(end_min.yz, end_max.yz, color.yz); let sv_mask = max(start_mask, end_mask); // create t1 < t2 < t3, based on the rotated hue starting with start_max.x representing 0. let t3 = end_max.x - start_max.x; // check hue range if t3 > 1.0 { return max(sv_mask.x, sv_mask.y); } let x = fract(color.x - start_max.x); var t1 = fract(start_min.x - start_max.x); var t2 = fract(end_min.x - start_max.x); return max(max(smoothstep(t1, 0.0, x), smoothstep(t2, t3, x)), max(sv_mask.x, sv_mask.y)); } fn rgb_to_hsv(rgb: vec3f) -> vec3f { let cmax = max(rgb.r, max(rgb.g, rgb.b)); let cmin = min(rgb.r, min(rgb.g, rgb.b)); let delta = cmax - cmin; var h = 0.0; var s = 0.0; let v = cmax; if cmax > cmin { s = delta / cmax; if rgb.r == cmax { h = (rgb.g - rgb.b) / delta; } else if rgb.g == cmax { h = 2.0 + (rgb.b - rgb.r) / delta; } else { h = 4.0 + (rgb.r - rgb.g) / delta; } h = fract(h / 6.0); } return vec3f(h, s, v); } //============================================================================================================ // // // Copyright (c) 2023, Qualcomm Innovation Center, Inc. All rights reserved. // SPDX-License-Identifier: BSD-3-Clause // //============================================================================================================ fn fastLanczos2(x: f32) -> f32 { var wA: f32 = x - 4.0; let wB: f32 = x * wA - wA; wA *= wA; return wB * wA; } fn weightY(dx: f32, dy: f32, c: f32, data: vec3f) -> vec2f { let stdA: f32 = data.x; let dir: vec2f = data.yz; let edgeDis: f32 = ((dx * dir.y) + (dy * dir.x)); let x: f32 = (((dx * dx) + (dy * dy)) + ((edgeDis * edgeDis) * ((clamp(((c * c) * stdA), 0.0, 1.0) * 0.7) + -1.0))); let w: f32 = fastLanczos2(x); return vec2f(w, w * c); } fn weightYned(dx: f32, dy: f32, c: f32, data: f32) -> vec2f { let stdA: f32 = data; let x: f32 = ((dx * dx) + (dy * dy)) * 0.55 + clamp(abs(c) * stdA, 0.0, 1.0); let w: f32 = fastLanczos2(x); return vec2f(w, w * c); } fn edgeDirection(left: vec4f, right: vec4f) -> vec2f { var dir: vec2f; let RxLz: f32 = (right.x + (-left.z)); let RwLy: f32 = (right.w + (-left.y)); var delta: vec2f; delta.x = (RxLz + RwLy); delta.y = (RxLz + (-RwLy)); let lengthInv: f32 = inverseSqrt((delta.x * delta.x + 3.075740e-05) + (delta.y * delta.y)); dir.x = (delta.x * lengthInv); dir.y = (delta.y * lengthInv); return dir; } fn sgsr(in_TEXCOORD0: vec4f, source_resolution_multiplier: f32) -> vec4f { // https://github.com/SnapdragonStudios/snapdragon-gsr/issues/2 let dim = vec2f(textureDimensions(stream_texture)) * source_resolution_multiplier; let viewport_info = vec4f(1/dim.x, 1/dim.y, dim.x, dim.y); var color: vec4f; let texSample = textureSampleLevel(stream_texture, stream_sampler, in_TEXCOORD0.xy, 0.0); color.x = texSample.x; color.y = texSample.y; color.z = texSample.z; // all of these 1 values are the OperationMode // see https://github.com/SnapdragonStudios/snapdragon-gsr/tree/main/sgsr/v1#operation-mode let imgCoord: vec2f = (in_TEXCOORD0.xy * viewport_info.zw) + vec2f(-0.5, 0.5); let imgCoordPixel: vec2f = floor(imgCoord); var coord: vec2f = (imgCoordPixel * viewport_info.xy); let pl: vec2f = (imgCoord + (-imgCoordPixel)); var left: vec4f = textureGather(1, stream_texture, stream_sampler, coord); let edgeVote: f32 = abs(left.z - left.y) + abs(color[1] - left.y) + abs(color[1] - left.z); if edgeVote > UPSCALE_EDGE_THRESHOLD { coord.x += viewport_info.x; var right: vec4f = textureGather(1, stream_texture, stream_sampler, coord + vec2f(viewport_info.x, 0.0)); var upDown: vec4f; let texGatherA = textureGather(1, stream_texture, stream_sampler, coord + vec2f(0.0, -viewport_info.y)); upDown.x = texGatherA.w; upDown.y = texGatherA.z; let texGatherB = textureGather(1, stream_texture, stream_sampler, coord + vec2f(0.0, viewport_info.y)); upDown.z = texGatherB.y; upDown.w = texGatherB.x; let mean: f32 = (left.y + left.z + right.x + right.w) * 0.25; left = left - vec4(mean); right = right - vec4(mean); upDown = upDown - vec4(mean); color.w = color[1] - mean; let sum: f32 = (((((abs(left.x) + abs(left.y)) + abs(left.z)) + abs(left.w)) + (((abs(right.x) + abs(right.y)) + abs(right.z)) + abs(right.w))) + (((abs(upDown.x) + abs(upDown.y)) + abs(upDown.z)) + abs(upDown.w))); let sumMean: f32 = 1.014185e+01 / sum; let stdA: f32 = (sumMean * sumMean); var aWY: vec2f; if UPSCALE_USE_EDGE_DIRECTION { let data = vec3f(stdA, edgeDirection(left, right)); aWY = weightY(pl.x, pl.y + 1.0, upDown.x, data); aWY += weightY(pl.x - 1.0, pl.y + 1.0, upDown.y, data); aWY += weightY(pl.x - 1.0, pl.y - 2.0, upDown.z, data); aWY += weightY(pl.x, pl.y - 2.0, upDown.w, data); aWY += weightY(pl.x + 1.0, pl.y - 1.0, left.x, data); aWY += weightY(pl.x, pl.y - 1.0, left.y, data); aWY += weightY(pl.x, pl.y, left.z, data); aWY += weightY(pl.x + 1.0, pl.y, left.w, data); aWY += weightY(pl.x - 1.0, pl.y - 1.0, right.x, data); aWY += weightY(pl.x - 2.0, pl.y - 1.0, right.y, data); aWY += weightY(pl.x - 2.0, pl.y, right.z, data); aWY += weightY(pl.x - 1.0, pl.y, right.w, data); } else { let data: f32 = stdA; aWY = weightYned(pl.x, pl.y + 1.0, upDown.x, data); aWY += weightYned(pl.x - 1.0, pl.y + 1.0, upDown.y, data); aWY += weightYned(pl.x - 1.0, pl.y - 2.0, upDown.z, data); aWY += weightYned(pl.x, pl.y - 2.0, upDown.w, data); aWY += weightYned(pl.x + 1.0, pl.y - 1.0, left.x, data); aWY += weightYned(pl.x, pl.y - 1.0, left.y, data); aWY += weightYned(pl.x, pl.y, left.z, data); aWY += weightYned(pl.x + 1.0, pl.y, left.w, data); aWY += weightYned(pl.x - 1.0, pl.y - 1.0, right.x, data); aWY += weightYned(pl.x - 2.0, pl.y - 1.0, right.y, data); aWY += weightYned(pl.x - 2.0, pl.y, right.z, data); aWY += weightYned(pl.x - 1.0, pl.y, right.w, data); } let finalY: f32 = aWY.y / aWY.x; let maxY: f32 = max(max(left.y, left.z), max(right.x, right.w)); let minY: f32 = min(min(left.y, left.z), min(right.x, right.w)); var deltaY: f32 = clamp(UPSCALE_EDGE_SHARPNESS * finalY, minY, maxY) - color.w; //smooth high contrast input deltaY = clamp(deltaY, -23.0 / 255.0, 23.0 / 255.0); color.x = clamp((color.x + deltaY), 0.0, 1.0); color.y = clamp((color.y + deltaY), 0.0, 1.0); color.z = clamp((color.z + deltaY), 0.0, 1.0); } color.w = 1.0; //assume alpha channel is not used return color; } ================================================ FILE: alvr/graphics/src/lib.rs ================================================ mod lobby; mod staging; mod stream; pub use lobby::*; pub use stream::*; use alvr_common::{ DeviceMotion, Fov, Pose, glam::{Mat4, UVec2, Vec4}, }; use glow::{self as gl, HasContext}; use khronos_egl as egl; use std::{ffi::c_void, ptr}; use wgpu::{ Device, Extent3d, Instance, Queue, Texture, TextureDescriptor, TextureDimension, TextureFormat, TextureUsages, TextureView, }; pub const SDR_FORMAT: TextureFormat = TextureFormat::Rgba8Unorm; pub const SDR_FORMAT_GL: u32 = gl::RGBA8; pub const GL_TEXTURE_EXTERNAL_OES: u32 = 0x8D65; pub const MAX_PUSH_CONSTANTS_SIZE: u32 = 128; type CreateImageFn = unsafe extern "C" fn( egl::EGLDisplay, egl::EGLContext, egl::Enum, egl::EGLClientBuffer, *const egl::Int, ) -> egl::EGLImage; type DestroyImageFn = unsafe extern "C" fn(egl::EGLDisplay, egl::EGLImage) -> egl::Boolean; type GetNativeClientBufferFn = unsafe extern "C" fn(*const c_void) -> egl::EGLClientBuffer; type ImageTargetTexture2DFn = unsafe extern "C" fn(egl::Enum, egl::EGLImage); pub struct HandData { pub grip_motion: Option, pub detached_grip_motion: Option, pub skeleton_joints: Option<[Pose; 26]>, } pub fn check_error(gl: &gl::Context, message_context: &str) { let err = unsafe { gl.get_error() }; if err != glow::NO_ERROR { alvr_common::error!("gl error {message_context} -> {err}"); std::process::abort(); } } macro_rules! ck { ($gl_ctx:ident.$($gl_cmd:tt)*) => {{ let res = $gl_ctx.$($gl_cmd)*; #[cfg(debug_assertions)] crate::check_error(&$gl_ctx, &format!("{}:{}: {}", file!(), line!(), stringify!($($gl_cmd)*))); res }}; } pub(crate) use ck; fn projection_from_fov(fov: Fov) -> Mat4 { const NEAR: f32 = 0.1; let tanl = f32::tan(fov.left); let tanr = f32::tan(fov.right); let tanu = f32::tan(fov.up); let tand = f32::tan(fov.down); let a = 2.0 / (tanr - tanl); let b = 2.0 / (tanu - tand); let c = (tanr + tanl) / (tanr - tanl); let d = (tanu + tand) / (tanu - tand); // note: for wgpu compatibility, the b and d components should be flipped. Maybe a bug in the // viewport handling in wgpu? Mat4::from_cols( Vec4::new(a, 0.0, c, 0.0), Vec4::new(0.0, -b, -d, 0.0), Vec4::new(0.0, 0.0, -1.0, -NEAR), Vec4::new(0.0, 0.0, -1.0, 0.0), ) .transpose() } pub fn choose_swapchain_format(supported_formats: &[u32], enable_hdr: bool) -> u32 { // Priority-sorted list of swapchain formats we'll accept-- let mut app_supported_swapchain_formats = vec![gl::SRGB8_ALPHA8, gl::RGBA8]; // float16 is required for HDR output. However, float16 swapchains // have a high perf cost, so only use these if HDR is enabled. if enable_hdr { app_supported_swapchain_formats.insert(0, gl::RGBA16F); } for format in app_supported_swapchain_formats { if supported_formats.contains(&format) { return format; } } // If we can't enumerate, default to a required format gl::RGBA8 } pub fn gl_format_to_wgpu(format: u32) -> TextureFormat { match format { gl::SRGB8_ALPHA8 => TextureFormat::Rgba8UnormSrgb, gl::RGBA8 => TextureFormat::Rgba8Unorm, gl::RGBA16F => TextureFormat::Rgba16Float, _ => panic!("Unsupported GL format: {format}"), } } pub fn create_texture(device: &Device, resolution: UVec2, format: TextureFormat) -> Texture { device.create_texture(&TextureDescriptor { label: None, size: Extent3d { width: resolution.x, height: resolution.y, depth_or_array_layers: 1, }, mip_level_count: 1, sample_count: 1, dimension: TextureDimension::D2, format, usage: TextureUsages::COPY_DST | TextureUsages::TEXTURE_BINDING, view_formats: &[], }) } #[cfg(not(any(target_os = "macos", target_os = "ios")))] fn create_texture_from_gles( device: &Device, texture: u32, resolution: UVec2, format: TextureFormat, ) -> Texture { use std::num::NonZeroU32; use wgpu::{ TextureUses, hal::{self, MemoryFlags, api}, }; let size = Extent3d { width: resolution.x, height: resolution.y, depth_or_array_layers: 1, }; unsafe { let hal_texture = device.as_hal::(|device| { device.unwrap().texture_from_raw( NonZeroU32::new(texture).unwrap(), &hal::TextureDescriptor { label: None, size, mip_level_count: 1, sample_count: 1, dimension: TextureDimension::D2, format, usage: TextureUses::COLOR_TARGET, memory_flags: MemoryFlags::empty(), view_formats: vec![], }, Some(Box::new(|| ())), ) }); device.create_texture_from_hal::( hal_texture, &TextureDescriptor { label: None, size, mip_level_count: 1, sample_count: 1, dimension: TextureDimension::D2, format, usage: TextureUsages::RENDER_ATTACHMENT, view_formats: &[], }, ) } } #[cfg(any(target_os = "macos", target_os = "ios"))] fn create_texture_from_gles(_: &Device, _: u32, _: UVec2, _: TextureFormat) -> Texture { unimplemented!() } // This is used to convert OpenXR swapchains to wgpu pub fn create_gl_swapchain( device: &Device, gl_textures: &[u32], resolution: UVec2, format: TextureFormat, ) -> Vec { gl_textures .iter() .map(|gl_tex| { create_texture_from_gles(device, *gl_tex, resolution, format) .create_view(&Default::default()) }) .collect() } pub struct GraphicsContext { _instance: Instance, #[cfg(not(any(windows, target_os = "macos", target_os = "ios")))] adapter: wgpu::Adapter, device: Device, queue: Queue, pub egl_display: egl::Display, pub egl_config: egl::Config, pub egl_context: egl::Context, pub gl_context: gl::Context, #[cfg(not(any(windows, target_os = "macos", target_os = "ios")))] dummy_surface: egl::Surface, create_image: CreateImageFn, destroy_image: DestroyImageFn, get_native_client_buffer: GetNativeClientBufferFn, image_target_texture_2d: ImageTargetTexture2DFn, } impl GraphicsContext { #[cfg(not(any(windows, target_os = "macos", target_os = "ios")))] pub fn new_gl() -> Self { use std::mem; use wgpu::{ Backends, DeviceDescriptor, Features, InstanceDescriptor, InstanceFlags, Limits, MemoryHints, Trace, hal::api, }; const CREATE_IMAGE_FN_STR: &str = "eglCreateImageKHR"; const DESTROY_IMAGE_FN_STR: &str = "eglDestroyImageKHR"; const GET_NATIVE_CLIENT_BUFFER_FN_STR: &str = "eglGetNativeClientBufferANDROID"; const IMAGE_TARGET_TEXTURE_2D_FN_STR: &str = "glEGLImageTargetTexture2DOES"; let flags = if cfg!(debug_assertions) { InstanceFlags::DEBUG | InstanceFlags::VALIDATION } else { InstanceFlags::empty() }; let instance = Instance::new(&InstanceDescriptor { backends: Backends::GL, flags, ..Default::default() }); let adapter = instance.enumerate_adapters(Backends::GL).remove(0); let (device, queue) = pollster::block_on(adapter.request_device(&DeviceDescriptor { label: None, required_features: Features::PUSH_CONSTANTS, required_limits: Limits { max_push_constant_size: MAX_PUSH_CONSTANTS_SIZE, ..adapter.limits() }, memory_hints: MemoryHints::Performance, trace: Trace::Off, })) .unwrap(); let raw_instance = unsafe { instance.as_hal::() }.unwrap(); let egl_display = raw_instance.raw_display(); let egl_config = raw_instance.egl_config(); let ( egl_context, gl_context, dummy_surface, create_image, destroy_image, get_native_client_buffer, image_target_texture_2d, ) = unsafe { adapter.as_hal::(|raw_adapter| { let adapter_context = raw_adapter.unwrap().adapter_context(); let egl_instance = adapter_context.egl_instance().unwrap(); let egl_context = egl::Context::from_ptr(adapter_context.raw_context()); const PBUFFER_ATTRIBS: [i32; 5] = [egl::WIDTH, 16, egl::HEIGHT, 16, egl::NONE]; let dummy_surface = egl_instance .create_pbuffer_surface(egl_display, egl_config, &PBUFFER_ATTRIBS) .unwrap(); egl_instance .make_current( egl_display, Some(dummy_surface), Some(dummy_surface), Some(egl_context), ) .unwrap(); let gl_context = gl::Context::from_loader_function(|fn_name| { egl_instance .get_proc_address(fn_name) .map_or(ptr::null(), |f| f as *const c_void) }); let get_fn_ptr = |fn_name| { egl_instance .get_proc_address(fn_name) .map_or(ptr::null(), |f| f as *const c_void) }; let create_image: CreateImageFn = mem::transmute(get_fn_ptr(CREATE_IMAGE_FN_STR)); let destroy_image: DestroyImageFn = mem::transmute(get_fn_ptr(DESTROY_IMAGE_FN_STR)); let get_native_client_buffer: GetNativeClientBufferFn = mem::transmute(get_fn_ptr(GET_NATIVE_CLIENT_BUFFER_FN_STR)); let image_target_texture_2d: ImageTargetTexture2DFn = mem::transmute(get_fn_ptr(IMAGE_TARGET_TEXTURE_2D_FN_STR)); ( egl_context, gl_context, dummy_surface, create_image, destroy_image, get_native_client_buffer, image_target_texture_2d, ) }) }; Self { _instance: instance, adapter, device, queue, egl_display, egl_config, egl_context, gl_context, dummy_surface, create_image, destroy_image, get_native_client_buffer, image_target_texture_2d, } } #[cfg(any(windows, target_os = "macos", target_os = "ios"))] pub fn new_gl() -> Self { unimplemented!() } pub fn make_current(&self) { #[cfg(not(any(windows, target_os = "macos", target_os = "ios")))] unsafe { self.adapter .as_hal::(|raw_adapter| { let egl_instance = raw_adapter .unwrap() .adapter_context() .egl_instance() .unwrap(); egl_instance .make_current( self.egl_display, Some(self.dummy_surface), Some(self.dummy_surface), Some(self.egl_context), ) .unwrap(); }) }; } /// # Safety /// `buffer` must be a valid AHardwareBuffer. /// `texture` must be a valid GL texture. pub unsafe fn render_ahardwarebuffer_using_texture( &self, buffer: *const c_void, texture: gl::Texture, render_cb: impl FnOnce(), ) { const EGL_NATIVE_BUFFER_ANDROID: u32 = 0x3140; if !buffer.is_null() { let client_buffer = unsafe { (self.get_native_client_buffer)(buffer) }; check_error(&self.gl_context, "get_native_client_buffer"); let image = unsafe { (self.create_image)( self.egl_display.as_ptr(), egl::NO_CONTEXT, EGL_NATIVE_BUFFER_ANDROID, client_buffer, ptr::null(), ) }; check_error(&self.gl_context, "create_image"); unsafe { self.gl_context .bind_texture(GL_TEXTURE_EXTERNAL_OES, Some(texture)) }; check_error(&self.gl_context, "bind texture OES"); unsafe { (self.image_target_texture_2d)(GL_TEXTURE_EXTERNAL_OES, image) }; check_error(&self.gl_context, "image_target_texture_2d"); render_cb(); unsafe { (self.destroy_image)(self.egl_display.as_ptr(), image) }; check_error(&self.gl_context, "destroy_image"); } } } #[cfg(not(windows))] impl Default for GraphicsContext { fn default() -> Self { Self::new_gl() } } ================================================ FILE: alvr/graphics/src/lobby.rs ================================================ use super::{GraphicsContext, MAX_PUSH_CONSTANTS_SIZE, SDR_FORMAT}; use crate::HandData; use alvr_common::{ BodySkeleton, DeviceMotion, ViewParams, glam::{IVec2, Mat4, Quat, UVec2, Vec3}, }; use glyph_brush_layout::{ FontId, GlyphPositioner, HorizontalAlign, Layout, SectionGeometry, SectionText, VerticalAlign, ab_glyph::{Font, FontRef, ScaleFont}, }; use std::{f32::consts::FRAC_PI_2, mem, rc::Rc}; use wgpu::{ BindGroup, BindGroupDescriptor, BindGroupEntry, BindGroupLayout, BindGroupLayoutDescriptor, BindGroupLayoutEntry, BindingResource, BindingType, BlendComponent, BlendFactor, BlendOperation, BlendState, Color, ColorTargetState, ColorWrites, CommandEncoderDescriptor, Device, Extent3d, FilterMode, FragmentState, LoadOp, Operations, Origin3d, PipelineLayoutDescriptor, PrimitiveState, PrimitiveTopology, PushConstantRange, RenderPass, RenderPassColorAttachment, RenderPassDescriptor, RenderPipeline, RenderPipelineDescriptor, SamplerBindingType, SamplerDescriptor, ShaderModuleDescriptor, ShaderStages, StoreOp, TexelCopyBufferLayout, TexelCopyTextureInfo, Texture, TextureAspect, TextureSampleType, TextureView, TextureViewDimension, VertexState, include_wgsl, }; const TRANSFORM_CONST_SIZE: u32 = mem::size_of::() as u32; const OBJECT_TYPE_CONST_SIZE: u32 = mem::size_of::() as u32; const FLOOR_SIDE_CONST_SIZE: u32 = mem::size_of::() as u32; const COLOR_CONST_SIZE: u32 = mem::size_of::() as u32; const QUAD_PUSH_CONTANTS_SIZE: u32 = TRANSFORM_CONST_SIZE + OBJECT_TYPE_CONST_SIZE + FLOOR_SIDE_CONST_SIZE; const LINE_PUSH_CONTANTS_SIZE: u32 = TRANSFORM_CONST_SIZE + COLOR_CONST_SIZE; const _: () = assert!( QUAD_PUSH_CONTANTS_SIZE <= MAX_PUSH_CONSTANTS_SIZE && LINE_PUSH_CONTANTS_SIZE <= MAX_PUSH_CONSTANTS_SIZE, "Push constants size exceeds the maximum size" ); const TRANSFORM_CONST_OFFSET: u32 = 0; const OBJECT_TYPE_CONST_OFFSET: u32 = TRANSFORM_CONST_SIZE; const FLOOR_SIDE_CONST_OFFSET: u32 = OBJECT_TYPE_CONST_OFFSET + OBJECT_TYPE_CONST_SIZE; const COLOR_CONST_OFFSET: u32 = TRANSFORM_CONST_SIZE; const FLOOR_SIDE: f32 = 300.0; const HUD_DIST: f32 = 5.0; const HUD_SIDE: f32 = 3.5; const HUD_TEXTURE_SIDE: usize = 1024; const FONT_SIZE: f32 = 50.0; const FAST_BORDER_OFFSETS: [IVec2; 8] = [ IVec2::new(0, -3), IVec2::new(2, -2), IVec2::new(3, 0), IVec2::new(2, 2), IVec2::new(0, 3), IVec2::new(-2, 2), IVec2::new(-3, 0), IVec2::new(-2, -2), ]; const MAX_BORDER_OFFSET: i32 = 3; const HAND_SKELETON_BONES: [(usize, usize); 19] = [ // Thumb (2, 3), (3, 4), (4, 5), // Index (6, 7), (7, 8), (8, 9), (9, 10), // Middle (11, 12), (12, 13), (13, 14), (14, 15), // Ring (16, 17), (17, 18), (18, 19), (19, 20), // Pinky (21, 22), (22, 23), (23, 24), (24, 25), ]; const BODY_JOINT_RELATIONS_FB: [(usize, usize); 30] = [ // Spine (1, 2), (2, 3), (3, 4), (4, 5), (5, 6), (6, 7), // Left arm (5, 8), (8, 9), (9, 10), (10, 11), (11, 12), // Right arm (5, 13), (13, 14), (14, 15), (15, 16), (16, 17), // Left leg (1, 18), (18, 19), (19, 20), (20, 21), (21, 22), (22, 23), (23, 24), // Right leg (1, 25), (25, 26), (26, 27), (27, 28), (28, 29), (29, 30), (30, 31), ]; const BODY_JOINT_RELATIONS_BD: [(usize, usize); 23] = [ // Left leg (0, 1), (1, 4), (4, 7), (7, 10), // Right leg (0, 2), (2, 5), (5, 8), (8, 11), // Spine (0, 3), (3, 6), (6, 9), (9, 12), (12, 15), // Left arm (9, 13), (13, 16), (16, 18), (18, 20), (20, 22), // Right arm (9, 14), (14, 17), (17, 19), (19, 21), (21, 23), ]; fn create_pipeline( device: &Device, label: &str, bind_group_layouts: &[&BindGroupLayout], push_constants_len: u32, shader: ShaderModuleDescriptor, topology: PrimitiveTopology, ) -> RenderPipeline { let shader_module = device.create_shader_module(shader); device.create_render_pipeline(&RenderPipelineDescriptor { label: Some(label), // Note: Layout cannot be inferred because of a bug with push constants layout: Some(&device.create_pipeline_layout(&PipelineLayoutDescriptor { label: Some(label), bind_group_layouts, push_constant_ranges: &[PushConstantRange { stages: ShaderStages::VERTEX_FRAGMENT, range: 0..push_constants_len, }], })), vertex: VertexState { module: &shader_module, entry_point: None, compilation_options: Default::default(), buffers: &[], }, primitive: PrimitiveState { topology, ..Default::default() }, depth_stencil: None, multisample: Default::default(), fragment: Some(FragmentState { module: &shader_module, entry_point: None, compilation_options: Default::default(), targets: &[Some(ColorTargetState { format: SDR_FORMAT, blend: Some(BlendState { color: BlendComponent { src_factor: BlendFactor::SrcAlpha, dst_factor: BlendFactor::OneMinusSrcAlpha, operation: BlendOperation::Add, }, alpha: BlendComponent { src_factor: BlendFactor::One, dst_factor: BlendFactor::OneMinusSrcAlpha, operation: BlendOperation::Add, }, }), write_mask: ColorWrites::ALL, })], }), multiview: None, cache: None, }) } pub struct LobbyViewParams { pub swapchain_index: u32, pub view_params: ViewParams, } pub struct LobbyRenderer { context: Rc, quad_pipeline: RenderPipeline, line_pipeline: RenderPipeline, hud_texture: Texture, bind_group: BindGroup, render_targets: [Vec; 2], } impl LobbyRenderer { pub fn new( context: Rc, view_resolution: UVec2, swapchain_textures: [Vec; 2], initial_hud_message: &str, ) -> Self { let device = &context.device; let hud_texture = super::create_texture(device, UVec2::ONE * HUD_TEXTURE_SIDE as u32, SDR_FORMAT); let bind_group_layout = device.create_bind_group_layout(&BindGroupLayoutDescriptor { label: None, entries: &[ BindGroupLayoutEntry { binding: 0, visibility: ShaderStages::FRAGMENT, ty: BindingType::Texture { sample_type: TextureSampleType::Float { filterable: true }, view_dimension: TextureViewDimension::D2, multisampled: false, }, count: None, }, BindGroupLayoutEntry { binding: 1, visibility: ShaderStages::FRAGMENT, ty: BindingType::Sampler(SamplerBindingType::Filtering), count: None, }, ], }); let quad_pipeline = create_pipeline( device, "lobby_quad", &[&bind_group_layout], QUAD_PUSH_CONTANTS_SIZE, include_wgsl!("../resources/lobby_quad.wgsl"), PrimitiveTopology::TriangleStrip, ); let line_pipeline = create_pipeline( device, "lobby_line", &[], LINE_PUSH_CONTANTS_SIZE, include_wgsl!("../resources/lobby_line.wgsl"), PrimitiveTopology::LineList, ); let bind_group = device.create_bind_group(&BindGroupDescriptor { label: None, layout: &bind_group_layout, entries: &[ BindGroupEntry { binding: 0, resource: BindingResource::TextureView( &hud_texture.create_view(&Default::default()), ), }, BindGroupEntry { binding: 1, resource: BindingResource::Sampler(&device.create_sampler( &SamplerDescriptor { mag_filter: FilterMode::Linear, min_filter: FilterMode::Linear, ..Default::default() }, )), }, ], }); let render_targets = [ super::create_gl_swapchain(device, &swapchain_textures[0], view_resolution, SDR_FORMAT), super::create_gl_swapchain(device, &swapchain_textures[1], view_resolution, SDR_FORMAT), ]; let this = Self { context, quad_pipeline, line_pipeline, hud_texture, bind_group, render_targets, }; this.update_hud_message(initial_hud_message); this } pub fn update_hud_message(&self, message: &str) { let ubuntu_font = FontRef::try_from_slice(include_bytes!("../resources/Ubuntu-Medium.ttf")).unwrap(); let section_glyphs = Layout::default() .h_align(HorizontalAlign::Center) .v_align(VerticalAlign::Center) .calculate_glyphs( &[&ubuntu_font], &SectionGeometry { screen_position: ( HUD_TEXTURE_SIDE as f32 / 2_f32, HUD_TEXTURE_SIDE as f32 / 2_f32, ), ..Default::default() }, &[SectionText { text: message, scale: FONT_SIZE.into(), font_id: FontId(0), }], ); let scaled_font = ubuntu_font.as_scaled(FONT_SIZE); let mut buffer = vec![0; HUD_TEXTURE_SIDE * HUD_TEXTURE_SIDE * 4]; for section_glyph in section_glyphs { if let Some(outlined) = scaled_font.outline_glyph(section_glyph.glyph) { let bounds = outlined.px_bounds(); outlined.draw(|x, y, alpha| { let x = x as i32 + bounds.min.x as i32; let y = y as i32 + bounds.min.y as i32; if x >= MAX_BORDER_OFFSET && y >= MAX_BORDER_OFFSET && x < HUD_TEXTURE_SIDE as i32 - MAX_BORDER_OFFSET && y < HUD_TEXTURE_SIDE as i32 - MAX_BORDER_OFFSET { let coord = (y as usize * HUD_TEXTURE_SIDE + x as usize) * 4; let value = (alpha * 255.0) as u8; buffer[coord] = value; buffer[coord + 1] = value; buffer[coord + 2] = value; // Render opacity with border for offset in &FAST_BORDER_OFFSETS { let coord = ((y + offset.y) as usize * HUD_TEXTURE_SIDE + (x + offset.x) as usize) * 4; buffer[coord + 3] = u8::max(buffer[coord + 3], value); } } }); } } self.context.queue.write_texture( TexelCopyTextureInfo { texture: &self.hud_texture, mip_level: 0, origin: Origin3d::ZERO, aspect: TextureAspect::All, }, &buffer, TexelCopyBufferLayout { offset: 0, bytes_per_row: Some(HUD_TEXTURE_SIDE as u32 * 4), rows_per_image: Some(HUD_TEXTURE_SIDE as u32), }, Extent3d { width: HUD_TEXTURE_SIDE as u32, height: HUD_TEXTURE_SIDE as u32, depth_or_array_layers: 1, }, ); } pub fn render( &self, view_params: [LobbyViewParams; 2], hand_data: [HandData; 2], body_skeleton: Option, additional_motions: Option>, render_background: bool, show_velocities: bool, ) { let mut encoder = self .context .device .create_command_encoder(&CommandEncoderDescriptor { label: Some("lobby_command_encoder"), }); for (view_idx, view_input) in view_params.iter().enumerate() { let view = Mat4::from_rotation_translation( view_input.view_params.pose.orientation, view_input.view_params.pose.position, ) .inverse(); let view_proj = super::projection_from_fov(view_input.view_params.fov) * view; let clear_color = if render_background { Color { r: 0.0, g: 0.0, b: 0.02, a: 1.0, } } else { Color::TRANSPARENT }; let mut pass = encoder.begin_render_pass(&RenderPassDescriptor { label: Some(&format!("lobby_view_{view_idx}")), color_attachments: &[Some(RenderPassColorAttachment { view: &self.render_targets[view_idx][view_input.swapchain_index as usize], resolve_target: None, ops: Operations { load: LoadOp::Clear(clear_color), store: StoreOp::Store, }, })], ..Default::default() }); fn transform_draw(pass: &mut RenderPass, transform: Mat4, vertices_count: u32) { let data = transform .to_cols_array() .iter() .flat_map(|v| v.to_le_bytes()) .collect::>(); pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, TRANSFORM_CONST_OFFSET, &data, ); pass.draw(0..vertices_count, 0..1); } // Draw the following geometry in the correct order (depth buffer is disabled) // Bind quad pipeline pass.set_pipeline(&self.quad_pipeline); pass.set_bind_group(0, &self.bind_group, &[]); if render_background { // Render ground pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, OBJECT_TYPE_CONST_OFFSET, &0_u32.to_le_bytes(), ); pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, FLOOR_SIDE_CONST_OFFSET, &FLOOR_SIDE.to_le_bytes(), ); let transform = view_proj * Mat4::from_rotation_x(-FRAC_PI_2) * Mat4::from_scale(Vec3::ONE * FLOOR_SIDE); transform_draw(&mut pass, transform, 4); } // Render HUD pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, OBJECT_TYPE_CONST_OFFSET, &1_u32.to_le_bytes(), ); for i in 0..4 { let transform = Mat4::from_rotation_y(FRAC_PI_2 * i as f32) * Mat4::from_translation(Vec3::new(0.0, HUD_SIDE / 2.0, -HUD_DIST)) * Mat4::from_scale(Vec3::ONE * HUD_SIDE); transform_draw(&mut pass, view_proj * transform, 4); } fn draw_crosshair( pass: &mut RenderPass, motion: &DeviceMotion, view_proj: Mat4, show_velocities: bool, color: &[u8; 4], ) { let hand_transform = Mat4::from_scale_rotation_translation( Vec3::ONE * 0.2, motion.pose.orientation, motion.pose.position, ); // Draw crosshair let segment_rotations = [ Mat4::IDENTITY, Mat4::from_rotation_y(FRAC_PI_2), Mat4::from_rotation_x(FRAC_PI_2), ]; pass.set_push_constants(ShaderStages::VERTEX_FRAGMENT, COLOR_CONST_OFFSET, color); for rot in &segment_rotations { let transform = hand_transform * *rot * Mat4::from_scale(Vec3::ONE * 0.5) * Mat4::from_translation(Vec3::Z * 0.5); transform_draw(pass, view_proj * transform, 2); } if show_velocities { // Draw linear velocity let transform = Mat4::from_scale_rotation_translation( Vec3::ONE * motion.linear_velocity.length() * 0.2, Quat::from_rotation_arc(-Vec3::Z, motion.linear_velocity.normalize()), motion.pose.position, ); pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, COLOR_CONST_OFFSET, &[255, 0, 0, 255], ); transform_draw(pass, view_proj * transform, 2); // Draw angular velocity let transform = Mat4::from_scale_rotation_translation( Vec3::ONE * motion.angular_velocity.length() * 0.01, Quat::from_rotation_arc(-Vec3::Z, motion.angular_velocity.normalize()), motion.pose.position, ); pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, COLOR_CONST_OFFSET, &[0, 255, 0, 255], ); transform_draw(pass, view_proj * transform, 2); } } // Render hands and body skeleton pass.set_pipeline(&self.line_pipeline); for data in &hand_data { if let Some(skeleton) = data.skeleton_joints { pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, COLOR_CONST_OFFSET, &[255, 255, 255, 255], ); for (joint1_idx, joint2_idx) in HAND_SKELETON_BONES { let j1_pose = skeleton[joint1_idx]; let j2_pose = skeleton[joint2_idx]; let transform = Mat4::from_scale_rotation_translation( Vec3::ONE * Vec3::distance(j1_pose.position, j2_pose.position), j1_pose.orientation, j1_pose.position, ); transform_draw(&mut pass, view_proj * transform, 2); } } if let Some(motion) = data.grip_motion { draw_crosshair( &mut pass, &motion, view_proj, show_velocities, &[255, 255, 255, 255], ); } if let Some(motion) = data.detached_grip_motion { draw_crosshair( &mut pass, &motion, view_proj, show_velocities, &[50, 50, 50, 255], ); } } if let Some(motions) = &additional_motions { for motion in motions { draw_crosshair( &mut pass, motion, view_proj, show_velocities, &[255, 255, 255, 255], ); } } if let Some(skeleton) = &body_skeleton { let (joints, relations) = match skeleton { BodySkeleton::Fb(skeleton) => { let mut joints = skeleton.upper_body.to_vec(); if let Some(lower_body) = skeleton.lower_body { joints.extend(lower_body); } let relations = BODY_JOINT_RELATIONS_FB.as_slice(); (joints, relations) } BodySkeleton::Bd(joints) => { (joints.0.to_vec(), BODY_JOINT_RELATIONS_BD.as_slice()) } }; for (joint1_idx, joint2_idx) in relations { if let (Some(Some(j1_pose)), Some(Some(j2_pose))) = (joints.get(*joint1_idx), joints.get(*joint2_idx)) { let transform = Mat4::from_scale_rotation_translation( Vec3::ONE * Vec3::distance(j1_pose.position, j2_pose.position), Quat::from_rotation_arc( -Vec3::Z, (j2_pose.position - j1_pose.position).normalize(), ), j1_pose.position, ); transform_draw(&mut pass, view_proj * transform, 2); } } } } self.context.queue.submit(Some(encoder.finish())); } } ================================================ FILE: alvr/graphics/src/staging.rs ================================================ use super::{GraphicsContext, ck}; use crate::GL_TEXTURE_EXTERNAL_OES; use alvr_common::glam::{IVec2, UVec2}; use glow::{self as gl, HasContext}; use std::{ffi::c_void, rc::Rc}; fn create_program( gl: &gl::Context, vertex_shader_source: &str, fragment_shader_source: &str, ) -> gl::Program { unsafe { let vertex_shader = ck!(gl.create_shader(gl::VERTEX_SHADER).unwrap()); ck!(gl.shader_source(vertex_shader, vertex_shader_source)); ck!(gl.compile_shader(vertex_shader)); if !gl.get_shader_compile_status(vertex_shader) { panic!( "Failed to compile vertex shader: {}", gl.get_shader_info_log(vertex_shader) ); } let fragment_shader = ck!(gl.create_shader(gl::FRAGMENT_SHADER).unwrap()); ck!(gl.shader_source(fragment_shader, fragment_shader_source)); ck!(gl.compile_shader(fragment_shader)); if !gl.get_shader_compile_status(fragment_shader) { panic!( "Failed to compile fragment shader: {}", gl.get_shader_info_log(fragment_shader) ); } let program = ck!(gl.create_program().unwrap()); ck!(gl.attach_shader(program, vertex_shader)); ck!(gl.attach_shader(program, fragment_shader)); ck!(gl.link_program(program)); if !gl.get_program_link_status(program) { panic!( "Failed to link program: {}", gl.get_program_info_log(program) ); } ck!(gl.delete_shader(vertex_shader)); ck!(gl.delete_shader(fragment_shader)); program } } pub struct StagingRenderer { context: Rc, program: gl::Program, view_idx_uloc: gl::UniformLocation, surface_texture: gl::Texture, framebuffers: [gl::Framebuffer; 2], viewport_size: IVec2, } impl StagingRenderer { pub fn new( context: Rc, staging_textures: [gl::Texture; 2], view_resolution: UVec2, fix_limited_range: bool, ) -> Self { let gl = &context.gl_context; context.make_current(); // Add #defines into the shader after the first line let mut frag_lines: Vec<&str> = include_str!("../resources/staging_fragment.glsl") .lines() .collect(); if fix_limited_range { frag_lines.insert(1, "#line 0 1\n#define FIX_LIMITED_RANGE"); } let frag_str = frag_lines.join("\n"); let program = create_program( gl, include_str!("../resources/staging_vertex.glsl"), frag_str.as_str(), ); unsafe { // This is an external surface and storage should not be initialized let surface_texture = ck!(gl.create_texture().unwrap()); let mut framebuffers = vec![]; for tex in staging_textures { let framebuffer = ck!(gl.create_framebuffer().unwrap()); ck!(gl.bind_framebuffer(gl::DRAW_FRAMEBUFFER, Some(framebuffer))); ck!(gl.framebuffer_texture_2d( gl::DRAW_FRAMEBUFFER, gl::COLOR_ATTACHMENT0, gl::TEXTURE_2D, Some(tex), 0, )); framebuffers.push(framebuffer); } ck!(gl.bind_framebuffer(gl::FRAMEBUFFER, None)); let view_idx_uloc = ck!(gl.get_uniform_location(program, "view_idx")).unwrap(); Self { context, program, surface_texture, view_idx_uloc, framebuffers: framebuffers.try_into().unwrap(), viewport_size: view_resolution.as_ivec2(), } } } #[allow(unused_variables)] pub fn render(&self, hardware_buffer: *mut c_void) { let gl = &self.context.gl_context; self.context.make_current(); unsafe { self.context.render_ahardwarebuffer_using_texture( hardware_buffer, self.surface_texture, || { ck!(gl.use_program(Some(self.program))); ck!(gl.viewport(0, 0, self.viewport_size.x, self.viewport_size.y)); ck!(gl.disable(gl::SCISSOR_TEST)); ck!(gl.disable(gl::STENCIL_TEST)); for (i, framebuffer) in self.framebuffers.iter().enumerate() { ck!(gl.bind_framebuffer(gl::DRAW_FRAMEBUFFER, Some(*framebuffer))); ck!(gl.active_texture(gl::TEXTURE0)); ck!(gl.bind_texture(GL_TEXTURE_EXTERNAL_OES, Some(self.surface_texture))); ck!(gl.bind_sampler(0, None)); ck!(gl.uniform_1_i32(Some(&self.view_idx_uloc), i as i32)); ck!(gl.draw_arrays(gl::TRIANGLE_STRIP, 0, 4)); } }, ) }; } } impl Drop for StagingRenderer { fn drop(&mut self) { let gl = &self.context.gl_context; self.context.make_current(); unsafe { ck!(gl.delete_program(self.program)); ck!(gl.delete_texture(self.surface_texture)); for framebuffer in &self.framebuffers { ck!(gl.delete_framebuffer(*framebuffer)); } } } } ================================================ FILE: alvr/graphics/src/stream.rs ================================================ use super::{GraphicsContext, MAX_PUSH_CONSTANTS_SIZE, staging::StagingRenderer}; use alvr_common::{ ViewParams, glam::{self, Mat4, UVec2, Vec3, Vec4}, }; use alvr_session::{FoveatedEncodingConfig, PassthroughMode, UpscalingConfig}; use std::{ffi::c_void, iter, mem, rc::Rc}; use wgpu::{ BindGroup, BindGroupDescriptor, BindGroupEntry, BindGroupLayoutDescriptor, BindGroupLayoutEntry, BindingResource, BindingType, Color, ColorTargetState, ColorWrites, FragmentState, LoadOp, PipelineCompilationOptions, PipelineLayoutDescriptor, PrimitiveState, PrimitiveTopology, PushConstantRange, RenderPass, RenderPassColorAttachment, RenderPassDescriptor, RenderPipeline, RenderPipelineDescriptor, SamplerBindingType, SamplerDescriptor, ShaderStages, StoreOp, TextureSampleType, TextureView, TextureViewDescriptor, TextureViewDimension, VertexState, include_wgsl, }; const FLOAT_SIZE: u32 = mem::size_of::() as u32; const U32_SIZE: u32 = mem::size_of::() as u32; const VEC4_SIZE: u32 = mem::size_of::() as u32; const TRANSFORM_SIZE: u32 = mem::size_of::() as u32; const TRANSFORM_CONST_OFFSET: u32 = 0; const VIEW_INDEX_CONST_OFFSET: u32 = TRANSFORM_SIZE; const PASSTHROUGH_MODE_OFFSET: u32 = VIEW_INDEX_CONST_OFFSET + U32_SIZE; const ALPHA_CONST_OFFSET: u32 = PASSTHROUGH_MODE_OFFSET + U32_SIZE; const CK_CHANNEL0_CONST_OFFSET: u32 = ALPHA_CONST_OFFSET + FLOAT_SIZE + U32_SIZE; const CK_CHANNEL1_CONST_OFFSET: u32 = CK_CHANNEL0_CONST_OFFSET + VEC4_SIZE; const CK_CHANNEL2_CONST_OFFSET: u32 = CK_CHANNEL1_CONST_OFFSET + VEC4_SIZE; const PUSH_CONSTANTS_SIZE: u32 = CK_CHANNEL2_CONST_OFFSET + VEC4_SIZE; const _: () = assert!( PUSH_CONSTANTS_SIZE <= MAX_PUSH_CONSTANTS_SIZE, "Push constants size exceeds the maximum size" ); pub struct StreamViewParams { pub swapchain_index: u32, pub input_view_params: ViewParams, pub output_view_params: ViewParams, } #[derive(Debug)] struct ViewObjects { bind_group: BindGroup, render_target: Vec, } pub struct StreamRenderer { context: Rc, staging_renderer: StagingRenderer, pipeline: RenderPipeline, views_objects: [ViewObjects; 2], } impl StreamRenderer { #[expect(clippy::too_many_arguments)] #[cfg_attr(any(target_os = "macos", target_os = "ios"), expect(unused))] pub fn new( context: Rc, base_view_resolution: UVec2, target_view_resolution: UVec2, swapchain_textures: [Vec; 2], target_format: u32, foveated_encoding: Option, enable_srgb_correction: bool, fix_limited_range: bool, encoding_gamma: f32, upscaling: Option, ) -> Self { let device = &context.device; let target_format = super::gl_format_to_wgpu(target_format); let bind_group_layout = device.create_bind_group_layout(&BindGroupLayoutDescriptor { label: None, entries: &[ BindGroupLayoutEntry { binding: 0, visibility: ShaderStages::FRAGMENT, ty: BindingType::Texture { sample_type: TextureSampleType::Float { filterable: true }, view_dimension: TextureViewDimension::D2, multisampled: false, }, count: None, }, BindGroupLayoutEntry { binding: 1, visibility: ShaderStages::FRAGMENT, ty: BindingType::Sampler(SamplerBindingType::Filtering), count: None, }, ], }); let shader_module = device.create_shader_module(include_wgsl!("../resources/stream.wgsl")); let mut constants = vec![]; constants.extend([ ("ENABLE_SRGB_CORRECTION", enable_srgb_correction.into()), ("ENCODING_GAMMA", encoding_gamma.into()), ]); let staging_resolution = if let Some(foveated_encoding) = foveated_encoding { let (staging_resolution, ffe_constants) = foveated_encoding_shader_constants(base_view_resolution, foveated_encoding); constants.extend(ffe_constants); staging_resolution } else { base_view_resolution }; if let Some(upscaling) = upscaling { constants.extend([ ("ENABLE_UPSCALING", true.into()), ( "UPSCALE_USE_EDGE_DIRECTION", upscaling.edge_direction.into(), ), ( "UPSCALE_EDGE_THRESHOLD", (upscaling.edge_threshold / 255.0).into(), ), ("UPSCALE_EDGE_SHARPNESS", upscaling.edge_sharpness.into()), ]); }; let pipeline = device.create_render_pipeline(&RenderPipelineDescriptor { label: None, // Note: Layout cannot be inferred because of a bug with push constants layout: Some(&device.create_pipeline_layout(&PipelineLayoutDescriptor { label: None, bind_group_layouts: &[&bind_group_layout], push_constant_ranges: &[PushConstantRange { stages: ShaderStages::VERTEX_FRAGMENT, range: 0..PUSH_CONSTANTS_SIZE, }], })), vertex: VertexState { module: &shader_module, entry_point: None, compilation_options: PipelineCompilationOptions { constants: &constants, zero_initialize_workgroup_memory: false, }, buffers: &[], }, primitive: PrimitiveState { topology: PrimitiveTopology::TriangleStrip, ..Default::default() }, depth_stencil: None, multisample: Default::default(), fragment: Some(FragmentState { module: &shader_module, entry_point: None, compilation_options: PipelineCompilationOptions { constants: &constants, zero_initialize_workgroup_memory: false, }, targets: &[Some(ColorTargetState { format: target_format, blend: None, write_mask: ColorWrites::ALL, })], }), multiview: None, cache: None, }); let sampler = device.create_sampler(&SamplerDescriptor { mag_filter: wgpu::FilterMode::Linear, min_filter: wgpu::FilterMode::Linear, ..Default::default() }); let mut view_objects = vec![]; let mut staging_textures_gl = vec![]; for target_swapchain in &swapchain_textures { let staging_texture = super::create_texture(device, staging_resolution, target_format); let bind_group = device.create_bind_group(&BindGroupDescriptor { label: None, layout: &bind_group_layout, entries: &[ BindGroupEntry { binding: 0, resource: BindingResource::TextureView( &staging_texture.create_view(&TextureViewDescriptor::default()), ), }, BindGroupEntry { binding: 1, resource: BindingResource::Sampler(&sampler), }, ], }); let render_target = super::create_gl_swapchain( device, target_swapchain, target_view_resolution, target_format, ); view_objects.push(ViewObjects { bind_group, render_target, }); #[cfg(not(any(target_os = "macos", target_os = "ios")))] { let staging_texture_gl = unsafe { staging_texture.as_hal::(|tex| { let wgpu::hal::gles::TextureInner::Texture { raw, .. } = tex.unwrap().inner else { panic!("invalid texture type"); }; raw }) }; staging_textures_gl.push(staging_texture_gl); } } let staging_renderer = StagingRenderer::new( Rc::clone(&context), staging_textures_gl.try_into().unwrap(), staging_resolution, fix_limited_range, ); Self { context, staging_renderer, pipeline, views_objects: view_objects.try_into().unwrap(), } } /// # Safety /// `hardware_buffer` must be a valid pointer to a ANativeWindowBuffer. pub fn render( &self, hardware_buffer: *mut c_void, view_params: [StreamViewParams; 2], passthrough: Option<&PassthroughMode>, ) { // if hardware_buffer is available copy stream to staging texture if !hardware_buffer.is_null() { self.staging_renderer.render(hardware_buffer); } let mut encoder = self .context .device .create_command_encoder(&Default::default()); for (view_idx, view_params) in view_params.iter().enumerate() { let mut render_pass = encoder.begin_render_pass(&RenderPassDescriptor { label: None, color_attachments: &[Some(RenderPassColorAttachment { view: &self.views_objects[view_idx].render_target [view_params.swapchain_index as usize], resolve_target: None, ops: wgpu::Operations { load: LoadOp::Clear(Color::BLACK), store: StoreOp::Store, }, })], ..Default::default() }); let input_fov = view_params.input_view_params.fov; let tanl = f32::tan(input_fov.left); let tanr = f32::tan(input_fov.right); let tanu = f32::tan(input_fov.up); let tand = f32::tan(input_fov.down); let width = tanr - tanl; let height = tanu - tand; let quad_depth = 1000.0; let output_mat4 = Mat4::from_translation(view_params.output_view_params.pose.position) * Mat4::from_quat(view_params.output_view_params.pose.orientation); let input_mat4 = Mat4::from_translation(view_params.input_view_params.pose.position) * Mat4::from_quat(view_params.input_view_params.pose.orientation); // The image is at z = -1.0, so we use tangents for the size let model_mat = Mat4::from_translation(Vec3 { x: 0.0, y: 0.0, z: -quad_depth * 0.5, }) * Mat4::from_scale(Vec3::new(quad_depth, quad_depth, quad_depth * 0.5)) * Mat4::from_translation(Vec3::new( width / 2.0 + tanl, height / 2.0 + tand, -1.0, )) * Mat4::from_scale(Vec3::new(width, height, 1.)); let view_mat = output_mat4.inverse() * input_mat4; let proj_mat = super::projection_from_fov(view_params.output_view_params.fov); let transform = proj_mat * view_mat * model_mat; let transform_bytes = transform .to_cols_array() .iter() .flat_map(|v| v.to_le_bytes()) .collect::>(); render_pass.set_pipeline(&self.pipeline); render_pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, TRANSFORM_CONST_OFFSET, &transform_bytes, ); render_pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, VIEW_INDEX_CONST_OFFSET, &(view_idx as u32).to_le_bytes(), ); render_pass.set_bind_group(0, &self.views_objects[view_idx].bind_group, &[]); set_passthrough_push_constants(&mut render_pass, passthrough); render_pass.draw(0..4, 0..1); } self.context.queue.submit(iter::once(encoder.finish())); } } fn set_passthrough_push_constants(render_pass: &mut RenderPass, config: Option<&PassthroughMode>) { const DEG_TO_NORM: f32 = 1. / 360.; fn set_u32(render_pass: &mut RenderPass, offset: u32, value: u32) { render_pass.set_push_constants(ShaderStages::VERTEX_FRAGMENT, offset, &value.to_le_bytes()); } fn set_float(render_pass: &mut RenderPass, offset: u32, value: f32) { render_pass.set_push_constants(ShaderStages::VERTEX_FRAGMENT, offset, &value.to_le_bytes()); } fn set_vec4(render_pass: &mut RenderPass, offset: u32, value: Vec4) { render_pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, offset, &value.x.to_le_bytes(), ); render_pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, offset + FLOAT_SIZE, &value.y.to_le_bytes(), ); render_pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, offset + 2 * FLOAT_SIZE, &value.z.to_le_bytes(), ); render_pass.set_push_constants( ShaderStages::VERTEX_FRAGMENT, offset + 3 * FLOAT_SIZE, &value.w.to_le_bytes(), ); } match config { None => { set_u32(render_pass, PASSTHROUGH_MODE_OFFSET, 0); set_float(render_pass, ALPHA_CONST_OFFSET, 1.); } Some(PassthroughMode::Blend { threshold, .. }) => { set_u32(render_pass, PASSTHROUGH_MODE_OFFSET, 0); set_float(render_pass, ALPHA_CONST_OFFSET, 1. - threshold); } Some(PassthroughMode::RgbChromaKey(config)) => { set_u32(render_pass, PASSTHROUGH_MODE_OFFSET, 1); let norm = |v| v as f32 / 255.; let red = norm(config.red); let green = norm(config.green); let blue = norm(config.blue); let thresh = norm(config.distance_threshold); let up_feather = 1. + config.feathering; let down_feather = 1. - config.feathering; let range_vec = thresh * Vec4::new(-up_feather, -down_feather, down_feather, up_feather); set_vec4(render_pass, CK_CHANNEL0_CONST_OFFSET, red + range_vec); set_vec4(render_pass, CK_CHANNEL1_CONST_OFFSET, green + range_vec); set_vec4(render_pass, CK_CHANNEL2_CONST_OFFSET, blue + range_vec); } Some(PassthroughMode::HsvChromaKey(config)) => { set_u32(render_pass, PASSTHROUGH_MODE_OFFSET, 2); set_vec4( render_pass, CK_CHANNEL0_CONST_OFFSET, Vec4::new( config.hue_start_max_deg, config.hue_start_min_deg, config.hue_end_min_deg, config.hue_end_max_deg, ) * DEG_TO_NORM, ); set_vec4( render_pass, CK_CHANNEL1_CONST_OFFSET, Vec4::new( config.saturation_start_max, config.saturation_start_min, config.saturation_end_min, config.saturation_end_max, ), ); set_vec4( render_pass, CK_CHANNEL2_CONST_OFFSET, Vec4::new( config.value_start_max, config.value_start_min, config.value_end_min, config.value_end_max, ), ); } } } pub fn foveated_encoding_shader_constants( expanded_view_resolution: UVec2, config: FoveatedEncodingConfig, ) -> (UVec2, Vec<(&'static str, f64)>) { let view_resolution = expanded_view_resolution.as_vec2(); let center_size = glam::vec2(config.center_size_x, config.center_size_y); let center_shift = glam::vec2(config.center_shift_x, config.center_shift_y); let edge_ratio = glam::vec2(config.edge_ratio_x, config.edge_ratio_y); let edge_size = view_resolution - center_size * view_resolution; let center_size_aligned = 1. - (edge_size / (edge_ratio * 2.)).ceil() * (edge_ratio * 2.) / view_resolution; let edge_size_aligned = view_resolution - center_size_aligned * view_resolution; let center_shift_aligned = (center_shift * edge_size_aligned / (edge_ratio * 2.)).ceil() * (edge_ratio * 2.) / edge_size_aligned; let foveation_scale = center_size_aligned + (1. - center_size_aligned) / edge_ratio; let optimized_view_resolution = foveation_scale * view_resolution; let optimized_view_resolution_aligned = optimized_view_resolution.map(|v| (v / 32.).ceil() * 32.); let view_ratio_aligned = optimized_view_resolution / optimized_view_resolution_aligned; let c0 = (1. - center_size_aligned) * 0.5; let c1 = (edge_ratio - 1.) * c0 * (center_shift_aligned + 1.) / edge_ratio; let c2 = (edge_ratio - 1.) * center_size_aligned + 1.; let lo_bound = c0 * (center_shift_aligned + 1.); let hi_bound = c0 * (center_shift_aligned - 1.) + 1.; let lo_bound_c = c0 * (center_shift_aligned + 1.) / c2; let hi_bound_c = c0 * (center_shift_aligned - 1.) / c2 + 1.; let a_left = c2 * (1. - edge_ratio) / (edge_ratio * lo_bound_c); let b_left = (c1 + c2 * lo_bound_c) / lo_bound_c; let a_right = c2 * (edge_ratio - 1.) / (edge_ratio * (1. - hi_bound_c)); let b_right = (c2 - edge_ratio * c1 - 2. * edge_ratio * c2 + c2 * edge_ratio * (1. - hi_bound_c) + edge_ratio) / (edge_ratio * (1. - hi_bound_c)); let c_right = (c2 * edge_ratio - c2) * (c1 - hi_bound_c + c2 * hi_bound_c) / (edge_ratio * (1. - hi_bound_c) * (1. - hi_bound_c)); let constants = [ ("ENABLE_FFE", 1.), ("VIEW_WIDTH_RATIO", view_ratio_aligned.x), ("VIEW_HEIGHT_RATIO", view_ratio_aligned.y), ("EDGE_X_RATIO", edge_ratio.x), ("EDGE_Y_RATIO", edge_ratio.y), ("C1_X", c1.x), ("C1_Y", c1.y), ("C2_X", c2.x), ("C2_Y", c2.y), ("LO_BOUND_X", lo_bound.x), ("LO_BOUND_Y", lo_bound.y), ("HI_BOUND_X", hi_bound.x), ("HI_BOUND_Y", hi_bound.y), ("A_LEFT_X", a_left.x), ("A_LEFT_Y", a_left.y), ("B_LEFT_X", b_left.x), ("B_LEFT_Y", b_left.y), ("A_RIGHT_X", a_right.x), ("A_RIGHT_Y", a_right.y), ("B_RIGHT_X", b_right.x), ("B_RIGHT_Y", b_right.y), ("C_RIGHT_X", c_right.x), ("C_RIGHT_Y", c_right.y), ] .iter() .map(|(k, v)| (*k, *v as f64)) .collect(); (optimized_view_resolution_aligned.as_uvec2(), constants) } pub fn compute_target_view_resolution( resolution: UVec2, upscaling: &Option, ) -> UVec2 { let mut target_resolution = resolution.as_vec2(); if let Some(upscaling) = upscaling { target_resolution *= upscaling.upscale_factor; } target_resolution.as_uvec2() } ================================================ FILE: alvr/gui_common/Cargo.toml ================================================ [package] name = "alvr_gui_common" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true egui = "0.32" ================================================ FILE: alvr/gui_common/src/basic_components/button_group.rs ================================================ use crate::DisplayString; use egui::Ui; // todo: use a custom widget pub fn button_group_clicked( ui: &mut Ui, options: &[DisplayString], selection: &mut String, ) -> bool { let mut clicked = false; for id in options { let res = ui.selectable_value(selection, (**id).clone(), &id.display); if res.clicked() { clicked = true; } if cfg!(debug_assertions) { res.on_hover_text((**id).clone()); } } clicked } ================================================ FILE: alvr/gui_common/src/basic_components/mod.rs ================================================ mod button_group; mod modal; mod switch; pub use button_group::*; pub use modal::*; pub use switch::*; ================================================ FILE: alvr/gui_common/src/basic_components/modal.rs ================================================ use egui::{Align, Align2, Context, Layout, Ui, Window}; use std::fmt::{self, Display, Formatter}; #[derive(Clone, PartialEq)] pub enum ModalButton { Ok, Cancel, Close, Custom(String), } impl Display for ModalButton { fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result { match self { ModalButton::Ok => write!(f, "OK"), ModalButton::Cancel => write!(f, "Cancel"), ModalButton::Close => write!(f, "Close"), ModalButton::Custom(text) => write!(f, "{text}"), } } } pub fn modal( context: &Context, title: &str, content: Option, buttons: &[ModalButton], width: Option, ) -> Option { let mut response = None; let mut window = Window::new(title) .anchor(Align2::CENTER_CENTER, (0.0, 0.0)) .collapsible(false) .resizable(false); if let Some(w) = width { window = window.min_width(w).max_width(w); } window.show(context, |ui| { ui.vertical_centered_justified(|ui| { if let Some(content) = content { ui.add_space(10.0); content(ui); ui.add_space(10.0); } ui.columns(buttons.len(), |cols| { for (idx, response_type) in buttons.iter().enumerate() { cols[idx].with_layout(Layout::top_down_justified(Align::Center), |ui| { if ui.button(response_type.to_string()).clicked() { response = Some(response_type.clone()); } }); } }); }); }); response } ================================================ FILE: alvr/gui_common/src/basic_components/switch.rs ================================================ use crate::theme; use egui::{self, Response, Sense, StrokeKind, Ui, WidgetInfo, WidgetType}; pub fn switch(ui: &mut Ui, on: &mut bool) -> Response { let desired_size = theme::SWITCH_DOT_DIAMETER * egui::vec2(2.0, 1.0); let (rect, mut response) = ui.allocate_exact_size(desired_size, Sense::click()); if response.clicked() { *on = !*on; response.mark_changed(); } response.widget_info(|| WidgetInfo::selected(WidgetType::Checkbox, true, *on, "")); let how_on = ui.ctx().animate_bool(response.id, *on); let visuals = ui.style().interact_selectable(&response, *on); let rect = rect.expand(visuals.expansion); let radius = 0.5 * rect.height(); ui.painter().rect( rect, radius, visuals.bg_fill, visuals.bg_stroke, StrokeKind::Middle, ); let circle_x = egui::lerp((rect.left() + radius)..=(rect.right() - radius), how_on); let center = egui::pos2(circle_x, rect.center().y); ui.painter() .circle(center, 0.75 * radius, visuals.bg_fill, visuals.fg_stroke); response } ================================================ FILE: alvr/gui_common/src/lib.rs ================================================ mod basic_components; pub mod theme; pub use basic_components::*; use std::{ops::Deref, sync::atomic::AtomicUsize}; pub fn get_id() -> usize { static NEXT_ID: AtomicUsize = AtomicUsize::new(0); NEXT_ID.fetch_add(1, std::sync::atomic::Ordering::Relaxed) } #[derive(Clone)] pub struct DisplayString { pub id: String, pub display: String, } impl From<(String, String)> for DisplayString { fn from((id, display): (String, String)) -> Self { Self { id, display } } } impl Deref for DisplayString { type Target = String; fn deref(&self) -> &String { &self.id } } ================================================ FILE: alvr/gui_common/src/theme.rs ================================================ use egui::{self, Color32, Context, CornerRadius, Stroke, TextStyle, ThemePreference, Visuals}; pub const ACCENT: Color32 = Color32::from_rgb(0, 76, 176); pub const BG: Color32 = Color32::from_rgb(30, 30, 30); pub const LIGHTER_BG: Color32 = Color32::from_rgb(36, 36, 36); pub const SECTION_BG: Color32 = Color32::from_rgb(36, 36, 36); pub const DARKER_BG: Color32 = Color32::from_rgb(26, 26, 26); pub const SEPARATOR_BG: Color32 = Color32::from_rgb(69, 69, 69); pub const FG: Color32 = Color32::from_rgb(250, 250, 250); pub const SCROLLBAR_DOT_DIAMETER: f32 = 20.0; pub const SWITCH_DOT_DIAMETER: f32 = SCROLLBAR_DOT_DIAMETER; pub const FRAME_PADDING: f32 = 10.0; pub const CORNER_RADIUS: u8 = 10; pub const FRAME_TEXT_SPACING: f32 = 5.0; pub const OK_GREEN: Color32 = Color32::GREEN; pub const KO_RED: Color32 = Color32::RED; pub mod log_colors { use egui::epaint::Color32; pub const ERROR_LIGHT: Color32 = Color32::from_rgb(255, 50, 50); pub const WARNING_LIGHT: Color32 = Color32::from_rgb(205, 147, 9); pub const INFO_LIGHT: Color32 = Color32::from_rgb(134, 171, 241); pub const DEBUG_LIGHT: Color32 = Color32::LIGHT_GRAY; pub const EVENT_LIGHT: Color32 = Color32::GRAY; } // Graph colors pub mod graph_colors { use egui::Color32; // Colors taken from https://colorhunt.co/palette/ff6b6bffd93d6bcb774d96ff pub const RENDER_EXTERNAL: Color32 = Color32::from_rgb(64, 64, 64); pub const RENDER_EXTERNAL_LABEL: Color32 = Color32::GRAY; pub const RENDER: Color32 = Color32::RED; pub const IDLE: Color32 = Color32::from_rgb(255, 217, 61); pub const TRANSCODE: Color32 = Color32::from_rgb(107, 203, 119); pub const NETWORK: Color32 = Color32::from_rgb(77, 150, 255); pub const SERVER_FPS: Color32 = Color32::LIGHT_BLUE; pub const CLIENT_FPS: Color32 = Color32::KHAKI; pub const INITIAL_CALCULATED_THROUGHPUT: Color32 = Color32::GRAY; pub const ENCODER_DECODER_LATENCY_LIMITER: Color32 = TRANSCODE; pub const NETWORK_LATENCY_LIMITER: Color32 = NETWORK; pub const MIN_MAX_LATENCY_THROUGHPUT: Color32 = Color32::RED; pub const REQUESTED_BITRATE: Color32 = Color32::GREEN; pub const RECORDED_THROUGHPUT: Color32 = Color32::KHAKI; pub const RECORDED_BITRATE: Color32 = super::FG; } pub fn set_theme(ctx: &Context) { ctx.set_theme(ThemePreference::Dark); let mut style = (*ctx.style()).clone(); style.spacing.slider_width = 200_f32; // slider width can only be set globally style.spacing.interact_size.x = 35.0; style.spacing.interact_size.y = 35.0; style.spacing.item_spacing = egui::vec2(15.0, 15.0); style.spacing.button_padding = egui::vec2(10.0, 10.0); style.spacing.window_margin = egui::Margin::from(FRAME_PADDING); style.text_styles.get_mut(&TextStyle::Body).unwrap().size = 14.0; style.interaction.tooltip_delay = 0.0; ctx.set_style(style); let mut visuals = Visuals::dark(); let corner_radius = CornerRadius::same(CORNER_RADIUS); visuals.widgets.active.bg_fill = ACCENT; visuals.widgets.active.fg_stroke = Stroke::new(1.0, FG); visuals.widgets.active.corner_radius = corner_radius; visuals.widgets.inactive.fg_stroke = Stroke::new(1.0, FG); visuals.widgets.inactive.corner_radius = corner_radius; visuals.widgets.hovered.corner_radius = corner_radius; visuals.widgets.open.bg_fill = SEPARATOR_BG; visuals.widgets.open.corner_radius = corner_radius; visuals.selection.bg_fill = ACCENT; visuals.selection.stroke = Stroke::new(1.0, FG); visuals.faint_bg_color = DARKER_BG; visuals.widgets.noninteractive.bg_fill = BG; visuals.widgets.noninteractive.fg_stroke = Stroke::new(1.0, FG); visuals.widgets.noninteractive.bg_stroke = Stroke::new(0.5, SEPARATOR_BG); visuals.widgets.noninteractive.corner_radius = CornerRadius::same(CORNER_RADIUS + FRAME_PADDING as u8); // Frame corner radius ctx.set_visuals(visuals); } ================================================ FILE: alvr/launcher/Cargo.toml ================================================ [package] name = "alvr_launcher" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_adb.workspace = true alvr_common.workspace = true alvr_filesystem.workspace = true alvr_gui_common.workspace = true alvr_system_info.workspace = true anyhow = "1" eframe = "0.32" flate2 = "1.0.18" futures-util = "0.3.28" ico = "0.4" open = "5" reqwest = { version = "0.12", default-features = false, features = [ "rustls-tls", "stream", "json", "http2", ] } serde_json = "1" tar = "0.4" tokio = { version = "1", features = ["rt-multi-thread"] } zip = "4" [target.'cfg(windows)'.build-dependencies] winres = "0.1" ================================================ FILE: alvr/launcher/build.rs ================================================ #[cfg(windows)] fn main() { let mut resource = winres::WindowsResource::new(); resource.set_icon("../dashboard/resources/dashboard.ico"); resource.compile().unwrap(); } #[cfg(not(windows))] fn main() {} ================================================ FILE: alvr/launcher/src/actions.rs ================================================ use crate::{ InstallationInfo, Progress, ReleaseChannelsInfo, ReleaseInfo, UiMessage, WorkerMessage, }; use alvr_common::{ToAny, anyhow::Result, semver::Version}; use anyhow::{Context, bail}; use flate2::read::GzDecoder; use futures_util::StreamExt; use std::{ env, fs::{self, File}, io::{Cursor, Write}, path::PathBuf, process::Command, sync::mpsc::{Receiver, Sender}, }; const APK_NAME: &str = "client.apk"; pub fn installations_dir() -> PathBuf { data_dir().join("installations") } pub fn worker( ui_message_receiver: Receiver, worker_message_sender: Sender, ) { tokio::runtime::Runtime::new() .expect("Failed to create tokio runtime") .block_on(async { let req_client = reqwest::Client::builder() .user_agent("ALVR-Launcher") .build() .unwrap(); let version_data = match fetch_all_releases(&req_client).await { Ok(data) => data, Err(e) => { eprintln!("Error fetching version data: {e}"); return; } }; worker_message_sender .send(WorkerMessage::ReleaseChannelsInfo(version_data)) .unwrap(); loop { let Ok(message) = ui_message_receiver.recv() else { return; }; let res = match message { UiMessage::Quit => return, UiMessage::InstallServer { release_info, session_version, } => { install_server( &worker_message_sender, release_info, session_version, &req_client, ) .await } UiMessage::InstallClient(release_info) => { install_and_launch_apk(&worker_message_sender, release_info) } }; match res { Ok(()) => worker_message_sender.send(WorkerMessage::Done).unwrap(), Err(e) => worker_message_sender .send(WorkerMessage::Error(e.to_string())) .unwrap(), } } }); } async fn fetch_all_releases(client: &reqwest::Client) -> Result { Ok(ReleaseChannelsInfo { stable: fetch_releases_for_repo( client, "https://api.github.com/repos/alvr-org/ALVR/releases", ) .await?, nightly: fetch_releases_for_repo( client, "https://api.github.com/repos/alvr-org/ALVR-nightly/releases", ) .await?, }) } async fn fetch_releases_for_repo(client: &reqwest::Client, url: &str) -> Result> { let response: serde_json::Value = client.get(url).send().await?.json().await?; let mut releases = Vec::new(); for value in response.as_array().to_any()? { releases.push(ReleaseInfo { version: value["tag_name"].as_str().to_any()?.into(), assets: value["assets"] .as_array() .to_any()? .iter() .filter_map(|value| { Some(( value["name"].as_str()?.into(), value["browser_download_url"].as_str()?.into(), )) }) .collect(), }) } Ok(releases) } pub fn get_release( release_channels_info: &ReleaseChannelsInfo, version: &str, ) -> Option { release_channels_info .stable .iter() .find(|release| release.version == version) .cloned() .or_else(|| { release_channels_info .nightly .iter() .find(|release| release.version == version) .cloned() }) } fn install_and_launch_apk( worker_message_sender: &Sender, release: ReleaseInfo, ) -> Result<()> { worker_message_sender.send(WorkerMessage::ProgressUpdate(Progress { message: "Starting install".into(), progress: 0.0, }))?; let root = installations_dir().join(&release.version); let apk_name = "alvr_client_android.apk"; let apk_path = root.join(apk_name); if !apk_path.exists() { let apk_url = release .assets .get(apk_name) .ok_or(anyhow::anyhow!("Unable to determine download URL"))?; let apk_buffer = alvr_adb::commands::download(apk_url, |downloaded, total| { let progress = total.map_or(0.0, |t| downloaded as f32 / t as f32); worker_message_sender .send(WorkerMessage::ProgressUpdate(Progress { message: "Downloading Client APK".into(), progress, })) .ok(); })?; let mut file = File::create(&apk_path)?; file.write_all(&apk_buffer)?; } let layout = alvr_filesystem::Layout::new(&root); let adb_path = alvr_adb::commands::require_adb(&layout, |downloaded, total| { let progress = total.map_or(0.0, |t| downloaded as f32 / t as f32); worker_message_sender .send(WorkerMessage::ProgressUpdate(Progress { message: "Downloading ADB".into(), progress, })) .ok(); })?; let device_serial = alvr_adb::commands::list_devices(&adb_path)? .iter() .find_map(|d| d.serial.clone()) .ok_or(anyhow::anyhow!("Failed to find connected device"))?; let v = if release.version.starts_with('v') { release.version[1..].to_string() } else { release.version }; let version = Version::parse(&v).context("Failed to parse release version")?; let stable = version.pre.is_empty() && !version.build.contains("nightly"); let application_id = if stable { alvr_system_info::PACKAGE_NAME_GITHUB_STABLE } else { alvr_system_info::PACKAGE_NAME_GITHUB_DEV }; if alvr_adb::commands::is_package_installed(&adb_path, &device_serial, application_id)? { worker_message_sender.send(WorkerMessage::ProgressUpdate(Progress { message: "Uninstalling old APK".into(), progress: 0.0, }))?; alvr_adb::commands::uninstall_package(&adb_path, &device_serial, application_id)?; } worker_message_sender.send(WorkerMessage::ProgressUpdate(Progress { message: "Installing new APK".into(), progress: 0.0, }))?; alvr_adb::commands::install_package(&adb_path, &device_serial, &apk_path.to_string_lossy())?; alvr_adb::commands::start_application(&adb_path, &device_serial, application_id)?; Ok(()) } async fn download( worker_message_sender: &Sender, message: &str, url: &str, client: &reqwest::Client, ) -> Result> { let res = client.get(url).send().await?; let total_size = res.content_length(); let mut stream = res.bytes_stream(); let mut buffer = Vec::new(); while let Some(item) = stream.next().await { buffer.extend(item?); match total_size { Some(total_size) => { worker_message_sender.send(WorkerMessage::ProgressUpdate(Progress { message: message.into(), progress: buffer.len() as f32 / total_size as f32, }))? } None => worker_message_sender.send(WorkerMessage::ProgressUpdate(Progress { message: format!("{message} (Progress unavailable)"), progress: 0.5, }))?, } } Ok(buffer) } async fn install_server( worker_message_sender: &Sender, release_info: ReleaseInfo, session_version: Option, req_client: &reqwest::Client, ) -> Result<()> { worker_message_sender.send(WorkerMessage::ProgressUpdate(Progress { message: "Starting install".into(), progress: 0.0, }))?; let file_name = if cfg!(windows) { "alvr_streamer_windows.zip" } else { "alvr_streamer_linux.tar.gz" }; let url = release_info .assets .get(file_name) .ok_or(anyhow::anyhow!("Unable to determine download link"))?; let buffer = download( worker_message_sender, "Downloading Streamer", url, req_client, ) .await?; let installation_dir = installations_dir().join(&release_info.version); fs::create_dir_all(&installation_dir)?; let mut buffer = Cursor::new(buffer); if cfg!(windows) { zip::ZipArchive::new(&mut buffer)?.extract(&installation_dir)?; } else { tar::Archive::new(&mut GzDecoder::new(&mut buffer)).unpack(&installation_dir)?; } if let Some(session_version) = session_version { if !cfg!(windows) { unreachable!("The session copying code should only be hit on Windows!") } for inst in get_installations() { if inst.version == session_version { let source = alvr_filesystem::filesystem_layout_from_openvr_driver_root_dir( &installations_dir().join(session_version), ) .unwrap() .session(); let destination = alvr_filesystem::filesystem_layout_from_openvr_driver_root_dir( &installation_dir, ) .unwrap() .session(); fs::copy(source, destination)?; break; } } } Ok(()) } pub fn data_dir() -> PathBuf { if cfg!(target_os = "linux") { PathBuf::from(env::var("HOME").expect("Failed to determine home directory")) .join(".local/share/ALVR-Launcher") } else { env::current_exe() .expect("Unable to determine executable directory") .parent() .unwrap() .to_owned() } } pub fn get_installations() -> Vec { match fs::read_dir(installations_dir()) { Ok(entries) => entries .into_iter() .filter_map(|entry| { entry .ok() .filter(|entry| match entry.file_type() { Ok(file_type) => file_type.is_dir(), Err(e) => { eprintln!("Failed to read entry file type: {e}"); false } }) .map(|entry| { let has_session_json = if cfg!(windows) { alvr_filesystem::filesystem_layout_from_openvr_driver_root_dir( &entry.path(), ) .map(|layout| layout.session().exists()) .unwrap_or(false) } else { // On linux, the launcher does not need to manage the session files false }; InstallationInfo { version: entry.file_name().to_string_lossy().into(), is_apk_downloaded: entry.path().join(APK_NAME).exists(), has_session_json, } }) }) .collect(), Err(e) => { eprintln!("Failed to read versions dir: {e}"); Vec::new() } } } pub fn launch_dashboard(version: &str) -> Result<()> { let installation_dir = installations_dir().join(version); let dashboard_path = if cfg!(windows) { installation_dir.join("ALVR Dashboard.exe") } else if cfg!(target_os = "linux") { installation_dir.join("alvr_streamer_linux/bin/alvr_dashboard") } else { bail!("Unsupported platform") }; Command::new(dashboard_path).spawn()?; Ok(()) } pub fn delete_installation(version: &str) -> Result<()> { fs::remove_dir_all(installations_dir().join(version))?; Ok(()) } ================================================ FILE: alvr/launcher/src/main.rs ================================================ mod actions; mod ui; use eframe::egui::{IconData, ViewportBuilder}; use ico::IconDir; use std::{collections::BTreeMap, env, fs, io::Cursor, sync::mpsc, thread}; use ui::Launcher; pub struct ReleaseChannelsInfo { stable: Vec, nightly: Vec, } pub struct Progress { message: String, progress: f32, } pub enum WorkerMessage { ReleaseChannelsInfo(ReleaseChannelsInfo), ProgressUpdate(Progress), Done, Error(String), } #[derive(Clone)] pub struct ReleaseInfo { version: String, assets: BTreeMap, } pub enum UiMessage { InstallServer { release_info: ReleaseInfo, session_version: Option, }, InstallClient(ReleaseInfo), Quit, } pub struct InstallationInfo { version: String, is_apk_downloaded: bool, has_session_json: bool, // Only relevant on Windows } fn main() { let (worker_message_sender, worker_message_receiver) = mpsc::channel::(); let (ui_message_sender, ui_message_receiver) = mpsc::channel::(); let worker_handle = thread::spawn(|| actions::worker(ui_message_receiver, worker_message_sender)); let ico = IconDir::read(Cursor::new(include_bytes!( "../../dashboard/resources/dashboard.ico" ))) .unwrap(); let image = ico.entries().first().unwrap().decode().unwrap(); // Workaround for the steam deck if fs::read_to_string("/sys/devices/virtual/dmi/id/board_vendor") .map(|vendor| vendor.trim() == "Valve") .unwrap_or(false) { unsafe { env::set_var("WINIT_X11_SCALE_FACTOR", "1") }; } eframe::run_native( "ALVR Launcher", eframe::NativeOptions { viewport: ViewportBuilder::default() .with_app_id("alvr.launcher") .with_inner_size((700.0, 400.0)) .with_icon(IconData { rgba: image.rgba_data().to_owned(), width: image.width(), height: image.height(), }), ..Default::default() }, Box::new(move |cc| { Ok(Box::new(Launcher::new( cc, worker_message_receiver, ui_message_sender, ))) }), ) .expect("Failed to run eframe"); worker_handle.join().unwrap(); } ================================================ FILE: alvr/launcher/src/ui.rs ================================================ use crate::{InstallationInfo, Progress, ReleaseChannelsInfo, UiMessage, WorkerMessage, actions}; use alvr_gui_common::ModalButton; use eframe::{ egui::{ self, Button, CentralPanel, ComboBox, Context, Frame, Grid, Layout, ProgressBar, RichText, Ui, ViewportCommand, }, emath::Align, epaint::Color32, }; use std::{ mem, sync::mpsc::{Receiver, Sender}, }; enum State { Default, Installing(Progress), Error(String), } #[derive(Default)] enum PopupType { #[default] None, DeleteInstallation(String), EditVersion(String), AddVersion { version_selection: Version, session_version_selection: Option, }, } #[derive(Clone, PartialEq, Eq)] enum ReleaseChannelType { Stable, Nightly, } #[derive(Clone, PartialEq, Eq)] struct Version { string: String, release_channel: ReleaseChannelType, } pub struct Launcher { worker_message_receiver: Receiver, ui_message_sender: Sender, state: State, release_channels_info: Option, installations: Vec, popup: PopupType, } impl Launcher { pub fn new( cc: &eframe::CreationContext, worker_message_receiver: Receiver, ui_message_sender: Sender, ) -> Self { alvr_gui_common::theme::set_theme(&cc.egui_ctx); Self { worker_message_receiver, ui_message_sender, state: State::Default, release_channels_info: None, installations: actions::get_installations(), popup: PopupType::None, } } fn version_popup( &self, ctx: &Context, mut version: Version, mut session_version: Option, ) -> PopupType { let response = alvr_gui_common::modal( ctx, "Add version", { // Safety: unwrap is safe because the "Add release" button is available after populating the release_channels_info. let release_channels_info = self.release_channels_info.as_ref().unwrap(); Some(|ui: &mut Ui| { let version_str = version.string.clone(); let versions: Vec<_> = match &version.release_channel { ReleaseChannelType::Stable => release_channels_info .stable .iter() .map(|release| Version { string: release.version.clone(), release_channel: ReleaseChannelType::Stable, }) .collect(), ReleaseChannelType::Nightly => release_channels_info .nightly .iter() .map(|release| Version { string: release.version.clone(), release_channel: ReleaseChannelType::Nightly, }) .collect(), }; let installations_with_session: Vec<_> = self .installations .iter() .filter(|installation| installation.has_session_json) .map(|installation| installation.version.clone()) .collect(); Grid::new("add-version-grid").num_columns(2).show(ui, |ui| { ui.label("Channel"); ui.with_layout(Layout::right_to_left(Align::Min), |ui| { let channel_str = match version.release_channel { ReleaseChannelType::Stable => "Stable", ReleaseChannelType::Nightly => "Nightly", }; ComboBox::from_id_salt("channel") .selected_text(channel_str) .show_ui(ui, |ui| { ui.selectable_value( &mut version, Version { string: release_channels_info.stable[0].version.clone(), release_channel: ReleaseChannelType::Stable, }, "Stable", ); ui.selectable_value( &mut version, Version { string: release_channels_info.nightly[0] .version .clone(), release_channel: ReleaseChannelType::Nightly, }, "Nightly", ); }) }); ui.end_row(); ui.label("Version"); ui.with_layout(Layout::right_to_left(Align::Min), |ui| { ComboBox::from_id_salt("version") .selected_text(version_str) .show_ui(ui, |ui| { for ver in versions { ui.selectable_value(&mut version, ver.clone(), ver.string); } }) }); ui.end_row(); if cfg!(windows) { ui.label("Copy session from:"); ui.with_layout(Layout::right_to_left(Align::Min), |ui| { ComboBox::from_id_salt("session") .selected_text(session_version.clone().unwrap_or("None".into())) .show_ui(ui, |ui| { ui.selectable_value(&mut session_version, None, "None"); for ver_str in installations_with_session { ui.selectable_value( &mut session_version, Some(ver_str.clone()), ver_str, ); } }) }); ui.end_row(); } }); }) }, &[ModalButton::Cancel, ModalButton::Custom("Install".into())], None, ); match response { Some(ModalButton::Cancel) => PopupType::None, Some(ModalButton::Custom(_)) => { let release_info = match &version.release_channel { ReleaseChannelType::Stable => self .release_channels_info .as_ref() .unwrap() .stable .iter() .find(|release| release.version == version.string) .unwrap() .clone(), ReleaseChannelType::Nightly => self .release_channels_info .as_ref() .unwrap() .nightly .iter() .find(|release| release.version == version.string) .unwrap() .clone(), }; self.ui_message_sender .send(UiMessage::InstallServer { release_info, session_version, }) .ok(); PopupType::None } _ => PopupType::AddVersion { version_selection: version, session_version_selection: session_version, }, } } fn edit_popup(&self, ctx: &Context, version: String) -> PopupType { let mut delete_version = false; let response = alvr_gui_common::modal( ctx, "Edit version", Some(|ui: &mut Ui| { ui.with_layout(Layout::top_down_justified(Align::Center), |ui| { delete_version = ui.button("Delete version").clicked(); }); }), &[ModalButton::Close], None, ); if delete_version { PopupType::DeleteInstallation(version) } else if matches!(response, Some(ModalButton::Close)) { PopupType::None } else { PopupType::EditVersion(version) } } fn delete_popup(&mut self, ctx: &Context, version: String) -> PopupType { let response = alvr_gui_common::modal( ctx, "Are you sure?", Some({ let version = version.clone(); move |ui: &mut Ui| { ui.with_layout(Layout::top_down(Align::Center), |ui| { ui.label(format!("This will permanently delete version {version}")); }); } }), &[ ModalButton::Cancel, ModalButton::Custom("Delete version".into()), ], None, ); match response { Some(ModalButton::Cancel) => PopupType::None, Some(ModalButton::Custom(_)) => { if let Err(e) = actions::delete_installation(&version) { self.state = State::Error(format!("Failed to delete version: {e}")); } self.installations = actions::get_installations(); PopupType::None } _ => PopupType::DeleteInstallation(version), } } } impl eframe::App for Launcher { fn update(&mut self, ctx: &Context, _: &mut eframe::Frame) { while let Ok(msg) = self.worker_message_receiver.try_recv() { match msg { WorkerMessage::ReleaseChannelsInfo(data) => self.release_channels_info = Some(data), WorkerMessage::ProgressUpdate(progress) => { self.state = State::Installing(progress); } WorkerMessage::Done => { // Refresh installations self.installations = actions::get_installations(); self.state = State::Default; } WorkerMessage::Error(e) => self.state = State::Error(e), } } CentralPanel::default().show(ctx, |ui| match &self.state { State::Default => { ui.with_layout(Layout::top_down(Align::Center), |ui| { ui.label(RichText::new("ALVR Launcher").size(25.0).strong()); ui.label(match &self.release_channels_info { Some(data) => format!("Latest stable release: {}", data.stable[0].version), None => "Fetching latest release...".into(), }); for installation in &self.installations { let path = actions::installations_dir().join(&installation.version); Frame::group(ui.style()) .fill(alvr_gui_common::theme::SECTION_BG) .inner_margin(egui::vec2(10.0, 5.0)) .show(ui, |ui| { Grid::new(&installation.version) .num_columns(2) .show(ui, |ui| { ui.label(&installation.version); ui.with_layout(Layout::right_to_left(Align::Min), |ui| { if ui.button("Edit").clicked() { self.popup = PopupType::EditVersion( installation.version.clone(), ); } if ui.button("Open directory").clicked() { open::that_in_background(path); } let release_info = self .release_channels_info .as_ref() .and_then(|info| { actions::get_release( info, &installation.version, ) }); if ui .add_enabled( release_info.is_some() || installation.is_apk_downloaded, Button::new("Install APK"), ) .clicked() { if let Some(release_info) = release_info { self.ui_message_sender .send(UiMessage::InstallClient( release_info, )) .ok(); } else { self.state = State::Error( "Failed to get release info".into(), ); } }; if ui.button("Launch").clicked() { match actions::launch_dashboard( &installation.version, ) { Ok(()) => { self.ui_message_sender .send(UiMessage::Quit) .ok(); ctx.send_viewport_cmd( ViewportCommand::Close, ); } Err(e) => { self.state = State::Error(e.to_string()); } } } }) }) }); } if ui .add_enabled( self.release_channels_info.is_some(), Button::new("Add version"), ) .clicked() { self.popup = PopupType::AddVersion { version_selection: Version { string: self.release_channels_info.as_ref().unwrap().stable[0] .version .clone(), release_channel: ReleaseChannelType::Stable, }, session_version_selection: None, }; } let popup = match mem::take(&mut self.popup) { PopupType::AddVersion { version_selection, session_version_selection, } => self.version_popup(ctx, version_selection, session_version_selection), PopupType::EditVersion(version) => self.edit_popup(ctx, version), PopupType::DeleteInstallation(version) => self.delete_popup(ctx, version), PopupType::None => PopupType::None, }; self.popup = popup; }); } State::Installing(progress) => { ui.with_layout(Layout::top_down(Align::Center), |ui| { ui.label(&progress.message); ui.add(ProgressBar::new(progress.progress).animate(true)); }); } State::Error(e) => { let e = e.clone(); // Avoid borrowing issues with the closure for the layout ui.with_layout(Layout::top_down(Align::Center), |ui| { ui.colored_label(Color32::LIGHT_RED, "Error!"); ui.label(e); if ui.button("Close").clicked() { self.state = State::Default; } }); } }); if ctx.input(|i| i.viewport().close_requested()) { self.ui_message_sender.send(UiMessage::Quit).ok(); } } } ================================================ FILE: alvr/packets/Cargo.toml ================================================ [package] name = "alvr_packets" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_session.workspace = true serde = { version = "1", features = ["derive"] } serde_json = "1" ================================================ FILE: alvr/packets/src/lib.rs ================================================ use alvr_common::{ BodySkeleton, ConnectionState, DeviceMotion, LogSeverity, Pose, ViewParams, anyhow::Result, glam::{Quat, UVec2, Vec2}, semver::Version, }; use alvr_session::{ ClientsidePostProcessingConfig, CodecType, PassthroughMode, PerformanceLevel, SessionConfig, Settings, }; use serde::{Deserialize, Serialize}; use serde_json as json; use std::{ collections::HashSet, fmt::{self, Debug}, net::IpAddr, time::Duration, }; pub const TRACKING: u16 = 0; pub const HAPTICS: u16 = 1; pub const AUDIO: u16 = 2; pub const VIDEO: u16 = 3; pub const STATISTICS: u16 = 4; #[derive(Serialize, Deserialize, Clone)] pub struct VideoStreamingCapabilitiesExt { // Nothing for now } #[derive(Serialize, Deserialize, Clone)] pub struct VideoStreamingCapabilities { pub default_view_resolution: UVec2, pub max_view_resolution: UVec2, pub refresh_rates: Vec, pub microphone_sample_rate: u32, pub foveated_encoding: bool, pub encoder_high_profile: bool, pub encoder_10_bits: bool, pub encoder_av1: bool, pub prefer_10bit: bool, pub preferred_encoding_gamma: f32, pub prefer_hdr: bool, pub ext_str: String, } impl VideoStreamingCapabilities { pub fn with_ext(self, ext: VideoStreamingCapabilitiesExt) -> Self { Self { ext_str: json::to_string(&ext).unwrap(), ..self } } pub fn ext(&self) -> Result { let _ext_json = json::from_str::(&self.ext_str)?; // decode values here Ok(VideoStreamingCapabilitiesExt {}) } } #[derive(Serialize, Deserialize)] pub struct ConnectionAcceptedInfo { pub client_protocol_id: u64, pub platform_string: String, pub server_ip: IpAddr, pub streaming_capabilities: Option, } #[derive(Serialize, Deserialize)] pub enum ClientConnectionResult { ConnectionAccepted(Box), ClientStandby, } #[derive(Serialize, Deserialize)] pub struct NegotiatedStreamingConfigExt { // Nothing for now } #[derive(Serialize, Deserialize, Clone)] pub struct NegotiatedStreamingConfig { pub view_resolution: UVec2, pub refresh_rate_hint: f32, pub game_audio_sample_rate: u32, pub enable_foveated_encoding: bool, pub encoding_gamma: f32, pub enable_hdr: bool, pub wired: bool, pub ext_str: String, } impl NegotiatedStreamingConfig { pub fn with_ext(self, ext: NegotiatedStreamingConfigExt) -> Self { Self { ext_str: json::to_string(&ext).unwrap(), ..self } } pub fn ext(&self) -> Result { let _ext_json = json::from_str::(&self.ext_str)?; // decode values here Ok(NegotiatedStreamingConfigExt {}) } } #[derive(Serialize, Deserialize)] pub struct StreamConfigPacket { pub session: String, // JSON session that allows for extrapolation pub negotiated: NegotiatedStreamingConfig, } #[derive(Serialize, Deserialize, Clone)] pub struct StreamConfig { pub server_version: Version, pub settings: Settings, pub negotiated_config: NegotiatedStreamingConfig, } impl StreamConfigPacket { pub fn new(session: &SessionConfig, negotiated: NegotiatedStreamingConfig) -> Result { Ok(Self { session: json::to_string(session)?, negotiated, }) } pub fn to_stream_config(self) -> Result { let mut session_config = SessionConfig::default(); session_config.merge_from_json(&json::from_str(&self.session)?)?; let settings = session_config.to_settings(); Ok(StreamConfig { server_version: session_config.server_version, settings, negotiated_config: self.negotiated, }) } } #[derive(Serialize, Deserialize, Clone)] pub struct DecoderInitializationConfig { pub codec: CodecType, pub config_buffer: Vec, // e.g. SPS + PPS NALs pub ext_str: String, } #[derive(Serialize, Deserialize)] pub enum ServerControlPacket { StartStream, DecoderConfig(DecoderInitializationConfig), Restarting, KeepAlive, RealTimeConfig(RealTimeConfig), Reserved(String), ReservedBuffer(Vec), } #[derive(Serialize, Deserialize, Clone)] pub struct BatteryInfo { pub device_id: u64, pub gauge_value: f32, // range [0, 1] pub is_plugged: bool, } #[derive(Serialize, Deserialize, Clone, Copy, Debug)] pub enum ButtonValue { Binary(bool), Scalar(f32), } #[derive(Serialize, Deserialize)] pub struct ButtonEntry { pub path_id: u64, pub value: ButtonValue, } #[derive(Serialize, Deserialize)] pub enum ClientControlPacket { PlayspaceSync(Option), RequestIdr, KeepAlive, StreamReady, // This flag notifies the server the client streaming socket is ready listening LocalViewParams([ViewParams; 2]), // In relation to head Battery(BatteryInfo), Buttons(Vec), ActiveInteractionProfile { device_id: u64, profile_id: u64, input_ids: HashSet, }, Log { level: LogSeverity, message: String, }, ProximityState(bool), Reserved(String), ReservedBuffer(Vec), } #[derive(Serialize, Deserialize, Clone, Debug)] pub enum FaceExpressions { Fb(Vec), // 70 values Pico(Vec), // 52 values Htc { eye: Option>, // 14 values lip: Option>, // 37 values }, } #[derive(Serialize, Deserialize, Clone, Default, Debug)] pub struct FaceData { // Can be used for foveated eye tracking pub eyes_combined: Option, // Should be used only for social presence pub eyes_social: [Option; 2], pub face_expressions: Option, } #[derive(Serialize, Deserialize)] pub struct TrackingData { pub poll_timestamp: Duration, pub device_motions: Vec<(u64, DeviceMotion)>, pub hand_skeletons: [Option<[Pose; 26]>; 2], pub face: FaceData, pub body: Option, } #[derive(Serialize, Deserialize)] pub struct VideoPacketHeader { pub timestamp: Duration, pub global_view_params: [ViewParams; 2], pub is_idr: bool, } #[derive(Serialize, Deserialize)] pub struct Haptics { pub device_id: u64, pub duration: Duration, pub frequency: f32, pub amplitude: f32, } #[derive(Serialize, Deserialize, Clone)] pub enum PathSegment { Name(String), Index(usize), } impl Debug for PathSegment { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { PathSegment::Name(name) => write!(f, "{name}"), PathSegment::Index(index) => write!(f, "[{index}]"), } } } impl From<&str> for PathSegment { fn from(value: &str) -> Self { PathSegment::Name(value.to_owned()) } } impl From for PathSegment { fn from(value: String) -> Self { PathSegment::Name(value) } } impl From for PathSegment { fn from(value: usize) -> Self { PathSegment::Index(value) } } // todo: support indices pub fn parse_path(path: &str) -> Vec { path.split('.').map(|s| s.into()).collect() } #[derive(Serialize, Deserialize, Clone, Debug)] pub enum ClientConnectionsAction { AddIfMissing { trusted: bool, manual_ips: Vec, }, SetDisplayName(String), Trust, SetManualIps(Vec), RemoveEntry, UpdateCurrentIp(Option), SetConnectionState(ConnectionState), } #[derive(Serialize, Deserialize, Default, Clone)] pub struct ClientStatistics { pub target_timestamp: Duration, // identifies the frame pub frame_interval: Duration, pub video_decode: Duration, pub video_decoder_queue: Duration, pub rendering: Duration, pub vsync_queue: Duration, pub total_pipeline_latency: Duration, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct PathValuePair { pub path: Vec, pub value: json::Value, } #[derive(Serialize, Deserialize, Debug)] pub enum FirewallRulesAction { Add, Remove, } // Note: server sends a packet to the client at low frequency, binary encoding, without ensuring // compatibility between different versions, even if within the same major version. #[derive(Serialize, Deserialize, PartialEq, Clone)] pub struct RealTimeConfig { pub passthrough: Option, pub clientside_post_processing: Option, pub cpu_performance_level: Option, pub gpu_performance_level: Option, pub ext_str: String, } impl RealTimeConfig { pub fn from_settings(settings: &Settings) -> Self { Self { passthrough: settings.video.passthrough.clone().into_option(), clientside_post_processing: settings .video .clientside_post_processing .clone() .into_option(), cpu_performance_level: settings.headset.performance_level.clone().cpu.into_option(), gpu_performance_level: settings.headset.performance_level.clone().gpu.into_option(), ext_str: String::new(), // No extensions for now } } } ================================================ FILE: alvr/server_core/Cargo.toml ================================================ [package] name = "alvr_server_core" version.workspace = true edition.workspace = true rust-version.workspace = true authors = ["alvr-org", "Valve Corporation"] license = "MIT" [lib] crate-type = ["rlib", "cdylib"] [features] trace-performance = ["profiling/profile-with-tracy"] [dependencies] alvr_adb.workspace = true alvr_audio.workspace = true alvr_common.workspace = true alvr_events.workspace = true alvr_filesystem.workspace = true alvr_packets.workspace = true alvr_server_io.workspace = true alvr_session.workspace = true alvr_sockets.workspace = true ash = "0.38" axum = { version = "0.8", features = ["ws"] } chrono = "0.4" fern = "0.7" flume = "0.11" mdns-sd = "0.14" profiling = { version = "1", optional = true } rfd = "0.15" rosc = "0.11" tokio = { version = "1", features = [ "rt-multi-thread", "macros", "process", "io-util", "net", "fs", ] } tower-http = { version = "0.6", features = ["cors", "set-header"] } serde = "1" serde_json = "1" sysinfo = "0.37" ================================================ FILE: alvr/server_core/cbindgen.toml ================================================ language = "C" header = "/* ALVR is licensed under the MIT license. https://github.com/alvr-org/ALVR/blob/master/LICENSE */" pragma_once = true autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */" cpp_compat = true tab_width = 4 documentation_style = "c99" [enum] rename_variants = "QualifiedScreamingSnakeCase" [parse] parse_deps = true include = ["alvr_common"] ================================================ FILE: alvr/server_core/src/bitrate.rs ================================================ use alvr_common::SlidingWindowAverage; use alvr_events::BitrateDirectives; use alvr_session::{ BitrateAdaptiveFramerateConfig, BitrateConfig, BitrateMode, settings_schema::Switch, }; use std::{ collections::VecDeque, time::{Duration, Instant}, }; const UPDATE_INTERVAL: Duration = Duration::from_secs(1); pub struct DynamicEncoderParams { pub bitrate_bps: f32, pub framerate: f32, } pub struct BitrateManager { nominal_frame_interval: Duration, frame_interval_average: SlidingWindowAverage, // note: why packet_sizes_bits_history is a queue and not a sliding average? Because some // network samples will be dropped but not any packet size sample packet_bytes_history: VecDeque<(Duration, usize)>, packet_bytes_average: SlidingWindowAverage, network_latency_average: SlidingWindowAverage, encoder_latency_average: SlidingWindowAverage, decoder_latency_overstep_count: usize, last_frame_instant: Instant, last_update_instant: Instant, dynamic_decoder_max_bytes_per_frame: f32, previous_config: Option, update_needed: bool, } impl BitrateManager { pub fn new(max_history_size: usize, initial_framerate: f32) -> Self { Self { nominal_frame_interval: Duration::from_secs_f32(1. / initial_framerate), frame_interval_average: SlidingWindowAverage::new( Duration::from_millis(16), max_history_size, ), packet_bytes_history: VecDeque::new(), packet_bytes_average: SlidingWindowAverage::new(50000.0, max_history_size), network_latency_average: SlidingWindowAverage::new( Duration::from_millis(5), max_history_size, ), encoder_latency_average: SlidingWindowAverage::new( Duration::from_millis(5), max_history_size, ), decoder_latency_overstep_count: 0, last_frame_instant: Instant::now(), last_update_instant: Instant::now(), dynamic_decoder_max_bytes_per_frame: f32::MAX, previous_config: None, update_needed: true, } } // Note: This is used to calculate the framerate/frame interval. The frame present is the most // accurate event for this use. pub fn report_frame_present(&mut self, config: &Switch) { let now = Instant::now(); let interval = now - self.last_frame_instant; self.last_frame_instant = now; if let Some(config) = config.as_option() { let interval_ratio = interval.as_secs_f32() / self.frame_interval_average.get_average().as_secs_f32(); self.frame_interval_average.submit_sample(interval); if interval_ratio > config.framerate_reset_threshold_multiplier || interval_ratio < 1.0 / config.framerate_reset_threshold_multiplier { // Clear most of the samples, keep some for stability self.frame_interval_average.retain(5); self.update_needed = true; } } } pub fn report_frame_encoded( &mut self, timestamp: Duration, encoder_latency: Duration, size_bytes: usize, ) { self.encoder_latency_average.submit_sample(encoder_latency); self.packet_bytes_history.push_back((timestamp, size_bytes)); } // decoder_latency is used to learn a suitable maximum bitrate bound to avoid decoder runaway // latency pub fn report_frame_latencies( &mut self, config: &BitrateMode, timestamp: Duration, network_latency: Duration, decoder_latency: Duration, ) { if network_latency.is_zero() { return; } while let Some(&(history_timestamp, size_bytes)) = self.packet_bytes_history.front() { if history_timestamp == timestamp { self.packet_bytes_average.submit_sample(size_bytes as f32); self.network_latency_average.submit_sample(network_latency); self.packet_bytes_history.pop_front(); break; } else { self.packet_bytes_history.pop_front(); } } if let BitrateMode::Adaptive { decoder_latency_limiter: Switch::Enabled(config), .. } = &config { if decoder_latency > Duration::from_millis(config.max_decoder_latency_ms) { self.decoder_latency_overstep_count += 1; if self.decoder_latency_overstep_count == config.latency_overstep_frames { self.dynamic_decoder_max_bytes_per_frame = f32::min( self.packet_bytes_average.get_average(), self.dynamic_decoder_max_bytes_per_frame, ) * config .latency_overstep_multiplier; self.update_needed = true; self.decoder_latency_overstep_count = 0; } } else { self.decoder_latency_overstep_count = 0; } } } pub fn get_encoder_params( &mut self, config: &BitrateConfig, ) -> Option<(DynamicEncoderParams, BitrateDirectives)> { let now = Instant::now(); if self.previous_config.as_ref() != Some(config) { self.previous_config = Some(config.clone()); // Continue method. Always update bitrate in this case } else if !self.update_needed && (now < self.last_update_instant + UPDATE_INTERVAL || matches!(config.mode, BitrateMode::ConstantMbps(_))) { return None; } self.last_update_instant = now; self.update_needed = false; let frame_interval = if config.adapt_to_framerate.enabled() { self.frame_interval_average.get_average() } else { self.nominal_frame_interval }; let mut bitrate_directives = BitrateDirectives::default(); let bitrate_bps = match &config.mode { BitrateMode::ConstantMbps(bitrate_mbps) => *bitrate_mbps as f32 * 1e6, BitrateMode::Adaptive { saturation_multiplier, max_throughput_mbps, min_throughput_mbps, max_network_latency_ms, encoder_latency_limiter, decoder_latency_limiter, } => { let packet_bytes_average = self.packet_bytes_average.get_average(); let network_latency_average_s = self.network_latency_average.get_average().as_secs_f32(); let mut throughput_bps = packet_bytes_average * 8.0 * saturation_multiplier / network_latency_average_s; bitrate_directives.scaled_calculated_throughput_bps = Some(throughput_bps); if decoder_latency_limiter.enabled() { throughput_bps = f32::min(throughput_bps, self.dynamic_decoder_max_bytes_per_frame); bitrate_directives.decoder_latency_limiter_bps = Some(self.dynamic_decoder_max_bytes_per_frame); } if let Switch::Enabled(max_ms) = max_network_latency_ms { let max_bps = throughput_bps * (*max_ms as f32 / 1000.0) / network_latency_average_s; throughput_bps = f32::min(throughput_bps, max_bps); bitrate_directives.network_latency_limiter_bps = Some(max_bps); } if let Switch::Enabled(config) = encoder_latency_limiter { // Note: this assumes linear relationship between bitrate and encoder latency // but this may not be the case let saturation = self.encoder_latency_average.get_average().as_secs_f32() / self.nominal_frame_interval.as_secs_f32(); let max_bps = throughput_bps * config.max_saturation_multiplier / saturation; bitrate_directives.encoder_latency_limiter_bps = Some(max_bps); if saturation > config.max_saturation_multiplier { throughput_bps = f32::min(throughput_bps, max_bps); } } if let Switch::Enabled(max) = max_throughput_mbps { let max_bps = *max as f32 * 1e6; throughput_bps = f32::min(throughput_bps, max_bps); bitrate_directives.manual_max_throughput_bps = Some(max_bps); } if let Switch::Enabled(min) = min_throughput_mbps { let min_bps = *min as f32 * 1e6; throughput_bps = f32::max(throughput_bps, min_bps); bitrate_directives.manual_min_throughput_bps = Some(min_bps); } // NB: Here we assign the calculated throughput to the requested bitrate. This is // crucial for the working of the adaptive bitrate algorithm. The goal is to // optimally occupy the available bandwidth, which is when the bitrate corresponds // to the throughput. throughput_bps } }; bitrate_directives.requested_bitrate_bps = bitrate_bps; Some(( DynamicEncoderParams { bitrate_bps, framerate: 1.0 / f32::min(frame_interval.as_secs_f32(), 1.0), }, bitrate_directives, )) } } ================================================ FILE: alvr/server_core/src/c_api.rs ================================================ #![allow(dead_code, unused_variables)] #![allow(clippy::missing_safety_doc)] use crate::{ SESSION_MANAGER, ServerCoreContext, ServerCoreEvent, logging_backend, tracking::HandType, }; use alvr_common::{ AlvrCodecType, AlvrPose, AlvrViewParams, log, parking_lot::{Mutex, RwLock}, }; use alvr_packets::{ButtonEntry, ButtonValue, Haptics}; use alvr_session::CodecType; use std::{ collections::{HashMap, VecDeque}, ffi::{CStr, CString, c_char}, path::PathBuf, ptr, str::FromStr, sync::{LazyLock, mpsc}, time::{Duration, Instant}, }; static SERVER_CORE_CONTEXT: RwLock> = RwLock::new(None); static EVENTS_RECEIVER: Mutex>> = Mutex::new(None); static BUTTONS_QUEUE: Mutex>> = Mutex::new(VecDeque::new()); #[repr(C)] pub struct AlvrDeviceMotion { pub pose: AlvrPose, pub linear_velocity: [f32; 3], pub angular_velocity: [f32; 3], } #[repr(u8)] pub enum AlvrHandType { Left = 0, Right = 1, } #[repr(C)] pub union AlvrButtonValue { pub scalar: bool, pub float: f32, } // the profile is implied #[repr(C)] pub struct AlvrButtonEntry { pub id: u64, pub value: AlvrButtonValue, } #[repr(C)] pub struct AlvrBatteryInfo { pub device_id: u64, /// range [0, 1] pub gauge_value: f32, pub is_plugged: bool, } #[repr(u8)] pub enum AlvrEvent { ClientConnected, ClientDisconnected, Battery(AlvrBatteryInfo), PlayspaceSync([f32; 2]), LocalViewParams([AlvrViewParams; 2]), // In relation to head TrackingUpdated { sample_timestamp_ns: u64 }, ButtonsUpdated, RequestIDR, CaptureFrame, RestartPending, ShutdownPending, ProximityState(bool), } #[repr(C)] pub struct AlvrTargetConfig { game_render_width: u32, game_render_height: u32, stream_width: u32, stream_height: u32, } #[repr(C)] pub struct AlvrDeviceConfig { device_id: u64, interaction_profile_id: u64, } #[repr(C)] pub struct AlvrDynamicEncoderParams { bitrate_bps: f32, framerate: f32, } fn string_to_c_str(buffer: *mut c_char, value: &str) -> u64 { let cstring = CString::new(value).unwrap(); if !buffer.is_null() { unsafe { ptr::copy_nonoverlapping(cstring.as_ptr(), buffer, cstring.as_bytes_with_nul().len()); } } cstring.as_bytes_with_nul().len() as u64 } // Get ALVR server time. The libalvr user should provide timestamps in the provided time frame of // reference in the following functions #[unsafe(no_mangle)] pub extern "C" fn alvr_get_time_ns() -> u64 { Instant::now().elapsed().as_nanos() as u64 } // The libalvr user is responsible of interpreting values and calling functions using // device/input/output identifiers obtained using this function #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_path_to_id(path_string: *const c_char) -> u64 { alvr_common::hash_string(unsafe { CStr::from_ptr(path_string) }.to_str().unwrap()) } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_error(string_ptr: *const c_char) { alvr_common::show_e(unsafe { CStr::from_ptr(string_ptr) }.to_string_lossy()); } pub unsafe fn log(level: log::Level, string_ptr: *const c_char) { log::log!( level, "{}", unsafe { CStr::from_ptr(string_ptr) }.to_string_lossy() ); } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_warn(string_ptr: *const c_char) { unsafe { log(log::Level::Warn, string_ptr) }; } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_info(string_ptr: *const c_char) { unsafe { log(log::Level::Info, string_ptr) }; } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_dbg_server_impl(string_ptr: *const c_char) { alvr_common::dbg_server_impl!( "{}", unsafe { CStr::from_ptr(string_ptr) }.to_string_lossy() ); } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_dbg_encoder(string_ptr: *const c_char) { alvr_common::dbg_encoder!( "{}", unsafe { CStr::from_ptr(string_ptr) }.to_string_lossy() ); } // Should not be used in production #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_log_periodically(tag_ptr: *const c_char, message_ptr: *const c_char) { const INTERVAL: Duration = Duration::from_secs(1); static LASTEST_TAG_TIMESTAMPS: LazyLock>> = LazyLock::new(|| Mutex::new(HashMap::new())); let tag = unsafe { CStr::from_ptr(tag_ptr) }.to_string_lossy(); let message = unsafe { CStr::from_ptr(message_ptr) }.to_string_lossy(); let mut timestamps_ref = LASTEST_TAG_TIMESTAMPS.lock(); let old_timestamp = timestamps_ref .entry(tag.to_string()) .or_insert_with(Instant::now); if *old_timestamp + INTERVAL < Instant::now() { *old_timestamp += INTERVAL; log::warn!("{}: {}", tag, message); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_get_settings_json(buffer: *mut c_char) -> u64 { string_to_c_str(buffer, &serde_json::to_string(&crate::settings()).unwrap()) } /// This must be called before alvr_initialize() #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_initialize_environment( config_dir: *const c_char, log_dir: *const c_char, ) { let config_dir = PathBuf::from_str(unsafe { CStr::from_ptr(config_dir) }.to_str().unwrap()).unwrap(); let log_dir = PathBuf::from_str(unsafe { CStr::from_ptr(log_dir) }.to_str().unwrap()).unwrap(); crate::initialize_environment(alvr_filesystem::Layout { config_dir, log_dir, ..Default::default() }); } /// Either session_log_path or crash_log_path can be null, in which case log is outputted to /// stdout/stderr on Windows. #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_initialize_logging( session_log_path: *const c_char, crash_log_path: *const c_char, ) { let session_log_path = (!session_log_path.is_null()).then(|| { PathBuf::from_str( unsafe { CStr::from_ptr(session_log_path) } .to_str() .unwrap(), ) .unwrap() }); let crash_log_path = (!crash_log_path.is_null()).then(|| { PathBuf::from_str(unsafe { CStr::from_ptr(crash_log_path) }.to_str().unwrap()).unwrap() }); logging_backend::init_logging(session_log_path, crash_log_path); } #[unsafe(no_mangle)] pub extern "C" fn alvr_initialize() -> AlvrTargetConfig { let (context, receiver) = ServerCoreContext::new(); *SERVER_CORE_CONTEXT.write() = Some(context); *EVENTS_RECEIVER.lock() = Some(receiver); let session_manager_lock = SESSION_MANAGER.read(); let restart_settings = &session_manager_lock.session().openvr_config; AlvrTargetConfig { game_render_width: restart_settings.target_eye_resolution_width, game_render_height: restart_settings.target_eye_resolution_height, stream_width: restart_settings.eye_resolution_width, stream_height: restart_settings.eye_resolution_height, } } #[unsafe(no_mangle)] pub extern "C" fn alvr_start_connection() { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.start_connection(); } } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_poll_event(out_event: *mut AlvrEvent, timeout_ns: u64) -> bool { if let Some(receiver) = &*EVENTS_RECEIVER.lock() && let Ok(event) = receiver.recv_timeout(Duration::from_nanos(timeout_ns)) { match event { ServerCoreEvent::ClientConnected => unsafe { *out_event = AlvrEvent::ClientConnected; }, ServerCoreEvent::ClientDisconnected => unsafe { *out_event = AlvrEvent::ClientDisconnected; }, ServerCoreEvent::Battery(battery) => unsafe { *out_event = AlvrEvent::Battery(AlvrBatteryInfo { device_id: battery.device_id, gauge_value: battery.gauge_value, is_plugged: battery.is_plugged, }); }, ServerCoreEvent::PlayspaceSync(bounds) => unsafe { *out_event = AlvrEvent::PlayspaceSync(bounds.to_array()) }, ServerCoreEvent::LocalViewParams(config) => unsafe { *out_event = AlvrEvent::LocalViewParams([ alvr_common::to_capi_view_params(&config[0]), alvr_common::to_capi_view_params(&config[1]), ]) }, ServerCoreEvent::Tracking { poll_timestamp } => unsafe { *out_event = AlvrEvent::TrackingUpdated { sample_timestamp_ns: poll_timestamp.as_nanos() as u64, }; }, ServerCoreEvent::Buttons(entries) => { BUTTONS_QUEUE.lock().push_back(entries); unsafe { *out_event = AlvrEvent::ButtonsUpdated }; } ServerCoreEvent::RequestIDR => unsafe { *out_event = AlvrEvent::RequestIDR }, ServerCoreEvent::CaptureFrame => unsafe { *out_event = AlvrEvent::CaptureFrame }, ServerCoreEvent::RestartPending => unsafe { *out_event = AlvrEvent::RestartPending; }, ServerCoreEvent::ShutdownPending => unsafe { *out_event = AlvrEvent::ShutdownPending; }, ServerCoreEvent::GameRenderLatencyFeedback(_) | ServerCoreEvent::SetOpenvrProperty { .. } => {} // implementation not needed ServerCoreEvent::ProximityState(headset_is_worn) => unsafe { *out_event = AlvrEvent::ProximityState(headset_is_worn); }, } true } else { false } } /// Returns false if there is no tracking sample for the requested sample timestamp #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_get_device_motion( device_id: u64, sample_timestamp_ns: u64, out_motion: *mut AlvrDeviceMotion, ) -> bool { if let Some(context) = &*SERVER_CORE_CONTEXT.read() && let Some(motion) = context.get_device_motion(device_id, Duration::from_nanos(sample_timestamp_ns)) { unsafe { *out_motion = AlvrDeviceMotion { pose: alvr_common::to_capi_pose(&motion.pose), linear_velocity: motion.linear_velocity.to_array(), angular_velocity: motion.angular_velocity.to_array(), }; } true } else { false } } /// out_skeleton must be an array of length 26 /// Returns false if there is no tracking sample for the requested sample timestamp #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_get_hand_skeleton( hand_type: AlvrHandType, sample_timestamp_ns: u64, out_skeleton: *mut AlvrPose, ) -> bool { if let Some(context) = &*SERVER_CORE_CONTEXT.read() && let Some(skeleton) = context.get_hand_skeleton( match hand_type { AlvrHandType::Left => HandType::Left, AlvrHandType::Right => HandType::Right, }, Duration::from_nanos(sample_timestamp_ns), ) { for (i, joint_pose) in skeleton.iter().enumerate() { unsafe { *out_skeleton.add(i) = alvr_common::to_capi_pose(joint_pose) }; } true } else { false } } /// Call with null out_entries to get the buffer length /// call with non-null out_entries to get the buttons and advanced the internal queue #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_get_buttons(out_entries: *mut AlvrButtonEntry) -> u64 { let entries_count = BUTTONS_QUEUE.lock().front().map_or(0, |e| e.len()) as u64; if out_entries.is_null() { return entries_count; } if let Some(button_entries) = BUTTONS_QUEUE.lock().pop_front() { for (i, entry) in button_entries.into_iter().enumerate() { let out_entry = unsafe { &mut *out_entries.add(i) }; out_entry.id = entry.path_id; match entry.value { ButtonValue::Binary(value) => out_entry.value.scalar = value, ButtonValue::Scalar(value) => out_entry.value.float = value, } } entries_count } else { 0 } } #[unsafe(no_mangle)] pub extern "C" fn alvr_send_haptics( device_id: u64, duration_s: f32, frequency: f32, amplitude: f32, ) { if let Ok(duration) = Duration::try_from_secs_f32(duration_s) && let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.send_haptics(Haptics { device_id, duration, frequency, amplitude, }); } } #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_set_video_config_nals( codec: AlvrCodecType, buffer_ptr: *const u8, len: i32, ) { let codec = match codec { AlvrCodecType::H264 => CodecType::H264, AlvrCodecType::Hevc => CodecType::Hevc, AlvrCodecType::AV1 => CodecType::AV1, }; let mut config_buffer = vec![0; len as usize]; unsafe { ptr::copy_nonoverlapping(buffer_ptr, config_buffer.as_mut_ptr(), len as usize) }; if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.set_video_config_nals(config_buffer, codec); } } /// global_view_params must be an array of length 2 #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_send_video_nal( timestamp_ns: u64, global_view_params: *const AlvrViewParams, is_idr: bool, buffer_ptr: *mut u8, len: i32, ) { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { let buffer = unsafe { std::slice::from_raw_parts(buffer_ptr, len as usize) }; let global_view_params = unsafe { [ alvr_common::from_capi_view_params(&(*global_view_params)), alvr_common::from_capi_view_params(&(*global_view_params.add(1))), ] }; context.send_video_nal( Duration::from_nanos(timestamp_ns), global_view_params, is_idr, buffer.to_vec(), ); } } /// Returns true if updated #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_get_dynamic_encoder_params( out_params: *mut AlvrDynamicEncoderParams, ) -> bool { if let Some(context) = &*SERVER_CORE_CONTEXT.read() && let Some(params) = context.get_dynamic_encoder_params() { unsafe { (*out_params).bitrate_bps = params.bitrate_bps; (*out_params).framerate = params.framerate; } true } else { false } } #[unsafe(no_mangle)] pub extern "C" fn alvr_report_composed(timestamp_ns: u64, offset_ns: u64) { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.report_composed( Duration::from_nanos(timestamp_ns), Duration::from_nanos(offset_ns), ); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_report_present(timestamp_ns: u64, offset_ns: u64) { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.report_present( Duration::from_nanos(timestamp_ns), Duration::from_nanos(offset_ns), ); } } /// Retr un true if a valid value is provided #[unsafe(no_mangle)] pub unsafe extern "C" fn alvr_duration_until_next_vsync(out_ns: *mut u64) -> bool { if let Some(context) = &*SERVER_CORE_CONTEXT.read() && let Some(duration) = context.duration_until_next_vsync() { unsafe { *out_ns = duration.as_nanos() as u64 }; true } else { false } } #[unsafe(no_mangle)] pub extern "C" fn alvr_restart() { if let Some(context) = SERVER_CORE_CONTEXT.write().take() { context.restart(); } } #[unsafe(no_mangle)] pub extern "C" fn alvr_shutdown() { SERVER_CORE_CONTEXT.write().take(); } ================================================ FILE: alvr/server_core/src/connection.rs ================================================ use crate::{ ConnectionContext, FILESYSTEM_LAYOUT, SESSION_MANAGER, ServerCoreEvent, bitrate::BitrateManager, hand_gestures::HandGestureManager, input_mapping::ButtonMappingManager, sockets::WelcomeSocket, statistics::StatisticsManager, tracking::{self, TrackingManager}, }; use alvr_adb::{WiredConnection, WiredConnectionStatus}; use alvr_common::{ AnyhowToCon, BUTTON_INFO, CONTROLLER_PROFILE_INFO, ConResult, ConnectionError, ConnectionState, LifecycleState, QUEST_CONTROLLER_PROFILE_PATH, con_bail, dbg_connection, debug, error, glam::{UVec2, Vec2}, info, parking_lot::{Condvar, Mutex, RwLock}, settings_schema::Switch, warn, }; use alvr_events::{AdbEvent, ButtonEvent, EventType}; use alvr_packets::{ AUDIO, ClientConnectionResult, ClientConnectionsAction, ClientControlPacket, ClientStatistics, HAPTICS, NegotiatedStreamingConfig, NegotiatedStreamingConfigExt, RealTimeConfig, STATISTICS, ServerControlPacket, StreamConfigPacket, TRACKING, TrackingData, VIDEO, VideoPacketHeader, }; use alvr_session::{ BodyTrackingSinkConfig, CodecType, ControllersEmulationMode, FrameSize, H264Profile, OpenvrConfig, SessionConfig, SocketProtocol, }; use alvr_sockets::{ CONTROL_PORT, KEEPALIVE_INTERVAL, KEEPALIVE_TIMEOUT, PeerType, ProtoControlSocket, StreamSocketBuilder, WIRED_CLIENT_HOSTNAME, }; use std::{ collections::HashMap, net::{IpAddr, Ipv4Addr}, process::Command, sync::{Arc, mpsc::RecvTimeoutError}, thread, time::{Duration, Instant}, }; const RETRY_CONNECT_MIN_INTERVAL: Duration = Duration::from_secs(1); const HANDSHAKE_ACTION_TIMEOUT: Duration = Duration::from_secs(2); pub const STREAMING_RECV_TIMEOUT: Duration = Duration::from_millis(500); const REAL_TIME_UPDATE_INTERVAL: Duration = Duration::from_secs(1); const MAX_UNREAD_PACKETS: usize = 10; // Applies per stream pub struct VideoPacket { pub header: VideoPacketHeader, pub payload: Vec, } fn align32(value: f32) -> u32 { ((value / 32.).floor() * 32.) as u32 } fn is_streaming(client_hostname: &str) -> bool { SESSION_MANAGER .read() .client_list() .get(client_hostname) .is_some_and(|c| c.connection_state == ConnectionState::Streaming) } pub fn contruct_openvr_config(session: &SessionConfig) -> OpenvrConfig { let old_config = session.openvr_config.clone(); let settings = session.to_settings(); let mut controller_is_tracker = false; let mut controller_profile = 0; let mut use_separate_hand_trackers = false; let controllers_enabled = if let Switch::Enabled(config) = &settings.headset.controllers { controller_is_tracker = matches!(config.emulation_mode, ControllersEmulationMode::ViveTracker); // These numbers don't mean anything, they're just for triggering SteamVR resets. // Gaps are included in the numbering to make adding other controllers // a bit easier though. controller_profile = match config.emulation_mode { ControllersEmulationMode::RiftSTouch => 0, ControllersEmulationMode::Quest1Touch => 1, ControllersEmulationMode::Quest2Touch => 2, ControllersEmulationMode::Quest3Plus => 3, ControllersEmulationMode::QuestPro => 4, ControllersEmulationMode::Pico4 => 10, ControllersEmulationMode::ValveIndex => 20, ControllersEmulationMode::ViveWand => 40, ControllersEmulationMode::ViveTracker => 41, ControllersEmulationMode::PSVR2Sense => 60, ControllersEmulationMode::Custom { .. } => 500, }; use_separate_hand_trackers = config .hand_skeleton .as_option() .is_some_and(|c| c.steamvr_input_2_0); true } else { false }; let body_tracking_vive_enabled = if let Switch::Enabled(config) = &settings.headset.body_tracking { matches!(config.sink, BodyTrackingSinkConfig::FakeViveTracker) } else if let Switch::Enabled(config) = settings.headset.multimodal_tracking { config.detached_controllers_steamvr_sink } else { false }; // Should be true if using full body tracking let body_tracking_has_legs = settings .headset .body_tracking .as_option() .map(|c| c.sources.meta.prefer_full_body) .unwrap_or(false); let mut foveation_center_size_x = 0.0; let mut foveation_center_size_y = 0.0; let mut foveation_center_shift_x = 0.0; let mut foveation_center_shift_y = 0.0; let mut foveation_edge_ratio_x = 0.0; let mut foveation_edge_ratio_y = 0.0; let enable_foveated_encoding = if let Switch::Enabled(config) = settings.video.foveated_encoding { foveation_center_size_x = config.center_size_x; foveation_center_size_y = config.center_size_y; foveation_center_shift_x = config.center_shift_x; foveation_center_shift_y = config.center_shift_y; foveation_edge_ratio_x = config.edge_ratio_x; foveation_edge_ratio_y = config.edge_ratio_y; true } else { false }; let mut brightness = 0.0; let mut contrast = 0.0; let mut saturation = 0.0; let mut gamma = 0.0; let mut sharpening = 0.0; let enable_color_correction = if let Switch::Enabled(config) = settings.video.color_correction { brightness = config.brightness; contrast = config.contrast; saturation = config.saturation; gamma = config.gamma; sharpening = config.sharpening; true } else { false }; let nvenc_overrides = settings.video.encoder_config.nvenc; let amf_controls = settings.video.encoder_config.amf; let hdr_controls = settings.video.encoder_config.hdr; OpenvrConfig { tracking_ref_only: settings.headset.tracking_ref_only, enable_vive_tracker_proxy: settings.headset.enable_vive_tracker_proxy, minimum_idr_interval_ms: settings.connection.minimum_idr_interval_ms, adapter_index: settings.video.adapter_index, codec: settings.video.preferred_codec as _, h264_profile: settings.video.encoder_config.h264_profile as u32, rate_control_mode: settings.video.encoder_config.rate_control_mode as u32, filler_data: settings.video.encoder_config.filler_data, entropy_coding: settings.video.encoder_config.entropy_coding as u32, force_hdr_srgb_correction: hdr_controls.force_hdr_srgb_correction, clamp_hdr_extended_range: hdr_controls.clamp_hdr_extended_range, enable_amf_pre_analysis: amf_controls.enable_pre_analysis, enable_vbaq: settings.video.encoder_config.enable_vbaq, enable_amf_hmqb: amf_controls.enable_hmqb, use_amf_preproc: amf_controls.use_preproc, amf_preproc_sigma: amf_controls.preproc_sigma, amf_preproc_tor: amf_controls.preproc_tor, nvenc_quality_preset: nvenc_overrides.quality_preset as u32, encoder_quality_preset: settings.video.encoder_config.quality_preset as u32, force_sw_encoding: settings .video .encoder_config .software .force_software_encoding, sw_thread_count: settings.video.encoder_config.software.thread_count, controllers_enabled, controller_is_tracker, body_tracking_vive_enabled, body_tracking_has_legs, enable_foveated_encoding, foveation_center_size_x, foveation_center_size_y, foveation_center_shift_x, foveation_center_shift_y, foveation_edge_ratio_x, foveation_edge_ratio_y, enable_color_correction, brightness, contrast, saturation, gamma, sharpening, linux_async_compute: settings.extra.patches.linux_async_compute, linux_async_reprojection: settings.extra.patches.linux_async_reprojection, nvenc_tuning_preset: nvenc_overrides.tuning_preset as u32, nvenc_multi_pass: nvenc_overrides.multi_pass as u32, nvenc_adaptive_quantization_mode: nvenc_overrides.adaptive_quantization_mode as u32, nvenc_low_delay_key_frame_scale: nvenc_overrides.low_delay_key_frame_scale, nvenc_refresh_rate: nvenc_overrides.refresh_rate, enable_intra_refresh: nvenc_overrides.enable_intra_refresh, intra_refresh_period: nvenc_overrides.intra_refresh_period, intra_refresh_count: nvenc_overrides.intra_refresh_count, max_num_ref_frames: nvenc_overrides.max_num_ref_frames, gop_length: nvenc_overrides.gop_length, p_frame_strategy: nvenc_overrides.p_frame_strategy, nvenc_rate_control_mode: nvenc_overrides.rate_control_mode, rc_buffer_size: nvenc_overrides.rc_buffer_size, rc_initial_delay: nvenc_overrides.rc_initial_delay, rc_max_bitrate: nvenc_overrides.rc_max_bitrate, rc_average_bitrate: nvenc_overrides.rc_average_bitrate, nvenc_enable_weighted_prediction: nvenc_overrides.enable_weighted_prediction, capture_frame_dir: settings.extra.capture.capture_frame_dir, amd_bitrate_corruption_fix: settings.video.bitrate.image_corruption_fix, use_separate_hand_trackers, _controller_profile: controller_profile, _server_impl_debug: settings.extra.logging.debug_groups.server_impl, _client_impl_debug: settings.extra.logging.debug_groups.client_impl, _server_core_debug: settings.extra.logging.debug_groups.server_core, _client_core_debug: settings.extra.logging.debug_groups.client_core, _connection_debug: settings.extra.logging.debug_groups.connection, _sockets_debug: settings.extra.logging.debug_groups.sockets, _server_gfx_debug: settings.extra.logging.debug_groups.server_gfx, _client_gfx_debug: settings.extra.logging.debug_groups.client_gfx, _encoder_debug: settings.extra.logging.debug_groups.encoder, _decoder_debug: settings.extra.logging.debug_groups.decoder, ..old_config } } // Alternate connection trials with manual IPs and clients discovered on the local network pub fn handshake_loop(ctx: Arc, lifecycle_state: Arc>) { dbg_connection!("handshake_loop: Begin"); let welcome_socket = match WelcomeSocket::new() { Ok(socket) => socket, Err(e) => { error!("Failed to create discovery socket: {e:?}"); return; } }; let mut wired_connection = None; while *lifecycle_state.read() != LifecycleState::ShuttingDown { dbg_connection!("handshake_loop: Try connect to wired device"); let mut wired_client_ips = HashMap::new(); if SESSION_MANAGER .read() .client_list() .iter() .any(|(hostname, info)| { info.connection_state == ConnectionState::Disconnected && hostname.as_str() == WIRED_CLIENT_HOSTNAME }) { // Make sure the wired connection is created once and kept alive let wired_connection = if let Some(connection) = &wired_connection { connection } else { let connection = match WiredConnection::new( FILESYSTEM_LAYOUT.get().unwrap(), |downloaded, maybe_total| { if let Some(total) = maybe_total { alvr_events::send_event(EventType::Adb(AdbEvent { download_progress: downloaded as f32 / total as f32, })); }; }, ) { Ok(connection) => connection, Err(e) => { error!("{e:?}"); thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } }; wired_connection = Some(connection); wired_connection.as_ref().unwrap() }; let stream_port; let client_type; let client_autolaunch; { let session_manager_lock = SESSION_MANAGER.read(); let connection = &session_manager_lock.settings().connection; stream_port = connection.stream_port; client_type = connection.wired_client_type.clone(); client_autolaunch = connection.wired_client_autolaunch.as_option().cloned(); } let status = match wired_connection.setup( CONTROL_PORT, stream_port, &client_type, client_autolaunch, ) { Ok(status) => status, Err(e) => { error!("{e:?}"); thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } }; #[cfg_attr(not(debug_assertions), expect(unused_variables))] if let WiredConnectionStatus::NotReady(s) = status { dbg_connection!("handshake_loop: Wired connection not ready: {s}"); thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } let client_ip = IpAddr::V4(Ipv4Addr::LOCALHOST); wired_client_ips.insert(client_ip, WIRED_CLIENT_HOSTNAME.to_owned()); } if !wired_client_ips.is_empty() && try_connect( Arc::clone(&ctx), Arc::clone(&lifecycle_state), wired_client_ips, ) .is_ok() { thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } dbg_connection!("handshake_loop: Try connect to manual IPs"); let available_manual_client_ips = { let mut manual_client_ips = HashMap::new(); for (hostname, connection_info) in SESSION_MANAGER .read() .client_list() .iter() .filter(|(hostname, info)| { info.connection_state == ConnectionState::Disconnected && hostname.as_str() != WIRED_CLIENT_HOSTNAME }) { for ip in &connection_info.manual_ips { manual_client_ips.insert(*ip, hostname.clone()); } } manual_client_ips }; if !available_manual_client_ips.is_empty() && try_connect( Arc::clone(&ctx), Arc::clone(&lifecycle_state), available_manual_client_ips, ) .is_ok() { thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } let discovery_config = SESSION_MANAGER .read() .settings() .connection .client_discovery .clone(); if let Switch::Enabled(config) = discovery_config { dbg_connection!("handshake_loop: Discovering clients"); let clients = match welcome_socket.recv_all() { Ok(clients) => clients, Err(e) => { warn!("mDNS listening error: {e:?}"); thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } }; if clients.is_empty() { thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } for (client_hostname, client_ip) in clients { let trusted = { let mut session_manager = SESSION_MANAGER.write(); session_manager.update_client_connections( client_hostname.clone(), ClientConnectionsAction::AddIfMissing { trusted: false, manual_ips: vec![], }, ); if config.auto_trust_clients { session_manager.update_client_connections( client_hostname.clone(), ClientConnectionsAction::Trust, ); } session_manager .client_list() .get(&client_hostname) .is_some_and(|c| c.trusted) }; // do not attempt connection if the client is already connected if trusted && SESSION_MANAGER .read() .client_list() .get(&client_hostname) .is_some_and(|c| c.connection_state == ConnectionState::Disconnected) && let Err(e) = try_connect( Arc::clone(&ctx), Arc::clone(&lifecycle_state), [(client_ip, client_hostname.clone())].into_iter().collect(), ) { error!("Could not initiate connection for {client_hostname}: {e}"); } thread::sleep(RETRY_CONNECT_MIN_INTERVAL); } } else { thread::sleep(RETRY_CONNECT_MIN_INTERVAL); } } alvr_common::dbg_connection!("handshake_loop: Joining connection threads"); // At this point, LIFECYCLE_STATE == ShuttingDown, so all threads are already terminating for thread in ctx.connection_threads.lock().drain(..) { thread.join().ok(); } alvr_common::dbg_connection!("handshake_loop: End"); } fn try_connect( ctx: Arc, lifecycle_state: Arc>, mut client_ips: HashMap, ) -> ConResult { dbg_connection!("try_connect: Finding client and creating control socket"); let (proto_socket, client_ip) = ProtoControlSocket::connect_to( Duration::from_secs(1), PeerType::AnyClient(client_ips.keys().cloned().collect()), )?; let Some(client_hostname) = client_ips.remove(&client_ip) else { con_bail!("unreachable"); }; dbg_connection!("try_connect: Pushing new client connection thread"); ctx.connection_threads.lock().push(thread::spawn({ let ctx = Arc::clone(&ctx); move || { if let Err(e) = connection_pipeline( Arc::clone(&ctx), lifecycle_state, proto_socket, client_hostname.clone(), client_ip, ) { error!("Handshake error for {client_hostname}: {e}"); } let mut clients_to_be_removed = ctx.clients_to_be_removed.lock(); let action = if clients_to_be_removed.contains(&client_hostname) { clients_to_be_removed.remove(&client_hostname); ClientConnectionsAction::RemoveEntry } else { ClientConnectionsAction::SetConnectionState(ConnectionState::Disconnected) }; SESSION_MANAGER .write() .update_client_connections(client_hostname, action); } })); Ok(()) } fn connection_pipeline( ctx: Arc, lifecycle_state: Arc>, mut proto_socket: ProtoControlSocket, client_hostname: String, client_ip: IpAddr, ) -> ConResult { dbg_connection!("connection_pipeline: Begin"); // This session lock will make sure settings and client list cannot be changed while connecting // to thos client, no other client can connect until handshake is finished. It will then be // temporarily relocked while shutting down the threads. let mut session_manager_lock = SESSION_MANAGER.write(); dbg_connection!("connection_pipeline: Setting client state in session"); session_manager_lock.update_client_connections( client_hostname.clone(), ClientConnectionsAction::SetConnectionState(ConnectionState::Connecting), ); session_manager_lock.update_client_connections( client_hostname.clone(), ClientConnectionsAction::UpdateCurrentIp(Some(client_ip)), ); let disconnect_notif = Arc::new(Condvar::new()); dbg_connection!("connection_pipeline: Getting client status packet"); let connection_result = match proto_socket.recv(HANDSHAKE_ACTION_TIMEOUT) { Ok(r) => r, Err(ConnectionError::TryAgain(e)) => { debug!( "Failed to recive client connection packet. This is normal for USB connection.\n{e}" ); return Ok(()); } Err(e) => return Err(e), }; let maybe_streaming_caps = if let ClientConnectionResult::ConnectionAccepted(info) = connection_result { session_manager_lock.update_client_connections( client_hostname.clone(), ClientConnectionsAction::SetDisplayName(info.platform_string), ); if info.client_protocol_id != alvr_common::protocol_id_u64() { warn!( "Trusted client is incompatible! Expected protocol ID: {}, found: {}", alvr_common::protocol_id_u64(), info.client_protocol_id, ); return Ok(()); } info.streaming_capabilities } else { debug!("Found client in standby. Retrying"); return Ok(()); }; let Some(streaming_caps) = maybe_streaming_caps else { con_bail!("Only streaming clients are supported for now"); }; dbg_connection!("connection_pipeline: setting up negotiated streaming config"); let initial_settings = session_manager_lock.settings().clone(); fn get_view_res(config: FrameSize, default_res: UVec2) -> UVec2 { let res = match config { FrameSize::Scale(scale) => default_res.as_vec2() * scale, FrameSize::Absolute { width, height } => { let width = width as f32; Vec2::new( width, height.map_or_else( || { let default_res = default_res.as_vec2(); width * default_res.y / default_res.x }, |h| h as f32, ), ) } }; UVec2::new(align32(res.x), align32(res.y)) } let mut transcoding_view_resolution = get_view_res( initial_settings.video.transcoding_view_resolution.clone(), streaming_caps.default_view_resolution, ); if transcoding_view_resolution.x > streaming_caps.max_view_resolution.x || transcoding_view_resolution.y > streaming_caps.max_view_resolution.y { warn!( "Chosen resolution {}x{} exceeds client maximum supported resolution of {}x{}. \ Using maximum supported resolution at same aspect ratio.", transcoding_view_resolution.x, transcoding_view_resolution.y, streaming_caps.max_view_resolution.x, streaming_caps.max_view_resolution.y, ); let transcoding_ratio = transcoding_view_resolution.x as f32 / transcoding_view_resolution.y as f32; if transcoding_ratio > streaming_caps.max_view_resolution.x as f32 / streaming_caps.max_view_resolution.y as f32 { transcoding_view_resolution = UVec2::new( align32(streaming_caps.max_view_resolution.x as f32), align32(streaming_caps.max_view_resolution.x as f32 / transcoding_ratio), ); } else { transcoding_view_resolution = UVec2::new( align32(streaming_caps.max_view_resolution.y as f32 * transcoding_ratio), align32(streaming_caps.max_view_resolution.y as f32), ); } } let emulated_headset_view_resolution = get_view_res( initial_settings .video .emulated_headset_view_resolution .clone(), streaming_caps.default_view_resolution, ); let fps = { let mut best_match = 0_f32; let mut min_diff = f32::MAX; for rate in &streaming_caps.refresh_rates { let diff = (*rate - initial_settings.video.preferred_fps).abs(); if diff < min_diff { best_match = *rate; min_diff = diff; } } best_match }; if !streaming_caps .refresh_rates .contains(&initial_settings.video.preferred_fps) { warn!("Chosen refresh rate not supported. Using {fps}Hz"); } let enable_foveated_encoding = if let Switch::Enabled(config) = &initial_settings.video.foveated_encoding { let enable = streaming_caps.foveated_encoding || config.force_enable; if !enable { warn!("Foveated encoding is not supported by the client."); } enable } else { false }; let encoder_profile = if initial_settings.video.encoder_config.h264_profile == H264Profile::High { let profile = if streaming_caps.encoder_high_profile { H264Profile::High } else { H264Profile::Main }; if profile != H264Profile::High { warn!("High profile encoding is not supported by the client."); } profile } else { initial_settings.video.encoder_config.h264_profile }; let mut enable_10_bits_encoding = initial_settings .video .encoder_config .use_10bit .unwrap_or(streaming_caps.prefer_10bit); if enable_10_bits_encoding && !streaming_caps.encoder_10_bits { warn!("10 bits encoding is not supported by the client."); enable_10_bits_encoding = false } let enable_hdr = initial_settings .video .encoder_config .hdr .enable .unwrap_or(streaming_caps.prefer_hdr); let encoding_gamma = initial_settings .video .encoder_config .encoding_gamma .unwrap_or(streaming_caps.preferred_encoding_gamma); let codec = if initial_settings.video.preferred_codec == CodecType::AV1 { let codec = if streaming_caps.encoder_av1 { CodecType::AV1 } else { CodecType::Hevc }; if codec != CodecType::AV1 { warn!("AV1 encoding is not supported by the client."); } codec } else { initial_settings.video.preferred_codec }; #[cfg(not(target_os = "windows"))] let game_audio_sample_rate = 44100; #[cfg(target_os = "windows")] let game_audio_sample_rate = if let Switch::Enabled(game_audio_config) = &initial_settings.audio.game_audio { let game_audio_device = alvr_audio::new_output(game_audio_config.device.as_ref()).to_con()?; if let Switch::Enabled(microphone_config) = &initial_settings.audio.microphone && matches!( microphone_config.devices, alvr_session::MicrophoneDevicesConfig::VAC | alvr_session::MicrophoneDevicesConfig::VBCable ) { let (sink, _) = alvr_audio::new_virtual_microphone_pair(microphone_config.devices.clone()) .to_con()?; // VoiceMeeter and Custom devices may have arbitrary internal routing. // Therefore, we cannot detect the loopback issue without knowing the routing. if alvr_audio::is_same_device(&game_audio_device, &sink) { con_bail!("Game audio and microphone cannot point to the same device!"); } } alvr_audio::input_sample_rate(&game_audio_device).to_con()? } else { 0 }; let wired = client_ip.is_loopback(); dbg_connection!("connection_pipeline: send streaming config"); let stream_config_packet = StreamConfigPacket::new( session_manager_lock.session(), NegotiatedStreamingConfig { view_resolution: transcoding_view_resolution, refresh_rate_hint: fps, game_audio_sample_rate, enable_foveated_encoding, encoding_gamma, enable_hdr, wired, ext_str: String::new(), } .with_ext(NegotiatedStreamingConfigExt {}), ) .to_con()?; proto_socket.send(&stream_config_packet).to_con()?; let (mut control_sender, mut control_receiver) = proto_socket.split(STREAMING_RECV_TIMEOUT).to_con()?; let mut new_openvr_config = contruct_openvr_config(session_manager_lock.session()); new_openvr_config.eye_resolution_width = transcoding_view_resolution.x; new_openvr_config.eye_resolution_height = transcoding_view_resolution.y; new_openvr_config.target_eye_resolution_width = emulated_headset_view_resolution.x; new_openvr_config.target_eye_resolution_height = emulated_headset_view_resolution.y; new_openvr_config.refresh_rate = fps as _; new_openvr_config.enable_foveated_encoding = enable_foveated_encoding; new_openvr_config.h264_profile = encoder_profile as _; new_openvr_config.use_10bit_encoder = enable_10_bits_encoding; new_openvr_config.enable_hdr = enable_hdr; new_openvr_config.encoding_gamma = encoding_gamma; new_openvr_config.codec = codec as _; if session_manager_lock.session().openvr_config != new_openvr_config { session_manager_lock.session_mut().openvr_config = new_openvr_config; control_sender.send(&ServerControlPacket::Restarting).ok(); crate::notify_restart_driver(); } dbg_connection!("connection_pipeline: Send StartStream packet"); control_sender .send(&ServerControlPacket::StartStream) .to_con()?; let signal = control_receiver.recv(HANDSHAKE_ACTION_TIMEOUT)?; if !matches!(signal, ClientControlPacket::StreamReady) { con_bail!("Got unexpected packet waiting for stream ack"); } dbg_connection!("connection_pipeline: Got StreamReady packet"); *ctx.statistics_manager.write() = Some(StatisticsManager::new( initial_settings.connection.statistics_history_size, Duration::from_secs_f32(1.0 / fps), if let Switch::Enabled(config) = &initial_settings.headset.controllers { config.steamvr_pipeline_frames } else { 0.0 }, )); *ctx.bitrate_manager.lock() = BitrateManager::new(initial_settings.video.bitrate.history_size, fps); let stream_protocol = if wired { SocketProtocol::Tcp } else { initial_settings.connection.stream_protocol }; dbg_connection!("connection_pipeline: StreamSocket connect_to_client"); let mut stream_socket = StreamSocketBuilder::connect_to_client( HANDSHAKE_ACTION_TIMEOUT, client_ip, initial_settings.connection.stream_port, stream_protocol, initial_settings.connection.dscp, initial_settings.connection.server_buffer_config, initial_settings.connection.packet_size as _, )?; let mut video_sender = stream_socket.request_stream(VIDEO); let game_audio_sender: alvr_sockets::StreamSender<()> = stream_socket.request_stream(AUDIO); let mut microphone_receiver: alvr_sockets::StreamReceiver<()> = stream_socket.subscribe_to_stream(AUDIO, MAX_UNREAD_PACKETS); let tracking_receiver = stream_socket.subscribe_to_stream::(TRACKING, MAX_UNREAD_PACKETS); let haptics_sender = stream_socket.request_stream(HAPTICS); let mut statics_receiver = stream_socket.subscribe_to_stream::(STATISTICS, MAX_UNREAD_PACKETS); let (video_channel_sender, video_channel_receiver) = std::sync::mpsc::sync_channel(initial_settings.connection.max_queued_server_video_frames); *ctx.video_channel_sender.lock() = Some(video_channel_sender); *ctx.haptics_sender.lock() = Some(haptics_sender); let video_send_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let client_hostname = client_hostname.clone(); move || { while is_streaming(&client_hostname) { let VideoPacket { mut header, payload, } = match video_channel_receiver.recv_timeout(STREAMING_RECV_TIMEOUT) { Ok(packet) => packet, Err(RecvTimeoutError::Timeout) => continue, Err(RecvTimeoutError::Disconnected) => return, }; ctx.tracking_manager .read() .unrecenter_view_params(&mut header.global_view_params); // todo: use get_buffer and make encoder write to socket buffers directly to avoid copy video_sender .send_header_with_payload(&header, &payload) .ok(); } } }); #[cfg_attr(target_os = "linux", allow(unused_variables))] let game_audio_thread = if let Switch::Enabled(config) = initial_settings.audio.game_audio.clone() { #[cfg(windows)] let ctx = Arc::clone(&ctx); let client_hostname = client_hostname.clone(); thread::spawn(move || { #[cfg(not(target_os = "linux"))] while is_streaming(&client_hostname) { { let device = match alvr_audio::new_output(config.device.as_ref()) { Ok(data) => data, Err(e) => { warn!("New audio device failed: {e:?}"); thread::sleep(RETRY_CONNECT_MIN_INTERVAL); continue; } }; #[cfg(windows)] if let Ok(id) = alvr_audio::get_windows_device_id(&device) { let prop = alvr_session::OpenvrProperty { key: alvr_session::OpenvrPropKey::AudioDefaultPlaybackDeviceIdString, value: id, }; ctx.events_sender .send(ServerCoreEvent::SetOpenvrProperty { device_id: *alvr_common::HEAD_ID, prop, }) .ok(); } else { continue; }; if let Err(e) = alvr_audio::record_audio_blocking( Arc::new({ let client_hostname = client_hostname.clone(); move || is_streaming(&client_hostname) }), game_audio_sender.clone(), &device, 2, config.mute_when_streaming, ) { error!("Audio record error: {e:?}"); } #[cfg(windows)] if let Ok(id) = alvr_audio::new_output(None) .and_then(|d| alvr_audio::get_windows_device_id(&d)) { let prop = alvr_session::OpenvrProperty { key: alvr_session::OpenvrPropKey::AudioDefaultPlaybackDeviceIdString, value: id, }; ctx.events_sender .send(ServerCoreEvent::SetOpenvrProperty { device_id: *alvr_common::HEAD_ID, prop, }) .ok(); } } } }) } else { thread::spawn(|| ()) }; #[cfg(not(target_os = "linux"))] let microphone_thread = if let Switch::Enabled(config) = initial_settings.audio.microphone.clone() { #[allow(unused_variables)] let (sink, source) = alvr_audio::new_virtual_microphone_pair(config.devices).to_con()?; #[cfg(windows)] if let Ok(id) = alvr_audio::get_windows_device_id(&source) { ctx.events_sender .send(ServerCoreEvent::SetOpenvrProperty { device_id: *alvr_common::HEAD_ID, prop: alvr_session::OpenvrProperty { key: alvr_session::OpenvrPropKey::AudioDefaultRecordingDeviceIdString, value: id, }, }) .ok(); } let client_hostname = client_hostname.clone(); thread::spawn(move || { alvr_common::show_err(alvr_audio::play_audio_loop( { let client_hostname = client_hostname.clone(); move || is_streaming(&client_hostname) }, &sink, 1, streaming_caps.microphone_sample_rate, config.buffering, &mut microphone_receiver, )); }) } else { thread::spawn(|| ()) }; #[cfg(target_os = "linux")] let microphone_thread = { use alvr_audio::linux::{self, AudioInfo}; let mic = if let Switch::Enabled(config) = initial_settings.audio.microphone.clone() { Some(( AudioInfo { sample_rate: streaming_caps.microphone_sample_rate, channel_count: 1, }, config.buffering, )) } else { None }; let audio_info = initial_settings .audio .game_audio .enabled() .then_some(AudioInfo { sample_rate: game_audio_sample_rate, channel_count: 2, }); if mic.is_some() || audio_info.is_some() { let client_hostname = client_hostname.clone(); thread::spawn(move || { linux::audio_loop( { let client_hostname = client_hostname.clone(); move || is_streaming(&client_hostname) }, game_audio_sender, audio_info, &mut microphone_receiver, mic, ); }) } else { thread::spawn(|| ()) } }; *ctx.tracking_manager.write() = TrackingManager::new(initial_settings.connection.statistics_history_size); let hand_gesture_manager = Arc::new(Mutex::new(HandGestureManager::new())); let tracking_receive_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let hand_gesture_manager = Arc::clone(&hand_gesture_manager); let initial_settings = initial_settings.clone(); let client_hostname = client_hostname.clone(); move || { tracking::tracking_loop( &ctx, initial_settings, hand_gesture_manager, tracking_receiver, || is_streaming(&client_hostname), ); } }); let statistics_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let client_hostname = client_hostname.clone(); move || { while is_streaming(&client_hostname) { let data = match statics_receiver.recv(STREAMING_RECV_TIMEOUT) { Ok(stats) => stats, Err(ConnectionError::TryAgain(_)) => continue, Err(ConnectionError::Other(_)) => return, }; let Ok(client_stats) = data.get_header() else { return; }; if let Some(stats) = &mut *ctx.statistics_manager.write() { let timestamp = client_stats.target_timestamp; let decoder_latency = client_stats.video_decode; let (network_latency, game_latency) = stats.report_statistics(client_stats); ctx.events_sender .send(ServerCoreEvent::GameRenderLatencyFeedback(game_latency)) .ok(); let session_manager_lock = SESSION_MANAGER.read(); ctx.bitrate_manager.lock().report_frame_latencies( &session_manager_lock.settings().video.bitrate.mode, timestamp, network_latency, decoder_latency, ); } } } }); let control_sender = Arc::new(Mutex::new(control_sender)); let real_time_update_thread = thread::spawn({ let control_sender = Arc::clone(&control_sender); let client_hostname = client_hostname.clone(); move || { let mut previous_config = None; while is_streaming(&client_hostname) { let config = { let session_manager_lock = SESSION_MANAGER.read(); let settings = session_manager_lock.settings(); RealTimeConfig::from_settings(settings) }; let same_config = previous_config.as_ref().is_some_and(|prev| config == *prev); if !same_config { previous_config = Some(config.clone()); control_sender .lock() .send(&ServerControlPacket::RealTimeConfig(config)) .ok(); } thread::sleep(REAL_TIME_UPDATE_INTERVAL); } } }); let keepalive_thread = thread::spawn({ let control_sender = Arc::clone(&control_sender); let disconnect_notif = Arc::clone(&disconnect_notif); let client_hostname = client_hostname.clone(); move || { while is_streaming(&client_hostname) { if let Err(e) = control_sender.lock().send(&ServerControlPacket::KeepAlive) { info!("Client disconnected. Cause: {e:?}"); disconnect_notif.notify_one(); return; } thread::sleep(KEEPALIVE_INTERVAL); } } }); let control_receive_thread = thread::spawn({ let ctx = Arc::clone(&ctx); let controllers_config = session_manager_lock .settings() .headset .controllers .as_option(); let mut controller_button_mapping_manager = controllers_config.map(|config| { if let Some(mappings) = &config.button_mappings { ButtonMappingManager::new_manual(mappings) } else { ButtonMappingManager::new_automatic( &CONTROLLER_PROFILE_INFO .get(&alvr_common::hash_string(QUEST_CONTROLLER_PROFILE_PATH)) .unwrap() .button_set, &config.emulation_mode, &config.button_mapping_config, ) } }); let controllers_emulation_mode = controllers_config.map(|config| config.emulation_mode.clone()); let disconnect_notif = Arc::clone(&disconnect_notif); let control_sender = Arc::clone(&control_sender); let client_hostname = client_hostname.clone(); move || { let mut disconnection_deadline = Instant::now() + KEEPALIVE_TIMEOUT; while is_streaming(&client_hostname) { let packet = match control_receiver.recv(STREAMING_RECV_TIMEOUT) { Ok(packet) => packet, Err(ConnectionError::TryAgain(_)) => { if Instant::now() > disconnection_deadline { info!("Client disconnected. Timeout"); break; } else { continue; } } Err(e) => { info!("Client disconnected. Cause: {e}"); break; } }; match packet { ClientControlPacket::PlayspaceSync(packet) => { if !initial_settings.headset.tracking_ref_only { let session_manager_lock = SESSION_MANAGER.read(); let config = &session_manager_lock.settings().headset; ctx.tracking_manager.write().recenter( config.position_recentering_mode, config.rotation_recentering_mode, ); let area = packet.unwrap_or(Vec2::new(2.0, 2.0)); let wh = area.x * area.y; if wh.is_finite() && wh > 0.0 { info!("Received new playspace with size: {}", area); ctx.events_sender .send(ServerCoreEvent::PlayspaceSync(area)) .ok(); } else { warn!("Received invalid playspace size: {}", area); ctx.events_sender .send(ServerCoreEvent::PlayspaceSync(Vec2::new(2.0, 2.0))) .ok(); } } } ClientControlPacket::RequestIdr => { if let Some(config) = ctx.decoder_config.lock().clone() { control_sender .lock() .send(&ServerControlPacket::DecoderConfig(config)) .ok(); } ctx.events_sender.send(ServerCoreEvent::RequestIDR).ok(); } ClientControlPacket::LocalViewParams(params) => { ctx.events_sender .send(ServerCoreEvent::LocalViewParams(params)) .ok(); } ClientControlPacket::Battery(packet) => { ctx.events_sender .send(ServerCoreEvent::Battery(packet.clone())) .ok(); if let Some(stats) = &mut *ctx.statistics_manager.write() { stats.report_battery( packet.device_id, packet.gauge_value, packet.is_plugged, ); } } ClientControlPacket::Buttons(entries) => { { let session_manager_lock = SESSION_MANAGER.read(); if session_manager_lock .settings() .extra .logging .log_button_presses { alvr_events::send_event(EventType::Buttons( entries .iter() .map(|e| ButtonEvent { path: BUTTON_INFO.get(&e.path_id).map_or_else( || format!("Unknown (ID: {:#16x})", e.path_id), |info| info.path.to_owned(), ), value: e.value, }) .collect(), )); } } if let Some(manager) = &mut controller_button_mapping_manager { let button_entries = entries .iter() .flat_map(|entry| manager.map_button(entry)) .collect::>(); if !button_entries.is_empty() { ctx.events_sender .send(ServerCoreEvent::Buttons(button_entries)) .ok(); } }; } ClientControlPacket::ActiveInteractionProfile { input_ids, .. } => { controller_button_mapping_manager = if let Switch::Enabled(config) = &SESSION_MANAGER.read().settings().headset.controllers { if let Some(mappings) = &config.button_mappings { Some(ButtonMappingManager::new_manual(mappings)) } else { controllers_emulation_mode.as_ref().map(|emulation_mode| { ButtonMappingManager::new_automatic( &input_ids, emulation_mode, &config.button_mapping_config, ) }) } } else { None }; } ClientControlPacket::Log { level, message } => { info!("Client {client_hostname}: [{level:?}] {message}") } ClientControlPacket::KeepAlive | ClientControlPacket::StreamReady => (), ClientControlPacket::ProximityState(headset_is_worn) => { ctx.events_sender .send(ServerCoreEvent::ProximityState(headset_is_worn)) .ok(); } ClientControlPacket::Reserved(_) | ClientControlPacket::ReservedBuffer(_) => (), } disconnection_deadline = Instant::now() + KEEPALIVE_TIMEOUT; } disconnect_notif.notify_one() } }); let stream_receive_thread = thread::spawn({ let disconnect_notif = Arc::clone(&disconnect_notif); let client_hostname = client_hostname.clone(); move || { while is_streaming(&client_hostname) { match stream_socket.recv() { Ok(()) => (), Err(ConnectionError::TryAgain(_)) => continue, Err(e) => { info!("Client disconnected. Cause: {e}"); disconnect_notif.notify_one(); return; } } } } }); let lifecycle_check_thread = thread::spawn({ let disconnect_notif = Arc::clone(&disconnect_notif); let client_hostname = client_hostname.clone(); move || { while SESSION_MANAGER .read() .client_list() .get(&client_hostname) .is_some_and(|c| c.connection_state == ConnectionState::Streaming) && *lifecycle_state.read() == LifecycleState::Resumed { thread::sleep(STREAMING_RECV_TIMEOUT); } disconnect_notif.notify_one() } }); { if initial_settings.connection.enable_on_connect_script { let on_connect_script = FILESYSTEM_LAYOUT.get().map(|l| l.connect_script()).unwrap(); info!( "Running on connect script (connect): {}", on_connect_script.display() ); if let Err(e) = Command::new(&on_connect_script) .env("ACTION", "connect") .spawn() { warn!("Failed to run connect script: {e}"); } } } if initial_settings.extra.capture.startup_video_recording { info!("Creating recording file"); crate::create_recording_file(&ctx, session_manager_lock.settings()); } session_manager_lock.update_client_connections( client_hostname.clone(), ClientConnectionsAction::SetConnectionState(ConnectionState::Streaming), ); ctx.events_sender .send(ServerCoreEvent::ClientConnected) .ok(); dbg_connection!("connection_pipeline: handshake finished; unlocking streams"); alvr_common::wait_rwlock(&disconnect_notif, &mut session_manager_lock); dbg_connection!("connection_pipeline: Begin connection shutdown"); // This requests shutdown from threads *ctx.video_channel_sender.lock() = None; *ctx.haptics_sender.lock() = None; *ctx.video_recording_file.lock() = None; session_manager_lock.update_client_connections( client_hostname, ClientConnectionsAction::SetConnectionState(ConnectionState::Disconnecting), ); let enable_on_disconnect_script = session_manager_lock .settings() .connection .enable_on_disconnect_script; if enable_on_disconnect_script { let on_disconnect_script = FILESYSTEM_LAYOUT .get() .map(|l| l.disconnect_script()) .unwrap(); info!( "Running on disconnect script (disconnect): {}", on_disconnect_script.display() ); if let Err(e) = Command::new(&on_disconnect_script) .env("ACTION", "disconnect") .spawn() { warn!("Failed to run disconnect script: {e}"); } } // Allow threads to shutdown correctly drop(session_manager_lock); // Ensure shutdown of threads dbg_connection!("connection_pipeline: Shutdown threads"); video_send_thread.join().ok(); game_audio_thread.join().ok(); microphone_thread.join().ok(); tracking_receive_thread.join().ok(); statistics_thread.join().ok(); real_time_update_thread.join().ok(); control_receive_thread.join().ok(); stream_receive_thread.join().ok(); keepalive_thread.join().ok(); lifecycle_check_thread.join().ok(); ctx.events_sender .send(ServerCoreEvent::ClientDisconnected) .ok(); dbg_connection!("connection_pipeline: End"); Ok(()) } ================================================ FILE: alvr/server_core/src/hand_gestures.rs ================================================ use crate::input_mapping::ButtonMappingManager; use alvr_common::{ glam::{Vec2, Vec3}, *, }; use alvr_packets::{ButtonEntry, ButtonValue}; use alvr_session::HandTrackingInteractionConfig; use std::{ collections::{HashMap, HashSet}, hash::Hash, sync::LazyLock, time::{Duration, SystemTime, UNIX_EPOCH}, }; fn lerp_pose(a: Pose, b: Pose, fac: f32) -> Pose { Pose { orientation: a.orientation.lerp(b.orientation, fac), position: a.position.lerp(b.position, fac), } } pub static HAND_GESTURE_BUTTON_SET: LazyLock> = LazyLock::new(|| { HashSet::from([ *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID, *LEFT_MENU_CLICK_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_TRIGGER_VALUE_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_THUMBSTICK_X_ID, *LEFT_THUMBSTICK_Y_ID, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_TRIGGER_VALUE_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_THUMBSTICK_X_ID, *RIGHT_THUMBSTICK_Y_ID, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, ]) }); #[derive(Debug, Clone)] pub struct HandGesture { pub id: HandGestureId, pub active: bool, pub clicked: bool, pub touching: bool, pub value: f32, } pub struct GestureAction { last_activated: u128, last_deactivated: u128, entering: bool, entering_since: u128, exiting: bool, exiting_since: u128, active: bool, } #[derive(Debug, Clone, Copy, Eq, Hash, PartialEq)] #[allow(dead_code)] pub enum HandGestureId { // Pinches ThumbIndexPinch, ThumbMiddlePinch, ThumbRingPinch, ThumbLittlePinch, // Curls ThumbCurl, IndexCurl, MiddleCurl, RingCurl, LittleCurl, GripCurl, // Complex JoystickX, JoystickY, } pub struct HandGestureManager { gesture_data_left: HashMap, gesture_data_right: HashMap, } impl HandGestureManager { pub fn new() -> Self { Self { gesture_data_left: HashMap::new(), gesture_data_right: HashMap::new(), } } pub fn get_active_gestures( &mut self, hand_skeleton: &[Pose; 26], config: &HandTrackingInteractionConfig, device_id: u64, ) -> Vec { // global joints let gj = hand_skeleton; // if we model the tip of the finger as a spherical object, we should account for its radius // these are intentionally under the average by ~5mm since the touch and trigger distances are already configurable in settings let thumb_rad: f32 = 0.0075; // average thumb is ~20mm in diameter let index_rad: f32 = 0.0065; // average index finger is ~18mm in diameter let middle_rad: f32 = 0.0065; // average middle finger is ~18mm in diameter let ring_rad: f32 = 0.006; // average ring finger is ~17mm in diameter let little_rad: f32 = 0.005; // average pinky finger is ~15mm in diameter let palm_depth: f32 = 0.005; // average palm bones are ~10mm from the skin // we add the radius of the finger and thumb because we're measuring the distance between the surface of them, not their centers let pinch_min = config.pinch_touch_distance * 0.01; let pinch_max = config.pinch_trigger_distance * 0.01; let curl_min = config.curl_touch_distance * 0.01; let curl_max = config.curl_trigger_distance * 0.01; let palm: Pose = gj[0]; let thumb_tip: Pose = gj[5]; let index_metacarpal: Pose = gj[6]; let index_proximal: Pose = gj[7]; let index_intermediate: Pose = gj[8]; let index_distal: Pose = gj[9]; let index_tip: Pose = gj[10]; let middle_metacarpal: Pose = gj[11]; let middle_proximal: Pose = gj[12]; let middle_tip: Pose = gj[15]; let ring_metacarpal: Pose = gj[16]; let ring_proximal: Pose = gj[17]; let ring_tip: Pose = gj[20]; let little_metacarpal: Pose = gj[21]; let little_proximal: Pose = gj[22]; let little_tip: Pose = gj[25]; let mut gestures = [ // Thumb & index pinch HandGesture { id: HandGestureId::ThumbIndexPinch, active: self.is_gesture_active( HandGestureId::ThumbIndexPinch, thumb_tip, thumb_rad, index_tip, index_rad, pinch_max, config.repeat_delay, config.activation_delay, config.deactivation_delay, device_id, ), clicked: self .test_gesture_dist(thumb_tip, thumb_rad, index_tip, index_rad, pinch_min), touching: self .test_gesture_dist(thumb_tip, thumb_rad, index_tip, index_rad, pinch_max), value: self.get_gesture_hover( thumb_tip, thumb_rad, index_tip, index_rad, pinch_min, pinch_max, ), }, // Thumb & middle pinch HandGesture { id: HandGestureId::ThumbMiddlePinch, active: self.is_gesture_active( HandGestureId::ThumbMiddlePinch, thumb_tip, thumb_rad, middle_tip, middle_rad, pinch_max, config.repeat_delay, config.activation_delay, config.deactivation_delay, device_id, ), clicked: self .test_gesture_dist(thumb_tip, thumb_rad, middle_tip, middle_rad, pinch_min), touching: self .test_gesture_dist(thumb_tip, thumb_rad, middle_tip, middle_rad, pinch_max), value: self.get_gesture_hover( thumb_tip, thumb_rad, middle_tip, middle_rad, pinch_min, pinch_max, ), }, // Thumb & ring pinch HandGesture { id: HandGestureId::ThumbRingPinch, active: self.is_gesture_active( HandGestureId::ThumbRingPinch, thumb_tip, thumb_rad, ring_tip, ring_rad, pinch_max, config.repeat_delay, config.activation_delay, config.deactivation_delay, device_id, ), clicked: self .test_gesture_dist(thumb_tip, thumb_rad, ring_tip, ring_rad, pinch_min), touching: self .test_gesture_dist(thumb_tip, thumb_rad, ring_tip, ring_rad, pinch_max), value: self.get_gesture_hover( thumb_tip, thumb_rad, ring_tip, ring_rad, pinch_min, pinch_max, ), }, // Thumb & little pinch HandGesture { id: HandGestureId::ThumbLittlePinch, active: self.is_gesture_active( HandGestureId::ThumbLittlePinch, thumb_tip, thumb_rad, little_tip, little_rad, pinch_max, config.repeat_delay, config.activation_delay, config.deactivation_delay, device_id, ), clicked: self .test_gesture_dist(thumb_tip, thumb_rad, little_tip, little_rad, pinch_min), touching: self .test_gesture_dist(thumb_tip, thumb_rad, little_tip, little_rad, pinch_max), value: self.get_gesture_hover( thumb_tip, thumb_rad, little_tip, little_rad, pinch_min, pinch_max, ), }, ] .into_iter() .collect::>(); // Finger curls let thumb_curl = self.get_gesture_hover(palm, palm_depth, thumb_tip, thumb_rad, curl_min, curl_max); let index_curl = self.get_gesture_hover( lerp_pose(index_metacarpal, index_proximal, 0.5), palm_depth, index_tip, index_rad, curl_min, curl_max, ); let middle_curl = self.get_gesture_hover( lerp_pose(middle_metacarpal, middle_proximal, 0.5), palm_depth, middle_tip, middle_rad, curl_min, curl_max, ); let ring_curl = self.get_gesture_hover( lerp_pose(ring_metacarpal, ring_proximal, 0.5), palm_depth, ring_tip, ring_rad, curl_min, curl_max, ); let little_curl = self.get_gesture_hover( lerp_pose(little_metacarpal, little_proximal, 0.5), palm_depth, little_tip, little_rad, curl_min, curl_max, ); // Grip let grip_curl = (middle_curl + ring_curl + little_curl) / 3.0; let grip_active = grip_curl > 0.0; gestures.push(HandGesture { id: HandGestureId::GripCurl, active: grip_active, clicked: grip_curl == 1.0, touching: grip_curl > 0.0, value: grip_curl, }); // Joystick let joystick_range = config.joystick_range * 0.01; let joystick_center = lerp_pose(index_intermediate, index_distal, 0.5); let joystick_up = joystick_center .orientation .mul_vec3(if device_id == *HAND_LEFT_ID { Vec3::X } else { Vec3::NEG_X }); let joystick_horizontal_vec = index_intermediate .orientation .mul_vec3(if device_id == *HAND_LEFT_ID { Vec3::Y } else { Vec3::NEG_Y }); let joystick_vertical_vec = index_intermediate.orientation.mul_vec3(Vec3::Z); let joystick_offset_horizontal_direction = if device_id == *HAND_LEFT_ID { 1.0 } else { -1.0 }; let joystick_pos = self.get_joystick_values( joystick_center, thumb_tip, joystick_range, joystick_horizontal_vec, joystick_vertical_vec, config.joystick_offset_horizontal * 0.01 * joystick_offset_horizontal_direction, config.joystick_offset_vertical * 0.01, ); let joystick_contact = index_curl >= 0.75 && grip_curl > 0.5 && joystick_center.position.distance(thumb_tip.position) <= joystick_range * 3.0 && (thumb_tip.position - joystick_center.position).dot(joystick_up) / joystick_up.length() <= joystick_range * 2.0; let joystick_deadzone: f32 = config.joystick_deadzone * 0.01; gestures.push(HandGesture { id: HandGestureId::ThumbCurl, active: thumb_curl >= 0.0, touching: thumb_curl >= 0.0, clicked: thumb_curl >= 0.5, value: thumb_curl, }); gestures.push(HandGesture { id: HandGestureId::JoystickX, active: joystick_contact, touching: joystick_contact, clicked: false, value: if joystick_contact && joystick_pos.x.abs() >= joystick_deadzone { joystick_pos.x } else { 0.0 }, }); gestures.push(HandGesture { id: HandGestureId::JoystickY, active: joystick_contact, touching: joystick_contact, clicked: false, value: if joystick_contact && joystick_pos.y.abs() >= joystick_deadzone { joystick_pos.y } else { 0.0 }, }); gestures } #[allow(clippy::too_many_arguments)] fn is_gesture_active( &mut self, gesture_id: HandGestureId, first_anchor: Pose, first_radius: f32, second_anchor: Pose, second_radius: f32, activation_dist: f32, repeat_delay: u32, in_delay: u32, out_delay: u32, device_id: u64, ) -> bool { let in_range = first_anchor.position.distance(second_anchor.position) < (activation_dist + first_radius + second_radius); let gesture_data = if device_id == *HAND_LEFT_ID { &mut self.gesture_data_left } else { &mut self.gesture_data_right }; gesture_data.entry(gesture_id).or_insert(GestureAction { last_activated: 0, last_deactivated: 0, entering: false, entering_since: 0, exiting: false, exiting_since: 0, active: false, }); let g: &mut GestureAction = gesture_data.get_mut(&gesture_id).unwrap(); // Disable entering/exiting state if we leave/enter range if in_range { g.exiting = false; } else { g.entering = false; } // Get current time, for comparison let time_millis = SystemTime::now() .duration_since(UNIX_EPOCH) .unwrap_or(Duration::from_millis(0)) .as_millis(); // Transitioning from inactive to active if in_range && !g.active { // Don't transition state unless the duration of repeat_delay has passed since last deactivation if g.last_deactivated < time_millis - u128::from(repeat_delay) { if g.entering { // Don't transition state unless gesture has been in range for the duration of in_delay if g.entering_since < time_millis - u128::from(in_delay) { g.last_activated = time_millis; g.entering = false; g.active = true; } } else { // Begin tracking entering state g.entering = true; g.entering_since = time_millis; } } } // Transitioning from inactive to active if !in_range && g.active { if g.exiting { // Don't transition state unless gesture has been out of range for the duration of out_delay if g.exiting_since < time_millis - u128::from(out_delay) { g.last_deactivated = time_millis; g.exiting = false; g.active = false; } } else { // Begin tracking exiting state g.exiting = true; g.exiting_since = time_millis; } } g.active } fn test_gesture_dist( &self, first_anchor: Pose, first_radius: f32, second_anchor: Pose, second_radius: f32, activation_dist: f32, ) -> bool { first_anchor.position.distance(second_anchor.position) < (activation_dist + first_radius + second_radius) } fn get_gesture_hover( &self, first_anchor: Pose, first_radius: f32, second_anchor: Pose, second_radius: f32, min_dist: f32, max_dist: f32, ) -> f32 { (1.0 - (first_anchor.position.distance(second_anchor.position) - min_dist - first_radius - second_radius) / (max_dist + first_radius + second_radius)) .clamp(0.0, 1.0) } #[allow(clippy::too_many_arguments)] fn get_joystick_values( &self, center: Pose, anchor: Pose, joy_radius: f32, hori_vec: Vec3, vert_vec: Vec3, offset_hori: f32, offset_vert: f32, ) -> Vec2 { let x = (anchor.position - center.position).dot(hori_vec) / hori_vec.length() + offset_hori; let y = (anchor.position - center.position).dot(vert_vec) / vert_vec.length() + offset_vert; Vec2 { x: (x / joy_radius).clamp(-1.0, 1.0), y: (y / joy_radius).clamp(-1.0, 1.0), } } } fn get_click_bind_for_gesture(device_id: u64, gesture_id: HandGestureId) -> Option { if device_id == *HAND_LEFT_ID { match gesture_id { HandGestureId::ThumbIndexPinch => Some(*LEFT_TRIGGER_CLICK_ID), HandGestureId::ThumbMiddlePinch => Some(*LEFT_Y_CLICK_ID), HandGestureId::ThumbRingPinch => Some(*LEFT_X_CLICK_ID), HandGestureId::ThumbLittlePinch => Some(*LEFT_MENU_CLICK_ID), HandGestureId::GripCurl => Some(*LEFT_SQUEEZE_CLICK_ID), HandGestureId::ThumbCurl => Some(*LEFT_THUMBSTICK_CLICK_ID), _ => None, } } else { match gesture_id { HandGestureId::ThumbIndexPinch => Some(*RIGHT_TRIGGER_CLICK_ID), HandGestureId::ThumbMiddlePinch => Some(*RIGHT_B_CLICK_ID), HandGestureId::ThumbRingPinch => Some(*RIGHT_A_CLICK_ID), HandGestureId::GripCurl => Some(*RIGHT_SQUEEZE_CLICK_ID), HandGestureId::ThumbCurl => Some(*RIGHT_THUMBSTICK_CLICK_ID), _ => None, } } } fn get_touch_bind_for_gesture(device_id: u64, gesture_id: HandGestureId) -> Option { if device_id == *HAND_LEFT_ID { match gesture_id { HandGestureId::ThumbIndexPinch => Some(*LEFT_TRIGGER_TOUCH_ID), HandGestureId::ThumbMiddlePinch => Some(*LEFT_Y_TOUCH_ID), HandGestureId::ThumbRingPinch => Some(*LEFT_X_TOUCH_ID), HandGestureId::JoystickX | HandGestureId::JoystickY => Some(*LEFT_THUMBSTICK_TOUCH_ID), _ => None, } } else { match gesture_id { HandGestureId::ThumbIndexPinch => Some(*RIGHT_TRIGGER_TOUCH_ID), HandGestureId::ThumbMiddlePinch => Some(*RIGHT_B_TOUCH_ID), HandGestureId::ThumbRingPinch => Some(*RIGHT_A_TOUCH_ID), HandGestureId::JoystickX | HandGestureId::JoystickY => Some(*RIGHT_THUMBSTICK_TOUCH_ID), _ => None, } } } fn get_hover_bind_for_gesture(device_id: u64, gesture_id: HandGestureId) -> Option { if device_id == *HAND_LEFT_ID { match gesture_id { HandGestureId::ThumbIndexPinch => Some(*LEFT_TRIGGER_VALUE_ID), HandGestureId::GripCurl => Some(*LEFT_SQUEEZE_VALUE_ID), HandGestureId::JoystickX => Some(*LEFT_THUMBSTICK_X_ID), HandGestureId::JoystickY => Some(*LEFT_THUMBSTICK_Y_ID), _ => None, } } else { match gesture_id { HandGestureId::ThumbIndexPinch => Some(*RIGHT_TRIGGER_VALUE_ID), HandGestureId::GripCurl => Some(*RIGHT_SQUEEZE_VALUE_ID), HandGestureId::JoystickX => Some(*RIGHT_THUMBSTICK_X_ID), HandGestureId::JoystickY => Some(*RIGHT_THUMBSTICK_Y_ID), _ => None, } } } pub fn trigger_hand_gesture_actions( button_mapping_manager: &mut ButtonMappingManager, device_id: u64, gestures: &[HandGesture], only_touch: bool, ) -> Vec { let mut button_entries = vec![]; for gesture in gestures { // Click bind if !only_touch && let Some(click_bind) = get_click_bind_for_gesture(device_id, gesture.id) { button_entries.append(&mut button_mapping_manager.map_button(&ButtonEntry { path_id: click_bind, value: ButtonValue::Binary(gesture.active && gesture.clicked), })); } // Touch bind if let Some(touch_bind) = get_touch_bind_for_gesture(device_id, gesture.id) { button_entries.append(&mut button_mapping_manager.map_button(&ButtonEntry { path_id: touch_bind, value: ButtonValue::Binary(gesture.active && gesture.touching), })); } // Hover bind if !only_touch && let Some(hover_bind) = get_hover_bind_for_gesture(device_id, gesture.id) { button_entries.append(&mut button_mapping_manager.map_button(&ButtonEntry { path_id: hover_bind, value: ButtonValue::Scalar(if gesture.active { gesture.value } else { 0.0 }), })); } } button_entries } ================================================ FILE: alvr/server_core/src/haptics.rs ================================================ use alvr_packets::Haptics; use alvr_session::HapticsConfig; use std::time::Duration; pub fn map_haptics(config: &HapticsConfig, haptics: Haptics) -> Haptics { Haptics { duration: Duration::max( haptics.duration, Duration::from_secs_f32(config.min_duration_s), ), amplitude: config.intensity_multiplier * f32::powf(haptics.amplitude, config.amplitude_curve), ..haptics } } ================================================ FILE: alvr/server_core/src/input_mapping.rs ================================================ use alvr_common::*; use alvr_packets::{ButtonEntry, ButtonValue}; use alvr_session::{ AutomaticButtonMappingConfig, BinaryToScalarStates, ButtonBindingTarget, ButtonMappingType, ControllersEmulationMode, HysteresisThreshold, Range, }; use std::collections::{HashMap, HashSet}; pub fn registered_button_set( controllers_emulation_mode: &ControllersEmulationMode, ) -> HashSet { match &controllers_emulation_mode { ControllersEmulationMode::RiftSTouch | ControllersEmulationMode::Quest1Touch | ControllersEmulationMode::Quest2Touch | ControllersEmulationMode::Quest3Plus | ControllersEmulationMode::QuestPro => CONTROLLER_PROFILE_INFO .get(&QUEST_CONTROLLER_PROFILE_ID) .unwrap() .button_set .clone(), ControllersEmulationMode::Pico4 => CONTROLLER_PROFILE_INFO .get(&PICO4_CONTROLLER_PROFILE_ID) .unwrap() .button_set .clone(), ControllersEmulationMode::PSVR2Sense => CONTROLLER_PROFILE_INFO .get(&PSVR2_CONTROLLER_PROFILE_ID) .unwrap() .button_set .clone(), ControllersEmulationMode::ValveIndex => CONTROLLER_PROFILE_INFO .get(&INDEX_CONTROLLER_PROFILE_ID) .unwrap() .button_set .clone(), ControllersEmulationMode::ViveWand => CONTROLLER_PROFILE_INFO .get(&VIVE_CONTROLLER_PROFILE_ID) .unwrap() .button_set .clone(), ControllersEmulationMode::ViveTracker => HashSet::new(), ControllersEmulationMode::Custom { button_set, .. } => button_set .iter() .map(|b| alvr_common::hash_string(b)) .collect(), } } pub struct BindingTarget { destination: u64, mapping_type: ButtonMappingType, binary_conditions: Vec, } // Inputs relative to the same physical button #[derive(Clone, Copy)] pub struct ButtonInputs { click: Option, touch: Option, value: Option, force: Option, } fn click(click: u64) -> ButtonInputs { ButtonInputs { click: Some(click), touch: None, value: None, force: None, } } fn ct(set: &HashSet, click: u64, touch: u64) -> ButtonInputs { ButtonInputs { click: Some(click), touch: set.contains(&touch).then_some(touch), value: None, force: None, } } fn value(value: u64) -> ButtonInputs { ButtonInputs { click: None, touch: None, value: Some(value), force: None, } } fn ctv(set: &HashSet, click: u64, touch: u64, value: u64) -> ButtonInputs { ButtonInputs { click: set.contains(&click).then_some(click), touch: set.contains(&touch).then_some(touch), value: set.contains(&value).then_some(value), force: None, } } fn ctvf(set: &HashSet, click: u64, touch: u64, value: u64, force: u64) -> ButtonInputs { ButtonInputs { click: set.contains(&click).then_some(click), touch: set.contains(&touch).then_some(touch), value: set.contains(&value).then_some(value), force: set.contains(&force).then_some(force), } } fn passthrough(target: u64) -> BindingTarget { BindingTarget { destination: target, mapping_type: ButtonMappingType::Passthrough, binary_conditions: vec![], } } fn binary_to_scalar(target: u64, map: BinaryToScalarStates) -> BindingTarget { BindingTarget { destination: target, mapping_type: ButtonMappingType::BinaryToScalar(map), binary_conditions: vec![], } } fn hysteresis_threshold(target: u64, map: HysteresisThreshold) -> BindingTarget { BindingTarget { destination: target, mapping_type: ButtonMappingType::HysteresisThreshold(map), binary_conditions: vec![], } } fn remap(target: u64, map: Range) -> BindingTarget { BindingTarget { destination: target, mapping_type: ButtonMappingType::Remap(map), binary_conditions: vec![], } } // Map two buttons with eterogeneous inputs fn map_button_pair_automatic( source: ButtonInputs, destination: ButtonInputs, config: &AutomaticButtonMappingConfig, ) -> impl Iterator)> { let click_to_value = BinaryToScalarStates { off: 0.0, on: 1.0 }; let mut entries = vec![]; if let Some(source_click) = source.click { let mut targets = vec![]; if let Some(destination_click) = destination.click { targets.push(passthrough(destination_click)); } if source.touch.is_none() && let Some(destination_touch) = destination.touch { targets.push(passthrough(destination_touch)); } if source.value.is_none() && let Some(destination_value) = destination.value { targets.push(binary_to_scalar(destination_value, click_to_value)); } entries.push((source_click, targets)); } if let Some(source_touch) = source.touch { let mut targets = vec![]; if let Some(destination_touch) = destination.touch { targets.push(passthrough(destination_touch)); } entries.push((source_touch, targets)); } if let Some(source_value) = source.value { let mut targets = vec![]; let mut remap_for_touch = false; let mut remap_for_force = false; if source.click.is_none() && let Some(destination_click) = destination.click { targets.push(hysteresis_threshold( destination_click, config.click_threshold, )); } if source.touch.is_none() && let Some(destination_touch) = destination.touch { targets.push(hysteresis_threshold( destination_touch, config.touch_threshold, )); remap_for_touch = true; } if source.force.is_none() && let Some(destination_force) = destination.force { targets.push(remap( destination_force, Range { min: config.force_threshold, max: 1.0, }, )); remap_for_force = true; } if let Some(destination_value) = destination.value { if !remap_for_touch && !remap_for_force { targets.push(passthrough(destination_value)); } else { let low = if remap_for_touch { config.touch_threshold.value } else { 0.0 }; let high = if remap_for_force { config.force_threshold } else { 1.0 }; targets.push(remap( destination_value, Range { min: low, max: high, }, )); } } entries.push((source_value, targets)); } entries.into_iter() } pub fn automatic_bindings( source_set: &HashSet, destination_set: &HashSet, config: &AutomaticButtonMappingConfig, ) -> HashMap> { let s_set = source_set; let d_set = destination_set; let mut bindings = HashMap::new(); // Menu buttons if s_set.contains(&*LEFT_MENU_CLICK_ID) { let click = click(*LEFT_MENU_CLICK_ID); if d_set.contains(&*LEFT_MENU_CLICK_ID) { bindings.extend(map_button_pair_automatic(click, click, config)); } else if d_set.contains(&*LEFT_SYSTEM_CLICK_ID) { bindings.extend(map_button_pair_automatic( click, ct(s_set, *LEFT_SYSTEM_CLICK_ID, *LEFT_SYSTEM_TOUCH_ID), config, )); } } if s_set.contains(&*LEFT_SYSTEM_CLICK_ID) { let click = click(*LEFT_SYSTEM_CLICK_ID); if d_set.contains(&*LEFT_SYSTEM_CLICK_ID) { bindings.extend(map_button_pair_automatic( click, ct(s_set, *LEFT_SYSTEM_CLICK_ID, *LEFT_SYSTEM_TOUCH_ID), config, )); } else if d_set.contains(&*LEFT_MENU_CLICK_ID) { bindings.extend(map_button_pair_automatic(click, click, config)); } } if s_set.contains(&*RIGHT_MENU_CLICK_ID) { let click = click(*RIGHT_MENU_CLICK_ID); if d_set.contains(&*RIGHT_MENU_CLICK_ID) { bindings.extend(map_button_pair_automatic(click, click, config)); } else if d_set.contains(&*RIGHT_SYSTEM_CLICK_ID) { bindings.extend(map_button_pair_automatic( click, ct(s_set, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SYSTEM_TOUCH_ID), config, )); } } if s_set.contains(&*RIGHT_SYSTEM_CLICK_ID) { let click = click(*RIGHT_SYSTEM_CLICK_ID); if d_set.contains(&*RIGHT_SYSTEM_CLICK_ID) { bindings.extend(map_button_pair_automatic( click, ct(s_set, *RIGHT_SYSTEM_CLICK_ID, *RIGHT_SYSTEM_TOUCH_ID), config, )); } else if d_set.contains(&*RIGHT_MENU_CLICK_ID) { bindings.extend(map_button_pair_automatic(click, click, config)); } } // A/X buttons if s_set.contains(&*LEFT_X_CLICK_ID) { let source = ct(s_set, *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID); if d_set.contains(&*LEFT_X_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_X_CLICK_ID, *LEFT_X_TOUCH_ID), config, )); } else if d_set.contains(&*LEFT_A_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_A_CLICK_ID, *LEFT_A_TOUCH_ID), config, )); } else if d_set.contains(&*LEFT_TRACKPAD_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_TRACKPAD_CLICK_ID, *LEFT_TRACKPAD_TOUCH_ID), config, )); } } if s_set.contains(&*RIGHT_A_CLICK_ID) { let source = ct(s_set, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID); if d_set.contains(&*RIGHT_A_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *RIGHT_A_CLICK_ID, *RIGHT_A_TOUCH_ID), config, )); } else if d_set.contains(&*RIGHT_TRACKPAD_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *RIGHT_TRACKPAD_CLICK_ID, *RIGHT_TRACKPAD_TOUCH_ID), config, )); } } // B/Y buttons if s_set.contains(&*LEFT_Y_CLICK_ID) { let source = ct(s_set, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID); if d_set.contains(&*LEFT_Y_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_Y_CLICK_ID, *LEFT_Y_TOUCH_ID), config, )); } else if d_set.contains(&*LEFT_B_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_B_CLICK_ID, *LEFT_B_TOUCH_ID), config, )); } } if s_set.contains(&*RIGHT_B_CLICK_ID) && d_set.contains(&*RIGHT_B_CLICK_ID) { bindings.extend(map_button_pair_automatic( ct(s_set, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID), ct(d_set, *RIGHT_B_CLICK_ID, *RIGHT_B_TOUCH_ID), config, )); } // Squeeze buttons if (s_set.contains(&*LEFT_SQUEEZE_CLICK_ID) || s_set.contains(&*LEFT_SQUEEZE_VALUE_ID)) && (d_set.contains(&*LEFT_SQUEEZE_CLICK_ID) || d_set.contains(&*LEFT_SQUEEZE_VALUE_ID)) { bindings.extend(map_button_pair_automatic( ctvf( s_set, *LEFT_SQUEEZE_CLICK_ID, *LEFT_SQUEEZE_TOUCH_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_SQUEEZE_FORCE_ID, ), ctvf( d_set, *LEFT_SQUEEZE_CLICK_ID, *LEFT_SQUEEZE_TOUCH_ID, *LEFT_SQUEEZE_VALUE_ID, *LEFT_SQUEEZE_FORCE_ID, ), config, )); } if (s_set.contains(&*RIGHT_SQUEEZE_CLICK_ID) || s_set.contains(&*RIGHT_SQUEEZE_VALUE_ID)) && (d_set.contains(&*RIGHT_SQUEEZE_CLICK_ID) || d_set.contains(&*RIGHT_SQUEEZE_VALUE_ID)) { bindings.extend(map_button_pair_automatic( ctvf( s_set, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_SQUEEZE_TOUCH_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_SQUEEZE_FORCE_ID, ), ctvf( d_set, *RIGHT_SQUEEZE_CLICK_ID, *RIGHT_SQUEEZE_TOUCH_ID, *RIGHT_SQUEEZE_VALUE_ID, *RIGHT_SQUEEZE_FORCE_ID, ), config, )); } // Trigger buttons if (s_set.contains(&*LEFT_TRIGGER_CLICK_ID) || s_set.contains(&*LEFT_TRIGGER_VALUE_ID)) && (d_set.contains(&*LEFT_TRIGGER_CLICK_ID) || d_set.contains(&*LEFT_TRIGGER_VALUE_ID)) { bindings.extend(map_button_pair_automatic( ctv( s_set, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_TRIGGER_VALUE_ID, ), ctv( d_set, *LEFT_TRIGGER_CLICK_ID, *LEFT_TRIGGER_TOUCH_ID, *LEFT_TRIGGER_VALUE_ID, ), config, )); } if (s_set.contains(&*RIGHT_TRIGGER_CLICK_ID) || s_set.contains(&*RIGHT_TRIGGER_VALUE_ID)) && (d_set.contains(&*RIGHT_TRIGGER_CLICK_ID) || d_set.contains(&*RIGHT_TRIGGER_VALUE_ID)) { bindings.extend(map_button_pair_automatic( ctv( s_set, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_TRIGGER_VALUE_ID, ), ctv( d_set, *RIGHT_TRIGGER_CLICK_ID, *RIGHT_TRIGGER_TOUCH_ID, *RIGHT_TRIGGER_VALUE_ID, ), config, )); } // Thumbsticks if s_set.contains(&*LEFT_THUMBSTICK_X_ID) { let x = value(*LEFT_THUMBSTICK_X_ID); let y = value(*LEFT_THUMBSTICK_Y_ID); if d_set.contains(&*LEFT_THUMBSTICK_X_ID) { bindings.extend(map_button_pair_automatic(x, x, config)); bindings.extend(map_button_pair_automatic(y, y, config)); } else if d_set.contains(&*LEFT_TRACKPAD_X_ID) { bindings.extend(map_button_pair_automatic( x, value(*LEFT_TRACKPAD_X_ID), config, )); bindings.extend(map_button_pair_automatic( y, value(*LEFT_TRACKPAD_Y_ID), config, )); } } if s_set.contains(&*LEFT_THUMBSTICK_CLICK_ID) { let source = ct(s_set, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID); if d_set.contains(&*LEFT_THUMBSTICK_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_THUMBSTICK_CLICK_ID, *LEFT_THUMBSTICK_TOUCH_ID), config, )); } else if d_set.contains(&*LEFT_TRACKPAD_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *LEFT_TRACKPAD_CLICK_ID, *LEFT_TRACKPAD_TOUCH_ID), config, )); } } if s_set.contains(&*RIGHT_THUMBSTICK_X_ID) { let x = value(*RIGHT_THUMBSTICK_X_ID); let y = value(*RIGHT_THUMBSTICK_Y_ID); if d_set.contains(&*RIGHT_THUMBSTICK_X_ID) { bindings.extend(map_button_pair_automatic(x, x, config)); bindings.extend(map_button_pair_automatic(y, y, config)); } else if d_set.contains(&*RIGHT_TRACKPAD_X_ID) { bindings.extend(map_button_pair_automatic( x, value(*RIGHT_TRACKPAD_X_ID), config, )); bindings.extend(map_button_pair_automatic( y, value(*RIGHT_TRACKPAD_Y_ID), config, )); } } if s_set.contains(&*RIGHT_THUMBSTICK_CLICK_ID) { let source = ct( s_set, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, ); if d_set.contains(&*RIGHT_THUMBSTICK_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct( d_set, *RIGHT_THUMBSTICK_CLICK_ID, *RIGHT_THUMBSTICK_TOUCH_ID, ), config, )); } else if d_set.contains(&*RIGHT_TRACKPAD_CLICK_ID) { bindings.extend(map_button_pair_automatic( source, ct(d_set, *RIGHT_TRACKPAD_CLICK_ID, *RIGHT_TRACKPAD_TOUCH_ID), config, )); } } // Thumbrests if s_set.contains(&*LEFT_THUMBREST_TOUCH_ID) { let source = value(*LEFT_THUMBREST_TOUCH_ID); if d_set.contains(&*LEFT_THUMBREST_TOUCH_ID) { bindings.extend(map_button_pair_automatic(source, source, config)); } else if d_set.contains(&*LEFT_TRACKPAD_TOUCH_ID) { bindings.extend(map_button_pair_automatic( source, value(*LEFT_TRACKPAD_TOUCH_ID), config, )); } } if s_set.contains(&*RIGHT_THUMBREST_TOUCH_ID) { let source = value(*RIGHT_THUMBREST_TOUCH_ID); if d_set.contains(&*RIGHT_THUMBREST_TOUCH_ID) { bindings.extend(map_button_pair_automatic(source, source, config)); } else if d_set.contains(&*RIGHT_TRACKPAD_TOUCH_ID) { bindings.extend(map_button_pair_automatic( source, value(*RIGHT_TRACKPAD_TOUCH_ID), config, )); } } bindings } pub struct ButtonMappingManager { mappings: HashMap>, binary_source_states: HashMap, hysteresis_states: HashMap>, } impl ButtonMappingManager { pub fn new_automatic( source: &HashSet, controllers_emulation_mode: &ControllersEmulationMode, button_mapping_config: &AutomaticButtonMappingConfig, ) -> Self { let button_set = registered_button_set(controllers_emulation_mode); Self { mappings: automatic_bindings(source, &button_set, button_mapping_config), binary_source_states: HashMap::new(), hysteresis_states: HashMap::new(), } } pub fn new_manual(mappings: &[(String, Vec)]) -> Self { let mappings = mappings .iter() .map(|(key, value)| { ( alvr_common::hash_string(key), value .iter() .map(|b| BindingTarget { destination: alvr_common::hash_string(&b.destination), mapping_type: b.mapping_type.clone(), binary_conditions: b .binary_conditions .iter() .map(|c| alvr_common::hash_string(c)) .collect(), }) .collect(), ) }) .collect(); Self { mappings, binary_source_states: HashMap::new(), hysteresis_states: HashMap::new(), } } // Apply any button changes that are mapped to this specific button pub fn map_button(&mut self, source_button: &ButtonEntry) -> Vec { if let ButtonValue::Binary(value) = source_button.value { let val_ref = self .binary_source_states .entry(source_button.path_id) .or_default(); if value == *val_ref { return vec![]; } // NB: Update value *val_ref = value; } let mut destination_buttons = vec![]; if let Some(mappings) = self.mappings.get(&source_button.path_id) { 'mapping: for mapping in mappings { let destination_value = match (&mapping.mapping_type, source_button.value) { (ButtonMappingType::Passthrough, value) => value, ( ButtonMappingType::HysteresisThreshold(threshold), ButtonValue::Scalar(value), ) => { let state = self .hysteresis_states .entry(source_button.path_id) .or_default() .entry(mapping.destination) .or_default(); if *state && value < threshold.value - threshold.deviation { *state = false; } else if !*state && value > threshold.value + threshold.deviation { *state = true; } else { // No change needed continue; } ButtonValue::Binary(*state) } (ButtonMappingType::BinaryToScalar(levels), ButtonValue::Binary(value)) => { if value { ButtonValue::Scalar(levels.on) } else { ButtonValue::Scalar(levels.off) } } (ButtonMappingType::Remap(range), ButtonValue::Scalar(value)) => { let value = (value - range.min) / (range.max - range.min); ButtonValue::Scalar(value.clamp(0.0, 1.0)) } _ => { error!("Failed to map button!"); continue; } }; for source_id in &mapping.binary_conditions { if !self .binary_source_states .get(source_id) .copied() .unwrap_or(false) { continue 'mapping; } } destination_buttons.push(ButtonEntry { path_id: mapping.destination, value: destination_value, }); } } else { let button_name = BUTTON_INFO .get(&source_button.path_id) .map_or("Unknown", |info| info.path); info!("Received button not mapped: {button_name}"); } destination_buttons } } ================================================ FILE: alvr/server_core/src/lib.rs ================================================ mod bitrate; mod c_api; mod connection; mod hand_gestures; mod haptics; mod input_mapping; mod logging_backend; mod sockets; mod statistics; mod tracking; mod web_server; pub use c_api::*; pub use logging_backend::init_logging; pub use tracking::HandType; use crate::connection::VideoPacket; use alvr_common::{ ConnectionState, DEVICE_ID_TO_PATH, DeviceMotion, LifecycleState, Pose, RelaxedAtomic, ViewParams, dbg_server_core, error, glam::Vec2, parking_lot::{Mutex, RwLock}, settings_schema::Switch, warn, }; use alvr_events::{EventType, HapticsEvent}; use alvr_filesystem as afs; use alvr_packets::{ BatteryInfo, ButtonEntry, ClientConnectionsAction, DecoderInitializationConfig, Haptics, VideoPacketHeader, }; use alvr_server_io::ServerSessionManager; use alvr_session::{CodecType, OpenvrProperty, Settings}; use alvr_sockets::StreamSender; use bitrate::{BitrateManager, DynamicEncoderParams}; use statistics::StatisticsManager; use std::{ collections::HashSet, env, ffi::OsStr, fs::File, io::Write, sync::{ Arc, LazyLock, OnceLock, atomic::{AtomicBool, Ordering}, mpsc::{self, SyncSender, TrySendError}, }, thread::{self, JoinHandle}, time::{Duration, Instant}, }; use tokio::{runtime::Runtime, sync::broadcast}; use tracking::TrackingManager; static FILESYSTEM_LAYOUT: OnceLock = OnceLock::new(); // This is lazily initialized when initializing logging or ServerCoreContext. So FILESYSTEM_LAYOUT // needs to be initialized first using initialize_environment(). // NB: this must remain a global because only one instance should exist for the whole application // execution time. static SESSION_MANAGER: LazyLock> = LazyLock::new(|| { RwLock::new(ServerSessionManager::new( FILESYSTEM_LAYOUT.get().map(|l| l.session()), )) }); pub fn initialize_environment(layout: afs::Layout) { FILESYSTEM_LAYOUT.set(layout).unwrap(); // This ensures that the session is written to disk SESSION_MANAGER.write().session_mut(); } pub enum ServerCoreEvent { SetOpenvrProperty { device_id: u64, prop: OpenvrProperty, }, ClientConnected, ClientDisconnected, Battery(BatteryInfo), PlayspaceSync(Vec2), LocalViewParams([ViewParams; 2]), // In relation to head Tracking { poll_timestamp: Duration, }, Buttons(Vec), // Note: this is after mapping RequestIDR, CaptureFrame, GameRenderLatencyFeedback(Duration), // only used for SteamVR ShutdownPending, RestartPending, ProximityState(bool), } pub struct ConnectionContext { events_sender: mpsc::Sender, statistics_manager: RwLock>, bitrate_manager: Mutex, tracking_manager: RwLock, decoder_config: Mutex>, video_mirror_sender: Mutex>>>, video_recording_file: Mutex>, connection_threads: Mutex>>, clients_to_be_removed: Mutex>, video_channel_sender: Mutex>>, haptics_sender: Mutex>>, } pub fn create_recording_file(connection_context: &ConnectionContext, settings: &Settings) { let codec = settings.video.preferred_codec; let ext = match codec { CodecType::H264 => "h264", CodecType::Hevc => "h265", CodecType::AV1 => "av1", }; let path = FILESYSTEM_LAYOUT.get().unwrap().log_dir.join(format!( "recording.{}.{ext}", chrono::Local::now().format("%F.%H-%M-%S") )); match File::create(path) { Ok(mut file) => { if let Some(config) = &*connection_context.decoder_config.lock() { file.write_all(&config.config_buffer).ok(); } *connection_context.video_recording_file.lock() = Some(file); connection_context .events_sender .send(ServerCoreEvent::RequestIDR) .ok(); } Err(e) => { error!("Failed to record video on disk: {e}"); } } } pub fn notify_restart_driver() { if sysinfo::System::new_all() .processes_by_name(OsStr::new(&afs::dashboard_fname())) .next() .is_some() { alvr_events::send_event(EventType::ServerRequestsSelfRestart); } else { error!("Cannot restart SteamVR. No dashboard process found on local device."); } } pub fn settings() -> Settings { SESSION_MANAGER.read().settings().clone() } pub fn registered_button_set() -> HashSet { let session_manager = SESSION_MANAGER.read(); if let Switch::Enabled(input_mapping) = &session_manager.settings().headset.controllers { input_mapping::registered_button_set(&input_mapping.emulation_mode) } else { HashSet::new() } } pub struct ServerCoreContext { lifecycle_state: Arc>, is_restarting: RelaxedAtomic, connection_context: Arc, connection_thread: Arc>>>, webserver_runtime: Option, } impl ServerCoreContext { pub fn new() -> (Self, mpsc::Receiver) { dbg_server_core!("Creating"); if SESSION_MANAGER .read() .settings() .extra .logging .prefer_backtrace { unsafe { env::set_var("RUST_BACKTRACE", "1") }; } SESSION_MANAGER.write().clean_client_list(); let (events_sender, events_receiver) = mpsc::channel(); // Create a temporary StatisticsManager until a headset connects let initial_settings = SESSION_MANAGER.read().settings().clone(); let stats = StatisticsManager::new( initial_settings.connection.statistics_history_size, Duration::from_secs_f32(1.0 / 90.0), if let Switch::Enabled(config) = &initial_settings.headset.controllers { config.steamvr_pipeline_frames } else { 0.0 }, ); let connection_context = Arc::new(ConnectionContext { events_sender, statistics_manager: RwLock::new(Some(stats)), bitrate_manager: Mutex::new(BitrateManager::new(256, 60.0)), tracking_manager: RwLock::new(TrackingManager::new( initial_settings.connection.statistics_history_size, )), decoder_config: Mutex::new(None), video_mirror_sender: Mutex::new(None), video_recording_file: Mutex::new(None), connection_threads: Mutex::new(Vec::new()), clients_to_be_removed: Mutex::new(HashSet::new()), video_channel_sender: Mutex::new(None), haptics_sender: Mutex::new(None), }); let webserver_runtime = Runtime::new().unwrap(); webserver_runtime.spawn({ let connection_context = Arc::clone(&connection_context); async move { alvr_common::show_err(web_server::web_server(connection_context).await) } }); ( Self { lifecycle_state: Arc::new(RwLock::new(LifecycleState::StartingUp)), is_restarting: RelaxedAtomic::new(false), connection_context, connection_thread: Arc::new(RwLock::new(None)), webserver_runtime: Some(webserver_runtime), }, events_receiver, ) } pub fn start_connection(&self) { dbg_server_core!("start_connection"); // Note: Idle state is not used on the server side *self.lifecycle_state.write() = LifecycleState::Resumed; let connection_context = Arc::clone(&self.connection_context); let lifecycle_state = Arc::clone(&self.lifecycle_state); *self.connection_thread.write() = Some(thread::spawn(move || { connection::handshake_loop(connection_context, lifecycle_state); })); } pub fn get_device_motion( &self, device_id: u64, sample_timestamp: Duration, ) -> Option { dbg_server_core!("get_device_motion: dev={device_id} sample_ts={sample_timestamp:?}"); self.connection_context .tracking_manager .read() .get_device_motion(device_id, sample_timestamp) } pub fn get_hand_skeleton( &self, hand_type: HandType, timestamp: Duration, ) -> Option<[Pose; 26]> { dbg_server_core!("get_hand_skeleton: hand={hand_type:?} ts={timestamp:?}"); self.connection_context .tracking_manager .read() .get_hand_skeleton(hand_type, timestamp) .copied() } pub fn get_motion_to_photon_latency(&self) -> Duration { dbg_server_core!("get_motion_to_photon_latency"); let latency = self .connection_context .statistics_manager .read() .as_ref() .map(|stats| stats.motion_to_photon_latency_average()) .unwrap_or_default(); let max_prediction = Duration::from_millis(SESSION_MANAGER.read().settings().headset.max_prediction_ms); if latency > max_prediction { warn!("Latency is too high. Clamping prediction"); max_prediction } else { latency } } pub fn get_tracker_pose_time_offset(&self) -> Duration { dbg_server_core!("get_tracker_pose_time_offset"); self.connection_context .statistics_manager .read() .as_ref() .map(|stats| stats.tracker_pose_time_offset()) .unwrap_or_default() } pub fn send_haptics(&self, haptics: Haptics) { dbg_server_core!("send_haptics"); let haptics_config = { let session_manager_lock = SESSION_MANAGER.read(); if session_manager_lock.settings().extra.logging.log_haptics { alvr_events::send_event(EventType::Haptics(HapticsEvent { path: DEVICE_ID_TO_PATH.get(&haptics.device_id).map_or_else( || format!("Unknown (ID: {:#16x})", haptics.device_id), |p| (*p).to_owned(), ), duration: haptics.duration, frequency: haptics.frequency, amplitude: haptics.amplitude, })) } session_manager_lock .settings() .headset .controllers .as_option() .and_then(|c| c.haptics.as_option().cloned()) }; if let (Some(config), Some(sender)) = ( haptics_config, &mut *self.connection_context.haptics_sender.lock(), ) { sender .send_header(&haptics::map_haptics(&config, haptics)) .ok(); } } pub fn set_video_config_nals(&self, config_buffer: Vec, codec: CodecType) { dbg_server_core!("set_video_config_nals"); if let Some(sender) = &*self.connection_context.video_mirror_sender.lock() { sender.send(config_buffer.clone()).ok(); } if let Some(file) = &mut *self.connection_context.video_recording_file.lock() { file.write_all(&config_buffer).ok(); } *self.connection_context.decoder_config.lock() = Some(DecoderInitializationConfig { codec, config_buffer, ext_str: String::new(), }); } pub fn send_video_nal( &self, timestamp: Duration, global_view_params: [ViewParams; 2], is_idr: bool, nal_buffer: Vec, ) { dbg_server_core!("send_video_nal"); // start in the corrupts state, the client didn't receive the initial IDR yet. static STREAM_CORRUPTED: AtomicBool = AtomicBool::new(true); static LAST_IDR_INSTANT: LazyLock> = LazyLock::new(|| Mutex::new(Instant::now())); if let Some(sender) = &*self.connection_context.video_channel_sender.lock() { let buffer_size = nal_buffer.len(); if is_idr { STREAM_CORRUPTED.store(false, Ordering::SeqCst); } if let Switch::Enabled(config) = &SESSION_MANAGER .read() .settings() .extra .capture .rolling_video_files && Instant::now() > *LAST_IDR_INSTANT.lock() + Duration::from_secs(config.duration_s) { self.connection_context .events_sender .send(ServerCoreEvent::RequestIDR) .ok(); if is_idr { create_recording_file( &self.connection_context, SESSION_MANAGER.read().settings(), ); *LAST_IDR_INSTANT.lock() = Instant::now(); } } if !STREAM_CORRUPTED.load(Ordering::SeqCst) || !SESSION_MANAGER .read() .settings() .connection .avoid_video_glitching { if let Some(sender) = &*self.connection_context.video_mirror_sender.lock() { sender.send(nal_buffer.clone()).ok(); } if let Some(file) = &mut *self.connection_context.video_recording_file.lock() { file.write_all(&nal_buffer).ok(); } let sender_result = sender.try_send(VideoPacket { header: VideoPacketHeader { timestamp, global_view_params, is_idr, }, payload: nal_buffer, }); if matches!(sender_result, Err(TrySendError::Full(_))) { STREAM_CORRUPTED.store(true, Ordering::SeqCst); self.connection_context .events_sender .send(ServerCoreEvent::RequestIDR) .ok(); warn!("Dropping video packet. Reason: Can't push to network"); } } else { warn!("Dropping video packet. Reason: Waiting for IDR frame"); } if let Some(stats) = &mut *self.connection_context.statistics_manager.write() { let encoder_latency = stats.report_frame_encoded(timestamp, buffer_size); self.connection_context .bitrate_manager .lock() .report_frame_encoded(timestamp, encoder_latency, buffer_size); } } } pub fn get_dynamic_encoder_params(&self) -> Option { dbg_server_core!("get_dynamic_encoder_params"); let pair = { let session_manager_lock = SESSION_MANAGER.read(); self.connection_context .bitrate_manager .lock() .get_encoder_params(&session_manager_lock.settings().video.bitrate) }; if let Some((params, stats)) = pair { if let Some(stats_manager) = &mut *self.connection_context.statistics_manager.write() { stats_manager.report_throughput_stats(stats); } Some(params) } else { None } } pub fn report_composed(&self, target_timestamp: Duration, offset: Duration) { dbg_server_core!("report_composed"); if let Some(stats) = &mut *self.connection_context.statistics_manager.write() { stats.report_frame_composed(target_timestamp, offset); } } pub fn report_present(&self, target_timestamp: Duration, offset: Duration) { dbg_server_core!("report_present"); if let Some(stats) = &mut *self.connection_context.statistics_manager.write() { stats.report_frame_present(target_timestamp, offset); } let session_manager_lock = SESSION_MANAGER.read(); self.connection_context .bitrate_manager .lock() .report_frame_present( &session_manager_lock .settings() .video .bitrate .adapt_to_framerate, ); } pub fn duration_until_next_vsync(&self) -> Option { dbg_server_core!("duration_until_next_vsync"); self.connection_context .statistics_manager .write() .as_mut() .map(|stats| stats.duration_until_next_vsync()) } pub fn restart(self) { dbg_server_core!("restart"); self.is_restarting.set(true); // drop is called here for self } } impl Drop for ServerCoreContext { fn drop(&mut self) { dbg_server_core!("Drop"); // Invoke connection runtimes shutdown *self.lifecycle_state.write() = LifecycleState::ShuttingDown; dbg_server_core!("Setting clients as Disconnecting"); { let mut session_manager_lock = SESSION_MANAGER.write(); let hostnames = session_manager_lock .client_list() .iter() .filter(|&(_, info)| { !matches!( info.connection_state, ConnectionState::Disconnected | ConnectionState::Disconnecting ) }) .map(|(hostname, _)| hostname.clone()) .collect::>(); for hostname in hostnames { session_manager_lock.update_client_connections( hostname, ClientConnectionsAction::SetConnectionState(ConnectionState::Disconnecting), ); } } dbg_server_core!("Joining connection thread"); if let Some(thread) = self.connection_thread.write().take() { thread.join().ok(); } // apply openvr config for the next launch dbg_server_core!("Setting restart settings chache"); { let mut session_manager_lock = SESSION_MANAGER.write(); session_manager_lock.session_mut().openvr_config = connection::contruct_openvr_config(session_manager_lock.session()); } // todo: check if this is still needed while SESSION_MANAGER .read() .client_list() .iter() .any(|(_, info)| info.connection_state != ConnectionState::Disconnected) { thread::sleep(Duration::from_millis(100)); } // Dropping the webserver runtime is bugged on linux and will prevent StemVR shutdown if !cfg!(target_os = "linux") { self.webserver_runtime.take(); } } } ================================================ FILE: alvr/server_core/src/logging_backend.rs ================================================ use crate::SESSION_MANAGER; use alvr_common::{LogEntry, LogSeverity, log::LevelFilter}; use alvr_events::{Event, EventType}; use chrono::Local; use fern::Dispatch; use std::{fs, path::PathBuf, sync::LazyLock}; use tokio::sync::broadcast; static CHANNEL_CAPACITY: usize = 256; pub static EVENTS_SENDER: LazyLock> = LazyLock::new(|| broadcast::channel(CHANNEL_CAPACITY).0); pub fn init_logging(session_log_path: Option, crash_log_path: Option) { let debug_groups_config = SESSION_MANAGER .read() .settings() .extra .logging .debug_groups .clone(); let mut log_dispatch = Dispatch::new() // Note: meta::target() is in the format :: .filter({ let debug_groups_config = debug_groups_config.clone(); move |meta| { !meta.target().starts_with("mdns_sd") && (meta.level() <= LevelFilter::Info || alvr_common::filter_debug_groups(meta.target(), &debug_groups_config)) } }) .format(move |out, message, record| { let maybe_event = format!("{message}"); let event_type = if maybe_event.starts_with('{') && maybe_event.ends_with('}') { serde_json::from_str(&maybe_event).unwrap() } else if record.level() == LevelFilter::Debug && alvr_common::is_enabled_debug_group(record.target(), &debug_groups_config) { EventType::DebugGroup { group: record.target().to_string(), message: message.to_string(), } } else { EventType::Log(LogEntry { severity: LogSeverity::from_log_level(record.level()), content: message.to_string(), }) }; let event = Event { timestamp: Local::now().format("%H:%M:%S.%3f").to_string(), event_type, }; out.finish(format_args!( "{} [{}] {}", event.timestamp, event.event_type_string(), event.message(), )); EVENTS_SENDER.send(event).ok(); }); if cfg!(debug_assertions) { log_dispatch = log_dispatch.level(LevelFilter::Debug) } else { log_dispatch = log_dispatch.level(LevelFilter::Info); } log_dispatch = if let Some(path) = session_log_path { log_dispatch.chain( fs::OpenOptions::new() .write(true) .create(true) .truncate(true) .open(path) .unwrap(), ) } else if cfg!(target_os = "linux") { // this sink is required to make sure all log gets processed and forwarded to the websocket log_dispatch.chain( fs::OpenOptions::new() .write(true) .open("/dev/null") .unwrap(), ) } else { log_dispatch.chain(std::io::stdout()) }; log_dispatch = if let Some(path) = crash_log_path { log_dispatch.chain( Dispatch::new() .level(LevelFilter::Error) .chain(fern::log_file(path).unwrap()), ) } else if cfg!(target_os = "linux") { log_dispatch.chain( fs::OpenOptions::new() .write(true) .open("/dev/null") .unwrap(), ) } else { log_dispatch.chain(std::io::stderr()) }; log_dispatch.apply().unwrap(); fn popup_callback(title: &str, message: &str, severity: LogSeverity) { let level = match severity { LogSeverity::Error => rfd::MessageLevel::Error, LogSeverity::Warning => rfd::MessageLevel::Warning, LogSeverity::Info | LogSeverity::Debug => rfd::MessageLevel::Info, }; rfd::MessageDialog::new() .set_title(title) .set_description(message) .set_level(level) .show(); } alvr_common::set_popup_callback(popup_callback); alvr_common::set_panic_hook(); } ================================================ FILE: alvr/server_core/src/sockets.rs ================================================ use alvr_common::{ ToAny, anyhow::{Result, bail}, warn, }; use flume::TryRecvError; use mdns_sd::{Receiver, ServiceDaemon, ServiceEvent}; use std::{collections::HashMap, net::IpAddr}; pub struct WelcomeSocket { mdns_receiver: Receiver, } impl WelcomeSocket { pub fn new() -> Result { let mdns_receiver = ServiceDaemon::new()?.browse(alvr_sockets::MDNS_SERVICE_TYPE)?; Ok(Self { mdns_receiver }) } // Returns: client IP, client hostname pub fn recv_all(&self) -> Result> { let mut clients = HashMap::new(); loop { match self.mdns_receiver.try_recv() { Ok(event) => { if let ServiceEvent::ServiceResolved(info) = event { let hostname = info .get_property_val_str(alvr_sockets::MDNS_DEVICE_ID_KEY) .unwrap_or_else(|| info.get_hostname()); let address = *info.get_addresses().iter().next().to_any()?; let client_protocol = info .get_property_val_str(alvr_sockets::MDNS_PROTOCOL_KEY) .to_any()?; let server_protocol = alvr_common::protocol_id(); let client_is_dev = client_protocol.contains("-dev"); let server_is_dev = server_protocol.contains("-dev"); if client_protocol != server_protocol { let reason = if client_is_dev && server_is_dev { "Please use matching nightly versions." } else if client_is_dev { "Please use nightly server or stable client." } else if server_is_dev { "Please use stable server or nightly client." } else { "Please use matching stable versions." }; let protocols = format!( "Protocols: server={server_protocol}, client={client_protocol}" ); warn!("Found incompatible client {hostname}! {reason}\n{protocols}"); } clients.insert(hostname.into(), address); } } Err(TryRecvError::Empty) => break, Err(e) => bail!(e), } } Ok(clients) } } ================================================ FILE: alvr/server_core/src/statistics.rs ================================================ use alvr_common::{HEAD_ID, SlidingWindowAverage}; use alvr_events::{BitrateDirectives, EventType, GraphStatistics, StatisticsSummary}; use alvr_packets::ClientStatistics; use std::{ collections::{HashMap, VecDeque}, time::{Duration, Instant}, }; const FULL_REPORT_INTERVAL: Duration = Duration::from_millis(500); const EPS_INTERVAL: Duration = Duration::from_micros(1); pub struct HistoryFrame { target_timestamp: Duration, tracking_received: Instant, frame_present: Instant, frame_composed: Instant, frame_encoded: Instant, video_packet_bytes: usize, total_pipeline_latency: Duration, } impl Default for HistoryFrame { fn default() -> Self { let now = Instant::now(); Self { target_timestamp: Duration::ZERO, tracking_received: now, frame_present: now, frame_composed: now, frame_encoded: now, video_packet_bytes: 0, total_pipeline_latency: Duration::ZERO, } } } #[derive(Default, Clone)] struct BatteryData { gauge_value: f32, is_plugged: bool, } pub struct StatisticsManager { history_buffer: VecDeque, max_history_size: usize, last_full_report_instant: Instant, last_frame_present_instant: Instant, last_frame_present_interval: Duration, video_packets_total: usize, video_packets_partial_sum: usize, video_bytes_total: usize, video_bytes_partial_sum: usize, battery_gauges: HashMap, steamvr_pipeline_latency: Duration, motion_to_photon_latency_average: SlidingWindowAverage, last_vsync_time: Instant, frame_interval: Duration, last_throughput_directives: BitrateDirectives, } impl StatisticsManager { // history size used to calculate average total pipeline latency pub fn new( max_history_size: usize, nominal_server_frame_interval: Duration, steamvr_pipeline_frames: f32, ) -> Self { Self { history_buffer: VecDeque::new(), max_history_size, last_full_report_instant: Instant::now(), last_frame_present_instant: Instant::now(), last_frame_present_interval: Duration::ZERO, video_packets_total: 0, video_packets_partial_sum: 0, video_bytes_total: 0, video_bytes_partial_sum: 0, battery_gauges: HashMap::new(), steamvr_pipeline_latency: Duration::from_secs_f32( steamvr_pipeline_frames * nominal_server_frame_interval.as_secs_f32(), ), motion_to_photon_latency_average: SlidingWindowAverage::new( Duration::ZERO, max_history_size, ), last_vsync_time: Instant::now(), frame_interval: nominal_server_frame_interval, last_throughput_directives: BitrateDirectives::default(), } } pub fn report_tracking_received(&mut self, target_timestamp: Duration) { if !self .history_buffer .iter() .any(|frame| frame.target_timestamp == target_timestamp) { self.history_buffer.push_front(HistoryFrame { target_timestamp, tracking_received: Instant::now(), ..Default::default() }); } if self.history_buffer.len() > self.max_history_size { self.history_buffer.pop_back(); } } pub fn report_frame_present(&mut self, target_timestamp: Duration, offset: Duration) { if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.target_timestamp == target_timestamp) { let now = Instant::now() - offset; self.last_frame_present_interval = now.saturating_duration_since(self.last_frame_present_instant); self.last_frame_present_instant = now; frame.frame_present = now; } } pub fn report_frame_composed(&mut self, target_timestamp: Duration, offset: Duration) { if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.target_timestamp == target_timestamp) { frame.frame_composed = Instant::now() - offset; } } // returns encoding interval pub fn report_frame_encoded( &mut self, target_timestamp: Duration, bytes_count: usize, ) -> Duration { self.video_packets_total += 1; self.video_packets_partial_sum += 1; self.video_bytes_total += bytes_count; self.video_bytes_partial_sum += bytes_count; if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.target_timestamp == target_timestamp) { frame.frame_encoded = Instant::now(); frame.video_packet_bytes = bytes_count; frame .frame_encoded .saturating_duration_since(frame.frame_composed) } else { Duration::ZERO } } pub fn report_battery(&mut self, device_id: u64, gauge_value: f32, is_plugged: bool) { *self.battery_gauges.entry(device_id).or_default() = BatteryData { gauge_value, is_plugged, }; } pub fn report_throughput_stats(&mut self, stats: BitrateDirectives) { self.last_throughput_directives = stats; } // Called every frame. Some statistics are reported once every frame // Returns (network latency, game time latency) pub fn report_statistics(&mut self, client_stats: ClientStatistics) -> (Duration, Duration) { self.motion_to_photon_latency_average .submit_sample(client_stats.total_pipeline_latency); if let Some(frame) = self .history_buffer .iter_mut() .find(|frame| frame.target_timestamp == client_stats.target_timestamp) { frame.total_pipeline_latency = client_stats.total_pipeline_latency; let game_time_latency = frame .frame_present .saturating_duration_since(frame.tracking_received); let server_compositor_latency = frame .frame_composed .saturating_duration_since(frame.frame_present); let encoder_latency = frame .frame_encoded .saturating_duration_since(frame.frame_composed); // The network latency cannot be estiamed directly. It is what's left of the total // latency after subtracting all other latency intervals. In particular it contains the // transport latency of the tracking packet and the interval between the first video // packet is sent and the last video packet is received for a specific frame. // For safety, use saturating_sub to avoid a crash if for some reason the network // latency is miscalculated as negative. let network_latency = frame.total_pipeline_latency.saturating_sub( game_time_latency + server_compositor_latency + encoder_latency + client_stats.video_decode + client_stats.video_decoder_queue + client_stats.rendering + client_stats.vsync_queue, ); let client_fps = 1.0 / Duration::max(client_stats.frame_interval, EPS_INTERVAL).as_secs_f32(); let server_fps = 1.0 / Duration::max(self.last_frame_present_interval, EPS_INTERVAL).as_secs_f32(); if self.last_full_report_instant + FULL_REPORT_INTERVAL < Instant::now() { self.last_full_report_instant += FULL_REPORT_INTERVAL; let interval_secs = FULL_REPORT_INTERVAL.as_secs_f32(); alvr_events::send_event(EventType::StatisticsSummary(StatisticsSummary { video_packets_total: self.video_packets_total, video_packets_per_sec: (self.video_packets_partial_sum as f32 / interval_secs) as _, video_mbytes_total: (self.video_bytes_total as f32 / 1e6) as usize, video_mbits_per_sec: self.video_bytes_partial_sum as f32 * 8. / 1e6 / interval_secs, total_latency_ms: client_stats.total_pipeline_latency.as_secs_f32() * 1000., network_latency_ms: network_latency.as_secs_f32() * 1000., encode_latency_ms: encoder_latency.as_secs_f32() * 1000., decode_latency_ms: client_stats.video_decode.as_secs_f32() * 1000., client_fps: client_fps as _, server_fps: server_fps as _, battery_hmd: (self .battery_gauges .get(&HEAD_ID) .cloned() .unwrap_or_default() .gauge_value * 100.) as u32, hmd_plugged: self .battery_gauges .get(&HEAD_ID) .cloned() .unwrap_or_default() .is_plugged, })); self.video_packets_partial_sum = 0; self.video_bytes_partial_sum = 0; } let packet_bits = frame.video_packet_bytes as f32 * 8.0; let throughput_bps = packet_bits / Duration::max(network_latency, EPS_INTERVAL).as_secs_f32(); let bitrate_bps = packet_bits / Duration::max(self.last_frame_present_interval, EPS_INTERVAL).as_secs_f32(); // todo: use target timestamp in nanoseconds. the dashboard needs to use the first // timestamp as the graph time origin. alvr_events::send_event(EventType::GraphStatistics(GraphStatistics { total_pipeline_latency_s: client_stats.total_pipeline_latency.as_secs_f32(), game_time_s: game_time_latency.as_secs_f32(), server_compositor_s: server_compositor_latency.as_secs_f32(), encoder_s: encoder_latency.as_secs_f32(), network_s: network_latency.as_secs_f32(), decoder_s: client_stats.video_decode.as_secs_f32(), decoder_queue_s: client_stats.video_decoder_queue.as_secs_f32(), client_compositor_s: client_stats.rendering.as_secs_f32(), vsync_queue_s: client_stats.vsync_queue.as_secs_f32(), client_fps, server_fps, bitrate_directives: self.last_throughput_directives.clone(), throughput_bps, bitrate_bps, })); (network_latency, game_time_latency) } else { (Duration::ZERO, Duration::ZERO) } } pub fn motion_to_photon_latency_average(&self) -> Duration { self.motion_to_photon_latency_average.get_average() } pub fn tracker_pose_time_offset(&self) -> Duration { // This is the opposite of the client's StatisticsManager::tracker_prediction_offset(). self.steamvr_pipeline_latency } // NB: this call is non-blocking, waiting should be done externally pub fn duration_until_next_vsync(&mut self) -> Duration { let now = Instant::now(); // update the last vsync if it's too old while self.last_vsync_time + self.frame_interval < now { self.last_vsync_time += self.frame_interval; } (self.last_vsync_time + self.frame_interval).saturating_duration_since(now) } } ================================================ FILE: alvr/server_core/src/tracking/body.rs ================================================ use alvr_common::{ BODY_CHEST_ID, BODY_HIPS_ID, BODY_LEFT_ELBOW_ID, BODY_LEFT_FOOT_ID, BODY_LEFT_KNEE_ID, BODY_RIGHT_ELBOW_ID, BODY_RIGHT_FOOT_ID, BODY_RIGHT_KNEE_ID, BodySkeleton, DETACHED_CONTROLLER_LEFT_ID, DETACHED_CONTROLLER_RIGHT_ID, DeviceMotion, GENERIC_TRACKER_1_ID, GENERIC_TRACKER_2_ID, GENERIC_TRACKER_3_ID, HEAD_ID, anyhow::Result, glam::Vec3, }; use alvr_session::BodyTrackingSinkConfig; use rosc::{OscMessage, OscPacket, OscType}; use std::{collections::HashMap, net::UdpSocket, sync::LazyLock}; const CHEST_FB: usize = 5; const HIPS_FB: usize = 1; const LEFT_ARM_LOWER_FB: usize = 11; const RIGHT_ARM_LOWER_FB: usize = 16; const LEFT_LOWER_LEG_META: usize = 1; const LEFT_FOOT_BALL_META: usize = 6; const RIGHT_LOWER_LEG_META: usize = 8; const RIGHT_FOOT_BALL_META: usize = 13; const PELVIS_BD: usize = 0; const LEFT_KNEE_BD: usize = 4; const RIGHT_KNEE_BD: usize = 5; const SPINE3_BD: usize = 9; const LEFT_FOOT_BD: usize = 10; const RIGHT_FOOT_BD: usize = 11; const LEFT_ELBOW_BD: usize = 18; const RIGHT_ELBOW_BD: usize = 19; static BODY_TRACKER_OSC_PATH_MAP: LazyLock> = LazyLock::new(|| { HashMap::from([ (*HEAD_ID, "/tracking/trackers/head/"), (*BODY_CHEST_ID, "/tracking/trackers/1/"), (*BODY_HIPS_ID, "/tracking/trackers/2/"), (*BODY_LEFT_ELBOW_ID, "/tracking/trackers/3/"), (*BODY_RIGHT_ELBOW_ID, "/tracking/trackers/4/"), (*BODY_LEFT_KNEE_ID, "/tracking/trackers/5/"), (*BODY_LEFT_FOOT_ID, "/tracking/trackers/6/"), (*BODY_RIGHT_KNEE_ID, "/tracking/trackers/7/"), (*BODY_RIGHT_FOOT_ID, "/tracking/trackers/8/"), ]) }); pub struct BodyTrackingSink { config: BodyTrackingSinkConfig, socket: Option, } impl BodyTrackingSink { pub fn new(config: BodyTrackingSinkConfig, local_osc_port: u16) -> Result { match config { BodyTrackingSinkConfig::VrchatBodyOsc { port } => { let socket = UdpSocket::bind(format!("127.0.0.1:{local_osc_port}"))?; socket.connect(format!("127.0.0.1:{port}"))?; Ok(Self { config, socket: Some(socket), }) } BodyTrackingSinkConfig::FakeViveTracker => Ok(Self { config, socket: None, }), } } fn send_osc_message(&self, path: &str, args: Vec) { if let Some(socket) = &self.socket { socket .send( &rosc::encoder::encode(&OscPacket::Message(OscMessage { addr: path.into(), args, })) .unwrap(), ) .ok(); } } pub fn send_tracking(&self, device_motions: &[(u64, DeviceMotion)]) { match self.config { BodyTrackingSinkConfig::VrchatBodyOsc { .. } => { for (id, motion) in device_motions { if BODY_TRACKER_OSC_PATH_MAP.contains_key(id) { // Only do position because rotation isn't quite right let position = motion.pose.position; self.send_osc_message( format!( "{}{}", BODY_TRACKER_OSC_PATH_MAP.get(id).unwrap(), "position" ) .as_str(), vec![ OscType::Float(position.x), OscType::Float(position.y), OscType::Float(-position.z), ], ); } } } BodyTrackingSinkConfig::FakeViveTracker => {} } } } pub fn get_default_body_trackers_from_detached_controllers( device_motions: &[(u64, DeviceMotion)], ) -> Vec<(u64, DeviceMotion)> { let mut poses = Vec::new(); for (id, motion) in device_motions { if *id == *DETACHED_CONTROLLER_LEFT_ID { poses.push((*BODY_LEFT_ELBOW_ID, *motion)); } else if *id == *DETACHED_CONTROLLER_RIGHT_ID { poses.push((*BODY_RIGHT_ELBOW_ID, *motion)); } } poses } // TODO: make this customizable pub fn get_default_body_trackers_from_motion_trackers_bd( device_motions: &[(u64, DeviceMotion)], ) -> Vec<(u64, DeviceMotion)> { let mut poses = Vec::new(); for (id, motion) in device_motions { if *id == *GENERIC_TRACKER_1_ID { poses.push((*BODY_HIPS_ID, *motion)); } else if *id == *GENERIC_TRACKER_2_ID { poses.push((*BODY_LEFT_FOOT_ID, *motion)); } else if *id == *GENERIC_TRACKER_3_ID { poses.push((*BODY_RIGHT_FOOT_ID, *motion)); } } poses } // Obtain predefined joints as trackers // TODO: make this customizable pub fn extract_default_trackers(skeleton: &BodySkeleton) -> Vec<(u64, DeviceMotion)> { let mut poses = Vec::new(); match skeleton { BodySkeleton::Fb(skeleton) => { if let Some(pose) = skeleton.upper_body[CHEST_FB] { poses.push((*BODY_CHEST_ID, pose)); } if let Some(pose) = skeleton.upper_body[HIPS_FB] { poses.push((*BODY_HIPS_ID, pose)); } if let Some(pose) = skeleton.upper_body[LEFT_ARM_LOWER_FB] { poses.push((*BODY_LEFT_ELBOW_ID, pose)); } if let Some(pose) = skeleton.upper_body[RIGHT_ARM_LOWER_FB] { poses.push((*BODY_RIGHT_ELBOW_ID, pose)); } if let Some(lower_body) = skeleton.lower_body { if let Some(pose) = lower_body[LEFT_LOWER_LEG_META] { poses.push((*BODY_LEFT_KNEE_ID, pose)); } if let Some(pose) = lower_body[LEFT_FOOT_BALL_META] { poses.push((*BODY_LEFT_FOOT_ID, pose)); } if let Some(pose) = lower_body[RIGHT_LOWER_LEG_META] { poses.push((*BODY_RIGHT_KNEE_ID, pose)); } if let Some(pose) = lower_body[RIGHT_FOOT_BALL_META] { poses.push((*BODY_RIGHT_FOOT_ID, pose)); } } } BodySkeleton::Bd(skeleton) => { if let Some(pose) = skeleton.0[SPINE3_BD] { poses.push((*BODY_HIPS_ID, pose)); } if let Some(pose) = skeleton.0[PELVIS_BD] { poses.push((*BODY_CHEST_ID, pose)); } if let Some(pose) = skeleton.0[LEFT_ELBOW_BD] { poses.push((*BODY_LEFT_ELBOW_ID, pose)); } if let Some(pose) = skeleton.0[RIGHT_ELBOW_BD] { poses.push((*BODY_RIGHT_ELBOW_ID, pose)); } if let Some(pose) = skeleton.0[LEFT_KNEE_BD] { poses.push((*BODY_LEFT_KNEE_ID, pose)); } if let Some(pose) = skeleton.0[LEFT_FOOT_BD] { poses.push((*BODY_LEFT_FOOT_ID, pose)); } if let Some(pose) = skeleton.0[RIGHT_KNEE_BD] { poses.push((*BODY_RIGHT_KNEE_ID, pose)); } if let Some(pose) = skeleton.0[RIGHT_FOOT_BD] { poses.push((*BODY_RIGHT_FOOT_ID, pose)); } } } poses .iter() .map(|(id, pose)| { ( *id, DeviceMotion { pose: *pose, linear_velocity: Vec3::ZERO, angular_velocity: Vec3::ZERO, }, ) }) .collect() } ================================================ FILE: alvr/server_core/src/tracking/face.rs ================================================ use alvr_common::{anyhow::Result, glam::EulerRot}; use alvr_packets::{FaceData, FaceExpressions}; use alvr_session::FaceTrackingSinkConfig; use rosc::{OscMessage, OscPacket, OscType}; use std::{f32::consts::PI, net::UdpSocket}; const RAD_TO_DEG: f32 = 180.0 / PI; const VRCFT_PORT: u16 = 0xA1F7; pub struct FaceTrackingSink { config: FaceTrackingSinkConfig, socket: UdpSocket, packet_buffer: Vec, } impl FaceTrackingSink { pub fn new(config: FaceTrackingSinkConfig, local_osc_port: u16) -> Result { let port = match config { FaceTrackingSinkConfig::VrchatEyeOsc { port } => port, FaceTrackingSinkConfig::VrcFaceTracking => VRCFT_PORT, }; let socket = UdpSocket::bind(format!("127.0.0.1:{local_osc_port}"))?; socket.connect(format!("127.0.0.1:{port}"))?; Ok(Self { config, socket, packet_buffer: vec![], }) } fn send_osc_message(&self, path: &str, args: Vec) { self.socket .send( &rosc::encoder::encode(&OscPacket::Message(OscMessage { addr: path.into(), args, })) .unwrap(), ) .ok(); } fn append_packet_vrcft(&mut self, prefix: [u8; 8], data: &[f32]) { self.packet_buffer.extend(prefix); for val in data { self.packet_buffer.extend(val.to_le_bytes()); } } pub fn send_tracking(&mut self, face_data: &FaceData) { match self.config { FaceTrackingSinkConfig::VrchatEyeOsc { .. } => { if let [Some(left), Some(right)] = face_data.eyes_social { let (left_pitch, left_yaw, _) = left.to_euler(EulerRot::XYZ); let (right_pitch, right_yaw, _) = right.to_euler(EulerRot::XYZ); self.send_osc_message( "/tracking/eye/LeftRightPitchYaw", vec![ OscType::Float(-left_pitch * RAD_TO_DEG), OscType::Float(-left_yaw * RAD_TO_DEG), OscType::Float(-right_pitch * RAD_TO_DEG), OscType::Float(-right_yaw * RAD_TO_DEG), ], ); } else if let Some(quat) = face_data.eyes_combined { let (pitch, yaw, _) = quat.to_euler(EulerRot::XYZ); self.send_osc_message( "/tracking/eye/CenterPitchYaw", vec![ OscType::Float(-pitch * RAD_TO_DEG), OscType::Float(-yaw * RAD_TO_DEG), ], ); } let (left_eye_blink, right_eye_blink) = match &face_data.face_expressions { Some(FaceExpressions::Fb(items)) => (Some(items[12]), Some(items[13])), Some(FaceExpressions::Pico(items)) => (Some(items[28]), Some(items[38])), Some(FaceExpressions::Htc { eye, .. }) => { (eye.as_ref().map(|v| v[0]), eye.as_ref().map(|v| v[2])) } _ => (None, None), }; if let (Some(left), Some(right)) = (left_eye_blink, right_eye_blink) { self.send_osc_message( "/tracking/eye/EyesClosedAmount", vec![OscType::Float((left + right) / 2.0)], ); } else if let Some(blink) = left_eye_blink.or(right_eye_blink) { self.send_osc_message( "/tracking/eye/EyesClosedAmount", vec![OscType::Float(blink)], ); } } FaceTrackingSinkConfig::VrcFaceTracking => { self.packet_buffer.clear(); if let [Some(left_quat), Some(right_quat)] = face_data.eyes_social { let mut vec = left_quat.to_array().to_vec(); vec.extend_from_slice(&right_quat.to_array()); self.append_packet_vrcft(*b"EyesQuat", &vec); } else if let Some(quat) = face_data.eyes_combined { self.append_packet_vrcft(*b"CombQuat", &quat.to_array()); } match &face_data.face_expressions { Some(FaceExpressions::Fb(items)) => { self.append_packet_vrcft(*b"Face2Fb\0", items); } Some(FaceExpressions::Pico(items)) => { self.append_packet_vrcft(*b"FacePico", items); } Some(FaceExpressions::Htc { eye, lip }) => { if let Some(arr) = eye { self.append_packet_vrcft(*b"EyesHtc\0", arr); } if let Some(arr) = lip { self.append_packet_vrcft(*b"LipHtc\0\0", arr); } } None => (), } self.socket.send(&self.packet_buffer).ok(); } } } } ================================================ FILE: alvr/server_core/src/tracking/mod.rs ================================================ mod body; mod face; mod vmc; pub use body::*; pub use face::*; pub use vmc::*; use crate::{ ConnectionContext, SESSION_MANAGER, ServerCoreEvent, connection::STREAMING_RECV_TIMEOUT, hand_gestures::{self, HAND_GESTURE_BUTTON_SET, HandGestureManager}, input_mapping::ButtonMappingManager, }; use alvr_common::{ BODY_CHEST_ID, BODY_HIPS_ID, BODY_LEFT_ELBOW_ID, BODY_LEFT_FOOT_ID, BODY_LEFT_KNEE_ID, BODY_RIGHT_ELBOW_ID, BODY_RIGHT_FOOT_ID, BODY_RIGHT_KNEE_ID, ConnectionError, DEVICE_ID_TO_PATH, DeviceMotion, HAND_LEFT_ID, HAND_RIGHT_ID, HEAD_ID, Pose, ViewParams, glam::{Quat, Vec3}, parking_lot::Mutex, }; use alvr_events::{EventType, TrackingEvent}; use alvr_packets::TrackingData; use alvr_session::{ BodyTrackingConfig, HeadsetConfig, PositionRecenteringMode, RotationRecenteringMode, Settings, VMCConfig, settings_schema::Switch, }; use alvr_sockets::StreamReceiver; use std::{ cmp::Ordering, collections::{HashMap, VecDeque}, f32::consts::PI, sync::Arc, time::Duration, }; const DEG_TO_RAD: f32 = PI / 180.0; #[derive(Debug)] pub enum HandType { Left = 0, Right = 1, } // todo: Move this struct to Settings and use it for every tracked device #[derive(Default)] struct MotionConfig { // Position offset applied after rotation offset pose_offset: Pose, linear_velocity_cutoff: f32, angular_velocity_cutoff: f32, } pub struct TrackingManager { last_head_pose: Pose, // client's reference space inverse_recentering_origin: Pose, // client's reference space device_motions_history: HashMap>, hand_skeletons_history: [VecDeque<(Duration, [Pose; 26])>; 2], max_history_size: usize, } impl TrackingManager { pub fn new(max_history_size: usize) -> TrackingManager { TrackingManager { last_head_pose: Pose::IDENTITY, inverse_recentering_origin: Pose::IDENTITY, device_motions_history: HashMap::new(), hand_skeletons_history: [VecDeque::new(), VecDeque::new()], max_history_size, } } pub fn recenter( &mut self, position_recentering_mode: PositionRecenteringMode, rotation_recentering_mode: RotationRecenteringMode, ) { let position = match position_recentering_mode { PositionRecenteringMode::Disabled => Vec3::ZERO, PositionRecenteringMode::LocalFloor => { let mut pos = self.last_head_pose.position; pos.y = 0.0; pos } PositionRecenteringMode::Local { view_height } => { self.last_head_pose.position - Vec3::new(0.0, view_height, 0.0) } }; let orientation = match rotation_recentering_mode { RotationRecenteringMode::Disabled => Quat::IDENTITY, RotationRecenteringMode::Yaw => { let mut rot = self.last_head_pose.orientation; // extract yaw rotation rot.x = 0.0; rot.z = 0.0; rot = rot.normalize(); rot } RotationRecenteringMode::Tilted => self.last_head_pose.orientation, }; self.inverse_recentering_origin = Pose { position, orientation, } .inverse(); } pub fn recenter_pose(&self, pose: Pose) -> Pose { self.inverse_recentering_origin * pose } pub fn recenter_motion(&self, motion: DeviceMotion) -> DeviceMotion { self.inverse_recentering_origin * motion } // Performs all kinds of tracking transformations, driven by settings. pub fn report_device_motions( &mut self, headset_config: &HeadsetConfig, timestamp: Duration, device_motions: &[(u64, DeviceMotion)], ) { let mut device_motion_configs = HashMap::new(); device_motion_configs.insert(*HEAD_ID, MotionConfig::default()); device_motion_configs.extend([ (*BODY_CHEST_ID, MotionConfig::default()), (*BODY_HIPS_ID, MotionConfig::default()), (*BODY_LEFT_ELBOW_ID, MotionConfig::default()), (*BODY_RIGHT_ELBOW_ID, MotionConfig::default()), (*BODY_LEFT_KNEE_ID, MotionConfig::default()), (*BODY_LEFT_FOOT_ID, MotionConfig::default()), (*BODY_RIGHT_KNEE_ID, MotionConfig::default()), (*BODY_RIGHT_FOOT_ID, MotionConfig::default()), ]); if let Switch::Enabled(controllers) = &headset_config.controllers { device_motion_configs.insert( *HAND_LEFT_ID, MotionConfig { pose_offset: Pose::IDENTITY, linear_velocity_cutoff: controllers.linear_velocity_cutoff, angular_velocity_cutoff: controllers.angular_velocity_cutoff * DEG_TO_RAD, }, ); device_motion_configs.insert( *HAND_RIGHT_ID, MotionConfig { pose_offset: Pose::IDENTITY, linear_velocity_cutoff: controllers.linear_velocity_cutoff, angular_velocity_cutoff: controllers.angular_velocity_cutoff * DEG_TO_RAD, }, ); } for &(device_id, mut motion) in device_motions { if device_id == *HEAD_ID { self.last_head_pose = motion.pose; } if let Some(config) = device_motion_configs.get(&device_id) { motion = self.recenter_motion(motion); motion.pose = motion.pose * config.pose_offset; fn cutoff(v: Vec3, threshold: f32) -> Vec3 { if v.length_squared() > threshold * threshold { v } else { Vec3::ZERO } } motion.linear_velocity = cutoff(motion.linear_velocity, config.linear_velocity_cutoff); motion.angular_velocity = cutoff(motion.angular_velocity, config.angular_velocity_cutoff); } if let Some(motions) = self.device_motions_history.get_mut(&device_id) { motions.push_front((timestamp, motion)); if motions.len() > self.max_history_size { motions.pop_back(); } } else { self.device_motions_history .insert(device_id, VecDeque::from(vec![(timestamp, motion)])); } } } // If the exact sample_timestamp is not found, use the closest one if it's not older. This makes // sure that we return None if there is no newer sample and always return Some otherwise. pub fn get_device_motion( &self, device_id: u64, sample_timestamp: Duration, ) -> Option { self.device_motions_history .get(&device_id) .and_then(|motions| { // Get first element to initialize a valid motion reference if let Some((_, motion)) = motions.front() { let mut best_timestamp_diff = Duration::MAX; let mut best_motion_ref = motion; // Note: we are iterating from most recent to oldest for (ts, m) in motions { match ts.cmp(&sample_timestamp) { Ordering::Equal => return Some(*m), Ordering::Greater => { let diff = ts.saturating_sub(sample_timestamp); if diff < best_timestamp_diff { best_timestamp_diff = diff; best_motion_ref = m; } } Ordering::Less => continue, } } (best_timestamp_diff != Duration::MAX).then_some(*best_motion_ref) } else { None } }) } pub fn report_hand_skeleton( &mut self, hand_type: HandType, timestamp: Duration, mut skeleton: [Pose; 26], ) { for pose in &mut skeleton { *pose = self.recenter_pose(*pose); } let skeleton_history = &mut self.hand_skeletons_history[hand_type as usize]; skeleton_history.push_back((timestamp, skeleton)); if skeleton_history.len() > self.max_history_size { skeleton_history.pop_front(); } } pub fn get_hand_skeleton( &self, hand_type: HandType, sample_timestamp: Duration, ) -> Option<&[Pose; 26]> { self.hand_skeletons_history[hand_type as usize] .iter() .find(|(timestamp, _)| *timestamp == sample_timestamp) .map(|(_, skeleton)| skeleton) } pub fn unrecenter_view_params(&self, view_params: &mut [ViewParams; 2]) { for params in view_params { params.pose = self.inverse_recentering_origin.inverse() * params.pose; } } } pub fn tracking_loop( ctx: &ConnectionContext, initial_settings: Settings, hand_gesture_manager: Arc>, mut tracking_receiver: StreamReceiver, is_streaming: impl Fn() -> bool, ) { let mut gestures_button_mapping_manager = initial_settings .headset .controllers .as_option() .map(|config| { ButtonMappingManager::new_automatic( &HAND_GESTURE_BUTTON_SET, &config.emulation_mode, &config.button_mapping_config, ) }); let mut face_tracking_sink = initial_settings .headset .face_tracking .into_option() .and_then(|config| { FaceTrackingSink::new(config.sink, initial_settings.connection.osc_local_port).ok() }); let mut body_tracking_sink = initial_settings .headset .body_tracking .into_option() .and_then(|config| { BodyTrackingSink::new(config.sink, initial_settings.connection.osc_local_port).ok() }); let mut vmc_sink = initial_settings .headset .vmc .into_option() .and_then(|config| VMCSink::new(config).ok()); while is_streaming() { let data = match tracking_receiver.recv(STREAMING_RECV_TIMEOUT) { Ok(tracking) => tracking, Err(ConnectionError::TryAgain(_)) => continue, Err(ConnectionError::Other(_)) => return, }; let Ok(mut tracking) = data.get_header() else { return; }; let timestamp = tracking.poll_timestamp; if let Some(stats) = &mut *ctx.statistics_manager.write() { stats.report_tracking_received(timestamp); } let controllers_config = { let data_lock = SESSION_MANAGER.read(); data_lock .settings() .headset .controllers .clone() .into_option() }; let device_motion_keys = { let mut tracking_manager_lock = ctx.tracking_manager.write(); let session_manager_lock = SESSION_MANAGER.read(); let headset_config = &session_manager_lock.settings().headset; tracking.device_motions.extend_from_slice( &body::get_default_body_trackers_from_detached_controllers( &tracking.device_motions, ), ); tracking.device_motions.extend_from_slice( &body::get_default_body_trackers_from_motion_trackers_bd(&tracking.device_motions), ); if let Some(skeleton) = &tracking.body { tracking .device_motions .extend_from_slice(&body::extract_default_trackers(skeleton)); } let device_motion_keys = tracking .device_motions .iter() .map(|(id, _)| *id) .collect::>(); let velocity_multiplier = session_manager_lock.settings().extra.velocities_multiplier; tracking.device_motions.iter_mut().for_each(|(_, motion)| { motion.linear_velocity *= velocity_multiplier; motion.angular_velocity *= velocity_multiplier; }); tracking_manager_lock.report_device_motions( headset_config, timestamp, &tracking.device_motions, ); if let Some(skeleton) = tracking.hand_skeletons[0] { tracking_manager_lock.report_hand_skeleton(HandType::Left, timestamp, skeleton); } if let Some(skeleton) = tracking.hand_skeletons[1] { tracking_manager_lock.report_hand_skeleton(HandType::Right, timestamp, skeleton); } if let Some(sink) = &mut face_tracking_sink { sink.send_tracking(&tracking.face); } if session_manager_lock.settings().extra.logging.log_tracking { let device_motions = device_motion_keys .iter() .filter_map(move |id| { Some(( (*DEVICE_ID_TO_PATH.get(id)?).into(), tracking_manager_lock .get_device_motion(*id, timestamp) .unwrap(), )) }) .collect::>(); alvr_events::send_event(EventType::Tracking(Box::new(TrackingEvent { device_motions, hand_skeletons: tracking.hand_skeletons, face: tracking.face, }))) } device_motion_keys }; // Handle hand gestures if let (Some(gestures_config), Some(gestures_button_mapping_manager)) = ( controllers_config .as_ref() .and_then(|c| c.hand_tracking_interaction.as_option()), &mut gestures_button_mapping_manager, ) { let mut hand_gesture_manager_lock = hand_gesture_manager.lock(); if !device_motion_keys.contains(&*HAND_LEFT_ID) && let Some(hand_skeleton) = tracking.hand_skeletons[0] { ctx.events_sender .send(ServerCoreEvent::Buttons( hand_gestures::trigger_hand_gesture_actions( gestures_button_mapping_manager, *HAND_LEFT_ID, &hand_gesture_manager_lock.get_active_gestures( &hand_skeleton, gestures_config, *HAND_LEFT_ID, ), gestures_config.only_touch, ), )) .ok(); } if !device_motion_keys.contains(&*HAND_RIGHT_ID) && let Some(hand_skeleton) = tracking.hand_skeletons[1] { ctx.events_sender .send(ServerCoreEvent::Buttons( hand_gestures::trigger_hand_gesture_actions( gestures_button_mapping_manager, *HAND_RIGHT_ID, &hand_gesture_manager_lock.get_active_gestures( &hand_skeleton, gestures_config, *HAND_RIGHT_ID, ), gestures_config.only_touch, ), )) .ok(); } } ctx.events_sender .send(ServerCoreEvent::Tracking { poll_timestamp: tracking.poll_timestamp, }) .ok(); let publish_vmc = matches!( SESSION_MANAGER.read().settings().headset.vmc, Switch::Enabled(VMCConfig { publish: true, .. }) ); if publish_vmc { let orientation_correction = matches!( SESSION_MANAGER.read().settings().headset.vmc, Switch::Enabled(VMCConfig { orientation_correction: true, .. }) ); if let Some(sink) = &mut vmc_sink { let tracking_manager_lock = ctx.tracking_manager.read(); let device_motions = device_motion_keys .iter() .map(move |id| { ( *id, tracking_manager_lock .get_device_motion(*id, timestamp) .unwrap(), ) }) .collect::>(); if let Some(skeleton) = tracking.hand_skeletons[0] { sink.send_hand_tracking(HandType::Left, &skeleton, orientation_correction); } if let Some(skeleton) = tracking.hand_skeletons[1] { sink.send_hand_tracking(HandType::Right, &skeleton, orientation_correction); } sink.send_tracking(&device_motions, orientation_correction); } } let track_body = matches!( SESSION_MANAGER.read().settings().headset.body_tracking, Switch::Enabled(BodyTrackingConfig { tracked: true, .. }) ); if track_body && let Some(sink) = &mut body_tracking_sink { let tracking_manager_lock = ctx.tracking_manager.read(); let device_motions = device_motion_keys .iter() .map(move |id| { ( *id, tracking_manager_lock .get_device_motion(*id, timestamp) .unwrap(), ) }) .collect::>(); sink.send_tracking(&device_motions); } } } ================================================ FILE: alvr/server_core/src/tracking/vmc.rs ================================================ use crate::tracking::HandType; use alvr_common::{ BODY_CHEST_ID, BODY_HIPS_ID, BODY_LEFT_ELBOW_ID, BODY_LEFT_FOOT_ID, BODY_LEFT_KNEE_ID, BODY_RIGHT_ELBOW_ID, BODY_RIGHT_FOOT_ID, BODY_RIGHT_KNEE_ID, DeviceMotion, HAND_LEFT_ID, HAND_RIGHT_ID, HEAD_ID, Pose, anyhow::Result, glam::Quat, }; use alvr_session::VMCConfig; use rosc::{OscMessage, OscPacket, OscType}; use std::{collections::HashMap, net::UdpSocket, sync::LazyLock}; // Transform DeviceMotion into Unity HumanBodyBones // https://docs.unity3d.com/ScriptReference/HumanBodyBones.html static DEVICE_MOTIONS_VMC_MAP: LazyLock> = LazyLock::new(|| { HashMap::from([ (*HAND_LEFT_ID, "LeftHand"), (*HAND_RIGHT_ID, "RightHand"), (*BODY_CHEST_ID, "Chest"), (*BODY_HIPS_ID, "Hips"), (*BODY_LEFT_ELBOW_ID, "LeftLowerArm"), (*BODY_RIGHT_ELBOW_ID, "RightLowerArm"), (*BODY_LEFT_KNEE_ID, "LeftLowerLeg"), (*BODY_LEFT_FOOT_ID, "LeftFoot"), (*BODY_RIGHT_KNEE_ID, "RightLowerLeg"), (*BODY_RIGHT_FOOT_ID, "RightFoot"), (*HEAD_ID, "Head"), ]) }); #[expect(clippy::approx_constant)] static DEVICE_MOTIONS_ROTATION_MAP: LazyLock> = LazyLock::new(|| { HashMap::from([ ( *HAND_LEFT_ID, Quat::from_xyzw(-0.03538, 0.25483, -0.00000, -0.96634), ), ( *HAND_RIGHT_ID, Quat::from_xyzw(-0.05859, -0.20524, -0.00000, 0.97696), ), ( *BODY_CHEST_ID, Quat::from_xyzw(-0.49627, 0.49516, -0.43469, -0.56531), ), ( *BODY_HIPS_ID, Quat::from_xyzw(-0.49274, 0.49568, -0.42416, -0.57584), ), ( *BODY_RIGHT_ELBOW_ID, Quat::from_xyzw(-0.63465, -0.11567, 0.00000, 0.76410), ), ( *BODY_LEFT_KNEE_ID, Quat::from_xyzw(0.51049, 0.47862, 0.42815, -0.57185), ), ( *BODY_LEFT_FOOT_ID, Quat::from_xyzw(-0.59103, 0.38818, 0.00000, -0.70711), ), ( *BODY_RIGHT_KNEE_ID, Quat::from_xyzw(-0.52823, 0.45434, -0.58530, -0.41470), ), ( *BODY_RIGHT_FOOT_ID, Quat::from_xyzw(0.70228, -0.08246, 0.70711, 0.00000), ), ]) }); static HAND_SKELETON_VMC_MAP: [[(usize, &str); 1]; 2] = [[(0, "LeftHand")], [(0, "RightHand")]]; static HAND_SKELETON_ROTATIONS: LazyLock<[HashMap; 2]> = LazyLock::new(|| { [ HashMap::from([(0, Quat::from_xyzw(-0.03566, 0.25481, 0.00000, -0.96633))]), HashMap::from([(0, Quat::from_xyzw(-0.05880, -0.20574, -0.00000, 0.97684))]), ] }); pub struct VMCSink { socket: Option, } impl VMCSink { pub fn new(config: VMCConfig) -> Result { let socket = UdpSocket::bind("0.0.0.0:0")?; socket.connect(format!("{}:{}", config.host, config.port))?; Ok(Self { socket: Some(socket), }) } fn send_osc_message(&self, path: &str, args: Vec) { if let Some(socket) = &self.socket { socket .send( &rosc::encoder::encode(&OscPacket::Message(OscMessage { addr: path.into(), args, })) .unwrap(), ) .ok(); } } pub fn send_hand_tracking( &self, hand_type: HandType, skeleton: &[Pose; 26], orientation_correction: bool, ) { let hand_id = hand_type as usize; for (part, vmc_str) in HAND_SKELETON_VMC_MAP[hand_id] { let corrected_orientation = { let mut q = skeleton[part].orientation; if orientation_correction { if HAND_SKELETON_ROTATIONS[hand_id].contains_key(&part) { q *= *HAND_SKELETON_ROTATIONS[hand_id].get(&part).unwrap(); } q.z = -q.z; q.w = -q.w; } q }; self.send_osc_message( "/VMC/Ext/Bone/Pos", vec![ OscType::String(vmc_str.to_string()), OscType::Float(skeleton[part].position.x), OscType::Float(skeleton[part].position.y), OscType::Float(skeleton[part].position.z), OscType::Float(corrected_orientation.x), OscType::Float(corrected_orientation.y), OscType::Float(corrected_orientation.z), OscType::Float(corrected_orientation.w), ], ); } } pub fn send_tracking( &self, device_motions: &[(u64, DeviceMotion)], orientation_correction: bool, ) { for (id, motion) in device_motions { if DEVICE_MOTIONS_VMC_MAP.contains_key(id) { let corrected_orientation = { let mut q = motion.pose.orientation; if orientation_correction { if DEVICE_MOTIONS_ROTATION_MAP.contains_key(id) { q *= *DEVICE_MOTIONS_ROTATION_MAP.get(id).unwrap(); } q.z = -q.z; q.w = -q.w; } q }; self.send_osc_message( "/VMC/Ext/Bone/Pos", vec![ OscType::String((*DEVICE_MOTIONS_VMC_MAP.get(id).unwrap()).to_string()), OscType::Float(motion.pose.position.x), OscType::Float(motion.pose.position.y), OscType::Float(motion.pose.position.z), OscType::Float(corrected_orientation.x), OscType::Float(corrected_orientation.y), OscType::Float(corrected_orientation.z), OscType::Float(corrected_orientation.w), ], ); } } } } ================================================ FILE: alvr/server_core/src/web_server.rs ================================================ use crate::{ ConnectionContext, FILESYSTEM_LAYOUT, SESSION_MANAGER, ServerCoreEvent, logging_backend::EVENTS_SENDER, }; use alvr_common::{ConnectionState, LogEntry, anyhow::Result, error, info, log}; use alvr_events::{ButtonEvent, EventType}; use alvr_packets::{ButtonEntry, ClientConnectionsAction, FirewallRulesAction, PathValuePair}; use alvr_session::SessionConfig; use axum::{ Json, Router, extract::{Request, State, WebSocketUpgrade, ws::Message}, http::{ HeaderValue, Method, StatusCode, header::{CACHE_CONTROL, CONTENT_TYPE}, }, middleware, response::Response, routing, }; use serde_json as json; use std::{net::SocketAddr, path::PathBuf, sync::Arc}; use tokio::{net::TcpListener, sync::broadcast::error::RecvError}; use tower_http::{ cors::{self, CorsLayer}, set_header::SetResponseHeaderLayer, }; const X_ALVR: &str = "X-ALVR"; // This is the actual core part of cors // We require the X-ALVR header, but the browser forces a cors preflight // if the site tries to send a request with it set since it's not-whitelisted // // The dashboard can just set the header and be allowed through without the preflight // thus not getting blocked by allow_untrusted_http being disabled async fn ensure_preflight(request: Request, next: middleware::Next) -> Response { if request.headers().contains_key(X_ALVR) || request.method() == Method::OPTIONS { next.run(request).await } else { Response::builder() .status(StatusCode::BAD_REQUEST) .body(format!("missing {X_ALVR} header").into()) .unwrap() } } pub async fn web_server(connection_context: Arc) -> Result<()> { let allow_untrusted_http; let web_server_port; { let session_manager = SESSION_MANAGER.read(); allow_untrusted_http = session_manager.settings().connection.allow_untrusted_http; web_server_port = session_manager.settings().connection.web_server_port; } let mut cors = CorsLayer::new() .allow_methods([Method::GET, Method::POST]) .allow_headers([CONTENT_TYPE, X_ALVR.parse().unwrap()]); if allow_untrusted_http { cors = cors.allow_origin(cors::Any); } let router = Router::new() .nest( "/api", Router::new() .route("/events", routing::get(events_websocket)) .route("/log", routing::post(set_log)) .nest( "/session", Router::new() .route("/", routing::get(get_session).post(update_session)) .route("/values", routing::post(set_session_values)) .route( "/client-connections", routing::post(update_client_connections), ), ) .route("/buttons", routing::post(set_buttons)) .route("/insert-idr", routing::post(insert_idr)) .route("/capture-frame", routing::post(capture_frame)) .nest( "/recording", Router::new() .route("/start", routing::post(start_recording)) .route("/stop", routing::post(stop_recording)), ) .nest( "/firewall-rules", Router::new() .route("/add", routing::post(add_firewall_rules)) .route("/remove", routing::post(remove_firewall_rules)), ) .nest( "/drivers", Router::new() .route("/", routing::get(get_driver_list)) .route("/register-alvr", routing::post(register_alvr_driver)) .route("/unregister", routing::post(unregister_driver)), ) .nest( "/steamvr", Router::new() .route("/restart", routing::post(restart_steamvr)) .route("/shutdown", routing::post(shutdown_steamvr)), ) .route( "/version", routing::get(async || alvr_common::ALVR_VERSION.to_string()), ) .route("/ping", routing::get(async || ())), ) .layer(cors) .layer(SetResponseHeaderLayer::overriding( CACHE_CONTROL, HeaderValue::from_static("no-cache, no-store, must-revalidate"), )) .layer(middleware::from_fn(ensure_preflight)) .with_state(connection_context); axum::serve( TcpListener::bind(SocketAddr::new([0, 0, 0, 0].into(), web_server_port)) .await .unwrap(), router, ) .await?; Ok(()) } async fn events_websocket(ws: WebSocketUpgrade) -> Response { ws.on_upgrade(async |mut ws| { let mut events_receiver = EVENTS_SENDER.subscribe(); loop { match events_receiver.recv().await { Ok(event) => { if let Err(e) = ws .send(Message::Text(json::to_string(&event).unwrap().into())) .await { info!("Failed to send event with websocket: {e}"); break; } } Err(RecvError::Lagged(_)) => (), Err(RecvError::Closed) => break, } } }) } async fn set_log(Json(entry): Json) { let level = entry.severity.into_log_level(); log::log!(level, "{}", entry.content); } async fn get_session() { alvr_events::send_event(EventType::Session(Box::new( crate::SESSION_MANAGER.read().session().clone(), ))); } async fn update_session(Json(config): Json) { *SESSION_MANAGER.write().session_mut() = config; } async fn set_session_values(Json(descs): Json>) { SESSION_MANAGER.write().set_session_values(descs).ok(); } async fn update_client_connections( State(ctx): State>, Json((hostname, mut action)): Json<(String, ClientConnectionsAction)>, ) { let mut session_manager = SESSION_MANAGER.write(); if matches!(action, ClientConnectionsAction::RemoveEntry) && let Some(entry) = session_manager.client_list().get(&hostname) && entry.connection_state != ConnectionState::Disconnected { ctx.clients_to_be_removed.lock().insert(hostname.clone()); action = ClientConnectionsAction::SetConnectionState(ConnectionState::Disconnecting); } session_manager.update_client_connections(hostname, action); } async fn insert_idr(State(ctx): State>) { ctx.events_sender.send(ServerCoreEvent::RequestIDR).ok(); } async fn capture_frame(State(ctx): State>) { ctx.events_sender.send(ServerCoreEvent::CaptureFrame).ok(); } async fn start_recording(State(ctx): State>) { crate::create_recording_file(&ctx, crate::SESSION_MANAGER.read().settings()) } async fn stop_recording(State(ctx): State>) { *ctx.video_recording_file.lock() = None; } async fn add_firewall_rules() { if let Err(e) = alvr_server_io::firewall_rules(FirewallRulesAction::Add, FILESYSTEM_LAYOUT.get().unwrap()) { error!("Failed to add firewall rules! code: {e}"); } else { info!("Successfully added firewall rules!"); } } async fn remove_firewall_rules() { if let Err(e) = alvr_server_io::firewall_rules( FirewallRulesAction::Remove, FILESYSTEM_LAYOUT.get().unwrap(), ) { error!("Failed to remove firewall rules! code: {e}"); } else { info!("Successfully removed firewall rules!"); } } async fn get_driver_list() { if let Ok(list) = alvr_server_io::get_registered_drivers() { alvr_events::send_event(EventType::DriversList(list)); } } async fn register_alvr_driver() { alvr_server_io::driver_registration( &[FILESYSTEM_LAYOUT .get() .unwrap() .openvr_driver_root_dir .clone()], true, ) .ok(); if let Ok(list) = alvr_server_io::get_registered_drivers() { alvr_events::send_event(EventType::DriversList(list)); } } async fn unregister_driver(Json(path): Json) { alvr_server_io::driver_registration(&[path], false).ok(); if let Ok(list) = alvr_server_io::get_registered_drivers() { alvr_events::send_event(EventType::DriversList(list)); } } async fn restart_steamvr(State(ctx): State>) { ctx.events_sender.send(ServerCoreEvent::RestartPending).ok(); } async fn shutdown_steamvr(State(ctx): State>) { ctx.events_sender .send(ServerCoreEvent::ShutdownPending) .ok(); } async fn set_buttons( State(ctx): State>, Json(button_events): Json>, ) { let button_entries = button_events .iter() .map(|b| ButtonEntry { path_id: alvr_common::hash_string(&b.path), value: b.value, }) .collect(); ctx.events_sender .send(ServerCoreEvent::Buttons(button_entries)) .ok(); } ================================================ FILE: alvr/server_io/Cargo.toml ================================================ [package] name = "alvr_server_io" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_events.workspace = true alvr_filesystem.workspace = true alvr_packets.workspace = true alvr_session.workspace = true encoding_rs_io = "0.1" dirs = "6" runas = "^1.2" # version 1.1 is broken serde_json = "1" ================================================ FILE: alvr/server_io/README.md ================================================ # alvr_server_io Contains functionality for data storage and system info retrieval. Shared between server and dashboard executable. ================================================ FILE: alvr/server_io/src/firewall.rs ================================================ use crate::openvrpaths; use alvr_packets::FirewallRulesAction; use std::{ env, fs, path::{Path, PathBuf}, process::Command, }; fn netsh_add_rule_command_string(rule_name: &str, program_path: &Path) -> String { format!( "netsh advfirewall firewall add rule name=\"{}\" dir=in program=\"{}\" action=allow", rule_name, program_path.to_string_lossy() ) } fn netsh_delete_rule_command_string(rule_name: &str) -> String { format!("netsh advfirewall firewall delete rule name=\"{rule_name}\"") } // Errors: // 1: firewall rule is already set // 126: pkexec request dismissed // other: command failed pub fn firewall_rules( action: FirewallRulesAction, filesystem_layout: &alvr_filesystem::Layout, ) -> Result<(), i32> { let exit_status = if cfg!(target_os = "linux") { let action = if matches!(action, FirewallRulesAction::Add) { "add" } else { "remove" }; // run as normal user since we use pkexec to sudo Command::new("bash") .arg( PathBuf::from("../").join( filesystem_layout .firewall_script_dir .join("alvr_fw_config.sh"), ), ) .arg(action) .status() .map_err(|_| -1)? } else { let script_path = env::temp_dir().join("alvr_firewall_rules.bat"); let firewall_rules_script_content = if matches!(action, FirewallRulesAction::Add) { format!( "{}\n{}", netsh_add_rule_command_string( "SteamVR ALVR vrserver", &openvrpaths::steamvr_root_dir() .map_err(|_| -1)? .join("bin") .join("win64") .join("vrserver.exe") ), netsh_add_rule_command_string( "SteamVR ALVR vrserver", &openvrpaths::steamvr_root_dir() .map_err(|_| -1)? .join("bin") .join("win32") .join("vrserver.exe") ), ) } else { netsh_delete_rule_command_string("SteamVR ALVR vrserver") }; fs::write(&script_path, firewall_rules_script_content).map_err(|_| -1)?; // run with admin privileges runas::Command::new(script_path) .gui(true) // UAC, if available .status() .map_err(|_| -1)? }; if exit_status.success() { Ok(()) } else { Err(exit_status.code().unwrap()) } } ================================================ FILE: alvr/server_io/src/lib.rs ================================================ mod firewall; mod openvr_drivers; mod openvrpaths; pub use firewall::*; pub use openvr_drivers::*; pub use openvrpaths::*; use alvr_common::{ ConnectionState, anyhow::{Result, bail}, error, info, }; use alvr_events::EventType; use alvr_packets::{ClientConnectionsAction, PathSegment, PathValuePair}; use alvr_session::{ClientConnectionConfig, SessionConfig, Settings}; use serde_json as json; use std::{ collections::{HashMap, hash_map::Entry}, fmt::{self, Debug}, fs, ops::{Deref, DerefMut}, path::{Path, PathBuf}, }; fn save_session(session: &SessionConfig, path: &Path) -> Result<()> { fs::write(path, json::to_string_pretty(session)?)?; Ok(()) } // SessionConfig wrapper that saves session.json on destruction. pub struct SessionLock<'a> { session_desc: &'a mut SessionConfig, session_path: Option<&'a Path>, settings: &'a mut Settings, } impl Deref for SessionLock<'_> { type Target = SessionConfig; fn deref(&self) -> &SessionConfig { self.session_desc } } impl DerefMut for SessionLock<'_> { fn deref_mut(&mut self) -> &mut SessionConfig { self.session_desc } } impl Drop for SessionLock<'_> { fn drop(&mut self) { if let Some(session_path) = self.session_path { save_session(self.session_desc, session_path).ok(); } *self.settings = self.session_desc.to_settings(); alvr_events::send_event(EventType::Session(Box::new(self.session_desc.clone()))); } } // Correct usage: // SessionManager should be used behind a Mutex. Each write of the session should be preceded by a // read, within the same lock. // fixme: the dashboard is doing this wrong because it is holding its own session state. If read and // write need to happen on separate threads, a critical region should be implemented. pub struct ServerSessionManager { session_config: SessionConfig, settings: Settings, session_path: Option, } impl ServerSessionManager { pub fn new(session_path: Option) -> Self { let session_config = if let Some(session_path) = &session_path { let config_dir = session_path.parent().unwrap(); fs::create_dir_all(config_dir).ok(); Self::load_session(session_path, config_dir) } else { SessionConfig::default() }; Self { session_config: session_config.clone(), settings: session_config.to_settings(), session_path, } } fn load_session(session_path: &Path, config_dir: &Path) -> SessionConfig { let session_string = fs::read_to_string(session_path).unwrap_or_default(); if session_string.is_empty() { return SessionConfig::default(); } let session_json = json::from_str::(&session_string) .unwrap_or_else(|e| { error!( "{} {} {}\n{}", "Failed to load session.json.", "Its contents will be reset and the original file content stored as session_invalid.json.", "See error message below for details:", e ); json::Value::Null }); if session_json.is_null() { fs::write(config_dir.join("session_invalid.json"), &session_string).ok(); return SessionConfig::default(); } json::from_value(session_json.clone()).unwrap_or_else(|_| { fs::write(config_dir.join("session_old.json"), &session_string).ok(); let mut session_desc = SessionConfig::default(); match session_desc.merge_from_json(&session_json) { Ok(_) => info!( "{} {}", "Session extrapolated successfully.", "Old session.json is stored as session_old.json" ), Err(e) => error!( "{} {} {}", "Error while extrapolating session.", "Old session.json is stored as session_old.json.", e ), } // not essential, but useful to avoid duplicated errors save_session(&session_desc, session_path).ok(); session_desc }) } // prefer settings() pub fn session(&self) -> &SessionConfig { &self.session_config } pub fn session_mut(&mut self) -> SessionLock<'_> { SessionLock { session_desc: &mut self.session_config, session_path: self.session_path.as_deref(), settings: &mut self.settings, } } pub fn settings(&self) -> &Settings { &self.settings } // Note: "value" can be any session subtree, in json format. pub fn set_session_values(&mut self, descs: Vec) -> Result<()> { let mut session_json = serde_json::to_value(self.session_config.clone()).unwrap(); for desc in descs { let mut session_ref = &mut session_json; for segment in &desc.path { session_ref = match segment { PathSegment::Name(name) => { if let Some(name) = session_ref.get_mut(name) { name } else { bail!("From path {:?}: segment \"{name}\" not found", desc.path); } } PathSegment::Index(index) => { if let Some(index) = session_ref.get_mut(index) { index } else { bail!("From path {:?}: segment [{index}] not found", desc.path); } } }; } *session_ref = desc.value.clone(); } // session_json has been updated self.session_config = serde_json::from_value(session_json)?; self.settings = self.session_config.to_settings(); if let Some(session_path) = &self.session_path { save_session(&self.session_config, session_path)?; } alvr_events::send_event(EventType::Session(Box::new(self.session_config.clone()))); Ok(()) } pub fn client_list(&self) -> &HashMap { &self.session_config.client_connections } pub fn update_client_connections(&mut self, hostname: String, action: ClientConnectionsAction) { let mut client_connections = self.session_config.client_connections.clone(); let maybe_client_entry = client_connections.entry(hostname); let mut updated = false; match action { ClientConnectionsAction::AddIfMissing { trusted, manual_ips, } => { if let Entry::Vacant(new_entry) = maybe_client_entry { let client_connection_desc = ClientConnectionConfig { display_name: "Unknown".into(), current_ip: None, manual_ips: manual_ips.into_iter().collect(), trusted, connection_state: ConnectionState::Disconnected, }; new_entry.insert(client_connection_desc); updated = true; } } ClientConnectionsAction::SetDisplayName(name) => { if let Entry::Occupied(mut entry) = maybe_client_entry { entry.get_mut().display_name = name; updated = true; } } ClientConnectionsAction::Trust => { if let Entry::Occupied(mut entry) = maybe_client_entry { entry.get_mut().trusted = true; updated = true; } } ClientConnectionsAction::SetManualIps(ips) => { if let Entry::Occupied(mut entry) = maybe_client_entry { entry.get_mut().manual_ips = ips.into_iter().collect(); updated = true; } } ClientConnectionsAction::RemoveEntry => { if let Entry::Occupied(entry) = maybe_client_entry { entry.remove_entry(); updated = true; } } ClientConnectionsAction::UpdateCurrentIp(current_ip) => { if let Entry::Occupied(mut entry) = maybe_client_entry && entry.get().current_ip != current_ip { entry.get_mut().current_ip = current_ip; updated = true; } } ClientConnectionsAction::SetConnectionState(state) => { if let Entry::Occupied(mut entry) = maybe_client_entry && entry.get().connection_state != state { entry.get_mut().connection_state = state; updated = true; } } } if updated { self.session_config.client_connections = client_connections; if let Some(session_path) = &self.session_path { save_session(&self.session_config, session_path).ok(); } alvr_events::send_event(EventType::Session(Box::new(self.session_config.clone()))); } } pub fn client_hostnames(&self) -> Vec { self.session_config .client_connections .keys() .cloned() .collect() } // Run at the start of dashboard or server pub fn clean_client_list(&mut self) { let connections = self.client_list().clone(); for (hostname, connection) in connections { if connection.trusted { self.update_client_connections( hostname, ClientConnectionsAction::SetConnectionState(ConnectionState::Disconnected), ) } else { self.update_client_connections(hostname, ClientConnectionsAction::RemoveEntry); } } for hostname in self.client_hostnames() { self.update_client_connections( hostname.clone(), ClientConnectionsAction::UpdateCurrentIp(None), ); } } } impl Debug for ServerSessionManager { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{:?}", self.session_path) } } ================================================ FILE: alvr/server_io/src/openvr_drivers.rs ================================================ use crate::openvrpaths; use alvr_common::{ ToAny, anyhow::{Result, bail}, }; use serde_json as json; use std::{ collections::{HashMap, HashSet}, fs, path::PathBuf, }; pub fn get_registered_drivers() -> Result> { Ok(openvrpaths::from_openvr_paths( openvrpaths::load_openvr_paths_json()? .get_mut("external_drivers") .to_any()?, )) } pub fn driver_registration(driver_paths: &[PathBuf], register: bool) -> Result<()> { let mut openvr_paths_json = openvrpaths::load_openvr_paths_json()?; let paths_json_ref = openvr_paths_json.get_mut("external_drivers").to_any()?; let mut paths: HashSet<_> = openvrpaths::from_openvr_paths(paths_json_ref) .into_iter() .collect(); if register { paths.extend(driver_paths.iter().cloned()); } else { for path in driver_paths { paths.remove(path); } } // write into openvr_paths_json, the other fields are preserved *paths_json_ref = openvrpaths::to_openvr_paths(paths.into_iter().collect::>().as_slice()); openvrpaths::save_openvr_paths_json(&openvr_paths_json) } pub fn get_driver_dir_from_registered() -> Result { for dir in get_registered_drivers()? { let maybe_driver_name = || -> Result<_> { let manifest_string = fs::read_to_string(dir.join("driver.vrdrivermanifest"))?; let mut manifest_map = json::from_str::>(&manifest_string)?; manifest_map.remove("name").to_any() }(); if let Ok(json::Value::String(str)) = maybe_driver_name && str == "alvr_server" { return Ok(dir); } } bail!("ALVR driver path not registered") } ================================================ FILE: alvr/server_io/src/openvrpaths.rs ================================================ use alvr_common::{ ToAny, anyhow::{Result, bail}, }; use encoding_rs_io::DecodeReaderBytes; use serde_json as json; use std::{ fs::{self, File}, io::Read, path::PathBuf, }; fn openvr_source_file_path() -> Result { let path = if cfg!(windows) { dirs::cache_dir() } else { dirs::config_dir() } .to_any()? .join("openvr/openvrpaths.vrpath"); if path.exists() { Ok(path) } else { bail!("{} does not exist", path.to_string_lossy()) } } pub fn steamvr_settings_file_path() -> Result { let path = if cfg!(windows) { // N.B. if ever implementing this: given Steam can be installed on another // drive, etc., this should probably start by looking at Windows registry keys. bail!("Not implemented for Windows.") // Original motive for implementation had little reason for Windows. } else { dirs::data_dir() } .to_any()? .join("Steam/config/steamvr.vrsettings"); if path.exists() { Ok(path) } else { bail!("{} does not exist", path.to_string_lossy()) } } pub fn load_openvr_paths_json() -> Result { let file = File::open(openvr_source_file_path()?)?; let mut file_content_decoded = String::new(); DecodeReaderBytes::new(&file).read_to_string(&mut file_content_decoded)?; let value = json::from_str(&file_content_decoded)?; Ok(value) } pub fn save_openvr_paths_json(openvr_paths: &json::Value) -> Result<()> { let file_content = json::to_string_pretty(openvr_paths)?; fs::write(openvr_source_file_path()?, file_content)?; Ok(()) } pub fn from_openvr_paths(paths: &json::Value) -> Vec { let Some(paths_vec) = paths.as_array() else { return vec![]; }; paths_vec .iter() .filter_map(json::Value::as_str) .map(|s| PathBuf::from(s.replace(r"\\", r"\"))) .collect() } pub fn to_openvr_paths(paths: &[PathBuf]) -> json::Value { let paths_vec = paths .iter() .map(|p| p.to_string_lossy().into()) .map(json::Value::String) // backslashes gets duplicated here .collect::>(); json::Value::Array(paths_vec) } fn get_single_openvr_path(path_type: &str) -> Result { let openvr_paths_json = load_openvr_paths_json()?; let paths_json = openvr_paths_json.get(path_type).to_any()?; from_openvr_paths(paths_json).first().cloned().to_any() } pub fn steamvr_root_dir() -> Result { get_single_openvr_path("runtime") } ================================================ FILE: alvr/server_openvr/Cargo.toml ================================================ [package] name = "alvr_server_openvr" version.workspace = true edition.workspace = true rust-version.workspace = true authors = ["alvr-org", "Valve Corporation"] license = "MIT" [lib] crate-type = ["cdylib"] [features] gpl = [] # Enable for FFmpeg support on Windows. Always enabled on Linux [dependencies] alvr_common.workspace = true alvr_filesystem.workspace = true alvr_packets.workspace = true alvr_server_core.workspace = true alvr_server_io.workspace = true alvr_session.workspace = true serde_json = "1" sysinfo = "0.37" [build-dependencies] alvr_filesystem = { path = "../filesystem" } bindgen = "0.72" cc = { version = "1", features = ["parallel"] } walkdir = "2" [target.'cfg(target_os = "linux")'.build-dependencies] pkg-config = "0.3" ================================================ FILE: alvr/server_openvr/LICENSE ================================================ Copyright (c) 2018-2019 polygraphene Copyright (c) 2020-2024 alvr-org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: alvr/server_openvr/LICENSE-Valve ================================================ Copyright (c) 2015, Valve Corporation All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ================================================ FILE: alvr/server_openvr/build.rs ================================================ use std::{env, path::PathBuf}; fn get_ffmpeg_path() -> PathBuf { let ffmpeg_path = alvr_filesystem::deps_dir() .join(if cfg!(target_os = "linux") { "linux" } else { "windows" }) .join("ffmpeg"); if cfg!(target_os = "linux") { ffmpeg_path.join("alvr_build") } else { ffmpeg_path } } #[cfg(all(target_os = "linux", feature = "gpl"))] fn get_linux_x264_path() -> PathBuf { alvr_filesystem::deps_dir().join("linux/x264/alvr_build") } fn main() { let platform_name = env::var("CARGO_CFG_TARGET_OS").unwrap(); let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); let platform_subpath = match platform_name.as_str() { "windows" => "cpp/platform/win32", "linux" => "cpp/platform/linux", "macos" => "cpp/platform/macos", _ => panic!(), }; let common_iter = walkdir::WalkDir::new("cpp") .into_iter() .filter_entry(|entry| { entry.file_name() != "tools" && entry.file_name() != "platform" && (platform_name != "macos" || entry.file_name() != "amf") && (platform_name != "linux" || entry.file_name() != "amf") }); let platform_iter = walkdir::WalkDir::new(platform_subpath).into_iter(); let cpp_paths = common_iter .chain(platform_iter) .filter_map(|maybe_entry| maybe_entry.ok()) .map(|entry| entry.into_path()) .collect::>(); let source_files_paths = cpp_paths.iter().filter(|path| { path.extension() .filter(|ext| { let ext_str = ext.to_string_lossy(); ext_str == "c" || ext_str == "cpp" }) .is_some() }); let mut build = cc::Build::new(); build .cpp(true) .std("c++17") .files(source_files_paths) .include(alvr_filesystem::workspace_dir().join("openvr/headers")) .include("cpp"); if platform_name == "windows" { build .debug(false) // This is because we cannot link to msvcrtd (see below) .flag("/permissive-") .define("NOMINMAX", None) .define("_WINSOCKAPI_", None) .define("_MBCS", None) .define("_MT", None); } else if platform_name == "macos" { build.define("__APPLE__", None); } #[cfg(debug_assertions)] build.define("ALVR_DEBUG_LOG", None); let gpl_or_linux = cfg!(feature = "gpl") || cfg!(target_os = "linux"); if gpl_or_linux { let ffmpeg_path = get_ffmpeg_path(); assert!(ffmpeg_path.join("include").exists()); build.include(ffmpeg_path.join("include")); } #[cfg(all(target_os = "linux", feature = "gpl"))] { let x264_path = get_linux_x264_path(); assert!(x264_path.join("include").exists()); build.include(x264_path.join("include")); } #[cfg(feature = "gpl")] build.define("ALVR_GPL", None); #[cfg(target_os = "windows")] { let vpl_path = alvr_filesystem::deps_dir().join("windows/libvpl/alvr_build"); let vpl_include_path = vpl_path.join("include"); let vpl_lib_path = vpl_path.join("lib"); println!( "cargo:rustc-link-search=native={}", vpl_lib_path.to_string_lossy() ); build.define("ONEVPL_EXPERIMENTAL", None); build.include(vpl_include_path); println!("cargo:rustc-link-lib=vpl"); } build.compile("bindings"); #[cfg(all(target_os = "linux", feature = "gpl"))] { let x264_path = get_linux_x264_path(); let x264_lib_path = x264_path.join("lib"); println!( "cargo:rustc-link-search=native={}", x264_lib_path.to_string_lossy() ); let x264_pkg_path = x264_lib_path.join("pkgconfig"); assert!(x264_pkg_path.exists()); let x264_pkg_path = x264_pkg_path.to_string_lossy().to_string(); unsafe { env::set_var( "PKG_CONFIG_PATH", env::var("PKG_CONFIG_PATH").map_or(x264_pkg_path.clone(), |old| { format!("{x264_pkg_path}:{old}") }), ) }; println!("cargo:rustc-link-lib=static=x264"); pkg_config::Config::new() .statik(true) .probe("x264") .unwrap(); } // ffmpeg if gpl_or_linux { let ffmpeg_path = get_ffmpeg_path(); let ffmpeg_lib_path = ffmpeg_path.join("lib"); assert!(ffmpeg_lib_path.exists()); println!( "cargo:rustc-link-search=native={}", ffmpeg_lib_path.to_string_lossy() ); #[cfg(target_os = "linux")] { let ffmpeg_pkg_path = ffmpeg_lib_path.join("pkgconfig"); assert!(ffmpeg_pkg_path.exists()); let ffmpeg_pkg_path = ffmpeg_pkg_path.to_string_lossy().to_string(); unsafe { env::set_var( "PKG_CONFIG_PATH", env::var("PKG_CONFIG_PATH").map_or(ffmpeg_pkg_path.clone(), |old| { format!("{ffmpeg_pkg_path}:{old}") }), ) }; let pkg = pkg_config::Config::new().statik(true).to_owned(); for lib in ["libavutil", "libavfilter", "libavcodec"] { pkg.probe(lib).unwrap(); } } #[cfg(windows)] for lib in ["avutil", "avfilter", "avcodec", "swscale"] { println!("cargo:rustc-link-lib={lib}"); } } bindgen::builder() .clang_arg("-xc++") .header("cpp/alvr_server/bindings.h") .derive_default(true) .generate() .unwrap() .write_to_file(out_dir.join("bindings.rs")) .unwrap(); if platform_name == "linux" { println!( "cargo:rustc-link-search=native={}", alvr_filesystem::workspace_dir() .join("openvr/lib/linux64") .to_string_lossy() ); println!("cargo:rustc-link-lib=openvr_api"); } else if platform_name == "windows" { println!( "cargo:rustc-link-search=native={}", alvr_filesystem::workspace_dir() .join("openvr/lib/win64") .to_string_lossy() ); println!("cargo:rustc-link-lib=openvr_api"); } #[cfg(target_os = "linux")] { pkg_config::Config::new().probe("vulkan").unwrap(); #[cfg(not(feature = "gpl"))] { pkg_config::Config::new().probe("x264").unwrap(); } // fail build if there are undefined symbols in final library println!("cargo:rustc-cdylib-link-arg=-Wl,--no-undefined"); } for path in cpp_paths { println!("cargo:rerun-if-changed={}", path.to_string_lossy()); } } ================================================ FILE: alvr/server_openvr/cpp/ALVR-common/common-utils.cpp ================================================ #include "common-utils.h" #include #include std::wstring ToWstring(const std::string& src) { // TODO: src is really UTF-8? std::wstring_convert> converter; return converter.from_bytes(src); } std::string ToUTF8(const std::wstring& src) { std::wstring_convert> converter; return converter.to_bytes(src); } ================================================ FILE: alvr/server_openvr/cpp/ALVR-common/common-utils.h ================================================ #pragma once #include std::wstring ToWstring(const std::string& src); std::string ToUTF8(const std::wstring& src); ================================================ FILE: alvr/server_openvr/cpp/ALVR-common/exception.cpp ================================================ #include "exception.h" #include "common-utils.h" #include #include Exception FormatExceptionV(const char* format, va_list args) { char buf[1024]; vsprintf(buf, format, args); return Exception(buf); } Exception FormatException(const char* format, ...) { va_list args; va_start(args, format); Exception e = FormatExceptionV(format, args); va_end(args); return e; } ================================================ FILE: alvr/server_openvr/cpp/ALVR-common/exception.h ================================================ #pragma once #include #include class Exception : public std::exception { public: Exception(std::string what) : m_what(what) { } Exception() { } const char* what() const noexcept override { return m_what.c_str(); } private: std::string m_what; }; Exception FormatExceptionV(const char* format, va_list args); Exception FormatException(const char* format, ...); ================================================ FILE: alvr/server_openvr/cpp/ALVR-common/packet_types.h ================================================ #ifndef ALVRCLIENT_PACKETTYPES_H #define ALVRCLIENT_PACKETTYPES_H #include "../alvr_server/bindings.h" #include #include enum ALVR_CODEC { ALVR_CODEC_H264 = 0, ALVR_CODEC_HEVC = 1, ALVR_CODEC_AV1 = 2, }; enum ALVR_H264_PROFILE { ALVR_H264_PROFILE_HIGH = 0, ALVR_H264_PROFILE_MAIN = 1, ALVR_H264_PROFILE_BASELINE = 2, }; enum ALVR_RATE_CONTROL_METHOD { ALVR_CBR = 0, ALVR_VBR = 1, }; enum ALVR_ENTROPY_CODING { ALVR_CABAC = 0, ALVR_CAVLC = 1, }; enum ALVR_ENCODER_QUALITY_PRESET { ALVR_QUALITY = 0, ALVR_BALANCED = 1, ALVR_SPEED = 2 }; enum ALVR_INPUT { ALVR_INPUT_FINGER_INDEX, ALVR_INPUT_FINGER_MIDDLE, ALVR_INPUT_FINGER_RING, ALVR_INPUT_FINGER_PINKY, }; #define ALVR_BUTTON_FLAG(input) (1ULL << input) #endif // ALVRCLIENT_PACKETTYPES_H ================================================ FILE: alvr/server_openvr/cpp/alvr_server/ChaperoneUpdater.cpp ================================================ #include "ALVR-common/packet_types.h" #include "Logger.h" #include "bindings.h" #include #include #ifndef __APPLE__ // Workaround symbol clash in openvr.h / openvr_driver.h namespace alvr_chaperone { #include } using namespace alvr_chaperone; #endif std::mutex chaperone_mutex; bool isOpenvrInit = false; void InitOpenvrClient() { Debug("InitOpenvrClient"); #ifndef __APPLE__ std::unique_lock lock(chaperone_mutex); if (isOpenvrInit) { return; } vr::EVRInitError error; // Background needed for VRCompositor()->GetTrackingSpace() vr::VR_Init(&error, vr::VRApplication_Background); if (error != vr::VRInitError_None) { Warn("Failed to init OpenVR client! Error: %d", error); return; } isOpenvrInit = true; #endif } void ShutdownOpenvrClient() { Debug("ShutdownOpenvrClient"); #ifndef __APPLE__ std::unique_lock lock(chaperone_mutex); if (!isOpenvrInit) { return; } isOpenvrInit = false; vr::VR_Shutdown(); #endif } bool IsOpenvrClientReady() { return isOpenvrInit; } void _SetChaperoneArea(float areaWidth, float areaHeight) { Debug("SetChaperoneArea"); #ifndef __APPLE__ std::unique_lock lock(chaperone_mutex); const vr::HmdMatrix34_t MATRIX_IDENTITY = { { { 1.0, 0.0, 0.0, 0.0 }, { 0.0, 1.0, 0.0, 0.0 }, { 0.0, 0.0, 1.0, 0.0 } } }; float perimeterPoints[4][2]; perimeterPoints[0][0] = -1.0f * areaWidth; perimeterPoints[0][1] = -1.0f * areaHeight; perimeterPoints[1][0] = -1.0f * areaWidth; perimeterPoints[1][1] = 1.0f * areaHeight; perimeterPoints[2][0] = 1.0f * areaWidth; perimeterPoints[2][1] = 1.0f * areaHeight; perimeterPoints[3][0] = 1.0f * areaWidth; perimeterPoints[3][1] = -1.0f * areaHeight; auto setup = vr::VRChaperoneSetup(); if (setup != nullptr) { vr::VRChaperoneSetup()->SetWorkingPerimeter( reinterpret_cast(perimeterPoints), 4 ); vr::VRChaperoneSetup()->SetWorkingStandingZeroPoseToRawTrackingPose(&MATRIX_IDENTITY); vr::VRChaperoneSetup()->SetWorkingSeatedZeroPoseToRawTrackingPose(&MATRIX_IDENTITY); vr::VRChaperoneSetup()->SetWorkingPlayAreaSize(areaWidth, areaHeight); vr::VRChaperoneSetup()->CommitWorkingCopy(vr::EChaperoneConfigFile_Live); } auto settings = vr::VRSettings(); if (settings != nullptr) { // Hide SteamVR Chaperone vr::VRSettings()->SetFloat( vr::k_pch_CollisionBounds_Section, vr::k_pch_CollisionBounds_FadeDistance_Float, 0.0f ); } #endif } #ifdef __linux__ std::unique_ptr GetInvZeroPose() { Debug("GetInvZeroPose"); std::unique_lock lock(chaperone_mutex); if (!isOpenvrInit) { return nullptr; } auto mat = std::make_unique(); // revert pulls live into working copy vr::VRChaperoneSetup()->RevertWorkingCopy(); auto compositor = vr::VRCompositor(); if (compositor == nullptr) { return nullptr; } if (compositor->GetTrackingSpace() == vr::TrackingUniverseStanding) { vr::VRChaperoneSetup()->GetWorkingStandingZeroPoseToRawTrackingPose(mat.get()); } else { vr::VRChaperoneSetup()->GetWorkingSeatedZeroPoseToRawTrackingPose(mat.get()); } return mat; } #endif ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Controller.cpp ================================================ #include "Controller.h" #include "Logger.h" #include "Paths.h" #include "Settings.h" #include "Utils.h" #include "include/openvr_math.h" #include #include #include Controller::Controller(uint64_t deviceID, vr::EVRSkeletalTrackingLevel skeletonLevel) : TrackedDevice( deviceID, Settings::Instance().m_controllerIsTracker ? vr::TrackedDeviceClass_GenericTracker : vr::TrackedDeviceClass_Controller ) , m_skeletonLevel(skeletonLevel) { Debug("Controller::constructor deviceID=%llu", deviceID); } bool Controller::activate() { Debug("Controller::Activate deviceID=%llu", this->device_id); auto vr_driver_input = vr::VRDriverInput(); SetOpenvrProps((void*)this, this->device_id); RegisterButtons((void*)this, this->device_id); vr_driver_input->CreateHapticComponent(this->prop_container, "/output/haptic", &m_compHaptic); vr_driver_input->CreateScalarComponent( this->prop_container, "/input/finger/index", &m_buttonHandles[ALVR_INPUT_FINGER_INDEX], vr::VRScalarType_Absolute, vr::VRScalarUnits_NormalizedOneSided ); vr_driver_input->CreateScalarComponent( this->prop_container, "/input/finger/middle", &m_buttonHandles[ALVR_INPUT_FINGER_MIDDLE], vr::VRScalarType_Absolute, vr::VRScalarUnits_NormalizedOneSided ); vr_driver_input->CreateScalarComponent( this->prop_container, "/input/finger/ring", &m_buttonHandles[ALVR_INPUT_FINGER_RING], vr::VRScalarType_Absolute, vr::VRScalarUnits_NormalizedOneSided ); vr_driver_input->CreateScalarComponent( this->prop_container, "/input/finger/pinky", &m_buttonHandles[ALVR_INPUT_FINGER_PINKY], vr::VRScalarType_Absolute, vr::VRScalarUnits_NormalizedOneSided ); if (this->device_id == HAND_LEFT_ID || this->device_id == HAND_TRACKER_LEFT_ID) { vr_driver_input->CreateSkeletonComponent( this->prop_container, "/input/skeleton/left", "/skeleton/hand/left", "/pose/raw", m_skeletonLevel, nullptr, 0U, &m_compSkeleton ); } else { vr_driver_input->CreateSkeletonComponent( this->prop_container, "/input/skeleton/right", "/skeleton/hand/right", "/pose/raw", m_skeletonLevel, nullptr, 0U, &m_compSkeleton ); } // NB: here we set some initial values for the hand skeleton to fix the frozen hand bug { vr::VRBoneTransform_t boneTransforms[SKELETON_BONE_COUNT]; GetBoneTransform(false, boneTransforms); vr_driver_input->UpdateSkeletonComponent( m_compSkeleton, vr::VRSkeletalMotionRange_WithController, boneTransforms, SKELETON_BONE_COUNT ); vr_driver_input->UpdateSkeletonComponent( m_compSkeleton, vr::VRSkeletalMotionRange_WithoutController, boneTransforms, SKELETON_BONE_COUNT ); } return true; } vr::VRInputComponentHandle_t Controller::getHapticComponent() { return m_compHaptic; } void Controller::RegisterButton(uint64_t id) { Debug("Controller::RegisterButton deviceID=%llu", this->device_id); ButtonInfo buttonInfo; if (device_id == HAND_LEFT_ID) { buttonInfo = LEFT_CONTROLLER_BUTTON_MAPPING[id]; } else { buttonInfo = RIGHT_CONTROLLER_BUTTON_MAPPING[id]; } if (buttonInfo.type == ButtonType::Binary) { for (auto path : buttonInfo.steamvr_paths) { vr::VRDriverInput()->CreateBooleanComponent( this->prop_container, path, &m_buttonHandles[PathStringToHash(path)] ); } } else { auto scalarType = buttonInfo.type == ButtonType::ScalarOneSided ? vr::VRScalarUnits_NormalizedOneSided : vr::VRScalarUnits_NormalizedTwoSided; for (auto path : buttonInfo.steamvr_paths) { vr::VRDriverInput()->CreateScalarComponent( this->prop_container, path, &m_buttonHandles[PathStringToHash(path)], vr::VRScalarType_Absolute, scalarType ); } } } void Controller::SetButton(uint64_t id, FfiButtonValue value) { Debug("Controller::SetButton deviceID=%llu buttonID=%llu", this->device_id, id); if (!this->last_pose.poseIsValid) { return; } for (auto id : ALVR_TO_STEAMVR_PATH_IDS[id]) { if (value.type == BUTTON_TYPE_BINARY) { vr::VRDriverInput()->UpdateBooleanComponent( m_buttonHandles[id], (bool)value.binary, 0.0 ); } else { vr::VRDriverInput()->UpdateScalarComponent(m_buttonHandles[id], value.scalar, 0.0); } } // todo: remove when moving inferred controller hand skeleton to rust if (id == LEFT_A_TOUCH_ID || id == LEFT_B_TOUCH_ID || id == LEFT_X_TOUCH_ID || id == LEFT_Y_TOUCH_ID || id == LEFT_TRACKPAD_TOUCH_ID || id == LEFT_THUMBSTICK_TOUCH_ID || id == LEFT_THUMBREST_TOUCH_ID || id == RIGHT_A_TOUCH_ID || id == RIGHT_B_TOUCH_ID || id == RIGHT_TRACKPAD_TOUCH_ID || id == RIGHT_THUMBSTICK_TOUCH_ID || id == RIGHT_THUMBREST_TOUCH_ID) { m_currentThumbTouch = value.binary; } else if (id == LEFT_TRIGGER_TOUCH_ID || id == RIGHT_TRIGGER_TOUCH_ID) { m_currentTriggerTouch = value.binary; } else if (id == LEFT_TRIGGER_VALUE_ID || id == RIGHT_TRIGGER_VALUE_ID) { m_triggerValue = value.scalar; } else if (id == LEFT_SQUEEZE_VALUE_ID || id == RIGHT_SQUEEZE_VALUE_ID) { m_gripValue = value.scalar; } } bool Controller::OnPoseUpdate(uint64_t targetTimestampNs, float predictionS, FfiHandData handData) { if (this->object_id == vr::k_unTrackedDeviceIndexInvalid) { return false; } auto controllerMotion = handData.controllerMotion; auto handSkeleton = handData.handSkeleton; bool enabledAsHandTracker = handData.isHandTracker && (device_id == HAND_TRACKER_LEFT_ID || device_id == HAND_TRACKER_RIGHT_ID); bool enabledAsController = !handData.isHandTracker && (device_id == HAND_LEFT_ID || device_id == HAND_RIGHT_ID); bool enabled = (controllerMotion != nullptr || handSkeleton != nullptr) && (enabledAsHandTracker || enabledAsController); Debug( "%s %s: enabled: %d, ctrl: %d, hand: %d", (device_id == HAND_TRACKER_LEFT_ID || device_id == HAND_TRACKER_RIGHT_ID) ? "hand tracker" : "controller", (device_id == HAND_TRACKER_LEFT_ID || device_id == HAND_LEFT_ID) ? "left" : "right", enabled, handData.controllerMotion != nullptr, handData.handSkeleton != nullptr ); auto vr_driver_input = vr::VRDriverInput(); auto pose = vr::DriverPose_t {}; pose.poseIsValid = enabled; pose.deviceIsConnected = enabled; pose.result = enabled ? vr::TrackingResult_Running_OK : vr::TrackingResult_Uninitialized; pose.qDriverFromHeadRotation = HmdQuaternion_Init(1, 0, 0, 0); pose.qWorldFromDriverRotation = HmdQuaternion_Init(1, 0, 0, 0); if (controllerMotion != nullptr) { auto m = controllerMotion; pose.qRotation = HmdQuaternion_Init( m->pose.orientation.w, m->pose.orientation.x, m->pose.orientation.y, m->pose.orientation.z ); pose.vecPosition[0] = m->pose.position[0]; pose.vecPosition[1] = m->pose.position[1]; pose.vecPosition[2] = m->pose.position[2]; pose.vecVelocity[0] = m->linearVelocity[0]; pose.vecVelocity[1] = m->linearVelocity[1]; pose.vecVelocity[2] = m->linearVelocity[2]; pose.vecAngularVelocity[0] = m->angularVelocity[0]; pose.vecAngularVelocity[1] = m->angularVelocity[1]; pose.vecAngularVelocity[2] = m->angularVelocity[2]; } else if (handSkeleton != nullptr) { auto r = handSkeleton->jointRotations[0]; pose.qRotation = HmdQuaternion_Init(r.w, r.x, r.y, r.z); auto p = handSkeleton->jointPositions[0]; pose.vecPosition[0] = p[0]; pose.vecPosition[1] = p[1]; pose.vecPosition[2] = p[2]; // If possible, use the last stored m_pose and timestamp // to calculate the velocities of the current pose. double linearVelocity[3] = { 0.0, 0.0, 0.0 }; vr::HmdVector3d_t angularVelocity = { 0.0, 0.0, 0.0 }; if (handData.predictHandSkeleton && this->last_pose.poseIsValid) { double dt = ((double)targetTimestampNs - (double)m_poseTargetTimestampNs) / NS_PER_S; if (dt > 0.0) { linearVelocity[0] = (pose.vecPosition[0] - this->last_pose.vecPosition[0]) / dt; linearVelocity[1] = (pose.vecPosition[1] - this->last_pose.vecPosition[1]) / dt; linearVelocity[2] = (pose.vecPosition[2] - this->last_pose.vecPosition[2]) / dt; angularVelocity = AngularVelocityBetweenQuats(this->last_pose.qRotation, pose.qRotation, dt); } } pose.vecVelocity[0] = linearVelocity[0]; pose.vecVelocity[1] = linearVelocity[1]; pose.vecVelocity[2] = linearVelocity[2]; pose.vecAngularVelocity[0] = angularVelocity.v[0]; pose.vecAngularVelocity[1] = angularVelocity.v[1]; pose.vecAngularVelocity[2] = angularVelocity.v[2]; } pose.poseTimeOffset = predictionS; this->submit_pose(pose); m_poseTargetTimestampNs = targetTimestampNs; // Early return to skip updating the skeleton if (!enabled) { return false; } else if (handSkeleton != nullptr) { vr::VRBoneTransform_t boneTransform[SKELETON_BONE_COUNT] = {}; boneTransform[0].orientation.w = 1.0; boneTransform[0].orientation.x = 0.0; boneTransform[0].orientation.y = 0.0; boneTransform[0].orientation.z = 0.0; boneTransform[0].position.v[0] = 0.0; boneTransform[0].position.v[1] = 0.0; boneTransform[0].position.v[2] = 0.0; boneTransform[0].position.v[3] = 1.0; // NB: start from index 1 to skip the root bone for (int j = 1; j < 31; j++) { boneTransform[j].orientation.w = handSkeleton->jointRotations[j].w; boneTransform[j].orientation.x = handSkeleton->jointRotations[j].x; boneTransform[j].orientation.y = handSkeleton->jointRotations[j].y; boneTransform[j].orientation.z = handSkeleton->jointRotations[j].z; boneTransform[j].position.v[0] = handSkeleton->jointPositions[j][0]; boneTransform[j].position.v[1] = handSkeleton->jointPositions[j][1]; boneTransform[j].position.v[2] = handSkeleton->jointPositions[j][2]; boneTransform[j].position.v[3] = 1.0; } vr_driver_input->UpdateSkeletonComponent( m_compSkeleton, vr::VRSkeletalMotionRange_WithController, boneTransform, SKELETON_BONE_COUNT ); vr_driver_input->UpdateSkeletonComponent( m_compSkeleton, vr::VRSkeletalMotionRange_WithoutController, boneTransform, SKELETON_BONE_COUNT ); float rotThumb = (handSkeleton->jointRotations[2].z + handSkeleton->jointRotations[2].y + handSkeleton->jointRotations[3].z + handSkeleton->jointRotations[3].y + handSkeleton->jointRotations[4].z + handSkeleton->jointRotations[4].y) * 0.67f; float rotIndex = (handSkeleton->jointRotations[7].z + handSkeleton->jointRotations[8].z + handSkeleton->jointRotations[9].z) * 0.67f; float rotMiddle = (handSkeleton->jointRotations[12].z + handSkeleton->jointRotations[13].z + handSkeleton->jointRotations[14].z) * 0.67f; float rotRing = (handSkeleton->jointRotations[17].z + handSkeleton->jointRotations[18].z + handSkeleton->jointRotations[19].z) * 0.67f; float rotPinky = (handSkeleton->jointRotations[22].z + handSkeleton->jointRotations[23].z + handSkeleton->jointRotations[24].z) * 0.67f; vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_INDEX], rotIndex, 0.0 ); vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_MIDDLE], rotMiddle, 0.0 ); vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_RING], rotRing, 0.0 ); vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_PINKY], rotPinky, 0.0 ); } else if (controllerMotion != nullptr) { if (m_lastThumbTouch != m_currentThumbTouch) { m_thumbTouchAnimationProgress += 1.f / ANIMATION_FRAME_COUNT; if (m_thumbTouchAnimationProgress > 1.f) { m_thumbTouchAnimationProgress = 0; m_lastThumbTouch = m_currentThumbTouch; } } else { m_thumbTouchAnimationProgress = 0; } if (m_lastTriggerTouch != m_currentTriggerTouch) { m_indexTouchAnimationProgress += 1.f / ANIMATION_FRAME_COUNT; if (m_indexTouchAnimationProgress > 1.f) { m_indexTouchAnimationProgress = 0; m_lastTriggerTouch = m_currentTriggerTouch; } } else { m_indexTouchAnimationProgress = 0; } float indexCurl = 0.0; if (m_triggerValue > 0.0) { indexCurl = 0.5 + m_triggerValue * 0.5; } else if (m_lastTriggerTouch == 0) { indexCurl = m_indexTouchAnimationProgress * 0.5; } else { indexCurl = 0.5 - m_indexTouchAnimationProgress * 0.5; } vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_INDEX], indexCurl, 0.0 ); vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_MIDDLE], m_gripValue, 0.0 ); // Ring and pinky fingers are not tracked. Infer a more natural pose. if (m_currentThumbTouch) { vr_driver_input->UpdateScalarComponent(m_buttonHandles[ALVR_INPUT_FINGER_RING], 1, 0.0); vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_PINKY], 1, 0.0 ); } else { vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_RING], m_gripValue, 0.0 ); vr_driver_input->UpdateScalarComponent( m_buttonHandles[ALVR_INPUT_FINGER_PINKY], m_gripValue, 0.0 ); } vr::VRBoneTransform_t boneTransforms[SKELETON_BONE_COUNT]; // Perform whatever logic is necessary to convert your device's input into a // skeletal pose, first to create a pose "With Controller", that is as close to the // pose of the user's real hand as possible GetBoneTransform(true, boneTransforms); // Then update the WithController pose on the component with those transforms vr::EVRInputError err = vr_driver_input->UpdateSkeletonComponent( m_compSkeleton, vr::VRSkeletalMotionRange_WithController, boneTransforms, SKELETON_BONE_COUNT ); if (err != vr::VRInputError_None) { // Handle failure case Error("UpdateSkeletonComponentfailed. Error: %i\n", err); } GetBoneTransform(false, boneTransforms); // Then update the WithoutController pose on the component err = vr_driver_input->UpdateSkeletonComponent( m_compSkeleton, vr::VRSkeletalMotionRange_WithoutController, boneTransforms, SKELETON_BONE_COUNT ); if (err != vr::VRInputError_None) { // Handle failure case Error("UpdateSkeletonComponentfailed. Error: %i\n", err); } } return false; } void GetThumbBoneTransform( bool withController, bool isLeftHand, bool touch, vr::VRBoneTransform_t outBoneTransform[] ) { if (isLeftHand) { if (touch) { if (withController) { outBoneTransform[2] = { { -0.017303f, 0.032567f, 0.025281f, 1.f }, { 0.317609f, 0.528344f, 0.213134f, 0.757991f } }; outBoneTransform[3] = { { 0.040406f, 0.000000f, -0.000000f, 1.f }, { 0.991742f, 0.085317f, 0.019416f, 0.093765f } }; outBoneTransform[4] = { { 0.032517f, -0.000000f, 0.000000f, 1.f }, { 0.959385f, -0.012202f, -0.031055f, 0.280120f } }; } else { outBoneTransform[2] = { { -0.016426f, 0.030866f, 0.025118f, 1.f }, { 0.403850f, 0.595704f, 0.082451f, 0.689380f } }; outBoneTransform[3] = { { 0.040406f, 0.000000f, -0.000000f, 1.f }, { 0.989655f, -0.090426f, 0.028457f, 0.107691f } }; outBoneTransform[4] = { { 0.032517f, 0.000000f, 0.000000f, 1.f }, { 0.988590f, 0.143978f, 0.041520f, 0.015363f } }; } } else { outBoneTransform[2] = { { -0.012083f, 0.028070f, 0.025050f, 1 }, { 0.464112f, 0.567418f, 0.272106f, 0.623374f } }; outBoneTransform[3] = { { 0.040406f, 0.000000f, -0.000000f, 1 }, { 0.994838f, 0.082939f, 0.019454f, 0.055130f } }; outBoneTransform[4] = { { 0.032517f, 0.000000f, 0.000000f, 1 }, { 0.974793f, -0.003213f, 0.021867f, -0.222015f } }; } outBoneTransform[5] = { { 0.030464f, -0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, 0.000000f, 0.000000f } }; } else { if (touch) { if (withController) { outBoneTransform[2] = { { 0.017303f, 0.032567f, 0.025281f, 1 }, { 0.528344f, -0.317609f, 0.757991f, -0.213134f } }; outBoneTransform[3] = { { -0.040406f, -0.000000f, 0.000000f, 1 }, { 0.991742f, 0.085317f, 0.019416f, 0.093765f } }; outBoneTransform[4] = { { -0.032517f, 0.000000f, -0.000000f, 1 }, { 0.959385f, -0.012202f, -0.031055f, 0.280120f } }; } else { outBoneTransform[2] = { { 0.016426f, 0.030866f, 0.025118f, 1 }, { 0.595704f, -0.403850f, 0.689380f, -0.082451f } }; outBoneTransform[3] = { { -0.040406f, -0.000000f, 0.000000f, 1 }, { 0.989655f, -0.090426f, 0.028457f, 0.107691f } }; outBoneTransform[4] = { { -0.032517f, -0.000000f, -0.000000f, 1 }, { 0.988590f, 0.143978f, 0.041520f, 0.015363f } }; } } else { outBoneTransform[2] = { { 0.012330f, 0.028661f, 0.025049f, 1 }, { 0.571059f, -0.451277f, 0.630056f, -0.270685f } }; outBoneTransform[3] = { { -0.040406f, -0.000000f, 0.000000f, 1 }, { 0.994565f, 0.078280f, 0.018282f, 0.066177f } }; outBoneTransform[4] = { { -0.032517f, -0.000000f, -0.000000f, 1 }, { 0.977658f, -0.003039f, 0.020722f, -0.209156f } }; } outBoneTransform[5] = { { -0.030464f, 0.000000f, 0.000000f, 1 }, { 1.000000f, -0.000000f, 0.000000f, 0.000000f } }; } } void GetTriggerBoneTransform( bool withController, bool isLeftHand, bool touch, float click, vr::VRBoneTransform_t outBoneTransform[] ) { if (click) { if (withController) { if (isLeftHand) { outBoneTransform[6] = { { -0.003925f, 0.027171f, 0.014640f, 1 }, { 0.666448f, 0.430031f, -0.455947f, 0.403772f } }; outBoneTransform[7] = { { 0.076015f, -0.005124f, 0.000239f, 1 }, { -0.956011f, -0.000025f, 0.158355f, -0.246913f } }; outBoneTransform[8] = { { 0.043930f, -0.000000f, -0.000000f, 1 }, { -0.944138f, -0.043351f, 0.014947f, -0.326345f } }; outBoneTransform[9] = { { 0.028695f, 0.000000f, 0.000000f, 1 }, { -0.912149f, 0.003626f, 0.039888f, -0.407898f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.529359f, 0.540512f, -0.463783f, 0.461011f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.847397f, -0.257141f, -0.139135f, 0.443213f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.874907f, 0.009875f, 0.026584f, 0.483460f } }; outBoneTransform[14] = { { 0.033266f, -0.000000f, 0.000000f, 1 }, { 0.894578f, -0.036774f, -0.050597f, 0.442513f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, -0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.500244f, 0.530784f, -0.516215f, 0.448939f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.831617f, -0.242931f, -0.139695f, 0.479461f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.769163f, -0.001746f, 0.001363f, 0.639049f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.968615f, -0.064538f, -0.046586f, 0.235477f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.474671f, 0.434670f, -0.653212f, 0.398827f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.798788f, -0.199577f, -0.094418f, 0.559636f } }; outBoneTransform[23] = { { 0.030220f, 0.000002f, -0.000000f, 1 }, { 0.853087f, 0.001644f, -0.000913f, 0.521765f } }; outBoneTransform[24] = { { 0.018187f, -0.000002f, 0.000000f, 1 }, { 0.974249f, 0.052491f, 0.003591f, 0.219249f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[26] = { { 0.006629f, 0.026690f, 0.061870f, 1 }, { 0.805084f, -0.018369f, 0.584788f, -0.097597f } }; outBoneTransform[27] = { { -0.007882f, -0.040478f, 0.039337f, 1 }, { -0.322494f, 0.932092f, 0.121861f, 0.111140f } }; outBoneTransform[28] = { { 0.017136f, -0.032633f, 0.080682f, 1 }, { -0.169466f, 0.800083f, 0.571006f, 0.071415f } }; outBoneTransform[29] = { { 0.011144f, -0.028727f, 0.108366f, 1 }, { -0.076328f, 0.788280f, 0.605097f, 0.081527f } }; outBoneTransform[30] = { { 0.011333f, -0.026044f, 0.128585f, 1 }, { -0.144791f, 0.737451f, 0.656958f, -0.060069f } }; } else { outBoneTransform[6] = { { -0.003925f, 0.027171f, 0.014640f, 1 }, { 0.666448f, 0.430031f, -0.455947f, 0.403772f } }; outBoneTransform[7] = { { 0.076015f, -0.005124f, 0.000239f, 1 }, { -0.956011f, -0.000025f, 0.158355f, -0.246913f } }; outBoneTransform[8] = { { 0.043930f, -0.000000f, -0.000000f, 1 }, { -0.944138f, -0.043351f, 0.014947f, -0.326345f } }; outBoneTransform[9] = { { 0.028695f, 0.000000f, 0.000000f, 1 }, { -0.912149f, 0.003626f, 0.039888f, -0.407898f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.529359f, 0.540512f, -0.463783f, 0.461011f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.847397f, -0.257141f, -0.139135f, 0.443213f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.874907f, 0.009875f, 0.026584f, 0.483460f } }; outBoneTransform[14] = { { 0.033266f, -0.000000f, 0.000000f, 1 }, { 0.894578f, -0.036774f, -0.050597f, 0.442513f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, -0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.500244f, 0.530784f, -0.516215f, 0.448939f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.831617f, -0.242931f, -0.139695f, 0.479461f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.769163f, -0.001746f, 0.001363f, 0.639049f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.968615f, -0.064538f, -0.046586f, 0.235477f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.474671f, 0.434670f, -0.653212f, 0.398827f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.798788f, -0.199577f, -0.094418f, 0.559636f } }; outBoneTransform[23] = { { 0.030220f, 0.000002f, -0.000000f, 1 }, { 0.853087f, 0.001644f, -0.000913f, 0.521765f } }; outBoneTransform[24] = { { 0.018187f, -0.000002f, 0.000000f, 1 }, { 0.974249f, 0.052491f, 0.003591f, 0.219249f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[26] = { { 0.006629f, 0.026690f, 0.061870f, 1 }, { 0.805084f, -0.018369f, 0.584788f, -0.097597f } }; outBoneTransform[27] = { { -0.007882f, -0.040478f, 0.039337f, 1 }, { -0.322494f, 0.932092f, 0.121861f, 0.111140f } }; outBoneTransform[28] = { { 0.017136f, -0.032633f, 0.080682f, 1 }, { -0.169466f, 0.800083f, 0.571006f, 0.071415f } }; outBoneTransform[29] = { { 0.011144f, -0.028727f, 0.108366f, 1 }, { -0.076328f, 0.788280f, 0.605097f, 0.081527f } }; outBoneTransform[30] = { { 0.011333f, -0.026044f, 0.128585f, 1 }, { -0.144791f, 0.737451f, 0.656958f, -0.060069f } }; } } else { if (isLeftHand) { outBoneTransform[6] = { { 0.003802f, 0.021514f, 0.012803f, 1 }, { 0.617314f, 0.395175f, -0.510874f, 0.449185f } }; outBoneTransform[7] = { { 0.074204f, -0.005002f, 0.000234f, 1 }, { 0.737291f, -0.032006f, -0.115013f, 0.664944f } }; outBoneTransform[8] = { { 0.043287f, -0.000000f, -0.000000f, 1 }, { 0.611381f, 0.003287f, 0.003823f, 0.791320f } }; outBoneTransform[9] = { { 0.028275f, 0.000000f, 0.000000f, 1 }, { 0.745389f, -0.000684f, -0.000945f, 0.666629f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.004885f, 0.006885f, 0.016480f, 1 }, { 0.522678f, 0.527374f, -0.469333f, 0.477923f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.826071f, -0.121321f, 0.017267f, 0.550082f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.956676f, 0.013210f, 0.009330f, 0.290704f } }; outBoneTransform[14] = { { 0.033266f, 0.000000f, 0.000000f, 1 }, { 0.979740f, -0.001605f, -0.019412f, 0.199323f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.001696f, -0.006648f, 0.016418f, 1 }, { 0.509620f, 0.540794f, -0.504891f, 0.439220f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.955009f, -0.065344f, -0.063228f, 0.282294f } }; outBoneTransform[18] = { { 0.040577f, 0.000000f, 0.000000f, 1 }, { 0.953823f, -0.000972f, 0.000697f, 0.300366f } }; outBoneTransform[19] = { { 0.028698f, -0.000000f, -0.000000f, 1 }, { 0.977627f, -0.001163f, -0.011433f, 0.210033f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { -0.001792f, -0.019041f, 0.015254f, 1 }, { 0.518602f, 0.511152f, -0.596086f, 0.338315f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.978584f, -0.045398f, -0.103083f, 0.172297f } }; outBoneTransform[23] = { { 0.030154f, 0.000000f, 0.000000f, 1 }, { 0.970479f, -0.000068f, -0.002025f, 0.241175f } }; outBoneTransform[24] = { { 0.018187f, 0.000000f, 0.000000f, 1 }, { 0.997053f, -0.000687f, -0.052009f, -0.056395f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[26] = { { -0.005193f, 0.054191f, 0.060030f, 1 }, { 0.747374f, 0.182388f, 0.599615f, 0.220518f } }; outBoneTransform[27] = { { 0.000171f, 0.016473f, 0.096515f, 1 }, { -0.006456f, 0.022747f, -0.932927f, -0.359287f } }; outBoneTransform[28] = { { -0.038019f, -0.074839f, 0.046941f, 1 }, { -0.199973f, 0.698334f, -0.635627f, -0.261380f } }; outBoneTransform[29] = { { -0.036836f, -0.089774f, 0.081969f, 1 }, { -0.191006f, 0.756582f, -0.607429f, -0.148761f } }; outBoneTransform[30] = { { -0.030241f, -0.086049f, 0.119881f, 1 }, { -0.019037f, 0.779368f, -0.612017f, -0.132881f } }; } else { outBoneTransform[6] = { { -0.003802f, 0.021514f, 0.012803f, 1 }, { 0.395174f, -0.617314f, 0.449185f, 0.510874f } }; outBoneTransform[7] = { { -0.074204f, 0.005002f, -0.000234f, 1 }, { 0.737291f, -0.032006f, -0.115013f, 0.664944f } }; outBoneTransform[8] = { { -0.043287f, 0.000000f, 0.000000f, 1 }, { 0.611381f, 0.003287f, 0.003823f, 0.791320f } }; outBoneTransform[9] = { { -0.028275f, -0.000000f, -0.000000f, 1 }, { 0.745389f, -0.000684f, -0.000945f, 0.666629f } }; outBoneTransform[10] = { { -0.022821f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { -0.004885f, 0.006885f, 0.016480f, 1 }, { 0.527233f, -0.522513f, 0.478085f, 0.469510f } }; outBoneTransform[12] = { { -0.070953f, -0.000779f, -0.000997f, 1 }, { 0.826317f, -0.120120f, 0.019005f, 0.549918f } }; outBoneTransform[13] = { { -0.043108f, -0.000000f, -0.000000f, 1 }, { 0.958363f, 0.013484f, 0.007380f, 0.285138f } }; outBoneTransform[14] = { { -0.033266f, -0.000000f, -0.000000f, 1 }, { 0.977901f, -0.001431f, -0.018078f, 0.208279f } }; outBoneTransform[15] = { { -0.025892f, 0.000000f, -0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { -0.001696f, -0.006648f, 0.016418f, 1 }, { 0.541481f, -0.508179f, 0.441001f, 0.504054f } }; outBoneTransform[17] = { { -0.065876f, -0.001786f, -0.000693f, 1 }, { 0.953780f, -0.064506f, -0.058812f, 0.287548f } }; outBoneTransform[18] = { { -0.040577f, -0.000000f, -0.000000f, 1 }, { 0.954761f, -0.000983f, 0.000698f, 0.297372f } }; outBoneTransform[19] = { { -0.028698f, 0.000000f, 0.000000f, 1 }, { 0.976924f, -0.001344f, -0.010281f, 0.213335f } }; outBoneTransform[20] = { { -0.022430f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { 0.001792f, -0.019041f, 0.015254f, 1 }, { 0.510569f, -0.514906f, 0.341115f, 0.598191f } }; outBoneTransform[22] = { { -0.062878f, -0.002844f, -0.000332f, 1 }, { 0.979195f, -0.043879f, -0.095103f, 0.173800f } }; outBoneTransform[23] = { { -0.030154f, -0.000000f, -0.000000f, 1 }, { 0.971387f, -0.000102f, -0.002019f, 0.237494f } }; outBoneTransform[24] = { { -0.018187f, -0.000000f, -0.000000f, 1 }, { 0.997961f, 0.000800f, -0.051911f, -0.037114f } }; outBoneTransform[25] = { { -0.018018f, -0.000000f, 0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[26] = { { 0.004392f, 0.055515f, 0.060253f, 1 }, { 0.745924f, 0.156756f, -0.597950f, -0.247953f } }; outBoneTransform[27] = { { -0.000171f, 0.016473f, 0.096515f, 1 }, { -0.006456f, 0.022747f, 0.932927f, 0.359287f } }; outBoneTransform[28] = { { 0.038119f, -0.074730f, 0.046338f, 1 }, { -0.207931f, 0.699835f, 0.632631f, 0.258406f } }; outBoneTransform[29] = { { 0.035492f, -0.089519f, 0.081636f, 1 }, { -0.197555f, 0.760574f, 0.601098f, 0.145535f } }; outBoneTransform[30] = { { 0.029073f, -0.085957f, 0.119561f, 1 }, { -0.031423f, 0.791013f, 0.597190f, 0.129133f } }; } } } else if (touch) { if (withController) { if (isLeftHand) { outBoneTransform[6] = { { -0.003925f, 0.027171f, 0.014640f, 1 }, { 0.666448f, 0.430031f, -0.455947f, 0.403772f } }; outBoneTransform[7] = { { 0.074204f, -0.005002f, 0.000234f, 1 }, { -0.951843f, 0.009717f, 0.158611f, -0.262188f } }; outBoneTransform[8] = { { 0.043930f, -0.000000f, -0.000000f, 1 }, { -0.973045f, -0.044676f, 0.010341f, -0.226012f } }; outBoneTransform[9] = { { 0.028695f, 0.000000f, 0.000000f, 1 }, { -0.935253f, -0.002881f, 0.023037f, -0.353217f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.529359f, 0.540512f, -0.463783f, 0.461011f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.847397f, -0.257141f, -0.139135f, 0.443213f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.874907f, 0.009875f, 0.026584f, 0.483460f } }; outBoneTransform[14] = { { 0.033266f, -0.000000f, 0.000000f, 1 }, { 0.894578f, -0.036774f, -0.050597f, 0.442513f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, -0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.500244f, 0.530784f, -0.516215f, 0.448939f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.831617f, -0.242931f, -0.139695f, 0.479461f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.769163f, -0.001746f, 0.001363f, 0.639049f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.968615f, -0.064538f, -0.046586f, 0.235477f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.474671f, 0.434670f, -0.653212f, 0.398827f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.798788f, -0.199577f, -0.094418f, 0.559636f } }; outBoneTransform[23] = { { 0.030220f, 0.000002f, -0.000000f, 1 }, { 0.853087f, 0.001644f, -0.000913f, 0.521765f } }; outBoneTransform[24] = { { 0.018187f, -0.000002f, 0.000000f, 1 }, { 0.974249f, 0.052491f, 0.003591f, 0.219249f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[26] = { { 0.006629f, 0.026690f, 0.061870f, 1 }, { 0.805084f, -0.018369f, 0.584788f, -0.097597f } }; outBoneTransform[27] = { { -0.009005f, -0.041708f, 0.037992f, 1 }, { -0.338860f, 0.939952f, -0.007564f, 0.040082f } }; outBoneTransform[28] = { { 0.017136f, -0.032633f, 0.080682f, 1 }, { -0.169466f, 0.800083f, 0.571006f, 0.071415f } }; outBoneTransform[29] = { { 0.011144f, -0.028727f, 0.108366f, 1 }, { -0.076328f, 0.788280f, 0.605097f, 0.081527f } }; outBoneTransform[30] = { { 0.011333f, -0.026044f, 0.128585f, 1 }, { -0.144791f, 0.737451f, 0.656958f, -0.060069f } }; } else { outBoneTransform[6] = { { -0.003925f, 0.027171f, 0.014640f, 1 }, { 0.666448f, 0.430031f, -0.455947f, 0.403772f } }; outBoneTransform[7] = { { 0.074204f, -0.005002f, 0.000234f, 1 }, { -0.951843f, 0.009717f, 0.158611f, -0.262188f } }; outBoneTransform[8] = { { 0.043930f, -0.000000f, -0.000000f, 1 }, { -0.973045f, -0.044676f, 0.010341f, -0.226012f } }; outBoneTransform[9] = { { 0.028695f, 0.000000f, 0.000000f, 1 }, { -0.935253f, -0.002881f, 0.023037f, -0.353217f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.529359f, 0.540512f, -0.463783f, 0.461011f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.847397f, -0.257141f, -0.139135f, 0.443213f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.874907f, 0.009875f, 0.026584f, 0.483460f } }; outBoneTransform[14] = { { 0.033266f, -0.000000f, 0.000000f, 1 }, { 0.894578f, -0.036774f, -0.050597f, 0.442513f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, -0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.500244f, 0.530784f, -0.516215f, 0.448939f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.831617f, -0.242931f, -0.139695f, 0.479461f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.769163f, -0.001746f, 0.001363f, 0.639049f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.968615f, -0.064538f, -0.046586f, 0.235477f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.474671f, 0.434670f, -0.653212f, 0.398827f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.798788f, -0.199577f, -0.094418f, 0.559636f } }; outBoneTransform[23] = { { 0.030220f, 0.000002f, -0.000000f, 1 }, { 0.853087f, 0.001644f, -0.000913f, 0.521765f } }; outBoneTransform[24] = { { 0.018187f, -0.000002f, 0.000000f, 1 }, { 0.974249f, 0.052491f, 0.003591f, 0.219249f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[26] = { { 0.006629f, 0.026690f, 0.061870f, 1 }, { 0.805084f, -0.018369f, 0.584788f, -0.097597f } }; outBoneTransform[27] = { { -0.009005f, -0.041708f, 0.037992f, 1 }, { -0.338860f, 0.939952f, -0.007564f, 0.040082f } }; outBoneTransform[28] = { { 0.017136f, -0.032633f, 0.080682f, 1 }, { -0.169466f, 0.800083f, 0.571006f, 0.071415f } }; outBoneTransform[29] = { { 0.011144f, -0.028727f, 0.108366f, 1 }, { -0.076328f, 0.788280f, 0.605097f, 0.081527f } }; outBoneTransform[30] = { { 0.011333f, -0.026044f, 0.128585f, 1 }, { -0.144791f, 0.737451f, 0.656958f, -0.060069f } }; } } else { if (isLeftHand) { outBoneTransform[6] = { { 0.002693f, 0.023387f, 0.013573f, 1 }, { 0.626743f, 0.404630f, -0.499840f, 0.440032f } }; outBoneTransform[7] = { { 0.074204f, -0.005002f, 0.000234f, 1 }, { 0.869067f, -0.019031f, -0.093524f, 0.485400f } }; outBoneTransform[8] = { { 0.043512f, -0.000000f, -0.000000f, 1 }, { 0.834068f, 0.020722f, 0.003930f, 0.551259f } }; outBoneTransform[9] = { { 0.028422f, 0.000000f, 0.000000f, 1 }, { 0.890556f, 0.000289f, -0.009290f, 0.454779f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.003937f, 0.006967f, 0.016424f, 1 }, { 0.531603f, 0.532690f, -0.459598f, 0.471602f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.906933f, -0.142169f, -0.015445f, 0.396261f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.975787f, 0.014996f, 0.010867f, 0.217936f } }; outBoneTransform[14] = { { 0.033266f, 0.000000f, 0.000000f, 1 }, { 0.992777f, -0.002096f, -0.021403f, 0.118029f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.001282f, -0.006612f, 0.016394f, 1 }, { 0.513688f, 0.543325f, -0.502550f, 0.434011f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.971280f, -0.068108f, -0.073480f, 0.215818f } }; outBoneTransform[18] = { { 0.040619f, 0.000000f, 0.000000f, 1 }, { 0.976566f, -0.001379f, 0.000441f, 0.215216f } }; outBoneTransform[19] = { { 0.028715f, -0.000000f, -0.000000f, 1 }, { 0.987232f, -0.000977f, -0.011919f, 0.158838f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { -0.002032f, -0.019020f, 0.015240f, 1 }, { 0.521784f, 0.511917f, -0.594340f, 0.335325f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.982925f, -0.053050f, -0.108004f, 0.139206f } }; outBoneTransform[23] = { { 0.030177f, 0.000000f, 0.000000f, 1 }, { 0.979798f, 0.000394f, -0.001374f, 0.199982f } }; outBoneTransform[24] = { { 0.018187f, 0.000000f, 0.000000f, 1 }, { 0.997410f, -0.000172f, -0.051977f, -0.049724f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[26] = { { -0.004857f, 0.053377f, 0.060017f, 1 }, { 0.751040f, 0.174397f, 0.601473f, 0.209178f } }; outBoneTransform[27] = { { -0.013234f, -0.004327f, 0.069740f, 1 }, { -0.119277f, 0.262590f, -0.888979f, -0.355718f } }; outBoneTransform[28] = { { -0.037500f, -0.074514f, 0.046899f, 1 }, { -0.204942f, 0.706005f, -0.626220f, -0.259623f } }; outBoneTransform[29] = { { -0.036251f, -0.089302f, 0.081732f, 1 }, { -0.194045f, 0.764033f, -0.596592f, -0.150590f } }; outBoneTransform[30] = { { -0.029633f, -0.085595f, 0.119439f, 1 }, { -0.025015f, 0.787219f, -0.601140f, -0.135243f } }; } else { outBoneTransform[6] = { { -0.002693f, 0.023387f, 0.013573f, 1 }, { 0.404698f, -0.626951f, 0.439894f, 0.499645f } }; outBoneTransform[7] = { { -0.074204f, 0.005002f, -0.000234f, 1 }, { 0.870303f, -0.017421f, -0.092515f, 0.483436f } }; outBoneTransform[8] = { { -0.043512f, 0.000000f, 0.000000f, 1 }, { 0.835972f, 0.018944f, 0.003312f, 0.548436f } }; outBoneTransform[9] = { { -0.028422f, -0.000000f, -0.000000f, 1 }, { 0.890326f, 0.000173f, -0.008504f, 0.455244f } }; outBoneTransform[10] = { { -0.022821f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { -0.003937f, 0.006967f, 0.016424f, 1 }, { 0.532293f, -0.531137f, 0.472074f, 0.460113f } }; outBoneTransform[12] = { { -0.070953f, -0.000779f, -0.000997f, 1 }, { 0.908154f, -0.139967f, -0.013210f, 0.394323f } }; outBoneTransform[13] = { { -0.043108f, -0.000000f, -0.000000f, 1 }, { 0.977887f, 0.015350f, 0.008912f, 0.208378f } }; outBoneTransform[14] = { { -0.033266f, -0.000000f, -0.000000f, 1 }, { 0.992487f, -0.002006f, -0.020888f, 0.120540f } }; outBoneTransform[15] = { { -0.025892f, 0.000000f, -0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { -0.001282f, -0.006612f, 0.016394f, 1 }, { 0.544460f, -0.511334f, 0.436935f, 0.501187f } }; outBoneTransform[17] = { { -0.065876f, -0.001786f, -0.000693f, 1 }, { 0.971233f, -0.064561f, -0.071188f, 0.217877f } }; outBoneTransform[18] = { { -0.040619f, -0.000000f, -0.000000f, 1 }, { 0.978211f, -0.001419f, 0.000451f, 0.207607f } }; outBoneTransform[19] = { { -0.028715f, 0.000000f, 0.000000f, 1 }, { 0.987488f, -0.001166f, -0.010852f, 0.157314f } }; outBoneTransform[20] = { { -0.022430f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { 0.002032f, -0.019020f, 0.015240f, 1 }, { 0.513640f, -0.518192f, 0.337332f, 0.594860f } }; outBoneTransform[22] = { { -0.062878f, -0.002844f, -0.000332f, 1 }, { 0.983501f, -0.050059f, -0.104491f, 0.138930f } }; outBoneTransform[23] = { { -0.030177f, -0.000000f, -0.000000f, 1 }, { 0.981170f, 0.000501f, -0.001363f, 0.193138f } }; outBoneTransform[24] = { { -0.018187f, -0.000000f, -0.000000f, 1 }, { 0.997801f, 0.000487f, -0.051933f, -0.041173f } }; outBoneTransform[25] = { { -0.018018f, -0.000000f, 0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[26] = { { 0.004574f, 0.055518f, 0.060226f, 1 }, { 0.745334f, 0.161961f, -0.597782f, -0.246784f } }; outBoneTransform[27] = { { 0.013831f, -0.004360f, 0.069547f, 1 }, { -0.117443f, 0.257604f, 0.890065f, 0.357255f } }; outBoneTransform[28] = { { 0.038220f, -0.074817f, 0.046428f, 1 }, { -0.205767f, 0.697939f, 0.635107f, 0.259191f } }; outBoneTransform[29] = { { 0.035802f, -0.089658f, 0.081733f, 1 }, { -0.196007f, 0.758396f, 0.604341f, 0.145564f } }; outBoneTransform[30] = { { 0.029364f, -0.086069f, 0.119701f, 1 }, { -0.028444f, 0.787767f, 0.601616f, 0.129123f } }; } } } else { // no touch if (isLeftHand) { outBoneTransform[6] = { { 0.000632f, 0.026866f, 0.015002f, 1 }, { 0.644251f, 0.421979f, -0.478202f, 0.422133f } }; outBoneTransform[7] = { { 0.074204f, -0.005002f, 0.000234f, 1 }, { 0.995332f, 0.007007f, -0.039124f, 0.087949f } }; outBoneTransform[8] = { { 0.043930f, -0.000000f, -0.000000f, 1 }, { 0.997891f, 0.045808f, 0.002142f, -0.045943f } }; outBoneTransform[9] = { { 0.028695f, 0.000000f, 0.000000f, 1 }, { 0.999649f, 0.001850f, -0.022782f, -0.013409f } }; outBoneTransform[10] = { { 0.022821f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.546723f, 0.541277f, -0.442520f, 0.460749f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.980294f, -0.167261f, -0.078959f, 0.069368f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.997947f, 0.018493f, 0.013192f, 0.059886f } }; outBoneTransform[14] = { { 0.033266f, 0.000000f, 0.000000f, 1 }, { 0.997394f, -0.003328f, -0.028225f, -0.066315f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.516692f, 0.550144f, -0.495548f, 0.429888f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.990420f, -0.058696f, -0.101820f, 0.072495f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.999545f, -0.002240f, 0.000004f, 0.030081f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.999102f, -0.000721f, -0.012693f, 0.040420f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.526918f, 0.523940f, -0.584025f, 0.326740f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.986609f, -0.059615f, -0.135163f, 0.069132f } }; outBoneTransform[23] = { { 0.030220f, 0.000000f, 0.000000f, 1 }, { 0.994317f, 0.001896f, -0.000132f, 0.106446f } }; outBoneTransform[24] = { { 0.018187f, 0.000000f, 0.000000f, 1 }, { 0.995931f, -0.002010f, -0.052079f, -0.073526f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[26] = { { -0.006059f, 0.056285f, 0.060064f, 1 }, { 0.737238f, 0.202745f, 0.594267f, 0.249441f } }; outBoneTransform[27] = { { -0.040416f, -0.043018f, 0.019345f, 1 }, { -0.290330f, 0.623527f, -0.663809f, -0.293734f } }; outBoneTransform[28] = { { -0.039354f, -0.075674f, 0.047048f, 1 }, { -0.187047f, 0.678062f, -0.659285f, -0.265683f } }; outBoneTransform[29] = { { -0.038340f, -0.090987f, 0.082579f, 1 }, { -0.183037f, 0.736793f, -0.634757f, -0.143936f } }; outBoneTransform[30] = { { -0.031806f, -0.087214f, 0.121015f, 1 }, { -0.003659f, 0.758407f, -0.639342f, -0.126678f } }; } else { outBoneTransform[6] = { { -0.000632f, 0.026866f, 0.015002f, 1 }, { 0.421833f, -0.643793f, 0.422458f, 0.478661f } }; outBoneTransform[7] = { { -0.074204f, 0.005002f, -0.000234f, 1 }, { 0.994784f, 0.007053f, -0.041286f, 0.093009f } }; outBoneTransform[8] = { { -0.043930f, 0.000000f, 0.000000f, 1 }, { 0.998404f, 0.045905f, 0.002780f, -0.032767f } }; outBoneTransform[9] = { { -0.028695f, -0.000000f, -0.000000f, 1 }, { 0.999704f, 0.001955f, -0.022774f, -0.008282f } }; outBoneTransform[10] = { { -0.022821f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, 0.000000f } }; outBoneTransform[11] = { { -0.002177f, 0.007120f, 0.016319f, 1 }, { 0.541874f, -0.547427f, 0.459996f, 0.441701f } }; outBoneTransform[12] = { { -0.070953f, -0.000779f, -0.000997f, 1 }, { 0.979837f, -0.168061f, -0.075910f, 0.076899f } }; outBoneTransform[13] = { { -0.043108f, -0.000000f, -0.000000f, 1 }, { 0.997271f, 0.018278f, 0.013375f, 0.070266f } }; outBoneTransform[14] = { { -0.033266f, -0.000000f, -0.000000f, 1 }, { 0.998402f, -0.003143f, -0.026423f, -0.049849f } }; outBoneTransform[15] = { { -0.025892f, 0.000000f, -0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { -0.000513f, -0.006545f, 0.016348f, 1 }, { 0.548983f, -0.519068f, 0.426914f, 0.496920f } }; outBoneTransform[17] = { { -0.065876f, -0.001786f, -0.000693f, 1 }, { 0.989791f, -0.065882f, -0.096417f, 0.081716f } }; outBoneTransform[18] = { { -0.040697f, -0.000000f, -0.000000f, 1 }, { 0.999102f, -0.002168f, -0.000020f, 0.042317f } }; outBoneTransform[19] = { { -0.028747f, 0.000000f, 0.000000f, 1 }, { 0.998584f, -0.000674f, -0.012714f, 0.051653f } }; outBoneTransform[20] = { { -0.022430f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { 0.002478f, -0.018981f, 0.015214f, 1 }, { 0.518597f, -0.527304f, 0.328264f, 0.587580f } }; outBoneTransform[22] = { { -0.062878f, -0.002844f, -0.000332f, 1 }, { 0.987294f, -0.063356f, -0.125964f, 0.073274f } }; outBoneTransform[23] = { { -0.030220f, -0.000000f, -0.000000f, 1 }, { 0.993413f, 0.001573f, -0.000147f, 0.114578f } }; outBoneTransform[24] = { { -0.018187f, -0.000000f, -0.000000f, 1 }, { 0.997047f, -0.000695f, -0.052009f, -0.056495f } }; outBoneTransform[25] = { { -0.018018f, -0.000000f, 0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[26] = { { 0.005198f, 0.054204f, 0.060030f, 1 }, { 0.747318f, 0.182508f, -0.599586f, -0.220688f } }; outBoneTransform[27] = { { 0.038779f, -0.042973f, 0.019824f, 1 }, { -0.297445f, 0.639373f, 0.648910f, 0.285734f } }; outBoneTransform[28] = { { 0.038027f, -0.074844f, 0.046941f, 1 }, { -0.199898f, 0.698218f, 0.635767f, 0.261406f } }; outBoneTransform[29] = { { 0.036845f, -0.089781f, 0.081973f, 1 }, { -0.190960f, 0.756469f, 0.607591f, 0.148733f } }; outBoneTransform[30] = { { 0.030251f, -0.086056f, 0.119887f, 1 }, { -0.018948f, 0.779249f, 0.612180f, 0.132846f } }; } } } void GetGripClickBoneTransform( bool withController, bool isLeftHand, vr::VRBoneTransform_t outBoneTransform[] ) { if (withController) { if (isLeftHand) { outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.529359f, 0.540512f, -0.463783f, 0.461011f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { -0.831727f, 0.270927f, 0.175647f, -0.451638f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { -0.854886f, -0.008231f, -0.028107f, -0.517990f } }; outBoneTransform[14] = { { 0.033266f, -0.000000f, 0.000000f, 1 }, { -0.825759f, 0.085208f, 0.086456f, -0.550805f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, -0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.500244f, 0.530784f, -0.516215f, 0.448939f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.831617f, -0.242931f, -0.139695f, 0.479461f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.769163f, -0.001746f, 0.001363f, 0.639049f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.968615f, -0.064537f, -0.046586f, 0.235477f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.474671f, 0.434670f, -0.653212f, 0.398827f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.798788f, -0.199577f, -0.094418f, 0.559636f } }; outBoneTransform[23] = { { 0.030220f, 0.000002f, -0.000000f, 1 }, { 0.853087f, 0.001644f, -0.000913f, 0.521765f } }; outBoneTransform[24] = { { 0.018187f, -0.000002f, 0.000000f, 1 }, { 0.974249f, 0.052491f, 0.003591f, 0.219249f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[28] = { { 0.016642f, -0.029992f, 0.083200f, 1 }, { -0.094577f, 0.694550f, 0.702845f, 0.121100f } }; outBoneTransform[29] = { { 0.011144f, -0.028727f, 0.108366f, 1 }, { -0.076328f, 0.788280f, 0.605097f, 0.081527f } }; outBoneTransform[30] = { { 0.011333f, -0.026044f, 0.128585f, 1 }, { -0.144791f, 0.737451f, 0.656958f, -0.060069f } }; } else { outBoneTransform[11] = { { 0.002177f, 0.007120f, 0.016319f, 1 }, { 0.529359f, 0.540512f, -0.463783f, 0.461011f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { -0.831727f, 0.270927f, 0.175647f, -0.451638f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { -0.854886f, -0.008231f, -0.028107f, -0.517990f } }; outBoneTransform[14] = { { 0.033266f, -0.000000f, 0.000000f, 1 }, { -0.825759f, 0.085208f, 0.086456f, -0.550805f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, -0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.000513f, -0.006545f, 0.016348f, 1 }, { 0.500244f, 0.530784f, -0.516215f, 0.448939f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.831617f, -0.242931f, -0.139695f, 0.479461f } }; outBoneTransform[18] = { { 0.040697f, 0.000000f, 0.000000f, 1 }, { 0.769163f, -0.001746f, 0.001363f, 0.639049f } }; outBoneTransform[19] = { { 0.028747f, -0.000000f, -0.000000f, 1 }, { 0.968615f, -0.064537f, -0.046586f, 0.235477f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[21] = { { -0.002478f, -0.018981f, 0.015214f, 1 }, { 0.474671f, 0.434670f, -0.653212f, 0.398827f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.798788f, -0.199577f, -0.094418f, 0.559636f } }; outBoneTransform[23] = { { 0.030220f, 0.000002f, -0.000000f, 1 }, { 0.853087f, 0.001644f, -0.000913f, 0.521765f } }; outBoneTransform[24] = { { 0.018187f, -0.000002f, 0.000000f, 1 }, { 0.974249f, 0.052491f, 0.003591f, 0.219249f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[28] = { { 0.016642f, -0.029992f, 0.083200f, 1 }, { -0.094577f, 0.694550f, 0.702845f, 0.121100f } }; outBoneTransform[29] = { { 0.011144f, -0.028727f, 0.108366f, 1 }, { -0.076328f, 0.788280f, 0.605097f, 0.081527f } }; outBoneTransform[30] = { { 0.011333f, -0.026044f, 0.128585f, 1 }, { -0.144791f, 0.737451f, 0.656958f, -0.060069f } }; } } else { if (isLeftHand) { outBoneTransform[11] = { { 0.005787f, 0.006806f, 0.016534f, 1 }, { 0.514203f, 0.522315f, -0.478348f, 0.483700f } }; outBoneTransform[12] = { { 0.070953f, 0.000779f, 0.000997f, 1 }, { 0.723653f, -0.097901f, 0.048546f, 0.681458f } }; outBoneTransform[13] = { { 0.043108f, 0.000000f, 0.000000f, 1 }, { 0.637464f, -0.002366f, -0.002831f, 0.770472f } }; outBoneTransform[14] = { { 0.033266f, 0.000000f, 0.000000f, 1 }, { 0.658008f, 0.002610f, 0.003196f, 0.753000f } }; outBoneTransform[15] = { { 0.025892f, -0.000000f, 0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { 0.004123f, -0.006858f, 0.016563f, 1 }, { 0.489609f, 0.523374f, -0.520644f, 0.463997f } }; outBoneTransform[17] = { { 0.065876f, 0.001786f, 0.000693f, 1 }, { 0.759970f, -0.055609f, 0.011571f, 0.647471f } }; outBoneTransform[18] = { { 0.040331f, 0.000000f, 0.000000f, 1 }, { 0.664315f, 0.001595f, 0.001967f, 0.747449f } }; outBoneTransform[19] = { { 0.028489f, -0.000000f, -0.000000f, 1 }, { 0.626957f, -0.002784f, -0.003234f, 0.779042f } }; outBoneTransform[20] = { { 0.022430f, -0.000000f, 0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { 0.001131f, -0.019295f, 0.015429f, 1 }, { 0.479766f, 0.477833f, -0.630198f, 0.379934f } }; outBoneTransform[22] = { { 0.062878f, 0.002844f, 0.000332f, 1 }, { 0.827001f, 0.034282f, 0.003440f, 0.561144f } }; outBoneTransform[23] = { { 0.029874f, 0.000000f, 0.000000f, 1 }, { 0.702185f, -0.006716f, -0.009289f, 0.711903f } }; outBoneTransform[24] = { { 0.017979f, 0.000000f, 0.000000f, 1 }, { 0.676853f, 0.007956f, 0.009917f, 0.736009f } }; outBoneTransform[25] = { { 0.018018f, 0.000000f, -0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[28] = { { 0.000448f, 0.001536f, 0.116543f, 1 }, { -0.039357f, 0.105143f, -0.928833f, -0.353079f } }; outBoneTransform[29] = { { 0.003949f, -0.014869f, 0.130608f, 1 }, { -0.055071f, 0.068695f, -0.944016f, -0.317933f } }; outBoneTransform[30] = { { 0.003263f, -0.034685f, 0.139926f, 1 }, { 0.019690f, -0.100741f, -0.957331f, -0.270149f } }; } else { outBoneTransform[11] = { { -0.005787f, 0.006806f, 0.016534f, 1 }, { 0.522315f, -0.514203f, 0.483700f, 0.478348f } }; outBoneTransform[12] = { { -0.070953f, -0.000779f, -0.000997f, 1 }, { 0.723653f, -0.097901f, 0.048546f, 0.681458f } }; outBoneTransform[13] = { { -0.043108f, -0.000000f, -0.000000f, 1 }, { 0.637464f, -0.002366f, -0.002831f, 0.770472f } }; outBoneTransform[14] = { { -0.033266f, -0.000000f, -0.000000f, 1 }, { 0.658008f, 0.002610f, 0.003196f, 0.753000f } }; outBoneTransform[15] = { { -0.025892f, 0.000000f, -0.000000f, 1 }, { 0.999195f, 0.000000f, 0.000000f, 0.040126f } }; outBoneTransform[16] = { { -0.004123f, -0.006858f, 0.016563f, 1 }, { 0.523374f, -0.489609f, 0.463997f, 0.520644f } }; outBoneTransform[17] = { { -0.065876f, -0.001786f, -0.000693f, 1 }, { 0.759970f, -0.055609f, 0.011571f, 0.647471f } }; outBoneTransform[18] = { { -0.040331f, -0.000000f, -0.000000f, 1 }, { 0.664315f, 0.001595f, 0.001967f, 0.747449f } }; outBoneTransform[19] = { { -0.028489f, 0.000000f, 0.000000f, 1 }, { 0.626957f, -0.002784f, -0.003234f, 0.779042f } }; outBoneTransform[20] = { { -0.022430f, 0.000000f, -0.000000f, 1 }, { 1.000000f, 0.000000f, 0.000000f, 0.000000f } }; outBoneTransform[21] = { { -0.001131f, -0.019295f, 0.015429f, 1 }, { 0.477833f, -0.479766f, 0.379935f, 0.630198f } }; outBoneTransform[22] = { { -0.062878f, -0.002844f, -0.000332f, 1 }, { 0.827001f, 0.034282f, 0.003440f, 0.561144f } }; outBoneTransform[23] = { { -0.029874f, -0.000000f, -0.000000f, 1 }, { 0.702185f, -0.006716f, -0.009289f, 0.711903f } }; outBoneTransform[24] = { { -0.017979f, -0.000000f, -0.000000f, 1 }, { 0.676853f, 0.007956f, 0.009917f, 0.736009f } }; outBoneTransform[25] = { { -0.018018f, -0.000000f, 0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, -0.000000f } }; outBoneTransform[28] = { { -0.000448f, 0.001536f, 0.116543f, 1 }, { -0.039357f, 0.105143f, 0.928833f, 0.353079f } }; outBoneTransform[29] = { { -0.003949f, -0.014869f, 0.130608f, 1 }, { -0.055071f, 0.068695f, 0.944016f, 0.317933f } }; outBoneTransform[30] = { { -0.003263f, -0.034685f, 0.139926f, 1 }, { 0.019690f, -0.100741f, 0.957331f, 0.270149f } }; } } } void Controller::GetBoneTransform(bool withController, vr::VRBoneTransform_t outBoneTransform[]) { auto isLeftHand = device_id == HAND_LEFT_ID; vr::VRBoneTransform_t boneTransform1[SKELETON_BONE_COUNT]; vr::VRBoneTransform_t boneTransform2[SKELETON_BONE_COUNT]; // root and wrist outBoneTransform[0] = { { 0.000000f, 0.000000f, 0.000000f, 1 }, { 1.000000f, -0.000000f, -0.000000f, 0.000000f } }; if (isLeftHand) { outBoneTransform[1] = { { -0.034038f, 0.036503f, 0.164722f, 1 }, { -0.055147f, -0.078608f, -0.920279f, 0.379296f } }; } else { outBoneTransform[1] = { { 0.034038f, 0.036503f, 0.164722f, 1 }, { -0.055147f, -0.078608f, 0.920279f, -0.379296f } }; } // thumb GetThumbBoneTransform(withController, isLeftHand, m_lastThumbTouch, boneTransform1); GetThumbBoneTransform(withController, isLeftHand, m_currentThumbTouch, boneTransform2); for (int boneIdx = 2; boneIdx < 6; boneIdx++) { outBoneTransform[boneIdx].position = Lerp( boneTransform1[boneIdx].position, boneTransform2[boneIdx].position, m_thumbTouchAnimationProgress ); outBoneTransform[boneIdx].orientation = Slerp( boneTransform1[boneIdx].orientation, boneTransform2[boneIdx].orientation, m_thumbTouchAnimationProgress ); } // trigger (index to pinky) if (m_triggerValue > 0) { GetTriggerBoneTransform(withController, isLeftHand, true, false, boneTransform1); GetTriggerBoneTransform(withController, isLeftHand, true, true, boneTransform2); for (int boneIdx = 6; boneIdx < SKELETON_BONE_COUNT; boneIdx++) { outBoneTransform[boneIdx].position = Lerp( boneTransform1[boneIdx].position, boneTransform2[boneIdx].position, m_triggerValue ); outBoneTransform[boneIdx].orientation = Slerp( boneTransform1[boneIdx].orientation, boneTransform2[boneIdx].orientation, m_triggerValue ); } } else { GetTriggerBoneTransform( withController, isLeftHand, m_lastTriggerTouch, false, boneTransform1 ); GetTriggerBoneTransform( withController, isLeftHand, m_currentTriggerTouch, false, boneTransform2 ); for (int boneIdx = 6; boneIdx < SKELETON_BONE_COUNT; boneIdx++) { outBoneTransform[boneIdx].position = Lerp( boneTransform1[boneIdx].position, boneTransform2[boneIdx].position, m_indexTouchAnimationProgress ); outBoneTransform[boneIdx].orientation = Slerp( boneTransform1[boneIdx].orientation, boneTransform2[boneIdx].orientation, m_indexTouchAnimationProgress ); } } // grip (middle to pinky) if (m_gripValue > 0) { GetGripClickBoneTransform(withController, isLeftHand, boneTransform2); for (int boneIdx = 11; boneIdx < 26; boneIdx++) { outBoneTransform[boneIdx].position = Lerp( outBoneTransform[boneIdx].position, boneTransform2[boneIdx].position, m_gripValue ); outBoneTransform[boneIdx].orientation = Slerp( outBoneTransform[boneIdx].orientation, boneTransform2[boneIdx].orientation, m_gripValue ); } for (int boneIdx = 28; boneIdx < SKELETON_BONE_COUNT; boneIdx++) { outBoneTransform[boneIdx].position = Lerp( outBoneTransform[boneIdx].position, boneTransform2[boneIdx].position, m_gripValue ); outBoneTransform[boneIdx].orientation = Slerp( outBoneTransform[boneIdx].orientation, boneTransform2[boneIdx].orientation, m_gripValue ); } } } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Controller.h ================================================ #pragma once #include "ALVR-common/packet_types.h" #include "TrackedDevice.h" #include "openvr_driver_wrap.h" #include class Controller : public TrackedDevice { public: Controller(uint64_t deviceID, vr::EVRSkeletalTrackingLevel skeletonLevel); virtual ~Controller() {}; void RegisterButton(uint64_t id); void SetButton(uint64_t id, FfiButtonValue value); bool OnPoseUpdate(uint64_t targetTimestampNs, float predictionS, FfiHandData handData); private: static const int SKELETON_BONE_COUNT = 31; static const int ANIMATION_FRAME_COUNT = 15; std::map m_buttonHandles; vr::VRInputComponentHandle_t m_compHaptic; vr::VRInputComponentHandle_t m_compSkeleton = vr::k_ulInvalidInputComponentHandle; vr::EVRSkeletalTrackingLevel m_skeletonLevel; uint64_t m_poseTargetTimestampNs; // These variables are used for controller hand animation // todo: move to rust float m_thumbTouchAnimationProgress = 0; float m_indexTouchAnimationProgress = 0; bool m_currentThumbTouch = false; bool m_lastThumbTouch = false; bool m_currentTriggerTouch = false; bool m_lastTriggerTouch = false; float m_triggerValue = 0; float m_gripValue = 0; vr::VRInputComponentHandle_t getHapticComponent(); void GetBoneTransform(bool withController, vr::VRBoneTransform_t outBoneTransform[]); // TrackedDevice bool activate() final; void* get_component([[maybe_unused]] const char* component_name_and_version) final { return nullptr; } }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/FakeViveTracker.cpp ================================================ #include "FakeViveTracker.h" #include "Logger.h" #include "Paths.h" #include "Settings.h" #include "Utils.h" #include "bindings.h" #include FakeViveTracker::FakeViveTracker(uint64_t deviceID) : TrackedDevice(deviceID, vr::TrackedDeviceClass_GenericTracker) { } bool FakeViveTracker::activate() { Debug("FakeViveTracker::Activate"); auto vr_properties = vr::VRProperties(); // Normally a vive tracker emulator would (logically) always set the tracking system to // "lighthouse" but in order to do space calibration with existing tools such as OpenVR Space // calibrator and be able to calibrate to/from ALVR HMD (and the proxy tracker) space to/from a // native HMD/tracked device which is already using "lighthouse" as the tracking system the // proxy tracker needs to be in a different tracking system to treat them differently and // prevent those tools doing the same space transform to the proxy tracker. vr_properties->SetStringProperty( this->prop_container, vr::Prop_TrackingSystemName_String, "ALVRTrackerCustom" ); //"lighthouse"); vr_properties->SetStringProperty( this->prop_container, vr::Prop_ModelNumber_String, "Vive Tracker Pro MV" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_SerialNumber_String, this->get_serial_number().c_str() ); // Changed vr_properties->SetStringProperty( this->prop_container, vr::Prop_RenderModelName_String, "{htc}vr_tracker_vive_1_0" ); vr_properties->SetBoolProperty(this->prop_container, vr::Prop_WillDriftInYaw_Bool, false); vr_properties->SetStringProperty(this->prop_container, vr::Prop_ManufacturerName_String, "HTC"); vr_properties->SetStringProperty( this->prop_container, vr::Prop_TrackingFirmwareVersion_String, "1541800000 RUNNER-WATCHMAN$runner-watchman@runner-watchman 2018-01-01 FPGA 512(2.56/0/0) " "BL 0 VRC 1541800000 Radio 1518800000" ); // Changed vr_properties->SetStringProperty( this->prop_container, vr::Prop_HardwareRevision_String, "product 128 rev 2.5.6 lot 2000/0/0 0" ); // Changed vr_properties->SetStringProperty( this->prop_container, vr::Prop_ConnectedWirelessDongle_String, "D0000BE000" ); // Changed vr_properties->SetBoolProperty(this->prop_container, vr::Prop_DeviceIsWireless_Bool, true); vr_properties->SetBoolProperty(this->prop_container, vr::Prop_DeviceIsCharging_Bool, false); vr_properties->SetFloatProperty( this->prop_container, vr::Prop_DeviceBatteryPercentage_Float, 1.f ); // Always charged vr::HmdMatrix34_t l_transform = { { { -1.f, 0.f, 0.f, 0.f }, { 0.f, 0.f, -1.f, 0.f }, { 0.f, -1.f, 0.f, 0.f } } }; vr_properties->SetProperty( this->prop_container, vr::Prop_StatusDisplayTransform_Matrix34, &l_transform, sizeof(vr::HmdMatrix34_t), vr::k_unHmdMatrix34PropertyTag ); vr_properties->SetBoolProperty( this->prop_container, vr::Prop_Firmware_UpdateAvailable_Bool, false ); vr_properties->SetBoolProperty( this->prop_container, vr::Prop_Firmware_ManualUpdate_Bool, false ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_Firmware_ManualUpdateURL_String, "https://developer.valvesoftware.com/wiki/SteamVR/HowTo_Update_Firmware" ); vr_properties->SetUint64Property( this->prop_container, vr::Prop_HardwareRevision_Uint64, 2214720000 ); // Changed vr_properties->SetUint64Property( this->prop_container, vr::Prop_FirmwareVersion_Uint64, 1541800000 ); // Changed vr_properties->SetUint64Property( this->prop_container, vr::Prop_FPGAVersion_Uint64, 512 ); // Changed vr_properties->SetUint64Property( this->prop_container, vr::Prop_VRCVersion_Uint64, 1514800000 ); // Changed vr_properties->SetUint64Property( this->prop_container, vr::Prop_RadioVersion_Uint64, 1518800000 ); // Changed vr_properties->SetUint64Property( this->prop_container, vr::Prop_DongleVersion_Uint64, 8933539758 ); // Changed, based on vr::Prop_ConnectedWirelessDongle_String above vr_properties->SetBoolProperty( this->prop_container, vr::Prop_DeviceProvidesBatteryStatus_Bool, true ); vr_properties->SetBoolProperty(this->prop_container, vr::Prop_DeviceCanPowerOff_Bool, true); vr_properties->SetStringProperty( this->prop_container, vr::Prop_Firmware_ProgrammingTarget_String, this->get_serial_number().c_str() ); vr_properties->SetInt32Property( this->prop_container, vr::Prop_DeviceClass_Int32, vr::TrackedDeviceClass_GenericTracker ); vr_properties->SetBoolProperty( this->prop_container, vr::Prop_Firmware_ForceUpdateRequired_Bool, false ); vr_properties->SetStringProperty(this->prop_container, vr::Prop_ResourceRoot_String, "htc"); const char* name; if (this->device_id == BODY_CHEST_ID) { name = "ALVR/tracker/chest"; } else if (this->device_id == BODY_HIPS_ID) { name = "ALVR/tracker/waist"; } else if (this->device_id == BODY_LEFT_FOOT_ID) { name = "ALVR/tracker/left_foot"; } else if (this->device_id == BODY_RIGHT_FOOT_ID) { name = "ALVR/tracker/right_foot"; } else if (this->device_id == BODY_LEFT_KNEE_ID) { name = "ALVR/tracker/left_knee"; } else if (this->device_id == BODY_RIGHT_KNEE_ID) { name = "ALVR/tracker/right_knee"; } else if (this->device_id == BODY_LEFT_ELBOW_ID) { name = "ALVR/tracker/left_elbow"; } else if (this->device_id == BODY_RIGHT_ELBOW_ID) { name = "ALVR/tracker/right_elbow"; } else { name = "ALVR/tracker/unknown"; } vr_properties->SetStringProperty( this->prop_container, vr::Prop_RegisteredDeviceType_String, name ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_InputProfilePath_String, "{htc}/input/vive_tracker_profile.json" ); vr_properties->SetBoolProperty(this->prop_container, vr::Prop_Identifiable_Bool, false); vr_properties->SetBoolProperty( this->prop_container, vr::Prop_Firmware_RemindUpdate_Bool, false ); vr_properties->SetInt32Property( this->prop_container, vr::Prop_ControllerRoleHint_Int32, vr::TrackedControllerRole_Invalid ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_ControllerType_String, "vive_tracker_waist" ); vr_properties->SetInt32Property( this->prop_container, vr::Prop_ControllerHandSelectionPriority_Int32, -1 ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceOff_String, "{htc}/icons/tracker_status_off.png" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceSearching_String, "{htc}/icons/tracker_status_searching.gif" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceSearchingAlert_String, "{htc}/icons/tracker_status_searching_alert.gif" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceReady_String, "{htc}/icons/tracker_status_ready.png" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceReadyAlert_String, "{htc}/icons/tracker_status_ready_alert.png" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceNotReady_String, "{htc}/icons/tracker_status_error.png" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceStandby_String, "{htc}/icons/tracker_status_standby.png" ); vr_properties->SetStringProperty( this->prop_container, vr::Prop_NamedIconPathDeviceAlertLow_String, "{htc}/icons/tracker_status_ready_low.png" ); vr_properties->SetBoolProperty(this->prop_container, vr::Prop_HasDisplayComponent_Bool, false); vr_properties->SetBoolProperty(this->prop_container, vr::Prop_HasCameraComponent_Bool, false); vr_properties->SetBoolProperty( this->prop_container, vr::Prop_HasDriverDirectModeComponent_Bool, false ); vr_properties->SetBoolProperty( this->prop_container, vr::Prop_HasVirtualDisplayComponent_Bool, false ); return true; } void FakeViveTracker::OnPoseUpdated(uint64_t targetTimestampNs, const FfiDeviceMotion* motion) { if (this->object_id == vr::k_unTrackedDeviceIndexInvalid) { return; } bool tracked = motion != nullptr; auto pose = vr::DriverPose_t {}; pose.poseIsValid = tracked; pose.deviceIsConnected = tracked; pose.result = tracked ? vr::TrackingResult_Running_OK : vr::TrackingResult_Uninitialized; pose.qWorldFromDriverRotation = HmdQuaternion_Init(1, 0, 0, 0); pose.qDriverFromHeadRotation = HmdQuaternion_Init(1, 0, 0, 0); if (motion != nullptr) { pose.qRotation = HmdQuaternion_Init( motion->pose.orientation.w, motion->pose.orientation.x, motion->pose.orientation.y, motion->pose.orientation.z ); pose.vecPosition[0] = motion->pose.position[0]; pose.vecPosition[1] = motion->pose.position[1]; pose.vecPosition[2] = motion->pose.position[2]; } this->submit_pose(pose); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/FakeViveTracker.h ================================================ #pragma once #include "TrackedDevice.h" #include "bindings.h" #include "openvr_driver_wrap.h" class FakeViveTracker : public TrackedDevice { public: FakeViveTracker(uint64_t deviceID); void OnPoseUpdated(uint64_t targetTimestampNs, const FfiDeviceMotion* motion); private: // TrackedDevice bool activate() final; void* get_component(const char*) final { return nullptr; } }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/HMD.cpp ================================================ #include "HMD.h" #include "Controller.h" #include "Logger.h" #include "Paths.h" #include "PoseHistory.h" #include "Settings.h" #include "Utils.h" #include "ViveTrackerProxy.h" #include "bindings.h" #ifdef _WIN32 #include "platform/win32/CEncoder.h" #elif __APPLE__ #include "platform/macos/CEncoder.h" #else #include "platform/linux/CEncoder.h" #endif Hmd::Hmd() : TrackedDevice( HEAD_ID, Settings::Instance().m_TrackingRefOnly ? vr::TrackedDeviceClass_TrackingReference : vr::TrackedDeviceClass_HMD ) , m_baseComponentsInitialized(false) , m_streamComponentsInitialized(false) { Debug("Hmd::constructor"); auto dummy_fov = FfiFov { -1.0, 1.0, 1.0, -1.0 }; auto dummy_pose = FfiPose { { 0, 0, 0, 1 }, { 0, 0, 0 } }; auto dummy_view_params = FfiViewParams { dummy_pose, dummy_fov }; this->view_params[0] = dummy_view_params; this->view_params[1] = dummy_view_params; m_poseHistory = std::make_shared(); if (Settings::Instance().m_enableViveTrackerProxy) { m_viveTrackerProxy = std::make_unique(*this); if (!vr::VRServerDriverHost()->TrackedDeviceAdded( m_viveTrackerProxy->GetSerialNumber(), vr::TrackedDeviceClass_GenericTracker, m_viveTrackerProxy.get() )) { Warn("Failed to register Vive tracker"); } } } Hmd::~Hmd() { Debug("Hmd::destructor"); if (m_encoder) { Debug("Hmd::~Hmd(): Stopping encoder...\n"); m_encoder->Stop(); m_encoder.reset(); } #ifdef _WIN32 if (m_D3DRender) { m_D3DRender->Shutdown(); m_D3DRender.reset(); } #endif } bool Hmd::activate() { Debug("Hmd::Activate"); auto vr_properties = vr::VRProperties(); SetOpenvrProps((void*)this, this->device_id); vr_properties->SetFloatProperty( this->prop_container, vr::Prop_DisplayFrequency_Float, static_cast(Settings::Instance().m_refreshRate) ); vr::VRDriverInput()->CreateBooleanComponent(this->prop_container, "/proximity", &m_proximity); #ifdef _WIN32 float originalIPD = vr::VRSettings()->GetFloat(vr::k_pch_SteamVR_Section, vr::k_pch_SteamVR_IPD_Float); vr::VRSettings()->SetFloat(vr::k_pch_SteamVR_Section, vr::k_pch_SteamVR_IPD_Float, 0.063); #endif HmdMatrix_SetIdentity(&m_eyeToHeadLeft); HmdMatrix_SetIdentity(&m_eyeToHeadRight); // Disable async reprojection on Linux. Windows interface uses IVRDriverDirectModeComponent // which never applies reprojection // Also Disable async reprojection on vulkan #ifndef _WIN32 vr::VRSettings()->SetBool( vr::k_pch_SteamVR_Section, vr::k_pch_SteamVR_EnableLinuxVulkanAsync_Bool, Settings::Instance().m_enableLinuxVulkanAsyncCompute ); vr::VRSettings()->SetBool( vr::k_pch_SteamVR_Section, vr::k_pch_SteamVR_DisableAsyncReprojection_Bool, !Settings::Instance().m_enableLinuxAsyncReprojection ); #endif if (!m_baseComponentsInitialized) { m_baseComponentsInitialized = true; if (this->device_class == vr::TrackedDeviceClass_HMD) { #ifdef _WIN32 m_D3DRender = std::make_shared(); // Use the same adapter as vrcompositor uses. If another adapter is used, vrcompositor // says "failed to open shared texture" and then crashes. It seems vrcompositor selects // always(?) first adapter. vrcompositor may use Intel iGPU when user sets it as primary // adapter. I don't know what happens on laptop which support optimus. // Prop_GraphicsAdapterLuid_Uint64 is only for redirect display and is ignored on direct // mode driver. So we can't specify an adapter for vrcompositor. m_nAdapterIndex is set // 0 on the dashboard. if (!m_D3DRender->Initialize(Settings::Instance().m_nAdapterIndex)) { Error( "Could not create graphics device for adapter %d. Requires a minimum of two " "graphics cards.\n", Settings::Instance().m_nAdapterIndex ); return false; } int32_t nDisplayAdapterIndex; if (!m_D3DRender->GetAdapterInfo(&nDisplayAdapterIndex, m_adapterName)) { Error("Failed to get primary adapter info!\n"); return false; } Info("Using %ls as primary graphics adapter.\n", m_adapterName.c_str()); Info("OSVer: %ls\n", GetWindowsOSVersion().c_str()); m_directModeComponent = std::make_shared(m_D3DRender, m_poseHistory); #endif } DriverReadyIdle(this->device_class == vr::TrackedDeviceClass_HMD); } if (this->device_class == vr::TrackedDeviceClass_HMD) { vr::VREvent_Data_t eventData; eventData.ipd = { 0.063 }; vr::VRServerDriverHost()->VendorSpecificEvent( this->object_id, vr::VREvent_IpdChanged, eventData, 0 ); } return true; } void* Hmd::get_component(const char* component_name_and_version) { Debug("Hmd::GetComponent %s", component_name_and_version); // NB: "this" pointer needs to be statically cast to point to the correct vtable auto name_and_vers = std::string(component_name_and_version); if (name_and_vers == vr::IVRDisplayComponent_Version) { return (vr::IVRDisplayComponent*)this; } #ifdef _WIN32 if (name_and_vers == vr::IVRDriverDirectModeComponent_Version) { return m_directModeComponent.get(); } #endif return nullptr; } void Hmd::OnPoseUpdated(uint64_t targetTimestampNs, FfiDeviceMotion motion) { Debug("Hmd::OnPoseUpdated"); if (this->object_id == vr::k_unTrackedDeviceIndexInvalid) { return; } auto pose = vr::DriverPose_t {}; pose.poseIsValid = true; pose.result = vr::TrackingResult_Running_OK; pose.deviceIsConnected = true; pose.qWorldFromDriverRotation = HmdQuaternion_Init(1, 0, 0, 0); pose.qDriverFromHeadRotation = HmdQuaternion_Init(1, 0, 0, 0); pose.qRotation = HmdQuaternion_Init( motion.pose.orientation.w, motion.pose.orientation.x, motion.pose.orientation.y, motion.pose.orientation.z ); pose.vecPosition[0] = motion.pose.position[0]; pose.vecPosition[1] = motion.pose.position[1]; pose.vecPosition[2] = motion.pose.position[2]; this->submit_pose(pose); m_poseHistory->OnPoseUpdated(targetTimestampNs, motion); if (m_viveTrackerProxy) m_viveTrackerProxy->update(); #if !defined(_WIN32) && !defined(__APPLE__) // This has to be set after initialization is done, because something in vrcompositor is // setting it to 90Hz in the meantime if (!m_refreshRateSet && m_encoder && m_encoder->IsConnected()) { m_refreshRateSet = true; vr::VRProperties()->SetFloatProperty( this->prop_container, vr::Prop_DisplayFrequency_Float, static_cast(Settings::Instance().m_refreshRate) ); } #endif } void Hmd::StartStreaming() { Debug("Hmd::StartStreaming"); vr::VRDriverInput()->UpdateBooleanComponent(m_proximity, true, 0.0); if (m_streamComponentsInitialized) { return; } // Spin up a separate thread to handle the overlapped encoding/transmit step. if (this->device_class == vr::TrackedDeviceClass_HMD) { #ifdef _WIN32 m_encoder = std::make_shared(); try { m_encoder->Initialize(m_D3DRender); } catch (Exception e) { Error( "Your GPU does not meet the requirements for video encoding. %s %s\n%s %s\n", "If you get this error after changing some settings, you can revert them by", "deleting the file \"session.json\" in the installation folder.", "Failed to initialize CEncoder:", e.what() ); } m_encoder->Start(); m_directModeComponent->SetEncoder(m_encoder); #elif __APPLE__ m_encoder = std::make_shared(); #else m_encoder = std::make_shared(m_poseHistory); m_encoder->Start(); #endif m_encoder->OnStreamStart(); } m_streamComponentsInitialized = true; } void Hmd::StopStreaming() { Debug("Hmd::StopStreaming"); vr::VRDriverInput()->UpdateBooleanComponent(m_proximity, false, 0.0); } void Hmd::SetViewParams(const FfiViewParams params[2]) { Debug("Hmd::SetViewParams"); this->view_params[0] = params[0]; this->view_params[1] = params[1]; // The OpenXR spec defines the HMD position as the midpoint // between the eyes, so conversion to this is handled by the // client. auto left_transform = pose_to_mat(params[0].pose); auto right_transform = pose_to_mat(params[1].pose); vr::VRServerDriverHost()->SetDisplayEyeToHead(object_id, left_transform, right_transform); auto left_proj = fov_to_tangents(params[0].fov); auto right_proj = fov_to_tangents(params[1].fov); vr::VRServerDriverHost()->SetDisplayProjectionRaw(object_id, left_proj, right_proj); #ifdef _WIN32 if (m_encoder) { m_encoder->SetViewParams(left_proj, left_transform, right_proj, right_transform); } #endif // todo: check if this is still needed vr::VRServerDriverHost()->VendorSpecificEvent( object_id, vr::VREvent_LensDistortionChanged, {}, 0 ); } void Hmd::SetProximityState(bool headsetIsWorn) { vr::VRDriverInput()->UpdateBooleanComponent(m_proximity, headsetIsWorn, 0.0); } void Hmd::GetWindowBounds(int32_t* pnX, int32_t* pnY, uint32_t* pnWidth, uint32_t* pnHeight) { Debug( "Hmd::GetWindowBounds %dx%d - %dx%d\n", 0, 0, Settings::Instance().m_renderWidth, Settings::Instance().m_renderHeight ); *pnX = 0; *pnY = 0; *pnWidth = Settings::Instance().m_renderWidth; *pnHeight = Settings::Instance().m_renderHeight; } bool Hmd::IsDisplayRealDisplay() { #ifdef _WIN32 return false; #else return true; #endif } void Hmd::GetRecommendedRenderTargetSize(uint32_t* pnWidth, uint32_t* pnHeight) { *pnWidth = Settings::Instance().m_recommendedTargetWidth / 2; *pnHeight = Settings::Instance().m_recommendedTargetHeight; Debug("Hmd::GetRecommendedRenderTargetSize %dx%d\n", *pnWidth, *pnHeight); } void Hmd::GetEyeOutputViewport( vr::EVREye eEye, uint32_t* pnX, uint32_t* pnY, uint32_t* pnWidth, uint32_t* pnHeight ) { *pnY = 0; *pnWidth = Settings::Instance().m_renderWidth / 2; *pnHeight = Settings::Instance().m_renderHeight; if (eEye == vr::Eye_Left) { *pnX = 0; } else { *pnX = Settings::Instance().m_renderWidth / 2; } Debug("Hmd::GetEyeOutputViewport Eye=%d %dx%d %dx%d\n", eEye, *pnX, *pnY, *pnWidth, *pnHeight); } void Hmd::GetProjectionRaw(vr::EVREye eye, float* left, float* right, float* top, float* bottom) { auto proj = fov_to_tangents(this->view_params[eye].fov); *left = proj.vTopLeft.v[0]; *right = proj.vBottomRight.v[0]; *top = proj.vTopLeft.v[1]; *bottom = proj.vBottomRight.v[1]; Debug("Hmd::GetProjectionRaw Eye=%d %f %f %f %f\n", eye, *left, *right, *top, *bottom); } vr::DistortionCoordinates_t Hmd::ComputeDistortion(vr::EVREye, float u, float v) { return { { u, v }, { u, v }, { u, v } }; } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/HMD.h ================================================ #pragma once #include "ALVR-common/packet_types.h" #include "TrackedDevice.h" #include "openvr_driver_wrap.h" #include #ifdef _WIN32 #include "platform/win32/OvrDirectModeComponent.h" #endif class Controller; class Controller; class ViveTrackerProxy; class CEncoder; #ifdef _WIN32 class CD3DRender; #endif class PoseHistory; class Hmd : public TrackedDevice, vr::IVRDisplayComponent { public: std::shared_ptr m_poseHistory; std::shared_ptr m_encoder; Hmd(); virtual ~Hmd(); void OnPoseUpdated(uint64_t targetTimestampNs, FfiDeviceMotion motion); void StartStreaming(); void StopStreaming(); void SetViewParams(const FfiViewParams params[2]); void SetProximityState(bool headsetIsWorn); private: vr::VRInputComponentHandle_t m_proximity; FfiViewParams view_params[2]; bool m_baseComponentsInitialized; bool m_streamComponentsInitialized; vr::HmdMatrix34_t m_eyeToHeadLeft; vr::HmdMatrix34_t m_eyeToHeadRight; vr::HmdRect2_t m_eyeFoVLeft; vr::HmdRect2_t m_eyeFoVRight; std::wstring m_adapterName; #ifdef _WIN32 std::shared_ptr m_D3DRender; #endif #ifdef _WIN32 std::shared_ptr m_directModeComponent; #endif std::shared_ptr m_viveTrackerProxy; #ifndef _WIN32 bool m_refreshRateSet = false; #endif // TrackedDevice virtual bool activate() final; virtual void* get_component(const char* component_name_and_version) final; // IVRDisplayComponent virtual void GetWindowBounds(int32_t* x, int32_t* y, uint32_t* width, uint32_t* height); virtual bool IsDisplayOnDesktop() { return false; } virtual bool IsDisplayRealDisplay(); virtual void GetRecommendedRenderTargetSize(uint32_t* width, uint32_t* height); virtual void GetEyeOutputViewport( vr::EVREye eye, uint32_t* x, uint32_t* y, uint32_t* width, uint32_t* height ); virtual void GetProjectionRaw(vr::EVREye eEye, float* pfLeft, float* pfRight, float* pfTop, float* pfBottom); virtual vr::DistortionCoordinates_t ComputeDistortion(vr::EVREye eEye, float fU, float fV); }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/IDRScheduler.cpp ================================================ #include "IDRScheduler.h" #include "Utils.h" #include IDRScheduler::IDRScheduler() { } IDRScheduler::~IDRScheduler() { } void IDRScheduler::OnStreamStart() { m_minIDRFrameInterval = Settings::Instance().m_minimumIdrIntervalMs * 1000; m_scheduled = false; InsertIDR(); } void IDRScheduler::InsertIDR() { std::unique_lock lock(m_mutex); m_insertIDRTime = GetTimestampUs() - MIN_IDR_FRAME_INTERVAL * 2; m_scheduled = true; } bool IDRScheduler::CheckIDRInsertion() { std::unique_lock lock(m_mutex); if (m_scheduled) { if (m_insertIDRTime <= GetTimestampUs()) { m_scheduled = false; return true; } } return false; } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/IDRScheduler.h ================================================ #pragma once #include "Settings.h" #include #include class IDRScheduler { public: IDRScheduler(); ~IDRScheduler(); void OnStreamStart(); void InsertIDR(); bool CheckIDRInsertion(); private: static const int MIN_IDR_FRAME_INTERVAL = 100 * 1000; // 100-milliseconds uint64_t m_insertIDRTime = 0; bool m_scheduled = false; std::mutex m_mutex; uint64_t m_minIDRFrameInterval = MIN_IDR_FRAME_INTERVAL; }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Logger.cpp ================================================ #include "Logger.h" #include #include "bindings.h" #include "driverlog.h" void _log(const char* format, va_list args, void (*logFn)(const char*), bool driverLog = false) { char buf[1024]; int count = vsnprintf(buf, sizeof(buf), format, args); if (count > (int)sizeof(buf)) count = (int)sizeof(buf); if (count > 0 && buf[count - 1] == '\n') buf[count - 1] = '\0'; logFn(buf); if (driverLog) DriverLog(buf); } Exception MakeException(const char* format, ...) { va_list args; va_start(args, format); Exception e = FormatExceptionV(format, args); va_end(args); return e; } void Error(const char* format, ...) { va_list args; va_start(args, format); _log(format, args, LogError, true); va_end(args); } void Warn(const char* format, ...) { va_list args; va_start(args, format); _log(format, args, LogWarn, true); va_end(args); } void Info(const char* format, ...) { va_list args; va_start(args, format); // Don't log to SteamVR/writing to file for info level, this is mostly statistics info _log(format, args, LogInfo); va_end(args); } void Debug(const char* format, ...) { #ifdef ALVR_DEBUG_LOG va_list args; va_start(args, format); _log(format, args, LogDebug); va_end(args); #else (void)format; #endif } void LogPeriod(const char* tag, const char* format, ...) { va_list args; va_start(args, format); char buf[1024]; int count = vsnprintf(buf, sizeof(buf), format, args); if (count > (int)sizeof(buf)) count = (int)sizeof(buf); if (count > 0 && buf[count - 1] == '\n') buf[count - 1] = '\0'; LogPeriodically(tag, buf); va_end(args); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Logger.h ================================================ #pragma once #include "ALVR-common/exception.h" Exception MakeException(const char* format, ...); void Error(const char* format, ...); void Warn(const char* format, ...); void Info(const char* format, ...); void Debug(const char* format, ...); void LogPeriod(const char* tag, const char* format, ...); ================================================ FILE: alvr/server_openvr/cpp/alvr_server/NalParsing.cpp ================================================ #include "Logger.h" #include "Settings.h" #include "Utils.h" #include "bindings.h" #include #include static const char NAL_PREFIX_3B[] = { 0x00, 0x00, 0x01 }; static const char NAL_PREFIX_4B[] = { 0x00, 0x00, 0x00, 0x01 }; static const unsigned char H264_NAL_TYPE_SPS = 7; static const unsigned char HEVC_NAL_TYPE_VPS = 32; static const unsigned char H264_NAL_TYPE_AUD = 9; static const unsigned char HEVC_NAL_TYPE_AUD = 35; int8_t getNalPrefixSize(unsigned char* buf) { if (memcmp(buf, NAL_PREFIX_3B, sizeof(NAL_PREFIX_3B)) == 0) { return sizeof(NAL_PREFIX_3B); } else if (memcmp(buf, NAL_PREFIX_4B, sizeof(NAL_PREFIX_4B)) == 0) { return sizeof(NAL_PREFIX_4B); } else { return -1; } } /* Sends the (VPS + )SPS + PPS video configuration headers from H.264 or H.265 stream as a sequence of NALs. (VPS + )SPS + PPS have short size (8bytes + 28bytes in some environment), so we can assume SPS + PPS is contained in first fragment. */ void sendHeaders(int codec, unsigned char*& buf, int& len, int nalNum) { unsigned char* cursor = buf; int headersLen = 0; int foundHeaders = -1; // Offset by 1 header to find the length until the next header while (headersLen <= len) { if (headersLen + sizeof(NAL_PREFIX_4B) > (unsigned)len) { cursor++; headersLen++; continue; } int8_t prefixSize = getNalPrefixSize(cursor); if (prefixSize == -1) { cursor++; headersLen++; continue; } foundHeaders++; if (foundHeaders == nalNum) { break; } headersLen += prefixSize; cursor += prefixSize; } if (foundHeaders != nalNum) { return; } SetVideoConfigNals((const unsigned char*)buf, headersLen, codec); // move the cursor forward excluding config NALs buf = cursor; len -= headersLen; } void processH264Nals(unsigned char*& buf, int& len) { unsigned char prefixSize = getNalPrefixSize(buf); unsigned char nalType = buf[prefixSize] & 0x1F; if (nalType == H264_NAL_TYPE_AUD && len > prefixSize * 2 + 2) { buf += prefixSize + 2; len -= prefixSize + 2; prefixSize = getNalPrefixSize(buf); nalType = buf[prefixSize] & 0x1F; } if (nalType == H264_NAL_TYPE_SPS) { sendHeaders(ALVR_CODEC_H264, buf, len, 2); // 2 headers SPS and PPS } } void processHevcNals(unsigned char*& buf, int& len) { unsigned char prefixSize = getNalPrefixSize(buf); unsigned char nalType = (buf[prefixSize] >> 1) & 0x3F; if (nalType == HEVC_NAL_TYPE_AUD && len > prefixSize * 2 + 3) { buf += prefixSize + 3; len -= prefixSize + 3; prefixSize = getNalPrefixSize(buf); nalType = (buf[prefixSize] >> 1) & 0x3F; } if (nalType == HEVC_NAL_TYPE_VPS) { sendHeaders(ALVR_CODEC_HEVC, buf, len, 3); // 3 headers VPS, SPS and PPS } } void ParseFrameNals( int codec, unsigned char* buf, int len, unsigned long long targetTimestampNs, bool isIdr ) { static bool av1GotFrame = false; if ((unsigned)len < sizeof(NAL_PREFIX_4B)) { return; } if (codec == ALVR_CODEC_H264) { processH264Nals(buf, len); } else if (codec == ALVR_CODEC_HEVC) { processHevcNals(buf, len); } else if (codec == ALVR_CODEC_AV1 && !av1GotFrame) { av1GotFrame = true; SetVideoConfigNals(0, 0, codec); } VideoSend(targetTimestampNs, buf, len, isIdr); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Paths.cpp ================================================ #include "Paths.h" #include "bindings.h" uint64_t HEAD_ID; uint64_t HAND_LEFT_ID; uint64_t HAND_RIGHT_ID; uint64_t HAND_TRACKER_LEFT_ID; uint64_t HAND_TRACKER_RIGHT_ID; uint64_t BODY_CHEST_ID; uint64_t BODY_HIPS_ID; uint64_t BODY_LEFT_ELBOW_ID; uint64_t BODY_RIGHT_ELBOW_ID; uint64_t BODY_LEFT_KNEE_ID; uint64_t BODY_LEFT_FOOT_ID; uint64_t BODY_RIGHT_KNEE_ID; uint64_t BODY_RIGHT_FOOT_ID; uint64_t LEFT_A_TOUCH_ID; uint64_t LEFT_B_TOUCH_ID; uint64_t LEFT_X_TOUCH_ID; uint64_t LEFT_Y_TOUCH_ID; uint64_t LEFT_TRACKPAD_TOUCH_ID; uint64_t LEFT_THUMBSTICK_TOUCH_ID; uint64_t LEFT_THUMBREST_TOUCH_ID; uint64_t LEFT_TRIGGER_TOUCH_ID; uint64_t LEFT_TRIGGER_VALUE_ID; uint64_t LEFT_SQUEEZE_VALUE_ID; uint64_t RIGHT_A_TOUCH_ID; uint64_t RIGHT_B_TOUCH_ID; uint64_t RIGHT_TRACKPAD_TOUCH_ID; uint64_t RIGHT_THUMBSTICK_TOUCH_ID; uint64_t RIGHT_THUMBREST_TOUCH_ID; uint64_t RIGHT_TRIGGER_TOUCH_ID; uint64_t RIGHT_TRIGGER_VALUE_ID; uint64_t RIGHT_SQUEEZE_VALUE_ID; std::set BODY_IDS; std::map LEFT_CONTROLLER_BUTTON_MAPPING; std::map RIGHT_CONTROLLER_BUTTON_MAPPING; std::map> ALVR_TO_STEAMVR_PATH_IDS; void init_paths() { HEAD_ID = PathStringToHash("/user/head"); HAND_LEFT_ID = PathStringToHash("/user/hand/left"); HAND_RIGHT_ID = PathStringToHash("/user/hand/right"); HAND_TRACKER_LEFT_ID = PathStringToHash("/user/hand_tracker/left"); HAND_TRACKER_RIGHT_ID = PathStringToHash("/user/hand_tracker/right"); BODY_CHEST_ID = PathStringToHash("/user/body/chest"); BODY_HIPS_ID = PathStringToHash("/user/body/waist"); BODY_LEFT_ELBOW_ID = PathStringToHash("/user/body/left_elbow"); BODY_RIGHT_ELBOW_ID = PathStringToHash("/user/body/right_elbow"); BODY_LEFT_KNEE_ID = PathStringToHash("/user/body/left_knee"); BODY_LEFT_FOOT_ID = PathStringToHash("/user/body/left_foot"); BODY_RIGHT_KNEE_ID = PathStringToHash("/user/body/right_knee"); BODY_RIGHT_FOOT_ID = PathStringToHash("/user/body/right_foot"); LEFT_A_TOUCH_ID = PathStringToHash("/user/hand/left/input/a/touch"); LEFT_B_TOUCH_ID = PathStringToHash("/user/hand/left/input/b/touch"); LEFT_X_TOUCH_ID = PathStringToHash("/user/hand/left/input/x/touch"); LEFT_Y_TOUCH_ID = PathStringToHash("/user/hand/left/input/y/touch"); LEFT_TRACKPAD_TOUCH_ID = PathStringToHash("/user/hand/left/input/trackpad/touch"); LEFT_THUMBSTICK_TOUCH_ID = PathStringToHash("/user/hand/left/input/thumbstick/touch"); LEFT_THUMBREST_TOUCH_ID = PathStringToHash("/user/hand/left/input/thumbrest/touch"); LEFT_TRIGGER_TOUCH_ID = PathStringToHash("/user/hand/left/input/trigger/touch"); LEFT_TRIGGER_VALUE_ID = PathStringToHash("/user/hand/left/input/trigger/value"); LEFT_SQUEEZE_VALUE_ID = PathStringToHash("/user/hand/left/input/squeeze/value"); RIGHT_A_TOUCH_ID = PathStringToHash("/user/hand/right/input/a/touch"); RIGHT_B_TOUCH_ID = PathStringToHash("/user/hand/right/input/b/touch"); RIGHT_TRACKPAD_TOUCH_ID = PathStringToHash("/user/hand/right/input/trackpad/touch"); RIGHT_THUMBSTICK_TOUCH_ID = PathStringToHash("/user/hand/right/input/thumbstick/touch"); RIGHT_THUMBREST_TOUCH_ID = PathStringToHash("/user/hand/right/input/thumbrest/touch"); RIGHT_TRIGGER_TOUCH_ID = PathStringToHash("/user/hand/right/input/trigger/touch"); RIGHT_TRIGGER_VALUE_ID = PathStringToHash("/user/hand/right/input/trigger/value"); RIGHT_SQUEEZE_VALUE_ID = PathStringToHash("/user/hand/right/input/squeeze/value"); BODY_IDS.insert(BODY_CHEST_ID); BODY_IDS.insert(BODY_HIPS_ID); BODY_IDS.insert(BODY_LEFT_ELBOW_ID); BODY_IDS.insert(BODY_RIGHT_ELBOW_ID); BODY_IDS.insert(BODY_LEFT_KNEE_ID); BODY_IDS.insert(BODY_LEFT_FOOT_ID); BODY_IDS.insert(BODY_RIGHT_KNEE_ID); BODY_IDS.insert(BODY_RIGHT_FOOT_ID); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/system/click"), { { "/input/system/click", "/input/left_ps/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/system/touch"), { { "/input/system/touch", "/input/left_ps/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/menu/click"), { { "/input/system/click", "/input/application_menu/click", "/input/create/click" }, ButtonType::Binary } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/menu/touch"), { { "/input/system/touch", "/input/application_menu/touch", "/input/create/touch" }, ButtonType::Binary } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/a/click"), { { "/input/a/click", "/input/cross/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/a/touch"), { { "/input/a/touch", "/input/cross/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/b/click"), { { "/input/b/click", "/input/circle/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/b/touch"), { { "/input/b/touch", "/input/circle/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/x/click"), { { "/input/x/click", "/input/square/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/x/touch"), { { "/input/x/touch", "/input/square/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/y/click"), { { "/input/y/click", "/input/triangle/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/y/touch"), { { "/input/y/touch", "/input/triangle/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/squeeze/click"), { { "/input/grip/click", "/input/l1/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/squeeze/touch"), { { "/input/grip/touch", "/input/l1/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/squeeze/value"), { { "/input/grip/value", "/input/l1/value" }, ButtonType::ScalarOneSided } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/squeeze/force"), { { "/input/grip/force" }, ButtonType::ScalarOneSided } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trigger/click"), { { "/input/trigger/click", "/input/l2/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trigger/value"), { { "/input/trigger/value", "/input/l2/value" }, ButtonType::ScalarOneSided } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trigger/touch"), { { "/input/trigger/touch", "/input/l2/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/thumbstick/x"), { { "/input/joystick/x", "/input/thumbstick/x", "/input/left_stick/x" }, ButtonType::ScalarTwoSided } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/thumbstick/y"), { { "/input/joystick/y", "/input/thumbstick/y", "/input/left_stick/y" }, ButtonType::ScalarTwoSided } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/thumbstick/click"), { { "/input/joystick/click", "/input/thumbstick/click", "/input/left_stick/click" }, ButtonType::Binary } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/thumbstick/touch"), { { "/input/joystick/touch", "/input/thumbstick/touch", "/input/left_stick/touch" }, ButtonType::Binary } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trackpad/x"), { { "/input/trackpad/x" }, ButtonType::ScalarTwoSided } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trackpad/y"), { { "/input/trackpad/y" }, ButtonType::ScalarTwoSided } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trackpad/click" ), { { "/input/trackpad/click" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/left/input/trackpad/force"), { { "/input/trackpad/force" }, ButtonType::ScalarOneSided } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/trackpad/touch" ), { { "/input/trackpad/touch" }, ButtonType::Binary } }); LEFT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/left/input/thumbrest/touch" ), { { "/input/thumbrest/touch" }, ButtonType::Binary } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/system/click"), { { "/input/system/click", "/input/right_ps/click" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/system/touch"), { { "/input/system/touch", "/input/right_ps/touch" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/menu/click"), { { "/input/system/click", "/input/application_menu/click", "/input/options/click" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/menu/touch"), { { "/input/system/touch", "/input/application_menu/touch", "/input/options/touch" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/right/input/a/click"), { { "/input/a/click", "/input/cross/click" }, ButtonType::Binary } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/right/input/a/touch"), { { "/input/a/touch", "/input/cross/touch" }, ButtonType::Binary } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/right/input/b/click"), { { "/input/b/click", "/input/circle/click" }, ButtonType::Binary } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/right/input/b/touch"), { { "/input/b/touch", "/input/circle/touch" }, ButtonType::Binary } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/squeeze/click"), { { "/input/grip/click", "/input/r1/click" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/squeeze/touch"), { { "/input/grip/touch", "/input/r1/touch" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/squeeze/value"), { { "/input/grip/value", "/input/r1/value" }, ButtonType::ScalarOneSided } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/squeeze/force"), { { "/input/grip/force" }, ButtonType::ScalarOneSided } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/trigger/click"), { { "/input/trigger/click", "/input/r2/click" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/trigger/value"), { { "/input/trigger/value", "/input/r2/value" }, ButtonType::ScalarOneSided } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/trigger/touch"), { { "/input/trigger/touch", "/input/r2/touch" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/thumbstick/x"), { { "/input/joystick/x", "/input/thumbstick/x", "/input/right_stick/x" }, ButtonType::ScalarTwoSided } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/thumbstick/y"), { { "/input/joystick/y", "/input/thumbstick/y", "/input/right_stick/y" }, ButtonType::ScalarTwoSided } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/thumbstick/click"), { { "/input/joystick/click", "/input/thumbstick/click", "/input/right_stick/click" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/thumbstick/touch"), { { "/input/joystick/touch", "/input/thumbstick/touch", "/input/right_stick/touch" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/right/input/trackpad/x"), { { "/input/trackpad/x" }, ButtonType::ScalarTwoSided } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert({ PathStringToHash("/user/hand/right/input/trackpad/y"), { { "/input/trackpad/y" }, ButtonType::ScalarTwoSided } }); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/trackpad/click"), { { "/input/trackpad/click" }, ButtonType::Binary } } ); LEFT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/trackpad/force"), { { "/input/trackpad/force" }, ButtonType::ScalarOneSided } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/trackpad/touch"), { { "/input/trackpad/touch" }, ButtonType::Binary } } ); RIGHT_CONTROLLER_BUTTON_MAPPING.insert( { PathStringToHash("/user/hand/right/input/thumbrest/touch"), { { "/input/thumbrest/touch" }, ButtonType::Binary } } ); for (auto hand : { LEFT_CONTROLLER_BUTTON_MAPPING, RIGHT_CONTROLLER_BUTTON_MAPPING }) { for (auto info : hand) { std::vector ids; for (auto path : info.second.steamvr_paths) { ids.push_back(PathStringToHash(path)); } ALVR_TO_STEAMVR_PATH_IDS.insert({ info.first, ids }); } } } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Paths.h ================================================ #pragma once #include #include #include #include #include "openvr_driver_wrap.h" extern uint64_t HEAD_ID; extern uint64_t HAND_LEFT_ID; extern uint64_t HAND_RIGHT_ID; extern uint64_t HAND_TRACKER_LEFT_ID; extern uint64_t HAND_TRACKER_RIGHT_ID; extern uint64_t BODY_CHEST_ID; extern uint64_t BODY_HIPS_ID; extern uint64_t BODY_LEFT_ELBOW_ID; extern uint64_t BODY_RIGHT_ELBOW_ID; extern uint64_t BODY_LEFT_KNEE_ID; extern uint64_t BODY_LEFT_FOOT_ID; extern uint64_t BODY_RIGHT_KNEE_ID; extern uint64_t BODY_RIGHT_FOOT_ID; // These values are needed to determine the hand skeleton when holding a controller. // todo: move inferred hand skeleton to rust extern uint64_t LEFT_A_TOUCH_ID; extern uint64_t LEFT_B_TOUCH_ID; extern uint64_t LEFT_X_TOUCH_ID; extern uint64_t LEFT_Y_TOUCH_ID; extern uint64_t LEFT_TRACKPAD_TOUCH_ID; extern uint64_t LEFT_THUMBSTICK_TOUCH_ID; extern uint64_t LEFT_THUMBREST_TOUCH_ID; extern uint64_t LEFT_TRIGGER_TOUCH_ID; extern uint64_t LEFT_TRIGGER_VALUE_ID; extern uint64_t LEFT_SQUEEZE_TOUCH_ID; extern uint64_t LEFT_SQUEEZE_VALUE_ID; extern uint64_t RIGHT_A_TOUCH_ID; extern uint64_t RIGHT_B_TOUCH_ID; extern uint64_t RIGHT_TRACKPAD_TOUCH_ID; extern uint64_t RIGHT_THUMBSTICK_TOUCH_ID; extern uint64_t RIGHT_THUMBREST_TOUCH_ID; extern uint64_t RIGHT_TRIGGER_TOUCH_ID; extern uint64_t RIGHT_TRIGGER_VALUE_ID; extern uint64_t RIGHT_SQUEEZE_TOUCH_ID; extern uint64_t RIGHT_SQUEEZE_VALUE_ID; enum class ButtonType { Binary, ScalarOneSided, ScalarTwoSided, }; struct ButtonInfo { std::vector steamvr_paths; ButtonType type; }; extern std::set BODY_IDS; extern std::map LEFT_CONTROLLER_BUTTON_MAPPING; extern std::map RIGHT_CONTROLLER_BUTTON_MAPPING; extern std::map> ALVR_TO_STEAMVR_PATH_IDS; void init_paths(); ================================================ FILE: alvr/server_openvr/cpp/alvr_server/PoseHistory.cpp ================================================ #include "PoseHistory.h" #include "Logger.h" #include "Utils.h" #include "include/openvr_math.h" #include #include void PoseHistory::OnPoseUpdated(uint64_t targetTimestampNs, FfiDeviceMotion motion) { // Put pose history buffer TrackingHistoryFrame history; history.targetTimestampNs = targetTimestampNs; history.motion = motion; HmdMatrix_QuatToMat( motion.pose.orientation.w, motion.pose.orientation.x, motion.pose.orientation.y, motion.pose.orientation.z, &history.rotationMatrix ); std::unique_lock lock(m_mutex); if (!m_transformIdentity) { vr::HmdMatrix34_t rotation = vrmath::matMul33(m_transform, history.rotationMatrix); history.rotationMatrix = rotation; } if (m_poseBuffer.size() == 0) { m_poseBuffer.push_back(history); } else { if (m_poseBuffer.back().targetTimestampNs != targetTimestampNs) { // New track info m_poseBuffer.push_back(history); } } // The value should match with the client's MAXIMUM_TRACKING_FRAMES in ovr_context.cpp if (m_poseBuffer.size() > 120 * 3) { m_poseBuffer.pop_front(); } } std::optional PoseHistory::GetBestPoseMatch(const vr::HmdMatrix34_t& pose) const { std::unique_lock lock(m_mutex); float minDiff = 100000; auto minIt = m_poseBuffer.begin(); for (auto it = m_poseBuffer.begin(); it != m_poseBuffer.end(); ++it) { float distance = 0; // Rotation matrix composes a part of ViewMatrix of TrackingInfo. // Be carefull of transpose. // And bottom side and right side of matrix should not be compared, because pPose does not // contain that part of matrix. for (int i = 0; i < 3; i++) { for (int j = 0; j < 3; j++) { distance += pow(it->rotationMatrix.m[j][i] - pose.m[j][i], 2); } } if (minDiff > distance) { minIt = it; minDiff = distance; } } if (minIt != m_poseBuffer.end()) { return *minIt; } Debug("PoseHistory::GetBestPoseMatch: No pose matched."); return {}; } std::optional PoseHistory::GetPoseAt(uint64_t timestampNs ) const { std::unique_lock lock(m_mutex); for (auto it = m_poseBuffer.rbegin(), end = m_poseBuffer.rend(); it != end; ++it) { if (it->targetTimestampNs == timestampNs) return *it; } Debug("PoseHistory::GetPoseAt: No pose matched."); return {}; } void PoseHistory::SetTransform(const vr::HmdMatrix34_t& transform) { std::unique_lock lock(m_mutex); m_transform = transform; for (int i = 0; i < 3; ++i) { for (int j = 0; j < 3; ++j) { if (transform.m[i][j] != (i == j ? 1 : 0)) { m_transformIdentity = false; return; } } } m_transformIdentity = true; } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/PoseHistory.h ================================================ #pragma once #include "ALVR-common/packet_types.h" #include "openvr_driver_wrap.h" #include #include #include class PoseHistory { public: struct TrackingHistoryFrame { uint64_t targetTimestampNs; FfiDeviceMotion motion; vr::HmdMatrix34_t rotationMatrix; }; void OnPoseUpdated(uint64_t targetTimestampNs, FfiDeviceMotion motion); std::optional GetBestPoseMatch(const vr::HmdMatrix34_t& pose) const; // Return the most recent pose known at the given timestamp std::optional GetPoseAt(uint64_t timestampNs) const; void SetTransform(const vr::HmdMatrix34_t& transform); private: mutable std::mutex m_mutex; std::list m_poseBuffer; vr::HmdMatrix34_t m_transform = { { { 1.0, 0.0, 0.0, 0.0 }, { 0.0, 1.0, 0.0, 0.0 }, { 0.0, 0.0, 1.0, 0.0 } } }; bool m_transformIdentity = true; }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Settings.cpp ================================================ #include "Settings.h" #include "Logger.h" #include "bindings.h" #include #include #include #include #include #define PICOJSON_USE_INT64 #include "include/picojson.h" using namespace std; extern uint64_t g_DriverTestMode; Settings Settings::m_Instance; Settings::Settings() : m_loaded(false) { } Settings::~Settings() { } void Settings::Load() { try { auto sessionFile = std::ifstream(g_sessionPath); auto json = std::string( std::istreambuf_iterator(sessionFile), std::istreambuf_iterator() ); picojson::value v; std::string err = picojson::parse(v, json); if (!err.empty()) { Error("Error on parsing session config (%s): %hs\n", g_sessionPath, err.c_str()); return; } auto config = v.get("openvr_config"); m_refreshRate = (int)config.get("refresh_rate").get(); m_renderWidth = config.get("eye_resolution_width").get() * 2; m_renderHeight = config.get("eye_resolution_height").get(); m_recommendedTargetWidth = config.get("target_eye_resolution_width").get() * 2; m_recommendedTargetHeight = config.get("target_eye_resolution_height").get(); m_nAdapterIndex = (int32_t)config.get("adapter_index").get(); m_captureFrameDir = config.get("capture_frame_dir").get(); m_enableFoveatedEncoding = config.get("enable_foveated_encoding").get(); m_foveationCenterSizeX = (float)config.get("foveation_center_size_x").get(); m_foveationCenterSizeY = (float)config.get("foveation_center_size_y").get(); m_foveationCenterShiftX = (float)config.get("foveation_center_shift_x").get(); m_foveationCenterShiftY = (float)config.get("foveation_center_shift_y").get(); m_foveationEdgeRatioX = (float)config.get("foveation_edge_ratio_x").get(); m_foveationEdgeRatioY = (float)config.get("foveation_edge_ratio_y").get(); m_enableColorCorrection = config.get("enable_color_correction").get(); m_brightness = (float)config.get("brightness").get(); m_contrast = (float)config.get("contrast").get(); m_saturation = (float)config.get("saturation").get(); m_gamma = (float)config.get("gamma").get(); m_sharpening = (float)config.get("sharpening").get(); m_codec = (int32_t)config.get("codec").get(); m_h264Profile = (int32_t)config.get("h264_profile").get(); m_rateControlMode = (uint32_t)config.get("rate_control_mode").get(); m_fillerData = config.get("filler_data").get(); m_entropyCoding = (uint32_t)config.get("entropy_coding").get(); m_use10bitEncoder = config.get("use_10bit_encoder").get(); m_encodingGamma = config.get("encoding_gamma").get(); m_enableHdr = config.get("enable_hdr").get(); m_forceHdrSrgbCorrection = config.get("force_hdr_srgb_correction").get(); m_clampHdrExtendedRange = config.get("clamp_hdr_extended_range").get(); m_enableAmfPreAnalysis = config.get("enable_amf_pre_analysis").get(); m_enableVbaq = config.get("enable_vbaq").get(); m_enableAmfHmqb = config.get("enable_amf_hmqb").get(); m_useAmfPreproc = config.get("use_amf_preproc").get(); m_amfPreProcSigma = (uint32_t)config.get("amf_preproc_sigma").get(); m_amfPreProcTor = (uint32_t)config.get("amf_preproc_tor").get(); m_encoderQualityPreset = (uint32_t)config.get("encoder_quality_preset").get(); m_amdBitrateCorruptionFix = (bool)config.get("amd_bitrate_corruption_fix").get(); m_nvencQualityPreset = (uint32_t)config.get("nvenc_quality_preset").get(); m_force_sw_encoding = config.get("force_sw_encoding").get(); m_swThreadCount = (int32_t)config.get("sw_thread_count").get(); m_nvencTuningPreset = (uint32_t)config.get("nvenc_tuning_preset").get(); m_nvencMultiPass = (uint32_t)config.get("nvenc_multi_pass").get(); m_nvencAdaptiveQuantizationMode = (uint32_t)config.get("nvenc_adaptive_quantization_mode").get(); m_nvencLowDelayKeyFrameScale = config.get("nvenc_low_delay_key_frame_scale").get(); m_nvencRefreshRate = config.get("nvenc_refresh_rate").get(); m_nvencEnableIntraRefresh = config.get("enable_intra_refresh").get(); m_nvencIntraRefreshPeriod = config.get("intra_refresh_period").get(); m_nvencIntraRefreshCount = config.get("intra_refresh_count").get(); m_nvencMaxNumRefFrames = config.get("max_num_ref_frames").get(); m_nvencGopLength = config.get("gop_length").get(); m_nvencPFrameStrategy = config.get("p_frame_strategy").get(); m_nvencRateControlMode = config.get("nvenc_rate_control_mode").get(); m_nvencRcBufferSize = config.get("rc_buffer_size").get(); m_nvencRcInitialDelay = config.get("rc_initial_delay").get(); m_nvencRcMaxBitrate = config.get("rc_max_bitrate").get(); m_nvencRcAverageBitrate = config.get("rc_average_bitrate").get(); m_nvencEnableWeightedPrediction = config.get("nvenc_enable_weighted_prediction").get(); m_minimumIdrIntervalMs = config.get("minimum_idr_interval_ms").get(); m_enableViveTrackerProxy = config.get("enable_vive_tracker_proxy").get(); m_TrackingRefOnly = config.get("tracking_ref_only").get(); m_enableLinuxVulkanAsyncCompute = config.get("linux_async_compute").get(); m_enableLinuxAsyncReprojection = config.get("linux_async_reprojection").get(); m_enableControllers = config.get("controllers_enabled").get(); m_controllerIsTracker = config.get("controller_is_tracker").get(); m_enableBodyTrackingFakeVive = config.get("body_tracking_vive_enabled").get(); m_bodyTrackingHasLegs = config.get("body_tracking_has_legs").get(); m_useSeparateHandTrackers = config.get("use_separate_hand_trackers").get(); Info("Render Target: %d %d\n", m_renderWidth, m_renderHeight); Info("Refresh Rate: %d\n", m_refreshRate); m_loaded = true; } catch (std::exception& e) { Error("Exception on parsing session config (%s): %hs\n", g_sessionPath, e.what()); } } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Settings.h ================================================ #pragma once #include "ALVR-common/packet_types.h" #include class Settings { static Settings m_Instance; bool m_loaded; Settings(); virtual ~Settings(); public: void Load(); static Settings& Instance() { return m_Instance; } bool IsLoaded() { return m_loaded; } int m_refreshRate; uint32_t m_renderWidth; uint32_t m_renderHeight; int32_t m_recommendedTargetWidth; int32_t m_recommendedTargetHeight; int32_t m_nAdapterIndex; std::string m_captureFrameDir; bool m_enableFoveatedEncoding; float m_foveationCenterSizeX; float m_foveationCenterSizeY; float m_foveationCenterShiftX; float m_foveationCenterShiftY; float m_foveationEdgeRatioX; float m_foveationEdgeRatioY; bool m_enableColorCorrection; float m_brightness; float m_contrast; float m_saturation; float m_gamma; float m_sharpening; int m_codec; int m_h264Profile; bool m_use10bitEncoder; double m_encodingGamma; bool m_enableHdr; bool m_forceHdrSrgbCorrection; bool m_clampHdrExtendedRange; bool m_enableAmfPreAnalysis; bool m_enableVbaq; bool m_enableAmfHmqb; bool m_useAmfPreproc; uint32_t m_amfPreProcSigma; uint32_t m_amfPreProcTor; uint32_t m_encoderQualityPreset; bool m_amdBitrateCorruptionFix; uint32_t m_nvencQualityPreset; uint32_t m_rateControlMode; bool m_fillerData; uint32_t m_entropyCoding; bool m_force_sw_encoding; uint32_t m_swThreadCount; uint32_t m_nvencTuningPreset; uint32_t m_nvencMultiPass; uint32_t m_nvencAdaptiveQuantizationMode; int64_t m_nvencLowDelayKeyFrameScale; int64_t m_nvencRefreshRate; bool m_nvencEnableIntraRefresh; int64_t m_nvencIntraRefreshPeriod; int64_t m_nvencIntraRefreshCount; int64_t m_nvencMaxNumRefFrames; int64_t m_nvencGopLength; int64_t m_nvencPFrameStrategy; int64_t m_nvencRateControlMode; int64_t m_nvencRcBufferSize; int64_t m_nvencRcInitialDelay; int64_t m_nvencRcMaxBitrate; int64_t m_nvencRcAverageBitrate; bool m_nvencEnableWeightedPrediction; uint64_t m_minimumIdrIntervalMs; bool m_enableViveTrackerProxy = false; bool m_TrackingRefOnly = false; bool m_enableLinuxVulkanAsyncCompute; bool m_enableLinuxAsyncReprojection; bool m_enableControllers; int m_controllerIsTracker = false; int m_enableBodyTrackingFakeVive = false; int m_bodyTrackingHasLegs = false; bool m_useSeparateHandTrackers = false; }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/TrackedDevice.cpp ================================================ #include "TrackedDevice.h" #include "Logger.h" #include "Utils.h" #include #include TrackedDevice::TrackedDevice(uint64_t device_id, vr::ETrackedDeviceClass device_class) : device_id(device_id) , device_class(device_class) { this->last_pose = vr::DriverPose_t {}; this->last_pose.poseIsValid = false; this->last_pose.deviceIsConnected = false; this->last_pose.result = vr::TrackingResult_Uninitialized; this->last_pose.qDriverFromHeadRotation = HmdQuaternion_Init(1, 0, 0, 0); this->last_pose.qWorldFromDriverRotation = HmdQuaternion_Init(1, 0, 0, 0); this->last_pose.qRotation = HmdQuaternion_Init(1, 0, 0, 0); } std::string TrackedDevice::get_serial_number() { auto size = GetSerialNumber(this->device_id, nullptr); auto buffer = std::vector(size); GetSerialNumber(this->device_id, &buffer[0]); return std::string(&buffer[0]); } void TrackedDevice::set_prop(FfiOpenvrProperty prop) { if (this->object_id == vr::k_unTrackedDeviceIndexInvalid) { return; } auto key = (vr::ETrackedDeviceProperty)prop.key; auto props = vr::VRProperties(); vr::ETrackedPropertyError result; if (prop.type == FfiOpenvrPropertyType::Bool) { result = props->SetBoolProperty(this->prop_container, key, prop.value.bool_); } else if (prop.type == FfiOpenvrPropertyType::Float) { result = props->SetFloatProperty(this->prop_container, key, prop.value.float_); } else if (prop.type == FfiOpenvrPropertyType::Int32) { result = props->SetInt32Property(this->prop_container, key, prop.value.int32); } else if (prop.type == FfiOpenvrPropertyType::Uint64) { result = props->SetUint64Property(this->prop_container, key, prop.value.uint64); } else if (prop.type == FfiOpenvrPropertyType::Vector3) { auto vec3 = vr::HmdVector3_t {}; vec3.v[0] = prop.value.vector3[0]; vec3.v[1] = prop.value.vector3[1]; vec3.v[2] = prop.value.vector3[2]; result = props->SetVec3Property(this->prop_container, key, vec3); } else if (prop.type == FfiOpenvrPropertyType::Double) { result = props->SetDoubleProperty(this->prop_container, key, prop.value.double_); } else if (prop.type == FfiOpenvrPropertyType::String) { result = props->SetStringProperty(this->prop_container, key, prop.value.string); } else { Error("Unreachable"); result = vr::TrackedProp_Success; } if (result != vr::TrackedProp_Success) { Error( "Error setting property %d: %s", key, vr::VRPropertiesRaw()->GetPropErrorNameFromEnum(result) ); } auto event_data = vr::VREvent_Data_t {}; event_data.property.container = this->prop_container; event_data.property.prop = key; vr::VRServerDriverHost()->VendorSpecificEvent( this->object_id, vr::VREvent_PropertyChanged, event_data, 0. ); } void TrackedDevice::submit_pose(vr::DriverPose_t pose) { this->last_pose = pose; vr::VRServerDriverHost()->TrackedDevicePoseUpdated( this->object_id, pose, sizeof(vr::DriverPose_t) ); } bool TrackedDevice::register_device(bool await_activation) { if (!vr::VRServerDriverHost()->TrackedDeviceAdded( this->get_serial_number().c_str(), this->device_class, (vr::ITrackedDeviceServerDriver*)this )) { Error("Failed to register device (%s)", this->get_serial_number().c_str()); return false; } if (await_activation) { auto lock = std::unique_lock(this->activation_mutex); this->activation_condvar.wait_for(lock, std::chrono::seconds(1), [this] { return this->activation_state != ActivationState::Pending; }); return this->activation_state == ActivationState::Success; } else { return true; } } vr::EVRInitError TrackedDevice::Activate(vr::TrackedDeviceIndex_t object_id) { this->object_id = object_id; this->prop_container = vr::VRProperties()->TrackedDeviceToPropertyContainer(this->object_id); { auto guard = std::lock_guard(this->activation_mutex); if (this->activate()) { this->activation_state = ActivationState::Success; } else { this->activation_state = ActivationState::Failure; } } this->activation_condvar.notify_one(); return this->activation_state == ActivationState::Success ? vr::VRInitError_None : vr::VRInitError_Driver_Failed; } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/TrackedDevice.h ================================================ #pragma once #include "bindings.h" #include "openvr_driver_wrap.h" #include #include #include #include enum class ActivationState { Pending, Success, Failure, }; class TrackedDevice : vr::ITrackedDeviceServerDriver { public: vr::TrackedDeviceIndex_t object_id = vr::k_unTrackedDeviceIndexInvalid; vr::PropertyContainerHandle_t prop_container = vr::k_ulInvalidPropertyContainer; vr::DriverPose_t last_pose; bool register_device(bool await_activation); void set_prop(FfiOpenvrProperty prop); protected: uint64_t device_id; vr::ETrackedDeviceClass device_class; TrackedDevice(uint64_t device_id, vr::ETrackedDeviceClass device_class); std::string get_serial_number(); void submit_pose(vr::DriverPose_t pose); virtual bool activate() = 0; virtual void* get_component(const char*) = 0; private: ActivationState activation_state = ActivationState::Pending; std::mutex activation_mutex = {}; std::condition_variable activation_condvar = {}; // ITrackedDeviceServerDriver vr::EVRInitError Activate(vr::TrackedDeviceIndex_t object_id) final; void Deactivate() final { this->device_id = vr::k_unTrackedDeviceIndexInvalid; this->prop_container = vr::k_ulInvalidPropertyContainer; } void EnterStandby() final { } void* GetComponent(const char* component_name_and_version) final { return get_component(component_name_and_version); } void DebugRequest(const char*, char* buffer, uint32_t buffer_size) final { if (buffer_size >= 1) buffer[0] = 0; } vr::DriverPose_t GetPose() final { return this->last_pose; } }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/Utils.h ================================================ #pragma once #include #ifdef _WIN32 #pragma warning(disable : 4005) #include #pragma warning(default : 4005) #include #include #include #include #include #include #include #include #define _USE_MATH_DEFINES #include #else #include #include #include #endif #include #include "ALVR-common/packet_types.h" #include "openvr_driver_wrap.h" const float DEG_TO_RAD = (float)(M_PI / 180.); const double NS_PER_S = 1000000000.0; // Get elapsed time in us from Unix Epoch inline uint64_t GetTimestampUs() { auto duration = std::chrono::system_clock::now().time_since_epoch(); return std::chrono::duration_cast(duration).count(); } #ifdef _WIN32 inline std::wstring GetErrorStr(HRESULT hr) { wchar_t* s = NULL; std::wstring ret; FormatMessageW( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, hr, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (LPWSTR)&s, 0, NULL ); ret = s; LocalFree(s); if (ret.size() >= 1 && ret[ret.size() - 1] == L'\n') { ret.erase(ret.size() - 1, 1); } if (ret.size() >= 1 && ret[ret.size() - 1] == L'\r') { ret.erase(ret.size() - 1, 1); } return ret; } #endif inline vr::HmdQuaternion_t HmdQuaternion_Init(double w, double x, double y, double z) { vr::HmdQuaternion_t quat; quat.w = w; quat.x = x; quat.y = y; quat.z = z; return quat; } inline vr::HmdRect2_t fov_to_tangents(FfiFov fov) { auto proj_bounds = vr::HmdRect2_t {}; proj_bounds.vTopLeft.v[0] = tanf(fov.left); proj_bounds.vBottomRight.v[0] = tanf(fov.right); proj_bounds.vTopLeft.v[1] = tanf(fov.down); proj_bounds.vBottomRight.v[1] = tanf(fov.up); return proj_bounds; } inline vr::HmdMatrix34_t pose_to_mat(FfiPose pose) { FfiQuat o = pose.orientation; vr::HmdMatrix34_t mat = {}; mat.m[0][0] = 1.0f - 2.0f * (o.y * o.y + o.z * o.z); mat.m[0][1] = 2.0f * (o.x * o.y - o.w * o.z); mat.m[0][2] = 2.0f * (o.x * o.z + o.w * o.y); mat.m[1][0] = 2.0f * (o.x * o.y + o.w * o.z); mat.m[1][1] = 1.0f - 2.0f * (o.x * o.x + o.z * o.z); mat.m[1][2] = 2.0f * (o.y * o.z - o.w * o.x); mat.m[2][0] = 2.0f * (o.x * o.z - o.w * o.y); mat.m[2][1] = 2.0f * (o.y * o.z + o.w * o.x); mat.m[2][2] = 1.0f - 2.0f * (o.x * o.x + o.y * o.y); mat.m[0][3] = pose.position[0]; mat.m[1][3] = pose.position[1]; mat.m[2][3] = pose.position[2]; return mat; } inline void HmdMatrix_SetIdentity(vr::HmdMatrix34_t* pMatrix) { pMatrix->m[0][0] = 1.f; pMatrix->m[0][1] = 0.f; pMatrix->m[0][2] = 0.f; pMatrix->m[0][3] = 0.f; pMatrix->m[1][0] = 0.f; pMatrix->m[1][1] = 1.f; pMatrix->m[1][2] = 0.f; pMatrix->m[1][3] = 0.f; pMatrix->m[2][0] = 0.f; pMatrix->m[2][1] = 0.f; pMatrix->m[2][2] = 1.f; pMatrix->m[2][3] = 0.f; } inline void HmdMatrix_QuatToMat(double w, double x, double y, double z, vr::HmdMatrix34_t* pMatrix) { pMatrix->m[0][0] = (float)(1.0f - 2.0f * y * y - 2.0f * z * z); pMatrix->m[0][1] = (float)(2.0f * x * y - 2.0f * z * w); pMatrix->m[0][2] = (float)(2.0f * x * z + 2.0f * y * w); pMatrix->m[0][3] = (float)(0.0f); pMatrix->m[1][0] = (float)(2.0f * x * y + 2.0f * z * w); pMatrix->m[1][1] = (float)(1.0f - 2.0f * x * x - 2.0f * z * z); pMatrix->m[1][2] = (float)(2.0f * y * z - 2.0f * x * w); pMatrix->m[1][3] = (float)(0.0f); pMatrix->m[2][0] = (float)(2.0f * x * z - 2.0f * y * w); pMatrix->m[2][1] = (float)(2.0f * y * z + 2.0f * x * w); pMatrix->m[2][2] = (float)(1.0f - 2.0f * x * x - 2.0f * y * y); pMatrix->m[2][3] = (float)(0.0f); } inline vr::HmdQuaternion_t EulerAngleToQuaternion(const double* yaw_pitch_roll) { vr::HmdQuaternion_t q; // Abbreviations for the various angular functions double cy = cos(yaw_pitch_roll[0] * 0.5); double sy = sin(yaw_pitch_roll[0] * 0.5); double cr = cos(yaw_pitch_roll[2] * 0.5); double sr = sin(yaw_pitch_roll[2] * 0.5); double cp = cos(yaw_pitch_roll[1] * 0.5); double sp = sin(yaw_pitch_roll[1] * 0.5); q.w = cy * cr * cp + sy * sr * sp; q.x = cy * sr * cp - sy * cr * sp; q.y = cy * cr * sp + sy * sr * cp; q.z = sy * cr * cp - cy * sr * sp; return q; } inline vr::HmdVector4_t Lerp(vr::HmdVector4_t& v1, vr::HmdVector4_t& v2, double lambda) { vr::HmdVector4_t res; res.v[0] = (float)((1 - lambda) * v1.v[0] + lambda * v2.v[0]); res.v[1] = (float)((1 - lambda) * v1.v[1] + lambda * v2.v[1]); res.v[2] = (float)((1 - lambda) * v1.v[2] + lambda * v2.v[2]); res.v[3] = 1; return res; } inline vr::HmdQuaternionf_t Slerp(vr::HmdQuaternionf_t& q1, vr::HmdQuaternionf_t& q2, double lambda) { if (q1.w != q2.w || q1.x != q2.x || q1.y != q2.y || q1.z != q2.z) { float dotproduct = q1.x * q2.x + q1.y * q2.y + q1.z * q2.z + q1.w * q2.w; float theta, st, sut, sout, coeff1, coeff2; // algorithm adapted from Shoemake's paper theta = (float)acos(dotproduct); if (theta < 0.0) theta = -theta; st = (float)sin(theta); sut = (float)sin(lambda * theta); sout = (float)sin((1 - lambda) * theta); coeff1 = sout / st; coeff2 = sut / st; vr::HmdQuaternionf_t res; res.w = coeff1 * q1.w + coeff2 * q2.w; res.x = coeff1 * q1.x + coeff2 * q2.x; res.y = coeff1 * q1.y + coeff2 * q2.y; res.z = coeff1 * q1.z + coeff2 * q2.z; float norm = res.w * res.w + res.x * res.x + res.y * res.y + res.z * res.z; res.w /= norm; res.x /= norm; res.y /= norm; res.z /= norm; return res; } else { return q1; } } // Sourced from https://mariogc.com/post/angular-velocity-quaternions/ inline vr::HmdVector3d_t AngularVelocityBetweenQuats( const vr::HmdQuaternion_t& q1, const vr::HmdQuaternion_t& q2, double dt ) { double r = (2.0f / dt); return { (q1.w * q2.x - q1.x * q2.w - q1.y * q2.z + q1.z * q2.y) * r, (q1.w * q2.y + q1.x * q2.z - q1.y * q2.w - q1.z * q2.x) * r, (q1.w * q2.z - q1.x * q2.y + q1.y * q2.x - q1.z * q2.w) * r }; } #ifdef _WIN32 typedef void(WINAPI* RtlGetVersion_FUNC)(OSVERSIONINFOEXW*); inline std::wstring GetWindowsOSVersion() { HMODULE hModule; OSVERSIONINFOEXW ver; hModule = LoadLibraryW(L"ntdll.dll"); if (hModule == NULL) { return L"Unknown"; } RtlGetVersion_FUNC RtlGetVersion = (RtlGetVersion_FUNC)GetProcAddress(hModule, "RtlGetVersion"); if (RtlGetVersion == NULL) { FreeLibrary(hModule); return L"Unknown"; } memset(&ver, 0, sizeof(ver)); ver.dwOSVersionInfoSize = sizeof(ver); RtlGetVersion(&ver); FreeLibrary(hModule); wchar_t buf[1000]; _snwprintf_s( buf, sizeof(buf) / sizeof(buf[0]), L"MajorVersion=%d MinorVersion=%d Build=%d", ver.dwMajorVersion, ver.dwMinorVersion, ver.dwBuildNumber ); return buf; } #endif ================================================ FILE: alvr/server_openvr/cpp/alvr_server/ViveTrackerProxy.cpp ================================================ #include "ViveTrackerProxy.h" #include "HMD.h" #include "Settings.h" #include ViveTrackerProxy::ViveTrackerProxy(Hmd& owner) : m_unObjectId(vr::k_unTrackedDeviceIndexInvalid) , m_HMDOwner(&owner) { } vr::DriverPose_t ViveTrackerProxy::GetPose() { assert(m_HMDOwner != nullptr); return m_HMDOwner->last_pose; } vr::EVRInitError ViveTrackerProxy::Activate(vr::TrackedDeviceIndex_t unObjectId) { auto vr_properties = vr::VRProperties(); m_unObjectId = unObjectId; assert(m_unObjectId != vr::k_unTrackedDeviceIndexInvalid); const auto propertyContainer = vr_properties->TrackedDeviceToPropertyContainer(m_unObjectId); // Normally a vive tracker emulator would (logically) always set the tracking system to // "lighthouse" but in order to do space calibration with existing tools such as OpenVR Space // calibrator and be able to calibrate to/from ALVR HMD (and the proxy tracker) space to/from a // native HMD/tracked device which is already using "lighthouse" as the tracking system the // proxy tracker needs to be in a different tracking system to treat them differently and // prevent those tools doing the same space transform to the proxy tracker. vr_properties->SetStringProperty( propertyContainer, vr::Prop_TrackingSystemName_String, "HeadTrackerCustom" ); //"lighthouse"); vr_properties->SetStringProperty( propertyContainer, vr::Prop_ModelNumber_String, "Vive Tracker Pro MV" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_SerialNumber_String, GetSerialNumber() ); // Changed vr_properties->SetStringProperty( propertyContainer, vr::Prop_RenderModelName_String, "{htc}vr_tracker_vive_1_0" ); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_WillDriftInYaw_Bool, false); vr_properties->SetStringProperty(propertyContainer, vr::Prop_ManufacturerName_String, "HTC"); vr_properties->SetStringProperty( propertyContainer, vr::Prop_TrackingFirmwareVersion_String, "1541800000 RUNNER-WATCHMAN$runner-watchman@runner-watchman 2018-01-01 FPGA 512(2.56/0/0) " "BL 0 VRC 1541800000 Radio 1518800000" ); // Changed vr_properties->SetStringProperty( propertyContainer, vr::Prop_HardwareRevision_String, "product 128 rev 2.5.6 lot 2000/0/0 0" ); // Changed vr_properties->SetStringProperty( propertyContainer, vr::Prop_ConnectedWirelessDongle_String, "D0000BE000" ); // Changed vr_properties->SetBoolProperty(propertyContainer, vr::Prop_DeviceIsWireless_Bool, true); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_DeviceIsCharging_Bool, false); vr_properties->SetFloatProperty( propertyContainer, vr::Prop_DeviceBatteryPercentage_Float, 1.f ); // Always charged vr::HmdMatrix34_t l_transform = { { { -1.f, 0.f, 0.f, 0.f }, { 0.f, 0.f, -1.f, 0.f }, { 0.f, -1.f, 0.f, 0.f } } }; vr_properties->SetProperty( propertyContainer, vr::Prop_StatusDisplayTransform_Matrix34, &l_transform, sizeof(vr::HmdMatrix34_t), vr::k_unHmdMatrix34PropertyTag ); vr_properties->SetBoolProperty( propertyContainer, vr::Prop_Firmware_UpdateAvailable_Bool, false ); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_Firmware_ManualUpdate_Bool, false); vr_properties->SetStringProperty( propertyContainer, vr::Prop_Firmware_ManualUpdateURL_String, "https://developer.valvesoftware.com/wiki/SteamVR/HowTo_Update_Firmware" ); vr_properties->SetUint64Property( propertyContainer, vr::Prop_HardwareRevision_Uint64, 2214720000 ); // Changed vr_properties->SetUint64Property( propertyContainer, vr::Prop_FirmwareVersion_Uint64, 1541800000 ); // Changed vr_properties->SetUint64Property( propertyContainer, vr::Prop_FPGAVersion_Uint64, 512 ); // Changed vr_properties->SetUint64Property( propertyContainer, vr::Prop_VRCVersion_Uint64, 1514800000 ); // Changed vr_properties->SetUint64Property( propertyContainer, vr::Prop_RadioVersion_Uint64, 1518800000 ); // Changed vr_properties->SetUint64Property( propertyContainer, vr::Prop_DongleVersion_Uint64, 8933539758 ); // Changed, based on vr::Prop_ConnectedWirelessDongle_String above vr_properties->SetBoolProperty( propertyContainer, vr::Prop_DeviceProvidesBatteryStatus_Bool, true ); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_DeviceCanPowerOff_Bool, true); vr_properties->SetStringProperty( propertyContainer, vr::Prop_Firmware_ProgrammingTarget_String, GetSerialNumber() ); vr_properties->SetInt32Property( propertyContainer, vr::Prop_DeviceClass_Int32, vr::TrackedDeviceClass_GenericTracker ); vr_properties->SetBoolProperty( propertyContainer, vr::Prop_Firmware_ForceUpdateRequired_Bool, false ); vr_properties->SetStringProperty(propertyContainer, vr::Prop_ResourceRoot_String, "htc"); vr_properties->SetStringProperty( propertyContainer, vr::Prop_RegisteredDeviceType_String, "ALVR/tracker/hmd_proxy" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_InputProfilePath_String, "{htc}/input/vive_tracker_profile.json" ); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_Identifiable_Bool, false); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_Firmware_RemindUpdate_Bool, false); vr_properties->SetInt32Property( propertyContainer, vr::Prop_ControllerRoleHint_Int32, vr::TrackedControllerRole_Invalid ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_ControllerType_String, "vive_tracker_waist" ); vr_properties->SetInt32Property( propertyContainer, vr::Prop_ControllerHandSelectionPriority_Int32, -1 ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceOff_String, "{htc}/icons/tracker_status_off.png" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceSearching_String, "{htc}/icons/tracker_status_searching.gif" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceSearchingAlert_String, "{htc}/icons/tracker_status_searching_alert.gif" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceReady_String, "{htc}/icons/tracker_status_ready.png" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceReadyAlert_String, "{htc}/icons/tracker_status_ready_alert.png" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceNotReady_String, "{htc}/icons/tracker_status_error.png" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceStandby_String, "{htc}/icons/tracker_status_standby.png" ); vr_properties->SetStringProperty( propertyContainer, vr::Prop_NamedIconPathDeviceAlertLow_String, "{htc}/icons/tracker_status_ready_low.png" ); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_HasDisplayComponent_Bool, false); vr_properties->SetBoolProperty(propertyContainer, vr::Prop_HasCameraComponent_Bool, false); vr_properties->SetBoolProperty( propertyContainer, vr::Prop_HasDriverDirectModeComponent_Bool, false ); vr_properties->SetBoolProperty( propertyContainer, vr::Prop_HasVirtualDisplayComponent_Bool, false ); return vr::VRInitError_None; } void ViveTrackerProxy::update() { auto newPose = GetPose(); vr::VRServerDriverHost()->TrackedDevicePoseUpdated( m_unObjectId, newPose, sizeof(vr::DriverPose_t) ); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/ViveTrackerProxy.h ================================================ #pragma once #include "openvr_driver_wrap.h" class Hmd; class ViveTrackerProxy final : public vr::ITrackedDeviceServerDriver { vr::TrackedDeviceIndex_t m_unObjectId; Hmd* m_HMDOwner; public: ViveTrackerProxy(Hmd& owner); ViveTrackerProxy(const ViveTrackerProxy&) = delete; ViveTrackerProxy& operator=(const ViveTrackerProxy&) = delete; constexpr inline const char* GetSerialNumber() const { return "ALVR HMD Tracker Proxy"; } virtual vr::EVRInitError Activate(vr::TrackedDeviceIndex_t unObjectId) override; virtual inline void Deactivate() override { m_unObjectId = vr::k_unTrackedDeviceIndexInvalid; } virtual inline void EnterStandby() override { } virtual inline void* GetComponent(const char* /*pchComponentNameAndVersion*/) override { // override this to add a component to a driver return nullptr; } virtual inline void DebugRequest( const char* /*pchRequest*/, char* pchResponseBuffer, uint32_t unResponseBufferSize ) override { if (unResponseBufferSize >= 1) pchResponseBuffer[0] = 0; } virtual vr::DriverPose_t GetPose() override; void update(); }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/alvr_server.cpp ================================================ #ifdef _WIN32 #include "platform/win32/CEncoder.h" #include #elif __APPLE__ #include "platform/macos/CEncoder.h" #else #include "platform/linux/CEncoder.h" #endif #include "Controller.h" #include "FakeViveTracker.h" #include "HMD.h" #include "Logger.h" #include "Paths.h" #include "PoseHistory.h" #include "Settings.h" #include "TrackedDevice.h" #include "bindings.h" #include "driverlog.h" #include "openvr_driver_wrap.h" #include #include #include #include #include #ifdef __linux__ #include "include/openvr_math.h" std::unique_ptr GetInvZeroPose(); std::unique_ptr GetRawZeroPose() { auto invZeroPose = GetInvZeroPose(); if (invZeroPose == nullptr) { return nullptr; } return std::make_unique(vrmath::matInv33(*invZeroPose)); } bool IsOpenvrClientReady(); #endif void _SetChaperoneArea(float areaWidth, float areaHeight); vr::EVREventType VendorEvent_ALVRDriverResync = (vr::EVREventType)(vr::VREvent_VendorSpecific_Reserved_Start + ((vr::EVREventType)0xC0)); static void load_debug_privilege(void) { #ifdef _WIN32 const DWORD flags = TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY; TOKEN_PRIVILEGES tp; HANDLE token; LUID val; if (!OpenProcessToken(GetCurrentProcess(), flags, &token)) { return; } if (!!LookupPrivilegeValue(NULL, SE_DEBUG_NAME, &val)) { tp.PrivilegeCount = 1; tp.Privileges[0].Luid = val; tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED; AdjustTokenPrivileges(token, false, &tp, sizeof(tp), NULL, NULL); } if (!!LookupPrivilegeValue(NULL, SE_INC_BASE_PRIORITY_NAME, &val)) { tp.PrivilegeCount = 1; tp.Privileges[0].Luid = val; tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED; if (!AdjustTokenPrivileges(token, false, &tp, sizeof(tp), NULL, NULL)) { Warn("[GPU PRIO FIX] Could not set privilege to increase GPU priority\n"); } } Debug("[GPU PRIO FIX] Succeeded to set some sort of priority.\n"); CloseHandle(token); #endif } class DriverProvider : public vr::IServerTrackedDeviceProvider { public: bool early_hmd_initialization = false; std::unique_ptr hmd; std::unique_ptr left_controller, right_controller; std::unique_ptr left_hand_tracker, right_hand_tracker; std::vector> generic_trackers; bool devices_initialized = false; bool shutdown_called = false; std::map tracked_devices; virtual vr::EVRInitError Init(vr::IVRDriverContext* pContext) override { Debug("DriverProvider::Init"); VR_INIT_SERVER_DRIVER_CONTEXT(pContext); InitDriverLog(vr::VRDriverLog()); if (this->early_hmd_initialization) { auto hmd = new Hmd(); // Note: we disable awaiting for Acivate() call. That will only be called after // IServerTrackedDeviceProvider::Init() (this function) returns. hmd->register_device(false); this->hmd = std::unique_ptr(hmd); this->tracked_devices.insert({ HEAD_ID, this->hmd.get() }); } return vr::VRInitError_None; } virtual void Cleanup() override { Debug("DriverProvider::Cleanup"); this->left_hand_tracker.reset(); this->right_hand_tracker.reset(); this->left_controller.reset(); this->right_controller.reset(); this->hmd.reset(); // this->generic_trackers.clear(); CleanupDriverLog(); VR_CLEANUP_SERVER_DRIVER_CONTEXT(); } virtual const char* const* GetInterfaceVersions() override { return vr::k_InterfaceVersions; } virtual const char* GetTrackedDeviceDriverVersion() { return vr::ITrackedDeviceServerDriver_Version; } virtual void RunFrame() override { vr::VREvent_t event; while (vr::VRServerDriverHost()->PollNextEvent(&event, sizeof(vr::VREvent_t))) { if (event.eventType == vr::VREvent_Input_HapticVibration) { Debug("DriverProvider: Received HapticVibration event"); vr::VREvent_HapticVibration_t haptics = event.data.hapticVibration; uint64_t id = 0; if (this->left_controller && haptics.containerHandle == this->left_controller->prop_container) { id = HAND_LEFT_ID; } else if (this->right_controller && haptics.containerHandle == this->right_controller->prop_container) { id = HAND_RIGHT_ID; } HapticsSend(id, haptics.fDurationSeconds, haptics.fFrequency, haptics.fAmplitude); } #ifdef __linux__ else if (event.eventType == vr::VREvent_ChaperoneUniverseHasChanged || event.eventType == vr::VREvent_ChaperoneRoomSetupFinished || event.eventType == vr::VREvent_ChaperoneFlushCache || event.eventType == vr::VREvent_ChaperoneSettingsHaveChanged || event.eventType == vr::VREvent_SeatedZeroPoseReset || event.eventType == vr::VREvent_StandingZeroPoseReset || event.eventType == vr::VREvent_SceneApplicationChanged || event.eventType == VendorEvent_ALVRDriverResync) { if (hmd && hmd->m_poseHistory) { auto rawZeroPose = GetRawZeroPose(); if (rawZeroPose != nullptr) { hmd->m_poseHistory->SetTransform(*rawZeroPose); } } } #endif } if (vr::VRServerDriverHost()->IsExiting() && !shutdown_called) { Debug("DriverProvider: Received shutdown event"); shutdown_called = true; ShutdownRuntime(); } } virtual bool ShouldBlockStandbyMode() override { return false; } virtual void EnterStandby() override { Debug("DriverProvider::EnterStandby"); } virtual void LeaveStandby() override { Debug("DriverProvider::LeaveStandby"); } } g_driver_provider; // bindigs for Rust const unsigned char* FRAME_RENDER_VS_CSO_PTR; unsigned int FRAME_RENDER_VS_CSO_LEN; const unsigned char* FRAME_RENDER_PS_CSO_PTR; unsigned int FRAME_RENDER_PS_CSO_LEN; const unsigned char* QUAD_SHADER_CSO_PTR; unsigned int QUAD_SHADER_CSO_LEN; const unsigned char* COMPRESS_AXIS_ALIGNED_CSO_PTR; unsigned int COMPRESS_AXIS_ALIGNED_CSO_LEN; const unsigned char* COLOR_CORRECTION_CSO_PTR; unsigned int COLOR_CORRECTION_CSO_LEN; const unsigned char* RGBTOYUV420_CSO_PTR; unsigned int RGBTOYUV420_CSO_LEN; const unsigned char* QUAD_SHADER_COMP_SPV_PTR; unsigned int QUAD_SHADER_COMP_SPV_LEN; const unsigned char* COLOR_SHADER_COMP_SPV_PTR; unsigned int COLOR_SHADER_COMP_SPV_LEN; const unsigned char* FFR_SHADER_COMP_SPV_PTR; unsigned int FFR_SHADER_COMP_SPV_LEN; const unsigned char* RGBTOYUV420_SHADER_COMP_SPV_PTR; unsigned int RGBTOYUV420_SHADER_COMP_SPV_LEN; const char* g_sessionPath; const char* g_driverRootDir; void (*LogError)(const char* stringPtr); void (*LogWarn)(const char* stringPtr); void (*LogInfo)(const char* stringPtr); void (*LogDebug)(const char* stringPtr); void (*LogEncoder)(const char* stringPtr); void (*LogPeriodically)(const char* tag, const char* stringPtr); void (*DriverReadyIdle)(bool setDefaultChaprone); void (*SetVideoConfigNals)(const unsigned char* configBuffer, int len, int codec); void (*VideoSend)(unsigned long long targetTimestampNs, unsigned char* buf, int len, bool isIdr); void (*HapticsSend)(unsigned long long path, float duration_s, float frequency, float amplitude); void (*ShutdownRuntime)(); unsigned long long (*PathStringToHash)(const char* path); void (*ReportPresent)(unsigned long long timestamp_ns, unsigned long long offset_ns); void (*ReportComposed)(unsigned long long timestamp_ns, unsigned long long offset_ns); FfiDynamicEncoderParams (*GetDynamicEncoderParams)(); unsigned long long (*GetSerialNumber)(unsigned long long deviceID, char* outString); void (*SetOpenvrProps)(void* instancePtr, unsigned long long deviceID); void (*RegisterButtons)(void* instancePtr, unsigned long long deviceID); void (*WaitForVSync)(); void CppInit(bool earlyHmdInitialization) { g_driver_provider.early_hmd_initialization = earlyHmdInitialization; HookCrashHandler(); // Initialize path constants init_paths(); Settings::Instance().Load(); load_debug_privilege(); } void* CppOpenvrEntryPoint(const char* interface_name, int* return_code) { if (std::string(interface_name) == vr::IServerTrackedDeviceProvider_Version) { *return_code = vr::VRInitError_None; return &g_driver_provider; } else { *return_code = vr::VRInitError_Init_InterfaceNotFound; return nullptr; } } bool InitializeStreaming() { Settings::Instance().Load(); if (!g_driver_provider.devices_initialized) { if (!g_driver_provider.early_hmd_initialization) { auto hmd = new Hmd(); if (!hmd->register_device(false)) { Error("Failed to register HMD"); return false; } g_driver_provider.hmd = std::unique_ptr(hmd); g_driver_provider.tracked_devices.insert({ HEAD_ID, g_driver_provider.hmd.get() }); } // Note: for controllers, hands and trackers don't bail out if registration fails if (Settings::Instance().m_enableControllers) { auto controllerSkeletonLevel = Settings::Instance().m_useSeparateHandTrackers ? vr::VRSkeletalTracking_Estimated : vr::VRSkeletalTracking_Partial; auto left_controller = new Controller(HAND_LEFT_ID, controllerSkeletonLevel); if (left_controller->register_device(true)) { g_driver_provider.left_controller = std::unique_ptr(left_controller); g_driver_provider.tracked_devices.insert( { HAND_LEFT_ID, g_driver_provider.left_controller.get() } ); } auto right_controller = new Controller(HAND_RIGHT_ID, controllerSkeletonLevel); if (right_controller->register_device(true)) { g_driver_provider.right_controller = std::unique_ptr(right_controller); g_driver_provider.tracked_devices.insert( { HAND_RIGHT_ID, g_driver_provider.right_controller.get() } ); } if (Settings::Instance().m_useSeparateHandTrackers) { auto left_hand_tracker = new Controller(HAND_TRACKER_LEFT_ID, vr::VRSkeletalTracking_Full); if (left_hand_tracker->register_device(true)) { g_driver_provider.left_hand_tracker = std::unique_ptr(left_hand_tracker); g_driver_provider.tracked_devices.insert( { HAND_TRACKER_LEFT_ID, g_driver_provider.left_hand_tracker.get() } ); } auto right_hand_tracker = new Controller(HAND_TRACKER_RIGHT_ID, vr::VRSkeletalTracking_Full); if (right_hand_tracker->register_device(true)) { g_driver_provider.right_hand_tracker = std::unique_ptr(right_hand_tracker); g_driver_provider.tracked_devices.insert( { HAND_TRACKER_RIGHT_ID, g_driver_provider.right_hand_tracker.get() } ); } } } if (Settings::Instance().m_enableBodyTrackingFakeVive) { auto chestTracker = std::make_unique(BODY_CHEST_ID); if (chestTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_CHEST_ID, chestTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(chestTracker)); } auto waistTracker = std::make_unique(BODY_HIPS_ID); if (waistTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_HIPS_ID, waistTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(waistTracker)); } auto leftElbowTracker = std::make_unique(BODY_LEFT_ELBOW_ID); if (leftElbowTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_LEFT_ELBOW_ID, leftElbowTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(leftElbowTracker)); } auto rightElbowTracker = std::make_unique(BODY_RIGHT_ELBOW_ID); if (rightElbowTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_RIGHT_ELBOW_ID, rightElbowTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(rightElbowTracker)); } if (Settings::Instance().m_bodyTrackingHasLegs) { auto leftKneeTracker = std::make_unique(BODY_LEFT_KNEE_ID); if (leftKneeTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_LEFT_KNEE_ID, leftKneeTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(leftKneeTracker)); } auto leftFootTracker = std::make_unique(BODY_LEFT_FOOT_ID); if (leftFootTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_LEFT_FOOT_ID, leftFootTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(leftFootTracker)); } auto rightKneeTracker = std::make_unique(BODY_RIGHT_KNEE_ID); if (rightKneeTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_RIGHT_KNEE_ID, rightKneeTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(rightKneeTracker)); } auto rightFootTracker = std::make_unique(BODY_RIGHT_FOOT_ID); if (rightFootTracker->register_device(true)) { g_driver_provider.tracked_devices.insert({ BODY_RIGHT_FOOT_ID, rightFootTracker.get() }); g_driver_provider.generic_trackers.push_back(std::move(rightFootTracker)); } } } g_driver_provider.devices_initialized = true; } if (g_driver_provider.hmd) { g_driver_provider.hmd->StartStreaming(); } return true; } void DeinitializeStreaming() { if (g_driver_provider.hmd) { g_driver_provider.hmd->StopStreaming(); } } void SendVSync() { vr::VRServerDriverHost()->VsyncEvent(0.0); } void RequestIDR() { if (g_driver_provider.hmd && g_driver_provider.hmd->m_encoder) { g_driver_provider.hmd->m_encoder->InsertIDR(); } } void SetTracking( unsigned long long targetTimestampNs, float controllerPoseTimeOffsetS, FfiDeviceMotion headMotion, FfiHandData leftHandData, FfiHandData rightHandData, const FfiDeviceMotion* bodyTrackerMotions, int bodyTrackerMotionCount ) { if (g_driver_provider.hmd) { g_driver_provider.hmd->OnPoseUpdated(targetTimestampNs, headMotion); } if (g_driver_provider.left_hand_tracker) { g_driver_provider.left_hand_tracker->OnPoseUpdate( targetTimestampNs, controllerPoseTimeOffsetS, leftHandData ); } if (g_driver_provider.left_controller) { g_driver_provider.left_controller->OnPoseUpdate( targetTimestampNs, controllerPoseTimeOffsetS, leftHandData ); } if (g_driver_provider.right_hand_tracker) { g_driver_provider.right_hand_tracker->OnPoseUpdate( targetTimestampNs, controllerPoseTimeOffsetS, rightHandData ); } if (g_driver_provider.right_controller) { g_driver_provider.right_controller->OnPoseUpdate( targetTimestampNs, controllerPoseTimeOffsetS, rightHandData ); } if (Settings::Instance().m_enableBodyTrackingFakeVive) { std::map motionsMap; for (int i = 0; i < bodyTrackerMotionCount; i++) { auto m = bodyTrackerMotions[i]; motionsMap.insert({ m.deviceID, m }); } for (auto id : BODY_IDS) { auto it = g_driver_provider.tracked_devices.find(id); if (it != g_driver_provider.tracked_devices.end()) { auto* maybeTracker = (FakeViveTracker*)it->second; auto res = motionsMap.find(id); auto* maybeMotion = res != motionsMap.end() ? &res->second : nullptr; maybeTracker->OnPoseUpdated(targetTimestampNs, maybeMotion); } } } } void RequestDriverResync() { if (g_driver_provider.hmd) { vr::VRServerDriverHost()->VendorSpecificEvent( g_driver_provider.hmd->object_id, VendorEvent_ALVRDriverResync, {}, 0 ); } } void ShutdownSteamvr() { if (g_driver_provider.hmd) { vr::VRServerDriverHost()->VendorSpecificEvent( g_driver_provider.hmd->object_id, vr::VREvent_DriverRequestedQuit, {}, 0 ); } } void SetOpenvrProperty(void* instancePtr, FfiOpenvrProperty prop) { ((TrackedDevice*)instancePtr)->set_prop(prop); } void SetOpenvrPropByDeviceID(unsigned long long deviceID, FfiOpenvrProperty prop) { auto device_it = g_driver_provider.tracked_devices.find(deviceID); if (device_it != g_driver_provider.tracked_devices.end()) { device_it->second->set_prop(prop); } } void RegisterButton(void* instancePtr, unsigned long long buttonID) { // Todo: move RegisterButton to generic TrackedDevice interface ((Controller*)instancePtr)->RegisterButton(buttonID); } void SetLocalViewParams(const FfiViewParams params[2]) { if (g_driver_provider.hmd) { g_driver_provider.hmd->SetViewParams(params); } } void SetBattery(unsigned long long deviceID, float gauge_value, bool is_plugged) { auto device_it = g_driver_provider.tracked_devices.find(deviceID); if (device_it != g_driver_provider.tracked_devices.end()) { vr::VRProperties()->SetFloatProperty( device_it->second->prop_container, vr::Prop_DeviceBatteryPercentage_Float, gauge_value ); vr::VRProperties()->SetBoolProperty( device_it->second->prop_container, vr::Prop_DeviceIsCharging_Bool, is_plugged ); } } void SetButton(unsigned long long buttonID, FfiButtonValue value) { if (LEFT_CONTROLLER_BUTTON_MAPPING.find(buttonID) != LEFT_CONTROLLER_BUTTON_MAPPING.end()) { if (g_driver_provider.left_controller) { g_driver_provider.left_controller->SetButton(buttonID, value); } if (g_driver_provider.left_hand_tracker) { g_driver_provider.left_hand_tracker->SetButton(buttonID, value); } } else if (RIGHT_CONTROLLER_BUTTON_MAPPING.find(buttonID) != RIGHT_CONTROLLER_BUTTON_MAPPING.end()) { if (g_driver_provider.right_controller) { g_driver_provider.right_controller->SetButton(buttonID, value); } if (g_driver_provider.right_hand_tracker) { g_driver_provider.right_hand_tracker->SetButton(buttonID, value); } } } void SetProximityState(bool headset_is_worn) { if (g_driver_provider.hmd) { g_driver_provider.hmd->SetProximityState(headset_is_worn); } } void SetChaperoneArea(float areaWidth, float areaHeight) { _SetChaperoneArea(areaWidth, areaHeight); } void CaptureFrame() { #ifndef __APPLE__ if (g_driver_provider.hmd && g_driver_provider.hmd->m_encoder) { g_driver_provider.hmd->m_encoder->CaptureFrame(); } #endif } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/bindings.h ================================================ #pragma once struct FfiFov { float left; float right; float up; float down; }; struct FfiQuat { float x; float y; float z; float w; }; struct FfiPose { FfiQuat orientation; float position[3]; }; struct FfiDeviceMotion { unsigned long long deviceID; FfiPose pose; float linearVelocity[3]; float angularVelocity[3]; }; struct FfiViewParams { FfiPose pose; FfiFov fov; }; struct FfiHandSkeleton { float jointPositions[31][3]; FfiQuat jointRotations[31]; }; struct FfiHandData { const FfiDeviceMotion* controllerMotion; const FfiHandSkeleton* handSkeleton; bool isHandTracker; bool predictHandSkeleton; }; enum FfiOpenvrPropertyType { Bool, Float, Int32, Uint64, Vector3, Double, String, }; union FfiOpenvrPropertyValue { unsigned int bool_; float float_; int int32; unsigned long long uint64; float vector3[3]; double double_; char string[256]; }; struct FfiOpenvrProperty { unsigned int key; FfiOpenvrPropertyType type; FfiOpenvrPropertyValue value; }; enum FfiButtonType { BUTTON_TYPE_BINARY, BUTTON_TYPE_SCALAR, }; struct FfiButtonValue { FfiButtonType type; union { unsigned int binary; float scalar; }; }; struct FfiDynamicEncoderParams { unsigned int updated; unsigned long long bitrate_bps; float framerate; }; extern "C" const unsigned char* FRAME_RENDER_VS_CSO_PTR; extern "C" unsigned int FRAME_RENDER_VS_CSO_LEN; extern "C" const unsigned char* FRAME_RENDER_PS_CSO_PTR; extern "C" unsigned int FRAME_RENDER_PS_CSO_LEN; extern "C" const unsigned char* QUAD_SHADER_CSO_PTR; extern "C" unsigned int QUAD_SHADER_CSO_LEN; extern "C" const unsigned char* COMPRESS_AXIS_ALIGNED_CSO_PTR; extern "C" unsigned int COMPRESS_AXIS_ALIGNED_CSO_LEN; extern "C" const unsigned char* COLOR_CORRECTION_CSO_PTR; extern "C" unsigned int COLOR_CORRECTION_CSO_LEN; extern "C" const unsigned char* RGBTOYUV420_CSO_PTR; extern "C" unsigned int RGBTOYUV420_CSO_LEN; extern "C" const unsigned char* QUAD_SHADER_COMP_SPV_PTR; extern "C" unsigned int QUAD_SHADER_COMP_SPV_LEN; extern "C" const unsigned char* COLOR_SHADER_COMP_SPV_PTR; extern "C" unsigned int COLOR_SHADER_COMP_SPV_LEN; extern "C" const unsigned char* FFR_SHADER_COMP_SPV_PTR; extern "C" unsigned int FFR_SHADER_COMP_SPV_LEN; extern "C" const unsigned char* RGBTOYUV420_SHADER_COMP_SPV_PTR; extern "C" unsigned int RGBTOYUV420_SHADER_COMP_SPV_LEN; extern "C" const char* g_sessionPath; extern "C" const char* g_driverRootDir; extern "C" void (*LogError)(const char* stringPtr); extern "C" void (*LogWarn)(const char* stringPtr); extern "C" void (*LogInfo)(const char* stringPtr); extern "C" void (*LogDebug)(const char* stringPtr); extern "C" void (*LogEncoder)(const char* stringPtr); extern "C" void (*LogPeriodically)(const char* tag, const char* stringPtr); extern "C" void (*DriverReadyIdle)(bool setDefaultChaprone); extern "C" void (*SetVideoConfigNals)(const unsigned char* configBuffer, int len, int codec); extern "C" void (*VideoSend)( unsigned long long targetTimestampNs, unsigned char* buf, int len, bool isIdr ); extern "C" void (*HapticsSend)( unsigned long long path, float duration_s, float frequency, float amplitude ); extern "C" void (*ShutdownRuntime)(); extern "C" unsigned long long (*PathStringToHash)(const char* path); extern "C" void (*ReportPresent)(unsigned long long timestamp_ns, unsigned long long offset_ns); extern "C" void (*ReportComposed)(unsigned long long timestamp_ns, unsigned long long offset_ns); extern "C" FfiDynamicEncoderParams (*GetDynamicEncoderParams)(); extern "C" unsigned long long (*GetSerialNumber)(unsigned long long deviceID, char* outString); extern "C" void (*SetOpenvrProps)(void* instancePtr, unsigned long long deviceID); extern "C" void (*RegisterButtons)(void* instancePtr, unsigned long long deviceID); extern "C" void (*WaitForVSync)(); extern "C" void CppInit(bool earlyHmdInitialization); extern "C" void* CppOpenvrEntryPoint(const char* pInterfaceName, int* pReturnCode); extern "C" bool InitializeStreaming(); extern "C" void DeinitializeStreaming(); extern "C" void SendVSync(); extern "C" void RequestIDR(); extern "C" void SetTracking( unsigned long long targetTimestampNs, float controllerPoseTimeOffsetS, FfiDeviceMotion headMotion, FfiHandData leftHandData, FfiHandData rightHandData, const FfiDeviceMotion* bodyTrackerMotions, int bodyTrackerMotionCount ); extern "C" void RequestDriverResync(); extern "C" void ShutdownSteamvr(); extern "C" void SetOpenvrProperty(void* instancePtr, FfiOpenvrProperty prop); extern "C" void SetOpenvrPropByDeviceID(unsigned long long deviceID, FfiOpenvrProperty prop); extern "C" void RegisterButton(void* instancePtr, unsigned long long buttonID); extern "C" void SetLocalViewParams(const FfiViewParams params[2]); extern "C" void SetBattery(unsigned long long deviceID, float gauge_value, bool is_plugged); extern "C" void SetButton(unsigned long long buttonID, FfiButtonValue value); extern "C" void SetProximityState(bool headset_is_worn); extern "C" void InitOpenvrClient(); extern "C" void ShutdownOpenvrClient(); extern "C" void SetChaperoneArea(float areaWidth, float areaHeight); extern "C" void CaptureFrame(); // NalParsing.cpp void ParseFrameNals( int codec, unsigned char* buf, int len, unsigned long long targetTimestampNs, bool isIdr ); // CrashHandler.cpp void HookCrashHandler(); ================================================ FILE: alvr/server_openvr/cpp/alvr_server/driverlog.cpp ================================================ //========= Copyright Valve Corporation ============// #include "driverlog.h" #include #include static vr::IVRDriverLog* s_pLogFile = NULL; #if !defined(WIN32) #define vsnprintf_s vsnprintf #endif bool InitDriverLog(vr::IVRDriverLog* pDriverLog) { if (s_pLogFile) return false; s_pLogFile = pDriverLog; return s_pLogFile != NULL; } void CleanupDriverLog() { s_pLogFile = NULL; } void DriverLogVarArgs(const char* pMsgFormat, va_list args) { char buf[1024]; vsnprintf_s(buf, sizeof(buf), pMsgFormat, args); if (s_pLogFile) s_pLogFile->Log(buf); } void DriverLog(const char* pMsgFormat, ...) { va_list args; va_start(args, pMsgFormat); DriverLogVarArgs(pMsgFormat, args); va_end(args); } void DebugDriverLog(const char* pMsgFormat, ...) { #ifdef _DEBUG va_list args; va_start(args, pMsgFormat); DriverLogVarArgs(pMsgFormat, args); va_end(args); #else (void)pMsgFormat; #endif } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/driverlog.h ================================================ //========= Copyright Valve Corporation ============// #ifndef DRIVERLOG_H #define DRIVERLOG_H #pragma once #include "openvr_driver_wrap.h" #include extern void DriverLog(const char* pchFormat, ...); extern void DriverLogVarArgs(const char* pMsgFormat, va_list args); // -------------------------------------------------------------------------- // Purpose: Write to the log file only in debug builds // -------------------------------------------------------------------------- extern void DebugDriverLog(const char* pchFormat, ...); extern bool InitDriverLog(vr::IVRDriverLog* pDriverLog); extern void CleanupDriverLog(); #endif // DRIVERLOG_H ================================================ FILE: alvr/server_openvr/cpp/alvr_server/include/openvr_math.h ================================================ #pragma once #include #include inline vr::HmdQuaternion_t operator+(const vr::HmdQuaternion_t& lhs, const vr::HmdQuaternion_t& rhs) { return { lhs.w + rhs.w, lhs.x + rhs.x, lhs.y + rhs.y, lhs.z + rhs.z }; } inline vr::HmdQuaternion_t operator-(const vr::HmdQuaternion_t& lhs, const vr::HmdQuaternion_t& rhs) { return{ lhs.w - rhs.w, lhs.x - rhs.x, lhs.y - rhs.y, lhs.z - rhs.z }; } inline vr::HmdQuaternion_t operator*(const vr::HmdQuaternion_t& lhs, const vr::HmdQuaternion_t& rhs) { return { (lhs.w * rhs.w) - (lhs.x * rhs.x) - (lhs.y * rhs.y) - (lhs.z * rhs.z), (lhs.w * rhs.x) + (lhs.x * rhs.w) + (lhs.y * rhs.z) - (lhs.z * rhs.y), (lhs.w * rhs.y) + (lhs.y * rhs.w) + (lhs.z * rhs.x) - (lhs.x * rhs.z), (lhs.w * rhs.z) + (lhs.z * rhs.w) + (lhs.x * rhs.y) - (lhs.y * rhs.x) }; } inline vr::HmdVector3d_t operator+(const vr::HmdVector3d_t& lhs, const vr::HmdVector3d_t& rhs) { return { lhs.v[0] + rhs.v[0], lhs.v[1] + rhs.v[1], lhs.v[2] + rhs.v[2] }; } inline vr::HmdVector3d_t operator+(const vr::HmdVector3d_t& lhs, const double(&rhs)[3]) { return{ lhs.v[0] + rhs[0], lhs.v[1] + rhs[1], lhs.v[2] + rhs[2] }; } inline vr::HmdVector3d_t operator-(const vr::HmdVector3d_t& lhs, const vr::HmdVector3d_t& rhs) { return{ lhs.v[0] - rhs.v[0], lhs.v[1] - rhs.v[1], lhs.v[2] - rhs.v[2] }; } inline vr::HmdVector3d_t operator-(const vr::HmdVector3d_t& lhs, const double (&rhs)[3]) { return{ lhs.v[0] - rhs[0], lhs.v[1] - rhs[1], lhs.v[2] - rhs[2] }; } inline vr::HmdVector3d_t operator*(const vr::HmdVector3d_t& lhs, const double rhs) { return{ lhs.v[0] * rhs, lhs.v[1] * rhs, lhs.v[2] * rhs }; } inline vr::HmdVector3d_t operator/(const vr::HmdVector3d_t& lhs, const double rhs) { return{ lhs.v[0] / rhs, lhs.v[1] / rhs, lhs.v[2] / rhs }; } namespace vrmath { template int signum(T v) { return (v > (T)0) ? 1 : ((v < (T)0) ? -1 : 0); } inline vr::HmdQuaternion_t quaternionFromRotationAxis(double rot, double ux, double uy, double uz) { auto ha = rot / 2; return{ std::cos(ha), ux * std::sin(ha), uy * std::sin(ha), uz * std::sin(ha) }; } inline vr::HmdQuaternion_t quaternionFromRotationX(double rot) { auto ha = rot / 2; return{ std::cos(ha), std::sin(ha), 0.0f, 0.0f }; } inline vr::HmdQuaternion_t quaternionFromRotationY(double rot) { auto ha = rot / 2; return{ std::cos(ha), 0.0f, std::sin(ha), 0.0f }; } inline vr::HmdQuaternion_t quaternionFromRotationZ(double rot) { auto ha = rot / 2; return{ std::cos(ha), 0.0f, 0.0f, std::sin(ha) }; } inline vr::HmdQuaternion_t quaternionFromYawPitchRoll(double yaw, double pitch, double roll) { return quaternionFromRotationY(yaw) * quaternionFromRotationX(pitch) * quaternionFromRotationZ(roll); } inline vr::HmdQuaternion_t quaternionFromRotationMatrix(const vr::HmdMatrix34_t& mat) { auto a = mat.m; vr::HmdQuaternion_t q; double trace = a[0][0] + a[1][1] + a[2][2]; if (trace > 0) { double s = 0.5 / sqrt(trace + 1.0); q.w = 0.25 / s; q.x = (a[1][2] - a[2][1]) * s; q.y = (a[2][0] - a[0][2]) * s; q.z = (a[0][1] - a[1][0]) * s; } else { if (a[0][0] > a[1][1] && a[0][0] > a[2][2]) { double s = 2.0 * sqrt(1.0 + a[0][0] - a[1][1] - a[2][2]); q.w = (a[1][2] - a[2][1]) / s; q.x = 0.25 * s; q.y = (a[1][0] + a[0][1]) / s; q.z = (a[2][0] + a[0][2]) / s; } else if (a[1][1] > a[2][2]) { double s = 2.0 * sqrt(1.0 + a[1][1] - a[0][0] - a[2][2]); q.w = (a[2][0] - a[0][2]) / s; q.x = (a[1][0] + a[0][1]) / s; q.y = 0.25 * s; q.z = (a[2][1] + a[1][2]) / s; } else { double s = 2.0 * sqrt(1.0 + a[2][2] - a[0][0] - a[1][1]); q.w = (a[0][1] - a[1][0]) / s; q.x = (a[2][0] + a[0][2]) / s; q.y = (a[2][1] + a[1][2]) / s; q.z = 0.25 * s; } } q.x = -q.x; q.y = -q.y; q.z = -q.z; return q; } inline vr::HmdQuaternion_t quaternionConjugate(const vr::HmdQuaternion_t& quat) { return { quat.w, -quat.x, -quat.y, -quat.z, }; } inline vr::HmdVector3d_t quaternionRotateVector(const vr::HmdQuaternion_t& quat, const vr::HmdVector3d_t& vector, bool reverse = false) { if (reverse) { vr::HmdQuaternion_t pin = { 0.0, vector.v[0], vector.v[1] , vector.v[2] }; auto pout = vrmath::quaternionConjugate(quat) * pin * quat; return {pout.x, pout.y, pout.z}; } else { vr::HmdQuaternion_t pin = { 0.0, vector.v[0], vector.v[1] , vector.v[2] }; auto pout = quat * pin * vrmath::quaternionConjugate(quat); return { pout.x, pout.y, pout.z }; } } inline vr::HmdVector3d_t quaternionRotateVector(const vr::HmdQuaternion_t& quat, const vr::HmdQuaternion_t& quatInv, const vr::HmdVector3d_t& vector, bool reverse = false) { if (reverse) { vr::HmdQuaternion_t pin = { 0.0, vector.v[0], vector.v[1] , vector.v[2] }; auto pout = quatInv * pin * quat; return{ pout.x, pout.y, pout.z }; } else { vr::HmdQuaternion_t pin = { 0.0, vector.v[0], vector.v[1] , vector.v[2] }; auto pout = quat * pin * quatInv; return{ pout.x, pout.y, pout.z }; } } inline vr::HmdVector3d_t quaternionRotateVector(const vr::HmdQuaternion_t& quat, const double (&vector)[3], bool reverse = false) { if (reverse) { vr::HmdQuaternion_t pin = { 0.0, vector[0], vector[1] , vector[2] }; auto pout = vrmath::quaternionConjugate(quat) * pin * quat; return{ pout.x, pout.y, pout.z }; } else { vr::HmdQuaternion_t pin = { 0.0, vector[0], vector[1] , vector[2] }; auto pout = quat * pin * vrmath::quaternionConjugate(quat); return{ pout.x, pout.y, pout.z }; } } inline vr::HmdVector3d_t quaternionRotateVector(const vr::HmdQuaternion_t& quat, const vr::HmdQuaternion_t& quatInv, const double(&vector)[3], bool reverse = false) { if (reverse) { vr::HmdQuaternion_t pin = { 0.0, vector[0], vector[1] , vector[2] }; auto pout = quatInv * pin * quat; return{ pout.x, pout.y, pout.z }; } else { vr::HmdQuaternion_t pin = { 0.0, vector[0], vector[1] , vector[2] }; auto pout = quat * pin * quatInv; return{ pout.x, pout.y, pout.z }; } } inline vr::HmdMatrix34_t matMul33(const vr::HmdMatrix34_t& a, const vr::HmdMatrix34_t& b) { vr::HmdMatrix34_t result; for (unsigned i = 0; i < 3; i++) { for (unsigned j = 0; j < 3; j++) { result.m[i][j] = 0.0f; for (unsigned k = 0; k < 3; k++) { result.m[i][j] += a.m[i][k] * b.m[k][j]; } } } return result; } inline vr::HmdVector3_t matMul33(const vr::HmdMatrix34_t& a, const vr::HmdVector3_t& b) { vr::HmdVector3_t result; for (unsigned i = 0; i < 3; i++) { result.v[i] = 0.0f; for (unsigned k = 0; k < 3; k++) { result.v[i] += a.m[i][k] * b.v[k]; }; } return result; } inline vr::HmdVector3d_t matMul33(const vr::HmdMatrix34_t& a, const vr::HmdVector3d_t& b) { vr::HmdVector3d_t result; for (unsigned i = 0; i < 3; i++) { result.v[i] = 0.0f; for (unsigned k = 0; k < 3; k++) { result.v[i] += a.m[i][k] * b.v[k]; }; } return result; } inline vr::HmdVector3_t matMul33(const vr::HmdVector3_t& a, const vr::HmdMatrix34_t& b) { vr::HmdVector3_t result; for (unsigned i = 0; i < 3; i++) { result.v[i] = 0.0f; for (unsigned k = 0; k < 3; k++) { result.v[i] += a.v[k] * b.m[k][i]; }; } return result; } inline vr::HmdVector3d_t matMul33(const vr::HmdVector3d_t& a, const vr::HmdMatrix34_t& b) { vr::HmdVector3d_t result; for (unsigned i = 0; i < 3; i++) { result.v[i] = 0.0f; for (unsigned k = 0; k < 3; k++) { result.v[i] += a.v[k] * b.m[k][i]; }; } return result; } inline vr::HmdMatrix34_t transposeMul33(const vr::HmdMatrix34_t& a) { vr::HmdMatrix34_t result; for (unsigned i = 0; i < 3; i++) { for (unsigned k = 0; k < 3; k++) { result.m[i][k] = a.m[k][i]; } } result.m[0][3] = a.m[0][3]; result.m[1][3] = a.m[1][3]; result.m[2][3] = a.m[2][3]; return result; } inline vr::HmdMatrix34_t matInv33(vr::HmdMatrix34_t matrix) { vr::HmdMatrix34_t result; float cofac00 = matrix.m[1][1] * matrix.m[2][2] - matrix.m[1][2] * matrix.m[2][1]; float cofac10 = matrix.m[1][2] * matrix.m[2][0] - matrix.m[1][0] * matrix.m[2][2]; float cofac20 = matrix.m[1][0] * matrix.m[2][1] - matrix.m[1][1] * matrix.m[2][0]; float det = matrix.m[0][0] * cofac00 + matrix.m[0][1] * cofac10 + matrix.m[0][2] * cofac20; if (det == 0) { vr::HmdMatrix34_t result = { { { 1.0, 0.0, 0.0, 0.0 }, { 0.0, 1.0, 0.0, 0.0 }, { 0.0, 0.0, 1.0, 0.0 } } }; return result; } float invDet = 1.0f / det; float cofac01 = matrix.m[0][2] * matrix.m[2][1] - matrix.m[0][1] * matrix.m[2][2]; float cofac02 = matrix.m[0][1] * matrix.m[1][2] - matrix.m[0][2] * matrix.m[1][1]; float cofac11 = matrix.m[0][0] * matrix.m[2][2] - matrix.m[0][2] * matrix.m[2][0]; float cofac12 = matrix.m[0][2] * matrix.m[1][0] - matrix.m[0][0] * matrix.m[1][2]; float cofac21 = matrix.m[0][1] * matrix.m[2][0] - matrix.m[0][0] * matrix.m[2][1]; float cofac22 = matrix.m[0][0] * matrix.m[1][1] - matrix.m[0][1] * matrix.m[1][0]; result.m[0][0] = invDet * cofac00; result.m[0][1] = invDet * cofac01; result.m[0][2] = invDet * cofac02; result.m[0][3] = 0.0f; result.m[1][0] = invDet * cofac10; result.m[1][1] = invDet * cofac11; result.m[1][2] = invDet * cofac12; result.m[1][3] = 0.0f; result.m[2][0] = invDet * cofac20; result.m[2][1] = invDet * cofac21; result.m[2][2] = invDet * cofac22; result.m[2][3] = 0.0f; return result; } } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/include/picojson.h ================================================ /* * Copyright 2009-2010 Cybozu Labs, Inc. * Copyright 2011-2014 Kazuho Oku * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ #ifndef picojson_h #define picojson_h #include #include #include #include #include #include #include #include #include #include #include #include #include // for isnan/isinf #if __cplusplus >= 201103L #include #else extern "C" { #ifdef _MSC_VER #include #elif defined(__INTEL_COMPILER) #include #else #include #endif } #endif #ifndef PICOJSON_USE_RVALUE_REFERENCE #if (defined(__cpp_rvalue_references) && __cpp_rvalue_references >= 200610) || (defined(_MSC_VER) && _MSC_VER >= 1600) #define PICOJSON_USE_RVALUE_REFERENCE 1 #else #define PICOJSON_USE_RVALUE_REFERENCE 0 #endif #endif // PICOJSON_USE_RVALUE_REFERENCE #ifndef PICOJSON_NOEXCEPT #if PICOJSON_USE_RVALUE_REFERENCE #define PICOJSON_NOEXCEPT noexcept #else #define PICOJSON_NOEXCEPT throw() #endif #endif // experimental support for int64_t (see README.mkdn for detail) #ifdef PICOJSON_USE_INT64 #define __STDC_FORMAT_MACROS #include #include #endif // to disable the use of localeconv(3), set PICOJSON_USE_LOCALE to 0 #ifndef PICOJSON_USE_LOCALE #define PICOJSON_USE_LOCALE 1 #endif #if PICOJSON_USE_LOCALE extern "C" { #include } #endif #ifndef PICOJSON_ASSERT #define PICOJSON_ASSERT(e) \ do { \ if (!(e)) \ throw std::runtime_error(#e); \ } while (0) #endif #ifdef _MSC_VER #define SNPRINTF _snprintf_s #pragma warning(push) #pragma warning(disable : 4244) // conversion from int to char #pragma warning(disable : 4127) // conditional expression is constant #pragma warning(disable : 4702) // unreachable code #else #define SNPRINTF snprintf #endif namespace picojson { enum { null_type, boolean_type, number_type, string_type, array_type, object_type #ifdef PICOJSON_USE_INT64 , int64_type #endif }; enum { INDENT_WIDTH = 2 }; struct null {}; class value { public: typedef std::vector array; typedef std::map object; union _storage { bool boolean_; double number_; #ifdef PICOJSON_USE_INT64 int64_t int64_; #endif std::string *string_; array *array_; object *object_; }; protected: int type_; _storage u_; public: value(); value(int type, bool); explicit value(bool b); #ifdef PICOJSON_USE_INT64 explicit value(int64_t i); #endif explicit value(double n); explicit value(const std::string &s); explicit value(const array &a); explicit value(const object &o); #if PICOJSON_USE_RVALUE_REFERENCE explicit value(std::string &&s); explicit value(array &&a); explicit value(object &&o); #endif explicit value(const char *s); value(const char *s, size_t len); ~value(); value(const value &x); value &operator=(const value &x); #if PICOJSON_USE_RVALUE_REFERENCE value(value &&x) PICOJSON_NOEXCEPT; value &operator=(value &&x) PICOJSON_NOEXCEPT; #endif void swap(value &x) PICOJSON_NOEXCEPT; template bool is() const; template const T &get() const; template T &get(); template void set(const T &); #if PICOJSON_USE_RVALUE_REFERENCE template void set(T &&); #endif bool evaluate_as_boolean() const; const value &get(const size_t idx) const; const value &get(const std::string &key) const; value &get(const size_t idx); value &get(const std::string &key); bool contains(const size_t idx) const; bool contains(const std::string &key) const; std::string to_str() const; template void serialize(Iter os, bool prettify = false) const; std::string serialize(bool prettify = false) const; private: template value(const T *); // intentionally defined to block implicit conversion of pointer to bool template static void _indent(Iter os, int indent); template void _serialize(Iter os, int indent) const; std::string _serialize(int indent) const; void clear(); }; typedef value::array array; typedef value::object object; inline value::value() : type_(null_type), u_() { } inline value::value(int type, bool) : type_(type), u_() { switch (type) { #define INIT(p, v) \ case p##type: \ u_.p = v; \ break INIT(boolean_, false); INIT(number_, 0.0); #ifdef PICOJSON_USE_INT64 INIT(int64_, 0); #endif INIT(string_, new std::string()); INIT(array_, new array()); INIT(object_, new object()); #undef INIT default: break; } } inline value::value(bool b) : type_(boolean_type), u_() { u_.boolean_ = b; } #ifdef PICOJSON_USE_INT64 inline value::value(int64_t i) : type_(int64_type), u_() { u_.int64_ = i; } #endif inline value::value(double n) : type_(number_type), u_() { if ( #ifdef _MSC_VER !_finite(n) #elif __cplusplus >= 201103L || !(defined(isnan) && defined(isinf)) std::isnan(n) || std::isinf(n) #else isnan(n) || isinf(n) #endif ) { throw std::overflow_error(""); } u_.number_ = n; } inline value::value(const std::string &s) : type_(string_type), u_() { u_.string_ = new std::string(s); } inline value::value(const array &a) : type_(array_type), u_() { u_.array_ = new array(a); } inline value::value(const object &o) : type_(object_type), u_() { u_.object_ = new object(o); } #if PICOJSON_USE_RVALUE_REFERENCE inline value::value(std::string &&s) : type_(string_type), u_() { u_.string_ = new std::string(std::move(s)); } inline value::value(array &&a) : type_(array_type), u_() { u_.array_ = new array(std::move(a)); } inline value::value(object &&o) : type_(object_type), u_() { u_.object_ = new object(std::move(o)); } #endif inline value::value(const char *s) : type_(string_type), u_() { u_.string_ = new std::string(s); } inline value::value(const char *s, size_t len) : type_(string_type), u_() { u_.string_ = new std::string(s, len); } inline void value::clear() { switch (type_) { #define DEINIT(p) \ case p##type: \ delete u_.p; \ break DEINIT(string_); DEINIT(array_); DEINIT(object_); #undef DEINIT default: break; } } inline value::~value() { clear(); } inline value::value(const value &x) : type_(x.type_), u_() { switch (type_) { #define INIT(p, v) \ case p##type: \ u_.p = v; \ break INIT(string_, new std::string(*x.u_.string_)); INIT(array_, new array(*x.u_.array_)); INIT(object_, new object(*x.u_.object_)); #undef INIT default: u_ = x.u_; break; } } inline value &value::operator=(const value &x) { if (this != &x) { value t(x); swap(t); } return *this; } #if PICOJSON_USE_RVALUE_REFERENCE inline value::value(value &&x) PICOJSON_NOEXCEPT : type_(null_type), u_() { swap(x); } inline value &value::operator=(value &&x) PICOJSON_NOEXCEPT { swap(x); return *this; } #endif inline void value::swap(value &x) PICOJSON_NOEXCEPT { std::swap(type_, x.type_); std::swap(u_, x.u_); } #define IS(ctype, jtype) \ template <> inline bool value::is() const { \ return type_ == jtype##_type; \ } IS(null, null) IS(bool, boolean) #ifdef PICOJSON_USE_INT64 IS(int64_t, int64) #endif IS(std::string, string) IS(array, array) IS(object, object) #undef IS template <> inline bool value::is() const { return type_ == number_type #ifdef PICOJSON_USE_INT64 || type_ == int64_type #endif ; } #define GET(ctype, var) \ template <> inline const ctype &value::get() const { \ PICOJSON_ASSERT("type mismatch! call is() before get()" && is()); \ return var; \ } \ template <> inline ctype &value::get() { \ PICOJSON_ASSERT("type mismatch! call is() before get()" && is()); \ return var; \ } GET(bool, u_.boolean_) GET(std::string, *u_.string_) GET(array, *u_.array_) GET(object, *u_.object_) #ifdef PICOJSON_USE_INT64 GET(double, (type_ == int64_type && (const_cast(this)->type_ = number_type, (const_cast(this)->u_.number_ = u_.int64_)), u_.number_)) GET(int64_t, u_.int64_) #else GET(double, u_.number_) #endif #undef GET #define SET(ctype, jtype, setter) \ template <> inline void value::set(const ctype &_val) { \ clear(); \ type_ = jtype##_type; \ setter \ } SET(bool, boolean, u_.boolean_ = _val;) SET(std::string, string, u_.string_ = new std::string(_val);) SET(array, array, u_.array_ = new array(_val);) SET(object, object, u_.object_ = new object(_val);) SET(double, number, u_.number_ = _val;) #ifdef PICOJSON_USE_INT64 SET(int64_t, int64, u_.int64_ = _val;) #endif #undef SET #if PICOJSON_USE_RVALUE_REFERENCE #define MOVESET(ctype, jtype, setter) \ template <> inline void value::set(ctype && _val) { \ clear(); \ type_ = jtype##_type; \ setter \ } MOVESET(std::string, string, u_.string_ = new std::string(std::move(_val));) MOVESET(array, array, u_.array_ = new array(std::move(_val));) MOVESET(object, object, u_.object_ = new object(std::move(_val));) #undef MOVESET #endif inline bool value::evaluate_as_boolean() const { switch (type_) { case null_type: return false; case boolean_type: return u_.boolean_; case number_type: return u_.number_ != 0; #ifdef PICOJSON_USE_INT64 case int64_type: return u_.int64_ != 0; #endif case string_type: return !u_.string_->empty(); default: return true; } } inline const value &value::get(const size_t idx) const { static value s_null; PICOJSON_ASSERT(is()); return idx < u_.array_->size() ? (*u_.array_)[idx] : s_null; } inline value &value::get(const size_t idx) { static value s_null; PICOJSON_ASSERT(is()); return idx < u_.array_->size() ? (*u_.array_)[idx] : s_null; } inline const value &value::get(const std::string &key) const { static value s_null; PICOJSON_ASSERT(is()); object::const_iterator i = u_.object_->find(key); return i != u_.object_->end() ? i->second : s_null; } inline value &value::get(const std::string &key) { static value s_null; PICOJSON_ASSERT(is()); object::iterator i = u_.object_->find(key); return i != u_.object_->end() ? i->second : s_null; } inline bool value::contains(const size_t idx) const { PICOJSON_ASSERT(is()); return idx < u_.array_->size(); } inline bool value::contains(const std::string &key) const { PICOJSON_ASSERT(is()); object::const_iterator i = u_.object_->find(key); return i != u_.object_->end(); } inline std::string value::to_str() const { switch (type_) { case null_type: return "null"; case boolean_type: return u_.boolean_ ? "true" : "false"; #ifdef PICOJSON_USE_INT64 case int64_type: { char buf[sizeof("-9223372036854775808")]; SNPRINTF(buf, sizeof(buf), "%" PRId64, u_.int64_); return buf; } #endif case number_type: { char buf[256]; double tmp; SNPRINTF(buf, sizeof(buf), fabs(u_.number_) < (1ULL << 53) && modf(u_.number_, &tmp) == 0 ? "%.f" : "%.17g", u_.number_); #if PICOJSON_USE_LOCALE char *decimal_point = localeconv()->decimal_point; if (strcmp(decimal_point, ".") != 0) { size_t decimal_point_len = strlen(decimal_point); for (char *p = buf; *p != '\0'; ++p) { if (strncmp(p, decimal_point, decimal_point_len) == 0) { return std::string(buf, p) + "." + (p + decimal_point_len); } } } #endif return buf; } case string_type: return *u_.string_; case array_type: return "array"; case object_type: return "object"; default: PICOJSON_ASSERT(0); #ifdef _MSC_VER __assume(0); #endif } return std::string(); } template void copy(const std::string &s, Iter oi) { std::copy(s.begin(), s.end(), oi); } template struct serialize_str_char { Iter oi; void operator()(char c) { switch (c) { #define MAP(val, sym) \ case val: \ copy(sym, oi); \ break MAP('"', "\\\""); MAP('\\', "\\\\"); MAP('/', "\\/"); MAP('\b', "\\b"); MAP('\f', "\\f"); MAP('\n', "\\n"); MAP('\r', "\\r"); MAP('\t', "\\t"); #undef MAP default: if (static_cast(c) < 0x20 || c == 0x7f) { char buf[7]; SNPRINTF(buf, sizeof(buf), "\\u%04x", c & 0xff); copy(buf, buf + 6, oi); } else { *oi++ = c; } break; } } }; template void serialize_str(const std::string &s, Iter oi) { *oi++ = '"'; serialize_str_char process_char = {oi}; std::for_each(s.begin(), s.end(), process_char); *oi++ = '"'; } template void value::serialize(Iter oi, bool prettify) const { return _serialize(oi, prettify ? 0 : -1); } inline std::string value::serialize(bool prettify) const { return _serialize(prettify ? 0 : -1); } template void value::_indent(Iter oi, int indent) { *oi++ = '\n'; for (int i = 0; i < indent * INDENT_WIDTH; ++i) { *oi++ = ' '; } } template void value::_serialize(Iter oi, int indent) const { switch (type_) { case string_type: serialize_str(*u_.string_, oi); break; case array_type: { *oi++ = '['; if (indent != -1) { ++indent; } for (array::const_iterator i = u_.array_->begin(); i != u_.array_->end(); ++i) { if (i != u_.array_->begin()) { *oi++ = ','; } if (indent != -1) { _indent(oi, indent); } i->_serialize(oi, indent); } if (indent != -1) { --indent; if (!u_.array_->empty()) { _indent(oi, indent); } } *oi++ = ']'; break; } case object_type: { *oi++ = '{'; if (indent != -1) { ++indent; } for (object::const_iterator i = u_.object_->begin(); i != u_.object_->end(); ++i) { if (i != u_.object_->begin()) { *oi++ = ','; } if (indent != -1) { _indent(oi, indent); } serialize_str(i->first, oi); *oi++ = ':'; if (indent != -1) { *oi++ = ' '; } i->second._serialize(oi, indent); } if (indent != -1) { --indent; if (!u_.object_->empty()) { _indent(oi, indent); } } *oi++ = '}'; break; } default: copy(to_str(), oi); break; } if (indent == 0) { *oi++ = '\n'; } } inline std::string value::_serialize(int indent) const { std::string s; _serialize(std::back_inserter(s), indent); return s; } template class input { protected: Iter cur_, end_; bool consumed_; int line_; public: input(const Iter &first, const Iter &last) : cur_(first), end_(last), consumed_(false), line_(1) { } int getc() { if (consumed_) { if (*cur_ == '\n') { ++line_; } ++cur_; } if (cur_ == end_) { consumed_ = false; return -1; } consumed_ = true; return *cur_ & 0xff; } void ungetc() { consumed_ = false; } Iter cur() const { if (consumed_) { input *self = const_cast *>(this); self->consumed_ = false; ++self->cur_; } return cur_; } int line() const { return line_; } void skip_ws() { while (1) { int ch = getc(); if (!(ch == ' ' || ch == '\t' || ch == '\n' || ch == '\r')) { ungetc(); break; } } } bool expect(const int expected) { skip_ws(); if (getc() != expected) { ungetc(); return false; } return true; } bool match(const std::string &pattern) { for (std::string::const_iterator pi(pattern.begin()); pi != pattern.end(); ++pi) { if (getc() != *pi) { ungetc(); return false; } } return true; } }; template inline int _parse_quadhex(input &in) { int uni_ch = 0, hex; for (int i = 0; i < 4; i++) { if ((hex = in.getc()) == -1) { return -1; } if ('0' <= hex && hex <= '9') { hex -= '0'; } else if ('A' <= hex && hex <= 'F') { hex -= 'A' - 0xa; } else if ('a' <= hex && hex <= 'f') { hex -= 'a' - 0xa; } else { in.ungetc(); return -1; } uni_ch = uni_ch * 16 + hex; } return uni_ch; } template inline bool _parse_codepoint(String &out, input &in) { int uni_ch; if ((uni_ch = _parse_quadhex(in)) == -1) { return false; } if (0xd800 <= uni_ch && uni_ch <= 0xdfff) { if (0xdc00 <= uni_ch) { // a second 16-bit of a surrogate pair appeared return false; } // first 16-bit of surrogate pair, get the next one if (in.getc() != '\\' || in.getc() != 'u') { in.ungetc(); return false; } int second = _parse_quadhex(in); if (!(0xdc00 <= second && second <= 0xdfff)) { return false; } uni_ch = ((uni_ch - 0xd800) << 10) | ((second - 0xdc00) & 0x3ff); uni_ch += 0x10000; } if (uni_ch < 0x80) { out.push_back(static_cast(uni_ch)); } else { if (uni_ch < 0x800) { out.push_back(static_cast(0xc0 | (uni_ch >> 6))); } else { if (uni_ch < 0x10000) { out.push_back(static_cast(0xe0 | (uni_ch >> 12))); } else { out.push_back(static_cast(0xf0 | (uni_ch >> 18))); out.push_back(static_cast(0x80 | ((uni_ch >> 12) & 0x3f))); } out.push_back(static_cast(0x80 | ((uni_ch >> 6) & 0x3f))); } out.push_back(static_cast(0x80 | (uni_ch & 0x3f))); } return true; } template inline bool _parse_string(String &out, input &in) { while (1) { int ch = in.getc(); if (ch < ' ') { in.ungetc(); return false; } else if (ch == '"') { return true; } else if (ch == '\\') { if ((ch = in.getc()) == -1) { return false; } switch (ch) { #define MAP(sym, val) \ case sym: \ out.push_back(val); \ break MAP('"', '\"'); MAP('\\', '\\'); MAP('/', '/'); MAP('b', '\b'); MAP('f', '\f'); MAP('n', '\n'); MAP('r', '\r'); MAP('t', '\t'); #undef MAP case 'u': if (!_parse_codepoint(out, in)) { return false; } break; default: return false; } } else { out.push_back(static_cast(ch)); } } return false; } template inline bool _parse_array(Context &ctx, input &in) { if (!ctx.parse_array_start()) { return false; } size_t idx = 0; if (in.expect(']')) { return ctx.parse_array_stop(idx); } do { if (!ctx.parse_array_item(in, idx)) { return false; } idx++; } while (in.expect(',')); return in.expect(']') && ctx.parse_array_stop(idx); } template inline bool _parse_object(Context &ctx, input &in) { if (!ctx.parse_object_start()) { return false; } if (in.expect('}')) { return true; } do { std::string key; if (!in.expect('"') || !_parse_string(key, in) || !in.expect(':')) { return false; } if (!ctx.parse_object_item(in, key)) { return false; } } while (in.expect(',')); return in.expect('}'); } template inline std::string _parse_number(input &in) { std::string num_str; while (1) { int ch = in.getc(); if (('0' <= ch && ch <= '9') || ch == '+' || ch == '-' || ch == 'e' || ch == 'E') { num_str.push_back(static_cast(ch)); } else if (ch == '.') { #if PICOJSON_USE_LOCALE num_str += localeconv()->decimal_point; #else num_str.push_back('.'); #endif } else { in.ungetc(); break; } } return num_str; } template inline bool _parse(Context &ctx, input &in) { in.skip_ws(); int ch = in.getc(); switch (ch) { #define IS(ch, text, op) \ case ch: \ if (in.match(text) && op) { \ return true; \ } else { \ return false; \ } IS('n', "ull", ctx.set_null()); IS('f', "alse", ctx.set_bool(false)); IS('t', "rue", ctx.set_bool(true)); #undef IS case '"': return ctx.parse_string(in); case '[': return _parse_array(ctx, in); case '{': return _parse_object(ctx, in); default: if (('0' <= ch && ch <= '9') || ch == '-') { double f; char *endp; in.ungetc(); std::string num_str(_parse_number(in)); if (num_str.empty()) { return false; } #ifdef PICOJSON_USE_INT64 { errno = 0; intmax_t ival = strtoimax(num_str.c_str(), &endp, 10); if (errno == 0 && std::numeric_limits::min() <= ival && ival <= std::numeric_limits::max() && endp == num_str.c_str() + num_str.size()) { ctx.set_int64(ival); return true; } } #endif f = strtod(num_str.c_str(), &endp); if (endp == num_str.c_str() + num_str.size()) { ctx.set_number(f); return true; } return false; } break; } in.ungetc(); return false; } class deny_parse_context { public: bool set_null() { return false; } bool set_bool(bool) { return false; } #ifdef PICOJSON_USE_INT64 bool set_int64(int64_t) { return false; } #endif bool set_number(double) { return false; } template bool parse_string(input &) { return false; } bool parse_array_start() { return false; } template bool parse_array_item(input &, size_t) { return false; } bool parse_array_stop(size_t) { return false; } bool parse_object_start() { return false; } template bool parse_object_item(input &, const std::string &) { return false; } }; class default_parse_context { protected: value *out_; public: default_parse_context(value *out) : out_(out) { } bool set_null() { *out_ = value(); return true; } bool set_bool(bool b) { *out_ = value(b); return true; } #ifdef PICOJSON_USE_INT64 bool set_int64(int64_t i) { *out_ = value(i); return true; } #endif bool set_number(double f) { *out_ = value(f); return true; } template bool parse_string(input &in) { *out_ = value(string_type, false); return _parse_string(out_->get(), in); } bool parse_array_start() { *out_ = value(array_type, false); return true; } template bool parse_array_item(input &in, size_t) { array &a = out_->get(); a.push_back(value()); default_parse_context ctx(&a.back()); return _parse(ctx, in); } bool parse_array_stop(size_t) { return true; } bool parse_object_start() { *out_ = value(object_type, false); return true; } template bool parse_object_item(input &in, const std::string &key) { object &o = out_->get(); default_parse_context ctx(&o[key]); return _parse(ctx, in); } private: default_parse_context(const default_parse_context &); default_parse_context &operator=(const default_parse_context &); }; class null_parse_context { public: struct dummy_str { void push_back(int) { } }; public: null_parse_context() { } bool set_null() { return true; } bool set_bool(bool) { return true; } #ifdef PICOJSON_USE_INT64 bool set_int64(int64_t) { return true; } #endif bool set_number(double) { return true; } template bool parse_string(input &in) { dummy_str s; return _parse_string(s, in); } bool parse_array_start() { return true; } template bool parse_array_item(input &in, size_t) { return _parse(*this, in); } bool parse_array_stop(size_t) { return true; } bool parse_object_start() { return true; } template bool parse_object_item(input &in, const std::string &) { return _parse(*this, in); } private: null_parse_context(const null_parse_context &); null_parse_context &operator=(const null_parse_context &); }; // obsolete, use the version below template inline std::string parse(value &out, Iter &pos, const Iter &last) { std::string err; pos = parse(out, pos, last, &err); return err; } template inline Iter _parse(Context &ctx, const Iter &first, const Iter &last, std::string *err) { input in(first, last); if (!_parse(ctx, in) && err != NULL) { char buf[64]; SNPRINTF(buf, sizeof(buf), "syntax error at line %d near: ", in.line()); *err = buf; while (1) { int ch = in.getc(); if (ch == -1 || ch == '\n') { break; } else if (ch >= ' ') { err->push_back(static_cast(ch)); } } } return in.cur(); } template inline Iter parse(value &out, const Iter &first, const Iter &last, std::string *err) { default_parse_context ctx(&out); return _parse(ctx, first, last, err); } inline std::string parse(value &out, const std::string &s) { std::string err; parse(out, s.begin(), s.end(), &err); return err; } inline std::string parse(value &out, std::istream &is) { std::string err; parse(out, std::istreambuf_iterator(is.rdbuf()), std::istreambuf_iterator(), &err); return err; } template struct last_error_t { static std::string s; }; template std::string last_error_t::s; inline void set_last_error(const std::string &s) { last_error_t::s = s; } inline const std::string &get_last_error() { return last_error_t::s; } inline bool operator==(const value &x, const value &y) { if (x.is()) return y.is(); #define PICOJSON_CMP(type) \ if (x.is()) \ return y.is() && x.get() == y.get() PICOJSON_CMP(bool); PICOJSON_CMP(double); PICOJSON_CMP(std::string); PICOJSON_CMP(array); PICOJSON_CMP(object); #undef PICOJSON_CMP PICOJSON_ASSERT(0); #ifdef _MSC_VER __assume(0); #endif return false; } inline bool operator!=(const value &x, const value &y) { return !(x == y); } } #if !PICOJSON_USE_RVALUE_REFERENCE namespace std { template <> inline void swap(picojson::value &x, picojson::value &y) { x.swap(y); } } #endif inline std::istream &operator>>(std::istream &is, picojson::value &x) { picojson::set_last_error(std::string()); const std::string err(picojson::parse(x, is)); if (!err.empty()) { picojson::set_last_error(err); is.setstate(std::ios::failbit); } return is; } inline std::ostream &operator<<(std::ostream &os, const picojson::value &x) { x.serialize(std::ostream_iterator(os)); return os; } #ifdef _MSC_VER #pragma warning(pop) #endif #endif ================================================ FILE: alvr/server_openvr/cpp/alvr_server/nvEncodeAPI.h ================================================ /* * This copyright notice applies to this header file only: * * Copyright (c) 2010-2022 NVIDIA Corporation * * Permission is hereby granted, free of charge, to any person * obtaining a copy of this software and associated documentation * files (the "Software"), to deal in the Software without * restriction, including without limitation the rights to use, * copy, modify, merge, publish, distribute, sublicense, and/or sell * copies of the software, and to permit persons to whom the * software is furnished to do so, subject to the following * conditions: * * The above copyright notice and this permission notice shall be * included in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. */ /** * \file nvEncodeAPI.h * NVIDIA GPUs - beginning with the Kepler generation - contain a hardware-based encoder * (referred to as NVENC) which provides fully-accelerated hardware-based video encoding. * NvEncodeAPI provides the interface for NVIDIA video encoder (NVENC). * \date 2011-2022 * This file contains the interface constants, structure definitions and function prototypes. */ #ifndef _NV_ENCODEAPI_H_ #define _NV_ENCODEAPI_H_ #include #ifdef _WIN32 #include #endif #ifdef _MSC_VER #ifndef _STDINT typedef __int32 int32_t; typedef unsigned __int32 uint32_t; typedef __int64 int64_t; typedef unsigned __int64 uint64_t; typedef signed char int8_t; typedef unsigned char uint8_t; typedef short int16_t; typedef unsigned short uint16_t; #endif #else #include #endif #ifdef __cplusplus extern "C" { #endif /** * \addtogroup ENCODER_STRUCTURE NvEncodeAPI Data structures * @{ */ #ifdef _WIN32 #define NVENCAPI __stdcall typedef RECT NVENC_RECT; #else #define NVENCAPI // ========================================================================================= #ifndef GUID_DEFINED #define GUID_DEFINED /*! * \struct GUID * Abstracts the GUID structure for non-windows platforms. */ // ========================================================================================= typedef struct _GUID { uint32_t Data1; /**< [in]: Specifies the first 8 hexadecimal digits of the GUID. */ uint16_t Data2; /**< [in]: Specifies the first group of 4 hexadecimal digits. */ uint16_t Data3; /**< [in]: Specifies the second group of 4 hexadecimal digits. */ uint8_t Data4[8]; /**< [in]: Array of 8 bytes. The first 2 bytes contain the third group of 4 hexadecimal digits. The remaining 6 bytes contain the final 12 hexadecimal digits. */ } GUID, *LPGUID; #endif // GUID /** * \struct _NVENC_RECT * Defines a Rectangle. Used in ::NV_ENC_PREPROCESS_FRAME. */ typedef struct _NVENC_RECT { uint32_t left; /**< [in]: X coordinate of the upper left corner of rectangular area to be specified. */ uint32_t top; /**< [in]: Y coordinate of the upper left corner of the rectangular area to be specified. */ uint32_t right; /**< [in]: X coordinate of the bottom right corner of the rectangular area to be specified. */ uint32_t bottom; /**< [in]: Y coordinate of the bottom right corner of the rectangular area to be specified. */ } NVENC_RECT; #endif // _WIN32 /** @} */ /* End of GUID and NVENC_RECT structure grouping*/ typedef void* NV_ENC_INPUT_PTR; /**< NVENCODE API input buffer */ typedef void* NV_ENC_OUTPUT_PTR; /**< NVENCODE API output buffer*/ typedef void* NV_ENC_REGISTERED_PTR; /**< A Resource that has been registered with NVENCODE API*/ typedef void* NV_ENC_CUSTREAM_PTR; /**< Pointer to CUstream*/ #define NVENCAPI_MAJOR_VERSION 12 #define NVENCAPI_MINOR_VERSION 0 #define NVENCAPI_VERSION (NVENCAPI_MAJOR_VERSION | (NVENCAPI_MINOR_VERSION << 24)) /** * Macro to generate per-structure version for use with API. */ #define NVENCAPI_STRUCT_VERSION(ver) ((uint32_t)NVENCAPI_VERSION | ((ver) << 16) | (0x7 << 28)) #define NVENC_INFINITE_GOPLENGTH 0xffffffff #define NV_MAX_SEQ_HDR_LEN (512) #ifdef __GNUC__ #define NV_ENC_DEPRECATED \ __attribute__((deprecated("WILL BE REMOVED IN A FUTURE VIDEO CODEC SDK VERSION"))) #elif defined(_MSC_VER) #define NV_ENC_DEPRECATED \ __declspec(deprecated("WILL BE REMOVED IN A FUTURE VIDEO CODEC SDK VERSION")) #endif // ========================================================================================= // Encode Codec GUIDS supported by the NvEncodeAPI interface. // ========================================================================================= // {6BC82762-4E63-4ca4-AA85-1E50F321F6BF} static const GUID NV_ENC_CODEC_H264_GUID = { 0x6bc82762, 0x4e63, 0x4ca4, { 0xaa, 0x85, 0x1e, 0x50, 0xf3, 0x21, 0xf6, 0xbf } }; // {790CDC88-4522-4d7b-9425-BDA9975F7603} static const GUID NV_ENC_CODEC_HEVC_GUID = { 0x790cdc88, 0x4522, 0x4d7b, { 0x94, 0x25, 0xbd, 0xa9, 0x97, 0x5f, 0x76, 0x3 } }; // {0A352289-0AA7-4759-862D-5D15CD16D254} static const GUID NV_ENC_CODEC_AV1_GUID = { 0x0a352289, 0x0aa7, 0x4759, { 0x86, 0x2d, 0x5d, 0x15, 0xcd, 0x16, 0xd2, 0x54 } }; // ========================================================================================= // * Encode Profile GUIDS supported by the NvEncodeAPI interface. // ========================================================================================= // {BFD6F8E7-233C-4341-8B3E-4818523803F4} static const GUID NV_ENC_CODEC_PROFILE_AUTOSELECT_GUID = { 0xbfd6f8e7, 0x233c, 0x4341, { 0x8b, 0x3e, 0x48, 0x18, 0x52, 0x38, 0x3, 0xf4 } }; // {0727BCAA-78C4-4c83-8C2F-EF3DFF267C6A} static const GUID NV_ENC_H264_PROFILE_BASELINE_GUID = { 0x727bcaa, 0x78c4, 0x4c83, { 0x8c, 0x2f, 0xef, 0x3d, 0xff, 0x26, 0x7c, 0x6a } }; // {60B5C1D4-67FE-4790-94D5-C4726D7B6E6D} static const GUID NV_ENC_H264_PROFILE_MAIN_GUID = { 0x60b5c1d4, 0x67fe, 0x4790, { 0x94, 0xd5, 0xc4, 0x72, 0x6d, 0x7b, 0x6e, 0x6d } }; // {E7CBC309-4F7A-4b89-AF2A-D537C92BE310} static const GUID NV_ENC_H264_PROFILE_HIGH_GUID = { 0xe7cbc309, 0x4f7a, 0x4b89, { 0xaf, 0x2a, 0xd5, 0x37, 0xc9, 0x2b, 0xe3, 0x10 } }; // {7AC663CB-A598-4960-B844-339B261A7D52} static const GUID NV_ENC_H264_PROFILE_HIGH_444_GUID = { 0x7ac663cb, 0xa598, 0x4960, { 0xb8, 0x44, 0x33, 0x9b, 0x26, 0x1a, 0x7d, 0x52 } }; // {40847BF5-33F7-4601-9084-E8FE3C1DB8B7} static const GUID NV_ENC_H264_PROFILE_STEREO_GUID = { 0x40847bf5, 0x33f7, 0x4601, { 0x90, 0x84, 0xe8, 0xfe, 0x3c, 0x1d, 0xb8, 0xb7 } }; // {B405AFAC-F32B-417B-89C4-9ABEED3E5978} static const GUID NV_ENC_H264_PROFILE_PROGRESSIVE_HIGH_GUID = { 0xb405afac, 0xf32b, 0x417b, { 0x89, 0xc4, 0x9a, 0xbe, 0xed, 0x3e, 0x59, 0x78 } }; // {AEC1BD87-E85B-48f2-84C3-98BCA6285072} static const GUID NV_ENC_H264_PROFILE_CONSTRAINED_HIGH_GUID = { 0xaec1bd87, 0xe85b, 0x48f2, { 0x84, 0xc3, 0x98, 0xbc, 0xa6, 0x28, 0x50, 0x72 } }; // {B514C39A-B55B-40fa-878F-F1253B4DFDEC} static const GUID NV_ENC_HEVC_PROFILE_MAIN_GUID = { 0xb514c39a, 0xb55b, 0x40fa, { 0x87, 0x8f, 0xf1, 0x25, 0x3b, 0x4d, 0xfd, 0xec } }; // {fa4d2b6c-3a5b-411a-8018-0a3f5e3c9be5} static const GUID NV_ENC_HEVC_PROFILE_MAIN10_GUID = { 0xfa4d2b6c, 0x3a5b, 0x411a, { 0x80, 0x18, 0x0a, 0x3f, 0x5e, 0x3c, 0x9b, 0xe5 } }; // For HEVC Main 444 8 bit and HEVC Main 444 10 bit profiles only // {51ec32b5-1b4c-453c-9cbd-b616bd621341} static const GUID NV_ENC_HEVC_PROFILE_FREXT_GUID = { 0x51ec32b5, 0x1b4c, 0x453c, { 0x9c, 0xbd, 0xb6, 0x16, 0xbd, 0x62, 0x13, 0x41 } }; // {5f2a39f5-f14e-4f95-9a9e-b76d568fcf97} static const GUID NV_ENC_AV1_PROFILE_MAIN_GUID = { 0x5f2a39f5, 0xf14e, 0x4f95, { 0x9a, 0x9e, 0xb7, 0x6d, 0x56, 0x8f, 0xcf, 0x97 } }; // ========================================================================================= // * Preset GUIDS supported by the NvEncodeAPI interface. // ========================================================================================= // {B2DFB705-4EBD-4C49-9B5F-24A777D3E587} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_DEFAULT_GUID = { 0xb2dfb705, 0x4ebd, 0x4c49, { 0x9b, 0x5f, 0x24, 0xa7, 0x77, 0xd3, 0xe5, 0x87 } }; // {60E4C59F-E846-4484-A56D-CD45BE9FDDF6} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_HP_GUID = { 0x60e4c59f, 0xe846, 0x4484, { 0xa5, 0x6d, 0xcd, 0x45, 0xbe, 0x9f, 0xdd, 0xf6 } }; // {34DBA71D-A77B-4B8F-9C3E-B6D5DA24C012} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_HQ_GUID = { 0x34dba71d, 0xa77b, 0x4b8f, { 0x9c, 0x3e, 0xb6, 0xd5, 0xda, 0x24, 0xc0, 0x12 } }; // {82E3E450-BDBB-4e40-989C-82A90DF9EF32} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_BD_GUID = { 0x82e3e450, 0xbdbb, 0x4e40, { 0x98, 0x9c, 0x82, 0xa9, 0xd, 0xf9, 0xef, 0x32 } }; // {49DF21C5-6DFA-4feb-9787-6ACC9EFFB726} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_LOW_LATENCY_DEFAULT_GUID = { 0x49df21c5, 0x6dfa, 0x4feb, { 0x97, 0x87, 0x6a, 0xcc, 0x9e, 0xff, 0xb7, 0x26 } }; // {C5F733B9-EA97-4cf9-BEC2-BF78A74FD105} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_LOW_LATENCY_HQ_GUID = { 0xc5f733b9, 0xea97, 0x4cf9, { 0xbe, 0xc2, 0xbf, 0x78, 0xa7, 0x4f, 0xd1, 0x5 } }; // {67082A44-4BAD-48FA-98EA-93056D150A58} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_LOW_LATENCY_HP_GUID = { 0x67082a44, 0x4bad, 0x48fa, { 0x98, 0xea, 0x93, 0x5, 0x6d, 0x15, 0xa, 0x58 } }; // {D5BFB716-C604-44e7-9BB8-DEA5510FC3AC} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_LOSSLESS_DEFAULT_GUID = { 0xd5bfb716, 0xc604, 0x44e7, { 0x9b, 0xb8, 0xde, 0xa5, 0x51, 0xf, 0xc3, 0xac } }; // {149998E7-2364-411d-82EF-179888093409} NV_ENC_DEPRECATED static const GUID NV_ENC_PRESET_LOSSLESS_HP_GUID = { 0x149998e7, 0x2364, 0x411d, { 0x82, 0xef, 0x17, 0x98, 0x88, 0x9, 0x34, 0x9 } }; // Performance degrades and quality improves as we move from P1 to P7. Presets P3 to P7 for H264 and // Presets P2 to P7 for HEVC have B frames enabled by default for HIGH_QUALITY and LOSSLESS tuning // info, and will not work with Weighted Prediction enabled. In case Weighted Prediction is // required, disable B frames by setting frameIntervalP = 1 {FC0A8D3E-45F8-4CF8-80C7-298871590EBF} static const GUID NV_ENC_PRESET_P1_GUID = { 0xfc0a8d3e, 0x45f8, 0x4cf8, { 0x80, 0xc7, 0x29, 0x88, 0x71, 0x59, 0xe, 0xbf } }; // {F581CFB8-88D6-4381-93F0-DF13F9C27DAB} static const GUID NV_ENC_PRESET_P2_GUID = { 0xf581cfb8, 0x88d6, 0x4381, { 0x93, 0xf0, 0xdf, 0x13, 0xf9, 0xc2, 0x7d, 0xab } }; // {36850110-3A07-441F-94D5-3670631F91F6} static const GUID NV_ENC_PRESET_P3_GUID = { 0x36850110, 0x3a07, 0x441f, { 0x94, 0xd5, 0x36, 0x70, 0x63, 0x1f, 0x91, 0xf6 } }; // {90A7B826-DF06-4862-B9D2-CD6D73A08681} static const GUID NV_ENC_PRESET_P4_GUID = { 0x90a7b826, 0xdf06, 0x4862, { 0xb9, 0xd2, 0xcd, 0x6d, 0x73, 0xa0, 0x86, 0x81 } }; // {21C6E6B4-297A-4CBA-998F-B6CBDE72ADE3} static const GUID NV_ENC_PRESET_P5_GUID = { 0x21c6e6b4, 0x297a, 0x4cba, { 0x99, 0x8f, 0xb6, 0xcb, 0xde, 0x72, 0xad, 0xe3 } }; // {8E75C279-6299-4AB6-8302-0B215A335CF5} static const GUID NV_ENC_PRESET_P6_GUID = { 0x8e75c279, 0x6299, 0x4ab6, { 0x83, 0x2, 0xb, 0x21, 0x5a, 0x33, 0x5c, 0xf5 } }; // {84848C12-6F71-4C13-931B-53E283F57974} static const GUID NV_ENC_PRESET_P7_GUID = { 0x84848c12, 0x6f71, 0x4c13, { 0x93, 0x1b, 0x53, 0xe2, 0x83, 0xf5, 0x79, 0x74 } }; /** * \addtogroup ENCODER_STRUCTURE NvEncodeAPI Data structures * @{ */ /** * Input frame encode modes */ typedef enum _NV_ENC_PARAMS_FRAME_FIELD_MODE { NV_ENC_PARAMS_FRAME_FIELD_MODE_FRAME = 0x01, /**< Frame mode */ NV_ENC_PARAMS_FRAME_FIELD_MODE_FIELD = 0x02, /**< Field mode */ NV_ENC_PARAMS_FRAME_FIELD_MODE_MBAFF = 0x03 /**< MB adaptive frame/field */ } NV_ENC_PARAMS_FRAME_FIELD_MODE; /** * Rate Control Modes */ typedef enum _NV_ENC_PARAMS_RC_MODE { NV_ENC_PARAMS_RC_CONSTQP = 0x0, /**< Constant QP mode */ NV_ENC_PARAMS_RC_VBR = 0x1, /**< Variable bitrate mode */ NV_ENC_PARAMS_RC_CBR = 0x2, /**< Constant bitrate mode */ NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ = 0x8, /**< Deprecated, use NV_ENC_PARAMS_RC_CBR + NV_ENC_TWO_PASS_QUARTER_RESOLUTION / NV_ENC_TWO_PASS_FULL_RESOLUTION + lowDelayKeyFrameScale=1 */ NV_ENC_PARAMS_RC_CBR_HQ = 0x10, /**< Deprecated, use NV_ENC_PARAMS_RC_CBR + NV_ENC_TWO_PASS_QUARTER_RESOLUTION / NV_ENC_TWO_PASS_FULL_RESOLUTION */ NV_ENC_PARAMS_RC_VBR_HQ = 0x20 /**< Deprecated, use NV_ENC_PARAMS_RC_VBR + NV_ENC_TWO_PASS_QUARTER_RESOLUTION / NV_ENC_TWO_PASS_FULL_RESOLUTION */ } NV_ENC_PARAMS_RC_MODE; /** * Multi Pass encoding */ typedef enum _NV_ENC_MULTI_PASS { NV_ENC_MULTI_PASS_DISABLED = 0x0, /**< Single Pass */ NV_ENC_TWO_PASS_QUARTER_RESOLUTION = 0x1, /**< Two Pass encoding is enabled where first Pass is quarter resolution */ NV_ENC_TWO_PASS_FULL_RESOLUTION = 0x2, /**< Two Pass encoding is enabled where first Pass is full resolution */ } NV_ENC_MULTI_PASS; /** * Emphasis Levels */ typedef enum _NV_ENC_EMPHASIS_MAP_LEVEL { NV_ENC_EMPHASIS_MAP_LEVEL_0 = 0x0, /**< Emphasis Map Level 0, for zero Delta QP value */ NV_ENC_EMPHASIS_MAP_LEVEL_1 = 0x1, /**< Emphasis Map Level 1, for very low Delta QP value */ NV_ENC_EMPHASIS_MAP_LEVEL_2 = 0x2, /**< Emphasis Map Level 2, for low Delta QP value */ NV_ENC_EMPHASIS_MAP_LEVEL_3 = 0x3, /**< Emphasis Map Level 3, for medium Delta QP value */ NV_ENC_EMPHASIS_MAP_LEVEL_4 = 0x4, /**< Emphasis Map Level 4, for high Delta QP value */ NV_ENC_EMPHASIS_MAP_LEVEL_5 = 0x5 /**< Emphasis Map Level 5, for very high Delta QP value */ } NV_ENC_EMPHASIS_MAP_LEVEL; /** * QP MAP MODE */ typedef enum _NV_ENC_QP_MAP_MODE { NV_ENC_QP_MAP_DISABLED = 0x0, /**< Value in NV_ENC_PIC_PARAMS::qpDeltaMap have no effect. */ NV_ENC_QP_MAP_EMPHASIS = 0x1, /**< Value in NV_ENC_PIC_PARAMS::qpDeltaMap will be treated as Emphasis level. Currently this is only supported for H264 */ NV_ENC_QP_MAP_DELTA = 0x2, /**< Value in NV_ENC_PIC_PARAMS::qpDeltaMap will be treated as QP delta map. */ NV_ENC_QP_MAP = 0x3, /**< Currently This is not supported. Value in NV_ENC_PIC_PARAMS::qpDeltaMap will be treated as QP value. */ } NV_ENC_QP_MAP_MODE; #define NV_ENC_PARAMS_RC_VBR_MINQP (NV_ENC_PARAMS_RC_MODE)0x4 /**< Deprecated */ #define NV_ENC_PARAMS_RC_2_PASS_QUALITY NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ /**< Deprecated */ #define NV_ENC_PARAMS_RC_2_PASS_FRAMESIZE_CAP NV_ENC_PARAMS_RC_CBR_HQ /**< Deprecated */ #define NV_ENC_PARAMS_RC_2_PASS_VBR NV_ENC_PARAMS_RC_VBR_HQ /**< Deprecated */ #define NV_ENC_PARAMS_RC_CBR2 NV_ENC_PARAMS_RC_CBR /**< Deprecated */ /** * Input picture structure */ typedef enum _NV_ENC_PIC_STRUCT { NV_ENC_PIC_STRUCT_FRAME = 0x01, /**< Progressive frame */ NV_ENC_PIC_STRUCT_FIELD_TOP_BOTTOM = 0x02, /**< Field encoding top field first */ NV_ENC_PIC_STRUCT_FIELD_BOTTOM_TOP = 0x03 /**< Field encoding bottom field first */ } NV_ENC_PIC_STRUCT; /** * Display picture structure * Currently, this enum is only used for deciding the number of clock timestamp sets in Picture * Timing SEI / Time Code SEI Otherwise, this has no impact on encoder behavior */ typedef enum _NV_ENC_DISPLAY_PIC_STRUCT { NV_ENC_PIC_STRUCT_DISPLAY_FRAME = 0x00, /**< Field encoding top field first */ NV_ENC_PIC_STRUCT_DISPLAY_FIELD_TOP_BOTTOM = 0x01, /**< Field encoding top field first */ NV_ENC_PIC_STRUCT_DISPLAY_FIELD_BOTTOM_TOP = 0x02, /**< Field encoding bottom field first */ NV_ENC_PIC_STRUCT_DISPLAY_FRAME_DOUBLING = 0x03, /**< Frame doubling */ NV_ENC_PIC_STRUCT_DISPLAY_FRAME_TRIPLING = 0x04 /**< Field tripling */ } NV_ENC_DISPLAY_PIC_STRUCT; /** * Input picture type */ typedef enum _NV_ENC_PIC_TYPE { NV_ENC_PIC_TYPE_P = 0x0, /**< Forward predicted */ NV_ENC_PIC_TYPE_B = 0x01, /**< Bi-directionally predicted picture */ NV_ENC_PIC_TYPE_I = 0x02, /**< Intra predicted picture */ NV_ENC_PIC_TYPE_IDR = 0x03, /**< IDR picture */ NV_ENC_PIC_TYPE_BI = 0x04, /**< Bi-directionally predicted with only Intra MBs */ NV_ENC_PIC_TYPE_SKIPPED = 0x05, /**< Picture is skipped */ NV_ENC_PIC_TYPE_INTRA_REFRESH = 0x06, /**< First picture in intra refresh cycle */ NV_ENC_PIC_TYPE_NONREF_P = 0x07, /**< Non reference P picture */ NV_ENC_PIC_TYPE_UNKNOWN = 0xFF /**< Picture type unknown */ } NV_ENC_PIC_TYPE; /** * Motion vector precisions */ typedef enum _NV_ENC_MV_PRECISION { NV_ENC_MV_PRECISION_DEFAULT = 0x0, /**< Driver selects Quarter-Pel motion vector precision by default */ NV_ENC_MV_PRECISION_FULL_PEL = 0x01, /**< Full-Pel motion vector precision */ NV_ENC_MV_PRECISION_HALF_PEL = 0x02, /**< Half-Pel motion vector precision */ NV_ENC_MV_PRECISION_QUARTER_PEL = 0x03 /**< Quarter-Pel motion vector precision */ } NV_ENC_MV_PRECISION; /** * Input buffer formats */ typedef enum _NV_ENC_BUFFER_FORMAT { NV_ENC_BUFFER_FORMAT_UNDEFINED = 0x00000000, /**< Undefined buffer format */ NV_ENC_BUFFER_FORMAT_NV12 = 0x00000001, /**< Semi-Planar YUV [Y plane followed by interleaved UV plane] */ NV_ENC_BUFFER_FORMAT_YV12 = 0x00000010, /**< Planar YUV [Y plane followed by V and U planes] */ NV_ENC_BUFFER_FORMAT_IYUV = 0x00000100, /**< Planar YUV [Y plane followed by U and V planes] */ NV_ENC_BUFFER_FORMAT_YUV444 = 0x00001000, /**< Planar YUV [Y plane followed by U and V planes] */ NV_ENC_BUFFER_FORMAT_YUV420_10BIT = 0x00010000, /**< 10 bit Semi-Planar YUV [Y plane followed by interleaved UV plane]. Each pixel of size 2 bytes. Most Significant 10 bits contain pixel data. */ NV_ENC_BUFFER_FORMAT_YUV444_10BIT = 0x00100000, /**< 10 bit Planar YUV444 [Y plane followed by U and V planes]. Each pixel of size 2 bytes. Most Significant 10 bits contain pixel data. */ NV_ENC_BUFFER_FORMAT_ARGB = 0x01000000, /**< 8 bit Packed A8R8G8B8. This is a word-ordered format where a pixel is represented by a 32-bit word with B in the lowest 8 bits, G in the next 8 bits, R in the 8 bits after that and A in the highest 8 bits. */ NV_ENC_BUFFER_FORMAT_ARGB10 = 0x02000000, /**< 10 bit Packed A2R10G10B10. This is a word-ordered format where a pixel is represented by a 32-bit word with B in the lowest 10 bits, G in the next 10 bits, R in the 10 bits after that and A in the highest 2 bits. */ NV_ENC_BUFFER_FORMAT_AYUV = 0x04000000, /**< 8 bit Packed A8Y8U8V8. This is a word-ordered format where a pixel is represented by a 32-bit word with V in the lowest 8 bits, U in the next 8 bits, Y in the 8 bits after that and A in the highest 8 bits. */ NV_ENC_BUFFER_FORMAT_ABGR = 0x10000000, /**< 8 bit Packed A8B8G8R8. This is a word-ordered format where a pixel is represented by a 32-bit word with R in the lowest 8 bits, G in the next 8 bits, B in the 8 bits after that and A in the highest 8 bits. */ NV_ENC_BUFFER_FORMAT_ABGR10 = 0x20000000, /**< 10 bit Packed A2B10G10R10. This is a word-ordered format where a pixel is represented by a 32-bit word with R in the lowest 10 bits, G in the next 10 bits, B in the 10 bits after that and A in the highest 2 bits. */ NV_ENC_BUFFER_FORMAT_U8 = 0x40000000, /**< Buffer format representing one-dimensional buffer. This format should be used only when registering the resource as output buffer, which will be used to write the encoded bit stream or H.264 ME only mode output. */ } NV_ENC_BUFFER_FORMAT; #define NV_ENC_BUFFER_FORMAT_NV12_PL NV_ENC_BUFFER_FORMAT_NV12 #define NV_ENC_BUFFER_FORMAT_YV12_PL NV_ENC_BUFFER_FORMAT_YV12 #define NV_ENC_BUFFER_FORMAT_IYUV_PL NV_ENC_BUFFER_FORMAT_IYUV #define NV_ENC_BUFFER_FORMAT_YUV444_PL NV_ENC_BUFFER_FORMAT_YUV444 /** * Encoding levels */ typedef enum _NV_ENC_LEVEL { NV_ENC_LEVEL_AUTOSELECT = 0, NV_ENC_LEVEL_H264_1 = 10, NV_ENC_LEVEL_H264_1b = 9, NV_ENC_LEVEL_H264_11 = 11, NV_ENC_LEVEL_H264_12 = 12, NV_ENC_LEVEL_H264_13 = 13, NV_ENC_LEVEL_H264_2 = 20, NV_ENC_LEVEL_H264_21 = 21, NV_ENC_LEVEL_H264_22 = 22, NV_ENC_LEVEL_H264_3 = 30, NV_ENC_LEVEL_H264_31 = 31, NV_ENC_LEVEL_H264_32 = 32, NV_ENC_LEVEL_H264_4 = 40, NV_ENC_LEVEL_H264_41 = 41, NV_ENC_LEVEL_H264_42 = 42, NV_ENC_LEVEL_H264_5 = 50, NV_ENC_LEVEL_H264_51 = 51, NV_ENC_LEVEL_H264_52 = 52, NV_ENC_LEVEL_H264_60 = 60, NV_ENC_LEVEL_H264_61 = 61, NV_ENC_LEVEL_H264_62 = 62, NV_ENC_LEVEL_HEVC_1 = 30, NV_ENC_LEVEL_HEVC_2 = 60, NV_ENC_LEVEL_HEVC_21 = 63, NV_ENC_LEVEL_HEVC_3 = 90, NV_ENC_LEVEL_HEVC_31 = 93, NV_ENC_LEVEL_HEVC_4 = 120, NV_ENC_LEVEL_HEVC_41 = 123, NV_ENC_LEVEL_HEVC_5 = 150, NV_ENC_LEVEL_HEVC_51 = 153, NV_ENC_LEVEL_HEVC_52 = 156, NV_ENC_LEVEL_HEVC_6 = 180, NV_ENC_LEVEL_HEVC_61 = 183, NV_ENC_LEVEL_HEVC_62 = 186, NV_ENC_TIER_HEVC_MAIN = 0, NV_ENC_TIER_HEVC_HIGH = 1, NV_ENC_LEVEL_AV1_2 = 0, NV_ENC_LEVEL_AV1_21 = 1, NV_ENC_LEVEL_AV1_22 = 2, NV_ENC_LEVEL_AV1_23 = 3, NV_ENC_LEVEL_AV1_3 = 4, NV_ENC_LEVEL_AV1_31 = 5, NV_ENC_LEVEL_AV1_32 = 6, NV_ENC_LEVEL_AV1_33 = 7, NV_ENC_LEVEL_AV1_4 = 8, NV_ENC_LEVEL_AV1_41 = 9, NV_ENC_LEVEL_AV1_42 = 10, NV_ENC_LEVEL_AV1_43 = 11, NV_ENC_LEVEL_AV1_5 = 12, NV_ENC_LEVEL_AV1_51 = 13, NV_ENC_LEVEL_AV1_52 = 14, NV_ENC_LEVEL_AV1_53 = 15, NV_ENC_LEVEL_AV1_6 = 16, NV_ENC_LEVEL_AV1_61 = 17, NV_ENC_LEVEL_AV1_62 = 18, NV_ENC_LEVEL_AV1_63 = 19, NV_ENC_LEVEL_AV1_7 = 20, NV_ENC_LEVEL_AV1_71 = 21, NV_ENC_LEVEL_AV1_72 = 22, NV_ENC_LEVEL_AV1_73 = 23, NV_ENC_LEVEL_AV1_AUTOSELECT, NV_ENC_TIER_AV1_0 = 0, NV_ENC_TIER_AV1_1 = 1 } NV_ENC_LEVEL; /** * Error Codes */ typedef enum _NVENCSTATUS { /** * This indicates that API call returned with no errors. */ NV_ENC_SUCCESS, /** * This indicates that no encode capable devices were detected. */ NV_ENC_ERR_NO_ENCODE_DEVICE, /** * This indicates that devices pass by the client is not supported. */ NV_ENC_ERR_UNSUPPORTED_DEVICE, /** * This indicates that the encoder device supplied by the client is not * valid. */ NV_ENC_ERR_INVALID_ENCODERDEVICE, /** * This indicates that device passed to the API call is invalid. */ NV_ENC_ERR_INVALID_DEVICE, /** * This indicates that device passed to the API call is no longer available and * needs to be reinitialized. The clients need to destroy the current encoder * session by freeing the allocated input output buffers and destroying the device * and create a new encoding session. */ NV_ENC_ERR_DEVICE_NOT_EXIST, /** * This indicates that one or more of the pointers passed to the API call * is invalid. */ NV_ENC_ERR_INVALID_PTR, /** * This indicates that completion event passed in ::NvEncEncodePicture() call * is invalid. */ NV_ENC_ERR_INVALID_EVENT, /** * This indicates that one or more of the parameter passed to the API call * is invalid. */ NV_ENC_ERR_INVALID_PARAM, /** * This indicates that an API call was made in wrong sequence/order. */ NV_ENC_ERR_INVALID_CALL, /** * This indicates that the API call failed because it was unable to allocate * enough memory to perform the requested operation. */ NV_ENC_ERR_OUT_OF_MEMORY, /** * This indicates that the encoder has not been initialized with * ::NvEncInitializeEncoder() or that initialization has failed. * The client cannot allocate input or output buffers or do any encoding * related operation before successfully initializing the encoder. */ NV_ENC_ERR_ENCODER_NOT_INITIALIZED, /** * This indicates that an unsupported parameter was passed by the client. */ NV_ENC_ERR_UNSUPPORTED_PARAM, /** * This indicates that the ::NvEncLockBitstream() failed to lock the output * buffer. This happens when the client makes a non blocking lock call to * access the output bitstream by passing NV_ENC_LOCK_BITSTREAM::doNotWait flag. * This is not a fatal error and client should retry the same operation after * few milliseconds. */ NV_ENC_ERR_LOCK_BUSY, /** * This indicates that the size of the user buffer passed by the client is * insufficient for the requested operation. */ NV_ENC_ERR_NOT_ENOUGH_BUFFER, /** * This indicates that an invalid struct version was used by the client. */ NV_ENC_ERR_INVALID_VERSION, /** * This indicates that ::NvEncMapInputResource() API failed to map the client * provided input resource. */ NV_ENC_ERR_MAP_FAILED, /** * This indicates encode driver requires more input buffers to produce an output * bitstream. If this error is returned from ::NvEncEncodePicture() API, this * is not a fatal error. If the client is encoding with B frames then, * ::NvEncEncodePicture() API might be buffering the input frame for re-ordering. * * A client operating in synchronous mode cannot call ::NvEncLockBitstream() * API on the output bitstream buffer if ::NvEncEncodePicture() returned the * ::NV_ENC_ERR_NEED_MORE_INPUT error code. * The client must continue providing input frames until encode driver returns * ::NV_ENC_SUCCESS. After receiving ::NV_ENC_SUCCESS status the client can call * ::NvEncLockBitstream() API on the output buffers in the same order in which * it has called ::NvEncEncodePicture(). */ NV_ENC_ERR_NEED_MORE_INPUT, /** * This indicates that the HW encoder is busy encoding and is unable to encode * the input. The client should call ::NvEncEncodePicture() again after few * milliseconds. */ NV_ENC_ERR_ENCODER_BUSY, /** * This indicates that the completion event passed in ::NvEncEncodePicture() * API has not been registered with encoder driver using ::NvEncRegisterAsyncEvent(). */ NV_ENC_ERR_EVENT_NOT_REGISTERD, /** * This indicates that an unknown internal error has occurred. */ NV_ENC_ERR_GENERIC, /** * This indicates that the client is attempting to use a feature * that is not available for the license type for the current system. */ NV_ENC_ERR_INCOMPATIBLE_CLIENT_KEY, /** * This indicates that the client is attempting to use a feature * that is not implemented for the current version. */ NV_ENC_ERR_UNIMPLEMENTED, /** * This indicates that the ::NvEncRegisterResource API failed to register the resource. */ NV_ENC_ERR_RESOURCE_REGISTER_FAILED, /** * This indicates that the client is attempting to unregister a resource * that has not been successfully registered. */ NV_ENC_ERR_RESOURCE_NOT_REGISTERED, /** * This indicates that the client is attempting to unmap a resource * that has not been successfully mapped. */ NV_ENC_ERR_RESOURCE_NOT_MAPPED, } NVENCSTATUS; /** * Encode Picture encode flags. */ typedef enum _NV_ENC_PIC_FLAGS { NV_ENC_PIC_FLAG_FORCEINTRA = 0x1, /**< Encode the current picture as an Intra picture */ NV_ENC_PIC_FLAG_FORCEIDR = 0x2, /**< Encode the current picture as an IDR picture. This flag is only valid when Picture type decision is taken by the Encoder [_NV_ENC_INITIALIZE_PARAMS::enablePTD == 1]. */ NV_ENC_PIC_FLAG_OUTPUT_SPSPPS = 0x4, /**< Write the sequence and picture header in encoded bitstream of the current picture */ NV_ENC_PIC_FLAG_EOS = 0x8, /**< Indicates end of the input stream */ } NV_ENC_PIC_FLAGS; /** * Memory heap to allocate input and output buffers. */ typedef enum _NV_ENC_MEMORY_HEAP { NV_ENC_MEMORY_HEAP_AUTOSELECT = 0, /**< Memory heap to be decided by the encoder driver based on the usage */ NV_ENC_MEMORY_HEAP_VID = 1, /**< Memory heap is in local video memory */ NV_ENC_MEMORY_HEAP_SYSMEM_CACHED = 2, /**< Memory heap is in cached system memory */ NV_ENC_MEMORY_HEAP_SYSMEM_UNCACHED = 3 /**< Memory heap is in uncached system memory */ } NV_ENC_MEMORY_HEAP; /** * B-frame used as reference modes */ typedef enum _NV_ENC_BFRAME_REF_MODE { NV_ENC_BFRAME_REF_MODE_DISABLED = 0x0, /**< B frame is not used for reference */ NV_ENC_BFRAME_REF_MODE_EACH = 0x1, /**< Each B-frame will be used for reference */ NV_ENC_BFRAME_REF_MODE_MIDDLE = 0x2, /**< Only(Number of B-frame)/2 th B-frame will be used for reference */ } NV_ENC_BFRAME_REF_MODE; /** * H.264 entropy coding modes. */ typedef enum _NV_ENC_H264_ENTROPY_CODING_MODE { NV_ENC_H264_ENTROPY_CODING_MODE_AUTOSELECT = 0x0, /**< Entropy coding mode is auto selected by the encoder driver */ NV_ENC_H264_ENTROPY_CODING_MODE_CABAC = 0x1, /**< Entropy coding mode is CABAC */ NV_ENC_H264_ENTROPY_CODING_MODE_CAVLC = 0x2 /**< Entropy coding mode is CAVLC */ } NV_ENC_H264_ENTROPY_CODING_MODE; /** * H.264 specific BDirect modes */ typedef enum _NV_ENC_H264_BDIRECT_MODE { NV_ENC_H264_BDIRECT_MODE_AUTOSELECT = 0x0, /**< BDirect mode is auto selected by the encoder driver */ NV_ENC_H264_BDIRECT_MODE_DISABLE = 0x1, /**< Disable BDirect mode */ NV_ENC_H264_BDIRECT_MODE_TEMPORAL = 0x2, /**< Temporal BDirect mode */ NV_ENC_H264_BDIRECT_MODE_SPATIAL = 0x3 /**< Spatial BDirect mode */ } NV_ENC_H264_BDIRECT_MODE; /** * H.264 specific FMO usage */ typedef enum _NV_ENC_H264_FMO_MODE { NV_ENC_H264_FMO_AUTOSELECT = 0x0, /**< FMO usage is auto selected by the encoder driver */ NV_ENC_H264_FMO_ENABLE = 0x1, /**< Enable FMO */ NV_ENC_H264_FMO_DISABLE = 0x2, /**< Disable FMO */ } NV_ENC_H264_FMO_MODE; /** * H.264 specific Adaptive Transform modes */ typedef enum _NV_ENC_H264_ADAPTIVE_TRANSFORM_MODE { NV_ENC_H264_ADAPTIVE_TRANSFORM_AUTOSELECT = 0x0, /**< Adaptive Transform 8x8 mode is auto selected by the encoder driver*/ NV_ENC_H264_ADAPTIVE_TRANSFORM_DISABLE = 0x1, /**< Adaptive Transform 8x8 mode disabled */ NV_ENC_H264_ADAPTIVE_TRANSFORM_ENABLE = 0x2, /**< Adaptive Transform 8x8 mode should be used */ } NV_ENC_H264_ADAPTIVE_TRANSFORM_MODE; /** * Stereo frame packing modes. */ typedef enum _NV_ENC_STEREO_PACKING_MODE { NV_ENC_STEREO_PACKING_MODE_NONE = 0x0, /**< No Stereo packing required */ NV_ENC_STEREO_PACKING_MODE_CHECKERBOARD = 0x1, /**< Checkerboard mode for packing stereo frames */ NV_ENC_STEREO_PACKING_MODE_COLINTERLEAVE = 0x2, /**< Column Interleave mode for packing stereo frames */ NV_ENC_STEREO_PACKING_MODE_ROWINTERLEAVE = 0x3, /**< Row Interleave mode for packing stereo frames */ NV_ENC_STEREO_PACKING_MODE_SIDEBYSIDE = 0x4, /**< Side-by-side mode for packing stereo frames */ NV_ENC_STEREO_PACKING_MODE_TOPBOTTOM = 0x5, /**< Top-Bottom mode for packing stereo frames */ NV_ENC_STEREO_PACKING_MODE_FRAMESEQ = 0x6 /**< Frame Sequential mode for packing stereo frames */ } NV_ENC_STEREO_PACKING_MODE; /** * Input Resource type */ typedef enum _NV_ENC_INPUT_RESOURCE_TYPE { NV_ENC_INPUT_RESOURCE_TYPE_DIRECTX = 0x0, /**< input resource type is a directx9 surface*/ NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR = 0x1, /**< input resource type is a cuda device pointer surface*/ NV_ENC_INPUT_RESOURCE_TYPE_CUDAARRAY = 0x2, /**< input resource type is a cuda array surface. This array must be a 2D array and the CUDA_ARRAY3D_SURFACE_LDST flag must have been specified when creating it. */ NV_ENC_INPUT_RESOURCE_TYPE_OPENGL_TEX = 0x3 /**< input resource type is an OpenGL texture */ } NV_ENC_INPUT_RESOURCE_TYPE; /** * Buffer usage */ typedef enum _NV_ENC_BUFFER_USAGE { NV_ENC_INPUT_IMAGE = 0x0, /**< Registered surface will be used for input image */ NV_ENC_OUTPUT_MOTION_VECTOR = 0x1, /**< Registered surface will be used for output of H.264 ME only mode. This buffer usage type is not supported for HEVC ME only mode. */ NV_ENC_OUTPUT_BITSTREAM = 0x2, /**< Registered surface will be used for output bitstream in encoding */ } NV_ENC_BUFFER_USAGE; /** * Encoder Device type */ typedef enum _NV_ENC_DEVICE_TYPE { NV_ENC_DEVICE_TYPE_DIRECTX = 0x0, /**< encode device type is a directx9 device */ NV_ENC_DEVICE_TYPE_CUDA = 0x1, /**< encode device type is a cuda device */ NV_ENC_DEVICE_TYPE_OPENGL = 0x2 /**< encode device type is an OpenGL device. Use of this device type is supported only on Linux */ } NV_ENC_DEVICE_TYPE; /** * Number of reference frames */ typedef enum _NV_ENC_NUM_REF_FRAMES { NV_ENC_NUM_REF_FRAMES_AUTOSELECT = 0x0, /**< Number of reference frames is auto selected by the encoder driver */ NV_ENC_NUM_REF_FRAMES_1 = 0x1, /**< Number of reference frames equal to 1 */ NV_ENC_NUM_REF_FRAMES_2 = 0x2, /**< Number of reference frames equal to 2 */ NV_ENC_NUM_REF_FRAMES_3 = 0x3, /**< Number of reference frames equal to 3 */ NV_ENC_NUM_REF_FRAMES_4 = 0x4, /**< Number of reference frames equal to 4 */ NV_ENC_NUM_REF_FRAMES_5 = 0x5, /**< Number of reference frames equal to 5 */ NV_ENC_NUM_REF_FRAMES_6 = 0x6, /**< Number of reference frames equal to 6 */ NV_ENC_NUM_REF_FRAMES_7 = 0x7 /**< Number of reference frames equal to 7 */ } NV_ENC_NUM_REF_FRAMES; /** * Encoder capabilities enumeration. */ typedef enum _NV_ENC_CAPS { /** * Maximum number of B-Frames supported. */ NV_ENC_CAPS_NUM_MAX_BFRAMES, /** * Rate control modes supported. * \n The API return value is a bitmask of the values in NV_ENC_PARAMS_RC_MODE. */ NV_ENC_CAPS_SUPPORTED_RATECONTROL_MODES, /** * Indicates HW support for field mode encoding. * \n 0 : Interlaced mode encoding is not supported. * \n 1 : Interlaced field mode encoding is supported. * \n 2 : Interlaced frame encoding and field mode encoding are both supported. */ NV_ENC_CAPS_SUPPORT_FIELD_ENCODING, /** * Indicates HW support for monochrome mode encoding. * \n 0 : Monochrome mode not supported. * \n 1 : Monochrome mode supported. */ NV_ENC_CAPS_SUPPORT_MONOCHROME, /** * Indicates HW support for FMO. * \n 0 : FMO not supported. * \n 1 : FMO supported. */ NV_ENC_CAPS_SUPPORT_FMO, /** * Indicates HW capability for Quarter pel motion estimation. * \n 0 : Quarter-Pel Motion Estimation not supported. * \n 1 : Quarter-Pel Motion Estimation supported. */ NV_ENC_CAPS_SUPPORT_QPELMV, /** * H.264 specific. Indicates HW support for BDirect modes. * \n 0 : BDirect mode encoding not supported. * \n 1 : BDirect mode encoding supported. */ NV_ENC_CAPS_SUPPORT_BDIRECT_MODE, /** * H264 specific. Indicates HW support for CABAC entropy coding mode. * \n 0 : CABAC entropy coding not supported. * \n 1 : CABAC entropy coding supported. */ NV_ENC_CAPS_SUPPORT_CABAC, /** * Indicates HW support for Adaptive Transform. * \n 0 : Adaptive Transform not supported. * \n 1 : Adaptive Transform supported. */ NV_ENC_CAPS_SUPPORT_ADAPTIVE_TRANSFORM, /** * Indicates HW support for Multi View Coding. * \n 0 : Multi View Coding not supported. * \n 1 : Multi View Coding supported. */ NV_ENC_CAPS_SUPPORT_STEREO_MVC, /** * Indicates HW support for encoding Temporal layers. * \n 0 : Encoding Temporal layers not supported. * \n 1 : Encoding Temporal layers supported. */ NV_ENC_CAPS_NUM_MAX_TEMPORAL_LAYERS, /** * Indicates HW support for Hierarchical P frames. * \n 0 : Hierarchical P frames not supported. * \n 1 : Hierarchical P frames supported. */ NV_ENC_CAPS_SUPPORT_HIERARCHICAL_PFRAMES, /** * Indicates HW support for Hierarchical B frames. * \n 0 : Hierarchical B frames not supported. * \n 1 : Hierarchical B frames supported. */ NV_ENC_CAPS_SUPPORT_HIERARCHICAL_BFRAMES, /** * Maximum Encoding level supported (See ::NV_ENC_LEVEL for details). */ NV_ENC_CAPS_LEVEL_MAX, /** * Minimum Encoding level supported (See ::NV_ENC_LEVEL for details). */ NV_ENC_CAPS_LEVEL_MIN, /** * Indicates HW support for separate colour plane encoding. * \n 0 : Separate colour plane encoding not supported. * \n 1 : Separate colour plane encoding supported. */ NV_ENC_CAPS_SEPARATE_COLOUR_PLANE, /** * Maximum output width supported. */ NV_ENC_CAPS_WIDTH_MAX, /** * Maximum output height supported. */ NV_ENC_CAPS_HEIGHT_MAX, /** * Indicates Temporal Scalability Support. * \n 0 : Temporal SVC encoding not supported. * \n 1 : Temporal SVC encoding supported. */ NV_ENC_CAPS_SUPPORT_TEMPORAL_SVC, /** * Indicates Dynamic Encode Resolution Change Support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Dynamic Encode Resolution Change not supported. * \n 1 : Dynamic Encode Resolution Change supported. */ NV_ENC_CAPS_SUPPORT_DYN_RES_CHANGE, /** * Indicates Dynamic Encode Bitrate Change Support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Dynamic Encode bitrate change not supported. * \n 1 : Dynamic Encode bitrate change supported. */ NV_ENC_CAPS_SUPPORT_DYN_BITRATE_CHANGE, /** * Indicates Forcing Constant QP On The Fly Support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Forcing constant QP on the fly not supported. * \n 1 : Forcing constant QP on the fly supported. */ NV_ENC_CAPS_SUPPORT_DYN_FORCE_CONSTQP, /** * Indicates Dynamic rate control mode Change Support. * \n 0 : Dynamic rate control mode change not supported. * \n 1 : Dynamic rate control mode change supported. */ NV_ENC_CAPS_SUPPORT_DYN_RCMODE_CHANGE, /** * Indicates Subframe readback support for slice-based encoding. If this feature is supported, * it can be enabled by setting enableSubFrameWrite = 1. \n 0 : Subframe readback not supported. * \n 1 : Subframe readback supported. */ NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK, /** * Indicates Constrained Encoding mode support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Constrained encoding mode not supported. * \n 1 : Constrained encoding mode supported. * If this mode is supported client can enable this during initialization. * Client can then force a picture to be coded as constrained picture where * in-loop filtering is disabled across slice boundaries and prediction vectors for inter * macroblocks in each slice will be restricted to the slice region. */ NV_ENC_CAPS_SUPPORT_CONSTRAINED_ENCODING, /** * Indicates Intra Refresh Mode Support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Intra Refresh Mode not supported. * \n 1 : Intra Refresh Mode supported. */ NV_ENC_CAPS_SUPPORT_INTRA_REFRESH, /** * Indicates Custom VBV Buffer Size support. It can be used for capping frame size. * Support added from NvEncodeAPI version 2.0. * \n 0 : Custom VBV buffer size specification from client, not supported. * \n 1 : Custom VBV buffer size specification from client, supported. */ NV_ENC_CAPS_SUPPORT_CUSTOM_VBV_BUF_SIZE, /** * Indicates Dynamic Slice Mode Support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Dynamic Slice Mode not supported. * \n 1 : Dynamic Slice Mode supported. */ NV_ENC_CAPS_SUPPORT_DYNAMIC_SLICE_MODE, /** * Indicates Reference Picture Invalidation Support. * Support added from NvEncodeAPI version 2.0. * \n 0 : Reference Picture Invalidation not supported. * \n 1 : Reference Picture Invalidation supported. */ NV_ENC_CAPS_SUPPORT_REF_PIC_INVALIDATION, /** * Indicates support for Pre-Processing. * The API return value is a bitmask of the values defined in ::NV_ENC_PREPROC_FLAGS */ NV_ENC_CAPS_PREPROC_SUPPORT, /** * Indicates support Async mode. * \n 0 : Async Encode mode not supported. * \n 1 : Async Encode mode supported. */ NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT, /** * Maximum MBs per frame supported. */ NV_ENC_CAPS_MB_NUM_MAX, /** * Maximum aggregate throughput in MBs per sec. */ NV_ENC_CAPS_MB_PER_SEC_MAX, /** * Indicates HW support for YUV444 mode encoding. * \n 0 : YUV444 mode encoding not supported. * \n 1 : YUV444 mode encoding supported. */ NV_ENC_CAPS_SUPPORT_YUV444_ENCODE, /** * Indicates HW support for lossless encoding. * \n 0 : lossless encoding not supported. * \n 1 : lossless encoding supported. */ NV_ENC_CAPS_SUPPORT_LOSSLESS_ENCODE, /** * Indicates HW support for Sample Adaptive Offset. * \n 0 : SAO not supported. * \n 1 : SAO encoding supported. */ NV_ENC_CAPS_SUPPORT_SAO, /** * Indicates HW support for Motion Estimation Only Mode. * \n 0 : MEOnly Mode not supported. * \n 1 : MEOnly Mode supported for I and P frames. * \n 2 : MEOnly Mode supported for I, P and B frames. */ NV_ENC_CAPS_SUPPORT_MEONLY_MODE, /** * Indicates HW support for lookahead encoding (enableLookahead=1). * \n 0 : Lookahead not supported. * \n 1 : Lookahead supported. */ NV_ENC_CAPS_SUPPORT_LOOKAHEAD, /** * Indicates HW support for temporal AQ encoding (enableTemporalAQ=1). * \n 0 : Temporal AQ not supported. * \n 1 : Temporal AQ supported. */ NV_ENC_CAPS_SUPPORT_TEMPORAL_AQ, /** * Indicates HW support for 10 bit encoding. * \n 0 : 10 bit encoding not supported. * \n 1 : 10 bit encoding supported. */ NV_ENC_CAPS_SUPPORT_10BIT_ENCODE, /** * Maximum number of Long Term Reference frames supported */ NV_ENC_CAPS_NUM_MAX_LTR_FRAMES, /** * Indicates HW support for Weighted Prediction. * \n 0 : Weighted Prediction not supported. * \n 1 : Weighted Prediction supported. */ NV_ENC_CAPS_SUPPORT_WEIGHTED_PREDICTION, /** * On managed (vGPU) platforms (Windows only), this API, in conjunction with other GRID * Management APIs, can be used to estimate the residual capacity of the hardware encoder on the * GPU as a percentage of the total available encoder capacity. This API can be called at any * time; i.e. during the encode session or before opening the encode session. If the available * encoder capacity is returned as zero, applications may choose to switch to software encoding * and continue to call this API (e.g. polling once per second) until capacity becomes * available. * * On bare metal (non-virtualized GPU) and linux platforms, this API always returns 100. */ NV_ENC_CAPS_DYNAMIC_QUERY_ENCODER_CAPACITY, /** * Indicates B as reference support. * \n 0 : B as reference is not supported. * \n 1 : each B-Frame as reference is supported. * \n 2 : only Middle B-frame as reference is supported. */ NV_ENC_CAPS_SUPPORT_BFRAME_REF_MODE, /** * Indicates HW support for Emphasis Level Map based delta QP computation. * \n 0 : Emphasis Level Map based delta QP not supported. * \n 1 : Emphasis Level Map based delta QP is supported. */ NV_ENC_CAPS_SUPPORT_EMPHASIS_LEVEL_MAP, /** * Minimum input width supported. */ NV_ENC_CAPS_WIDTH_MIN, /** * Minimum input height supported. */ NV_ENC_CAPS_HEIGHT_MIN, /** * Indicates HW support for multiple reference frames. */ NV_ENC_CAPS_SUPPORT_MULTIPLE_REF_FRAMES, /** * Indicates HW support for HEVC with alpha encoding. * \n 0 : HEVC with alpha encoding not supported. * \n 1 : HEVC with alpha encoding is supported. */ NV_ENC_CAPS_SUPPORT_ALPHA_LAYER_ENCODING, /** * Indicates number of Encoding engines present on GPU. */ NV_ENC_CAPS_NUM_ENCODER_ENGINES, /** * Indicates single slice intra refresh support. */ NV_ENC_CAPS_SINGLE_SLICE_INTRA_REFRESH, /** * Reserved - Not to be used by clients. */ NV_ENC_CAPS_EXPOSED_COUNT } NV_ENC_CAPS; /** * HEVC CU SIZE */ typedef enum _NV_ENC_HEVC_CUSIZE { NV_ENC_HEVC_CUSIZE_AUTOSELECT = 0, NV_ENC_HEVC_CUSIZE_8x8 = 1, NV_ENC_HEVC_CUSIZE_16x16 = 2, NV_ENC_HEVC_CUSIZE_32x32 = 3, NV_ENC_HEVC_CUSIZE_64x64 = 4, } NV_ENC_HEVC_CUSIZE; /** * AV1 PART SIZE */ typedef enum _NV_ENC_AV1_PART_SIZE { NV_ENC_AV1_PART_SIZE_AUTOSELECT = 0, NV_ENC_AV1_PART_SIZE_4x4 = 1, NV_ENC_AV1_PART_SIZE_8x8 = 2, NV_ENC_AV1_PART_SIZE_16x16 = 3, NV_ENC_AV1_PART_SIZE_32x32 = 4, NV_ENC_AV1_PART_SIZE_64x64 = 5, } NV_ENC_AV1_PART_SIZE; /** * Enums related to fields in VUI parameters. */ typedef enum _NV_ENC_VUI_VIDEO_FORMAT { NV_ENC_VUI_VIDEO_FORMAT_COMPONENT = 0, NV_ENC_VUI_VIDEO_FORMAT_PAL = 1, NV_ENC_VUI_VIDEO_FORMAT_NTSC = 2, NV_ENC_VUI_VIDEO_FORMAT_SECAM = 3, NV_ENC_VUI_VIDEO_FORMAT_MAC = 4, NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED = 5, } NV_ENC_VUI_VIDEO_FORMAT; typedef enum _NV_ENC_VUI_COLOR_PRIMARIES { NV_ENC_VUI_COLOR_PRIMARIES_UNDEFINED = 0, NV_ENC_VUI_COLOR_PRIMARIES_BT709 = 1, NV_ENC_VUI_COLOR_PRIMARIES_UNSPECIFIED = 2, NV_ENC_VUI_COLOR_PRIMARIES_RESERVED = 3, NV_ENC_VUI_COLOR_PRIMARIES_BT470M = 4, NV_ENC_VUI_COLOR_PRIMARIES_BT470BG = 5, NV_ENC_VUI_COLOR_PRIMARIES_SMPTE170M = 6, NV_ENC_VUI_COLOR_PRIMARIES_SMPTE240M = 7, NV_ENC_VUI_COLOR_PRIMARIES_FILM = 8, NV_ENC_VUI_COLOR_PRIMARIES_BT2020 = 9, NV_ENC_VUI_COLOR_PRIMARIES_SMPTE428 = 10, NV_ENC_VUI_COLOR_PRIMARIES_SMPTE431 = 11, NV_ENC_VUI_COLOR_PRIMARIES_SMPTE432 = 12, NV_ENC_VUI_COLOR_PRIMARIES_JEDEC_P22 = 22, } NV_ENC_VUI_COLOR_PRIMARIES; typedef enum _NV_ENC_VUI_TRANSFER_CHARACTERISTIC { NV_ENC_VUI_TRANSFER_CHARACTERISTIC_UNDEFINED = 0, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT709 = 1, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_UNSPECIFIED = 2, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_RESERVED = 3, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT470M = 4, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT470BG = 5, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE170M = 6, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE240M = 7, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_LINEAR = 8, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_LOG = 9, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_LOG_SQRT = 10, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_IEC61966_2_4 = 11, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT1361_ECG = 12, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB = 13, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT2020_10 = 14, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT2020_12 = 15, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE2084 = 16, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SMPTE428 = 17, NV_ENC_VUI_TRANSFER_CHARACTERISTIC_ARIB_STD_B67 = 18, } NV_ENC_VUI_TRANSFER_CHARACTERISTIC; typedef enum _NV_ENC_VUI_MATRIX_COEFFS { NV_ENC_VUI_MATRIX_COEFFS_RGB = 0, NV_ENC_VUI_MATRIX_COEFFS_BT709 = 1, NV_ENC_VUI_MATRIX_COEFFS_UNSPECIFIED = 2, NV_ENC_VUI_MATRIX_COEFFS_RESERVED = 3, NV_ENC_VUI_MATRIX_COEFFS_FCC = 4, NV_ENC_VUI_MATRIX_COEFFS_BT470BG = 5, NV_ENC_VUI_MATRIX_COEFFS_SMPTE170M = 6, NV_ENC_VUI_MATRIX_COEFFS_SMPTE240M = 7, NV_ENC_VUI_MATRIX_COEFFS_YCGCO = 8, NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL = 9, NV_ENC_VUI_MATRIX_COEFFS_BT2020_CL = 10, NV_ENC_VUI_MATRIX_COEFFS_SMPTE2085 = 11, } NV_ENC_VUI_MATRIX_COEFFS; /** * Input struct for querying Encoding capabilities. */ typedef struct _NV_ENC_CAPS_PARAM { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_CAPS_PARAM_VER */ NV_ENC_CAPS capsToQuery; /**< [in]: Specifies the encode capability to be queried. Client should pass a member for ::NV_ENC_CAPS enum. */ uint32_t reserved[62]; /**< [in]: Reserved and must be set to 0 */ } NV_ENC_CAPS_PARAM; /** NV_ENC_CAPS_PARAM struct version. */ #define NV_ENC_CAPS_PARAM_VER NVENCAPI_STRUCT_VERSION(1) /** * Encoder Output parameters */ typedef struct _NV_ENC_ENCODE_OUT_PARAMS { uint32_t version; /**< [out]: Struct version. */ uint32_t bitstreamSizeInBytes; /**< [out]: Encoded bitstream size in bytes */ uint32_t reserved[62]; /**< [out]: Reserved and must be set to 0 */ } NV_ENC_ENCODE_OUT_PARAMS; /** NV_ENC_ENCODE_OUT_PARAMS struct version. */ #define NV_ENC_ENCODE_OUT_PARAMS_VER NVENCAPI_STRUCT_VERSION(1) /** * Creation parameters for input buffer. */ typedef struct _NV_ENC_CREATE_INPUT_BUFFER { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_CREATE_INPUT_BUFFER_VER */ uint32_t width; /**< [in]: Input frame width */ uint32_t height; /**< [in]: Input frame height */ NV_ENC_MEMORY_HEAP memoryHeap; /**< [in]: Deprecated. Do not use */ NV_ENC_BUFFER_FORMAT bufferFmt; /**< [in]: Input buffer format */ uint32_t reserved; /**< [in]: Reserved and must be set to 0 */ NV_ENC_INPUT_PTR inputBuffer; /**< [out]: Pointer to input buffer */ void* pSysMemBuffer; /**< [in]: Pointer to existing system memory buffer */ uint32_t reserved1[57]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[63]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CREATE_INPUT_BUFFER; /** NV_ENC_CREATE_INPUT_BUFFER struct version. */ #define NV_ENC_CREATE_INPUT_BUFFER_VER NVENCAPI_STRUCT_VERSION(1) /** * Creation parameters for output bitstream buffer. */ typedef struct _NV_ENC_CREATE_BITSTREAM_BUFFER { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_CREATE_BITSTREAM_BUFFER_VER */ uint32_t size; /**< [in]: Deprecated. Do not use */ NV_ENC_MEMORY_HEAP memoryHeap; /**< [in]: Deprecated. Do not use */ uint32_t reserved; /**< [in]: Reserved and must be set to 0 */ NV_ENC_OUTPUT_PTR bitstreamBuffer; /**< [out]: Pointer to the output bitstream buffer */ void* bitstreamBufferPtr; /**< [out]: Reserved and should not be used */ uint32_t reserved1[58]; /**< [in]: Reserved and should be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and should be set to NULL */ } NV_ENC_CREATE_BITSTREAM_BUFFER; /** NV_ENC_CREATE_BITSTREAM_BUFFER struct version. */ #define NV_ENC_CREATE_BITSTREAM_BUFFER_VER NVENCAPI_STRUCT_VERSION(1) /** * Structs needed for ME only mode. */ typedef struct _NV_ENC_MVECTOR { int16_t mvx; /**< the x component of MV in quarter-pel units */ int16_t mvy; /**< the y component of MV in quarter-pel units */ } NV_ENC_MVECTOR; /** * Motion vector structure per macroblock for H264 motion estimation. */ typedef struct _NV_ENC_H264_MV_DATA { NV_ENC_MVECTOR mv[4]; /**< up to 4 vectors for 8x8 partition */ uint8_t mbType; /**< 0 (I), 1 (P), 2 (IPCM), 3 (B) */ uint8_t partitionType; /**< Specifies the block partition type. 0:16x16, 1:8x8, 2:16x8, 3:8x16 */ uint16_t reserved; /**< reserved padding for alignment */ uint32_t mbCost; } NV_ENC_H264_MV_DATA; /** * Motion vector structure per CU for HEVC motion estimation. */ typedef struct _NV_ENC_HEVC_MV_DATA { NV_ENC_MVECTOR mv[4]; /**< up to 4 vectors within a CU */ uint8_t cuType; /**< 0 (I), 1(P) */ uint8_t cuSize; /**< 0: 8x8, 1: 16x16, 2: 32x32, 3: 64x64 */ uint8_t partitionMode; /**< The CU partition mode 0 (2Nx2N), 1 (2NxN), 2(Nx2N), 3 (NxN), 4 (2NxnU), 5 (2NxnD), 6(nLx2N), 7 (nRx2N) */ uint8_t lastCUInCTB; /**< Marker to separate CUs in the current CTB from CUs in the next CTB */ } NV_ENC_HEVC_MV_DATA; /** * Creation parameters for output motion vector buffer for ME only mode. */ typedef struct _NV_ENC_CREATE_MV_BUFFER { uint32_t version; /**< [in]: Struct version. Must be set to NV_ENC_CREATE_MV_BUFFER_VER */ NV_ENC_OUTPUT_PTR mvBuffer; /**< [out]: Pointer to the output motion vector buffer */ uint32_t reserved1[255]; /**< [in]: Reserved and should be set to 0 */ void* reserved2[63]; /**< [in]: Reserved and should be set to NULL */ } NV_ENC_CREATE_MV_BUFFER; /** NV_ENC_CREATE_MV_BUFFER struct version*/ #define NV_ENC_CREATE_MV_BUFFER_VER NVENCAPI_STRUCT_VERSION(1) /** * QP value for frames */ typedef struct _NV_ENC_QP { uint32_t qpInterP; /**< [in]: Specifies QP value for P-frame. Even though this field is uint32_t for legacy reasons, the client should treat this as a signed parameter(int32_t) for cases in which negative QP values are to be specified. */ uint32_t qpInterB; /**< [in]: Specifies QP value for B-frame. Even though this field is uint32_t for legacy reasons, the client should treat this as a signed parameter(int32_t) for cases in which negative QP values are to be specified. */ uint32_t qpIntra; /**< [in]: Specifies QP value for Intra Frame. Even though this field is uint32_t for legacy reasons, the client should treat this as a signed parameter(int32_t) for cases in which negative QP values are to be specified. */ } NV_ENC_QP; /** * Rate Control Configuration Parameters */ typedef struct _NV_ENC_RC_PARAMS { uint32_t version; NV_ENC_PARAMS_RC_MODE rateControlMode; /**< [in]: Specifies the rate control mode. Check support for various rate control modes using ::NV_ENC_CAPS_SUPPORTED_RATECONTROL_MODES caps. */ NV_ENC_QP constQP; /**< [in]: Specifies the initial QP to be used for encoding, these values would be used for all frames if in Constant QP mode. */ uint32_t averageBitRate; /**< [in]: Specifies the average bitrate(in bits/sec) used for encoding. */ uint32_t maxBitRate; /**< [in]: Specifies the maximum bitrate for the encoded output. This is used for VBR and ignored for CBR mode. */ uint32_t vbvBufferSize; /**< [in]: Specifies the VBV(HRD) buffer size. in bits. Set 0 to use the default VBV buffer size. */ uint32_t vbvInitialDelay; /**< [in]: Specifies the VBV(HRD) initial delay in bits. Set 0 to use the default VBV initial delay .*/ uint32_t enableMinQP : 1; /**< [in]: Set this to 1 if minimum QP used for rate control. */ uint32_t enableMaxQP : 1; /**< [in]: Set this to 1 if maximum QP used for rate control. */ uint32_t enableInitialRCQP : 1; /**< [in]: Set this to 1 if user supplied initial QP is used for rate control. */ uint32_t enableAQ : 1; /**< [in]: Set this to 1 to enable adaptive quantization (Spatial). */ uint32_t reservedBitField1 : 1; /**< [in]: Reserved bitfields and must be set to 0. */ uint32_t enableLookahead : 1; /**< [in]: Set this to 1 to enable lookahead with depth (if lookahead is enabled, input frames must remain available to the encoder until encode completion) */ uint32_t disableIadapt : 1; /**< [in]: Set this to 1 to disable adaptive I-frame insertion at scene cuts (only has an effect when lookahead is enabled) */ uint32_t disableBadapt : 1; /**< [in]: Set this to 1 to disable adaptive B-frame decision (only has an effect when lookahead is enabled) */ uint32_t enableTemporalAQ : 1; /**< [in]: Set this to 1 to enable temporal AQ */ uint32_t zeroReorderDelay : 1; /**< [in]: Set this to 1 to indicate zero latency operation (no reordering delay, num_reorder_frames=0) */ uint32_t enableNonRefP : 1; /**< [in]: Set this to 1 to enable automatic insertion of non-reference P-frames (no effect if enablePTD=0) */ uint32_t strictGOPTarget : 1; /**< [in]: Set this to 1 to minimize GOP-to-GOP rate fluctuations */ uint32_t aqStrength : 4; /**< [in]: When AQ (Spatial) is enabled (i.e. NV_ENC_RC_PARAMS::enableAQ is set), this field is used to specify AQ strength. AQ strength scale is from 1 (low) - 15 (aggressive). If not set, strength is auto selected by driver. */ uint32_t reservedBitFields : 16; /**< [in]: Reserved bitfields and must be set to 0 */ NV_ENC_QP minQP; /**< [in]: Specifies the minimum QP used for rate control. Client must set NV_ENC_CONFIG::enableMinQP to 1. */ NV_ENC_QP maxQP; /**< [in]: Specifies the maximum QP used for rate control. Client must set NV_ENC_CONFIG::enableMaxQP to 1. */ NV_ENC_QP initialRCQP; /**< [in]: Specifies the initial QP used for rate control. Client must set NV_ENC_CONFIG::enableInitialRCQP to 1. */ uint32_t temporallayerIdxMask; /**< [in]: Specifies the temporal layers (as a bitmask) whose QPs have changed. Valid max bitmask is [2^NV_ENC_CAPS_NUM_MAX_TEMPORAL_LAYERS - 1]. Applicable only for constant QP mode (NV_ENC_RC_PARAMS::rateControlMode = NV_ENC_PARAMS_RC_CONSTQP). */ uint8_t temporalLayerQP[8]; /**< [in]: Specifies the temporal layer QPs used for rate control. Temporal layer index is used as the array index. Applicable only for constant QP mode (NV_ENC_RC_PARAMS::rateControlMode = NV_ENC_PARAMS_RC_CONSTQP). */ uint8_t targetQuality; /**< [in]: Target CQ (Constant Quality) level for VBR mode (range 0-51 with 0-automatic) */ uint8_t targetQualityLSB; /**< [in]: Fractional part of target quality (as 8.8 fixed point format) */ uint16_t lookaheadDepth; /**< [in]: Maximum depth of lookahead with range 0-(31 - number of B frames). lookaheadDepth is only used if enableLookahead=1.*/ uint8_t lowDelayKeyFrameScale; /**< [in]: Specifies the ratio of I frame bits to P frame bits in case of single frame VBV and CBR rate control mode, is set to 2 by default for low latency tuning info and 1 by default for ultra low latency tuning info */ int8_t yDcQPIndexOffset; /**< [in]: Specifies the value of 'deltaQ_y_dc' in AV1.*/ int8_t uDcQPIndexOffset; /**< [in]: Specifies the value of 'deltaQ_u_dc' in AV1.*/ int8_t vDcQPIndexOffset; /**< [in]: Specifies the value of 'deltaQ_v_dc' in AV1 (for future use only - deltaQ_v_dc is currently always internally set to same value as deltaQ_u_dc). */ NV_ENC_QP_MAP_MODE qpMapMode; /**< [in]: This flag is used to interpret values in array specified by NV_ENC_PIC_PARAMS::qpDeltaMap. Set this to NV_ENC_QP_MAP_EMPHASIS to treat values specified by NV_ENC_PIC_PARAMS::qpDeltaMap as Emphasis Level Map. Emphasis Level can be assigned any value specified in enum NV_ENC_EMPHASIS_MAP_LEVEL. Emphasis Level Map is used to specify regions to be encoded at varying levels of quality. The hardware encoder adjusts the quantization within the image as per the provided emphasis map, by adjusting the quantization parameter (QP) assigned to each macroblock. This adjustment is commonly called "Delta QP". The adjustment depends on the absolute QP decided by the rate control algorithm, and is applied after the rate control has decided each macroblock's QP. Since the Delta QP overrides rate control, enabling Emphasis Level Map may violate bitrate and VBV buffer size constraints. Emphasis Level Map is useful in situations where client has a priori knowledge of the image complexity (e.g. via use of NVFBC's Classification feature) and encoding those high-complexity areas at higher quality (lower QP) is important, even at the possible cost of violating bitrate/VBV buffer size constraints This feature is not supported when AQ( Spatial/Temporal) is enabled. This feature is only supported for H264 codec currently. Set this to NV_ENC_QP_MAP_DELTA to treat values specified by NV_ENC_PIC_PARAMS::qpDeltaMap as QP Delta. This specifies QP modifier to be applied on top of the QP chosen by rate control Set this to NV_ENC_QP_MAP_DISABLED to ignore NV_ENC_PIC_PARAMS::qpDeltaMap values. In this case, qpDeltaMap should be set to NULL. Other values are reserved for future use.*/ NV_ENC_MULTI_PASS multiPass; /**< [in]: This flag is used to enable multi-pass encoding for a given ::NV_ENC_PARAMS_RC_MODE. This flag is not valid for H264 and HEVC MEOnly mode */ uint32_t alphaLayerBitrateRatio; /**< [in]: Specifies the ratio in which bitrate should be split between base and alpha layer. A value 'x' for this field will split the target bitrate in a ratio of x : 1 between base and alpha layer. The default split ratio is 15.*/ int8_t cbQPIndexOffset; /**< [in]: Specifies the value of 'chroma_qp_index_offset' in H264 / 'pps_cb_qp_offset' in HEVC / 'deltaQ_u_ac' in AV1.*/ int8_t crQPIndexOffset; /**< [in]: Specifies the value of 'second_chroma_qp_index_offset' in H264 / 'pps_cr_qp_offset' in HEVC / 'deltaQ_v_ac' in AV1 (for future use only - deltaQ_v_ac is currently always internally set to same value as deltaQ_u_ac). */ uint16_t reserved2; uint32_t reserved[4]; } NV_ENC_RC_PARAMS; /** macro for constructing the version field of ::_NV_ENC_RC_PARAMS */ #define NV_ENC_RC_PARAMS_VER NVENCAPI_STRUCT_VERSION(1) #define MAX_NUM_CLOCK_TS 3 /** * Clock Timestamp set parameters * For H264, this structure is used to populate Picture Timing SEI when * NV_ENC_CONFIG_H264::enableTimeCode is set to 1. For HEVC, this structure is used to populate Time * Code SEI when NV_ENC_CONFIG_HEVC::enableTimeCodeSEI is set to 1. For more details, refer to Annex * D of ITU-T Specification. */ typedef struct _NV_ENC_CLOCK_TIMESTAMP_SET { uint32_t countingType : 1; /**< [in] Specifies the 'counting_type' */ uint32_t discontinuityFlag : 1; /**< [in] Specifies the 'discontinuity_flag' */ uint32_t cntDroppedFrames : 1; /**< [in] Specifies the 'cnt_dropped_flag' */ uint32_t nFrames : 8; /**< [in] Specifies the value of 'n_frames' */ uint32_t secondsValue : 6; /**< [in] Specifies the 'seconds_value' */ uint32_t minutesValue : 6; /**< [in] Specifies the 'minutes_value' */ uint32_t hoursValue : 5; /**< [in] Specifies the 'hours_value' */ uint32_t reserved2 : 4; /**< [in] Reserved and must be set to 0 */ uint32_t timeOffset; /**< [in] Specifies the 'time_offset_value' */ } NV_ENC_CLOCK_TIMESTAMP_SET; typedef struct _NV_ENC_TIME_CODE { NV_ENC_DISPLAY_PIC_STRUCT displayPicStruct; /**< [in] Display picStruct */ NV_ENC_CLOCK_TIMESTAMP_SET clockTimestamp[MAX_NUM_CLOCK_TS]; /**< [in] Clock Timestamp set */ } NV_ENC_TIME_CODE; /** * \struct _NV_ENC_CONFIG_H264_VUI_PARAMETERS * H264 Video Usability Info parameters */ typedef struct _NV_ENC_CONFIG_H264_VUI_PARAMETERS { uint32_t overscanInfoPresentFlag; /**< [in]: If set to 1 , it specifies that the overscanInfo is present */ uint32_t overscanInfo; /**< [in]: Specifies the overscan info(as defined in Annex E of the ITU-T Specification). */ uint32_t videoSignalTypePresentFlag; /**< [in]: If set to 1, it specifies that the videoFormat, videoFullRangeFlag and colourDescriptionPresentFlag are present. */ NV_ENC_VUI_VIDEO_FORMAT videoFormat; /**< [in]: Specifies the source video format(as defined in Annex E of the ITU-T Specification).*/ uint32_t videoFullRangeFlag; /**< [in]: Specifies the output range of the luma and chroma samples(as defined in Annex E of the ITU-T Specification). */ uint32_t colourDescriptionPresentFlag; /**< [in]: If set to 1, it specifies that the colourPrimaries, transferCharacteristics and colourMatrix are present. */ NV_ENC_VUI_COLOR_PRIMARIES colourPrimaries; /**< [in]: Specifies color primaries for converting to RGB(as defined in Annex E of the ITU-T Specification) */ NV_ENC_VUI_TRANSFER_CHARACTERISTIC transferCharacteristics; /**< [in]: Specifies the opto-electronic transfer characteristics to use (as defined in Annex E of the ITU-T Specification) */ NV_ENC_VUI_MATRIX_COEFFS colourMatrix; /**< [in]: Specifies the matrix coefficients used in deriving the luma and chroma from the RGB primaries (as defined in Annex E of the ITU-T Specification). */ uint32_t chromaSampleLocationFlag; /**< [in]: If set to 1 , it specifies that the chromaSampleLocationTop and chromaSampleLocationBot are present.*/ uint32_t chromaSampleLocationTop; /**< [in]: Specifies the chroma sample location for top field(as defined in Annex E of the ITU-T Specification) */ uint32_t chromaSampleLocationBot; /**< [in]: Specifies the chroma sample location for bottom field(as defined in Annex E of the ITU-T Specification) */ uint32_t bitstreamRestrictionFlag; /**< [in]: If set to 1, it specifies the bitstream restriction parameters are present in the bitstream.*/ uint32_t timingInfoPresentFlag; /**< [in]: If set to 1, it specifies that the timingInfo is present and the 'numUnitInTicks' and 'timeScale' fields are specified by the application. */ /**< [in]: If not set, the timingInfo may still be present with timing related fields calculated * internally basedon the frame rate specified by the application. */ uint32_t numUnitInTicks; /**< [in]: Specifies the number of time units of the clock(as defined in Annex E of the ITU-T Specification). */ uint32_t timeScale; /**< [in]: Specifies the frquency of the clock(as defined in Annex E of the ITU-T Specification). */ uint32_t reserved[12]; /**< [in]: Reserved and must be set to 0 */ } NV_ENC_CONFIG_H264_VUI_PARAMETERS; typedef NV_ENC_CONFIG_H264_VUI_PARAMETERS NV_ENC_CONFIG_HEVC_VUI_PARAMETERS; /** * \struct _NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE * External motion vector hint counts per block type. * H264 and AV1 support multiple hint while HEVC supports one hint for each valid candidate. */ typedef struct _NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE { uint32_t numCandsPerBlk16x16 : 4; /**< [in]: Supported for H264, HEVC. It Specifies the number of candidates per 16x16 block. */ uint32_t numCandsPerBlk16x8 : 4; /**< [in]: Supported for H264 only. Specifies the number of candidates per 16x8 block. */ uint32_t numCandsPerBlk8x16 : 4; /**< [in]: Supported for H264 only. Specifies the number of candidates per 8x16 block. */ uint32_t numCandsPerBlk8x8 : 4; /**< [in]: Supported for H264, HEVC. Specifies the number of candidates per 8x8 block. */ uint32_t numCandsPerSb : 8; /**< [in]: Supported for AV1 only. Specifies the number of candidates per SB. */ uint32_t reserved : 8; /**< [in]: Reserved for padding. */ uint32_t reserved1[3]; /**< [in]: Reserved for future use. */ } NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE; /** * \struct _NVENC_EXTERNAL_ME_HINT * External Motion Vector hint structure for H264 and HEVC. */ typedef struct _NVENC_EXTERNAL_ME_HINT { int32_t mvx : 12; /**< [in]: Specifies the x component of integer pixel MV (relative to current MB) S12.0. */ int32_t mvy : 10; /**< [in]: Specifies the y component of integer pixel MV (relative to current MB) S10.0 .*/ int32_t refidx : 5; /**< [in]: Specifies the reference index (31=invalid). Current we support only 1 reference frame per direction for external hints, so \p refidx must be 0. */ int32_t dir : 1; /**< [in]: Specifies the direction of motion estimation . 0=L0 1=L1.*/ int32_t partType : 2; /**< [in]: Specifies the block partition type.0=16x16 1=16x8 2=8x16 3=8x8 (blocks in partition must be consecutive).*/ int32_t lastofPart : 1; /**< [in]: Set to 1 for the last MV of (sub) partition */ int32_t lastOfMB : 1; /**< [in]: Set to 1 for the last MV of macroblock. */ } NVENC_EXTERNAL_ME_HINT; /** * \struct _NVENC_EXTERNAL_ME_SB_HINT * External Motion Vector SB hint structure for AV1 */ typedef struct _NVENC_EXTERNAL_ME_SB_HINT { int16_t refidx : 5; /**< [in]: Specifies the reference index (31=invalid) */ int16_t direction : 1; /**< [in]: Specifies the direction of motion estimation . 0=L0 1=L1.*/ int16_t bi : 1; /**< [in]: Specifies reference mode 0=single mv, 1=compound mv */ int16_t partition_type : 3; /**< [in]: Specifies the partition type: 0: 2NX2N, 1:2NxN, 2:Nx2N. reserved 3bits for future modes */ int16_t x8 : 3; /**< [in]: Specifies the current partition's top left x position in 8 pixel unit */ int16_t last_of_cu : 1; /**< [in]: Set to 1 for the last MV current CU */ int16_t last_of_sb : 1; /**< [in]: Set to 1 for the last MV of current SB */ int16_t reserved0 : 1; /**< [in]: Reserved and must be set to 0 */ int16_t mvx : 14; /**< [in]: Specifies the x component of integer pixel MV (relative to current MB) S12.2. */ int16_t cu_size : 2; /**< [in]: Specifies the CU size: 0: 8x8, 1: 16x16, 2:32x32, 3:64x64 */ int16_t mvy : 12; /**< [in]: Specifies the y component of integer pixel MV (relative to current MB) S10.2 .*/ int16_t y8 : 3; /**< [in]: Specifies the current partition's top left y position in 8 pixel unit */ int16_t reserved1 : 1; /**< [in]: Reserved and must be set to 0 */ } NVENC_EXTERNAL_ME_SB_HINT; /** * \struct _NV_ENC_CONFIG_H264 * H264 encoder configuration parameters */ typedef struct _NV_ENC_CONFIG_H264 { uint32_t enableTemporalSVC : 1; /**< [in]: Set to 1 to enable SVC temporal*/ uint32_t enableStereoMVC : 1; /**< [in]: Set to 1 to enable stereo MVC*/ uint32_t hierarchicalPFrames : 1; /**< [in]: Set to 1 to enable hierarchical P Frames */ uint32_t hierarchicalBFrames : 1; /**< [in]: Set to 1 to enable hierarchical B Frames */ uint32_t outputBufferingPeriodSEI : 1; /**< [in]: Set to 1 to write SEI buffering period syntax in the bitstream */ uint32_t outputPictureTimingSEI : 1; /**< [in]: Set to 1 to write SEI picture timing syntax in the bitstream. */ uint32_t outputAUD : 1; /**< [in]: Set to 1 to write access unit delimiter syntax in bitstream */ uint32_t disableSPSPPS : 1; /**< [in]: Set to 1 to disable writing of Sequence and Picture parameter info in bitstream */ uint32_t outputFramePackingSEI : 1; /**< [in]: Set to 1 to enable writing of frame packing arrangement SEI messages to bitstream */ uint32_t outputRecoveryPointSEI : 1; /**< [in]: Set to 1 to enable writing of recovery point SEI message */ uint32_t enableIntraRefresh : 1; /**< [in]: Set to 1 to enable gradual decoder refresh or intra refresh. If the GOP structure uses B frames this will be ignored */ uint32_t enableConstrainedEncoding : 1; /**< [in]: Set this to 1 to enable constrainedFrame encoding where each slice in the constrained picture is independent of other slices. Constrained encoding works only with rectangular slices. Check support for constrained encoding using ::NV_ENC_CAPS_SUPPORT_CONSTRAINED_ENCODING caps. */ uint32_t repeatSPSPPS : 1; /**< [in]: Set to 1 to enable writing of Sequence and Picture parameter for every IDR frame */ uint32_t enableVFR : 1; /**< [in]: Setting enableVFR=1 currently only sets the fixed_frame_rate_flag=0 in the VUI but otherwise has no impact on the encoder behavior. For more details please refer to E.1 VUI syntax of H.264 standard. Note, however, that NVENC does not support VFR encoding and rate control. */ uint32_t enableLTR : 1; /**< [in]: Set to 1 to enable LTR (Long Term Reference) frame support. LTR can be used in two modes: "LTR Trust" mode and "LTR Per Picture" mode. LTR Trust mode: In this mode, ltrNumFrames pictures after IDR are automatically marked as LTR. This mode is enabled by setting ltrTrustMode = 1. Use of LTR Trust mode is strongly discouraged as this mode may be deprecated in future. LTR Per Picture mode: In this mode, client can control whether the current picture should be marked as LTR. Enable this mode by setting ltrTrustMode = 0 and ltrMarkFrame = 1 for the picture to be marked as LTR. This is the preferred mode for using LTR. Note that LTRs are not supported if encoding session is configured with B-frames */ uint32_t qpPrimeYZeroTransformBypassFlag : 1; /**< [in]: To enable lossless encode set this to 1, set QP to 0 and RC_mode to NV_ENC_PARAMS_RC_CONSTQP and profile to HIGH_444_PREDICTIVE_PROFILE. Check support for lossless encoding using ::NV_ENC_CAPS_SUPPORT_LOSSLESS_ENCODE caps. */ uint32_t useConstrainedIntraPred : 1; /**< [in]: Set 1 to enable constrained intra prediction. */ uint32_t enableFillerDataInsertion : 1; /**< [in]: Set to 1 to enable insertion of filler data in the bitstream. This flag will take effect only when one of the CBR rate control modes (NV_ENC_PARAMS_RC_CBR, NV_ENC_PARAMS_RC_CBR_HQ, NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ) is in use and both NV_ENC_INITIALIZE_PARAMS::frameRateNum and NV_ENC_INITIALIZE_PARAMS::frameRateDen are set to non-zero values. Setting this field when NV_ENC_INITIALIZE_PARAMS::enableOutputInVidmem is also set is currently not supported and will make ::NvEncInitializeEncoder() return an error. */ uint32_t disableSVCPrefixNalu : 1; /**< [in]: Set to 1 to disable writing of SVC Prefix NALU preceding each slice in bitstream. Applicable only when temporal SVC is enabled (NV_ENC_CONFIG_H264::enableTemporalSVC = 1). */ uint32_t enableScalabilityInfoSEI : 1; /**< [in]: Set to 1 to enable writing of Scalability Information SEI message preceding each IDR picture in bitstream Applicable only when temporal SVC is enabled (NV_ENC_CONFIG_H264::enableTemporalSVC = 1). */ uint32_t singleSliceIntraRefresh : 1; /**< [in]: Set to 1 to maintain single slice in frames during intra refresh. Check support for single slice intra refresh using ::NV_ENC_CAPS_SINGLE_SLICE_INTRA_REFRESH caps. This flag will be ignored if the value returned for ::NV_ENC_CAPS_SINGLE_SLICE_INTRA_REFRESH caps is false. */ uint32_t enableTimeCode : 1; /**< [in]: Set to 1 to enable writing of clock timestamp sets in picture timing SEI. Note that this flag will be ignored for D3D12 interface. */ uint32_t reservedBitFields : 10; /**< [in]: Reserved bitfields and must be set to 0 */ uint32_t level; /**< [in]: Specifies the encoding level. Client is recommended to set this to NV_ENC_LEVEL_AUTOSELECT in order to enable the NvEncodeAPI interface to select the correct level. */ uint32_t idrPeriod; /**< [in]: Specifies the IDR interval. If not set, this is made equal to gopLength in NV_ENC_CONFIG.Low latency application client can set IDR interval to NVENC_INFINITE_GOPLENGTH so that IDR frames are not inserted automatically. */ uint32_t separateColourPlaneFlag; /**< [in]: Set to 1 to enable 4:4:4 separate colour planes */ uint32_t disableDeblockingFilterIDC; /**< [in]: Specifies the deblocking filter mode. Permissible value range: [0,2]. This flag corresponds to the flag disable_deblocking_filter_idc specified in section 7.4.3 of H.264 specification, which specifies whether the operation of the deblocking filter shall be disabled across some block edges of the slice and specifies for which edges the filtering is disabled. See section 7.4.3 of H.264 specification for more details.*/ uint32_t numTemporalLayers; /**< [in]: Specifies number of temporal layers to be used for hierarchical coding / temporal SVC. Valid value range is [1,::NV_ENC_CAPS_NUM_MAX_TEMPORAL_LAYERS] */ uint32_t spsId; /**< [in]: Specifies the SPS id of the sequence header */ uint32_t ppsId; /**< [in]: Specifies the PPS id of the picture header */ NV_ENC_H264_ADAPTIVE_TRANSFORM_MODE adaptiveTransformMode; /**< [in]: Specifies the AdaptiveTransform Mode. Check support for AdaptiveTransform mode using ::NV_ENC_CAPS_SUPPORT_ADAPTIVE_TRANSFORM caps. */ NV_ENC_H264_FMO_MODE fmoMode; /**< [in]: Specified the FMO Mode. Check support for FMO using ::NV_ENC_CAPS_SUPPORT_FMO caps. */ NV_ENC_H264_BDIRECT_MODE bdirectMode; /**< [in]: Specifies the BDirect mode. Check support for BDirect mode using ::NV_ENC_CAPS_SUPPORT_BDIRECT_MODE caps.*/ NV_ENC_H264_ENTROPY_CODING_MODE entropyCodingMode; /**< [in]: Specifies the entropy coding mode. Check support for CABAC mode using ::NV_ENC_CAPS_SUPPORT_CABAC caps. */ NV_ENC_STEREO_PACKING_MODE stereoMode; /**< [in]: Specifies the stereo frame packing mode which is to be signaled in frame packing arrangement SEI */ uint32_t intraRefreshPeriod; /**< [in]: Specifies the interval between successive intra refresh if enableIntrarefresh is set. Requires enableIntraRefresh to be set. Will be disabled if NV_ENC_CONFIG::gopLength is not set to NVENC_INFINITE_GOPLENGTH. */ uint32_t intraRefreshCnt; /**< [in]: Specifies the length of intra refresh in number of frames for periodic intra refresh. This value should be smaller than intraRefreshPeriod */ uint32_t maxNumRefFrames; /**< [in]: Specifies the DPB size used for encoding. Setting it to 0 will let driver use the default DPB size. The low latency application which wants to invalidate reference frame as an error resilience tool is recommended to use a large DPB size so that the encoder can keep old reference frames which can be used if recent frames are invalidated. */ uint32_t sliceMode; /**< [in]: This parameter in conjunction with sliceModeData specifies the way in which the picture is divided into slices sliceMode = 0 MB based slices, sliceMode = 1 Byte based slices, sliceMode = 2 MB row based slices, sliceMode = 3 numSlices in Picture. When forceIntraRefreshWithFrameCnt is set it will have priority over sliceMode setting When sliceMode == 0 and sliceModeData == 0 whole picture will be coded with one slice */ uint32_t sliceModeData; /**< [in]: Specifies the parameter needed for sliceMode. For: sliceMode = 0, sliceModeData specifies # of MBs in each slice (except last slice) sliceMode = 1, sliceModeData specifies maximum # of bytes in each slice (except last slice) sliceMode = 2, sliceModeData specifies # of MB rows in each slice (except last slice) sliceMode = 3, sliceModeData specifies number of slices in the picture. Driver will divide picture into slices optimally */ NV_ENC_CONFIG_H264_VUI_PARAMETERS h264VUIParameters; /**< [in]: Specifies the H264 video usability info parameters */ uint32_t ltrNumFrames; /**< [in]: Specifies the number of LTR frames. This parameter has different meaning in two LTR modes. In "LTR Trust" mode (ltrTrustMode = 1), encoder will mark the first ltrNumFrames base layer reference frames within each IDR interval as LTR. In "LTR Per Picture" mode (ltrTrustMode = 0 and ltrMarkFrame = 1), ltrNumFrames specifies maximum number of LTR frames in DPB. */ uint32_t ltrTrustMode; /**< [in]: Specifies the LTR operating mode. See comments near NV_ENC_CONFIG_H264::enableLTR for description of the two modes. Set to 1 to use "LTR Trust" mode of LTR operation. Clients are discouraged to use "LTR Trust" mode as this mode may be deprecated in future releases. Set to 0 when using "LTR Per Picture" mode of LTR operation. */ uint32_t chromaFormatIDC; /**< [in]: Specifies the chroma format. Should be set to 1 for yuv420 input, 3 for yuv444 input. Check support for YUV444 encoding using ::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE caps.*/ uint32_t maxTemporalLayers; /**< [in]: Specifies the max temporal layer used for temporal SVC / hierarchical coding. Defaut value of this field is NV_ENC_CAPS::NV_ENC_CAPS_NUM_MAX_TEMPORAL_LAYERS. Note that the value NV_ENC_CONFIG_H264::maxNumRefFrames should be greater than or equal to (NV_ENC_CONFIG_H264::maxTemporalLayers - 2) * 2, for NV_ENC_CONFIG_H264::maxTemporalLayers >= 2.*/ NV_ENC_BFRAME_REF_MODE useBFramesAsRef; /**< [in]: Specifies the B-Frame as reference mode. Check support for useBFramesAsRef mode using ::NV_ENC_CAPS_SUPPORT_BFRAME_REF_MODE caps.*/ NV_ENC_NUM_REF_FRAMES numRefL0; /**< [in]: Specifies max number of reference frames in reference picture list L0, that can be used by hardware for prediction of a frame. Check support for numRefL0 using ::NV_ENC_CAPS_SUPPORT_MULTIPLE_REF_FRAMES caps. */ NV_ENC_NUM_REF_FRAMES numRefL1; /**< [in]: Specifies max number of reference frames in reference picture list L1, that can be used by hardware for prediction of a frame. Check support for numRefL1 using ::NV_ENC_CAPS_SUPPORT_MULTIPLE_REF_FRAMES caps. */ uint32_t reserved1[267]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CONFIG_H264; /** * \struct _NV_ENC_CONFIG_HEVC * HEVC encoder configuration parameters to be set during initialization. */ typedef struct _NV_ENC_CONFIG_HEVC { uint32_t level; /**< [in]: Specifies the level of the encoded bitstream.*/ uint32_t tier; /**< [in]: Specifies the level tier of the encoded bitstream.*/ NV_ENC_HEVC_CUSIZE minCUSize; /**< [in]: Specifies the minimum size of luma coding unit.*/ NV_ENC_HEVC_CUSIZE maxCUSize; /**< [in]: Specifies the maximum size of luma coding unit. Currently NVENC SDK only supports maxCUSize equal to NV_ENC_HEVC_CUSIZE_32x32.*/ uint32_t useConstrainedIntraPred : 1; /**< [in]: Set 1 to enable constrained intra prediction. */ uint32_t disableDeblockAcrossSliceBoundary : 1; /**< [in]: Set 1 to disable in loop filtering across slice boundary.*/ uint32_t outputBufferingPeriodSEI : 1; /**< [in]: Set 1 to write SEI buffering period syntax in the bitstream */ uint32_t outputPictureTimingSEI : 1; /**< [in]: Set 1 to write SEI picture timing syntax in the bitstream */ uint32_t outputAUD : 1; /**< [in]: Set 1 to write Access Unit Delimiter syntax. */ uint32_t enableLTR : 1; /**< [in]: Set to 1 to enable LTR (Long Term Reference) frame support. LTR can be used in two modes: "LTR Trust" mode and "LTR Per Picture" mode. LTR Trust mode: In this mode, ltrNumFrames pictures after IDR are automatically marked as LTR. This mode is enabled by setting ltrTrustMode = 1. Use of LTR Trust mode is strongly discouraged as this mode may be deprecated in future releases. LTR Per Picture mode: In this mode, client can control whether the current picture should be marked as LTR. Enable this mode by setting ltrTrustMode = 0 and ltrMarkFrame = 1 for the picture to be marked as LTR. This is the preferred mode for using LTR. Note that LTRs are not supported if encoding session is configured with B-frames */ uint32_t disableSPSPPS : 1; /**< [in]: Set 1 to disable VPS, SPS and PPS signaling in the bitstream. */ uint32_t repeatSPSPPS : 1; /**< [in]: Set 1 to output VPS,SPS and PPS for every IDR frame.*/ uint32_t enableIntraRefresh : 1; /**< [in]: Set 1 to enable gradual decoder refresh or intra refresh. If the GOP structure uses B frames this will be ignored */ uint32_t chromaFormatIDC : 2; /**< [in]: Specifies the chroma format. Should be set to 1 for yuv420 input, 3 for yuv444 input.*/ uint32_t pixelBitDepthMinus8 : 3; /**< [in]: Specifies pixel bit depth minus 8. Should be set to 0 for 8 bit input, 2 for 10 bit input.*/ uint32_t enableFillerDataInsertion : 1; /**< [in]: Set to 1 to enable insertion of filler data in the bitstream. This flag will take effect only when one of the CBR rate control modes (NV_ENC_PARAMS_RC_CBR, NV_ENC_PARAMS_RC_CBR_HQ, NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ) is in use and both NV_ENC_INITIALIZE_PARAMS::frameRateNum and NV_ENC_INITIALIZE_PARAMS::frameRateDen are set to non-zero values. Setting this field when NV_ENC_INITIALIZE_PARAMS::enableOutputInVidmem is also set is currently not supported and will make ::NvEncInitializeEncoder() return an error. */ uint32_t enableConstrainedEncoding : 1; /**< [in]: Set this to 1 to enable constrainedFrame encoding where each slice in the constrained picture is independent of other slices. Constrained encoding works only with rectangular slices. Check support for constrained encoding using ::NV_ENC_CAPS_SUPPORT_CONSTRAINED_ENCODING caps. */ uint32_t enableAlphaLayerEncoding : 1; /**< [in]: Set this to 1 to enable HEVC encode with alpha layer. */ uint32_t singleSliceIntraRefresh : 1; /**< [in]: Set this to 1 to maintain single slice frames during intra refresh. Check support for single slice intra refresh using ::NV_ENC_CAPS_SINGLE_SLICE_INTRA_REFRESH caps. This flag will be ignored if the value returned for ::NV_ENC_CAPS_SINGLE_SLICE_INTRA_REFRESH caps is false. */ uint32_t outputRecoveryPointSEI : 1; /**< [in]: Set to 1 to enable writing of recovery point SEI message */ uint32_t outputTimeCodeSEI : 1; /**< [in]: Set 1 to write SEI time code syntax in the bitstream. Note that this flag will be ignored for D3D12 interface.*/ uint32_t reserved : 12; /**< [in]: Reserved bitfields.*/ uint32_t idrPeriod; /**< [in]: Specifies the IDR interval. If not set, this is made equal to gopLength in NV_ENC_CONFIG. Low latency application client can set IDR interval to NVENC_INFINITE_GOPLENGTH so that IDR frames are not inserted automatically. */ uint32_t intraRefreshPeriod; /**< [in]: Specifies the interval between successive intra refresh if enableIntrarefresh is set. Requires enableIntraRefresh to be set. Will be disabled if NV_ENC_CONFIG::gopLength is not set to NVENC_INFINITE_GOPLENGTH. */ uint32_t intraRefreshCnt; /**< [in]: Specifies the length of intra refresh in number of frames for periodic intra refresh. This value should be smaller than intraRefreshPeriod */ uint32_t maxNumRefFramesInDPB; /**< [in]: Specifies the maximum number of references frames in the DPB.*/ uint32_t ltrNumFrames; /**< [in]: This parameter has different meaning in two LTR modes. In "LTR Trust" mode (ltrTrustMode = 1), encoder will mark the first ltrNumFrames base layer reference frames within each IDR interval as LTR. In "LTR Per Picture" mode (ltrTrustMode = 0 and ltrMarkFrame = 1), ltrNumFrames specifies maximum number of LTR frames in DPB. These ltrNumFrames acts as a guidance to the encoder and are not necessarily honored. To achieve a right balance between the encoding quality and keeping LTR frames in the DPB queue, the encoder can internally limit the number of LTR frames. The number of LTR frames actually used depends upon the encoding preset being used; Faster encoding presets will use fewer LTR frames.*/ uint32_t vpsId; /**< [in]: Specifies the VPS id of the video parameter set */ uint32_t spsId; /**< [in]: Specifies the SPS id of the sequence header */ uint32_t ppsId; /**< [in]: Specifies the PPS id of the picture header */ uint32_t sliceMode; /**< [in]: This parameter in conjunction with sliceModeData specifies the way in which the picture is divided into slices sliceMode = 0 CTU based slices, sliceMode = 1 Byte based slices, sliceMode = 2 CTU row based slices, sliceMode = 3, numSlices in Picture When sliceMode == 0 and sliceModeData == 0 whole picture will be coded with one slice */ uint32_t sliceModeData; /**< [in]: Specifies the parameter needed for sliceMode. For: sliceMode = 0, sliceModeData specifies # of CTUs in each slice (except last slice) sliceMode = 1, sliceModeData specifies maximum # of bytes in each slice (except last slice) sliceMode = 2, sliceModeData specifies # of CTU rows in each slice (except last slice) sliceMode = 3, sliceModeData specifies number of slices in the picture. Driver will divide picture into slices optimally */ uint32_t maxTemporalLayersMinus1; /**< [in]: Specifies the max temporal layer used for hierarchical coding. */ NV_ENC_CONFIG_HEVC_VUI_PARAMETERS hevcVUIParameters; /**< [in]: Specifies the HEVC video usability info parameters */ uint32_t ltrTrustMode; /**< [in]: Specifies the LTR operating mode. See comments near NV_ENC_CONFIG_HEVC::enableLTR for description of the two modes. Set to 1 to use "LTR Trust" mode of LTR operation. Clients are discouraged to use "LTR Trust" mode as this mode may be deprecated in future releases. Set to 0 when using "LTR Per Picture" mode of LTR operation. */ NV_ENC_BFRAME_REF_MODE useBFramesAsRef; /**< [in]: Specifies the B-Frame as reference mode. Check support for useBFramesAsRef mode using ::NV_ENC_CAPS_SUPPORT_BFRAME_REF_MODE caps.*/ NV_ENC_NUM_REF_FRAMES numRefL0; /**< [in]: Specifies max number of reference frames in reference picture list L0, that can be used by hardware for prediction of a frame. Check support for numRefL0 using ::NV_ENC_CAPS_SUPPORT_MULTIPLE_REF_FRAMES caps. */ NV_ENC_NUM_REF_FRAMES numRefL1; /**< [in]: Specifies max number of reference frames in reference picture list L1, that can be used by hardware for prediction of a frame. Check support for numRefL1 using ::NV_ENC_CAPS_SUPPORT_MULTIPLE_REF_FRAMES caps. */ uint32_t reserved1[214]; /**< [in]: Reserved and must be set to 0.*/ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CONFIG_HEVC; #define NV_MAX_TILE_COLS_AV1 64 #define NV_MAX_TILE_ROWS_AV1 64 /** * \struct _NV_ENC_FILM_GRAIN_PARAMS_AV1 * AV1 Film Grain Parameters structure */ typedef struct _NV_ENC_FILM_GRAIN_PARAMS_AV1 { uint32_t applyGrain : 1; /**< [in]: Set to 1 to specify film grain should be added to frame */ uint32_t chromaScalingFromLuma : 1; /**< [in]: Set to 1 to specify the chroma scaling is inferred from luma scaling */ uint32_t overlapFlag : 1; /**< [in]: Set to 1 to indicate that overlap between film grain blocks should be applied*/ uint32_t clipToRestrictedRange : 1; /**< [in]: Set to 1 to clip values to restricted (studio) range after adding film grain */ uint32_t grainScalingMinus8 : 2; /**< [in]: Represents the shift - 8 applied to the values of the chroma component */ uint32_t arCoeffLag : 2; /**< [in]: Specifies the number of auto-regressive coefficients for luma and chroma */ uint32_t numYPoints : 4; /**< [in]: Specifies the number of points for the piecewise linear scaling function of the luma component */ uint32_t numCbPoints : 4; /**< [in]: Specifies the number of points for the piecewise linear scaling function of the cb component */ uint32_t numCrPoints : 4; /**< [in]: Specifies the number of points for the piecewise linear scaling function of the cr component */ uint32_t arCoeffShiftMinus6 : 2; /**< [in]: specifies the range of the auto-regressive coefficients */ uint32_t grainScaleShift : 2; /**< [in]: Specifies how much the Gaussian random numbers should be scaled down during the grain synthesi process */ uint32_t reserved1 : 8; /**< [in]: Reserved bits field - should be set to 0 */ uint8_t pointYValue[14]; /**< [in]: pointYValue[i]: x coordinate for i-th point of luma piecewise linear scaling function. Values on a scale of 0...255 */ uint8_t pointYScaling[14]; /**< [in]: pointYScaling[i]: i-th point output value of luma piecewise linear scaling function */ uint8_t pointCbValue[10]; /**< [in]: pointCbValue[i]: x coordinate for i-th point of cb piecewise linear scaling function. Values on a scale of 0...255 */ uint8_t pointCbScaling[10]; /**< [in]: pointCbScaling[i]: i-th point output value of cb piecewise linear scaling function */ uint8_t pointCrValue[10]; /**< [in]: pointCrValue[i]: x coordinate for i-th point of cr piecewise linear scaling function. Values on a scale of 0...255 */ uint8_t pointCrScaling[10]; /**< [in]: pointCrScaling[i]: i-th point output value of cr piecewise linear scaling function */ uint8_t arCoeffsYPlus128[24]; /**< [in]: Specifies auto-regressive coefficients used for the Y plane */ uint8_t arCoeffsCbPlus128[25]; /**< [in]: Specifies auto-regressive coefficients used for the U plane */ uint8_t arCoeffsCrPlus128[25]; /**< [in]: Specifies auto-regressive coefficients used for the V plane */ uint8_t reserved2[2]; /**< [in]: Reserved bytes - should be set to 0 */ uint8_t cbMult; /**< [in]: Represents a multiplier for the cb component used in derivation of the input index to the cb component scaling function */ uint8_t cbLumaMult; /**< [in]: represents a multiplier for the average luma component used in derivation of the input index to the cb component scaling function. */ uint16_t cbOffset; /**< [in]: Represents an offset used in derivation of the input index to the cb component scaling function */ uint8_t crMult; /**< [in]: Represents a multiplier for the cr component used in derivation of the input index to the cr component scaling function */ uint8_t crLumaMult; /**< [in]: represents a multiplier for the average luma component used in derivation of the input index to the cr component scaling function. */ uint16_t crOffset; /**< [in]: Represents an offset used in derivation of the input index to the cr component scaling function */ } NV_ENC_FILM_GRAIN_PARAMS_AV1; /** * \struct _NV_ENC_CONFIG_AV1 * AV1 encoder configuration parameters to be set during initialization. */ typedef struct _NV_ENC_CONFIG_AV1 { uint32_t level; /**< [in]: Specifies the level of the encoded bitstream.*/ uint32_t tier; /**< [in]: Specifies the level tier of the encoded bitstream.*/ NV_ENC_AV1_PART_SIZE minPartSize; /**< [in]: Specifies the minimum size of luma coding block partition.*/ NV_ENC_AV1_PART_SIZE maxPartSize; /**< [in]: Specifies the maximum size of luma coding block partition.*/ uint32_t outputAnnexBFormat : 1; /**< [in]: Set 1 to use Annex B format for bitstream output.*/ uint32_t enableTimingInfo : 1; /**< [in]: Set 1 to write Timing Info into sequence/frame headers */ uint32_t enableDecoderModelInfo : 1; /**< [in]: Set 1 to write Decoder Model Info into sequence/frame headers */ uint32_t enableFrameIdNumbers : 1; /**< [in]: Set 1 to write Frame id numbers in bitstream */ uint32_t disableSeqHdr : 1; /**< [in]: Set 1 to disable Sequence Header signaling in the bitstream. */ uint32_t repeatSeqHdr : 1; /**< [in]: Set 1 to output Sequence Header for every Key frame.*/ uint32_t enableIntraRefresh : 1; /**< [in]: Set 1 to enable gradual decoder refresh or intra refresh. If the GOP structure uses B frames this will be ignored */ uint32_t chromaFormatIDC : 2; /**< [in]: Specifies the chroma format. Should be set to 1 for yuv420 input (yuv444 input currently not supported).*/ uint32_t enableBitstreamPadding : 1; /**< [in]: Set 1 to enable bitstream padding. */ uint32_t enableCustomTileConfig : 1; /**< [in]: Set 1 to enable custom tile configuration: numTileColumns and numTileRows must have non zero values and tileWidths and tileHeights must point to a valid address */ uint32_t enableFilmGrainParams : 1; /**< [in]: Set 1 to enable custom film grain parameters: filmGrainParams must point to a valid address */ uint32_t inputPixelBitDepthMinus8 : 3; /**< [in]: Specifies pixel bit depth minus 8 of video input. Should be set to 0 for 8 bit input, 2 for 10 bit input.*/ uint32_t pixelBitDepthMinus8 : 3; /**< [in]: Specifies pixel bit depth minus 8 of encoded video. Should be set to 0 for 8 bit, 2 for 10 bit. HW will do the bitdepth conversion internally from inputPixelBitDepthMinus8 -> pixelBitDepthMinus8 if bit dpeths differ Support for 8 bit input to 10 bit encode conversion only */ uint32_t reserved : 14; /**< [in]: Reserved bitfields.*/ uint32_t idrPeriod; /**< [in]: Specifies the IDR/Key frame interval. If not set, this is made equal to gopLength in NV_ENC_CONFIG.Low latency application client can set IDR interval to NVENC_INFINITE_GOPLENGTH so that IDR frames are not inserted automatically. */ uint32_t intraRefreshPeriod; /**< [in]: Specifies the interval between successive intra refresh if enableIntrarefresh is set. Requires enableIntraRefresh to be set. Will be disabled if NV_ENC_CONFIG::gopLength is not set to NVENC_INFINITE_GOPLENGTH. */ uint32_t intraRefreshCnt; /**< [in]: Specifies the length of intra refresh in number of frames for periodic intra refresh. This value should be smaller than intraRefreshPeriod */ uint32_t maxNumRefFramesInDPB; /**< [in]: Specifies the maximum number of references frames in the DPB.*/ uint32_t numTileColumns; /**< [in]: This parameter in conjunction with the flag enableCustomTileConfig and the array tileWidths[] specifies the way in which the picture is divided into tile columns. When enableCustomTileConfig == 0, the picture will be uniformly divided into numTileColumns tile columns. If numTileColumns is not a power of 2, it will be rounded down to the next power of 2 value. If numTileColumns == 0, the picture will be coded with the smallest number of vertical tiles as allowed by standard. When enableCustomTileConfig == 1, numTileColumns must be > 0 and <= NV_MAX_TILE_COLS_AV1 and tileWidths must point to a valid array of numTileColumns entries. Entry i specifies the width in 64x64 CTU unit of tile colum i. The sum of all the entries should be equal to the picture width in 64x64 CTU units. */ uint32_t numTileRows; /**< [in]: This parameter in conjunction with the flag enableCustomTileConfig and the array tileHeights[] specifies the way in which the picture is divided into tiles rows When enableCustomTileConfig == 0, the picture will be uniformly divided into numTileRows tile rows. If numTileRows is not a power of 2, it will be rounded down to the next power of 2 value. If numTileRows == 0, the picture will be coded with the smallest number of horizontal tiles as allowed by standard. When enableCustomTileConfig == 1, numTileRows must be > 0 and <= NV_MAX_TILE_ROWS_AV1 and tileHeights must point to a valid array of numTileRows entries. Entry i specifies the height in 64x64 CTU unit of tile row i. The sum of all the entries should be equal to the picture hieght in 64x64 CTU units. */ uint32_t* tileWidths; /**< [in]: If enableCustomTileConfig == 1, tileWidths[i] specifies the width of tile column i in 64x64 CTU unit, with 0 <= i <= numTileColumns -1. */ uint32_t* tileHeights; /**< [in]: If enableCustomTileConfig == 1, tileHeights[i] specifies the height of tile row i in 64x64 CTU unit, with 0 <= i <= numTileRows -1. */ uint32_t maxTemporalLayersMinus1; /**< [in]: Specifies the max temporal layer used for hierarchical coding. */ NV_ENC_VUI_COLOR_PRIMARIES colorPrimaries; /**< [in]: as defined in section of ISO/IEC 23091-4/ITU-T H.273 */ NV_ENC_VUI_TRANSFER_CHARACTERISTIC transferCharacteristics; /**< [in]: as defined in section of ISO/IEC 23091-4/ITU-T H.273 */ NV_ENC_VUI_MATRIX_COEFFS matrixCoefficients; /**< [in]: as defined in section of ISO/IEC 23091-4/ITU-T H.273 */ uint32_t colorRange; /**< [in]: 0: studio swing representation - 1: full swing representation */ uint32_t chromaSamplePosition; /**< [in]: 0: unknown 1: Horizontally collocated with luma (0,0) sample, between two vertical samples 2: Co-located with luma (0,0) sample */ NV_ENC_BFRAME_REF_MODE useBFramesAsRef; /**< [in]: Specifies the B-Frame as reference mode. Check support for useBFramesAsRef mode using ::NV_ENC_CAPS_SUPPORT_BFRAME_REF_MODE caps.*/ NV_ENC_FILM_GRAIN_PARAMS_AV1* filmGrainParams; /**< [in]: If enableFilmGrainParams == 1, filmGrainParams must point to a valid NV_ENC_FILM_GRAIN_PARAMS_AV1 structure */ NV_ENC_NUM_REF_FRAMES numFwdRefs; /**< [in]: Specifies max number of forward reference frame used for prediction of a frame. It must be in range 1-4 (Last, Last2, last3 and Golden). It's a suggestive value not necessarily be honored always. */ NV_ENC_NUM_REF_FRAMES numBwdRefs; /**< [in]: Specifies max number of L1 list reference frame used for prediction of a frame. It must be in range 1-3 (Backward, Altref2, Altref). It's a suggestive value not necessarily be honored always. */ uint32_t reserved1[235]; /**< [in]: Reserved and must be set to 0.*/ void* reserved2[62]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CONFIG_AV1; /** * \struct _NV_ENC_CONFIG_H264_MEONLY * H264 encoder configuration parameters for ME only Mode * */ typedef struct _NV_ENC_CONFIG_H264_MEONLY { uint32_t disablePartition16x16 : 1; /**< [in]: Disable Motion Estimation on 16x16 blocks*/ uint32_t disablePartition8x16 : 1; /**< [in]: Disable Motion Estimation on 8x16 blocks*/ uint32_t disablePartition16x8 : 1; /**< [in]: Disable Motion Estimation on 16x8 blocks*/ uint32_t disablePartition8x8 : 1; /**< [in]: Disable Motion Estimation on 8x8 blocks*/ uint32_t disableIntraSearch : 1; /**< [in]: Disable Intra search during Motion Estimation*/ uint32_t bStereoEnable : 1; /**< [in]: Enable Stereo Mode for Motion Estimation where each view is independently executed*/ uint32_t reserved : 26; /**< [in]: Reserved and must be set to 0 */ uint32_t reserved1[255]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CONFIG_H264_MEONLY; /** * \struct _NV_ENC_CONFIG_HEVC_MEONLY * HEVC encoder configuration parameters for ME only Mode * */ typedef struct _NV_ENC_CONFIG_HEVC_MEONLY { uint32_t reserved[256]; /**< [in]: Reserved and must be set to 0 */ void* reserved1[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CONFIG_HEVC_MEONLY; /** * \struct _NV_ENC_CODEC_CONFIG * Codec-specific encoder configuration parameters to be set during initialization. */ typedef union _NV_ENC_CODEC_CONFIG { NV_ENC_CONFIG_H264 h264Config; /**< [in]: Specifies the H.264-specific encoder configuration. */ NV_ENC_CONFIG_HEVC hevcConfig; /**< [in]: Specifies the HEVC-specific encoder configuration. */ NV_ENC_CONFIG_AV1 av1Config; /**< [in]: Specifies the AV1-specific encoder configuration. */ NV_ENC_CONFIG_H264_MEONLY h264MeOnlyConfig; /**< [in]: Specifies the H.264-specific ME only encoder configuration. */ NV_ENC_CONFIG_HEVC_MEONLY hevcMeOnlyConfig; /**< [in]: Specifies the HEVC-specific ME only encoder configuration. */ uint32_t reserved[320]; /**< [in]: Reserved and must be set to 0 */ } NV_ENC_CODEC_CONFIG; /** * \struct _NV_ENC_CONFIG * Encoder configuration parameters to be set during initialization. */ typedef struct _NV_ENC_CONFIG { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_CONFIG_VER. */ GUID profileGUID; /**< [in]: Specifies the codec profile GUID. If client specifies \p NV_ENC_CODEC_PROFILE_AUTOSELECT_GUID the NvEncodeAPI interface will select the appropriate codec profile. */ uint32_t gopLength; /**< [in]: Specifies the number of pictures in one GOP. Low latency application client can set goplength to NVENC_INFINITE_GOPLENGTH so that keyframes are not inserted automatically. */ int32_t frameIntervalP; /**< [in]: Specifies the GOP pattern as follows: \p frameIntervalP = 0: I, 1: IPP, 2: IBP, 3: IBBP If goplength is set to NVENC_INFINITE_GOPLENGTH \p frameIntervalP should be set to 1. */ uint32_t monoChromeEncoding; /**< [in]: Set this to 1 to enable monochrome encoding for this session. */ NV_ENC_PARAMS_FRAME_FIELD_MODE frameFieldMode; /**< [in]: Specifies the frame/field mode. Check support for field encoding using ::NV_ENC_CAPS_SUPPORT_FIELD_ENCODING caps. Using a frameFieldMode other than NV_ENC_PARAMS_FRAME_FIELD_MODE_FRAME for RGB input is not supported. */ NV_ENC_MV_PRECISION mvPrecision; /**< [in]: Specifies the desired motion vector prediction precision. */ NV_ENC_RC_PARAMS rcParams; /**< [in]: Specifies the rate control parameters for the current encoding session. */ NV_ENC_CODEC_CONFIG encodeCodecConfig; /**< [in]: Specifies the codec specific config parameters through this union. */ uint32_t reserved[278]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_CONFIG; /** macro for constructing the version field of ::_NV_ENC_CONFIG */ #define NV_ENC_CONFIG_VER (NVENCAPI_STRUCT_VERSION(8) | (1 << 31)) /** * Tuning information of NVENC encoding (TuningInfo is not applicable to H264 and HEVC MEOnly * mode). */ typedef enum NV_ENC_TUNING_INFO { NV_ENC_TUNING_INFO_UNDEFINED = 0, /**< Undefined tuningInfo. Invalid value for encoding. */ NV_ENC_TUNING_INFO_HIGH_QUALITY = 1, /**< Tune presets for latency tolerant encoding.*/ NV_ENC_TUNING_INFO_LOW_LATENCY = 2, /**< Tune presets for low latency streaming.*/ NV_ENC_TUNING_INFO_ULTRA_LOW_LATENCY = 3, /**< Tune presets for ultra low latency streaming.*/ NV_ENC_TUNING_INFO_LOSSLESS = 4, /**< Tune presets for lossless encoding.*/ NV_ENC_TUNING_INFO_COUNT /**< Count number of tuningInfos. Invalid value. */ } NV_ENC_TUNING_INFO; /** * \struct _NV_ENC_INITIALIZE_PARAMS * Encode Session Initialization parameters. */ typedef struct _NV_ENC_INITIALIZE_PARAMS { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_INITIALIZE_PARAMS_VER. */ GUID encodeGUID; /**< [in]: Specifies the Encode GUID for which the encoder is being created. ::NvEncInitializeEncoder() API will fail if this is not set, or set to unsupported value. */ GUID presetGUID; /**< [in]: Specifies the preset for encoding. If the preset GUID is set then , the preset configuration will be applied before any other parameter. */ uint32_t encodeWidth; /**< [in]: Specifies the encode width. If not set ::NvEncInitializeEncoder() API will fail. */ uint32_t encodeHeight; /**< [in]: Specifies the encode height. If not set ::NvEncInitializeEncoder() API will fail. */ uint32_t darWidth; /**< [in]: Specifies the display aspect ratio width (H264/HEVC) or the render width (AV1). */ uint32_t darHeight; /**< [in]: Specifies the display aspect ratio height (H264/HEVC) or the render height (AV1). */ uint32_t frameRateNum; /**< [in]: Specifies the numerator for frame rate used for encoding in frames per second ( Frame rate = frameRateNum / frameRateDen ). */ uint32_t frameRateDen; /**< [in]: Specifies the denominator for frame rate used for encoding in frames per second ( Frame rate = frameRateNum / frameRateDen ). */ uint32_t enableEncodeAsync; /**< [in]: Set this to 1 to enable asynchronous mode and is expected to use events to get picture completion notification. */ uint32_t enablePTD; /**< [in]: Set this to 1 to enable the Picture Type Decision is be taken by the NvEncodeAPI interface. */ uint32_t reportSliceOffsets : 1; /**< [in]: Set this to 1 to enable reporting slice offsets in ::_NV_ENC_LOCK_BITSTREAM. NV_ENC_INITIALIZE_PARAMS::enableEncodeAsync must be set to 0 to use this feature. Client must set this to 0 if NV_ENC_CONFIG_H264::sliceMode is 1 on Kepler GPUs */ uint32_t enableSubFrameWrite : 1; /**< [in]: Set this to 1 to write out available bitstream to memory at subframe intervals. If enableSubFrameWrite = 1, then the hardware encoder returns data as soon as a slice (H264/HEVC) or tile (AV1) has completed encoding. This results in better encoding latency, but the downside is that the application has to keep polling via a call to nvEncLockBitstream API continuously to see if any encoded slice/tile data is available. Use this mode if you feel that the marginal reduction in latency from sub-frame encoding is worth the increase in complexity due to CPU-based polling. */ uint32_t enableExternalMEHints : 1; /**< [in]: Set to 1 to enable external ME hints for the current frame. For NV_ENC_INITIALIZE_PARAMS::enablePTD=1 with B frames, programming L1 hints is optional for B frames since Client doesn't know internal GOP structure. NV_ENC_PIC_PARAMS::meHintRefPicDist should preferably be set with enablePTD=1. */ uint32_t enableMEOnlyMode : 1; /**< [in]: Set to 1 to enable ME Only Mode .*/ uint32_t enableWeightedPrediction : 1; /**< [in]: Set this to 1 to enable weighted prediction. Not supported if encode session is configured for B-Frames (i.e. NV_ENC_CONFIG::frameIntervalP > 1 or preset >=P3 when tuningInfo = ::NV_ENC_TUNING_INFO_HIGH_QUALITY or tuningInfo = ::NV_ENC_TUNING_INFO_LOSSLESS. This is because preset >=p3 internally enables B frames when tuningInfo = ::NV_ENC_TUNING_INFO_HIGH_QUALITY or ::NV_ENC_TUNING_INFO_LOSSLESS). */ uint32_t enableOutputInVidmem : 1; /**< [in]: Set this to 1 to enable output of NVENC in video memory buffer created by application. This feature is not supported for HEVC ME only mode. */ uint32_t reservedBitFields : 26; /**< [in]: Reserved bitfields and must be set to 0 */ uint32_t privDataSize; /**< [in]: Reserved private data buffer size and must be set to 0 */ void* privData; /**< [in]: Reserved private data buffer and must be set to NULL */ NV_ENC_CONFIG* encodeConfig; /**< [in]: Specifies the advanced codec specific structure. If client has sent a valid codec config structure, it will override parameters set by the NV_ENC_INITIALIZE_PARAMS::presetGUID parameter. If set to NULL the NvEncodeAPI interface will use the NV_ENC_INITIALIZE_PARAMS::presetGUID to set the codec specific parameters. Client can also optionally query the NvEncodeAPI interface to get codec specific parameters for a presetGUID using ::NvEncGetEncodePresetConfig() API. It can then modify (if required) some of the codec config parameters and send down a custom config structure as part of ::_NV_ENC_INITIALIZE_PARAMS. Even in this case client is recommended to pass the same preset guid it has used in ::NvEncGetEncodePresetConfig() API to query the config structure; as NV_ENC_INITIALIZE_PARAMS::presetGUID. This will not override the custom config structure but will be used to determine other Encoder HW specific parameters not exposed in the API. */ uint32_t maxEncodeWidth; /**< [in]: Maximum encode width to be used for current Encode session. Client should allocate output buffers according to this dimension for dynamic resolution change. If set to 0, Encoder will not allow dynamic resolution change. */ uint32_t maxEncodeHeight; /**< [in]: Maximum encode height to be allowed for current Encode session. Client should allocate output buffers according to this dimension for dynamic resolution change. If set to 0, Encode will not allow dynamic resolution change. */ NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE maxMEHintCountsPerBlock [2]; /**< [in]: If Client wants to pass external motion vectors in NV_ENC_PIC_PARAMS::meExternalHints buffer it must specify the maximum number of hint candidates per block per direction for the encode session. The NV_ENC_INITIALIZE_PARAMS::maxMEHintCountsPerBlock[0] is for L0 predictors and NV_ENC_INITIALIZE_PARAMS::maxMEHintCountsPerBlock[1] is for L1 predictors. This client must also set NV_ENC_INITIALIZE_PARAMS::enableExternalMEHints to 1. */ NV_ENC_TUNING_INFO tuningInfo; /**< [in]: Tuning Info of NVENC encoding(TuningInfo is not applicable to H264 and HEVC meonly mode). */ NV_ENC_BUFFER_FORMAT bufferFormat; /**< [in]: Input buffer format. Used only when DX12 interface type is used */ uint32_t reserved[287]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_INITIALIZE_PARAMS; /** macro for constructing the version field of ::_NV_ENC_INITIALIZE_PARAMS */ #define NV_ENC_INITIALIZE_PARAMS_VER (NVENCAPI_STRUCT_VERSION(5) | (1 << 31)) /** * \struct _NV_ENC_RECONFIGURE_PARAMS * Encode Session Reconfigured parameters. */ typedef struct _NV_ENC_RECONFIGURE_PARAMS { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_RECONFIGURE_PARAMS_VER. */ NV_ENC_INITIALIZE_PARAMS reInitEncodeParams; /**< [in]: Encoder session re-initialization parameters. If reInitEncodeParams.encodeConfig is NULL and reInitEncodeParams.presetGUID is the same as the preset GUID specified on the call to NvEncInitializeEncoder(), EncodeAPI will continue to use the existing encode configuration. If reInitEncodeParams.encodeConfig is NULL and reInitEncodeParams.presetGUID is different from the preset GUID specified on the call to NvEncInitializeEncoder(), EncodeAPI will try to use the default configuration for the preset specified by reInitEncodeParams.presetGUID. In this case, reconfiguration may fail if the new configuration is incompatible with the existing configuration (e.g. the new configuration results in a change in the GOP structure). */ uint32_t resetEncoder : 1; /**< [in]: This resets the rate control states and other internal encoder states. This should be used only with an IDR frame. If NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1, encoder will force the frame type to IDR */ uint32_t forceIDR : 1; /**< [in]: Encode the current picture as an IDR picture. This flag is only valid when Picture type decision is taken by the Encoder [_NV_ENC_INITIALIZE_PARAMS::enablePTD == 1]. */ uint32_t reserved : 30; } NV_ENC_RECONFIGURE_PARAMS; /** macro for constructing the version field of ::_NV_ENC_RECONFIGURE_PARAMS */ #define NV_ENC_RECONFIGURE_PARAMS_VER (NVENCAPI_STRUCT_VERSION(1) | (1 << 31)) /** * \struct _NV_ENC_PRESET_CONFIG * Encoder preset config */ typedef struct _NV_ENC_PRESET_CONFIG { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_PRESET_CONFIG_VER. */ NV_ENC_CONFIG presetCfg; /**< [out]: preset config returned by the Nvidia Video Encoder interface. */ uint32_t reserved1[255]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_PRESET_CONFIG; /** macro for constructing the version field of ::_NV_ENC_PRESET_CONFIG */ #define NV_ENC_PRESET_CONFIG_VER (NVENCAPI_STRUCT_VERSION(4) | (1 << 31)) /** * \struct _NV_ENC_PIC_PARAMS_MVC * MVC-specific parameters to be sent on a per-frame basis. */ typedef struct _NV_ENC_PIC_PARAMS_MVC { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_PIC_PARAMS_MVC_VER. */ uint32_t viewID; /**< [in]: Specifies the view ID associated with the current input view. */ uint32_t temporalID; /**< [in]: Specifies the temporal ID associated with the current input view. */ uint32_t priorityID; /**< [in]: Specifies the priority ID associated with the current input view. Reserved and ignored by the NvEncodeAPI interface. */ uint32_t reserved1[12]; /**< [in]: Reserved and must be set to 0. */ void* reserved2[8]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_PIC_PARAMS_MVC; /** macro for constructing the version field of ::_NV_ENC_PIC_PARAMS_MVC */ #define NV_ENC_PIC_PARAMS_MVC_VER NVENCAPI_STRUCT_VERSION(1) /** * \union _NV_ENC_PIC_PARAMS_H264_EXT * H264 extension picture parameters */ typedef union _NV_ENC_PIC_PARAMS_H264_EXT { NV_ENC_PIC_PARAMS_MVC mvcPicParams; /**< [in]: Specifies the MVC picture parameters. */ uint32_t reserved1[32]; /**< [in]: Reserved and must be set to 0. */ } NV_ENC_PIC_PARAMS_H264_EXT; /** * \struct _NV_ENC_SEI_PAYLOAD * User SEI message */ typedef struct _NV_ENC_SEI_PAYLOAD { uint32_t payloadSize; /**< [in] SEI payload size in bytes. SEI payload must be byte aligned, as described in Annex D */ uint32_t payloadType; /**< [in] SEI payload types and syntax can be found in Annex D of the H.264 Specification. */ uint8_t* payload; /**< [in] pointer to user data */ } NV_ENC_SEI_PAYLOAD; #define NV_ENC_H264_SEI_PAYLOAD NV_ENC_SEI_PAYLOAD /** * \struct _NV_ENC_PIC_PARAMS_H264 * H264 specific enc pic params. sent on a per frame basis. */ typedef struct _NV_ENC_PIC_PARAMS_H264 { uint32_t displayPOCSyntax; /**< [in]: Specifies the display POC syntax This is required to be set if client is handling the picture type decision. */ uint32_t reserved3; /**< [in]: Reserved and must be set to 0 */ uint32_t refPicFlag; /**< [in]: Set to 1 for a reference picture. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t colourPlaneId; /**< [in]: Specifies the colour plane ID associated with the current input. */ uint32_t forceIntraRefreshWithFrameCnt; /**< [in]: Forces an intra refresh with duration equal to intraRefreshFrameCnt. When outputRecoveryPointSEI is set this is value is used for recovery_frame_cnt in recovery point SEI message forceIntraRefreshWithFrameCnt cannot be used if B frames are used in the GOP structure specified */ uint32_t constrainedFrame : 1; /**< [in]: Set to 1 if client wants to encode this frame with each slice completely independent of other slices in the frame. NV_ENC_INITIALIZE_PARAMS::enableConstrainedEncoding should be set to 1 */ uint32_t sliceModeDataUpdate : 1; /**< [in]: Set to 1 if client wants to change the sliceModeData field to specify new sliceSize Parameter When forceIntraRefreshWithFrameCnt is set it will have priority over sliceMode setting */ uint32_t ltrMarkFrame : 1; /**< [in]: Set to 1 if client wants to mark this frame as LTR */ uint32_t ltrUseFrames : 1; /**< [in]: Set to 1 if client allows encoding this frame using the LTR frames specified in ltrFrameBitmap */ uint32_t reservedBitFields : 28; /**< [in]: Reserved bit fields and must be set to 0 */ uint8_t* sliceTypeData; /**< [in]: Deprecated. */ uint32_t sliceTypeArrayCnt; /**< [in]: Deprecated. */ uint32_t seiPayloadArrayCnt; /**< [in]: Specifies the number of elements allocated in seiPayloadArray array. */ NV_ENC_SEI_PAYLOAD* seiPayloadArray; /**< [in]: Array of SEI payloads which will be inserted for this frame. */ uint32_t sliceMode; /**< [in]: This parameter in conjunction with sliceModeData specifies the way in which the picture is divided into slices sliceMode = 0 MB based slices, sliceMode = 1 Byte based slices, sliceMode = 2 MB row based slices, sliceMode = 3, numSlices in Picture When forceIntraRefreshWithFrameCnt is set it will have priority over sliceMode setting When sliceMode == 0 and sliceModeData == 0 whole picture will be coded with one slice */ uint32_t sliceModeData; /**< [in]: Specifies the parameter needed for sliceMode. For: sliceMode = 0, sliceModeData specifies # of MBs in each slice (except last slice) sliceMode = 1, sliceModeData specifies maximum # of bytes in each slice (except last slice) sliceMode = 2, sliceModeData specifies # of MB rows in each slice (except last slice) sliceMode = 3, sliceModeData specifies number of slices in the picture. Driver will divide picture into slices optimally */ uint32_t ltrMarkFrameIdx; /**< [in]: Specifies the long term referenceframe index to use for marking this frame as LTR.*/ uint32_t ltrUseFrameBitmap; /**< [in]: Specifies the associated bitmap of LTR frame indices to use when encoding this frame. */ uint32_t ltrUsageMode; /**< [in]: Not supported. Reserved for future use and must be set to 0. */ uint32_t forceIntraSliceCount; /**< [in]: Specifies the number of slices to be forced to Intra in the current picture. This option along with forceIntraSliceIdx[] array needs to be used with sliceMode = 3 only */ uint32_t* forceIntraSliceIdx; /**< [in]: Slice indices to be forced to intra in the current picture. Each slice index should be <= num_slices_in_picture -1. Index starts from 0 for first slice. The number of entries in this array should be equal to forceIntraSliceCount */ NV_ENC_PIC_PARAMS_H264_EXT h264ExtPicParams; /**< [in]: Specifies the H264 extension config parameters using this config. */ NV_ENC_TIME_CODE timeCode; /**< [in]: Specifies the clock timestamp sets used in picture timing SEI. Applicable only when NV_ENC_CONFIG_H264::enableTimeCode is set to 1. */ uint32_t reserved[203]; /**< [in]: Reserved and must be set to 0. */ void* reserved2[61]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_PIC_PARAMS_H264; /** * \struct _NV_ENC_PIC_PARAMS_HEVC * HEVC specific enc pic params. sent on a per frame basis. */ typedef struct _NV_ENC_PIC_PARAMS_HEVC { uint32_t displayPOCSyntax; /**< [in]: Specifies the display POC syntax This is required to be set if client is handling the picture type decision. */ uint32_t refPicFlag; /**< [in]: Set to 1 for a reference picture. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t temporalId; /**< [in]: Specifies the temporal id of the picture */ uint32_t forceIntraRefreshWithFrameCnt; /**< [in]: Forces an intra refresh with duration equal to intraRefreshFrameCnt. When outputRecoveryPointSEI is set this is value is used for recovery_frame_cnt in recovery point SEI message forceIntraRefreshWithFrameCnt cannot be used if B frames are used in the GOP structure specified */ uint32_t constrainedFrame : 1; /**< [in]: Set to 1 if client wants to encode this frame with each slice completely independent of other slices in the frame. NV_ENC_INITIALIZE_PARAMS::enableConstrainedEncoding should be set to 1 */ uint32_t sliceModeDataUpdate : 1; /**< [in]: Set to 1 if client wants to change the sliceModeData field to specify new sliceSize Parameter When forceIntraRefreshWithFrameCnt is set it will have priority over sliceMode setting */ uint32_t ltrMarkFrame : 1; /**< [in]: Set to 1 if client wants to mark this frame as LTR */ uint32_t ltrUseFrames : 1; /**< [in]: Set to 1 if client allows encoding this frame using the LTR frames specified in ltrFrameBitmap */ uint32_t reservedBitFields : 28; /**< [in]: Reserved bit fields and must be set to 0 */ uint8_t* sliceTypeData; /**< [in]: Array which specifies the slice type used to force intra slice for a particular slice. Currently supported only for NV_ENC_CONFIG_H264::sliceMode == 3. Client should allocate array of size sliceModeData where sliceModeData is specified in field of ::_NV_ENC_CONFIG_H264 Array element with index n corresponds to nth slice. To force a particular slice to intra client should set corresponding array element to NV_ENC_SLICE_TYPE_I all other array elements should be set to NV_ENC_SLICE_TYPE_DEFAULT */ uint32_t sliceTypeArrayCnt; /**< [in]: Client should set this to the number of elements allocated in sliceTypeData array. If sliceTypeData is NULL then this should be set to 0 */ uint32_t sliceMode; /**< [in]: This parameter in conjunction with sliceModeData specifies the way in which the picture is divided into slices sliceMode = 0 CTU based slices, sliceMode = 1 Byte based slices, sliceMode = 2 CTU row based slices, sliceMode = 3, numSlices in Picture When forceIntraRefreshWithFrameCnt is set it will have priority over sliceMode setting When sliceMode == 0 and sliceModeData == 0 whole picture will be coded with one slice */ uint32_t sliceModeData; /**< [in]: Specifies the parameter needed for sliceMode. For: sliceMode = 0, sliceModeData specifies # of CTUs in each slice (except last slice) sliceMode = 1, sliceModeData specifies maximum # of bytes in each slice (except last slice) sliceMode = 2, sliceModeData specifies # of CTU rows in each slice (except last slice) sliceMode = 3, sliceModeData specifies number of slices in the picture. Driver will divide picture into slices optimally */ uint32_t ltrMarkFrameIdx; /**< [in]: Specifies the long term reference frame index to use for marking this frame as LTR.*/ uint32_t ltrUseFrameBitmap; /**< [in]: Specifies the associated bitmap of LTR frame indices to use when encoding this frame. */ uint32_t ltrUsageMode; /**< [in]: Not supported. Reserved for future use and must be set to 0. */ uint32_t seiPayloadArrayCnt; /**< [in]: Specifies the number of elements allocated in seiPayloadArray array. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0. */ NV_ENC_SEI_PAYLOAD* seiPayloadArray; /**< [in]: Array of SEI payloads which will be inserted for this frame. */ NV_ENC_TIME_CODE timeCode; /**< [in]: Specifies the clock timestamp sets used in time code SEI. Applicable only when NV_ENC_CONFIG_HEVC::enableTimeCodeSEI is set to 1. */ uint32_t reserved2[237]; /**< [in]: Reserved and must be set to 0. */ void* reserved3[61]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_PIC_PARAMS_HEVC; #define NV_ENC_AV1_OBU_PAYLOAD NV_ENC_SEI_PAYLOAD /** * \struct _NV_ENC_PIC_PARAMS_AV1 * AV1 specific enc pic params. sent on a per frame basis. */ typedef struct _NV_ENC_PIC_PARAMS_AV1 { uint32_t displayPOCSyntax; /**< [in]: Specifies the display POC syntax This is required to be set if client is handling the picture type decision. */ uint32_t refPicFlag; /**< [in]: Set to 1 for a reference picture. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t temporalId; /**< [in]: Specifies the temporal id of the picture */ uint32_t forceIntraRefreshWithFrameCnt; /**< [in]: Forces an intra refresh with duration equal to intraRefreshFrameCnt. forceIntraRefreshWithFrameCnt cannot be used if B frames are used in the GOP structure specified */ uint32_t goldenFrameFlag : 1; /**< [in]: Encode frame as Golden Frame. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t arfFrameFlag : 1; /**< [in]: Encode frame as Alternate Reference Frame. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t arf2FrameFlag : 1; /**< [in]: Encode frame as Alternate Reference 2 Frame. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t bwdFrameFlag : 1; /**< [in]: Encode frame as Backward Reference Frame. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t overlayFrameFlag : 1; /**< [in]: Encode frame as overlay frame. A previously encoded frame with the same displayPOCSyntax value should be present in reference frame buffer. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t showExistingFrameFlag : 1; /**< [in]: When ovelayFrameFlag is set to 1, this flag controls the value of the show_existing_frame syntax element associated with the overlay frame. This flag is added to the interface as a placeholder. Its value is ignored for now and always assumed to be set to 1. This is ignored if NV_ENC_INITIALIZE_PARAMS::enablePTD is set to 1. */ uint32_t errorResilientModeFlag : 1; /**< [in]: encode frame independently from previously encoded frames */ uint32_t tileConfigUpdate : 1; /**< [in]: Set to 1 if client wants to overwrite the default tile configuration with the tile parameters specified below When forceIntraRefreshWithFrameCnt is set it will have priority over tileConfigUpdate setting */ uint32_t enableCustomTileConfig : 1; /**< [in]: Set 1 to enable custom tile configuration: numTileColumns and numTileRows must have non zero values and tileWidths and tileHeights must point to a valid address */ uint32_t filmGrainParamsUpdate : 1; /**< [in]: Set to 1 if client wants to update previous film grain parameters: filmGrainParams must point to a valid address and encoder must have been configured with film grain enabled */ uint32_t reservedBitFields : 22; /**< [in]: Reserved bitfields and must be set to 0 */ uint32_t numTileColumns; /**< [in]: This parameter in conjunction with the flag enableCustomTileConfig and the array tileWidths[] specifies the way in which the picture is divided into tile columns. When enableCustomTileConfig == 0, the picture will be uniformly divided into numTileColumns tile columns. If numTileColumns is not a power of 2, it will be rounded down to the next power of 2 value. If numTileColumns == 0, the picture will be coded with the smallest number of vertical tiles as allowed by standard. When enableCustomTileConfig == 1, numTileColumns must be > 0 and <= NV_MAX_TILE_COLS_AV1 and tileWidths must point to a valid array of numTileColumns entries. Entry i specifies the width in 64x64 CTU unit of tile colum i. The sum of all the entries should be equal to the picture width in 64x64 CTU units. */ uint32_t numTileRows; /**< [in]: This parameter in conjunction with the flag enableCustomTileConfig and the array tileHeights[] specifies the way in which the picture is divided into tiles rows When enableCustomTileConfig == 0, the picture will be uniformly divided into numTileRows tile rows. If numTileRows is not a power of 2, it will be rounded down to the next power of 2 value. If numTileRows == 0, the picture will be coded with the smallest number of horizontal tiles as allowed by standard. When enableCustomTileConfig == 1, numTileRows must be > 0 and <= NV_MAX_TILE_ROWS_AV1 and tileHeights must point to a valid array of numTileRows entries. Entry i specifies the height in 64x64 CTU unit of tile row i. The sum of all the entries should be equal to the picture hieght in 64x64 CTU units. */ uint32_t* tileWidths; /**< [in]: If enableCustomTileConfig == 1, tileWidths[i] specifies the width of tile column i in 64x64 CTU unit, with 0 <= i <= numTileColumns -1. */ uint32_t* tileHeights; /**< [in]: If enableCustomTileConfig == 1, tileHeights[i] specifies the height of tile row i in 64x64 CTU unit, with 0 <= i <= numTileRows -1. */ uint32_t obuPayloadArrayCnt; /**< [in]: Specifies the number of elements allocated in obuPayloadArray array. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0. */ NV_ENC_AV1_OBU_PAYLOAD* obuPayloadArray; /**< [in]: Array of OBU payloads which will be inserted for this frame. */ NV_ENC_FILM_GRAIN_PARAMS_AV1* filmGrainParams; /**< [in]: If filmGrainParamsUpdate == 1, filmGrainParams must point to a valid NV_ENC_FILM_GRAIN_PARAMS_AV1 structure */ uint32_t reserved2[247]; /**< [in]: Reserved and must be set to 0. */ void* reserved3[61]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_PIC_PARAMS_AV1; /** * Codec specific per-picture encoding parameters. */ typedef union _NV_ENC_CODEC_PIC_PARAMS { NV_ENC_PIC_PARAMS_H264 h264PicParams; /**< [in]: H264 encode picture params. */ NV_ENC_PIC_PARAMS_HEVC hevcPicParams; /**< [in]: HEVC encode picture params. */ NV_ENC_PIC_PARAMS_AV1 av1PicParams; /**< [in]: AV1 encode picture params. */ uint32_t reserved[256]; /**< [in]: Reserved and must be set to 0. */ } NV_ENC_CODEC_PIC_PARAMS; /** * \struct _NV_ENC_PIC_PARAMS * Encoding parameters that need to be sent on a per frame basis. */ typedef struct _NV_ENC_PIC_PARAMS { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_PIC_PARAMS_VER. */ uint32_t inputWidth; /**< [in]: Specifies the input frame width */ uint32_t inputHeight; /**< [in]: Specifies the input frame height */ uint32_t inputPitch; /**< [in]: Specifies the input buffer pitch. If pitch value is not known, set this to inputWidth. */ uint32_t encodePicFlags; /**< [in]: Specifies bit-wise OR of encode picture flags. See ::NV_ENC_PIC_FLAGS enum. */ uint32_t frameIdx; /**< [in]: Specifies the frame index associated with the input frame [optional]. */ uint64_t inputTimeStamp; /**< [in]: Specifies opaque data which is associated with the encoded frame, but not actually encoded in the output bitstream. This opaque data can be used later to uniquely refer to the corresponding encoded frame. For example, it can be used for identifying the frame to be invalidated in the reference picture buffer, if lost at the client. */ uint64_t inputDuration; /**< [in]: Specifies duration of the input picture */ NV_ENC_INPUT_PTR inputBuffer; /**< [in]: Specifies the input buffer pointer. Client must use a pointer obtained from ::NvEncCreateInputBuffer() or ::NvEncMapInputResource() APIs.*/ NV_ENC_OUTPUT_PTR outputBitstream; /**< [in]: Specifies the output buffer pointer. If NV_ENC_INITIALIZE_PARAMS::enableOutputInVidmem is set to 0, specifies the pointer to output buffer. Client should use a pointer obtained from ::NvEncCreateBitstreamBuffer() API. If NV_ENC_INITIALIZE_PARAMS::enableOutputInVidmem is set to 1, client should allocate buffer in video memory for NV_ENC_ENCODE_OUT_PARAMS struct and encoded bitstream data. Client should use a pointer obtained from ::NvEncMapInputResource() API, when mapping this output buffer and assign it to NV_ENC_PIC_PARAMS::outputBitstream. First 256 bytes of this buffer should be interpreted as NV_ENC_ENCODE_OUT_PARAMS struct followed by encoded bitstream data. Recommended size for output buffer is sum of size of NV_ENC_ENCODE_OUT_PARAMS struct and twice the input frame size for lower resolution eg. CIF and 1.5 times the input frame size for higher resolutions. If encoded bitstream size is greater than the allocated buffer size for encoded bitstream, then the output buffer will have encoded bitstream data equal to buffer size. All CUDA operations on this buffer must use the default stream. */ void* completionEvent; /**< [in]: Specifies an event to be signaled on completion of encoding of this Frame [only if operating in Asynchronous mode]. Each output buffer should be associated with a distinct event pointer. */ NV_ENC_BUFFER_FORMAT bufferFmt; /**< [in]: Specifies the input buffer format. */ NV_ENC_PIC_STRUCT pictureStruct; /**< [in]: Specifies structure of the input picture. */ NV_ENC_PIC_TYPE pictureType; /**< [in]: Specifies input picture type. Client required to be set explicitly by the client if the client has not set NV_ENC_INITALIZE_PARAMS::enablePTD to 1 while calling NvInitializeEncoder. */ NV_ENC_CODEC_PIC_PARAMS codecPicParams; /**< [in]: Specifies the codec specific per-picture encoding parameters. */ NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE meHintCountsPerBlock[2]; /**< [in]: For H264 and Hevc, specifies the number of hint candidates per block per direction for the current frame. meHintCountsPerBlock[0] is for L0 predictors and meHintCountsPerBlock[1] is for L1 predictors. The candidate count in NV_ENC_PIC_PARAMS::meHintCountsPerBlock[lx] must never exceed NV_ENC_INITIALIZE_PARAMS::maxMEHintCountsPerBlock[lx] provided during encoder initialization. */ NVENC_EXTERNAL_ME_HINT* meExternalHints; /**< [in]: For H264 and Hevc, Specifies the pointer to ME external hints for the current frame. The size of ME hint buffer should be equal to number of macroblocks * the total number of candidates per macroblock. The total number of candidates per MB per direction = 1*meHintCountsPerBlock[Lx].numCandsPerBlk16x16 + 2*meHintCountsPerBlock[Lx].numCandsPerBlk16x8 + 2*meHintCountsPerBlock[Lx].numCandsPerBlk8x8 + 4*meHintCountsPerBlock[Lx].numCandsPerBlk8x8. For frames using bidirectional ME , the total number of candidates for single macroblock is sum of total number of candidates per MB for each direction (L0 and L1) */ uint32_t reserved1[6]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[2]; /**< [in]: Reserved and must be set to NULL */ int8_t* qpDeltaMap; /**< [in]: Specifies the pointer to signed byte array containing value per MB for H264, per CTB for HEVC and per SB for AV1 in raster scan order for the current picture, which will be interpreted depending on NV_ENC_RC_PARAMS::qpMapMode. If NV_ENC_RC_PARAMS::qpMapMode is NV_ENC_QP_MAP_DELTA, qpDeltaMap specifies QP modifier per MB for H264, per CTB for HEVC and per SB for AV1. This QP modifier will be applied on top of the QP chosen by rate control. If NV_ENC_RC_PARAMS::qpMapMode is NV_ENC_QP_MAP_EMPHASIS, qpDeltaMap specifies Emphasis Level Map per MB for H264. This level value along with QP chosen by rate control is used to compute the QP modifier, which in turn is applied on top of QP chosen by rate control. If NV_ENC_RC_PARAMS::qpMapMode is NV_ENC_QP_MAP_DISABLED, value in qpDeltaMap will be ignored.*/ uint32_t qpDeltaMapSize; /**< [in]: Specifies the size in bytes of qpDeltaMap surface allocated by client and pointed to by NV_ENC_PIC_PARAMS::qpDeltaMap. Surface (array) should be picWidthInMbs * picHeightInMbs for H264, picWidthInCtbs * picHeightInCtbs for HEVC and picWidthInSbs * picHeightInSbs for AV1 */ uint32_t reservedBitFields; /**< [in]: Reserved bitfields and must be set to 0 */ uint16_t meHintRefPicDist[2]; /**< [in]: Specifies temporal distance for reference picture (NVENC_EXTERNAL_ME_HINT::refidx = 0) used during external ME with NV_ENC_INITALIZE_PARAMS::enablePTD = 1 . meHintRefPicDist[0] is for L0 hints and meHintRefPicDist[1] is for L1 hints. If not set, will internally infer distance of 1. Ignored for NV_ENC_INITALIZE_PARAMS::enablePTD = 0 */ NV_ENC_INPUT_PTR alphaBuffer; /**< [in]: Specifies the input alpha buffer pointer. Client must use a pointer obtained from ::NvEncCreateInputBuffer() or ::NvEncMapInputResource() APIs. Applicable only when encoding hevc with alpha layer is enabled. */ NVENC_EXTERNAL_ME_SB_HINT* meExternalSbHints; /**< [in]: For AV1,Specifies the pointer to ME external SB hints for the current frame. The size of ME hint buffer should be equal to meSbHintsCount. */ uint32_t meSbHintsCount; /**< [in]: For AV1, specifies the total number of external ME SB hint candidates for the frame NV_ENC_PIC_PARAMS::meSbHintsCount must never exceed the total number of SBs in frame * the max number of candidates per SB provided during encoder initialization. The max number of candidates per SB is maxMeHintCountsPerBlock[0].numCandsPerSb + maxMeHintCountsPerBlock[1].numCandsPerSb */ uint32_t reserved3[285]; /**< [in]: Reserved and must be set to 0 */ void* reserved4[58]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_PIC_PARAMS; /** Macro for constructing the version field of ::_NV_ENC_PIC_PARAMS */ #define NV_ENC_PIC_PARAMS_VER (NVENCAPI_STRUCT_VERSION(6) | (1 << 31)) /** * \struct _NV_ENC_MEONLY_PARAMS * MEOnly parameters that need to be sent on a per motion estimation basis. * NV_ENC_MEONLY_PARAMS::meExternalHints is supported for H264 only. */ typedef struct _NV_ENC_MEONLY_PARAMS { uint32_t version; /**< [in]: Struct version. Must be set to NV_ENC_MEONLY_PARAMS_VER.*/ uint32_t inputWidth; /**< [in]: Specifies the input frame width */ uint32_t inputHeight; /**< [in]: Specifies the input frame height */ NV_ENC_INPUT_PTR inputBuffer; /**< [in]: Specifies the input buffer pointer. Client must use a pointer obtained from NvEncCreateInputBuffer() or NvEncMapInputResource() APIs. */ NV_ENC_INPUT_PTR referenceFrame; /**< [in]: Specifies the reference frame pointer */ NV_ENC_OUTPUT_PTR mvBuffer; /**< [in]: Specifies the output buffer pointer. If NV_ENC_INITIALIZE_PARAMS::enableOutputInVidmem is set to 0, specifies the pointer to motion vector data buffer allocated by NvEncCreateMVBuffer. Client must lock mvBuffer using ::NvEncLockBitstream() API to get the motion vector data. If NV_ENC_INITIALIZE_PARAMS::enableOutputInVidmem is set to 1, client should allocate buffer in video memory for storing the motion vector data. The size of this buffer must be equal to total number of macroblocks multiplied by size of NV_ENC_H264_MV_DATA struct. Client should use a pointer obtained from ::NvEncMapInputResource() API, when mapping this output buffer and assign it to NV_ENC_MEONLY_PARAMS::mvBuffer. All CUDA operations on this buffer must use the default stream. */ NV_ENC_BUFFER_FORMAT bufferFmt; /**< [in]: Specifies the input buffer format. */ void* completionEvent; /**< [in]: Specifies an event to be signaled on completion of motion estimation of this Frame [only if operating in Asynchronous mode]. Each output buffer should be associated with a distinct event pointer. */ uint32_t viewID; /**< [in]: Specifies left or right viewID if NV_ENC_CONFIG_H264_MEONLY::bStereoEnable is set. viewID can be 0,1 if bStereoEnable is set, 0 otherwise. */ NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE meHintCountsPerBlock[2]; /**< [in]: Specifies the number of hint candidates per block for the current frame. meHintCountsPerBlock[0] is for L0 predictors. The candidate count in NV_ENC_PIC_PARAMS::meHintCountsPerBlock[lx] must never exceed NV_ENC_INITIALIZE_PARAMS::maxMEHintCountsPerBlock[lx] provided during encoder initialization. */ NVENC_EXTERNAL_ME_HINT* meExternalHints; /**< [in]: Specifies the pointer to ME external hints for the current frame. The size of ME hint buffer should be equal to number of macroblocks * the total number of candidates per macroblock. The total number of candidates per MB per direction = 1*meHintCountsPerBlock[Lx].numCandsPerBlk16x16 + 2*meHintCountsPerBlock[Lx].numCandsPerBlk16x8 + 2*meHintCountsPerBlock[Lx].numCandsPerBlk8x8 + 4*meHintCountsPerBlock[Lx].numCandsPerBlk8x8. For frames using bidirectional ME , the total number of candidates for single macroblock is sum of total number of candidates per MB for each direction (L0 and L1) */ uint32_t reserved1[243]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[59]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_MEONLY_PARAMS; /** NV_ENC_MEONLY_PARAMS struct version*/ #define NV_ENC_MEONLY_PARAMS_VER NVENCAPI_STRUCT_VERSION(3) /** * \struct _NV_ENC_LOCK_BITSTREAM * Bitstream buffer lock parameters. */ typedef struct _NV_ENC_LOCK_BITSTREAM { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_LOCK_BITSTREAM_VER. */ uint32_t doNotWait : 1; /**< [in]: If this flag is set, the NvEncodeAPI interface will return buffer pointer even if operation is not completed. If not set, the call will block until operation completes. */ uint32_t ltrFrame : 1; /**< [out]: Flag indicating this frame is marked as LTR frame */ uint32_t getRCStats : 1; /**< [in]: If this flag is set then lockBitstream call will add additional intra-inter MB count and average MVX, MVY */ uint32_t reservedBitFields : 29; /**< [in]: Reserved bit fields and must be set to 0 */ void* outputBitstream; /**< [in]: Pointer to the bitstream buffer being locked. */ uint32_t* sliceOffsets; /**< [in, out]: Array which receives the slice (H264/HEVC) or tile (AV1) offsets. This is not supported if NV_ENC_CONFIG_H264::sliceMode is 1 on Kepler GPUs. Array size must be equal to size of frame in MBs. */ uint32_t frameIdx; /**< [out]: Frame no. for which the bitstream is being retrieved. */ uint32_t hwEncodeStatus; /**< [out]: The NvEncodeAPI interface status for the locked picture. */ uint32_t numSlices; /**< [out]: Number of slices (H264/HEVC) or tiles (AV1) in the encoded picture. Will be reported only if NV_ENC_INITIALIZE_PARAMS::reportSliceOffsets set to 1. */ uint32_t bitstreamSizeInBytes; /**< [out]: Actual number of bytes generated and copied to the memory pointed by bitstreamBufferPtr. When HEVC alpha layer encoding is enabled, this field reports the total encoded size in bytes i.e it is the encoded size of the base plus the alpha layer. For AV1 when enablePTD is set, this field reports the total encoded size in bytes of all the encoded frames packed into the current output surface i.e. show frame plus all preceding no-show frames */ uint64_t outputTimeStamp; /**< [out]: Presentation timestamp associated with the encoded output. */ uint64_t outputDuration; /**< [out]: Presentation duration associates with the encoded output. */ void* bitstreamBufferPtr; /**< [out]: Pointer to the generated output bitstream. For MEOnly mode _NV_ENC_LOCK_BITSTREAM::bitstreamBufferPtr should be typecast to NV_ENC_H264_MV_DATA/NV_ENC_HEVC_MV_DATA pointer respectively for H264/HEVC */ NV_ENC_PIC_TYPE pictureType; /**< [out]: Picture type of the encoded picture. */ NV_ENC_PIC_STRUCT pictureStruct; /**< [out]: Structure of the generated output picture. */ uint32_t frameAvgQP; /**< [out]: Average QP of the frame. */ uint32_t frameSatd; /**< [out]: Total SATD cost for whole frame. */ uint32_t ltrFrameIdx; /**< [out]: Frame index associated with this LTR frame. */ uint32_t ltrFrameBitmap; /**< [out]: Bitmap of LTR frames indices which were used for encoding this frame. Value of 0 if no LTR frames were used. */ uint32_t temporalId; /**< [out]: TemporalId value of the frame when using temporalSVC encoding */ uint32_t reserved[12]; /**< [in]: Reserved and must be set to 0 */ uint32_t intraMBCount; /**< [out]: For H264, Number of Intra MBs in the encoded frame. For HEVC, Number of Intra CTBs in the encoded frame. For AV1, Number of Intra SBs in the encoded show frame. Supported only if _NV_ENC_LOCK_BITSTREAM::getRCStats set to 1. */ uint32_t interMBCount; /**< [out]: For H264, Number of Inter MBs in the encoded frame, includes skip MBs. For HEVC, Number of Inter CTBs in the encoded frame. For AV1, Number of Inter SBs in the encoded show frame. Supported only if _NV_ENC_LOCK_BITSTREAM::getRCStats set to 1. */ int32_t averageMVX; /**< [out]: Average Motion Vector in X direction for the encoded frame. Supported only if _NV_ENC_LOCK_BITSTREAM::getRCStats set to 1. */ int32_t averageMVY; /**< [out]: Average Motion Vector in y direction for the encoded frame. Supported only if _NV_ENC_LOCK_BITSTREAM::getRCStats set to 1. */ uint32_t alphaLayerSizeInBytes; /**< [out]: Number of bytes generated for the alpha layer in the encoded output. Applicable only when HEVC with alpha encoding is enabled. */ uint32_t reserved1[218]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_LOCK_BITSTREAM; /** Macro for constructing the version field of ::_NV_ENC_LOCK_BITSTREAM */ #define NV_ENC_LOCK_BITSTREAM_VER NVENCAPI_STRUCT_VERSION(2) /** * \struct _NV_ENC_LOCK_INPUT_BUFFER * Uncompressed Input Buffer lock parameters. */ typedef struct _NV_ENC_LOCK_INPUT_BUFFER { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_LOCK_INPUT_BUFFER_VER. */ uint32_t doNotWait : 1; /**< [in]: Set to 1 to make ::NvEncLockInputBuffer() a unblocking call. If the encoding is not completed, driver will return ::NV_ENC_ERR_ENCODER_BUSY error code. */ uint32_t reservedBitFields : 31; /**< [in]: Reserved bitfields and must be set to 0 */ NV_ENC_INPUT_PTR inputBuffer; /**< [in]: Pointer to the input buffer to be locked, client should pass the pointer obtained from ::NvEncCreateInputBuffer() or ::NvEncMapInputResource API. */ void* bufferDataPtr; /**< [out]: Pointed to the locked input buffer data. Client can only access input buffer using the \p bufferDataPtr. */ uint32_t pitch; /**< [out]: Pitch of the locked input buffer. */ uint32_t reserved1[251]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_LOCK_INPUT_BUFFER; /** Macro for constructing the version field of ::_NV_ENC_LOCK_INPUT_BUFFER */ #define NV_ENC_LOCK_INPUT_BUFFER_VER NVENCAPI_STRUCT_VERSION(1) /** * \struct _NV_ENC_MAP_INPUT_RESOURCE * Map an input resource to a Nvidia Encoder Input Buffer */ typedef struct _NV_ENC_MAP_INPUT_RESOURCE { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_MAP_INPUT_RESOURCE_VER. */ uint32_t subResourceIndex; /**< [in]: Deprecated. Do not use. */ void* inputResource; /**< [in]: Deprecated. Do not use. */ NV_ENC_REGISTERED_PTR registeredResource; /**< [in]: The Registered resource handle obtained by calling NvEncRegisterInputResource. */ NV_ENC_INPUT_PTR mappedResource; /**< [out]: Mapped pointer corresponding to the registeredResource. This pointer must be used in NV_ENC_PIC_PARAMS::inputBuffer parameter in ::NvEncEncodePicture() API. */ NV_ENC_BUFFER_FORMAT mappedBufferFmt; /**< [out]: Buffer format of the outputResource. This buffer format must be used in NV_ENC_PIC_PARAMS::bufferFmt if client using the above mapped resource pointer. */ uint32_t reserved1[251]; /**< [in]: Reserved and must be set to 0. */ void* reserved2[63]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_MAP_INPUT_RESOURCE; /** Macro for constructing the version field of ::_NV_ENC_MAP_INPUT_RESOURCE */ #define NV_ENC_MAP_INPUT_RESOURCE_VER NVENCAPI_STRUCT_VERSION(4) /** * \struct _NV_ENC_INPUT_RESOURCE_OPENGL_TEX * NV_ENC_REGISTER_RESOURCE::resourceToRegister must be a pointer to a variable of this type, * when NV_ENC_REGISTER_RESOURCE::resourceType is NV_ENC_INPUT_RESOURCE_TYPE_OPENGL_TEX */ typedef struct _NV_ENC_INPUT_RESOURCE_OPENGL_TEX { uint32_t texture; /**< [in]: The name of the texture to be used. */ uint32_t target; /**< [in]: Accepted values are GL_TEXTURE_RECTANGLE and GL_TEXTURE_2D. */ } NV_ENC_INPUT_RESOURCE_OPENGL_TEX; /** \struct NV_ENC_FENCE_POINT_D3D12 * Fence and fence value for synchronization. */ typedef struct _NV_ENC_FENCE_POINT_D3D12 { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_FENCE_POINT_D3D12_VER. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0. */ void* pFence; /**< [in]: Pointer to ID3D12Fence. This fence object is used for synchronization. */ uint64_t waitValue; /**< [in]: Fence value to reach or exceed before the GPU operation. */ uint64_t signalValue; /**< [in]: Fence value to set the fence to, after the GPU operation. */ uint32_t bWait : 1; /**< [in]: Wait on 'waitValue' if bWait is set to 1, before starting GPU operation. */ uint32_t bSignal : 1; /**< [in]: Signal on 'signalValue' if bSignal is set to 1, after GPU operation is complete. */ uint32_t reservedBitField : 30; /**< [in]: Reserved and must be set to 0. */ uint32_t reserved1[7]; /**< [in]: Reserved and must be set to 0. */ } NV_ENC_FENCE_POINT_D3D12; #define NV_ENC_FENCE_POINT_D3D12_VER NVENCAPI_STRUCT_VERSION(1) /** * \struct _NV_ENC_INPUT_RESOURCE_D3D12 * NV_ENC_PIC_PARAMS::inputBuffer and NV_ENC_PIC_PARAMS::alphaBuffer must be a pointer to a struct * of this type, when D3D12 interface is used */ typedef struct _NV_ENC_INPUT_RESOURCE_D3D12 { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_INPUT_RESOURCE_D3D12_VER. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0. */ NV_ENC_INPUT_PTR pInputBuffer; /**< [in]: Specifies the input surface pointer. Client must use a pointer obtained from NvEncMapInputResource() in NV_ENC_MAP_INPUT_RESOURCE::mappedResource when mapping the input surface. */ NV_ENC_FENCE_POINT_D3D12 inputFencePoint; /**< [in]: Specifies the fence and corresponding fence values to do GPU wait and signal. */ uint32_t reserved1[16]; /**< [in]: Reserved and must be set to 0. */ void* reserved2[16]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_INPUT_RESOURCE_D3D12; #define NV_ENC_INPUT_RESOURCE_D3D12_VER NVENCAPI_STRUCT_VERSION(1) /** * \struct _NV_ENC_OUTPUT_RESOURCE_D3D12 * NV_ENC_PIC_PARAMS::outputBitstream and NV_ENC_LOCK_BITSTREAM::outputBitstream must be a pointer * to a struct of this type, when D3D12 interface is used */ typedef struct _NV_ENC_OUTPUT_RESOURCE_D3D12 { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_OUTPUT_RESOURCE_D3D12_VER. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0. */ NV_ENC_INPUT_PTR pOutputBuffer; /**< [in]: Specifies the output buffer pointer. Client must use a pointer obtained from NvEncMapInputResource() in NV_ENC_MAP_INPUT_RESOURCE::mappedResource when mapping output bitstream buffer */ NV_ENC_FENCE_POINT_D3D12 outputFencePoint; /**< [in]: Specifies the fence and corresponding fence values to do GPU wait and signal.*/ uint32_t reserved1[16]; /**< [in]: Reserved and must be set to 0. */ void* reserved2[16]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_OUTPUT_RESOURCE_D3D12; #define NV_ENC_OUTPUT_RESOURCE_D3D12_VER NVENCAPI_STRUCT_VERSION(1) /** * \struct _NV_ENC_REGISTER_RESOURCE * Register a resource for future use with the Nvidia Video Encoder Interface. */ typedef struct _NV_ENC_REGISTER_RESOURCE { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_REGISTER_RESOURCE_VER. */ NV_ENC_INPUT_RESOURCE_TYPE resourceType; /**< [in]: Specifies the type of resource to be registered. Supported values are ::NV_ENC_INPUT_RESOURCE_TYPE_DIRECTX, ::NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR, ::NV_ENC_INPUT_RESOURCE_TYPE_OPENGL_TEX */ uint32_t width; /**< [in]: Input frame width. */ uint32_t height; /**< [in]: Input frame height. */ uint32_t pitch; /**< [in]: Input buffer pitch. For ::NV_ENC_INPUT_RESOURCE_TYPE_DIRECTX resources, set this to 0. For ::NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR resources, set this to the pitch as obtained from cuMemAllocPitch(), or to the width in bytes (if this resource was created by using cuMemAlloc()). This value must be a multiple of 4. For ::NV_ENC_INPUT_RESOURCE_TYPE_CUDAARRAY resources, set this to the width of the allocation in bytes (i.e. CUDA_ARRAY3D_DESCRIPTOR::Width * CUDA_ARRAY3D_DESCRIPTOR::NumChannels). For ::NV_ENC_INPUT_RESOURCE_TYPE_OPENGL_TEX resources, set this to the texture width multiplied by the number of components in the texture format. */ uint32_t subResourceIndex; /**< [in]: Subresource Index of the DirectX resource to be registered. Should be set to 0 for other interfaces. */ void* resourceToRegister; /**< [in]: Handle to the resource that is being registered. */ NV_ENC_REGISTERED_PTR registeredResource; /**< [out]: Registered resource handle. This should be used in future interactions with the Nvidia Video Encoder Interface. */ NV_ENC_BUFFER_FORMAT bufferFormat; /**< [in]: Buffer format of resource to be registered. */ NV_ENC_BUFFER_USAGE bufferUsage; /**< [in]: Usage of resource to be registered. */ NV_ENC_FENCE_POINT_D3D12* pInputFencePoint; /**< [in]: Specifies the input fence and corresponding fence values to do GPU wait and signal. To be used only when NV_ENC_REGISTER_RESOURCE::resourceToRegister represents D3D12 surface and NV_ENC_BUFFER_USAGE::bufferUsage is NV_ENC_INPUT_IMAGE. The fence NV_ENC_FENCE_POINT_D3D12::pFence and NV_ENC_FENCE_POINT_D3D12::waitValue will be used to do GPU wait before starting GPU operation, if NV_ENC_FENCE_POINT_D3D12::bWait is set. The fence NV_ENC_FENCE_POINT_D3D12::pFence and NV_ENC_FENCE_POINT_D3D12::signalValue will be used to do GPU signal when GPU operation finishes, if NV_ENC_FENCE_POINT_D3D12::bSignal is set. */ uint32_t reserved1[247]; /**< [in]: Reserved and must be set to 0. */ void* reserved2[61]; /**< [in]: Reserved and must be set to NULL. */ } NV_ENC_REGISTER_RESOURCE; /** Macro for constructing the version field of ::_NV_ENC_REGISTER_RESOURCE */ #define NV_ENC_REGISTER_RESOURCE_VER NVENCAPI_STRUCT_VERSION(4) /** * \struct _NV_ENC_STAT * Encode Stats structure. */ typedef struct _NV_ENC_STAT { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_STAT_VER. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0 */ NV_ENC_OUTPUT_PTR outputBitStream; /**< [out]: Specifies the pointer to output bitstream. */ uint32_t bitStreamSize; /**< [out]: Size of generated bitstream in bytes. */ uint32_t picType; /**< [out]: Picture type of encoded picture. See ::NV_ENC_PIC_TYPE. */ uint32_t lastValidByteOffset; /**< [out]: Offset of last valid bytes of completed bitstream */ uint32_t sliceOffsets[16]; /**< [out]: Offsets of each slice */ uint32_t picIdx; /**< [out]: Picture number */ uint32_t frameAvgQP; /**< [out]: Average QP of the frame. */ uint32_t ltrFrame : 1; /**< [out]: Flag indicating this frame is marked as LTR frame */ uint32_t reservedBitFields : 31; /**< [in]: Reserved bit fields and must be set to 0 */ uint32_t ltrFrameIdx; /**< [out]: Frame index associated with this LTR frame. */ uint32_t intraMBCount; /**< [out]: For H264, Number of Intra MBs in the encoded frame. For HEVC, Number of Intra CTBs in the encoded frame. */ uint32_t interMBCount; /**< [out]: For H264, Number of Inter MBs in the encoded frame, includes skip MBs. For HEVC, Number of Inter CTBs in the encoded frame. */ int32_t averageMVX; /**< [out]: Average Motion Vector in X direction for the encoded frame. */ int32_t averageMVY; /**< [out]: Average Motion Vector in y direction for the encoded frame. */ uint32_t reserved1[226]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_STAT; /** Macro for constructing the version field of ::_NV_ENC_STAT */ #define NV_ENC_STAT_VER NVENCAPI_STRUCT_VERSION(1) /** * \struct _NV_ENC_SEQUENCE_PARAM_PAYLOAD * Sequence and picture paramaters payload. */ typedef struct _NV_ENC_SEQUENCE_PARAM_PAYLOAD { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_INITIALIZE_PARAMS_VER. */ uint32_t inBufferSize; /**< [in]: Specifies the size of the spsppsBuffer provided by the client */ uint32_t spsId; /**< [in]: Specifies the SPS id to be used in sequence header. Default value is 0. */ uint32_t ppsId; /**< [in]: Specifies the PPS id to be used in picture header. Default value is 0. */ void* spsppsBuffer; /**< [in]: Specifies bitstream header pointer of size NV_ENC_SEQUENCE_PARAM_PAYLOAD::inBufferSize. It is the client's responsibility to manage this memory. */ uint32_t* outSPSPPSPayloadSize; /**< [out]: Size of the sequence and picture header in bytes. */ uint32_t reserved[250]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_SEQUENCE_PARAM_PAYLOAD; /** Macro for constructing the version field of ::_NV_ENC_SEQUENCE_PARAM_PAYLOAD */ #define NV_ENC_SEQUENCE_PARAM_PAYLOAD_VER NVENCAPI_STRUCT_VERSION(1) /** * Event registration/unregistration parameters. */ typedef struct _NV_ENC_EVENT_PARAMS { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_EVENT_PARAMS_VER. */ uint32_t reserved; /**< [in]: Reserved and must be set to 0 */ void* completionEvent; /**< [in]: Handle to event to be registered/unregistered with the NvEncodeAPI interface. */ uint32_t reserved1[253]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_EVENT_PARAMS; /** Macro for constructing the version field of ::_NV_ENC_EVENT_PARAMS */ #define NV_ENC_EVENT_PARAMS_VER NVENCAPI_STRUCT_VERSION(1) /** * Encoder Session Creation parameters */ typedef struct _NV_ENC_OPEN_ENCODE_SESSIONEX_PARAMS { uint32_t version; /**< [in]: Struct version. Must be set to ::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER. */ NV_ENC_DEVICE_TYPE deviceType; /**< [in]: Specified the device Type */ void* device; /**< [in]: Pointer to client device. */ void* reserved; /**< [in]: Reserved and must be set to 0. */ uint32_t apiVersion; /**< [in]: API version. Should be set to NVENCAPI_VERSION. */ uint32_t reserved1[253]; /**< [in]: Reserved and must be set to 0 */ void* reserved2[64]; /**< [in]: Reserved and must be set to NULL */ } NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS; /** Macro for constructing the version field of ::_NV_ENC_OPEN_ENCODE_SESSIONEX_PARAMS */ #define NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER NVENCAPI_STRUCT_VERSION(1) /** @} */ /* END ENCODER_STRUCTURE */ /** * \addtogroup ENCODE_FUNC NvEncodeAPI Functions * @{ */ // NvEncOpenEncodeSession /** * \brief Opens an encoding session. * * Deprecated. * * \return * ::NV_ENC_ERR_INVALID_CALL\n * */ NVENCSTATUS NVENCAPI NvEncOpenEncodeSession(void* device, uint32_t deviceType, void** encoder); // NvEncGetEncodeGuidCount /** * \brief Retrieves the number of supported encode GUIDs. * * The function returns the number of codec GUIDs supported by the NvEncodeAPI * interface. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [out] encodeGUIDCount * Number of supported encode GUIDs. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodeGUIDCount(void* encoder, uint32_t* encodeGUIDCount); // NvEncGetEncodeGUIDs /** * \brief Retrieves an array of supported encoder codec GUIDs. * * The function returns an array of codec GUIDs supported by the NvEncodeAPI interface. * The client must allocate an array where the NvEncodeAPI interface can * fill the supported GUIDs and pass the pointer in \p *GUIDs parameter. * The size of the array can be determined by using ::NvEncGetEncodeGUIDCount() API. * The Nvidia Encoding interface returns the number of codec GUIDs it has actually * filled in the GUID array in the \p GUIDCount parameter. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] guidArraySize * Number of GUIDs to retrieved. Should be set to the number retrieved using * ::NvEncGetEncodeGUIDCount. * \param [out] GUIDs * Array of supported Encode GUIDs. * \param [out] GUIDCount * Number of supported Encode GUIDs. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodeGUIDs(void* encoder, GUID* GUIDs, uint32_t guidArraySize, uint32_t* GUIDCount); // NvEncGetEncodeProfileGuidCount /** * \brief Retrieves the number of supported profile GUIDs. * * The function returns the number of profile GUIDs supported for a given codec. * The client must first enumerate the codec GUIDs supported by the NvEncodeAPI * interface. After determining the codec GUID, it can query the NvEncodeAPI * interface to determine the number of profile GUIDs supported for a particular * codec GUID. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * The codec GUID for which the profile GUIDs are being enumerated. * \param [out] encodeProfileGUIDCount * Number of encode profiles supported for the given encodeGUID. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodeProfileGUIDCount(void* encoder, GUID encodeGUID, uint32_t* encodeProfileGUIDCount); // NvEncGetEncodeProfileGUIDs /** * \brief Retrieves an array of supported encode profile GUIDs. * * The function returns an array of supported profile GUIDs for a particular * codec GUID. The client must allocate an array where the NvEncodeAPI interface * can populate the profile GUIDs. The client can determine the array size using * ::NvEncGetEncodeProfileGUIDCount() API. The client must also validiate that the * NvEncodeAPI interface supports the GUID the client wants to pass as \p encodeGUID * parameter. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * The encode GUID whose profile GUIDs are being enumerated. * \param [in] guidArraySize * Number of GUIDs to be retrieved. Should be set to the number retrieved using * ::NvEncGetEncodeProfileGUIDCount. * \param [out] profileGUIDs * Array of supported Encode Profile GUIDs * \param [out] GUIDCount * Number of valid encode profile GUIDs in \p profileGUIDs array. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodeProfileGUIDs( void* encoder, GUID encodeGUID, GUID* profileGUIDs, uint32_t guidArraySize, uint32_t* GUIDCount ); // NvEncGetInputFormatCount /** * \brief Retrieve the number of supported Input formats. * * The function returns the number of supported input formats. The client must * query the NvEncodeAPI interface to determine the supported input formats * before creating the input surfaces. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the number of supported input formats * is to be retrieved. * \param [out] inputFmtCount * Number of input formats supported for specified Encode GUID. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n */ NVENCSTATUS NVENCAPI NvEncGetInputFormatCount(void* encoder, GUID encodeGUID, uint32_t* inputFmtCount); // NvEncGetInputFormats /** * \brief Retrieves an array of supported Input formats * * Returns an array of supported input formats The client must use the input * format to create input surface using ::NvEncCreateInputBuffer() API. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the number of supported input formats * is to be retrieved. *\param [in] inputFmtArraySize * Size input format count array passed in \p inputFmts. *\param [out] inputFmts * Array of input formats supported for this Encode GUID. *\param [out] inputFmtCount * The number of valid input format types returned by the NvEncodeAPI * interface in \p inputFmts array. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetInputFormats( void* encoder, GUID encodeGUID, NV_ENC_BUFFER_FORMAT* inputFmts, uint32_t inputFmtArraySize, uint32_t* inputFmtCount ); // NvEncGetEncodeCaps /** * \brief Retrieves the capability value for a specified encoder attribute. * * The function returns the capability value for a given encoder attribute. The * client must validate the encodeGUID using ::NvEncGetEncodeGUIDs() API before * calling this function. The encoder attribute being queried are enumerated in * ::NV_ENC_CAPS_PARAM enum. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the capability attribute is to be retrieved. * \param [in] capsParam * Used to specify attribute being queried. Refer ::NV_ENC_CAPS_PARAM for more * details. * \param [out] capsVal * The value corresponding to the capability attribute being queried. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n */ NVENCSTATUS NVENCAPI NvEncGetEncodeCaps(void* encoder, GUID encodeGUID, NV_ENC_CAPS_PARAM* capsParam, int* capsVal); // NvEncGetEncodePresetCount /** * \brief Retrieves the number of supported preset GUIDs. * * The function returns the number of preset GUIDs available for a given codec. * The client must validate the codec GUID using ::NvEncGetEncodeGUIDs() API * before calling this function. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the number of supported presets is to * be retrieved. * \param [out] encodePresetGUIDCount * Receives the number of supported preset GUIDs. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodePresetCount(void* encoder, GUID encodeGUID, uint32_t* encodePresetGUIDCount); // NvEncGetEncodePresetGUIDs /** * \brief Receives an array of supported encoder preset GUIDs. * * The function returns an array of encode preset GUIDs available for a given codec. * The client can directly use one of the preset GUIDs based upon the use case * or target device. The preset GUID chosen can be directly used in * NV_ENC_INITIALIZE_PARAMS::presetGUID parameter to ::NvEncEncodePicture() API. * Alternately client can also use the preset GUID to retrieve the encoding config * parameters being used by NvEncodeAPI interface for that given preset, using * ::NvEncGetEncodePresetConfig() API. It can then modify preset config parameters * as per its use case and send it to NvEncodeAPI interface as part of * NV_ENC_INITIALIZE_PARAMS::encodeConfig parameter for NvEncInitializeEncoder() * API. * * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the list of supported presets is to be * retrieved. * \param [in] guidArraySize * Size of array of preset GUIDs passed in \p preset GUIDs * \param [out] presetGUIDs * Array of supported Encode preset GUIDs from the NvEncodeAPI interface * to client. * \param [out] encodePresetGUIDCount * Receives the number of preset GUIDs returned by the NvEncodeAPI * interface. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodePresetGUIDs( void* encoder, GUID encodeGUID, GUID* presetGUIDs, uint32_t guidArraySize, uint32_t* encodePresetGUIDCount ); // NvEncGetEncodePresetConfig /** * \brief Returns a preset config structure supported for given preset GUID. * * The function returns a preset config structure for a given preset GUID. * NvEncGetEncodePresetConfig() API is not applicable to AV1. * Before using this function the client must enumerate the preset GUIDs available for * a given codec. The preset config structure can be modified by the client depending * upon its use case and can be then used to initialize the encoder using * ::NvEncInitializeEncoder() API. The client can use this function only if it * wants to modify the NvEncodeAPI preset configuration, otherwise it can * directly use the preset GUID. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the list of supported presets is to be * retrieved. * \param [in] presetGUID * Preset GUID, corresponding to which the Encoding configurations is to be * retrieved. * \param [out] presetConfig * The requested Preset Encoder Attribute set. Refer ::_NV_ENC_CONFIG for * more details. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodePresetConfig( void* encoder, GUID encodeGUID, GUID presetGUID, NV_ENC_PRESET_CONFIG* presetConfig ); // NvEncGetEncodePresetConfigEx /** * \brief Returns a preset config structure supported for given preset GUID. * * The function returns a preset config structure for a given preset GUID and tuning info. * NvEncGetEncodePresetConfigEx() API is not applicable to H264 and HEVC meonly mode. * Before using this function the client must enumerate the preset GUIDs available for * a given codec. The preset config structure can be modified by the client depending * upon its use case and can be then used to initialize the encoder using * ::NvEncInitializeEncoder() API. The client can use this function only if it * wants to modify the NvEncodeAPI preset configuration, otherwise it can * directly use the preset GUID. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encodeGUID * Encode GUID, corresponding to which the list of supported presets is to be * retrieved. * \param [in] presetGUID * Preset GUID, corresponding to which the Encoding configurations is to be * retrieved. * \param [in] tuningInfo * tuning info, corresponding to which the Encoding configurations is to be * retrieved. * \param [out] presetConfig * The requested Preset Encoder Attribute set. Refer ::_NV_ENC_CONFIG for * more details. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodePresetConfigEx( void* encoder, GUID encodeGUID, GUID presetGUID, NV_ENC_TUNING_INFO tuningInfo, NV_ENC_PRESET_CONFIG* presetConfig ); // NvEncInitializeEncoder /** * \brief Initialize the encoder. * * This API must be used to initialize the encoder. The initialization parameter * is passed using \p *createEncodeParams The client must send the following * fields of the _NV_ENC_INITIALIZE_PARAMS structure with a valid value. * - NV_ENC_INITIALIZE_PARAMS::encodeGUID * - NV_ENC_INITIALIZE_PARAMS::encodeWidth * - NV_ENC_INITIALIZE_PARAMS::encodeHeight * * The client can pass a preset GUID directly to the NvEncodeAPI interface using * NV_ENC_INITIALIZE_PARAMS::presetGUID field. If the client doesn't pass * NV_ENC_INITIALIZE_PARAMS::encodeConfig structure, the codec specific parameters * will be selected based on the preset GUID. The preset GUID must have been * validated by the client using ::NvEncGetEncodePresetGUIDs() API. * If the client passes a custom ::_NV_ENC_CONFIG structure through * NV_ENC_INITIALIZE_PARAMS::encodeConfig , it will override the codec specific parameters * based on the preset GUID. It is recommended that even if the client passes a custom config, * it should also send a preset GUID. In this case, the preset GUID passed by the client * will not override any of the custom config parameters programmed by the client, * it is only used as a hint by the NvEncodeAPI interface to determine certain encoder parameters * which are not exposed to the client. * * There are two modes of operation for the encoder namely: * - Asynchronous mode * - Synchronous mode * * The client can select asynchronous or synchronous mode by setting the \p * enableEncodeAsync field in ::_NV_ENC_INITIALIZE_PARAMS to 1 or 0 respectively. *\par Asynchronous mode of operation: * The Asynchronous mode can be enabled by setting NV_ENC_INITIALIZE_PARAMS::enableEncodeAsync to 1. * The client operating in asynchronous mode must allocate completion event object * for each output buffer and pass the completion event object in the * ::NvEncEncodePicture() API. The client can create another thread and wait on * the event object to be signaled by NvEncodeAPI interface on completion of the * encoding process for the output frame. This should unblock the main thread from * submitting work to the encoder. When the event is signaled the client can call * NvEncodeAPI interfaces to copy the bitstream data using ::NvEncLockBitstream() * API. This is the preferred mode of operation. * * NOTE: Asynchronous mode is not supported on Linux. * *\par Synchronous mode of operation: * The client can select synchronous mode by setting NV_ENC_INITIALIZE_PARAMS::enableEncodeAsync to *0. The client working in synchronous mode can work in a single threaded or multi threaded mode. *The client need not allocate any event objects. The client can only lock the bitstream data after *NvEncodeAPI interface has returned * ::NV_ENC_SUCCESS from encode picture. The NvEncodeAPI interface can return * ::NV_ENC_ERR_NEED_MORE_INPUT error code from ::NvEncEncodePicture() API. The * client must not lock the output buffer in such case but should send the next * frame for encoding. The client must keep on calling ::NvEncEncodePicture() API * until it returns ::NV_ENC_SUCCESS. \n * The client must always lock the bitstream data in order in which it has submitted. * This is true for both asynchronous and synchronous mode. * *\par Picture type decision: * If the client is taking the picture type decision and it must disable the picture * type decision module in NvEncodeAPI by setting NV_ENC_INITIALIZE_PARAMS::enablePTD * to 0. In this case the client is required to send the picture in encoding * order to NvEncodeAPI by doing the re-ordering for B frames. \n * If the client doesn't want to take the picture type decision it can enable * picture type decision module in the NvEncodeAPI interface by setting * NV_ENC_INITIALIZE_PARAMS::enablePTD to 1 and send the input pictures in display * order. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] createEncodeParams * Refer ::_NV_ENC_INITIALIZE_PARAMS for details. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncInitializeEncoder(void* encoder, NV_ENC_INITIALIZE_PARAMS* createEncodeParams); // NvEncCreateInputBuffer /** * \brief Allocates Input buffer. * * This function is used to allocate an input buffer. The client must enumerate * the input buffer format before allocating the input buffer resources. The * NV_ENC_INPUT_PTR returned by the NvEncodeAPI interface in the * NV_ENC_CREATE_INPUT_BUFFER::inputBuffer field can be directly used in * ::NvEncEncodePicture() API. The number of input buffers to be allocated by the * client must be at least 4 more than the number of B frames being used for encoding. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] createInputBufferParams * Pointer to the ::NV_ENC_CREATE_INPUT_BUFFER structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncCreateInputBuffer(void* encoder, NV_ENC_CREATE_INPUT_BUFFER* createInputBufferParams); // NvEncDestroyInputBuffer /** * \brief Release an input buffers. * * This function is used to free an input buffer. If the client has allocated * any input buffer using ::NvEncCreateInputBuffer() API, it must free those * input buffers by calling this function. The client must release the input * buffers before destroying the encoder using ::NvEncDestroyEncoder() API. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] inputBuffer * Pointer to the input buffer to be released. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncDestroyInputBuffer(void* encoder, NV_ENC_INPUT_PTR inputBuffer); // NvEncSetIOCudaStreams /** * \brief Set input and output CUDA stream for specified encoder attribute. * * Encoding may involve CUDA pre-processing on the input and post-processing on encoded output. * This function is used to set input and output CUDA streams to pipeline the CUDA pre-processing * and post-processing tasks. Clients should call this function before the call to * NvEncUnlockInputBuffer(). If this function is not called, the default CUDA stream is used for * input and output processing. After a successful call to this function, the streams specified * in that call will replace the previously-used streams. * This API is supported for NVCUVID interface only. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] inputStream * Pointer to CUstream which is used to process ::NV_ENC_PIC_PARAMS::inputFrame for encode. * In case of ME-only mode, inputStream is used to process ::NV_ENC_MEONLY_PARAMS::inputBuffer and * ::NV_ENC_MEONLY_PARAMS::referenceFrame * \param [in] outputStream * Pointer to CUstream which is used to process ::NV_ENC_PIC_PARAMS::outputBuffer for encode. * In case of ME-only mode, outputStream is used to process ::NV_ENC_MEONLY_PARAMS::mvBuffer * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n */ NVENCSTATUS NVENCAPI NvEncSetIOCudaStreams( void* encoder, NV_ENC_CUSTREAM_PTR inputStream, NV_ENC_CUSTREAM_PTR outputStream ); // NvEncCreateBitstreamBuffer /** * \brief Allocates an output bitstream buffer * * This function is used to allocate an output bitstream buffer and returns a * NV_ENC_OUTPUT_PTR to bitstream buffer to the client in the * NV_ENC_CREATE_BITSTREAM_BUFFER::bitstreamBuffer field. * The client can only call this function after the encoder session has been * initialized using ::NvEncInitializeEncoder() API. The minimum number of output * buffers allocated by the client must be at least 4 more than the number of B * B frames being used for encoding. The client can only access the output * bitstream data by locking the \p bitstreamBuffer using the ::NvEncLockBitstream() * function. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] createBitstreamBufferParams * Pointer ::NV_ENC_CREATE_BITSTREAM_BUFFER for details. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncCreateBitstreamBuffer( void* encoder, NV_ENC_CREATE_BITSTREAM_BUFFER* createBitstreamBufferParams ); // NvEncDestroyBitstreamBuffer /** * \brief Release a bitstream buffer. * * This function is used to release the output bitstream buffer allocated using * the ::NvEncCreateBitstreamBuffer() function. The client must release the output * bitstreamBuffer using this function before destroying the encoder session. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] bitstreamBuffer * Pointer to the bitstream buffer being released. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncDestroyBitstreamBuffer(void* encoder, NV_ENC_OUTPUT_PTR bitstreamBuffer); // NvEncEncodePicture /** * \brief Submit an input picture for encoding. * * This function is used to submit an input picture buffer for encoding. The * encoding parameters are passed using \p *encodePicParams which is a pointer * to the ::_NV_ENC_PIC_PARAMS structure. * * If the client has set NV_ENC_INITIALIZE_PARAMS::enablePTD to 0, then it must * send a valid value for the following fields. * - NV_ENC_PIC_PARAMS::pictureType * - NV_ENC_PIC_PARAMS_H264::displayPOCSyntax (H264 only) * - NV_ENC_PIC_PARAMS_H264::frameNumSyntax(H264 only) * - NV_ENC_PIC_PARAMS_H264::refPicFlag(H264 only) * *\par MVC Encoding: * For MVC encoding the client must call encode picture API for each view separately * and must pass valid view id in NV_ENC_PIC_PARAMS_MVC::viewID field. Currently * NvEncodeAPI only support stereo MVC so client must send viewID as 0 for base * view and view ID as 1 for dependent view. * *\par Asynchronous Encoding * If the client has enabled asynchronous mode of encoding by setting * NV_ENC_INITIALIZE_PARAMS::enableEncodeAsync to 1 in the ::NvEncInitializeEncoder() * API ,then the client must send a valid NV_ENC_PIC_PARAMS::completionEvent. * Incase of asynchronous mode of operation, client can queue the ::NvEncEncodePicture() * API commands from the main thread and then queue output buffers to be processed * to a secondary worker thread. Before the locking the output buffers in the * secondary thread , the client must wait on NV_ENC_PIC_PARAMS::completionEvent * it has queued in ::NvEncEncodePicture() API call. The client must always process * completion event and the output buffer in the same order in which they have been * submitted for encoding. The NvEncodeAPI interface is responsible for any * re-ordering required for B frames and will always ensure that encoded bitstream * data is written in the same order in which output buffer is submitted. * The NvEncodeAPI interface may return ::NV_ENC_ERR_NEED_MORE_INPUT error code for * some ::NvEncEncodePicture() API calls but the client must not treat it as a fatal error. * The NvEncodeAPI interface might not be able to submit an input picture buffer for encoding * immediately due to re-ordering for B frames. *\code The below example shows how asynchronous encoding in case of 1 B frames ------------------------------------------------------------------------ Suppose the client allocated 4 input buffers(I1,I2..), 4 output buffers(O1,O2..) and 4 completion events(E1, E2, ...). The NvEncodeAPI interface will need to keep a copy of the input buffers for re-ordering and it allocates following internal buffers (NvI1, NvI2...). These internal buffers are managed by NvEncodeAPI and the client is not responsible for the allocating or freeing the memory of the internal buffers. a) The client main thread will queue the following encode frame calls. Note the picture type is unknown to the client, the decision is being taken by NvEncodeAPI interface. The client should pass ::_NV_ENC_PIC_PARAMS parameter consisting of allocated input buffer, output buffer and output events in successive ::NvEncEncodePicture() API calls along with other required encode picture params. For example: 1st EncodePicture parameters - (I1, O1, E1) 2nd EncodePicture parameters - (I2, O2, E2) 3rd EncodePicture parameters - (I3, O3, E3) b) NvEncodeAPI SW will receive the following encode Commands from the client. The left side shows input from client in the form (Input buffer, Output Buffer, Output Event). The right hand side shows a possible picture type decision take by the NvEncodeAPI interface. (I1, O1, E1) ---P1 Frame (I2, O2, E2) ---B2 Frame (I3, O3, E3) ---P3 Frame c) NvEncodeAPI interface will make a copy of the input buffers to its internal buffers for re-ordering. These copies are done as part of nvEncEncodePicture function call from the client and NvEncodeAPI interface is responsible for synchronization of copy operation with the actual encoding operation. I1 --> NvI1 I2 --> NvI2 I3 --> NvI3 d) The NvEncodeAPI encodes I1 as P frame and submits I1 to encoder HW and returns ::NV_ENC_SUCCESS. The NvEncodeAPI tries to encode I2 as B frame and fails with ::NV_ENC_ERR_NEED_MORE_INPUT error code. The error is not fatal and it notifies client that I2 is not submitted to encoder immediately. The NvEncodeAPI encodes I3 as P frame and submits I3 for encoding which will be used as backward reference frame for I2. The NvEncodeAPI then submits I2 for encoding and returns ::NV_ENC_SUCESS. Both the submission are part of the same ::NvEncEncodePicture() function call. e) After returning from ::NvEncEncodePicture() call , the client must queue the output bitstream processing work to the secondary thread. The output bitstream processing for asynchronous mode consist of first waiting on completion event(E1, E2..) and then locking the output bitstream buffer(O1, O2..) for reading the encoded data. The work queued to the secondary thread by the client is in the following order (I1, O1, E1) (I2, O2, E2) (I3, O3, E3) Note they are in the same order in which client calls ::NvEncEncodePicture() API in \p step a). f) NvEncodeAPI interface will do the re-ordering such that Encoder HW will receive the following encode commands: (NvI1, O1, E1) ---P1 Frame (NvI3, O2, E2) ---P3 Frame (NvI2, O3, E3) ---B2 frame g) After the encoding operations are completed, the events will be signaled by NvEncodeAPI interface in the following order : (O1, E1) ---P1 Frame ,output bitstream copied to O1 and event E1 signaled. (O2, E2) ---P3 Frame ,output bitstream copied to O2 and event E2 signaled. (O3, E3) ---B2 Frame ,output bitstream copied to O3 and event E3 signaled. h) The client must lock the bitstream data using ::NvEncLockBitstream() API in the order O1,O2,O3 to read the encoded data, after waiting for the events to be signaled in the same order i.e E1, E2 and E3.The output processing is done in the secondary thread in the following order: Waits on E1, copies encoded bitstream from O1 Waits on E2, copies encoded bitstream from O2 Waits on E3, copies encoded bitstream from O3 -Note the client will receive the events signaling and output buffer in the same order in which they have submitted for encoding. -Note the LockBitstream will have picture type field which will notify the output picture type to the clients. -Note the input, output buffer and the output completion event are free to be reused once NvEncodeAPI interfaced has signaled the event and the client has copied the data from the output buffer. * \endcode * *\par Synchronous Encoding * The client can enable synchronous mode of encoding by setting * NV_ENC_INITIALIZE_PARAMS::enableEncodeAsync to 0 in ::NvEncInitializeEncoder() API. * The NvEncodeAPI interface may return ::NV_ENC_ERR_NEED_MORE_INPUT error code for * some ::NvEncEncodePicture() API calls when NV_ENC_INITIALIZE_PARAMS::enablePTD * is set to 1, but the client must not treat it as a fatal error. The NvEncodeAPI * interface might not be able to submit an input picture buffer for encoding * immediately due to re-ordering for B frames. The NvEncodeAPI interface cannot * submit the input picture which is decided to be encoded as B frame as it waits * for backward reference from temporally subsequent frames. This input picture * is buffered internally and waits for more input picture to arrive. The client * must not call ::NvEncLockBitstream() API on the output buffers whose * ::NvEncEncodePicture() API returns ::NV_ENC_ERR_NEED_MORE_INPUT. The client must * wait for the NvEncodeAPI interface to return ::NV_ENC_SUCCESS before locking the * output bitstreams to read the encoded bitstream data. The following example * explains the scenario with synchronous encoding with 2 B frames. *\code The below example shows how synchronous encoding works in case of 1 B frames ----------------------------------------------------------------------------- Suppose the client allocated 4 input buffers(I1,I2..), 4 output buffers(O1,O2..) and 4 completion events(E1, E2, ...). The NvEncodeAPI interface will need to keep a copy of the input buffers for re-ordering and it allocates following internal buffers (NvI1, NvI2...). These internal buffers are managed by NvEncodeAPI and the client is not responsible for the allocating or freeing the memory of the internal buffers. The client calls ::NvEncEncodePicture() API with input buffer I1 and output buffer O1. The NvEncodeAPI decides to encode I1 as P frame and submits it to encoder HW and returns ::NV_ENC_SUCCESS. The client can now read the encoded data by locking the output O1 by calling NvEncLockBitstream API. The client calls ::NvEncEncodePicture() API with input buffer I2 and output buffer O2. The NvEncodeAPI decides to encode I2 as B frame and buffers I2 by copying it to internal buffer and returns ::NV_ENC_ERR_NEED_MORE_INPUT. The error is not fatal and it notifies client that it cannot read the encoded data by locking the output O2 by calling ::NvEncLockBitstream() API without submitting more work to the NvEncodeAPI interface. The client calls ::NvEncEncodePicture() with input buffer I3 and output buffer O3. The NvEncodeAPI decides to encode I3 as P frame and it first submits I3 for encoding which will be used as backward reference frame for I2. The NvEncodeAPI then submits I2 for encoding and returns ::NV_ENC_SUCESS. Both the submission are part of the same ::NvEncEncodePicture() function call. The client can now read the encoded data for both the frames by locking the output O2 followed by O3 ,by calling ::NvEncLockBitstream() API. The client must always lock the output in the same order in which it has submitted to receive the encoded bitstream in correct encoding order. * \endcode * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] encodePicParams * Pointer to the ::_NV_ENC_PIC_PARAMS structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_ENCODER_BUSY \n * ::NV_ENC_ERR_NEED_MORE_INPUT \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncEncodePicture(void* encoder, NV_ENC_PIC_PARAMS* encodePicParams); // NvEncLockBitstream /** * \brief Lock output bitstream buffer * * This function is used to lock the bitstream buffer to read the encoded data. * The client can only access the encoded data by calling this function. * The pointer to client accessible encoded data is returned in the * NV_ENC_LOCK_BITSTREAM::bitstreamBufferPtr field. The size of the encoded data * in the output buffer is returned in the NV_ENC_LOCK_BITSTREAM::bitstreamSizeInBytes * The NvEncodeAPI interface also returns the output picture type and picture structure * of the encoded frame in NV_ENC_LOCK_BITSTREAM::pictureType and * NV_ENC_LOCK_BITSTREAM::pictureStruct fields respectively. If the client has * set NV_ENC_LOCK_BITSTREAM::doNotWait to 1, the function might return * ::NV_ENC_ERR_LOCK_BUSY if client is operating in synchronous mode. This is not * a fatal failure if NV_ENC_LOCK_BITSTREAM::doNotWait is set to 1. In the above case the client can * retry the function after few milliseconds. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] lockBitstreamBufferParams * Pointer to the ::_NV_ENC_LOCK_BITSTREAM structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_LOCK_BUSY \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncLockBitstream(void* encoder, NV_ENC_LOCK_BITSTREAM* lockBitstreamBufferParams); // NvEncUnlockBitstream /** * \brief Unlock the output bitstream buffer * * This function is used to unlock the output bitstream buffer after the client * has read the encoded data from output buffer. The client must call this function * to unlock the output buffer which it has previously locked using ::NvEncLockBitstream() * function. Using a locked bitstream buffer in ::NvEncEncodePicture() API will cause * the function to fail. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] bitstreamBuffer * bitstream buffer pointer being unlocked * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncUnlockBitstream(void* encoder, NV_ENC_OUTPUT_PTR bitstreamBuffer); // NvLockInputBuffer /** * \brief Locks an input buffer * * This function is used to lock the input buffer to load the uncompressed YUV * pixel data into input buffer memory. The client must pass the NV_ENC_INPUT_PTR * it had previously allocated using ::NvEncCreateInputBuffer()in the * NV_ENC_LOCK_INPUT_BUFFER::inputBuffer field. * The NvEncodeAPI interface returns pointer to client accessible input buffer * memory in NV_ENC_LOCK_INPUT_BUFFER::bufferDataPtr field. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] lockInputBufferParams * Pointer to the ::_NV_ENC_LOCK_INPUT_BUFFER structure * * \return * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_LOCK_BUSY \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncLockInputBuffer(void* encoder, NV_ENC_LOCK_INPUT_BUFFER* lockInputBufferParams); // NvUnlockInputBuffer /** * \brief Unlocks the input buffer * * This function is used to unlock the input buffer memory previously locked for * uploading YUV pixel data. The input buffer must be unlocked before being used * again for encoding, otherwise NvEncodeAPI will fail the ::NvEncEncodePicture() * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] inputBuffer * Pointer to the input buffer that is being unlocked. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * * */ NVENCSTATUS NVENCAPI NvEncUnlockInputBuffer(void* encoder, NV_ENC_INPUT_PTR inputBuffer); // NvEncGetEncodeStats /** * \brief Get encoding statistics. * * This function is used to retrieve the encoding statistics. * This API is not supported when encode device type is CUDA. * Note that this API will be removed in future Video Codec SDK release. * Clients should use NvEncLockBitstream() API to retrieve the encoding statistics. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] encodeStats * Pointer to the ::_NV_ENC_STAT structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetEncodeStats(void* encoder, NV_ENC_STAT* encodeStats); // NvEncGetSequenceParams /** * \brief Get encoded sequence and picture header. * * This function can be used to retrieve the sequence and picture header out of * band. The client must call this function only after the encoder has been * initialized using ::NvEncInitializeEncoder() function. The client must * allocate the memory where the NvEncodeAPI interface can copy the bitstream * header and pass the pointer to the memory in NV_ENC_SEQUENCE_PARAM_PAYLOAD::spsppsBuffer. * The size of buffer is passed in the field NV_ENC_SEQUENCE_PARAM_PAYLOAD::inBufferSize. * The NvEncodeAPI interface will copy the bitstream header payload and returns * the actual size of the bitstream header in the field * NV_ENC_SEQUENCE_PARAM_PAYLOAD::outSPSPPSPayloadSize. * The client must call ::NvEncGetSequenceParams() function from the same thread which is * being used to call ::NvEncEncodePicture() function. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] sequenceParamPayload * Pointer to the ::_NV_ENC_SEQUENCE_PARAM_PAYLOAD structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetSequenceParams(void* encoder, NV_ENC_SEQUENCE_PARAM_PAYLOAD* sequenceParamPayload); // NvEncGetSequenceParamEx /** * \brief Get sequence and picture header. * * This function can be used to retrieve the sequence and picture header out of band, even when * encoder has not been initialized using ::NvEncInitializeEncoder() function. * The client must allocate the memory where the NvEncodeAPI interface can copy the bitstream * header and pass the pointer to the memory in NV_ENC_SEQUENCE_PARAM_PAYLOAD::spsppsBuffer. * The size of buffer is passed in the field NV_ENC_SEQUENCE_PARAM_PAYLOAD::inBufferSize. * If encoder has not been initialized using ::NvEncInitializeEncoder() function, client must * send NV_ENC_INITIALIZE_PARAMS as input. The NV_ENC_INITIALIZE_PARAMS passed must be same as the * one which will be used for initializing encoder using ::NvEncInitializeEncoder() function later. * If encoder is already initialized using ::NvEncInitializeEncoder() function, the provided * NV_ENC_INITIALIZE_PARAMS structure is ignored. The NvEncodeAPI interface will copy the bitstream * header payload and returns the actual size of the bitstream header in the field * NV_ENC_SEQUENCE_PARAM_PAYLOAD::outSPSPPSPayloadSize. The client must call * ::NvEncGetSequenceParamsEx() function from the same thread which is being used to call * ::NvEncEncodePicture() function. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] encInitParams * Pointer to the _NV_ENC_INITIALIZE_PARAMS structure. * \param [in,out] sequenceParamPayload * Pointer to the ::_NV_ENC_SEQUENCE_PARAM_PAYLOAD structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncGetSequenceParamEx( void* encoder, NV_ENC_INITIALIZE_PARAMS* encInitParams, NV_ENC_SEQUENCE_PARAM_PAYLOAD* sequenceParamPayload ); // NvEncRegisterAsyncEvent /** * \brief Register event for notification to encoding completion. * * This function is used to register the completion event with NvEncodeAPI * interface. The event is required when the client has configured the encoder to * work in asynchronous mode. In this mode the client needs to send a completion * event with every output buffer. The NvEncodeAPI interface will signal the * completion of the encoding process using this event. Only after the event is * signaled the client can get the encoded data using ::NvEncLockBitstream() function. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] eventParams * Pointer to the ::_NV_ENC_EVENT_PARAMS structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncRegisterAsyncEvent(void* encoder, NV_ENC_EVENT_PARAMS* eventParams); // NvEncUnregisterAsyncEvent /** * \brief Unregister completion event. * * This function is used to unregister completion event which has been previously * registered using ::NvEncRegisterAsyncEvent() function. The client must unregister * all events before destroying the encoder using ::NvEncDestroyEncoder() function. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] eventParams * Pointer to the ::_NV_ENC_EVENT_PARAMS structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncUnregisterAsyncEvent(void* encoder, NV_ENC_EVENT_PARAMS* eventParams); // NvEncMapInputResource /** * \brief Map an externally created input resource pointer for encoding. * * Maps an externally allocated input resource [using and returns a NV_ENC_INPUT_PTR * which can be used for encoding in the ::NvEncEncodePicture() function. The * mapped resource is returned in the field NV_ENC_MAP_INPUT_RESOURCE::outputResourcePtr. * The NvEncodeAPI interface also returns the buffer format of the mapped resource * in the field NV_ENC_MAP_INPUT_RESOURCE::outbufferFmt. * This function provides synchronization guarantee that any graphics work submitted * on the input buffer is completed before the buffer is used for encoding. This is * also true for compute (i.e. CUDA) work, provided that the previous workload using * the input resource was submitted to the default stream. * The client should not access any input buffer while they are mapped by the encoder. * For D3D12 interface type, this function does not provide synchronization guarantee. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] mapInputResParams * Pointer to the ::_NV_ENC_MAP_INPUT_RESOURCE structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_RESOURCE_NOT_REGISTERED \n * ::NV_ENC_ERR_MAP_FAILED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncMapInputResource(void* encoder, NV_ENC_MAP_INPUT_RESOURCE* mapInputResParams); // NvEncUnmapInputResource /** * \brief UnMaps a NV_ENC_INPUT_PTR which was mapped for encoding * * * UnMaps an input buffer which was previously mapped using ::NvEncMapInputResource() * API. The mapping created using ::NvEncMapInputResource() should be invalidated * using this API before the external resource is destroyed by the client. The client * must unmap the buffer after ::NvEncLockBitstream() API returns successfully for encode * work submitted using the mapped input buffer. * * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] mappedInputBuffer * Pointer to the NV_ENC_INPUT_PTR * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_RESOURCE_NOT_REGISTERED \n * ::NV_ENC_ERR_RESOURCE_NOT_MAPPED \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncUnmapInputResource(void* encoder, NV_ENC_INPUT_PTR mappedInputBuffer); // NvEncDestroyEncoder /** * \brief Destroy Encoding Session * * Destroys the encoder session previously created using ::NvEncOpenEncodeSession() * function. The client must flush the encoder before freeing any resources. In order * to flush the encoder the client must pass a NULL encode picture packet and either * wait for the ::NvEncEncodePicture() function to return in synchronous mode or wait * for the flush event to be signaled by the encoder in asynchronous mode. * The client must free all the input and output resources created using the * NvEncodeAPI interface before destroying the encoder. If the client is operating * in asynchronous mode, it must also unregister the completion events previously * registered. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncDestroyEncoder(void* encoder); // NvEncInvalidateRefFrames /** * \brief Invalidate reference frames * * Invalidates reference frame based on the time stamp provided by the client. * The encoder marks any reference frames or any frames which have been reconstructed * using the corrupt frame as invalid for motion estimation and uses older reference * frames for motion estimation. The encoder forces the current frame to be encoded * as an intra frame if no reference frames are left after invalidation process. * This is useful for low latency application for error resiliency. The client * is recommended to set NV_ENC_CONFIG_H264::maxNumRefFrames to a large value so * that encoder can keep a backup of older reference frames in the DPB and can use them * for motion estimation when the newer reference frames have been invalidated. * This API can be called multiple times. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] invalidRefFrameTimeStamp * Timestamp of the invalid reference frames which needs to be invalidated. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncInvalidateRefFrames(void* encoder, uint64_t invalidRefFrameTimeStamp); // NvEncOpenEncodeSessionEx /** * \brief Opens an encoding session. * * Opens an encoding session and returns a pointer to the encoder interface in * the \p **encoder parameter. The client should start encoding process by calling * this API first. * The client must pass a pointer to IDirect3DDevice9 device or CUDA context in the \p *device * parameter. For the OpenGL interface, \p device must be NULL. An OpenGL context must be current * when calling all NvEncodeAPI functions. If the creation of encoder session fails, the client must * call ::NvEncDestroyEncoder API before exiting. * * \param [in] openSessionExParams * Pointer to a ::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS structure. * \param [out] encoder * Encode Session pointer to the NvEncodeAPI interface. * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_NO_ENCODE_DEVICE \n * ::NV_ENC_ERR_UNSUPPORTED_DEVICE \n * ::NV_ENC_ERR_INVALID_DEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncOpenEncodeSessionEx(NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS* openSessionExParams, void** encoder); // NvEncRegisterResource /** * \brief Registers a resource with the Nvidia Video Encoder Interface. * * Registers a resource with the Nvidia Video Encoder Interface for book keeping. * The client is expected to pass the registered resource handle as well, while calling * ::NvEncMapInputResource API. * * \param [in] encoder * Pointer to the NVEncodeAPI interface. * * \param [in] registerResParams * Pointer to a ::_NV_ENC_REGISTER_RESOURCE structure * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_RESOURCE_REGISTER_FAILED \n * ::NV_ENC_ERR_GENERIC \n * ::NV_ENC_ERR_UNIMPLEMENTED \n * */ NVENCSTATUS NVENCAPI NvEncRegisterResource(void* encoder, NV_ENC_REGISTER_RESOURCE* registerResParams); // NvEncUnregisterResource /** * \brief Unregisters a resource previously registered with the Nvidia Video Encoder Interface. * * Unregisters a resource previously registered with the Nvidia Video Encoder Interface. * The client is expected to unregister any resource that it has registered with the * Nvidia Video Encoder Interface before destroying the resource. * * \param [in] encoder * Pointer to the NVEncodeAPI interface. * * \param [in] registeredResource * The registered resource pointer that was returned in ::NvEncRegisterResource. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_RESOURCE_NOT_REGISTERED \n * ::NV_ENC_ERR_GENERIC \n * ::NV_ENC_ERR_UNIMPLEMENTED \n * */ NVENCSTATUS NVENCAPI NvEncUnregisterResource(void* encoder, NV_ENC_REGISTERED_PTR registeredResource); // NvEncReconfigureEncoder /** * \brief Reconfigure an existing encoding session. * * Reconfigure an existing encoding session. * The client should call this API to change/reconfigure the parameter passed during * NvEncInitializeEncoder API call. * Currently Reconfiguration of following are not supported. * Change in GOP structure. * Change in sync-Async mode. * Change in MaxWidth & MaxHeight. * Change in PTD mode. * * Resolution change is possible only if maxEncodeWidth & maxEncodeHeight of * NV_ENC_INITIALIZE_PARAMS is set while creating encoder session. * * \param [in] encoder * Pointer to the NVEncodeAPI interface. * * \param [in] reInitEncodeParams * Pointer to a ::NV_ENC_RECONFIGURE_PARAMS structure. * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_NO_ENCODE_DEVICE \n * ::NV_ENC_ERR_UNSUPPORTED_DEVICE \n * ::NV_ENC_ERR_INVALID_DEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_GENERIC \n * */ NVENCSTATUS NVENCAPI NvEncReconfigureEncoder(void* encoder, NV_ENC_RECONFIGURE_PARAMS* reInitEncodeParams); // NvEncCreateMVBuffer /** * \brief Allocates output MV buffer for ME only mode. * * This function is used to allocate an output MV buffer. The size of the mvBuffer is * dependent on the frame height and width of the last ::NvEncCreateInputBuffer() call. * The NV_ENC_OUTPUT_PTR returned by the NvEncodeAPI interface in the * ::NV_ENC_CREATE_MV_BUFFER::mvBuffer field should be used in * ::NvEncRunMotionEstimationOnly() API. * Client must lock ::NV_ENC_CREATE_MV_BUFFER::mvBuffer using ::NvEncLockBitstream() API to get the * motion vector data. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in,out] createMVBufferParams * Pointer to the ::NV_ENC_CREATE_MV_BUFFER structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_GENERIC \n */ NVENCSTATUS NVENCAPI NvEncCreateMVBuffer(void* encoder, NV_ENC_CREATE_MV_BUFFER* createMVBufferParams); // NvEncDestroyMVBuffer /** * \brief Release an output MV buffer for ME only mode. * * This function is used to release the output MV buffer allocated using * the ::NvEncCreateMVBuffer() function. The client must release the output * mvBuffer using this function before destroying the encoder session. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] mvBuffer * Pointer to the mvBuffer being released. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n */ NVENCSTATUS NVENCAPI NvEncDestroyMVBuffer(void* encoder, NV_ENC_OUTPUT_PTR mvBuffer); // NvEncRunMotionEstimationOnly /** * \brief Submit an input picture and reference frame for motion estimation in ME only mode. * * This function is used to submit the input frame and reference frame for motion * estimation. The ME parameters are passed using *meOnlyParams which is a pointer * to ::_NV_ENC_MEONLY_PARAMS structure. * Client must lock ::NV_ENC_CREATE_MV_BUFFER::mvBuffer using ::NvEncLockBitstream() API to get the * motion vector data. to get motion vector data. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * \param [in] meOnlyParams * Pointer to the ::_NV_ENC_MEONLY_PARAMS structure. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n * ::NV_ENC_ERR_INVALID_ENCODERDEVICE \n * ::NV_ENC_ERR_DEVICE_NOT_EXIST \n * ::NV_ENC_ERR_UNSUPPORTED_PARAM \n * ::NV_ENC_ERR_OUT_OF_MEMORY \n * ::NV_ENC_ERR_INVALID_PARAM \n * ::NV_ENC_ERR_INVALID_VERSION \n * ::NV_ENC_ERR_NEED_MORE_INPUT \n * ::NV_ENC_ERR_ENCODER_NOT_INITIALIZED \n * ::NV_ENC_ERR_GENERIC \n */ NVENCSTATUS NVENCAPI NvEncRunMotionEstimationOnly(void* encoder, NV_ENC_MEONLY_PARAMS* meOnlyParams); // NvEncodeAPIGetMaxSupportedVersion /** * \brief Get the largest NvEncodeAPI version supported by the driver. * * This function can be used by clients to determine if the driver supports * the NvEncodeAPI header the application was compiled with. * * \param [out] version * Pointer to the requested value. The 4 least significant bits in the returned * indicate the minor version and the rest of the bits indicate the major * version of the largest supported version. * * \return * ::NV_ENC_SUCCESS \n * ::NV_ENC_ERR_INVALID_PTR \n */ NVENCSTATUS NVENCAPI NvEncodeAPIGetMaxSupportedVersion(uint32_t* version); // NvEncGetLastErrorString /** * \brief Get the description of the last error reported by the API. * * This function returns a null-terminated string that can be used by clients to better understand * the reason for failure of a previous API call. * * \param [in] encoder * Pointer to the NvEncodeAPI interface. * * \return * Pointer to buffer containing the details of the last error encountered by the API. */ const char* NVENCAPI NvEncGetLastErrorString(void* encoder); /// \cond API PFN /* * Defines API function pointers */ typedef NVENCSTATUS(NVENCAPI* PNVENCOPENENCODESESSION)( void* device, uint32_t deviceType, void** encoder ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEGUIDCOUNT)(void* encoder, uint32_t* encodeGUIDCount); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEGUIDS)( void* encoder, GUID* GUIDs, uint32_t guidArraySize, uint32_t* GUIDCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEPROFILEGUIDCOUNT)( void* encoder, GUID encodeGUID, uint32_t* encodeProfileGUIDCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEPROFILEGUIDS)( void* encoder, GUID encodeGUID, GUID* profileGUIDs, uint32_t guidArraySize, uint32_t* GUIDCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETINPUTFORMATCOUNT)( void* encoder, GUID encodeGUID, uint32_t* inputFmtCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETINPUTFORMATS)( void* encoder, GUID encodeGUID, NV_ENC_BUFFER_FORMAT* inputFmts, uint32_t inputFmtArraySize, uint32_t* inputFmtCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODECAPS)( void* encoder, GUID encodeGUID, NV_ENC_CAPS_PARAM* capsParam, int* capsVal ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEPRESETCOUNT)( void* encoder, GUID encodeGUID, uint32_t* encodePresetGUIDCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEPRESETGUIDS)( void* encoder, GUID encodeGUID, GUID* presetGUIDs, uint32_t guidArraySize, uint32_t* encodePresetGUIDCount ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEPRESETCONFIG)( void* encoder, GUID encodeGUID, GUID presetGUID, NV_ENC_PRESET_CONFIG* presetConfig ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODEPRESETCONFIGEX)( void* encoder, GUID encodeGUID, GUID presetGUID, NV_ENC_TUNING_INFO tuningInfo, NV_ENC_PRESET_CONFIG* presetConfig ); typedef NVENCSTATUS(NVENCAPI* PNVENCINITIALIZEENCODER)( void* encoder, NV_ENC_INITIALIZE_PARAMS* createEncodeParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCCREATEINPUTBUFFER)( void* encoder, NV_ENC_CREATE_INPUT_BUFFER* createInputBufferParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCDESTROYINPUTBUFFER)( void* encoder, NV_ENC_INPUT_PTR inputBuffer ); typedef NVENCSTATUS(NVENCAPI* PNVENCCREATEBITSTREAMBUFFER)( void* encoder, NV_ENC_CREATE_BITSTREAM_BUFFER* createBitstreamBufferParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCDESTROYBITSTREAMBUFFER)( void* encoder, NV_ENC_OUTPUT_PTR bitstreamBuffer ); typedef NVENCSTATUS(NVENCAPI* PNVENCENCODEPICTURE)( void* encoder, NV_ENC_PIC_PARAMS* encodePicParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCLOCKBITSTREAM)( void* encoder, NV_ENC_LOCK_BITSTREAM* lockBitstreamBufferParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCUNLOCKBITSTREAM)( void* encoder, NV_ENC_OUTPUT_PTR bitstreamBuffer ); typedef NVENCSTATUS(NVENCAPI* PNVENCLOCKINPUTBUFFER)( void* encoder, NV_ENC_LOCK_INPUT_BUFFER* lockInputBufferParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCUNLOCKINPUTBUFFER)(void* encoder, NV_ENC_INPUT_PTR inputBuffer); typedef NVENCSTATUS(NVENCAPI* PNVENCGETENCODESTATS)(void* encoder, NV_ENC_STAT* encodeStats); typedef NVENCSTATUS(NVENCAPI* PNVENCGETSEQUENCEPARAMS)( void* encoder, NV_ENC_SEQUENCE_PARAM_PAYLOAD* sequenceParamPayload ); typedef NVENCSTATUS(NVENCAPI* PNVENCREGISTERASYNCEVENT)( void* encoder, NV_ENC_EVENT_PARAMS* eventParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCUNREGISTERASYNCEVENT)( void* encoder, NV_ENC_EVENT_PARAMS* eventParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCMAPINPUTRESOURCE)( void* encoder, NV_ENC_MAP_INPUT_RESOURCE* mapInputResParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCUNMAPINPUTRESOURCE)( void* encoder, NV_ENC_INPUT_PTR mappedInputBuffer ); typedef NVENCSTATUS(NVENCAPI* PNVENCDESTROYENCODER)(void* encoder); typedef NVENCSTATUS(NVENCAPI* PNVENCINVALIDATEREFFRAMES)( void* encoder, uint64_t invalidRefFrameTimeStamp ); typedef NVENCSTATUS(NVENCAPI* PNVENCOPENENCODESESSIONEX)( NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS* openSessionExParams, void** encoder ); typedef NVENCSTATUS(NVENCAPI* PNVENCREGISTERRESOURCE)( void* encoder, NV_ENC_REGISTER_RESOURCE* registerResParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCUNREGISTERRESOURCE)( void* encoder, NV_ENC_REGISTERED_PTR registeredRes ); typedef NVENCSTATUS(NVENCAPI* PNVENCRECONFIGUREENCODER)( void* encoder, NV_ENC_RECONFIGURE_PARAMS* reInitEncodeParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCCREATEMVBUFFER)( void* encoder, NV_ENC_CREATE_MV_BUFFER* createMVBufferParams ); typedef NVENCSTATUS(NVENCAPI* PNVENCDESTROYMVBUFFER)(void* encoder, NV_ENC_OUTPUT_PTR mvBuffer); typedef NVENCSTATUS(NVENCAPI* PNVENCRUNMOTIONESTIMATIONONLY)( void* encoder, NV_ENC_MEONLY_PARAMS* meOnlyParams ); typedef const char*(NVENCAPI* PNVENCGETLASTERROR)(void* encoder); typedef NVENCSTATUS(NVENCAPI* PNVENCSETIOCUDASTREAMS)( void* encoder, NV_ENC_CUSTREAM_PTR inputStream, NV_ENC_CUSTREAM_PTR outputStream ); typedef NVENCSTATUS(NVENCAPI* PNVENCGETSEQUENCEPARAMEX)( void* encoder, NV_ENC_INITIALIZE_PARAMS* encInitParams, NV_ENC_SEQUENCE_PARAM_PAYLOAD* sequenceParamPayload ); /// \endcond /** @} */ /* END ENCODE_FUNC */ /** * \ingroup ENCODER_STRUCTURE * NV_ENCODE_API_FUNCTION_LIST */ typedef struct _NV_ENCODE_API_FUNCTION_LIST { uint32_t version; /**< [in]: Client should pass NV_ENCODE_API_FUNCTION_LIST_VER. */ uint32_t reserved; /**< [in]: Reserved and should be set to 0. */ PNVENCOPENENCODESESSION nvEncOpenEncodeSession; /**< [out]: Client should access ::NvEncOpenEncodeSession() API through this pointer. */ PNVENCGETENCODEGUIDCOUNT nvEncGetEncodeGUIDCount; /**< [out]: Client should access ::NvEncGetEncodeGUIDCount() API through this pointer. */ PNVENCGETENCODEPRESETCOUNT nvEncGetEncodeProfileGUIDCount; /**< [out]: Client should access ::NvEncGetEncodeProfileGUIDCount() API through this pointer.*/ PNVENCGETENCODEPRESETGUIDS nvEncGetEncodeProfileGUIDs; /**< [out]: Client should access ::NvEncGetEncodeProfileGUIDs() API through this pointer. */ PNVENCGETENCODEGUIDS nvEncGetEncodeGUIDs; /**< [out]: Client should access ::NvEncGetEncodeGUIDs() API through this pointer. */ PNVENCGETINPUTFORMATCOUNT nvEncGetInputFormatCount; /**< [out]: Client should access ::NvEncGetInputFormatCount() API through this pointer. */ PNVENCGETINPUTFORMATS nvEncGetInputFormats; /**< [out]: Client should access ::NvEncGetInputFormats() API through this pointer. */ PNVENCGETENCODECAPS nvEncGetEncodeCaps; /**< [out]: Client should access ::NvEncGetEncodeCaps() API through this pointer. */ PNVENCGETENCODEPRESETCOUNT nvEncGetEncodePresetCount; /**< [out]: Client should access ::NvEncGetEncodePresetCount() API through this pointer. */ PNVENCGETENCODEPRESETGUIDS nvEncGetEncodePresetGUIDs; /**< [out]: Client should access ::NvEncGetEncodePresetGUIDs() API through this pointer. */ PNVENCGETENCODEPRESETCONFIG nvEncGetEncodePresetConfig; /**< [out]: Client should access ::NvEncGetEncodePresetConfig() API through this pointer. */ PNVENCINITIALIZEENCODER nvEncInitializeEncoder; /**< [out]: Client should access ::NvEncInitializeEncoder() API through this pointer. */ PNVENCCREATEINPUTBUFFER nvEncCreateInputBuffer; /**< [out]: Client should access ::NvEncCreateInputBuffer() API through this pointer. */ PNVENCDESTROYINPUTBUFFER nvEncDestroyInputBuffer; /**< [out]: Client should access ::NvEncDestroyInputBuffer() API through this pointer. */ PNVENCCREATEBITSTREAMBUFFER nvEncCreateBitstreamBuffer; /**< [out]: Client should access ::NvEncCreateBitstreamBuffer() API through this pointer. */ PNVENCDESTROYBITSTREAMBUFFER nvEncDestroyBitstreamBuffer; /**< [out]: Client should access ::NvEncDestroyBitstreamBuffer() API through this pointer. */ PNVENCENCODEPICTURE nvEncEncodePicture; /**< [out]: Client should access ::NvEncEncodePicture() API through this pointer. */ PNVENCLOCKBITSTREAM nvEncLockBitstream; /**< [out]: Client should access ::NvEncLockBitstream() API through this pointer. */ PNVENCUNLOCKBITSTREAM nvEncUnlockBitstream; /**< [out]: Client should access ::NvEncUnlockBitstream() API through this pointer. */ PNVENCLOCKINPUTBUFFER nvEncLockInputBuffer; /**< [out]: Client should access ::NvEncLockInputBuffer() API through this pointer. */ PNVENCUNLOCKINPUTBUFFER nvEncUnlockInputBuffer; /**< [out]: Client should access ::NvEncUnlockInputBuffer() API through this pointer. */ PNVENCGETENCODESTATS nvEncGetEncodeStats; /**< [out]: Client should access ::NvEncGetEncodeStats() API through this pointer. */ PNVENCGETSEQUENCEPARAMS nvEncGetSequenceParams; /**< [out]: Client should access ::NvEncGetSequenceParams() API through this pointer. */ PNVENCREGISTERASYNCEVENT nvEncRegisterAsyncEvent; /**< [out]: Client should access ::NvEncRegisterAsyncEvent() API through this pointer. */ PNVENCUNREGISTERASYNCEVENT nvEncUnregisterAsyncEvent; /**< [out]: Client should access ::NvEncUnregisterAsyncEvent() API through this pointer. */ PNVENCMAPINPUTRESOURCE nvEncMapInputResource; /**< [out]: Client should access ::NvEncMapInputResource() API through this pointer. */ PNVENCUNMAPINPUTRESOURCE nvEncUnmapInputResource; /**< [out]: Client should access ::NvEncUnmapInputResource() API through this pointer. */ PNVENCDESTROYENCODER nvEncDestroyEncoder; /**< [out]: Client should access ::NvEncDestroyEncoder() API through this pointer. */ PNVENCINVALIDATEREFFRAMES nvEncInvalidateRefFrames; /**< [out]: Client should access ::NvEncInvalidateRefFrames() API through this pointer. */ PNVENCOPENENCODESESSIONEX nvEncOpenEncodeSessionEx; /**< [out]: Client should access ::NvEncOpenEncodeSession() API through this pointer. */ PNVENCREGISTERRESOURCE nvEncRegisterResource; /**< [out]: Client should access ::NvEncRegisterResource() API through this pointer. */ PNVENCUNREGISTERRESOURCE nvEncUnregisterResource; /**< [out]: Client should access ::NvEncUnregisterResource() API through this pointer. */ PNVENCRECONFIGUREENCODER nvEncReconfigureEncoder; /**< [out]: Client should access ::NvEncReconfigureEncoder() API through this pointer. */ void* reserved1; PNVENCCREATEMVBUFFER nvEncCreateMVBuffer; /**< [out]: Client should access ::NvEncCreateMVBuffer API through this pointer. */ PNVENCDESTROYMVBUFFER nvEncDestroyMVBuffer; /**< [out]: Client should access ::NvEncDestroyMVBuffer API through this pointer. */ PNVENCRUNMOTIONESTIMATIONONLY nvEncRunMotionEstimationOnly; /**< [out]: Client should access ::NvEncRunMotionEstimationOnly API through this pointer. */ PNVENCGETLASTERROR nvEncGetLastErrorString; /**< [out]: Client should access ::nvEncGetLastErrorString API through this pointer. */ PNVENCSETIOCUDASTREAMS nvEncSetIOCudaStreams; /**< [out]: Client should access ::nvEncSetIOCudaStreams API through this pointer. */ PNVENCGETENCODEPRESETCONFIGEX nvEncGetEncodePresetConfigEx; /**< [out]: Client should access ::NvEncGetEncodePresetConfigEx() API through this pointer. */ PNVENCGETSEQUENCEPARAMEX nvEncGetSequenceParamEx; /**< [out]: Client should access ::NvEncGetSequenceParamEx() API through this pointer. */ void* reserved2[277]; /**< [in]: Reserved and must be set to NULL */ } NV_ENCODE_API_FUNCTION_LIST; /** Macro for constructing the version field of ::_NV_ENCODEAPI_FUNCTION_LIST. */ #define NV_ENCODE_API_FUNCTION_LIST_VER NVENCAPI_STRUCT_VERSION(2) // NvEncodeAPICreateInstance /** * \ingroup ENCODE_FUNC * Entry Point to the NvEncodeAPI interface. * * Creates an instance of the NvEncodeAPI interface, and populates the * pFunctionList with function pointers to the API routines implemented by the * NvEncodeAPI interface. * * \param [out] functionList * * \return * ::NV_ENC_SUCCESS * ::NV_ENC_ERR_INVALID_PTR */ NVENCSTATUS NVENCAPI NvEncodeAPICreateInstance(NV_ENCODE_API_FUNCTION_LIST* functionList); #ifdef __cplusplus } #endif #endif ================================================ FILE: alvr/server_openvr/cpp/alvr_server/openvr_driver_wrap.h ================================================ #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Wunused-parameter" #include #pragma GCC diagnostic pop ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/ColorCorrectionPixelShader.hlsl ================================================ cbuffer ColorCorrectionParams { float renderWidth; float renderHeight; float brightness; float contrast; float saturation; float gamma; float sharpening; float _align; }; const static float DX = 1. / renderWidth; const static float DY = 1. / renderHeight; const static float sharpenNeighbourWeight = -sharpening / 8.; Texture2D sourceTexture; SamplerState bilinearSampler { Filter = MIN_MAG_LINEAR_MIP_POINT; AddressU = CLAMP; AddressV = CLAMP; }; float3 GetSharpenNeighborComponent(float2 uv, float xoff, float yoff) { return sourceTexture.Sample(bilinearSampler, uv + float2(xoff, yoff)).rgb * sharpenNeighbourWeight; } float3 blendLighten(float3 base, float3 blend) { return float3(max(base.r,blend.r),max(base.g,blend.g),max(base.b,blend.b)); } // https://forum.unity.com/threads/hue-saturation-brightness-contrast-shader.260649/ float4 main(float2 uv : TEXCOORD0) : SV_Target{ // sharpening float3 pixel = sourceTexture.Sample(bilinearSampler, uv).rgb * (sharpening + 1.); pixel += GetSharpenNeighborComponent(uv, -DX, -DY); pixel += GetSharpenNeighborComponent(uv, 0, -DY); pixel += GetSharpenNeighborComponent(uv, +DX, -DY); pixel += GetSharpenNeighborComponent(uv, +DX, 0); pixel += GetSharpenNeighborComponent(uv, +DX, +DY); pixel += GetSharpenNeighborComponent(uv, 0, +DY); pixel += GetSharpenNeighborComponent(uv, -DX, +DY); pixel += GetSharpenNeighborComponent(uv, -DX, 0); pixel += brightness; // brightness pixel = (pixel - 0.5) * contrast + 0.5f; // contast pixel = blendLighten(lerp(dot(pixel, float3(0.299, 0.587, 0.114)), pixel, saturation), pixel); // saturation + lighten only pixel = clamp(pixel, 0, 1); pixel = pow(pixel, 1. / gamma); // gamma return float4(pixel, 1); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/CompressAxisAlignedPixelShader.hlsl ================================================ // Compress to rectangular slices #include "FoveatedRendering.hlsli" Texture2D compositionTexture; SamplerState trilinearSampler { Filter = MIN_MAG_MIP_LINEAR; //AddressU = Wrap; //AddressV = Wrap; }; float4 main(float2 uv : TEXCOORD0) : SV_Target { bool isRightEye = uv.x > 0.5; float2 eyeUV = TextureToEyeUV(uv, isRightEye) / eyeSizeRatio; float2 c0 = (1. - centerSize) / 2.; float2 c1 = (edgeRatio - 1.) * c0 * (centerShift + 1.) / edgeRatio; float2 c2 = (edgeRatio - 1.) * centerSize + 1.; float2 loBound = c0 * (centerShift + 1.) / c2; float2 hiBound = c0 * (centerShift - 1.) / c2 + 1.; float2 underBound = float2(eyeUV.x < loBound.x, eyeUV.y < loBound.y); float2 inBound = float2(loBound.x < eyeUV.x && eyeUV.x < hiBound.x, loBound.y < eyeUV.y && eyeUV.y < hiBound.y); float2 overBound = float2(eyeUV.x > hiBound.x, eyeUV.y > hiBound.y); float2 center = eyeUV * c2 / edgeRatio + c1; float2 d2 = eyeUV * c2; float2 d3 = (eyeUV - 1.) * c2 + 1.; float2 g1 = eyeUV / loBound; float2 g2 = (1. - eyeUV) / (1. - hiBound); float2 leftEdge = g1 * center + (1. - g1) * d2; float2 rightEdge = g2 * center + (1. - g2) * d3; float2 compressedUV = underBound * leftEdge + inBound * center + overBound * rightEdge; return compositionTexture.Sample(trilinearSampler, EyeToTextureUV(compressedUV, isRightEye)); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/FoveatedRendering.hlsli ================================================ cbuffer FoveationVars { uint2 targetResolution; uint2 optimizedResolution; float2 eyeSizeRatio; float2 centerSize; float2 centerShift; float2 edgeRatio; }; float2 TextureToEyeUV(float2 textureUV, bool isRightEye) { // flip distortion horizontally for right eye // left: x * 2; right: (1 - x) * 2 return float2((textureUV.x + float(isRightEye) * (1. - 2. * textureUV.x)) * 2., textureUV.y); } float2 EyeToTextureUV(float2 eyeUV, bool isRightEye) { // saturate is used to avoid color bleeding between the two sides of the texture or with the black border when filtering //float2 clampedUV = saturate(eyeUV); // left: x / 2; right 1 - (x / 2) //return float2(clampedUV.x / 2. + float(isRightEye) * (1. - clampedUV.x), clampedUV.y); return float2(eyeUV.x * .5 + float(isRightEye) * (1. - eyeUV.x), eyeUV.y); } ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/FrameRender.fx ================================================ cbuffer FrameRenderParams { float encodingGamma; float _padding0; float _padding1; float _padding2; }; Texture2D txLeft : register(t0); Texture2D txRight : register(t1); SamplerState samLinear : register(s0); struct VS_INPUT { float4 Pos : POSITION; float2 Tex : TEXCOORD; uint View : VIEW; }; struct PS_INPUT { float4 Pos : SV_POSITION; float2 Tex : TEXCOORD; uint View : VIEW; }; #define SRGB_GAMMA_TO_NONLINEAR (1.0 / 2.4) #define SRGB_GAMMA_TO_LINEAR (2.4) float4 EncodingLinearToNonlinearRGB(float4 color, float gamma) { float4 nonlinearColor; nonlinearColor.r = (color.r <= 0.0) ? color.r : pow(color.r, gamma); nonlinearColor.g = (color.g <= 0.0) ? color.g : pow(color.g, gamma); nonlinearColor.b = (color.b <= 0.0) ? color.b : pow(color.b, gamma); nonlinearColor.a = (color.a <= 0.0) ? color.a : pow(color.a, gamma); return nonlinearColor; } float4 LinearToNonlinearRGB(float4 color, float gamma) { float4 nonlinearColor; nonlinearColor.r = (color.r <= 0.0031308) ? (color.r * 12.92) : (1.055 * pow(color.r, gamma) - 0.055); nonlinearColor.g = (color.g <= 0.0031308) ? (color.g * 12.92) : (1.055 * pow(color.g, gamma) - 0.055); nonlinearColor.b = (color.b <= 0.0031308) ? (color.b * 12.92) : (1.055 * pow(color.b, gamma) - 0.055); nonlinearColor.a = (color.a <= 0.0031308) ? (color.a * 12.92) : (1.055 * pow(color.a, gamma) - 0.055); return nonlinearColor; } float4 NonlinearToLinearRGB(float4 color, float gamma) { float4 linearColor; linearColor.r = (color.r <= 0.04045) ? (color.r / 12.92) : pow((color.r + 0.055) / 1.055, gamma); linearColor.g = (color.g <= 0.04045) ? (color.g / 12.92) : pow((color.g + 0.055) / 1.055, gamma); linearColor.b = (color.b <= 0.04045) ? (color.b / 12.92) : pow((color.b + 0.055) / 1.055, gamma); linearColor.a = (color.a <= 0.04045) ? (color.a / 12.92) : pow((color.a + 0.055) / 1.055, gamma); return linearColor; } PS_INPUT VS(VS_INPUT input) { PS_INPUT output = (PS_INPUT)0; output.Pos = input.Pos; output.Tex = input.Tex; output.View = input.View; return output; } float4 PS(PS_INPUT input) : SV_Target { float4 color = float4(1.0, 0.0, 0.0, 1.0); uint correctionType = (input.View >> 1) & 0xF; uint shouldClamp = (input.View >> 5); if ((input.View & 1) == 1) { color = txRight.Sample(samLinear, input.Tex); } else { color = txLeft.Sample(samLinear, input.Tex); } if (shouldClamp == (uint)1) { color = clamp(color, 0.0, 1.0); } color = EncodingLinearToNonlinearRGB(color, encodingGamma); if (correctionType == (uint)1) { // Left View sRGB color = LinearToNonlinearRGB(color, SRGB_GAMMA_TO_NONLINEAR); } else if (correctionType == (uint)2) { // Left View non-HDR linear color = NonlinearToLinearRGB(color, SRGB_GAMMA_TO_LINEAR); } if (shouldClamp == (uint)2) { color = clamp(color, 0.0, 1.0); } return color; }; ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/FrameRenderPS.hlsl ================================================ #include "FrameRender.fx" ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/FrameRenderVS.hlsl ================================================ #include "FrameRender.fx" ================================================ FILE: alvr/server_openvr/cpp/alvr_server/shader/rgbtoyuv420.hlsl ================================================ cbuffer YUVParams { float4 offset; float4 yCoeff; float4 uCoeff; float4 vCoeff; float renderWidth; float renderHeight; float _padding0; float _padding1; }; Texture2D sourceTexture; struct PS_OUTPUT { float4 plane_Y: SV_Target0; float4 plane_UV: SV_Target1; }; SamplerState bilinearSampler { Filter = MIN_MAG_LINEAR_MIP_POINT; AddressU = CLAMP; AddressV = CLAMP; }; PS_OUTPUT main(float2 uv : TEXCOORD0) { PS_OUTPUT output; uint2 uvTexels = uint2(uv * float2(renderWidth, renderHeight)); // Y @ 1x for YUV420 float3 point1 = sourceTexture.Sample(bilinearSampler, uv).rgb; float y = dot(point1, yCoeff.rgb) + offset.x; // UV @ 1/2x for YUV420 float2 image_uv = float2((uvTexels.x * 2) % renderWidth / renderWidth, (uvTexels.y * 2) % renderHeight / renderHeight); float3 point2 = sourceTexture.Sample(bilinearSampler, image_uv).rgb; float u = dot(point2, uCoeff.rgb) + offset.y; float v = dot(point2, vCoeff.rgb) + offset.z; output.plane_Y = float4(y, 0, 0, 1); output.plane_UV = float4(u, v, 0, 1); return output; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/CEncoder.cpp ================================================ #include "CEncoder.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "ALVR-common/packet_types.h" #include "EncodePipeline.h" #include "FrameRender.h" #include "alvr_server/Logger.h" #include "alvr_server/PoseHistory.h" #include "alvr_server/Settings.h" #include "ffmpeg_helper.h" #include "protocol.h" extern "C" { #include } CEncoder::CEncoder(std::shared_ptr poseHistory) : m_poseHistory(poseHistory) { } CEncoder::~CEncoder() { Stop(); } namespace { void read_exactly(pollfd pollfds, char* out, size_t size, std::atomic_bool& exiting) { while (not exiting and size != 0) { int timeout = 1; // poll api doesn't fit perfectly(100 mircoseconds) poll uses milliseconds // we do the best we can(1000 mircoseconds) pollfds.events = POLLIN; int count = poll(&pollfds, 1, timeout); if (count < 0) { throw MakeException("poll failed: %s", strerror(errno)); } else if (count == 1) { int s = read(pollfds.fd, out, size); if (s == -1) { throw MakeException("read failed: %s", strerror(errno)); } out += s; size -= s; } } } void read_latest(pollfd pollfds, char* out, size_t size, std::atomic_bool& exiting) { read_exactly(pollfds, out, size, exiting); while (not exiting) { int timeout = 0; // poll api fixes the original perfectly(0 microseconds) pollfds.events = POLLIN; int count = poll(&pollfds, 1, timeout); if (count == 0) return; read_exactly(pollfds, out, size, exiting); } } int accept_timeout(pollfd socket, std::atomic_bool& exiting) { while (not exiting) { int timeout = 15; // poll api also fits the original perfectly(15000 microseconds) socket.events = POLLIN; int count = poll(&socket, 1, timeout); if (count < 0) { throw MakeException("poll failed: %s", strerror(errno)); } else if (count == 1) { return accept4(socket.fd, NULL, NULL, SOCK_CLOEXEC); } } return -1; } void av_logfn(void*, int level, const char* data, va_list va) { if (level > #ifdef DEBUG AV_LOG_DEBUG) #else AV_LOG_INFO) #endif return; char buf[256]; vsnprintf(buf, sizeof(buf), data, va); if (level <= AV_LOG_ERROR) Error("Encoder: %s", buf); else Info("Encoder: %s", buf); } } // namespace void CEncoder::GetFds(int client, int (*received_fds)[6]) { struct msghdr msg; struct cmsghdr* cmsg; union { struct cmsghdr cm; u_int8_t pktinfo_sizer[sizeof(struct cmsghdr) + 1024]; } control_un; struct iovec iov[1]; char data[1]; int ret; msg.msg_control = &control_un; msg.msg_controllen = sizeof(control_un); msg.msg_flags = 0; msg.msg_name = NULL; msg.msg_namelen = 0; iov[0].iov_base = data; iov[0].iov_len = 1; msg.msg_iov = iov; msg.msg_iovlen = 1; ret = recvmsg(client, &msg, 0); if (ret == -1) { throw MakeException("recvmsg failed: %s", strerror(errno)); } for (cmsg = CMSG_FIRSTHDR(&msg); cmsg != NULL; cmsg = CMSG_NXTHDR(&msg, cmsg)) { if (cmsg->cmsg_level == SOL_SOCKET && cmsg->cmsg_type == SCM_RIGHTS) { memcpy(received_fds, CMSG_DATA(cmsg), sizeof(*received_fds)); break; } } if (cmsg == NULL) { throw MakeException("cmsg is NULL"); } } void CEncoder::Run() { Info("CEncoder::Run\n"); m_socketPath = getenv("XDG_RUNTIME_DIR"); m_socketPath += "/alvr-ipc"; int ret; // we don't really care about what happends with unlink, it's just incase we crashed before this // run ret = unlink(m_socketPath.c_str()); m_socket.fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0); struct sockaddr_un name; if (m_socket.fd == -1) { perror("socket"); exit(1); } memset(&name, 0, sizeof(name)); name.sun_family = AF_UNIX; strncpy(name.sun_path, m_socketPath.c_str(), sizeof(name.sun_path) - 1); ret = bind(m_socket.fd, (const struct sockaddr*)&name, sizeof(name)); if (ret == -1) { perror("bind"); exit(1); } ret = listen(m_socket.fd, 1024); if (ret == -1) { perror("listen"); exit(1); } Info("CEncoder Listening\n"); struct pollfd client; client.fd = accept_timeout(m_socket, m_exiting); if (m_exiting) return; init_packet init; client.events = POLLIN; read_exactly(client, (char*)&init, sizeof(init), m_exiting); if (m_exiting) return; // check that pointer types are null, other values would not make sense over a socket assert(init.image_create_info.queueFamilyIndexCount == 0); assert(init.image_create_info.pNext == NULL); char ifbuf[256]; char ifbuf2[256]; sprintf(ifbuf, "/proc/%d/cmdline", (int)init.source_pid); std::ifstream ifscmdl(ifbuf); ifscmdl >> ifbuf2; Info("CEncoder client connected, pid %d, cmdline %s\n", (int)init.source_pid, ifbuf2); try { GetFds(client.fd, &m_fds); m_connected = true; fprintf(stderr, "\n\nWe are initalizing Vulkan in CEncoder thread\n\n\n"); av_log_set_callback(av_logfn); alvr::VkContext vk_ctx(init.device_uuid.data(), {}); FrameRender render(vk_ctx, init, m_fds); auto output = render.CreateOutput(); alvr::VkFrame frame( vk_ctx, output.image, output.imageInfo, output.size, output.memory, output.drm ); auto encode_pipeline = alvr::EncodePipeline::Create( &render, vk_ctx, frame, output.imageInfo, render.GetEncodingWidth(), render.GetEncodingHeight() ); bool valid_timestamps = true; fprintf(stderr, "CEncoder starting to read present packets"); present_packet frame_info; while (not m_exiting) { read_latest(client, (char*)&frame_info, sizeof(frame_info), m_exiting); encode_pipeline->SetParams(GetDynamicEncoderParams()); auto pose = m_poseHistory->GetBestPoseMatch((const vr::HmdMatrix34_t&)frame_info.pose); if (!pose) { continue; } if (m_captureFrame) { m_captureFrame = false; render.CaptureInputFrame( Settings::Instance().m_captureFrameDir + "/alvr_frame_input.ppm" ); render.CaptureOutputFrame( Settings::Instance().m_captureFrameDir + "/alvr_frame_output.ppm" ); } render.Render(frame_info.image, frame_info.semaphore_value); if (!valid_timestamps) { ReportPresent(pose->targetTimestampNs, 0); ReportComposed(pose->targetTimestampNs, 0); } encode_pipeline->PushFrame(pose->targetTimestampNs, m_scheduler.CheckIDRInsertion()); static_assert(sizeof(frame_info.pose) == sizeof(vr::HmdMatrix34_t&)); alvr::FramePacket packet; if (!encode_pipeline->GetEncoded(packet)) { Error("Failed to get encoded data!"); continue; } if (valid_timestamps) { auto render_timestamps = render.GetTimestamps(); auto encode_timestamp = encode_pipeline->GetTimestamp(); uint64_t present_offset = render_timestamps.now - render_timestamps.renderBegin; uint64_t composed_offset = 0; valid_timestamps = render_timestamps.now != 0; if (encode_timestamp.gpu) { composed_offset = render_timestamps.now - encode_timestamp.gpu; } else if (encode_timestamp.cpu) { auto now = std::chrono::duration_cast( std::chrono::steady_clock::now().time_since_epoch() ) .count(); composed_offset = now - encode_timestamp.cpu; } else { composed_offset = render_timestamps.now - render_timestamps.renderComplete; } if (present_offset < composed_offset) { present_offset = composed_offset; } ReportPresent(pose->targetTimestampNs, present_offset); ReportComposed(pose->targetTimestampNs, composed_offset); } ParseFrameNals( encode_pipeline->GetCodec(), packet.data, packet.size, packet.pts, packet.isIDR ); } } catch (std::exception& e) { std::stringstream err; err << "error in encoder thread: " << e.what(); Error(err.str().c_str()); } client.events = POLLHUP; close(client.fd); } void CEncoder::Stop() { m_exiting = true; m_socket.events = POLLHUP; close(m_socket.fd); unlink(m_socketPath.c_str()); } void CEncoder::OnStreamStart() { m_scheduler.OnStreamStart(); } void CEncoder::InsertIDR() { m_scheduler.InsertIDR(); } void CEncoder::CaptureFrame() { m_captureFrame = true; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/CEncoder.h ================================================ #pragma once #include "alvr_server/IDRScheduler.h" #include "shared/threadtools.h" #include #include #include #include class PoseHistory; class CEncoder : public CThread { public: CEncoder(std::shared_ptr poseHistory); ~CEncoder(); bool Init() override { return true; } void Run() override; void Stop(); void OnStreamStart(); void InsertIDR(); bool IsConnected() { return m_connected; } void CaptureFrame(); private: void GetFds(int client, int (*fds)[6]); std::shared_ptr m_poseHistory; std::atomic_bool m_exiting { false }; IDRScheduler m_scheduler; pollfd m_socket; std::string m_socketPath; int m_fds[6]; bool m_connected = false; std::atomic_bool m_captureFrame = false; }; ================================================ FILE: alvr/server_openvr/cpp/platform/linux/CrashHandler.cpp ================================================ #include "../../alvr_server/bindings.h" void HookCrashHandler() { } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipeline.cpp ================================================ #include "EncodePipeline.h" #include "EncodePipelineNvEnc.h" #include "EncodePipelineSW.h" #include "EncodePipelineVAAPI.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "ffmpeg_helper.h" extern "C" { #include } void alvr::EncodePipeline::SetParams(FfiDynamicEncoderParams params) { if (params.updated) { encoder_ctx->bit_rate = params.bitrate_bps / params.framerate * 60.0; encoder_ctx->framerate = AVRational { 60, 1 }; encoder_ctx->rc_buffer_size = encoder_ctx->bit_rate / 60.0 * 1.1; encoder_ctx->rc_max_rate = encoder_ctx->bit_rate; encoder_ctx->rc_initial_buffer_occupancy = encoder_ctx->rc_buffer_size / 4 * 3; } } std::unique_ptr alvr::EncodePipeline::Create( Renderer* render, VkContext& vk_ctx, VkFrame& input_frame, VkImageCreateInfo& image_create_info, uint32_t width, uint32_t height ) { if (Settings::Instance().m_force_sw_encoding == false) { if (vk_ctx.nvidia) { try { auto nvenc = std::make_unique( render, vk_ctx, input_frame, image_create_info, width, height ); Info("Using NvEnc encoder"); return nvenc; } catch (std::exception& e) { Error( "Failed to create NvEnc encoder: %s\nPlease make sure you have installed CUDA " "runtime.", e.what() ); } } else { try { auto vaapi = std::make_unique( render, vk_ctx, input_frame, width, height ); Info("Using VAAPI encoder"); return vaapi; } catch (std::exception& e) { Error( "Failed to create VAAPI encoder: %s\nPlease make sure you have installed VAAPI " "runtime.", e.what() ); } } } auto sw = std::make_unique(render, width, height); Info("Using SW encoder"); return sw; } alvr::EncodePipeline::~EncodePipeline() { avcodec_free_context(&encoder_ctx); } bool alvr::EncodePipeline::GetEncoded(FramePacket& packet) { av_packet_free(&encoder_packet); encoder_packet = av_packet_alloc(); int err = avcodec_receive_packet(encoder_ctx, encoder_packet); if (err != 0) { av_packet_free(&encoder_packet); if (err == AVERROR(EAGAIN)) { return false; } throw alvr::AvException("failed to encode", err); } packet.data = encoder_packet->data; packet.size = encoder_packet->size; packet.pts = encoder_packet->pts; packet.isIDR = (encoder_packet->flags & AV_PKT_FLAG_KEY) != 0; return true; } int alvr::EncodePipeline::GetCodec() { return Settings::Instance().m_codec; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipeline.h ================================================ #pragma once #include "alvr_server/bindings.h" #include #include #include #include extern "C" struct AVCodecContext; extern "C" struct AVPacket; class Renderer; namespace alvr { class VkFrame; class VkFrameCtx; class VkContext; struct FramePacket { uint8_t* data; int size; uint64_t pts; bool isIDR; }; class EncodePipeline { public: struct Timestamp { uint64_t gpu = 0; uint64_t cpu = 0; }; virtual ~EncodePipeline(); virtual void PushFrame(uint64_t targetTimestampNs, bool idr) = 0; virtual bool GetEncoded(FramePacket& data); virtual Timestamp GetTimestamp() { return timestamp; } virtual int GetCodec(); virtual void SetParams(FfiDynamicEncoderParams params); static std::unique_ptr Create( Renderer* render, VkContext& vk_ctx, VkFrame& input_frame, VkImageCreateInfo& image_create_info, uint32_t width, uint32_t height ); protected: AVCodecContext* encoder_ctx = nullptr; // shall be initialized by child class AVPacket* encoder_packet = NULL; Timestamp timestamp = {}; }; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipelineNvEnc.cpp ================================================ #include "EncodePipelineNvEnc.h" #include "ALVR-common/packet_types.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "ffmpeg_helper.h" #include #include extern "C" { #include #include } namespace { const char* encoder(ALVR_CODEC codec) { switch (codec) { case ALVR_CODEC_H264: return "h264_nvenc"; case ALVR_CODEC_HEVC: return "hevc_nvenc"; case ALVR_CODEC_AV1: return "av1_nvenc"; } throw std::runtime_error("invalid codec " + std::to_string(codec)); } void set_hwframe_ctx(AVCodecContext* ctx, AVBufferRef* hw_device_ctx) { AVBufferRef* hw_frames_ref; AVHWFramesContext* frames_ctx = NULL; int err = 0; if (!(hw_frames_ref = av_hwframe_ctx_alloc(hw_device_ctx))) { throw std::runtime_error("Failed to create CUDA frame context."); } frames_ctx = (AVHWFramesContext*)(hw_frames_ref->data); frames_ctx->format = AV_PIX_FMT_CUDA; /** * We will recieve a frame from HW as AV_PIX_FMT_VULKAN which will converted to AV_PIX_FMT_BGRA * as SW format when we get it from HW. * But NVEnc support only BGR0 format and we easy can just to force it * Because: * AV_PIX_FMT_BGRA - 28 ///< packed BGRA 8:8:8:8, 32bpp, BGRABGRA... * AV_PIX_FMT_BGR0 - 123 ///< packed BGR 8:8:8, 32bpp, BGRXBGRX... X=unused/undefined * * We just to ignore the alpha channel and it's done */ frames_ctx->sw_format = AV_PIX_FMT_BGR0; frames_ctx->width = ctx->width; frames_ctx->height = ctx->height; if ((err = av_hwframe_ctx_init(hw_frames_ref)) < 0) { av_buffer_unref(&hw_frames_ref); throw alvr::AvException("Failed to initialize CUDA frame context:", err); } ctx->hw_frames_ctx = av_buffer_ref(hw_frames_ref); if (!ctx->hw_frames_ctx) err = AVERROR(ENOMEM); av_buffer_unref(&hw_frames_ref); } } // namespace alvr::EncodePipelineNvEnc::EncodePipelineNvEnc( Renderer* render, VkContext& vk_ctx, VkFrame& input_frame, VkImageCreateInfo& image_create_info, uint32_t width, uint32_t height ) { r = render; vk_frame_ctx = std::make_unique(vk_ctx, image_create_info); auto input_frame_ctx = (AVHWFramesContext*)vk_frame_ctx->ctx->data; assert(input_frame_ctx->sw_format == AV_PIX_FMT_BGRA); int err; vk_frame = input_frame.make_av_frame(*vk_frame_ctx); err = av_hwdevice_ctx_create_derived(&hw_ctx, AV_HWDEVICE_TYPE_CUDA, vk_ctx.ctx, 0); if (err < 0) { throw alvr::AvException("Failed to create a CUDA device:", err); } const auto& settings = Settings::Instance(); auto codec_id = ALVR_CODEC(settings.m_codec); const char* encoder_name = encoder(codec_id); const AVCodec* codec = avcodec_find_encoder_by_name(encoder_name); if (codec == nullptr) { throw std::runtime_error(std::string("Failed to find encoder ") + encoder_name); } encoder_ctx = avcodec_alloc_context3(codec); if (not encoder_ctx) { throw std::runtime_error("failed to allocate NvEnc encoder"); } switch (codec_id) { case ALVR_CODEC_H264: switch (settings.m_entropyCoding) { case ALVR_CABAC: av_opt_set(encoder_ctx->priv_data, "coder", "ac", 0); break; case ALVR_CAVLC: av_opt_set(encoder_ctx->priv_data, "coder", "vlc", 0); break; } break; case ALVR_CODEC_HEVC: break; case ALVR_CODEC_AV1: break; } switch (settings.m_rateControlMode) { case ALVR_CBR: av_opt_set(encoder_ctx->priv_data, "rc", "cbr", 0); break; case ALVR_VBR: av_opt_set(encoder_ctx->priv_data, "rc", "vbr", 0); break; } if (codec_id == ALVR_CODEC_H264) { switch (settings.m_h264Profile) { case ALVR_H264_PROFILE_BASELINE: av_opt_set(encoder_ctx->priv_data, "profile", "baseline", 0); break; case ALVR_H264_PROFILE_MAIN: av_opt_set(encoder_ctx->priv_data, "profile", "main", 0); break; default: case ALVR_H264_PROFILE_HIGH: av_opt_set(encoder_ctx->priv_data, "profile", "high", 0); break; } } char preset[] = "p0"; // replace 0 with preset number preset[1] += settings.m_nvencQualityPreset; av_opt_set(encoder_ctx->priv_data, "preset", preset, 0); if (settings.m_nvencAdaptiveQuantizationMode == 1) { av_opt_set_int(encoder_ctx->priv_data, "spatial_aq", 1, 0); } else if (settings.m_nvencAdaptiveQuantizationMode == 2) { av_opt_set_int(encoder_ctx->priv_data, "temporal_aq", 1, 0); } if (settings.m_nvencEnableWeightedPrediction) { av_opt_set_int(encoder_ctx->priv_data, "weighted_pred", 1, 0); } av_opt_set_int(encoder_ctx->priv_data, "tune", settings.m_nvencTuningPreset, 0); av_opt_set_int(encoder_ctx->priv_data, "zerolatency", 1, 0); // Delay isn't actually a delay instead its how many surfaces to encode at a time av_opt_set_int(encoder_ctx->priv_data, "delay", 1, 0); av_opt_set_int(encoder_ctx->priv_data, "forced-idr", 1, 0); // work around ffmpeg default not working for older NVIDIA cards av_opt_set_int(encoder_ctx->priv_data, "b_ref_mode", 0, 0); encoder_ctx->pix_fmt = AV_PIX_FMT_CUDA; encoder_ctx->width = width; encoder_ctx->height = height; encoder_ctx->time_base = { 1, (int)1e9 }; encoder_ctx->framerate = AVRational { settings.m_refreshRate, 1 }; encoder_ctx->sample_aspect_ratio = AVRational { 1, 1 }; encoder_ctx->max_b_frames = 0; encoder_ctx->gop_size = INT16_MAX; encoder_ctx->color_range = AVCOL_RANGE_JPEG; auto params = FfiDynamicEncoderParams {}; params.updated = true; params.bitrate_bps = 30'000'000; params.framerate = 60.0; SetParams(params); set_hwframe_ctx(encoder_ctx, hw_ctx); err = avcodec_open2(encoder_ctx, codec, NULL); if (err < 0) { throw alvr::AvException("Cannot open video encoder codec:", err); } hw_frame = av_frame_alloc(); } alvr::EncodePipelineNvEnc::~EncodePipelineNvEnc() { av_buffer_unref(&hw_ctx); av_frame_free(&hw_frame); } void alvr::EncodePipelineNvEnc::PushFrame(uint64_t targetTimestampNs, bool idr) { AVVkFrame* vkf = reinterpret_cast(vk_frame->data[0]); vkf->sem_value[0]++; VkTimelineSemaphoreSubmitInfo timelineInfo = {}; timelineInfo.sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO; timelineInfo.signalSemaphoreValueCount = 1; timelineInfo.pSignalSemaphoreValues = &vkf->sem_value[0]; VkPipelineStageFlags waitStage = VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT; VkSubmitInfo submitInfo = {}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.pNext = &timelineInfo; submitInfo.waitSemaphoreCount = 1; submitInfo.pWaitSemaphores = &r->GetOutput().semaphore; submitInfo.pWaitDstStageMask = &waitStage; submitInfo.signalSemaphoreCount = 1; submitInfo.pSignalSemaphores = &vkf->sem[0]; VK_CHECK(vkQueueSubmit(r->m_queue, 1, &submitInfo, nullptr)); int err = av_hwframe_get_buffer(encoder_ctx->hw_frames_ctx, hw_frame, 0); if (err < 0) { throw alvr::AvException("Failed to allocate CUDA frame", err); } err = av_hwframe_transfer_data(hw_frame, vk_frame.get(), 0); if (err < 0) { throw alvr::AvException("Failed to transfer Vulkan image to CUDA frame", err); } hw_frame->pict_type = idr ? AV_PICTURE_TYPE_I : AV_PICTURE_TYPE_NONE; hw_frame->pts = targetTimestampNs; if ((err = avcodec_send_frame(encoder_ctx, hw_frame)) < 0) { throw alvr::AvException("avcodec_send_frame failed:", err); } av_frame_unref(hw_frame); } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipelineNvEnc.h ================================================ #pragma once #include "EncodePipeline.h" #include #include extern "C" struct AVBufferRef; extern "C" struct AVCodecContext; extern "C" struct AVFrame; class Renderer; namespace alvr { class EncodePipelineNvEnc : public EncodePipeline { public: ~EncodePipelineNvEnc(); EncodePipelineNvEnc( Renderer* render, VkContext& vk_ctx, VkFrame& input_frame, VkImageCreateInfo& image_create_info, uint32_t width, uint32_t height ); void PushFrame(uint64_t targetTimestampNs, bool idr) override; private: Renderer* r = nullptr; std::unique_ptr vk_frame_ctx; AVBufferRef* hw_ctx = nullptr; std::unique_ptr> vk_frame; AVFrame* hw_frame = nullptr; }; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipelineSW.cpp ================================================ #include "EncodePipelineSW.h" #include #include "FormatConverter.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" namespace { void x264_log(void*, int level, const char* fmt, va_list args) { char buf[256]; vsnprintf(buf, sizeof(buf), fmt, args); switch (level) { case X264_LOG_ERROR: Error("x264: %s", buf); break; case X264_LOG_WARNING: Warn("x264: %s", buf); break; case X264_LOG_INFO: Info("x264: %s", buf); break; case X264_LOG_DEBUG: Debug("x264: %s", buf); break; default: break; } } } alvr::EncodePipelineSW::EncodePipelineSW(Renderer* render, uint32_t width, uint32_t height) { const auto& settings = Settings::Instance(); x264_param_default_preset(¶m, "ultrafast", "zerolatency"); param.pf_log = x264_log; param.i_log_level = X264_LOG_INFO; param.b_aud = 0; param.b_cabac = settings.m_entropyCoding == ALVR_CABAC; param.b_sliced_threads = true; param.i_threads = settings.m_swThreadCount; param.i_width = width; param.i_height = height; param.rc.i_rc_method = X264_RC_ABR; switch (settings.m_h264Profile) { case ALVR_H264_PROFILE_BASELINE: x264_param_apply_profile(¶m, "baseline"); break; case ALVR_H264_PROFILE_MAIN: x264_param_apply_profile(¶m, "main"); break; default: case ALVR_H264_PROFILE_HIGH: x264_param_apply_profile(¶m, "high"); break; } auto params = FfiDynamicEncoderParams {}; params.updated = true; params.bitrate_bps = 30'000'000; params.framerate = Settings::Instance().m_refreshRate; SetParams(params); enc = x264_encoder_open(¶m); if (!enc) { throw std::runtime_error("Failed to open encoder"); } x264_picture_init(&picture); picture.img.i_csp = X264_CSP_I420; picture.img.i_plane = 3; x264_picture_init(&picture_out); rgbtoyuv = new RgbToYuv420( render, render->GetOutput().image, render->GetOutput().imageInfo, render->GetOutput().semaphore ); } alvr::EncodePipelineSW::~EncodePipelineSW() { if (rgbtoyuv) { delete rgbtoyuv; } if (enc) { x264_encoder_close(enc); } } void alvr::EncodePipelineSW::PushFrame(uint64_t targetTimestampNs, bool idr) { rgbtoyuv->Convert(picture.img.plane, picture.img.i_stride); rgbtoyuv->Sync(); timestamp.cpu = std::chrono::duration_cast( std::chrono::steady_clock::now().time_since_epoch() ) .count(); picture.i_type = idr ? X264_TYPE_IDR : X264_TYPE_AUTO; pts = picture.i_pts = targetTimestampNs; is_idr = idr; int nnal = 0; nal_size = x264_encoder_encode(enc, &nal, &nnal, &picture, &picture_out); if (nal_size < 0) { throw std::runtime_error("x264 encoder_encode failed"); } } bool alvr::EncodePipelineSW::GetEncoded(FramePacket& packet) { if (!nal) { return false; } packet.size = nal_size; packet.data = nal[0].p_payload; packet.pts = pts; packet.isIDR = is_idr; return packet.size > 0; } void alvr::EncodePipelineSW::SetParams(FfiDynamicEncoderParams params) { if (!params.updated) { return; } // x264 doesn't work well with adaptive bitrate/fps param.i_fps_num = Settings::Instance().m_refreshRate; param.i_fps_den = 1; param.rc.i_bitrate = params.bitrate_bps / 1'000 * 1.4; // needs higher value to hit target bitrate param.rc.i_vbv_buffer_size = param.rc.i_bitrate / param.i_fps_num * 1.1; param.rc.i_vbv_max_bitrate = param.rc.i_bitrate; param.rc.f_vbv_buffer_init = 0.75; if (enc) { x264_encoder_reconfig(enc, ¶m); } } int alvr::EncodePipelineSW::GetCodec() { return ALVR_CODEC_H264; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipelineSW.h ================================================ #pragma once #include "EncodePipeline.h" #include class FormatConverter; namespace alvr { class EncodePipelineSW : public EncodePipeline { public: ~EncodePipelineSW(); EncodePipelineSW(Renderer* render, uint32_t width, uint32_t height); void PushFrame(uint64_t targetTimestampNs, bool idr) override; bool GetEncoded(FramePacket& packet) override; void SetParams(FfiDynamicEncoderParams params) override; int GetCodec() override; private: x264_t* enc = nullptr; x264_param_t param; x264_picture_t picture; x264_picture_t picture_out; x264_nal_t* nal = nullptr; int nal_size = 0; int64_t pts = 0; bool is_idr = false; FormatConverter* rgbtoyuv = nullptr; }; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipelineVAAPI.cpp ================================================ #include "EncodePipelineVAAPI.h" #include "ALVR-common/packet_types.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "ffmpeg_helper.h" #include extern "C" { #include #include #include #include #include #include } namespace { const char* encoder(ALVR_CODEC codec) { switch (codec) { case ALVR_CODEC_H264: return "h264_vaapi"; case ALVR_CODEC_HEVC: return "hevc_vaapi"; case ALVR_CODEC_AV1: return "av1_vaapi"; } throw std::runtime_error("invalid codec " + std::to_string(codec)); } void set_hwframe_ctx(AVCodecContext* ctx, AVBufferRef* hw_device_ctx) { AVBufferRef* hw_frames_ref; AVHWFramesContext* frames_ctx = NULL; int err = 0; if (!(hw_frames_ref = av_hwframe_ctx_alloc(hw_device_ctx))) { throw std::runtime_error("Failed to create VAAPI frame context."); } frames_ctx = (AVHWFramesContext*)(hw_frames_ref->data); frames_ctx->format = AV_PIX_FMT_VAAPI; frames_ctx->sw_format = (Settings::Instance().m_codec == ALVR_CODEC_HEVC || Settings::Instance().m_codec == ALVR_CODEC_AV1) && Settings::Instance().m_use10bitEncoder ? AV_PIX_FMT_P010 : AV_PIX_FMT_NV12; frames_ctx->width = ctx->width; frames_ctx->height = ctx->height; frames_ctx->initial_pool_size = 3; if ((err = av_hwframe_ctx_init(hw_frames_ref)) < 0) { av_buffer_unref(&hw_frames_ref); throw alvr::AvException("Failed to initialize VAAPI frame context:", err); } ctx->hw_frames_ctx = av_buffer_ref(hw_frames_ref); if (!ctx->hw_frames_ctx) err = AVERROR(ENOMEM); av_buffer_unref(&hw_frames_ref); } // Map the vulkan frames to corresponding vaapi frames AVFrame* map_frame(AVBufferRef* hw_frames_ref, AVBufferRef* drm_device_ctx, alvr::VkFrame& input_frame) { auto frames_ctx = (AVHWFramesContext*)(hw_frames_ref->data); AVFrame* mapped_frame = av_frame_alloc(); mapped_frame->format = AV_PIX_FMT_VAAPI; mapped_frame->hw_frames_ctx = av_buffer_ref(hw_frames_ref); AVBufferRef* drm_frames_ref = NULL; if (!(drm_frames_ref = av_hwframe_ctx_alloc(drm_device_ctx))) { throw std::runtime_error("Failed to create vulkan frame context."); } AVHWFramesContext* drm_frames_ctx = (AVHWFramesContext*)(drm_frames_ref->data); drm_frames_ctx->format = AV_PIX_FMT_DRM_PRIME; drm_frames_ctx->sw_format = frames_ctx->sw_format; drm_frames_ctx->width = frames_ctx->width; drm_frames_ctx->height = frames_ctx->height; drm_frames_ctx->initial_pool_size = 0; int err; if ((err = av_hwframe_ctx_init(drm_frames_ref)) < 0) { av_buffer_unref(&drm_frames_ref); throw alvr::AvException("Failed to initialize DRM frame context:", err); } AVFrame* vk_frame = av_frame_alloc(); vk_frame->width = frames_ctx->width; vk_frame->height = frames_ctx->height; vk_frame->hw_frames_ctx = drm_frames_ref; vk_frame->data[0] = (uint8_t*)(AVDRMFrameDescriptor*)input_frame; vk_frame->format = AV_PIX_FMT_DRM_PRIME; vk_frame->buf[0] = av_buffer_alloc(1); av_hwframe_map(mapped_frame, vk_frame, AV_HWFRAME_MAP_READ); av_frame_free(&vk_frame); av_buffer_unref(&hw_frames_ref); return mapped_frame; } // Import VA surface AVFrame* import_frame(AVBufferRef* hw_frames_ref, DrmImage& drm) { AVFrame* va_frame = av_frame_alloc(); int err = av_hwframe_get_buffer(hw_frames_ref, va_frame, 0); if (err < 0) { throw alvr::AvException("Failed to get hwframe buffer:", err); } AVFrame* mapped_frame = av_frame_alloc(); mapped_frame->format = AV_PIX_FMT_DRM_PRIME; err = av_hwframe_map(mapped_frame, va_frame, AV_HWFRAME_MAP_WRITE); if (err < 0) { throw alvr::AvException("Failed to export va frame:", err); } auto desc = reinterpret_cast(mapped_frame->data[0]); drm.fd = desc->objects[0].fd; drm.format = desc->layers[0].format; drm.modifier = desc->objects[0].format_modifier; drm.planes = desc->layers[0].nb_planes; for (uint32_t i = 0; i < drm.planes; ++i) { drm.strides[0] = desc->layers[0].planes[i].pitch; drm.offsets[0] = desc->layers[0].planes[i].offset; } return va_frame; } } alvr::EncodePipelineVAAPI::EncodePipelineVAAPI( Renderer* render, VkContext& vk_ctx, VkFrame& input_frame, uint32_t width, uint32_t height ) : r(render) { /* VAAPI Encoding pipeline * The encoding pipeline has 3 frame types: * - input vulkan frames, only used to initialize the mapped frames * - mapped frames, one per input frame, same format, and point to the same memory on the device * - encoder frame, with a format compatible with the encoder, created by the filter * Each frame type has a corresponding hardware frame context, the vulkan one is provided * * The pipeline is simply made of a scale_vaapi object, that does the conversion between formats * and the encoder that takes the converted frame and produces packets. */ int err = av_hwdevice_ctx_create( &hw_ctx, AV_HWDEVICE_TYPE_VAAPI, vk_ctx.devicePath.c_str(), NULL, 0 ); if (err < 0) { throw alvr::AvException("Failed to create a VAAPI device:", err); } drm_ctx = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_DRM); AVHWDeviceContext* hwctx = (AVHWDeviceContext*)drm_ctx->data; AVDRMDeviceContext* drmctx = (AVDRMDeviceContext*)hwctx->hwctx; drmctx->fd = -1; err = av_hwdevice_ctx_init(drm_ctx); if (err < 0) { throw alvr::AvException("Failed to create DRM device:", err); } const auto& settings = Settings::Instance(); auto codec_id = ALVR_CODEC(settings.m_codec); const char* encoder_name = encoder(codec_id); const AVCodec* codec = avcodec_find_encoder_by_name(encoder_name); if (codec == nullptr) { throw std::runtime_error(std::string("Failed to find encoder ") + encoder_name); } encoder_ctx = avcodec_alloc_context3(codec); if (not encoder_ctx) { throw std::runtime_error("failed to allocate VAAPI encoder"); } encoder_ctx->gop_size = INT_MAX; switch (codec_id) { case ALVR_CODEC_H264: switch (settings.m_h264Profile) { case ALVR_H264_PROFILE_BASELINE: encoder_ctx->profile = FF_PROFILE_H264_BASELINE; break; case ALVR_H264_PROFILE_MAIN: encoder_ctx->profile = FF_PROFILE_H264_MAIN; break; default: case ALVR_H264_PROFILE_HIGH: encoder_ctx->profile = FF_PROFILE_H264_HIGH; break; } switch (settings.m_entropyCoding) { case ALVR_CABAC: av_opt_set(encoder_ctx->priv_data, "coder", "ac", 0); break; case ALVR_CAVLC: av_opt_set(encoder_ctx->priv_data, "coder", "vlc", 0); break; } break; case ALVR_CODEC_HEVC: encoder_ctx->profile = Settings::Instance().m_use10bitEncoder ? FF_PROFILE_HEVC_MAIN_10 : FF_PROFILE_HEVC_MAIN; encoder_ctx->gop_size = INT16_MAX; break; case ALVR_CODEC_AV1: encoder_ctx->profile = FF_PROFILE_AV1_MAIN; break; } switch (settings.m_rateControlMode) { case ALVR_VBR: av_opt_set(encoder_ctx->priv_data, "rc_mode", "VBR", 0); break; case ALVR_CBR: default: av_opt_set(encoder_ctx->priv_data, "rc_mode", "CBR", 0); break; } av_opt_set_int(encoder_ctx->priv_data, "filler_data", settings.m_fillerData, 0); encoder_ctx->width = width; encoder_ctx->height = height; encoder_ctx->time_base = { 1, (int)1e9 }; encoder_ctx->sample_aspect_ratio = AVRational { 1, 1 }; encoder_ctx->pix_fmt = AV_PIX_FMT_VAAPI; encoder_ctx->max_b_frames = 0; encoder_ctx->color_range = AVCOL_RANGE_JPEG; auto params = FfiDynamicEncoderParams {}; params.updated = true; params.bitrate_bps = 30'000'000; params.framerate = settings.m_refreshRate; SetParams(params); vlVaQualityBits quality = {}; quality.vbaq_mode = Settings::Instance() .m_enableVbaq; // No noticable performance difference and should improve subjective // quality by allocating more bits to smooth areas switch (settings.m_encoderQualityPreset) { case ALVR_QUALITY: if (vk_ctx.amd) { quality.preset_mode = PRESET_MODE_QUALITY; encoder_ctx->compression_level = quality.quality; // (QUALITY preset, no pre-encoding, vbaq) } else if (vk_ctx.intel) { encoder_ctx->compression_level = 1; } break; case ALVR_BALANCED: if (vk_ctx.amd) { quality.preset_mode = PRESET_MODE_BALANCE; encoder_ctx->compression_level = quality.quality; // (BALANCE preset, no pre-encoding, vbaq) } else if (vk_ctx.intel) { encoder_ctx->compression_level = 4; } break; case ALVR_SPEED: default: if (vk_ctx.amd) { quality.preset_mode = PRESET_MODE_SPEED; encoder_ctx->compression_level = quality.quality; // (speed preset, no pre-encoding, vbaq) } else if (vk_ctx.intel) { encoder_ctx->compression_level = 7; } break; } av_opt_set_int(encoder_ctx->priv_data, "async_depth", 1, 0); set_hwframe_ctx(encoder_ctx, hw_ctx); err = avcodec_open2(encoder_ctx, codec, NULL); if (err < 0) { throw alvr::AvException("Cannot open video encoder codec:", err); } AVBufferRef* hw_frames_ref; if (!(hw_frames_ref = av_hwframe_ctx_alloc(hw_ctx))) { throw std::runtime_error("Failed to create VAAPI frame context."); } auto frames_ctx = (AVHWFramesContext*)(hw_frames_ref->data); frames_ctx->format = AV_PIX_FMT_VAAPI; frames_ctx->sw_format = input_frame.avFormat(); frames_ctx->width = input_frame.imageInfo().extent.width; frames_ctx->height = input_frame.imageInfo().extent.height; frames_ctx->initial_pool_size = 1; if ((err = av_hwframe_ctx_init(hw_frames_ref)) < 0) { av_buffer_unref(&hw_frames_ref); throw alvr::AvException("Failed to initialize VAAPI frame context:", err); } encoder_frame = av_frame_alloc(); if (vk_ctx.intel || getenv("ALVR_VAAPI_IMPORT_SURFACE")) { Info("Importing VA surface"); DrmImage drm; mapped_frame = import_frame(hw_frames_ref, drm); r->ImportOutput(drm); } else { mapped_frame = map_frame(hw_frames_ref, drm_ctx, input_frame); } filter_graph = avfilter_graph_alloc(); AVFilterInOut* outputs = avfilter_inout_alloc(); AVFilterInOut* inputs = avfilter_inout_alloc(); std::stringstream buffer_filter_args; buffer_filter_args << "video_size=" << mapped_frame->width << "x" << mapped_frame->height; buffer_filter_args << ":pix_fmt=" << mapped_frame->format; buffer_filter_args << ":time_base=" << encoder_ctx->time_base.num << "/" << encoder_ctx->time_base.den; if ((err = avfilter_graph_create_filter( &filter_in, avfilter_get_by_name("buffer"), "in", buffer_filter_args.str().c_str(), NULL, filter_graph ))) { throw alvr::AvException("filter_in creation failed:", err); } AVBufferSrcParameters* par = av_buffersrc_parameters_alloc(); memset(par, 0, sizeof(*par)); par->format = AV_PIX_FMT_NONE; par->hw_frames_ctx = av_buffer_ref(mapped_frame->hw_frames_ctx); av_buffersrc_parameters_set(filter_in, par); av_free(par); if ((err = avfilter_graph_create_filter( &filter_out, avfilter_get_by_name("buffersink"), "out", NULL, NULL, filter_graph ))) { throw alvr::AvException("filter_out creation failed:", err); } outputs->name = av_strdup("in"); outputs->filter_ctx = filter_in; outputs->pad_idx = 0; outputs->next = NULL; inputs->name = av_strdup("out"); inputs->filter_ctx = filter_out; inputs->pad_idx = 0; inputs->next = NULL; std::string filters = "scale_vaapi=out_range=full:format="; if ((Settings::Instance().m_codec == ALVR_CODEC_HEVC || Settings::Instance().m_codec == ALVR_CODEC_AV1) && Settings::Instance().m_use10bitEncoder) { filters += "p010"; } else { filters += "nv12"; } if ((err = avfilter_graph_parse_ptr(filter_graph, filters.c_str(), &inputs, &outputs, NULL)) < 0) { throw alvr::AvException("avfilter_graph_parse_ptr failed:", err); } avfilter_inout_free(&outputs); avfilter_inout_free(&inputs); for (unsigned i = 0; i < filter_graph->nb_filters; ++i) { filter_graph->filters[i]->hw_device_ctx = av_buffer_ref(hw_ctx); } if ((err = avfilter_graph_config(filter_graph, NULL))) { throw alvr::AvException("avfilter_graph_config failed:", err); } } alvr::EncodePipelineVAAPI::~EncodePipelineVAAPI() { // Commented because freeing it here causes a gpu reset, it should be cleaned up away // avcodec_free_context(&encoder_ctx); // avfilter_graph_free(&filter_graph); // av_frame_free(&mapped_frame); // av_frame_free(&encoder_frame); // av_buffer_unref(&hw_ctx); // av_buffer_unref(&drm_ctx); } void alvr::EncodePipelineVAAPI::PushFrame(uint64_t targetTimestampNs, bool idr) { r->Sync(); timestamp.cpu = std::chrono::duration_cast( std::chrono::steady_clock::now().time_since_epoch() ) .count(); int err = av_buffersrc_add_frame_flags( filter_in, mapped_frame, AV_BUFFERSRC_FLAG_PUSH | AV_BUFFERSRC_FLAG_KEEP_REF ); if (err != 0) { throw alvr::AvException("av_buffersrc_add_frame failed", err); } err = av_buffersink_get_frame(filter_out, encoder_frame); if (err != 0) { throw alvr::AvException("av_buffersink_get_frame failed", err); } encoder_frame->pict_type = idr ? AV_PICTURE_TYPE_I : AV_PICTURE_TYPE_NONE; encoder_frame->pts = targetTimestampNs; if ((err = avcodec_send_frame(encoder_ctx, encoder_frame)) < 0) { throw alvr::AvException("avcodec_send_frame failed: ", err); } av_frame_unref(encoder_frame); } void alvr::EncodePipelineVAAPI::SetParams(FfiDynamicEncoderParams params) { if (!params.updated) { return; } encoder_ctx->bit_rate = params.bitrate_bps; encoder_ctx->framerate = AVRational { int(params.framerate * 1000), 1000 }; encoder_ctx->rc_buffer_size = encoder_ctx->bit_rate / params.framerate; encoder_ctx->rc_max_rate = encoder_ctx->bit_rate; encoder_ctx->rc_initial_buffer_occupancy = encoder_ctx->rc_buffer_size; if (Settings::Instance().m_amdBitrateCorruptionFix) { RequestIDR(); } } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/EncodePipelineVAAPI.h ================================================ #pragma once #include "EncodePipeline.h" extern "C" struct AVBufferRef; extern "C" struct AVCodecContext; extern "C" struct AVFilterContext; extern "C" struct AVFilterGraph; extern "C" struct AVFrame; class Renderer; namespace alvr { #define PRESET_MODE_SPEED (0) #define PRESET_MODE_BALANCE (1) #define PRESET_MODE_QUALITY (2) enum EncoderQualityPreset { QUALITY = 0, BALANCED = 1, SPEED = 2 }; class EncodePipelineVAAPI : public EncodePipeline { public: ~EncodePipelineVAAPI(); EncodePipelineVAAPI( Renderer* render, VkContext& vk_ctx, VkFrame& input_frame, uint32_t width, uint32_t height ); void PushFrame(uint64_t targetTimestampNs, bool idr) override; void SetParams(FfiDynamicEncoderParams params) override; private: Renderer* r = nullptr; AVBufferRef* hw_ctx = nullptr; AVBufferRef* drm_ctx = nullptr; AVFrame* mapped_frame = nullptr; AVFrame* encoder_frame = nullptr; AVFilterGraph* filter_graph = nullptr; AVFilterContext* filter_in = nullptr; AVFilterContext* filter_out = nullptr; union vlVaQualityBits { unsigned int quality; struct { unsigned int valid_setting : 1; unsigned int preset_mode : 2; unsigned int pre_encode_mode : 1; unsigned int vbaq_mode : 1; unsigned int reservered : 27; }; }; }; ; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/FormatConverter.cpp ================================================ #include "FormatConverter.h" #include "alvr_server/bindings.h" FormatConverter::FormatConverter(Renderer* render) : r(render) { } FormatConverter::~FormatConverter() { for (const OutputImage& image : m_images) { vkUnmapMemory(r->m_dev, image.memory); vkDestroyImageView(r->m_dev, image.view, nullptr); vkDestroyImage(r->m_dev, image.image, nullptr); vkFreeMemory(r->m_dev, image.memory, nullptr); } vkDestroySemaphore(r->m_dev, m_output.semaphore, nullptr); vkDestroyQueryPool(r->m_dev, m_queryPool, nullptr); vkDestroyDescriptorSetLayout(r->m_dev, m_descriptorLayout, nullptr); vkDestroyImageView(r->m_dev, m_view, nullptr); vkDestroyShaderModule(r->m_dev, m_shader, nullptr); vkDestroyPipeline(r->m_dev, m_pipeline, nullptr); vkDestroyPipelineLayout(r->m_dev, m_pipelineLayout, nullptr); } void FormatConverter::init( VkImage image, VkImageCreateInfo imageCreateInfo, VkSemaphore semaphore, int count, const unsigned char* shaderData, unsigned shaderLen ) { m_images.resize(count); m_semaphore = semaphore; // Timestamp query VkQueryPoolCreateInfo queryPoolInfo = {}; queryPoolInfo.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO; queryPoolInfo.queryType = VK_QUERY_TYPE_TIMESTAMP; queryPoolInfo.queryCount = 1; VK_CHECK(vkCreateQueryPool(r->m_dev, &queryPoolInfo, nullptr, &m_queryPool)); // Command buffer VkCommandBufferAllocateInfo commandBufferInfo = {}; commandBufferInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; commandBufferInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; commandBufferInfo.commandPool = r->m_commandPool; commandBufferInfo.commandBufferCount = 1; VK_CHECK(vkAllocateCommandBuffers(r->m_dev, &commandBufferInfo, &m_commandBuffer)); // Descriptors VkDescriptorSetLayoutBinding descriptorBindings[2]; descriptorBindings[0] = {}; descriptorBindings[0].binding = 0; descriptorBindings[0].stageFlags = VK_SHADER_STAGE_COMPUTE_BIT; descriptorBindings[0].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorBindings[0].descriptorCount = 1; descriptorBindings[1] = {}; descriptorBindings[1].binding = 1; descriptorBindings[1].stageFlags = VK_SHADER_STAGE_COMPUTE_BIT; descriptorBindings[1].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorBindings[1].descriptorCount = count; VkDescriptorSetLayoutCreateInfo descriptorSetLayoutInfo = {}; descriptorSetLayoutInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO; descriptorSetLayoutInfo.flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_PUSH_DESCRIPTOR_BIT_KHR; descriptorSetLayoutInfo.bindingCount = 2; descriptorSetLayoutInfo.pBindings = descriptorBindings; VK_CHECK(vkCreateDescriptorSetLayout( r->m_dev, &descriptorSetLayoutInfo, nullptr, &m_descriptorLayout )); // Input image VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = imageCreateInfo.format; viewInfo.image = image; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VK_CHECK(vkCreateImageView(r->m_dev, &viewInfo, nullptr, &m_view)); // Output images for (int i = 0; i < count; ++i) { VkImageCreateInfo imageInfo = {}; imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO; imageInfo.imageType = VK_IMAGE_TYPE_2D; imageInfo.format = VK_FORMAT_R8_UNORM; imageInfo.extent.width = imageCreateInfo.extent.width; imageInfo.extent.height = imageCreateInfo.extent.height; imageInfo.extent.depth = 1; imageInfo.arrayLayers = 1; imageInfo.mipLevels = 1; imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; imageInfo.samples = VK_SAMPLE_COUNT_1_BIT; imageInfo.tiling = VK_IMAGE_TILING_LINEAR; imageInfo.usage = VK_IMAGE_USAGE_STORAGE_BIT; VK_CHECK(vkCreateImage(r->m_dev, &imageInfo, nullptr, &m_images[i].image)); VkMemoryRequirements memReqs; VkMemoryAllocateInfo memAllocInfo {}; memAllocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; vkGetImageMemoryRequirements(r->m_dev, m_images[i].image, &memReqs); memAllocInfo.allocationSize = memReqs.size; VkMemoryPropertyFlags memType = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; memAllocInfo.memoryTypeIndex = r->memoryTypeIndex(memType, memReqs.memoryTypeBits); VK_CHECK(vkAllocateMemory(r->m_dev, &memAllocInfo, nullptr, &m_images[i].memory)); VK_CHECK(vkBindImageMemory(r->m_dev, m_images[i].image, m_images[i].memory, 0)); VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = imageInfo.format; viewInfo.image = m_images[i].image; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VK_CHECK(vkCreateImageView(r->m_dev, &viewInfo, nullptr, &m_images[i].view)); VkImageMemoryBarrier imageBarrier = {}; imageBarrier.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; imageBarrier.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED; imageBarrier.newLayout = VK_IMAGE_LAYOUT_GENERAL; imageBarrier.image = m_images[i].image; imageBarrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; imageBarrier.subresourceRange.layerCount = 1; imageBarrier.subresourceRange.levelCount = 1; imageBarrier.srcAccessMask = 0; imageBarrier.dstAccessMask = VK_ACCESS_SHADER_WRITE_BIT; r->commandBufferBegin(); vkCmdPipelineBarrier( r->m_commandBuffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 0, nullptr, 0, nullptr, 1, &imageBarrier ); r->commandBufferSubmit(); VkImageSubresource subresource = {}; subresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; VkSubresourceLayout layout; vkGetImageSubresourceLayout(r->m_dev, m_images[i].image, &subresource, &layout); m_images[i].linesize = layout.rowPitch; VK_CHECK(vkMapMemory( r->m_dev, m_images[i].memory, 0, VK_WHOLE_SIZE, 0, reinterpret_cast(&m_images[i].mapped) )); } VkSemaphoreCreateInfo semInfo = {}; semInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO; VK_CHECK(vkCreateSemaphore(r->m_dev, &semInfo, nullptr, &m_output.semaphore)); // Shader VkShaderModuleCreateInfo moduleInfo = {}; moduleInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO; moduleInfo.codeSize = shaderLen; moduleInfo.pCode = (uint32_t*)shaderData; VK_CHECK(vkCreateShaderModule(r->m_dev, &moduleInfo, nullptr, &m_shader)); // Pipeline VkPipelineLayoutCreateInfo pipelineLayoutInfo = {}; pipelineLayoutInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO; pipelineLayoutInfo.setLayoutCount = 1; pipelineLayoutInfo.pSetLayouts = &m_descriptorLayout; VK_CHECK(vkCreatePipelineLayout(r->m_dev, &pipelineLayoutInfo, nullptr, &m_pipelineLayout)); VkPipelineShaderStageCreateInfo stageInfo = {}; stageInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO; stageInfo.stage = VK_SHADER_STAGE_COMPUTE_BIT; stageInfo.pName = "main"; stageInfo.module = m_shader; VkComputePipelineCreateInfo pipelineInfo = {}; pipelineInfo.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO; pipelineInfo.layout = m_pipelineLayout; pipelineInfo.stage = stageInfo; VK_CHECK(vkCreateComputePipelines(r->m_dev, nullptr, 1, &pipelineInfo, nullptr, &m_pipeline)); m_groupCountX = (imageCreateInfo.extent.width + 7) / 8; m_groupCountY = (imageCreateInfo.extent.height + 7) / 8; } void FormatConverter::Convert(uint8_t** data, int* linesize) { VkCommandBufferBeginInfo commandBufferBegin = {}; commandBufferBegin.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; VK_CHECK(vkBeginCommandBuffer(m_commandBuffer, &commandBufferBegin)); vkCmdResetQueryPool(m_commandBuffer, m_queryPool, 0, 1); vkCmdBindPipeline(m_commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, m_pipeline); std::vector descriptorWriteSets; VkDescriptorImageInfo descriptorImageInfoIn = {}; descriptorImageInfoIn.imageView = m_view; descriptorImageInfoIn.imageLayout = VK_IMAGE_LAYOUT_GENERAL; VkWriteDescriptorSet descriptorWriteSet = {}; descriptorWriteSet.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; descriptorWriteSet.descriptorCount = 1; descriptorWriteSet.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorWriteSet.pImageInfo = &descriptorImageInfoIn; descriptorWriteSet.dstBinding = 0; descriptorWriteSets.push_back(descriptorWriteSet); VkDescriptorImageInfo descriptorImageInfoOuts[3] = {}; for (size_t i = 0; i < m_images.size(); ++i) { descriptorImageInfoOuts[i].imageView = m_images[i].view; descriptorImageInfoOuts[i].imageLayout = VK_IMAGE_LAYOUT_GENERAL; descriptorWriteSet.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorWriteSet.pImageInfo = &descriptorImageInfoOuts[i]; descriptorWriteSet.dstBinding = 1; descriptorWriteSet.dstArrayElement = i; descriptorWriteSets.push_back(descriptorWriteSet); } r->d.vkCmdPushDescriptorSetKHR( m_commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, m_pipelineLayout, 0, descriptorWriteSets.size(), descriptorWriteSets.data() ); vkCmdDispatch(m_commandBuffer, m_groupCountX, m_groupCountY, 1); vkCmdWriteTimestamp(m_commandBuffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, m_queryPool, 0); vkEndCommandBuffer(m_commandBuffer); VkPipelineStageFlags waitStage = VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT; VkSubmitInfo submitInfo = {}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.waitSemaphoreCount = 1; submitInfo.pWaitSemaphores = &m_semaphore; submitInfo.pWaitDstStageMask = &waitStage; submitInfo.signalSemaphoreCount = 1; submitInfo.pSignalSemaphores = &m_output.semaphore; submitInfo.commandBufferCount = 1; submitInfo.pCommandBuffers = &m_commandBuffer; VK_CHECK(vkQueueSubmit(r->m_queue, 1, &submitInfo, nullptr)); for (size_t i = 0; i < m_images.size(); ++i) { data[i] = m_images[i].mapped; linesize[i] = m_images[i].linesize; } } void FormatConverter::Sync() { VkPipelineStageFlags waitStage = VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT; VkSubmitInfo submitInfo = {}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.waitSemaphoreCount = 1; submitInfo.pWaitSemaphores = &m_output.semaphore; submitInfo.pWaitDstStageMask = &waitStage; VK_CHECK(vkQueueSubmit(r->m_queue, 1, &submitInfo, r->m_fence)); VK_CHECK(vkWaitForFences(r->m_dev, 1, &r->m_fence, VK_TRUE, UINT64_MAX)); VK_CHECK(vkResetFences(r->m_dev, 1, &r->m_fence)); } uint64_t FormatConverter::GetTimestamp() { uint64_t query; VK_CHECK(vkGetQueryPoolResults( r->m_dev, m_queryPool, 0, 1, sizeof(uint64_t), &query, sizeof(uint64_t), VK_QUERY_RESULT_64_BIT )); return query * r->m_timestampPeriod; } RgbToYuv420::RgbToYuv420( Renderer* render, VkImage image, VkImageCreateInfo imageInfo, VkSemaphore semaphore ) : FormatConverter(render) { init( image, imageInfo, semaphore, 3, RGBTOYUV420_SHADER_COMP_SPV_PTR, RGBTOYUV420_SHADER_COMP_SPV_LEN ); } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/FormatConverter.h ================================================ #pragma once #include "Renderer.h" class FormatConverter { public: struct Output { VkSemaphore semaphore = VK_NULL_HANDLE; }; virtual ~FormatConverter(); Output GetOutput(); void Convert(uint8_t** data, int* linesize); void Sync(); uint64_t GetTimestamp(); protected: struct OutputImage { VkImage image = VK_NULL_HANDLE; VkDeviceMemory memory = VK_NULL_HANDLE; VkImageView view = VK_NULL_HANDLE; VkSemaphore semaphore = VK_NULL_HANDLE; VkDeviceSize linesize = 0; uint8_t* mapped = nullptr; }; explicit FormatConverter(Renderer* render); void init( VkImage image, VkImageCreateInfo imageCreateInfo, VkSemaphore semaphore, int count, const unsigned char* shaderData, unsigned shaderLen ); Renderer* r; VkQueryPool m_queryPool = VK_NULL_HANDLE; VkCommandBuffer m_commandBuffer = VK_NULL_HANDLE; VkDescriptorSetLayout m_descriptorLayout = VK_NULL_HANDLE; VkImageView m_view = VK_NULL_HANDLE; VkSemaphore m_semaphore = VK_NULL_HANDLE; VkShaderModule m_shader = VK_NULL_HANDLE; VkPipelineLayout m_pipelineLayout = VK_NULL_HANDLE; VkPipeline m_pipeline = VK_NULL_HANDLE; uint32_t m_groupCountX = 0; uint32_t m_groupCountY = 0; std::vector m_images; Output m_output; }; class RgbToYuv420 : public FormatConverter { public: explicit RgbToYuv420( Renderer* render, VkImage image, VkImageCreateInfo imageInfo, VkSemaphore semaphore ); }; ================================================ FILE: alvr/server_openvr/cpp/platform/linux/FrameRender.cpp ================================================ #include "FrameRender.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "alvr_server/bindings.h" #include #include FrameRender::FrameRender(alvr::VkContext& ctx, init_packet& init, int fds[]) : Renderer( ctx.get_vk_instance(), ctx.get_vk_device(), ctx.get_vk_phys_device(), ctx.get_vk_queue_family_index(), ctx.get_vk_device_extensions() ) { m_quadShaderSize = QUAD_SHADER_COMP_SPV_LEN; m_quadShaderCode = reinterpret_cast(QUAD_SHADER_COMP_SPV_PTR); Startup( init.image_create_info.extent.width, init.image_create_info.extent.height, init.image_create_info.format ); for (size_t i = 0; i < 3; ++i) { AddImage(init.image_create_info, init.mem_index, fds[2 * i], fds[2 * i + 1]); } m_width = Settings::Instance().m_renderWidth; m_height = Settings::Instance().m_renderHeight; Info("FrameRender: Input size %ux%u", m_width, m_height); if (Settings::Instance().m_force_sw_encoding) { m_handle = ExternalHandle::None; } else if (ctx.amd || ctx.intel) { m_handle = ExternalHandle::DmaBuf; } else if (ctx.nvidia) { m_handle = ExternalHandle::OpaqueFd; } setupCustomShaders("pre"); if (Settings::Instance().m_enableColorCorrection) { setupColorCorrection(); } if (Settings::Instance().m_enableFoveatedEncoding) { setupFoveatedRendering(); } setupCustomShaders("post"); if (m_pipelines.empty()) { RenderPipeline* pipeline = new RenderPipeline(this); pipeline->SetShader(QUAD_SHADER_COMP_SPV_PTR, QUAD_SHADER_COMP_SPV_LEN); m_pipelines.push_back(pipeline); AddPipeline(pipeline); } Info("FrameRender: Output size %ux%u", m_width, m_height); } FrameRender::~FrameRender() { for (RenderPipeline* pipeline : m_pipelines) { delete pipeline; } } FrameRender::Output FrameRender::CreateOutput() { Renderer::CreateOutput(m_width, m_height, m_handle); return GetOutput(); } uint32_t FrameRender::GetEncodingWidth() const { return m_width; } uint32_t FrameRender::GetEncodingHeight() const { return m_height; } void FrameRender::setupColorCorrection() { std::vector entries; #define ENTRY(x, v) \ m_colorCorrectionConstants.x = v; \ entries.push_back( \ { (uint32_t)entries.size(), offsetof(ColorCorrection, x), sizeof(ColorCorrection::x) } \ ); ENTRY(renderWidth, m_width); ENTRY(renderHeight, m_height); ENTRY(brightness, Settings::Instance().m_brightness); ENTRY(contrast, Settings::Instance().m_contrast + 1.f); ENTRY(saturation, Settings::Instance().m_saturation + 1.f); ENTRY(gamma, Settings::Instance().m_gamma); ENTRY(sharpening, Settings::Instance().m_sharpening); #undef ENTRY RenderPipeline* pipeline = new RenderPipeline(this); pipeline->SetShader(COLOR_SHADER_COMP_SPV_PTR, COLOR_SHADER_COMP_SPV_LEN); pipeline->SetConstants(&m_colorCorrectionConstants, std::move(entries)); m_pipelines.push_back(pipeline); AddPipeline(pipeline); } void FrameRender::setupFoveatedRendering() { float targetEyeWidth = (float)m_width / 2; float targetEyeHeight = (float)m_height; float centerSizeX = (float)Settings::Instance().m_foveationCenterSizeX; float centerSizeY = (float)Settings::Instance().m_foveationCenterSizeY; float centerShiftX = (float)Settings::Instance().m_foveationCenterShiftX; float centerShiftY = (float)Settings::Instance().m_foveationCenterShiftY; float edgeRatioX = (float)Settings::Instance().m_foveationEdgeRatioX; float edgeRatioY = (float)Settings::Instance().m_foveationEdgeRatioY; float edgeSizeX = targetEyeWidth - centerSizeX * targetEyeWidth; float edgeSizeY = targetEyeHeight - centerSizeY * targetEyeHeight; float centerSizeXAligned = 1. - ceil(edgeSizeX / (edgeRatioX * 2.)) * (edgeRatioX * 2.) / targetEyeWidth; float centerSizeYAligned = 1. - ceil(edgeSizeY / (edgeRatioY * 2.)) * (edgeRatioY * 2.) / targetEyeHeight; float edgeSizeXAligned = targetEyeWidth - centerSizeXAligned * targetEyeWidth; float edgeSizeYAligned = targetEyeHeight - centerSizeYAligned * targetEyeHeight; float centerShiftXAligned = ceil(centerShiftX * edgeSizeXAligned / (edgeRatioX * 2.)) * (edgeRatioX * 2.) / edgeSizeXAligned; float centerShiftYAligned = ceil(centerShiftY * edgeSizeYAligned / (edgeRatioY * 2.)) * (edgeRatioY * 2.) / edgeSizeYAligned; float foveationScaleX = (centerSizeXAligned + (1. - centerSizeXAligned) / edgeRatioX); float foveationScaleY = (centerSizeYAligned + (1. - centerSizeYAligned) / edgeRatioY); float optimizedEyeWidth = foveationScaleX * targetEyeWidth; float optimizedEyeHeight = foveationScaleY * targetEyeHeight; // round the frame dimensions to a number of pixel multiple of 32 for the encoder auto optimizedEyeWidthAligned = (uint32_t)ceil(optimizedEyeWidth / 32.f) * 32; auto optimizedEyeHeightAligned = (uint32_t)ceil(optimizedEyeHeight / 32.f) * 32; float eyeWidthRatioAligned = optimizedEyeWidth / optimizedEyeWidthAligned; float eyeHeightRatioAligned = optimizedEyeHeight / optimizedEyeHeightAligned; m_width = optimizedEyeWidthAligned * 2; m_height = optimizedEyeHeightAligned; std::vector entries; #define ENTRY(x, v) \ m_foveatedRenderingConstants.x = v; \ entries.push_back( \ { (uint32_t)entries.size(), offsetof(FoveationVars, x), sizeof(FoveationVars::x) } \ ); ENTRY(eyeWidthRatio, eyeWidthRatioAligned); ENTRY(eyeHeightRatio, eyeHeightRatioAligned); ENTRY(centerSizeX, centerSizeXAligned); ENTRY(centerSizeY, centerSizeYAligned); ENTRY(centerShiftX, centerShiftXAligned); ENTRY(centerShiftY, centerShiftYAligned); ENTRY(edgeRatioX, edgeRatioX); ENTRY(edgeRatioY, edgeRatioY); #undef ENTRY RenderPipeline* pipeline = new RenderPipeline(this); pipeline->SetShader(FFR_SHADER_COMP_SPV_PTR, FFR_SHADER_COMP_SPV_LEN); pipeline->SetConstants(&m_foveatedRenderingConstants, std::move(entries)); m_pipelines.push_back(pipeline); AddPipeline(pipeline); } void FrameRender::setupCustomShaders(const std::string& stage) { try { const std::filesystem::path shadersDir = std::filesystem::path(g_sessionPath).replace_filename("shaders"); for (const auto& entry : std::filesystem::directory_iterator(shadersDir / std::filesystem::path(stage))) { std::ifstream fs(entry.path(), std::ios::binary | std::ios::in); uint32_t magic = 0; fs.read((char*)&magic, sizeof(uint32_t)); if (magic != 0x07230203) { Warn("FrameRender: Shader file %s is not a SPIR-V file", entry.path().c_str()); continue; } Info( "FrameRender: Adding [%s] shader %s", stage.c_str(), entry.path().filename().c_str() ); RenderPipeline* pipeline = new RenderPipeline(this); pipeline->SetShader(entry.path().c_str()); m_pipelines.push_back(pipeline); AddPipeline(pipeline); } } catch (...) { } } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/FrameRender.h ================================================ #pragma once #include "Renderer.h" #include "ffmpeg_helper.h" #include "protocol.h" class FrameRender : public Renderer { public: explicit FrameRender(alvr::VkContext& ctx, init_packet& init, int fds[]); ~FrameRender(); Output CreateOutput(); uint32_t GetEncodingWidth() const; uint32_t GetEncodingHeight() const; private: struct ColorCorrection { float renderWidth; float renderHeight; float brightness; float contrast; float saturation; float gamma; float sharpening; }; struct FoveationVars { float eyeWidthRatio; float eyeHeightRatio; float centerSizeX; float centerSizeY; float centerShiftX; float centerShiftY; float edgeRatioX; float edgeRatioY; }; void setupColorCorrection(); void setupFoveatedRendering(); void setupCustomShaders(const std::string& stage); uint32_t m_width; uint32_t m_height; ExternalHandle m_handle = ExternalHandle::None; ColorCorrection m_colorCorrectionConstants; FoveationVars m_foveatedRenderingConstants; std::vector m_pipelines; }; ================================================ FILE: alvr/server_openvr/cpp/platform/linux/Renderer.cpp ================================================ #include "Renderer.h" #include #include #include #include #include #include #ifndef DRM_FORMAT_INVALID #define DRM_FORMAT_INVALID 0 #define fourcc_code(a, b, c, d) \ ((uint32_t)(a) | ((uint32_t)(b) << 8) | ((uint32_t)(c) << 16) | ((uint32_t)(d) << 24)) #define DRM_FORMAT_ARGB8888 fourcc_code('A', 'R', '2', '4') #define DRM_FORMAT_ABGR8888 fourcc_code('A', 'B', '2', '4') #define fourcc_mod_code(vendor, val) ((((uint64_t)vendor) << 56) | ((val) & 0x00ffffffffffffffULL)) #define DRM_FORMAT_MOD_INVALID fourcc_mod_code(0, ((1ULL << 56) - 1)) #define DRM_FORMAT_MOD_LINEAR fourcc_mod_code(0, 0) #define DRM_FORMAT_MOD_VENDOR_AMD 0x02 #define AMD_FMT_MOD_DCC_SHIFT 13 #define AMD_FMT_MOD_DCC_MASK 0x1 #define IS_AMD_FMT_MOD(val) (((val) >> 56) == DRM_FORMAT_MOD_VENDOR_AMD) #define AMD_FMT_MOD_GET(field, value) \ (((value) >> AMD_FMT_MOD_##field##_SHIFT) & AMD_FMT_MOD_##field##_MASK) #endif struct Vertex { float position[2]; }; static uint32_t to_drm_format(VkFormat format) { switch (format) { case VK_FORMAT_B8G8R8A8_UNORM: return DRM_FORMAT_ARGB8888; case VK_FORMAT_R8G8B8A8_UNORM: return DRM_FORMAT_ABGR8888; default: std::cerr << "Unsupported format " << format << std::endl; return DRM_FORMAT_INVALID; } } static bool filter_modifier(uint64_t modifier) { if (IS_AMD_FMT_MOD(modifier)) { // DCC not supported as encode input if (AMD_FMT_MOD_GET(DCC, modifier)) { return false; } } return true; } Renderer::Renderer( const VkInstance& inst, const VkDevice& dev, const VkPhysicalDevice& physDev, uint32_t queueIdx, const std::vector& devExtensions ) : m_inst(inst) , m_dev(dev) , m_physDev(physDev) , m_queueFamilyIndex(queueIdx) { auto checkExtension = [devExtensions](const char* name) { return std::find_if( devExtensions.begin(), devExtensions.end(), [name](const char* ext) { return strcmp(ext, name) == 0; } ) != devExtensions.end(); }; d.haveDmaBuf = checkExtension(VK_EXT_EXTERNAL_MEMORY_DMA_BUF_EXTENSION_NAME); d.haveDrmModifiers = checkExtension(VK_EXT_IMAGE_DRM_FORMAT_MODIFIER_EXTENSION_NAME); d.haveCalibratedTimestamps = checkExtension(VK_EXT_CALIBRATED_TIMESTAMPS_EXTENSION_NAME); if (!checkExtension(VK_KHR_PUSH_DESCRIPTOR_EXTENSION_NAME)) { throw std::runtime_error("Vulkan: Required extension " VK_KHR_PUSH_DESCRIPTOR_EXTENSION_NAME " not available"); } #define VK_LOAD_PFN(name) d.name = (PFN_##name)vkGetInstanceProcAddr(m_inst, #name) VK_LOAD_PFN(vkImportSemaphoreFdKHR); VK_LOAD_PFN(vkGetMemoryFdKHR); VK_LOAD_PFN(vkGetMemoryFdPropertiesKHR); VK_LOAD_PFN(vkGetImageDrmFormatModifierPropertiesEXT); VK_LOAD_PFN(vkGetCalibratedTimestampsEXT); VK_LOAD_PFN(vkCmdPushDescriptorSetKHR); #undef VK_LOAD_PFN VkPhysicalDeviceProperties props = {}; vkGetPhysicalDeviceProperties(m_physDev, &props); m_timestampPeriod = props.limits.timestampPeriod; } Renderer::~Renderer() { vkDeviceWaitIdle(m_dev); for (const InputImage& image : m_images) { vkDestroyImageView(m_dev, image.view, nullptr); vkDestroyImage(m_dev, image.image, nullptr); vkFreeMemory(m_dev, image.memory, nullptr); vkDestroySemaphore(m_dev, image.semaphore, nullptr); } for (const StagingImage& image : m_stagingImages) { vkDestroyImageView(m_dev, image.view, nullptr); vkDestroyImage(m_dev, image.image, nullptr); vkFreeMemory(m_dev, image.memory, nullptr); } vkDestroyImageView(m_dev, m_output.view, nullptr); vkDestroyImage(m_dev, m_output.image, nullptr); vkFreeMemory(m_dev, m_output.memory, nullptr); vkDestroySemaphore(m_dev, m_output.semaphore, nullptr); vkDestroyQueryPool(m_dev, m_queryPool, nullptr); vkDestroyCommandPool(m_dev, m_commandPool, nullptr); vkDestroySampler(m_dev, m_sampler, nullptr); vkDestroyDescriptorSetLayout(m_dev, m_descriptorLayout, nullptr); vkDestroyFence(m_dev, m_fence, nullptr); } void Renderer::Startup(uint32_t width, uint32_t height, VkFormat format) { m_format = format; m_imageSize.width = width; m_imageSize.height = height; vkGetDeviceQueue(m_dev, m_queueFamilyIndex, 0, &m_queue); // Timestamp query VkQueryPoolCreateInfo queryPoolInfo = {}; queryPoolInfo.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO; queryPoolInfo.queryType = VK_QUERY_TYPE_TIMESTAMP; queryPoolInfo.queryCount = 2; VK_CHECK(vkCreateQueryPool(m_dev, &queryPoolInfo, nullptr, &m_queryPool)); // Command buffer VkCommandPoolCreateInfo cmdPoolInfo = {}; cmdPoolInfo.sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO; cmdPoolInfo.queueFamilyIndex = m_queueFamilyIndex; cmdPoolInfo.flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT; VK_CHECK(vkCreateCommandPool(m_dev, &cmdPoolInfo, nullptr, &m_commandPool)); VkCommandBufferAllocateInfo commandBufferInfo = {}; commandBufferInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; commandBufferInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; commandBufferInfo.commandPool = m_commandPool; commandBufferInfo.commandBufferCount = 1; VK_CHECK(vkAllocateCommandBuffers(m_dev, &commandBufferInfo, &m_commandBuffer)); // Sampler VkSamplerCreateInfo samplerInfo = {}; samplerInfo.sType = VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO; samplerInfo.magFilter = VK_FILTER_LINEAR; samplerInfo.minFilter = VK_FILTER_LINEAR; samplerInfo.mipmapMode = VK_SAMPLER_MIPMAP_MODE_NEAREST; samplerInfo.addressModeU = VK_SAMPLER_ADDRESS_MODE_REPEAT; samplerInfo.addressModeV = VK_SAMPLER_ADDRESS_MODE_REPEAT; samplerInfo.addressModeW = VK_SAMPLER_ADDRESS_MODE_REPEAT; samplerInfo.anisotropyEnable = VK_TRUE; samplerInfo.maxAnisotropy = 16.0f; samplerInfo.borderColor = VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK; VK_CHECK(vkCreateSampler(m_dev, &samplerInfo, nullptr, &m_sampler)); // Descriptors VkDescriptorSetLayoutBinding descriptorBindings[2] = {}; descriptorBindings[0].descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER; descriptorBindings[0].stageFlags = VK_SHADER_STAGE_COMPUTE_BIT; descriptorBindings[0].descriptorCount = 1; descriptorBindings[0].pImmutableSamplers = &m_sampler; descriptorBindings[0].binding = 0; descriptorBindings[1].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorBindings[1].stageFlags = VK_SHADER_STAGE_COMPUTE_BIT; descriptorBindings[1].descriptorCount = 1; descriptorBindings[1].binding = 1; VkDescriptorSetLayoutCreateInfo descriptorSetLayoutInfo = {}; descriptorSetLayoutInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO; descriptorSetLayoutInfo.flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_PUSH_DESCRIPTOR_BIT_KHR; descriptorSetLayoutInfo.bindingCount = 2; descriptorSetLayoutInfo.pBindings = descriptorBindings; VK_CHECK( vkCreateDescriptorSetLayout(m_dev, &descriptorSetLayoutInfo, nullptr, &m_descriptorLayout) ); // Fence VkFenceCreateInfo fenceInfo = {}; fenceInfo.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO; VK_CHECK(vkCreateFence(m_dev, &fenceInfo, nullptr, &m_fence)); } void Renderer::AddImage( VkImageCreateInfo imageInfo, size_t memoryIndex, int imageFd, int semaphoreFd ) { VkExternalMemoryImageCreateInfo extMemImageInfo = {}; extMemImageInfo.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO; extMemImageInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; imageInfo.pNext = &extMemImageInfo; imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; VkImage image; VK_CHECK(vkCreateImage(m_dev, &imageInfo, nullptr, &image)); VkMemoryRequirements req; vkGetImageMemoryRequirements(m_dev, image, &req); VkMemoryDedicatedAllocateInfo dedicatedMemInfo = {}; dedicatedMemInfo.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO; dedicatedMemInfo.image = image; VkImportMemoryFdInfoKHR importMemInfo = {}; importMemInfo.sType = VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR; importMemInfo.pNext = &dedicatedMemInfo; importMemInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; importMemInfo.fd = imageFd; VkMemoryAllocateInfo memAllocInfo = {}; memAllocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; memAllocInfo.pNext = &importMemInfo; memAllocInfo.allocationSize = req.size; memAllocInfo.memoryTypeIndex = memoryIndex; VkDeviceMemory mem; VK_CHECK(vkAllocateMemory(m_dev, &memAllocInfo, nullptr, &mem)); VK_CHECK(vkBindImageMemory(m_dev, image, mem, 0)); VkSemaphoreTypeCreateInfo timelineInfo = {}; timelineInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO; timelineInfo.semaphoreType = VK_SEMAPHORE_TYPE_TIMELINE; VkSemaphoreCreateInfo semInfo = {}; semInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO; semInfo.pNext = &timelineInfo; VkSemaphore semaphore; VK_CHECK(vkCreateSemaphore(m_dev, &semInfo, nullptr, &semaphore)); VkImportSemaphoreFdInfoKHR impSemInfo = {}; impSemInfo.sType = VK_STRUCTURE_TYPE_IMPORT_SEMAPHORE_FD_INFO_KHR; impSemInfo.semaphore = semaphore; impSemInfo.handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT; impSemInfo.fd = semaphoreFd; VK_CHECK(d.vkImportSemaphoreFdKHR(m_dev, &impSemInfo)); VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = imageInfo.format; viewInfo.image = image; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VkImageView view; VK_CHECK(vkCreateImageView(m_dev, &viewInfo, nullptr, &view)); m_images.push_back({ image, VK_IMAGE_LAYOUT_UNDEFINED, mem, semaphore, view }); } void Renderer::AddPipeline(RenderPipeline* pipeline) { pipeline->Build(); m_pipelines.push_back(pipeline); if (m_pipelines.size() > 1 && m_stagingImages.size() < 2) { addStagingImage(m_imageSize.width, m_imageSize.height); } } void Renderer::CreateOutput(uint32_t width, uint32_t height, ExternalHandle handle) { m_output.imageInfo = {}; m_output.imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO; m_output.imageInfo.imageType = VK_IMAGE_TYPE_2D; m_output.imageInfo.format = m_format; m_output.imageInfo.extent.width = width; m_output.imageInfo.extent.height = height; m_output.imageInfo.extent.depth = 1; m_output.imageInfo.mipLevels = 1; m_output.imageInfo.arrayLayers = 1; m_output.imageInfo.samples = VK_SAMPLE_COUNT_1_BIT; m_output.imageInfo.usage = VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_SAMPLED_BIT; m_output.imageInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE; m_output.imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; std::vector modifierProps; VkExternalMemoryImageCreateInfo extMemImageInfo = {}; extMemImageInfo.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO; if (d.haveDrmModifiers && handle == ExternalHandle::DmaBuf) { VkImageDrmFormatModifierListCreateInfoEXT modifierListInfo = {}; modifierListInfo.sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT; m_output.imageInfo.pNext = &modifierListInfo; m_output.imageInfo.tiling = VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; VkDrmFormatModifierPropertiesListEXT modifierPropsList = {}; modifierPropsList.sType = VK_STRUCTURE_TYPE_DRM_FORMAT_MODIFIER_PROPERTIES_LIST_EXT; VkFormatProperties2 formatProps = {}; formatProps.sType = VK_STRUCTURE_TYPE_FORMAT_PROPERTIES_2; formatProps.pNext = &modifierPropsList; vkGetPhysicalDeviceFormatProperties2(m_physDev, m_output.imageInfo.format, &formatProps); modifierProps.resize(modifierPropsList.drmFormatModifierCount); modifierPropsList.pDrmFormatModifierProperties = modifierProps.data(); vkGetPhysicalDeviceFormatProperties2(m_physDev, m_output.imageInfo.format, &formatProps); std::vector imageModifiers; std::cout << "Available modifiers:" << std::endl; for (const VkDrmFormatModifierPropertiesEXT& prop : modifierProps) { std::cout << "modifier: " << prop.drmFormatModifier << " planes: " << prop.drmFormatModifierPlaneCount << std::endl; if (!filter_modifier(prop.drmFormatModifier)) { std::cout << " filtered" << std::endl; continue; } VkPhysicalDeviceImageDrmFormatModifierInfoEXT modInfo = {}; modInfo.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_DRM_FORMAT_MODIFIER_INFO_EXT; modInfo.drmFormatModifier = prop.drmFormatModifier; modInfo.sharingMode = m_output.imageInfo.sharingMode; modInfo.queueFamilyIndexCount = m_output.imageInfo.queueFamilyIndexCount; modInfo.pQueueFamilyIndices = m_output.imageInfo.pQueueFamilyIndices; VkPhysicalDeviceImageFormatInfo2 formatInfo = {}; formatInfo.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_FORMAT_INFO_2; formatInfo.pNext = &modInfo; formatInfo.format = m_output.imageInfo.format; formatInfo.type = m_output.imageInfo.imageType; formatInfo.tiling = m_output.imageInfo.tiling; formatInfo.usage = m_output.imageInfo.usage; formatInfo.flags = m_output.imageInfo.flags; VkImageFormatProperties2 imageFormatProps = {}; imageFormatProps.sType = VK_STRUCTURE_TYPE_IMAGE_FORMAT_PROPERTIES_2; imageFormatProps.pNext = NULL; VkResult r = vkGetPhysicalDeviceImageFormatProperties2( m_physDev, &formatInfo, &imageFormatProps ); if (r == VK_SUCCESS) { imageModifiers.push_back(prop.drmFormatModifier); } } modifierListInfo.drmFormatModifierCount = imageModifiers.size(); modifierListInfo.pDrmFormatModifiers = imageModifiers.data(); extMemImageInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; modifierListInfo.pNext = &extMemImageInfo; VK_CHECK(vkCreateImage(m_dev, &m_output.imageInfo, nullptr, &m_output.image)); } else if (d.haveDmaBuf && handle == ExternalHandle::DmaBuf) { extMemImageInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; m_output.imageInfo.pNext = &extMemImageInfo; m_output.imageInfo.tiling = VK_IMAGE_TILING_LINEAR; VK_CHECK(vkCreateImage(m_dev, &m_output.imageInfo, nullptr, &m_output.image)); } else if (handle == ExternalHandle::OpaqueFd) { extMemImageInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; m_output.imageInfo.pNext = &extMemImageInfo; m_output.imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL; VK_CHECK(vkCreateImage(m_dev, &m_output.imageInfo, nullptr, &m_output.image)); } else { m_output.imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL; VK_CHECK(vkCreateImage(m_dev, &m_output.imageInfo, nullptr, &m_output.image)); } VkMemoryDedicatedRequirements mdr = {}; mdr.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS; VkMemoryRequirements2 memoryReqs = {}; memoryReqs.sType = VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2; memoryReqs.pNext = &mdr; VkImageMemoryRequirementsInfo2 memoryReqsInfo = {}; memoryReqsInfo.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2; memoryReqsInfo.image = m_output.image; vkGetImageMemoryRequirements2(m_dev, &memoryReqsInfo, &memoryReqs); m_output.size = memoryReqs.memoryRequirements.size; VkExportMemoryAllocateInfo memory_export_info = {}; memory_export_info.sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO; memory_export_info.handleTypes = extMemImageInfo.handleTypes; VkMemoryDedicatedAllocateInfo memory_dedicated_info = {}; memory_dedicated_info.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO; memory_dedicated_info.image = m_output.image; if (handle != ExternalHandle::None) { memory_dedicated_info.pNext = &memory_export_info; } VkMemoryAllocateInfo memi = {}; memi.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; memi.pNext = &memory_dedicated_info; memi.allocationSize = memoryReqs.memoryRequirements.size; memi.memoryTypeIndex = memoryTypeIndex( VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, memoryReqs.memoryRequirements.memoryTypeBits ); VK_CHECK(vkAllocateMemory(m_dev, &memi, nullptr, &m_output.memory)); VkBindImageMemoryInfo bimi = {}; bimi.sType = VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO; bimi.image = m_output.image; bimi.memory = m_output.memory; bimi.memoryOffset = 0; VK_CHECK(vkBindImageMemory2(m_dev, 1, &bimi)); // DRM export if (d.haveDmaBuf) { VkMemoryGetFdInfoKHR memoryGetFdInfo = {}; memoryGetFdInfo.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR; memoryGetFdInfo.memory = m_output.memory; memoryGetFdInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; VkResult res = d.vkGetMemoryFdKHR(m_dev, &memoryGetFdInfo, &m_output.drm.fd); if (res != VK_SUCCESS) { std::cout << "vkGetMemoryFdKHR " << result_to_str(res) << std::endl; } else { if (d.haveDrmModifiers) { VkImageDrmFormatModifierPropertiesEXT imageDrmProps = {}; imageDrmProps.sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_PROPERTIES_EXT; d.vkGetImageDrmFormatModifierPropertiesEXT(m_dev, m_output.image, &imageDrmProps); if (res != VK_SUCCESS) { std::cout << "vkGetImageDrmFormatModifierPropertiesEXT " << result_to_str(res) << std::endl; } else { m_output.drm.modifier = imageDrmProps.drmFormatModifier; for (VkDrmFormatModifierPropertiesEXT prop : modifierProps) { if (prop.drmFormatModifier == m_output.drm.modifier) { m_output.drm.planes = prop.drmFormatModifierPlaneCount; } } } } else { m_output.drm.modifier = DRM_FORMAT_MOD_INVALID; m_output.drm.planes = 1; } for (uint32_t i = 0; i < m_output.drm.planes; i++) { VkImageSubresource subresource = {}; if (d.haveDrmModifiers) { subresource.aspectMask = VK_IMAGE_ASPECT_MEMORY_PLANE_0_BIT_EXT << i; } else { subresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; } VkSubresourceLayout layout; vkGetImageSubresourceLayout(m_dev, m_output.image, &subresource, &layout); m_output.drm.strides[i] = layout.rowPitch; m_output.drm.offsets[i] = layout.offset; } } m_output.drm.format = to_drm_format(m_output.imageInfo.format); } VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = m_output.imageInfo.format; viewInfo.image = m_output.image; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VK_CHECK(vkCreateImageView(m_dev, &viewInfo, nullptr, &m_output.view)); VkSemaphoreCreateInfo semInfo = {}; semInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO; VK_CHECK(vkCreateSemaphore(m_dev, &semInfo, nullptr, &m_output.semaphore)); } void Renderer::ImportOutput(const DrmImage& drm) { vkDestroyImageView(m_dev, m_output.view, nullptr); vkDestroyImage(m_dev, m_output.image, nullptr); vkFreeMemory(m_dev, m_output.memory, nullptr); m_output.drm = drm; m_output.imageInfo.tiling = VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; VkExternalMemoryImageCreateInfo extMemImageInfo = {}; extMemImageInfo.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO; extMemImageInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; m_output.imageInfo.pNext = &extMemImageInfo; VkSubresourceLayout layouts[4] = {}; for (uint32_t i = 0; i < drm.planes; ++i) { layouts[i].offset = drm.offsets[i]; layouts[i].rowPitch = drm.strides[i]; } VkImageDrmFormatModifierExplicitCreateInfoEXT modifierInfo = {}; modifierInfo.sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_EXPLICIT_CREATE_INFO_EXT; modifierInfo.drmFormatModifier = drm.modifier; modifierInfo.drmFormatModifierPlaneCount = drm.planes; modifierInfo.pPlaneLayouts = layouts; extMemImageInfo.pNext = &modifierInfo; VK_CHECK(vkCreateImage(m_dev, &m_output.imageInfo, NULL, &m_output.image)); VkMemoryFdPropertiesKHR fdProps = {}; fdProps.sType = VK_STRUCTURE_TYPE_MEMORY_FD_PROPERTIES_KHR; VK_CHECK(d.vkGetMemoryFdPropertiesKHR( m_dev, VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT, drm.fd, &fdProps )); VkImageMemoryRequirementsInfo2 memoryReqsInfo = {}; memoryReqsInfo.image = m_output.image; memoryReqsInfo.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2; VkMemoryRequirements2 memoryReqs = {}; memoryReqs.sType = VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2; vkGetImageMemoryRequirements2(m_dev, &memoryReqsInfo, &memoryReqs); VkMemoryAllocateInfo memoryAllocInfo = {}; memoryAllocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; memoryAllocInfo.allocationSize = memoryReqs.memoryRequirements.size; memoryAllocInfo.memoryTypeIndex = memoryTypeIndex( VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, memoryReqs.memoryRequirements.memoryTypeBits ); VkImportMemoryFdInfoKHR importMemInfo = {}; importMemInfo.sType = VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR; importMemInfo.fd = drm.fd; importMemInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; memoryAllocInfo.pNext = &importMemInfo; VkMemoryDedicatedAllocateInfo dedicatedMemInfo = {}; dedicatedMemInfo.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO; dedicatedMemInfo.image = m_output.image; importMemInfo.pNext = &dedicatedMemInfo; VK_CHECK(vkAllocateMemory(m_dev, &memoryAllocInfo, NULL, &m_output.memory)); VkBindImageMemoryInfo bindInfo = {}; bindInfo.sType = VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO; bindInfo.image = m_output.image; bindInfo.memory = m_output.memory; bindInfo.memoryOffset = 0; VK_CHECK(vkBindImageMemory2(m_dev, 1, &bindInfo)); VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = m_output.imageInfo.format; viewInfo.image = m_output.image; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VK_CHECK(vkCreateImageView(m_dev, &viewInfo, nullptr, &m_output.view)); } void Renderer::Render(uint32_t index, uint64_t waitValue) { if (!m_inputImageCapture.empty()) { VkSemaphoreWaitInfo waitInfo = {}; waitInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO; waitInfo.semaphoreCount = 1; waitInfo.pSemaphores = &m_images[index].semaphore; waitInfo.pValues = &waitValue; VK_CHECK(vkWaitSemaphores(m_dev, &waitInfo, UINT64_MAX)); dumpImage( m_images[index].image, m_images[index].view, m_images[index].layout, m_imageSize.width, m_imageSize.height, m_inputImageCapture ); m_inputImageCapture.clear(); } VkCommandBufferBeginInfo commandBufferBegin = {}; commandBufferBegin.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; VK_CHECK(vkBeginCommandBuffer(m_commandBuffer, &commandBufferBegin)); vkCmdResetQueryPool(m_commandBuffer, m_queryPool, 0, 2); vkCmdWriteTimestamp(m_commandBuffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, m_queryPool, 0); for (size_t i = 0; i < m_pipelines.size(); ++i) { VkRect2D rect = {}; VkImage in = VK_NULL_HANDLE; VkImageView inView = VK_NULL_HANDLE; VkImageLayout* inLayout = nullptr; VkImage out = VK_NULL_HANDLE; VkImageView outView = VK_NULL_HANDLE; VkImageLayout* outLayout = nullptr; if (i == 0) { auto& img = m_images[index]; in = img.image; inView = img.view; inLayout = &img.layout; } else { auto& img = m_stagingImages[(i - 1) % m_stagingImages.size()]; in = img.image; inView = img.view; inLayout = &img.layout; } if (i == m_pipelines.size() - 1) { out = m_output.image; outView = m_output.view; outLayout = &m_output.layout; rect.extent.width = m_output.imageInfo.extent.width; rect.extent.height = m_output.imageInfo.extent.height; } else { auto& img = m_stagingImages[i % m_stagingImages.size()]; out = img.image; outView = img.view; outLayout = &img.layout; rect.extent = m_imageSize; } VkImageMemoryBarrier imageBarrier = {}; imageBarrier.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; imageBarrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; imageBarrier.subresourceRange.layerCount = 1; imageBarrier.subresourceRange.levelCount = 1; std::vector imageBarriers; if (*inLayout != VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL) { imageBarrier.image = in; imageBarrier.oldLayout = *inLayout; *inLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; imageBarrier.newLayout = *inLayout; imageBarrier.srcAccessMask = 0; imageBarrier.dstAccessMask = VK_ACCESS_SHADER_READ_BIT; imageBarriers.push_back(imageBarrier); } if (*outLayout != VK_IMAGE_LAYOUT_GENERAL) { imageBarrier.image = out; imageBarrier.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED; *outLayout = VK_IMAGE_LAYOUT_GENERAL; imageBarrier.newLayout = *outLayout; imageBarrier.srcAccessMask = 0; imageBarrier.dstAccessMask = VK_ACCESS_SHADER_WRITE_BIT; imageBarriers.push_back(imageBarrier); } if (imageBarriers.size()) { vkCmdPipelineBarrier( m_commandBuffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 0, nullptr, 0, nullptr, imageBarriers.size(), imageBarriers.data() ); } m_pipelines[i]->Render(inView, outView, rect); } vkCmdWriteTimestamp(m_commandBuffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, m_queryPool, 1); VK_CHECK(vkEndCommandBuffer(m_commandBuffer)); VkTimelineSemaphoreSubmitInfo timelineInfo = {}; timelineInfo.sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO; timelineInfo.waitSemaphoreValueCount = 1; timelineInfo.pWaitSemaphoreValues = &waitValue; VkPipelineStageFlags waitStage = VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT; VkSubmitInfo submitInfo = {}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.pNext = &timelineInfo; submitInfo.waitSemaphoreCount = 1; submitInfo.pWaitSemaphores = &m_images[index].semaphore; submitInfo.pWaitDstStageMask = &waitStage; submitInfo.signalSemaphoreCount = 1; submitInfo.pSignalSemaphores = &m_output.semaphore; submitInfo.commandBufferCount = 1; submitInfo.pCommandBuffers = &m_commandBuffer; VK_CHECK(vkQueueSubmit(m_queue, 1, &submitInfo, nullptr)); } void Renderer::Sync() { VkPipelineStageFlags waitStage = VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT; VkSubmitInfo submitInfo = {}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.waitSemaphoreCount = 1; submitInfo.pWaitSemaphores = &m_output.semaphore; submitInfo.pWaitDstStageMask = &waitStage; VK_CHECK(vkQueueSubmit(m_queue, 1, &submitInfo, m_fence)); VK_CHECK(vkWaitForFences(m_dev, 1, &m_fence, VK_TRUE, UINT64_MAX)); VK_CHECK(vkResetFences(m_dev, 1, &m_fence)); } Renderer::Output& Renderer::GetOutput() { return m_output; } Renderer::Timestamps Renderer::GetTimestamps() { if (!d.haveCalibratedTimestamps) { return { 0, 0, 0 }; } uint64_t queries[2]; VK_CHECK(vkGetQueryPoolResults( m_dev, m_queryPool, 0, 2, 2 * sizeof(uint64_t), queries, sizeof(uint64_t), VK_QUERY_RESULT_64_BIT )); queries[0] *= m_timestampPeriod; queries[1] *= m_timestampPeriod; VkCalibratedTimestampInfoEXT timestampInfo = {}; timestampInfo.sType = VK_STRUCTURE_TYPE_CALIBRATED_TIMESTAMP_INFO_EXT; timestampInfo.timeDomain = VK_TIME_DOMAIN_DEVICE_EXT; uint64_t deviation; uint64_t timestamp; VK_CHECK(d.vkGetCalibratedTimestampsEXT(m_dev, 1, ×tampInfo, ×tamp, &deviation)); timestamp *= m_timestampPeriod; if (!m_outputImageCapture.empty()) { dumpImage( m_output.image, m_output.view, m_output.layout, m_output.imageInfo.extent.width, m_output.imageInfo.extent.height, m_outputImageCapture ); m_outputImageCapture.clear(); } return { timestamp, queries[0], queries[1] }; } void Renderer::CaptureInputFrame(const std::string& filename) { m_inputImageCapture = filename; } void Renderer::CaptureOutputFrame(const std::string& filename) { m_outputImageCapture = filename; } std::string Renderer::result_to_str(VkResult result) { switch (result) { #define VAL(x) \ case x: \ return #x VAL(VK_SUCCESS); VAL(VK_NOT_READY); VAL(VK_TIMEOUT); VAL(VK_EVENT_SET); VAL(VK_EVENT_RESET); VAL(VK_INCOMPLETE); VAL(VK_ERROR_OUT_OF_HOST_MEMORY); VAL(VK_ERROR_OUT_OF_DEVICE_MEMORY); VAL(VK_ERROR_INITIALIZATION_FAILED); VAL(VK_ERROR_DEVICE_LOST); VAL(VK_ERROR_MEMORY_MAP_FAILED); VAL(VK_ERROR_LAYER_NOT_PRESENT); VAL(VK_ERROR_EXTENSION_NOT_PRESENT); VAL(VK_ERROR_FEATURE_NOT_PRESENT); VAL(VK_ERROR_INCOMPATIBLE_DRIVER); VAL(VK_ERROR_TOO_MANY_OBJECTS); VAL(VK_ERROR_FORMAT_NOT_SUPPORTED); VAL(VK_ERROR_FRAGMENTED_POOL); VAL(VK_ERROR_OUT_OF_POOL_MEMORY); VAL(VK_ERROR_INVALID_EXTERNAL_HANDLE); VAL(VK_ERROR_SURFACE_LOST_KHR); VAL(VK_ERROR_NATIVE_WINDOW_IN_USE_KHR); VAL(VK_SUBOPTIMAL_KHR); VAL(VK_ERROR_OUT_OF_DATE_KHR); VAL(VK_ERROR_INCOMPATIBLE_DISPLAY_KHR); VAL(VK_ERROR_VALIDATION_FAILED_EXT); VAL(VK_ERROR_INVALID_SHADER_NV); VAL(VK_ERROR_INVALID_DRM_FORMAT_MODIFIER_PLANE_LAYOUT_EXT); VAL(VK_ERROR_NOT_PERMITTED_EXT); VAL(VK_RESULT_MAX_ENUM); #undef VAL default: return "Unknown VkResult"; } } void Renderer::commandBufferBegin() { VkCommandBufferBeginInfo commandBufferBegin = {}; commandBufferBegin.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; VK_CHECK(vkBeginCommandBuffer(m_commandBuffer, &commandBufferBegin)); } void Renderer::commandBufferSubmit() { VK_CHECK(vkEndCommandBuffer(m_commandBuffer)); VkSubmitInfo submitInfo = {}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.commandBufferCount = 1; submitInfo.pCommandBuffers = &m_commandBuffer; VkFenceCreateInfo fenceInfo = {}; fenceInfo.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO; VkFence fence; VK_CHECK(vkCreateFence(m_dev, &fenceInfo, nullptr, &fence)); VK_CHECK(vkQueueSubmit(m_queue, 1, &submitInfo, fence)); VK_CHECK(vkWaitForFences(m_dev, 1, &fence, VK_TRUE, UINT64_MAX)); vkDestroyFence(m_dev, fence, nullptr); } void Renderer::addStagingImage(uint32_t width, uint32_t height) { VkImageCreateInfo imageInfo = {}; imageInfo = {}; imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO; imageInfo.imageType = VK_IMAGE_TYPE_2D; imageInfo.format = m_format; imageInfo.extent.width = width; imageInfo.extent.height = height; imageInfo.extent.depth = 1; imageInfo.mipLevels = 1; imageInfo.arrayLayers = 1; imageInfo.samples = VK_SAMPLE_COUNT_1_BIT; imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL; imageInfo.usage = VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_SAMPLED_BIT; imageInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE; imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; VkImage image; VK_CHECK(vkCreateImage(m_dev, &imageInfo, nullptr, &image)); VkMemoryRequirements memoryReqs; vkGetImageMemoryRequirements(m_dev, image, &memoryReqs); VkMemoryAllocateInfo memoryAllocInfo = {}; memoryAllocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; memoryAllocInfo.allocationSize = memoryReqs.size; memoryAllocInfo.memoryTypeIndex = memoryTypeIndex(VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, memoryReqs.memoryTypeBits); VkDeviceMemory memory; VK_CHECK(vkAllocateMemory(m_dev, &memoryAllocInfo, nullptr, &memory)); VK_CHECK(vkBindImageMemory(m_dev, image, memory, 0)); VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = imageInfo.format; viewInfo.image = image; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VkImageView view; VK_CHECK(vkCreateImageView(m_dev, &viewInfo, nullptr, &view)); m_stagingImages.push_back({ image, VK_IMAGE_LAYOUT_UNDEFINED, memory, view }); } void Renderer::dumpImage( VkImage image, VkImageView imageView, VkImageLayout imageLayout, uint32_t width, uint32_t height, const std::string& filename ) { VkImageCreateInfo imageInfo = {}; imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO; imageInfo.imageType = VK_IMAGE_TYPE_2D; imageInfo.format = VK_FORMAT_R8G8B8A8_UNORM; imageInfo.extent.width = width; imageInfo.extent.height = height; imageInfo.extent.depth = 1; imageInfo.arrayLayers = 1; imageInfo.mipLevels = 1; imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; imageInfo.samples = VK_SAMPLE_COUNT_1_BIT; imageInfo.tiling = VK_IMAGE_TILING_LINEAR; imageInfo.usage = VK_IMAGE_USAGE_STORAGE_BIT; VkImage dstImage; VK_CHECK(vkCreateImage(m_dev, &imageInfo, nullptr, &dstImage)); VkMemoryRequirements memReqs; VkMemoryAllocateInfo memAllocInfo {}; memAllocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; vkGetImageMemoryRequirements(m_dev, dstImage, &memReqs); memAllocInfo.allocationSize = memReqs.size; memAllocInfo.memoryTypeIndex = memoryTypeIndex( VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, memReqs.memoryTypeBits ); VkDeviceMemory dstMemory; VK_CHECK(vkAllocateMemory(m_dev, &memAllocInfo, nullptr, &dstMemory)); VK_CHECK(vkBindImageMemory(m_dev, dstImage, dstMemory, 0)); VkImageViewCreateInfo viewInfo = {}; viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; viewInfo.format = imageInfo.format; viewInfo.image = dstImage; viewInfo.subresourceRange = {}; viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; viewInfo.subresourceRange.baseMipLevel = 0; viewInfo.subresourceRange.levelCount = 1; viewInfo.subresourceRange.baseArrayLayer = 0; viewInfo.subresourceRange.layerCount = 1; viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; VkImageView dstView; VK_CHECK(vkCreateImageView(m_dev, &viewInfo, nullptr, &dstView)); std::array imageBarrierIn; imageBarrierIn[0] = {}; imageBarrierIn[0].sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; imageBarrierIn[0].oldLayout = imageLayout; imageBarrierIn[0].newLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; imageBarrierIn[0].image = image; imageBarrierIn[0].subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; imageBarrierIn[0].subresourceRange.layerCount = 1; imageBarrierIn[0].subresourceRange.levelCount = 1; imageBarrierIn[0].srcAccessMask = 0; imageBarrierIn[0].dstAccessMask = VK_ACCESS_SHADER_READ_BIT; imageBarrierIn[1] = {}; imageBarrierIn[1].sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; imageBarrierIn[1].oldLayout = VK_IMAGE_LAYOUT_UNDEFINED; imageBarrierIn[1].newLayout = VK_IMAGE_LAYOUT_GENERAL; imageBarrierIn[1].image = dstImage; imageBarrierIn[1].subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; imageBarrierIn[1].subresourceRange.layerCount = 1; imageBarrierIn[1].subresourceRange.levelCount = 1; imageBarrierIn[1].srcAccessMask = 0; imageBarrierIn[1].dstAccessMask = VK_ACCESS_SHADER_WRITE_BIT; // Shader VkShaderModuleCreateInfo moduleInfo = {}; moduleInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO; moduleInfo.codeSize = m_quadShaderSize; moduleInfo.pCode = m_quadShaderCode; VkShaderModule shader; VK_CHECK(vkCreateShaderModule(m_dev, &moduleInfo, nullptr, &shader)); // Pipeline VkPipelineLayoutCreateInfo pipelineLayoutInfo = {}; pipelineLayoutInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO; pipelineLayoutInfo.setLayoutCount = 1; pipelineLayoutInfo.pSetLayouts = &m_descriptorLayout; VkPipelineLayout pipelineLayout; VK_CHECK(vkCreatePipelineLayout(m_dev, &pipelineLayoutInfo, nullptr, &pipelineLayout)); VkPipelineShaderStageCreateInfo stageInfo = {}; stageInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO; stageInfo.stage = VK_SHADER_STAGE_COMPUTE_BIT; stageInfo.pName = "main"; stageInfo.module = shader; VkComputePipelineCreateInfo pipelineInfo = {}; pipelineInfo.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO; pipelineInfo.layout = pipelineLayout; pipelineInfo.stage = stageInfo; VkPipeline pipeline; VK_CHECK(vkCreateComputePipelines(m_dev, nullptr, 1, &pipelineInfo, nullptr, &pipeline)); std::array imageBarrierOut; imageBarrierOut[0] = {}; imageBarrierOut[0].sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; imageBarrierOut[0].oldLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; imageBarrierOut[0].newLayout = imageLayout; imageBarrierOut[0].image = image; imageBarrierOut[0].subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; imageBarrierOut[0].subresourceRange.layerCount = 1; imageBarrierOut[0].subresourceRange.levelCount = 1; imageBarrierOut[0].srcAccessMask = 0; imageBarrierOut[0].dstAccessMask = 0; imageBarrierOut[1] = {}; imageBarrierOut[1].sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; imageBarrierOut[1].oldLayout = VK_IMAGE_LAYOUT_GENERAL; imageBarrierOut[1].newLayout = VK_IMAGE_LAYOUT_GENERAL; imageBarrierOut[1].image = dstImage; imageBarrierOut[1].subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; imageBarrierOut[1].subresourceRange.layerCount = 1; imageBarrierOut[1].subresourceRange.levelCount = 1; imageBarrierOut[1].srcAccessMask = 0; imageBarrierOut[1].dstAccessMask = VK_ACCESS_MEMORY_READ_BIT; std::vector descriptorWriteSets; VkDescriptorImageInfo descriptorImageInfoIn = {}; descriptorImageInfoIn.imageView = imageView; descriptorImageInfoIn.imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; VkDescriptorImageInfo descriptorImageInfoOut = {}; descriptorImageInfoOut.imageView = dstView; descriptorImageInfoOut.imageLayout = VK_IMAGE_LAYOUT_GENERAL; VkWriteDescriptorSet descriptorWriteSet = {}; descriptorWriteSet.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; descriptorWriteSet.descriptorCount = 1; descriptorWriteSet.descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER; descriptorWriteSet.pImageInfo = &descriptorImageInfoIn; descriptorWriteSet.dstBinding = 0; descriptorWriteSets.push_back(descriptorWriteSet); descriptorWriteSet.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorWriteSet.pImageInfo = &descriptorImageInfoOut; descriptorWriteSet.dstBinding = 1; descriptorWriteSets.push_back(descriptorWriteSet); commandBufferBegin(); vkCmdPipelineBarrier( m_commandBuffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 0, nullptr, 0, nullptr, imageBarrierIn.size(), imageBarrierIn.data() ); vkCmdBindPipeline(m_commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline); d.vkCmdPushDescriptorSetKHR( m_commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipelineLayout, 0, descriptorWriteSets.size(), descriptorWriteSets.data() ); vkCmdDispatch( m_commandBuffer, (imageInfo.extent.width + 7) / 8, (imageInfo.extent.height + 7) / 8, 1 ); vkCmdPipelineBarrier( m_commandBuffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 0, nullptr, 0, nullptr, imageBarrierOut.size(), imageBarrierOut.data() ); commandBufferSubmit(); VkImageSubresource subresource = {}; subresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; VkSubresourceLayout layout; vkGetImageSubresourceLayout(m_dev, dstImage, &subresource, &layout); const char* imageData; VK_CHECK(vkMapMemory(m_dev, dstMemory, 0, VK_WHOLE_SIZE, 0, (void**)&imageData)); imageData += layout.offset; std::ofstream file(filename, std::ios::out | std::ios::binary); // PPM header file << "P6\n" << width << "\n" << height << "\n" << 255 << "\n"; // PPM binary pixel data for (uint32_t y = 0; y < height; y++) { uint32_t* row = (uint32_t*)imageData; for (uint32_t x = 0; x < width; x++) { file.write((char*)row++, 3); } imageData += layout.rowPitch; } file.close(); std::cout << "Image saved to \"" << filename << "\"" << std::endl; vkUnmapMemory(m_dev, dstMemory); vkFreeMemory(m_dev, dstMemory, nullptr); vkDestroyImage(m_dev, dstImage, nullptr); vkDestroyImageView(m_dev, dstView, nullptr); vkDestroyShaderModule(m_dev, shader, nullptr); vkDestroyPipeline(m_dev, pipeline, nullptr); vkDestroyPipelineLayout(m_dev, pipelineLayout, nullptr); } uint32_t Renderer::memoryTypeIndex(VkMemoryPropertyFlags properties, uint32_t typeBits) const { VkPhysicalDeviceMemoryProperties prop; vkGetPhysicalDeviceMemoryProperties(m_physDev, &prop); for (uint32_t i = 0; i < prop.memoryTypeCount; i++) { if ((prop.memoryTypes[i].propertyFlags & properties) == properties && typeBits & (1 << i)) { return i; } } return 0xFFFFFFFF; } // RenderPipeline RenderPipeline::RenderPipeline(Renderer* render) : r(render) { } RenderPipeline::~RenderPipeline() { vkDestroyShaderModule(r->m_dev, m_shader, nullptr); vkDestroyPipeline(r->m_dev, m_pipeline, nullptr); vkDestroyPipelineLayout(r->m_dev, m_pipelineLayout, nullptr); } void RenderPipeline::SetShader(const char* filename) { std::ifstream is(filename, std::ios::binary | std::ios::in | std::ios::ate); if (!is.is_open()) { std::cerr << "Failed to open shader file: " << filename << std::endl; return; } size_t size = is.tellg(); is.seekg(0, std::ios::beg); std::vector data(size); is.read(data.data(), size); SetShader((unsigned char*)data.data(), size); } void RenderPipeline::SetShader(const unsigned char* data, unsigned len) { VkShaderModuleCreateInfo moduleInfo = {}; moduleInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO; moduleInfo.codeSize = len; moduleInfo.pCode = (uint32_t*)data; VK_CHECK(vkCreateShaderModule(r->m_dev, &moduleInfo, nullptr, &m_shader)); } void RenderPipeline::Build() { VkPipelineLayoutCreateInfo pipelineLayoutInfo = {}; pipelineLayoutInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO; pipelineLayoutInfo.setLayoutCount = 1; pipelineLayoutInfo.pSetLayouts = &r->m_descriptorLayout; VK_CHECK(vkCreatePipelineLayout(r->m_dev, &pipelineLayoutInfo, nullptr, &m_pipelineLayout)); VkSpecializationInfo specInfo = {}; specInfo.mapEntryCount = m_constantEntries.size(); specInfo.pMapEntries = m_constantEntries.data(); specInfo.dataSize = m_constantSize; specInfo.pData = m_constant; VkPipelineShaderStageCreateInfo stageInfo = {}; stageInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO; stageInfo.stage = VK_SHADER_STAGE_COMPUTE_BIT; stageInfo.pName = "main"; stageInfo.module = m_shader; if (m_constant) { stageInfo.pSpecializationInfo = &specInfo; } VkComputePipelineCreateInfo pipelineInfo = {}; pipelineInfo.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO; pipelineInfo.layout = m_pipelineLayout; pipelineInfo.stage = stageInfo; VK_CHECK(vkCreateComputePipelines(r->m_dev, nullptr, 1, &pipelineInfo, nullptr, &m_pipeline)); } void RenderPipeline::Render(VkImageView in, VkImageView out, VkRect2D outSize) { vkCmdBindPipeline(r->m_commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, m_pipeline); VkDescriptorImageInfo descriptorImageInfoIn = {}; descriptorImageInfoIn.imageView = in; descriptorImageInfoIn.imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; VkDescriptorImageInfo descriptorImageInfoOut = {}; descriptorImageInfoOut.imageView = out; descriptorImageInfoOut.imageLayout = VK_IMAGE_LAYOUT_GENERAL; VkWriteDescriptorSet descriptorWriteSets[2] = {}; descriptorWriteSets[0].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; descriptorWriteSets[0].descriptorCount = 1; descriptorWriteSets[0].descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER; descriptorWriteSets[0].pImageInfo = &descriptorImageInfoIn; descriptorWriteSets[0].dstBinding = 0; descriptorWriteSets[1].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; descriptorWriteSets[1].descriptorCount = 1; descriptorWriteSets[1].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE; descriptorWriteSets[1].pImageInfo = &descriptorImageInfoOut; descriptorWriteSets[1].dstBinding = 1; r->d.vkCmdPushDescriptorSetKHR( r->m_commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, m_pipelineLayout, 0, 2, descriptorWriteSets ); vkCmdDispatch( r->m_commandBuffer, (outSize.extent.width + 7) / 8, (outSize.extent.height + 7) / 8, 1 ); } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/Renderer.h ================================================ #pragma once #include #include #include #include #include #define VK_CHECK(f) \ { \ VkResult res = (f); \ if (res != VK_SUCCESS) { \ std::cerr << Renderer::result_to_str(res) << "at" << __FILE__ << ":" << __LINE__ \ << std::endl; \ throw std::runtime_error( \ "Vulkan: " + Renderer::result_to_str(res) + "at " __FILE__ ":" \ + std::to_string(__LINE__) \ ); \ } \ } struct DrmImage { int fd = -1; uint32_t format = 0; uint64_t modifier = 0; uint32_t planes = 0; std::array strides; std::array offsets; }; class RenderPipeline; class Renderer { public: enum class ExternalHandle { None, DmaBuf, OpaqueFd }; struct Output { VkImage image = VK_NULL_HANDLE; VkImageLayout layout = VK_IMAGE_LAYOUT_UNDEFINED; VkImageCreateInfo imageInfo; VkDeviceSize size = 0; VkDeviceMemory memory = VK_NULL_HANDLE; VkSemaphore semaphore = VK_NULL_HANDLE; // --- VkImageView view = VK_NULL_HANDLE; // --- DrmImage drm; }; struct Timestamps { uint64_t now; uint64_t renderBegin; uint64_t renderComplete; }; explicit Renderer( const VkInstance& inst, const VkDevice& dev, const VkPhysicalDevice& physDev, uint32_t queueIdx, const std::vector& devExtensions ); virtual ~Renderer(); void Startup(uint32_t width, uint32_t height, VkFormat format); void AddImage(VkImageCreateInfo imageInfo, size_t memoryIndex, int imageFd, int semaphoreFd); void AddPipeline(RenderPipeline* pipeline); void CreateOutput(uint32_t width, uint32_t height, ExternalHandle handle); void ImportOutput(const DrmImage& drm); void Render(uint32_t index, uint64_t waitValue); void Sync(); Output& GetOutput(); Timestamps GetTimestamps(); void CaptureInputFrame(const std::string& filename); void CaptureOutputFrame(const std::string& filename); static std::string result_to_str(VkResult result); // private: struct InputImage { VkImage image = VK_NULL_HANDLE; VkImageLayout layout = VK_IMAGE_LAYOUT_UNDEFINED; VkDeviceMemory memory = VK_NULL_HANDLE; VkSemaphore semaphore = VK_NULL_HANDLE; VkImageView view = VK_NULL_HANDLE; }; struct StagingImage { VkImage image = VK_NULL_HANDLE; VkImageLayout layout = VK_IMAGE_LAYOUT_UNDEFINED; VkDeviceMemory memory = VK_NULL_HANDLE; VkImageView view = VK_NULL_HANDLE; }; void commandBufferBegin(); void commandBufferSubmit(); void addStagingImage(uint32_t width, uint32_t height); void dumpImage( VkImage image, VkImageView imageView, VkImageLayout imageLayout, uint32_t width, uint32_t height, const std::string& filename ); uint32_t memoryTypeIndex(VkMemoryPropertyFlags properties, uint32_t typeBits) const; struct { PFN_vkImportSemaphoreFdKHR vkImportSemaphoreFdKHR = nullptr; PFN_vkGetMemoryFdKHR vkGetMemoryFdKHR = nullptr; PFN_vkGetMemoryFdPropertiesKHR vkGetMemoryFdPropertiesKHR = nullptr; PFN_vkGetImageDrmFormatModifierPropertiesEXT vkGetImageDrmFormatModifierPropertiesEXT = nullptr; PFN_vkGetCalibratedTimestampsEXT vkGetCalibratedTimestampsEXT = nullptr; PFN_vkCmdPushDescriptorSetKHR vkCmdPushDescriptorSetKHR = nullptr; bool haveDmaBuf = false; bool haveDrmModifiers = false; bool haveCalibratedTimestamps = false; } d; Output m_output; std::vector m_images; std::vector m_stagingImages; std::vector m_pipelines; VkInstance m_inst = VK_NULL_HANDLE; VkDevice m_dev = VK_NULL_HANDLE; VkPhysicalDevice m_physDev = VK_NULL_HANDLE; VkQueue m_queue = VK_NULL_HANDLE; uint32_t m_queueFamilyIndex = 0; VkFormat m_format = VK_FORMAT_UNDEFINED; VkExtent2D m_imageSize = { 0, 0 }; VkQueryPool m_queryPool = VK_NULL_HANDLE; VkCommandPool m_commandPool = VK_NULL_HANDLE; VkSampler m_sampler = VK_NULL_HANDLE; VkDescriptorSetLayout m_descriptorLayout = VK_NULL_HANDLE; VkCommandBuffer m_commandBuffer = VK_NULL_HANDLE; VkFence m_fence = VK_NULL_HANDLE; double m_timestampPeriod = 0; size_t m_quadShaderSize = 0; const uint32_t* m_quadShaderCode = nullptr; std::string m_inputImageCapture; std::string m_outputImageCapture; }; class RenderPipeline { public: explicit RenderPipeline(Renderer* render); virtual ~RenderPipeline(); void SetShader(const char* filename); void SetShader(const unsigned char* data, unsigned len); template void SetConstants(const T* data, std::vector&& entries) { m_constant = static_cast(data); m_constantSize = sizeof(T); m_constantEntries = std::move(entries); } private: void Build(); void Render(VkImageView in, VkImageView out, VkRect2D outSize); Renderer* r; VkShaderModule m_shader = VK_NULL_HANDLE; const void* m_constant = nullptr; uint32_t m_constantSize = 0; std::vector m_constantEntries; VkPipeline m_pipeline = VK_NULL_HANDLE; VkPipelineLayout m_pipelineLayout = VK_NULL_HANDLE; friend class Renderer; }; ================================================ FILE: alvr/server_openvr/cpp/platform/linux/ffmpeg_helper.cpp ================================================ #include "ffmpeg_helper.h" #include #include #include #include #include #include "alvr_server/Logger.h" #include "alvr_server/bindings.h" extern "C" { #include #include #include } namespace { // it seems that ffmpeg does not provide this mapping AVPixelFormat vk_format_to_av_format(vk::Format vk_fmt) { for (int f = AV_PIX_FMT_NONE; f < AV_PIX_FMT_NB; ++f) { auto current_fmt = av_vkfmt_from_pixfmt(AVPixelFormat(f)); if (current_fmt and *current_fmt == (VkFormat)vk_fmt) return AVPixelFormat(f); } throw std::runtime_error("unsupported vulkan pixel format " + std::to_string((VkFormat)vk_fmt)); } } std::string alvr::AvException::makemsg(const std::string& msg, int averror) { char av_msg[AV_ERROR_MAX_STRING_SIZE]; av_strerror(averror, av_msg, sizeof(av_msg)); return msg + " " + av_msg; } alvr::VkContext::VkContext( const uint8_t* deviceUUID, const std::vector& requiredDeviceExtensions ) { std::vector instance_extensions = { VK_KHR_GET_PHYSICAL_DEVICE_PROPERTIES_2_EXTENSION_NAME, VK_KHR_SURFACE_EXTENSION_NAME, }; std::vector device_extensions = { VK_KHR_EXTERNAL_MEMORY_FD_EXTENSION_NAME, VK_KHR_EXTERNAL_SEMAPHORE_FD_EXTENSION_NAME, VK_EXT_EXTERNAL_MEMORY_DMA_BUF_EXTENSION_NAME, VK_EXT_IMAGE_DRM_FORMAT_MODIFIER_EXTENSION_NAME, VK_KHR_EXTERNAL_SEMAPHORE_FD_EXTENSION_NAME, VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME, VK_KHR_PUSH_DESCRIPTOR_EXTENSION_NAME, VK_KHR_SAMPLER_YCBCR_CONVERSION_EXTENSION_NAME, VK_EXT_PHYSICAL_DEVICE_DRM_EXTENSION_NAME, VK_EXT_CALIBRATED_TIMESTAMPS_EXTENSION_NAME, }; device_extensions.insert( device_extensions.end(), requiredDeviceExtensions.begin(), requiredDeviceExtensions.end() ); uint32_t instanceExtensionCount = 0; vkEnumerateInstanceExtensionProperties(nullptr, &instanceExtensionCount, nullptr); std::vector instanceExts(instanceExtensionCount); vkEnumerateInstanceExtensionProperties(nullptr, &instanceExtensionCount, instanceExts.data()); for (const char* name : instance_extensions) { auto it = std::find_if( instanceExts.begin(), instanceExts.end(), [name](VkExtensionProperties e) { return strcmp(e.extensionName, name) == 0; } ); if (it != instanceExts.end()) { instanceExtensions.push_back(name); } } VkApplicationInfo appInfo = {}; appInfo.sType = VK_STRUCTURE_TYPE_APPLICATION_INFO; appInfo.pApplicationName = "ALVR"; appInfo.apiVersion = VK_API_VERSION_1_2; VkInstanceCreateInfo instanceInfo = {}; instanceInfo.sType = VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO; instanceInfo.pApplicationInfo = &appInfo; #ifdef DEBUG const char* validationLayers[] = { "VK_LAYER_KHRONOS_validation" }; instanceInfo.ppEnabledLayerNames = validationLayers; instanceInfo.enabledLayerCount = 1; #endif instanceInfo.enabledExtensionCount = instanceExtensions.size(); instanceInfo.ppEnabledExtensionNames = instanceExtensions.data(); VK_CHECK(vkCreateInstance(&instanceInfo, nullptr, &instance)); uint32_t deviceCount = 0; VK_CHECK(vkEnumeratePhysicalDevices(instance, &deviceCount, nullptr)); std::vector physicalDevices(deviceCount); VK_CHECK(vkEnumeratePhysicalDevices(instance, &deviceCount, physicalDevices.data())); for (VkPhysicalDevice dev : physicalDevices) { VkPhysicalDeviceVulkan11Properties props11 = {}; props11.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_1_PROPERTIES; VkPhysicalDeviceProperties2 props = {}; props.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2; props.pNext = &props11; vkGetPhysicalDeviceProperties2(dev, &props); if (memcmp(props11.deviceUUID, deviceUUID, VK_UUID_SIZE) == 0) { physicalDevice = dev; break; } } if (!physicalDevice && !physicalDevices.empty()) { Warn("Falling back to first device"); physicalDevice = physicalDevices[0]; } if (!physicalDevice) { throw std::runtime_error("Failed to find vulkan device."); } VkPhysicalDeviceDrmPropertiesEXT drmProps = {}; drmProps.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DRM_PROPERTIES_EXT; VkPhysicalDeviceProperties2 deviceProps = {}; deviceProps.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2; deviceProps.pNext = &drmProps; vkGetPhysicalDeviceProperties2(physicalDevice, &deviceProps); amd = deviceProps.properties.vendorID == 0x1002; intel = deviceProps.properties.vendorID == 0x8086; nvidia = deviceProps.properties.vendorID == 0x10de; Info("Using Vulkan device %s", deviceProps.properties.deviceName); uint32_t deviceExtensionCount = 0; VK_CHECK(vkEnumerateDeviceExtensionProperties( physicalDevice, nullptr, &deviceExtensionCount, nullptr )); std::vector deviceExts(deviceExtensionCount); VK_CHECK(vkEnumerateDeviceExtensionProperties( physicalDevice, nullptr, &deviceExtensionCount, deviceExts.data() )); for (const char* name : device_extensions) { auto it = std::find_if(deviceExts.begin(), deviceExts.end(), [name](VkExtensionProperties e) { return strcmp(e.extensionName, name) == 0; }); if (it != deviceExts.end()) { deviceExtensions.push_back(name); } } float queuePriority = 1.0; std::vector queueInfos; uint32_t queueFamilyCount; vkGetPhysicalDeviceQueueFamilyProperties(physicalDevice, &queueFamilyCount, nullptr); std::vector queueFamilyProperties(queueFamilyCount); vkGetPhysicalDeviceQueueFamilyProperties( physicalDevice, &queueFamilyCount, queueFamilyProperties.data() ); for (uint32_t i = 0; i < queueFamilyProperties.size(); ++i) { const bool graphics = queueFamilyProperties[i].queueFlags & VK_QUEUE_GRAPHICS_BIT; const bool compute = queueFamilyProperties[i].queueFlags & VK_QUEUE_COMPUTE_BIT; if (compute && (queueFamilyIndex == VK_QUEUE_FAMILY_IGNORED || !graphics)) { queueFamilyIndex = i; } VkDeviceQueueCreateInfo queueInfo = {}; queueInfo.sType = VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO; queueInfo.queueFamilyIndex = i; queueInfo.queueCount = 1; queueInfo.pQueuePriorities = &queuePriority; queueInfos.push_back(queueInfo); } VkPhysicalDeviceVulkan12Features features12 = {}; features12.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES; features12.timelineSemaphore = true; VkPhysicalDeviceFeatures2 features = {}; features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2; features.pNext = &features12; features.features.samplerAnisotropy = VK_TRUE; VkDeviceCreateInfo deviceInfo = {}; deviceInfo.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO; deviceInfo.pNext = &features; deviceInfo.queueCreateInfoCount = queueInfos.size(); deviceInfo.pQueueCreateInfos = queueInfos.data(); deviceInfo.enabledExtensionCount = deviceExtensions.size(); deviceInfo.ppEnabledExtensionNames = deviceExtensions.data(); VK_CHECK(vkCreateDevice(physicalDevice, &deviceInfo, nullptr, &device)); for (int i = 128; i < 136; ++i) { auto path = "/dev/dri/renderD" + std::to_string(i); int fd = open(path.c_str(), O_RDONLY); if (fd == -1) { continue; } struct stat s = {}; int ret = fstat(fd, &s); close(fd); if (ret != 0) { continue; } dev_t primaryDev = makedev(drmProps.primaryMajor, drmProps.primaryMinor); dev_t renderDev = makedev(drmProps.renderMajor, drmProps.renderMinor); if (primaryDev == s.st_rdev || renderDev == s.st_rdev) { devicePath = path; break; } } if (devicePath.empty()) { devicePath = "/dev/dri/renderD128"; } Info("Using device path %s", devicePath.c_str()); ctx = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_VULKAN); AVHWDeviceContext* hwctx = (AVHWDeviceContext*)ctx->data; AVVulkanDeviceContext* vkctx = (AVVulkanDeviceContext*)hwctx->hwctx; vkctx->alloc = nullptr; vkctx->inst = instance; vkctx->phys_dev = physicalDevice; vkctx->act_dev = device; vkctx->device_features = features; vkctx->queue_family_index = queueFamilyIndex; vkctx->nb_graphics_queues = 1; vkctx->queue_family_tx_index = queueFamilyIndex; vkctx->nb_tx_queues = 1; vkctx->queue_family_comp_index = queueFamilyIndex; vkctx->nb_comp_queues = 1; vkctx->get_proc_addr = vkGetInstanceProcAddr; vkctx->queue_family_encode_index = -1; vkctx->nb_encode_queues = 0; vkctx->queue_family_decode_index = -1; vkctx->nb_decode_queues = 0; char** inst_extensions = (char**)malloc(sizeof(char*) * instanceExtensions.size()); for (uint32_t i = 0; i < instanceExtensions.size(); ++i) { inst_extensions[i] = strdup(instanceExtensions[i]); } vkctx->enabled_inst_extensions = inst_extensions; vkctx->nb_enabled_inst_extensions = instanceExtensions.size(); char** dev_extensions = (char**)malloc(sizeof(char*) * deviceExtensions.size()); for (uint32_t i = 0; i < deviceExtensions.size(); ++i) { dev_extensions[i] = strdup(deviceExtensions[i]); } vkctx->enabled_dev_extensions = dev_extensions; vkctx->nb_enabled_dev_extensions = deviceExtensions.size(); int ret = av_hwdevice_ctx_init(ctx); if (ret) throw AvException("failed to initialize ffmpeg", ret); } alvr::VkContext::~VkContext() { av_buffer_unref(&ctx); vkDestroyDevice(device, nullptr); vkDestroyInstance(instance, nullptr); } alvr::VkFrameCtx::VkFrameCtx(VkContext& vkContext, vk::ImageCreateInfo image_create_info) { AVHWFramesContext* frames_ctx = NULL; int err = 0; if (!(ctx = av_hwframe_ctx_alloc(vkContext.ctx))) { throw std::runtime_error("Failed to create vulkan frame context."); } frames_ctx = (AVHWFramesContext*)(ctx->data); frames_ctx->format = AV_PIX_FMT_VULKAN; frames_ctx->sw_format = vk_format_to_av_format(image_create_info.format); frames_ctx->width = image_create_info.extent.width; frames_ctx->height = image_create_info.extent.height; frames_ctx->initial_pool_size = 0; if ((err = av_hwframe_ctx_init(ctx)) < 0) { av_buffer_unref(&ctx); throw alvr::AvException("Failed to initialize vulkan frame context:", err); } } alvr::VkFrameCtx::~VkFrameCtx() { av_buffer_unref(&ctx); } alvr::VkFrame::VkFrame( const VkContext& vk_ctx, VkImage image, VkImageCreateInfo image_info, VkDeviceSize size, VkDeviceMemory memory, DrmImage drm ) : vkimage(image) , vkimageinfo(image_info) { device = vk_ctx.get_vk_device(); avformat = vk_format_to_av_format(vk::Format(image_info.format)); av_drmframe = (AVDRMFrameDescriptor*)malloc(sizeof(AVDRMFrameDescriptor)); av_drmframe->nb_objects = 1; av_drmframe->objects[0].fd = drm.fd; av_drmframe->objects[0].size = size; av_drmframe->objects[0].format_modifier = drm.modifier; av_drmframe->nb_layers = 1; av_drmframe->layers[0].format = drm.format; av_drmframe->layers[0].nb_planes = drm.planes; for (uint32_t i = 0; i < drm.planes; ++i) { av_drmframe->layers[0].planes[i].object_index = 0; av_drmframe->layers[0].planes[i].pitch = drm.strides[i]; av_drmframe->layers[0].planes[i].offset = drm.offsets[i]; } av_vkframe = av_vk_frame_alloc(); av_vkframe->img[0] = image; av_vkframe->tiling = image_info.tiling; av_vkframe->mem[0] = memory; av_vkframe->size[0] = size; av_vkframe->layout[0] = VK_IMAGE_LAYOUT_UNDEFINED; VkExportSemaphoreCreateInfo exportInfo = {}; exportInfo.sType = VK_STRUCTURE_TYPE_EXPORT_SEMAPHORE_CREATE_INFO; exportInfo.handleTypes = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT; VkSemaphoreTypeCreateInfo timelineInfo = {}; timelineInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO; timelineInfo.pNext = &exportInfo; timelineInfo.semaphoreType = VK_SEMAPHORE_TYPE_TIMELINE; VkSemaphoreCreateInfo semInfo = {}; semInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO; semInfo.pNext = &timelineInfo; vkCreateSemaphore(device, &semInfo, nullptr, &av_vkframe->sem[0]); } alvr::VkFrame::~VkFrame() { free(av_drmframe); if (av_vkframe) { vkDestroySemaphore(device, av_vkframe->sem[0], nullptr); av_free(av_vkframe); } } std::unique_ptr> alvr::VkFrame::make_av_frame(VkFrameCtx& frame_ctx) { std::unique_ptr> frame { av_frame_alloc(), [](AVFrame* p) { av_frame_free(&p); } }; frame->width = vkimageinfo.extent.width; frame->height = vkimageinfo.extent.height; frame->hw_frames_ctx = av_buffer_ref(frame_ctx.ctx); frame->data[0] = (uint8_t*)av_vkframe; frame->format = AV_PIX_FMT_VULKAN; frame->buf[0] = av_buffer_alloc(1); frame->pts = std::chrono::steady_clock::now().time_since_epoch().count(); return frame; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/ffmpeg_helper.h ================================================ #pragma once #include #include #include extern "C" { #include #include #include #include #include #include #include #include #include #include #include } #include "Renderer.h" namespace alvr { // Utility class to build an exception from an ffmpeg return code. // Messages are rarely useful however. class AvException : public std::runtime_error { public: AvException(std::string msg, int averror) : std::runtime_error { makemsg(msg, averror) } { } private: static std::string makemsg(const std::string& msg, int averror); }; class VkContext { public: VkContext(const uint8_t* deviceUUID, const std::vector& requiredDeviceExtensions); ~VkContext(); VkDevice get_vk_device() const { return device; } VkInstance get_vk_instance() const { return instance; } VkPhysicalDevice get_vk_phys_device() const { return physicalDevice; } uint32_t get_vk_queue_family_index() const { return queueFamilyIndex; } std::vector get_vk_instance_extensions() const { return instanceExtensions; } std::vector get_vk_device_extensions() const { return deviceExtensions; } AVBufferRef* ctx = nullptr; VkInstance instance = VK_NULL_HANDLE; VkPhysicalDevice physicalDevice = VK_NULL_HANDLE; VkDevice device = VK_NULL_HANDLE; uint32_t queueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; std::vector instanceExtensions; std::vector deviceExtensions; bool amd = false; bool intel = false; bool nvidia = false; std::string devicePath; }; class VkFrameCtx { public: VkFrameCtx(VkContext& vkContext, vk::ImageCreateInfo image_create_info); ~VkFrameCtx(); AVBufferRef* ctx; }; class VkFrame { public: VkFrame( const VkContext& vk_ctx, VkImage image, VkImageCreateInfo image_info, VkDeviceSize size, VkDeviceMemory memory, DrmImage drm ); ~VkFrame(); VkImage image() { return vkimage; } VkImageCreateInfo imageInfo() { return vkimageinfo; } VkFormat format() { return vkimageinfo.format; } AVPixelFormat avFormat() { return avformat; } operator AVVkFrame*() const { return av_vkframe; } operator AVDRMFrameDescriptor*() const { return av_drmframe; } std::unique_ptr> make_av_frame(VkFrameCtx& frame_ctx); private: AVVkFrame* av_vkframe = nullptr; AVDRMFrameDescriptor* av_drmframe = nullptr; vk::Device device; VkImage vkimage; VkImageCreateInfo vkimageinfo; AVPixelFormat avformat; }; } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/protocol.h ================================================ #pragma once #include #include #include #include #include #include #include struct present_packet { uint32_t image; uint32_t frame; uint64_t semaphore_value; float pose[3][4]; }; struct init_packet { uint32_t num_images; std::array device_uuid; VkImageCreateInfo image_create_info; size_t mem_index; pid_t source_pid; }; ================================================ FILE: alvr/server_openvr/cpp/platform/linux/shader/color.comp ================================================ #version 450 layout (local_size_x = 8, local_size_y = 8, local_size_z = 1) in; layout (binding = 0) uniform sampler2D in_img; layout (binding = 1, rgba8) uniform writeonly image2D out_img; layout (constant_id = 0) const float renderWidth = 0.; layout (constant_id = 1) const float renderHeight = 0.; layout (constant_id = 2) const float brightness = 0.; layout (constant_id = 3) const float contrast = 0.; layout (constant_id = 4) const float saturation = 0.; layout (constant_id = 5) const float gamma = 0.; layout (constant_id = 6) const float sharpening = 0.; vec3 GetSharpenNeighborComponent(vec2 uv, float xoff, float yoff) { const float sharpenNeighbourWeight = -sharpening / 8.; return texture(in_img, uv + vec2(xoff, yoff)).rgb * sharpenNeighbourWeight; } vec3 blendLighten(vec3 base, vec3 blend) { return vec3(max(base.r, blend.r), max(base.g, blend.g), max(base.b, blend.b)); } // https://forum.unity.com/threads/hue-saturation-brightness-contrast-shader.260649/ void main() { ivec2 pos = ivec2(gl_GlobalInvocationID.xy); vec2 uv = (vec2(pos) + 0.5f) / imageSize(out_img); const float DX = 1. / renderWidth; const float DY = 1. / renderHeight; // sharpening vec3 pixel = texture(in_img, uv).rgb * (sharpening + 1.); pixel += GetSharpenNeighborComponent(uv, -DX, -DY); pixel += GetSharpenNeighborComponent(uv, 0, -DY); pixel += GetSharpenNeighborComponent(uv, +DX, -DY); pixel += GetSharpenNeighborComponent(uv, +DX, 0); pixel += GetSharpenNeighborComponent(uv, +DX, +DY); pixel += GetSharpenNeighborComponent(uv, 0, +DY); pixel += GetSharpenNeighborComponent(uv, -DX, +DY); pixel += GetSharpenNeighborComponent(uv, -DX, 0); pixel += brightness; // brightness pixel = (pixel - 0.5) * contrast + 0.5f; // contast pixel = blendLighten(mix(vec3(dot(pixel, vec3(0.299, 0.587, 0.114))), pixel, vec3(saturation)), pixel); // saturation + lighten only pixel = clamp(pixel, 0., 1.); pixel = pow(pixel, vec3(1. / gamma)); // gamma imageStore(out_img, pos, vec4(pixel, 1.)); } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/shader/ffr.comp ================================================ #version 450 layout (local_size_x = 8, local_size_y = 8, local_size_z = 1) in; layout (binding = 0) uniform sampler2D in_img; layout (binding = 1, rgba8) uniform writeonly image2D out_img; layout (constant_id = 0) const float eyeSizeRatioX = 0.; layout (constant_id = 1) const float eyeSizeRatioY = 0.; layout (constant_id = 2) const float centerSizeX = 0.; layout (constant_id = 3) const float centerSizeY = 0.; layout (constant_id = 4) const float centerShiftX = 0.; layout (constant_id = 5) const float centerShiftY = 0.; layout (constant_id = 6) const float edgeRatioX = 0.; layout (constant_id = 7) const float edgeRatioY = 0.; const vec2 eyeSizeRatio = vec2(eyeSizeRatioX, eyeSizeRatioY); const vec2 centerSize = vec2(centerSizeX, centerSizeY); const vec2 centerShift = vec2(centerShiftX, centerShiftY); const vec2 edgeRatio = vec2(edgeRatioX, edgeRatioY); vec2 TextureToEyeUV(vec2 textureUV, bool isRightEye) { // flip distortion horizontally for right eye // left: x * 2; right: (1 - x) * 2 return vec2((textureUV.x + float(isRightEye) * (1. - 2. * textureUV.x)) * 2., textureUV.y); } vec2 EyeToTextureUV(vec2 eyeUV, bool isRightEye) { // saturate is used to avoid color bleeding between the two sides of the texture or with the // black border when filtering // vec2 clampedUV = saturate(eyeUV); // left: x / 2; right 1 - (x / 2) // return vec2(clampedUV.x / 2. + float(isRightEye) * (1. - clampedUV.x), clampedUV.y); return vec2(eyeUV.x * .5 + float(isRightEye) * (1. - eyeUV.x), eyeUV.y); } void main() { ivec2 pos = ivec2(gl_GlobalInvocationID.xy); vec2 uv = (vec2(pos) + 0.5f) / imageSize(out_img); bool isRightEye = uv.x > 0.5; vec2 eyeUV = TextureToEyeUV(uv, isRightEye) / eyeSizeRatio; vec2 c0 = (1. - centerSize) * .5; vec2 c1 = (edgeRatio - 1.) * c0 * (centerShift + 1.) / edgeRatio; vec2 c2 = (edgeRatio - 1.) * centerSize + 1.; vec2 loBound = c0 * (centerShift + 1.) / c2; vec2 hiBound = c0 * (centerShift - 1.) / c2 + 1.; vec2 underBound = vec2(eyeUV.x < loBound.x, eyeUV.y < loBound.y); vec2 inBound = vec2(loBound.x < eyeUV.x && eyeUV.x < hiBound.x, loBound.y < eyeUV.y && eyeUV.y < hiBound.y); vec2 overBound = vec2(eyeUV.x > hiBound.x, eyeUV.y > hiBound.y); vec2 center = eyeUV * c2 / edgeRatio + c1; vec2 d2 = eyeUV * c2; vec2 d3 = (eyeUV - 1.) * c2 + 1.; vec2 g1 = eyeUV / loBound; vec2 g2 = (1. - eyeUV) / (1. - hiBound); vec2 leftEdge = g1 * center + (1. - g1) * d2; vec2 rightEdge = g2 * center + (1. - g2) * d3; vec2 compressedUV = underBound * leftEdge + inBound * center + overBound * rightEdge; imageStore(out_img, pos, texture(in_img, EyeToTextureUV(compressedUV, isRightEye))); } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/shader/quad.comp ================================================ #version 450 layout (local_size_x = 8, local_size_y = 8, local_size_z = 1) in; layout (binding = 0) uniform sampler2D in_img; layout (binding = 1, rgba8) uniform writeonly image2D out_img; void main() { ivec2 pos = ivec2(gl_GlobalInvocationID.xy); vec2 npos = (vec2(pos) + 0.5f) / imageSize(out_img); vec4 res = texture(in_img, npos); imageStore(out_img, pos, res); } ================================================ FILE: alvr/server_openvr/cpp/platform/linux/shader/rgbtoyuv420.comp ================================================ #version 450 layout (local_size_x = 8, local_size_y = 8, local_size_z = 1) in; layout (binding = 0, rgba8) uniform readonly image2D in_img; layout (binding = 1, r8) uniform writeonly image2D out_img[3]; /* FFmpeg/libavfilter/vf_scale_vulkan.c */ void main() { const mat4 yuv_matrix = mat4( 0.0, 1.0, 0.0, 0.0, 0.0, -0.5, 0.5, 0.0, 0.5, -0.5, 0, 0.0, 0.0, 0.0, 0.0, 1.0 ); ivec2 pos = ivec2(gl_GlobalInvocationID.xy); vec4 res = imageLoad(in_img, pos); res *= yuv_matrix; res *= vec4(219.0 / 255.0, 224.0 / 255.0, 224.0 / 255.0, 1.0); res += vec4(16.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0, 0.0); imageStore(out_img[0], pos, vec4(res.r, 0.0, 0.0, 0.0)); pos /= ivec2(2); imageStore(out_img[1], pos, vec4(res.g, 0.0, 0.0, 0.0)); imageStore(out_img[2], pos, vec4(res.b, 0.0, 0.0, 0.0)); } ================================================ FILE: alvr/server_openvr/cpp/platform/macos/CEncoder.h ================================================ #pragma once #include "shared/threadtools.h" class CEncoder : public CThread { public: CEncoder() { } ~CEncoder() { } bool Init() override { return true; } void Run() override { } void Stop() { } void OnStreamStart() { } void InsertIDR() { } }; ================================================ FILE: alvr/server_openvr/cpp/platform/macos/CrashHandler.cpp ================================================ #include "../../alvr_server/bindings.h" void HookCrashHandler() { } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/CEncoder.cpp ================================================ #include "CEncoder.h" CEncoder::CEncoder() : m_bExiting(false) , m_targetTimestampNs(0) { m_encodeFinished.Set(); } CEncoder::~CEncoder() { if (m_videoEncoder) { m_videoEncoder->Shutdown(); m_videoEncoder.reset(); } } void CEncoder::Initialize(std::shared_ptr d3dRender) { m_FrameRender = std::make_shared(d3dRender); m_FrameRender->Startup(); uint32_t encoderWidth, encoderHeight; m_FrameRender->GetEncodingResolution(&encoderWidth, &encoderHeight); Exception vplException; Exception vceException; Exception nvencException; #ifdef ALVR_GPL Exception swException; if (Settings::Instance().m_force_sw_encoding) { try { Debug("Try to use VideoEncoderSW.\n"); m_videoEncoder = std::make_shared(d3dRender, encoderWidth, encoderHeight); m_videoEncoder->Initialize(); return; } catch (Exception e) { swException = e; } } #endif try { Debug("Try to use VideoEncoderAMF.\n"); m_videoEncoder = std::make_shared(d3dRender, encoderWidth, encoderHeight); m_videoEncoder->Initialize(); return; } catch (Exception e) { vceException = e; } try { Debug("Try to use VideoEncoderNVENC.\n"); m_videoEncoder = std::make_shared(d3dRender, encoderWidth, encoderHeight); m_videoEncoder->Initialize(); return; } catch (Exception e) { nvencException = e; } try { Debug("Try to use VideoEncoderVPL.\n"); m_videoEncoder = std::make_shared(d3dRender, encoderWidth, encoderHeight); m_videoEncoder->Initialize(); return; } catch (Exception e) { vplException = e; } #ifdef ALVR_GPL try { Debug("Try to use VideoEncoderSW.\n"); m_videoEncoder = std::make_shared(d3dRender, encoderWidth, encoderHeight); m_videoEncoder->Initialize(); return; } catch (Exception e) { swException = e; } throw MakeException( "All VideoEncoder are not available. VCE: %s, NVENC: %s, VPL: %s, SW: %s", vceException.what(), nvencException.what(), vplException.what(), swException.what() ); #else throw MakeException( "All VideoEncoder are not available. VCE: %s, NVENC: %s, VPL: %s", vceException.what(), nvencException.what(), vplException.what() ); #endif } void CEncoder::SetViewParams( vr::HmdRect2_t projLeft, vr::HmdMatrix34_t eyeToHeadLeft, vr::HmdRect2_t projRight, vr::HmdMatrix34_t eyeToHeadRight ) { m_FrameRender->SetViewParams(projLeft, eyeToHeadLeft, projRight, eyeToHeadRight); } bool CEncoder::CopyToStaging( ID3D11Texture2D* pTexture[][2], vr::VRTextureBounds_t bounds[][2], vr::HmdMatrix34_t poses[], int layerCount, bool recentering, uint64_t presentationTime, uint64_t targetTimestampNs, const std::string& message, const std::string& debugText ) { m_presentationTime = presentationTime; m_targetTimestampNs = targetTimestampNs; m_FrameRender->Startup(); m_FrameRender->RenderFrame( pTexture, bounds, poses, layerCount, recentering, message, debugText ); return true; } void CEncoder::Run() { Debug("CEncoder: Start thread. Id=%d\n", GetCurrentThreadId()); SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_MOST_URGENT); while (!m_bExiting) { m_newFrameReady.Wait(); if (m_bExiting) break; if (m_FrameRender->GetTexture()) { m_videoEncoder->Transmit( m_FrameRender->GetTexture().Get(), m_presentationTime, m_targetTimestampNs, m_scheduler.CheckIDRInsertion() ); } m_encodeFinished.Set(); } } void CEncoder::Stop() { m_bExiting = true; m_newFrameReady.Set(); Join(); m_FrameRender.reset(); } void CEncoder::NewFrameReady() { m_encodeFinished.Reset(); m_newFrameReady.Set(); } void CEncoder::WaitForEncode() { m_encodeFinished.Wait(); } void CEncoder::OnStreamStart() { m_scheduler.OnStreamStart(); } void CEncoder::InsertIDR() { m_scheduler.InsertIDR(); } void CEncoder::CaptureFrame() { } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/CEncoder.h ================================================ #pragma once #include "shared/d3drender.h" #include "shared/threadtools.h" #include "FrameRender.h" #include "VideoEncoder.h" #include "VideoEncoderAMF.h" #include "VideoEncoderNVENC.h" #include "VideoEncoderVPL.h" #include "alvr_server/Utils.h" #include #include #include #include #include #include #ifdef ALVR_GPL #include "VideoEncoderSW.h" #endif #include "alvr_server/IDRScheduler.h" using Microsoft::WRL::ComPtr; //---------------------------------------------------------------------------- // Blocks on reading backbuffer from gpu, so WaitForPresent can return // as soon as we know rendering made it this frame. This step of the pipeline // should run about 3ms per frame. //---------------------------------------------------------------------------- class CEncoder : public CThread { public: CEncoder(); ~CEncoder(); void Initialize(std::shared_ptr d3dRender); void SetViewParams( vr::HmdRect2_t projLeft, vr::HmdMatrix34_t eyeToHeadLeft, vr::HmdRect2_t projRight, vr::HmdMatrix34_t eyeToHeadRight ); bool CopyToStaging( ID3D11Texture2D* pTexture[][2], vr::VRTextureBounds_t bounds[][2], vr::HmdMatrix34_t poses[], int layerCount, bool recentering, uint64_t presentationTime, uint64_t targetTimestampNs, const std::string& message, const std::string& debugText ); virtual void Run(); virtual void Stop(); void NewFrameReady(); void WaitForEncode(); void OnStreamStart(); void InsertIDR(); void CaptureFrame(); private: CThreadEvent m_newFrameReady, m_encodeFinished; std::shared_ptr m_videoEncoder; bool m_bExiting; uint64_t m_presentationTime; uint64_t m_targetTimestampNs; std::shared_ptr m_FrameRender; IDRScheduler m_scheduler; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/CrashHandler.cpp ================================================ #include "../../alvr_server/bindings.h" #include "../../alvr_server/Logger.h" #include "../../shared/backward.hpp" #include #include static LONG WINAPI handler(PEXCEPTION_POINTERS ptrs) { backward::StackTrace stacktrace; backward::Printer printer; std::ostringstream stream; stacktrace.load_from(ptrs->ExceptionRecord->ExceptionAddress); printer.print(stacktrace, stream); std::string str = stream.str(); Error("Unhandled exception: %X\n%s", ptrs->ExceptionRecord->ExceptionCode, str.c_str()); Sleep(2000); return EXCEPTION_EXECUTE_HANDLER; } void HookCrashHandler() { SetUnhandledExceptionFilter(handler); } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/FFR.cpp ================================================ #include "FFR.h" #include "alvr_server/Settings.h" #include "alvr_server/Utils.h" #include "alvr_server/bindings.h" using Microsoft::WRL::ComPtr; using namespace d3d_render_utils; namespace { struct FoveationVars { uint32_t targetEyeWidth; uint32_t targetEyeHeight; uint32_t optimizedEyeWidth; uint32_t optimizedEyeHeight; float eyeWidthRatio; float eyeHeightRatio; float centerSizeX; float centerSizeY; float centerShiftX; float centerShiftY; float edgeRatioX; float edgeRatioY; }; FoveationVars CalculateFoveationVars() { float targetEyeWidth = (float)Settings::Instance().m_renderWidth / 2; float targetEyeHeight = (float)Settings::Instance().m_renderHeight; float centerSizeX = (float)Settings::Instance().m_foveationCenterSizeX; float centerSizeY = (float)Settings::Instance().m_foveationCenterSizeY; float centerShiftX = (float)Settings::Instance().m_foveationCenterShiftX; float centerShiftY = (float)Settings::Instance().m_foveationCenterShiftY; float edgeRatioX = (float)Settings::Instance().m_foveationEdgeRatioX; float edgeRatioY = (float)Settings::Instance().m_foveationEdgeRatioY; float edgeSizeX = targetEyeWidth - centerSizeX * targetEyeWidth; float edgeSizeY = targetEyeHeight - centerSizeY * targetEyeHeight; float centerSizeXAligned = 1. - ceil(edgeSizeX / (edgeRatioX * 2.)) * (edgeRatioX * 2.) / targetEyeWidth; float centerSizeYAligned = 1. - ceil(edgeSizeY / (edgeRatioY * 2.)) * (edgeRatioY * 2.) / targetEyeHeight; float edgeSizeXAligned = targetEyeWidth - centerSizeXAligned * targetEyeWidth; float edgeSizeYAligned = targetEyeHeight - centerSizeYAligned * targetEyeHeight; float centerShiftXAligned = ceil(centerShiftX * edgeSizeXAligned / (edgeRatioX * 2.)) * (edgeRatioX * 2.) / edgeSizeXAligned; float centerShiftYAligned = ceil(centerShiftY * edgeSizeYAligned / (edgeRatioY * 2.)) * (edgeRatioY * 2.) / edgeSizeYAligned; float foveationScaleX = (centerSizeXAligned + (1. - centerSizeXAligned) / edgeRatioX); float foveationScaleY = (centerSizeYAligned + (1. - centerSizeYAligned) / edgeRatioY); float optimizedEyeWidth = foveationScaleX * targetEyeWidth; float optimizedEyeHeight = foveationScaleY * targetEyeHeight; // round the frame dimensions to a number of pixel multiple of 32 for the encoder auto optimizedEyeWidthAligned = (uint32_t)ceil(optimizedEyeWidth / 32.f) * 32; auto optimizedEyeHeightAligned = (uint32_t)ceil(optimizedEyeHeight / 32.f) * 32; float eyeWidthRatioAligned = optimizedEyeWidth / optimizedEyeWidthAligned; float eyeHeightRatioAligned = optimizedEyeHeight / optimizedEyeHeightAligned; return { (uint32_t)targetEyeWidth, (uint32_t)targetEyeHeight, optimizedEyeWidthAligned, optimizedEyeHeightAligned, eyeWidthRatioAligned, eyeHeightRatioAligned, centerSizeXAligned, centerSizeYAligned, centerShiftXAligned, centerShiftYAligned, edgeRatioX, edgeRatioY }; } } void FFR::GetOptimizedResolution(uint32_t* width, uint32_t* height) { auto fovVars = CalculateFoveationVars(); *width = fovVars.optimizedEyeWidth * 2; *height = fovVars.optimizedEyeHeight; } FFR::FFR(ID3D11Device* device) : mDevice(device) { } void FFR::Initialize(ID3D11Texture2D* compositionTexture) { auto fovVars = CalculateFoveationVars(); ComPtr foveatedRenderingBuffer = CreateBuffer(mDevice.Get(), fovVars); std::vector quadShaderCSO( QUAD_SHADER_CSO_PTR, QUAD_SHADER_CSO_PTR + QUAD_SHADER_CSO_LEN ); mQuadVertexShader = CreateVertexShader(mDevice.Get(), quadShaderCSO); mOptimizedTexture = CreateTexture( mDevice.Get(), fovVars.optimizedEyeWidth * 2, fovVars.optimizedEyeHeight, Settings::Instance().m_enableHdr ? DXGI_FORMAT_R16G16B16A16_FLOAT : DXGI_FORMAT_R8G8B8A8_UNORM_SRGB ); if (Settings::Instance().m_enableFoveatedEncoding) { std::vector compressAxisAlignedShaderCSO( COMPRESS_AXIS_ALIGNED_CSO_PTR, COMPRESS_AXIS_ALIGNED_CSO_PTR + COMPRESS_AXIS_ALIGNED_CSO_LEN ); auto compressAxisAlignedPipeline = RenderPipeline(mDevice.Get()); compressAxisAlignedPipeline.Initialize( { compositionTexture }, mQuadVertexShader.Get(), compressAxisAlignedShaderCSO, mOptimizedTexture.Get(), foveatedRenderingBuffer.Get() ); mPipelines.push_back(compressAxisAlignedPipeline); } else { mOptimizedTexture = compositionTexture; } } void FFR::Render() { for (auto& p : mPipelines) { p.Render(); } } ID3D11Texture2D* FFR::GetOutputTexture() { return mOptimizedTexture.Get(); } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/FFR.h ================================================ #pragma once #include "d3d-render-utils/RenderPipeline.h" class FFR { public: FFR(ID3D11Device* device); void Initialize(ID3D11Texture2D* compositionTexture); void Render(); void GetOptimizedResolution(uint32_t* width, uint32_t* height); ID3D11Texture2D* GetOutputTexture(); private: Microsoft::WRL::ComPtr mDevice; Microsoft::WRL::ComPtr mOptimizedTexture; Microsoft::WRL::ComPtr mQuadVertexShader; std::vector mPipelines; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/FrameRender.cpp ================================================ #include "FrameRender.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "alvr_server/Utils.h" #include "alvr_server/bindings.h" extern uint64_t g_DriverTestMode; using namespace d3d_render_utils; static const DirectX::XMFLOAT4X4 _identityMat = DirectX::XMFLOAT4X4( 1.0f, 0.0f, 0.0f, 0.0f, 0.0f, -1.0f, 0.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f, 0.0f, 0.0f, 0.0f, 1.0f ); static DirectX::XMMATRIX HmdMatrix_AsDxMat(const vr::HmdMatrix34_t& m) { // I think the negative Y basis is a handedness thing? DirectX::XMFLOAT4X4 f = DirectX::XMFLOAT4X4( m.m[0][0], m.m[1][0], m.m[2][0], 0.0f, -m.m[0][1], -m.m[1][1], -m.m[2][1], 0.0f, m.m[0][2], m.m[1][2], m.m[2][2], 0.0f, m.m[0][3], m.m[1][3], m.m[2][3], 1.0f ); return DirectX::XMLoadFloat4x4(&f); } static DirectX::XMMATRIX HmdMatrix_AsDxMatOrientOnly(vr::HmdMatrix34_t& m) { // I think the negative Y basis is a handedness thing? DirectX::XMFLOAT4X4 f = DirectX::XMFLOAT4X4( m.m[0][0], m.m[1][0], m.m[2][0], 0.0f, -m.m[0][1], -m.m[1][1], -m.m[2][1], 0.0f, m.m[0][2], m.m[1][2], m.m[2][2], 0.0f, 0.0f, 0.0f, 0.0f, 1.0f ); return DirectX::XMLoadFloat4x4(&f); } static DirectX::XMMATRIX HmdMatrix_AsDxMatPosOnly(const vr::HmdMatrix34_t& m) { DirectX::XMFLOAT4X4 f = DirectX::XMFLOAT4X4( 1.0f, 0.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f, m.m[0][3], m.m[1][3], m.m[2][3], 1.0f ); return DirectX::XMLoadFloat4x4(&f); } FrameRender::FrameRender(std::shared_ptr pD3DRender) : m_pD3DRender(pD3DRender) { // Set safe defaults for tangents and eye-to-HMD HmdMatrix_SetIdentity(&m_eyeToHead[0]); HmdMatrix_SetIdentity(&m_eyeToHead[1]); m_viewProj[0] = { -1.0f, 1.0f, 1.0f, -1.0f }; m_viewProj[1] = { -1.0f, 1.0f, 1.0f, -1.0f }; FrameRender::SetGpuPriority(m_pD3DRender->GetDevice()); } FrameRender::~FrameRender() { } bool FrameRender::Startup() { if (m_pStagingTexture) { return true; } // // Create staging texture // This is input texture of Video Encoder and is render target of both eyes. // D3D11_TEXTURE2D_DESC compositionTextureDesc; ZeroMemory(&compositionTextureDesc, sizeof(compositionTextureDesc)); compositionTextureDesc.Width = Settings::Instance().m_renderWidth; compositionTextureDesc.Height = Settings::Instance().m_renderHeight; compositionTextureDesc.Format = Settings::Instance().m_enableHdr ? DXGI_FORMAT_R16G16B16A16_FLOAT : DXGI_FORMAT_R8G8B8A8_UNORM_SRGB; compositionTextureDesc.MipLevels = 1; compositionTextureDesc.ArraySize = 1; compositionTextureDesc.SampleDesc.Count = 1; compositionTextureDesc.Usage = D3D11_USAGE_DEFAULT; compositionTextureDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET; ComPtr compositionTexture; if (FAILED(m_pD3DRender->GetDevice()->CreateTexture2D( &compositionTextureDesc, NULL, &compositionTexture ))) { Error("Failed to create staging texture!\n"); return false; } HRESULT hr = m_pD3DRender->GetDevice()->CreateRenderTargetView( compositionTexture.Get(), NULL, &m_pRenderTargetView ); if (FAILED(hr)) { Error("CreateRenderTargetView %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } D3D11_DEPTH_STENCIL_DESC depthStencilDesc; depthStencilDesc.DepthEnable = FALSE; depthStencilDesc.DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL; depthStencilDesc.DepthFunc = D3D11_COMPARISON_ALWAYS; depthStencilDesc.StencilEnable = FALSE; depthStencilDesc.StencilReadMask = 0xFF; depthStencilDesc.StencilWriteMask = 0xFF; // Stencil operations if pixel is front-facing depthStencilDesc.FrontFace.StencilFailOp = D3D11_STENCIL_OP_KEEP; depthStencilDesc.FrontFace.StencilDepthFailOp = D3D11_STENCIL_OP_INCR; depthStencilDesc.FrontFace.StencilPassOp = D3D11_STENCIL_OP_KEEP; depthStencilDesc.FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS; // Stencil operations if pixel is back-facing depthStencilDesc.BackFace.StencilFailOp = D3D11_STENCIL_OP_KEEP; depthStencilDesc.BackFace.StencilDepthFailOp = D3D11_STENCIL_OP_DECR; depthStencilDesc.BackFace.StencilPassOp = D3D11_STENCIL_OP_KEEP; depthStencilDesc.BackFace.StencilFunc = D3D11_COMPARISON_ALWAYS; m_pD3DRender->GetDevice()->CreateDepthStencilState(&depthStencilDesc, &m_depthStencilState); // Left eye viewport m_viewportL.Width = (float)Settings::Instance().m_renderWidth / 2.0; m_viewportL.Height = (float)Settings::Instance().m_renderHeight; m_viewportL.MinDepth = 0.0f; m_viewportL.MaxDepth = 1.0f; m_viewportL.TopLeftX = 0; m_viewportL.TopLeftY = 0; // Right eye viewport m_viewportR.Width = (float)Settings::Instance().m_renderWidth / 2.0; m_viewportR.Height = (float)Settings::Instance().m_renderHeight; m_viewportR.MinDepth = 0.0f; m_viewportR.MaxDepth = 1.0f; m_viewportR.TopLeftX = (float)Settings::Instance().m_renderWidth / 2.0; m_viewportR.TopLeftY = 0; // Final composition viewport m_viewport.Width = (float)Settings::Instance().m_renderWidth; m_viewport.Height = (float)Settings::Instance().m_renderHeight; m_viewport.MinDepth = 0.0f; m_viewport.MaxDepth = 1.0f; m_viewport.TopLeftX = 0; m_viewport.TopLeftY = 0; // Left eye scissor m_scissorL.bottom = 0.0f; m_scissorL.left = 0.0f; m_scissorL.right = (float)Settings::Instance().m_renderWidth / 2.0f; m_scissorL.top = (float)Settings::Instance().m_renderHeight; // Right eye scissor m_scissorR.bottom = 0.0f; m_scissorR.left = (float)Settings::Instance().m_renderWidth / 2.0f; m_scissorR.right = (float)Settings::Instance().m_renderWidth; m_scissorR.top = (float)Settings::Instance().m_renderHeight; // Final composition scissor m_scissor.bottom = 0.0f; m_scissor.left = 0.0f; m_scissor.right = (float)Settings::Instance().m_renderWidth; m_scissor.top = (float)Settings::Instance().m_renderHeight; // // Compile shaders // std::vector vshader( FRAME_RENDER_VS_CSO_PTR, FRAME_RENDER_VS_CSO_PTR + FRAME_RENDER_VS_CSO_LEN ); hr = m_pD3DRender->GetDevice()->CreateVertexShader( (const DWORD*)&vshader[0], vshader.size(), NULL, &m_pVertexShader ); if (FAILED(hr)) { Error("CreateVertexShader %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } std::vector pshader( FRAME_RENDER_PS_CSO_PTR, FRAME_RENDER_PS_CSO_PTR + FRAME_RENDER_PS_CSO_LEN ); hr = m_pD3DRender->GetDevice()->CreatePixelShader( (const DWORD*)&pshader[0], pshader.size(), NULL, &m_pPixelShader ); if (FAILED(hr)) { Error("CreatePixelShader %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } // // Create input layout // // Define the input layout D3D11_INPUT_ELEMENT_DESC layout[] = { { "POSITION", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, 0, D3D11_INPUT_PER_VERTEX_DATA, 0 }, { "TEXCOORD", 0, DXGI_FORMAT_R32G32_FLOAT, 0, 16, D3D11_INPUT_PER_VERTEX_DATA, 0 }, { "VIEW", 0, DXGI_FORMAT_R32_UINT, 0, 24, D3D11_INPUT_PER_VERTEX_DATA, 0 }, }; UINT numElements = ARRAYSIZE(layout); // Create the input layout hr = m_pD3DRender->GetDevice()->CreateInputLayout( layout, numElements, &vshader[0], vshader.size(), &m_pVertexLayout ); if (FAILED(hr)) { Error("CreateInputLayout %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } // Set the input layout m_pD3DRender->GetContext()->IASetInputLayout(m_pVertexLayout.Get()); // // Create frame render CBuffer // struct FrameRenderBuffer { float encodingGamma; float _align0; float _align1; float _align2; }; FrameRenderBuffer frameRenderStruct = { (float)(1.0 / Settings::Instance().m_encodingGamma), 0.0f, 0.0f, 0.0f }; m_pFrameRenderCBuffer = CreateBuffer(m_pD3DRender->GetDevice(), frameRenderStruct); // // Create vertex buffer // // Src texture has various geometry and we should use the part of the textures. // That part are defined by uv-coordinates of "bounds" passed to // IVRDriverDirectModeComponent::SubmitLayer. So we should update uv-coordinates for every // frames and layers. D3D11_BUFFER_DESC bd; ZeroMemory(&bd, sizeof(bd)); bd.Usage = D3D11_USAGE_DYNAMIC; bd.ByteWidth = sizeof(SimpleVertex) * 8; bd.BindFlags = D3D11_BIND_VERTEX_BUFFER; bd.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE; hr = m_pD3DRender->GetDevice()->CreateBuffer(&bd, NULL, &m_pVertexBuffer); if (FAILED(hr)) { Error("CreateBuffer 1 %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } // Set vertex buffer UINT stride = sizeof(SimpleVertex); UINT offset = 0; m_pD3DRender->GetContext()->IASetVertexBuffers( 0, 1, m_pVertexBuffer.GetAddressOf(), &stride, &offset ); // // Create index buffer // WORD indices[] = { 0, 1, 2, 0, 3, 1, 4, 5, 6, 4, 7, 5 }; bd.Usage = D3D11_USAGE_DEFAULT; bd.ByteWidth = sizeof(indices); bd.BindFlags = D3D11_BIND_INDEX_BUFFER; bd.CPUAccessFlags = 0; D3D11_SUBRESOURCE_DATA InitData; ZeroMemory(&InitData, sizeof(InitData)); InitData.pSysMem = indices; hr = m_pD3DRender->GetDevice()->CreateBuffer(&bd, &InitData, &m_pIndexBuffer); if (FAILED(hr)) { Error("CreateBuffer 2 %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } // Set index buffer m_pD3DRender->GetContext()->IASetIndexBuffer(m_pIndexBuffer.Get(), DXGI_FORMAT_R16_UINT, 0); // Set primitive topology m_pD3DRender->GetContext()->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST); // Create the sample state D3D11_SAMPLER_DESC sampDesc; ZeroMemory(&sampDesc, sizeof(sampDesc)); sampDesc.Filter = D3D11_FILTER_ANISOTROPIC; sampDesc.AddressU = D3D11_TEXTURE_ADDRESS_WRAP; sampDesc.AddressV = D3D11_TEXTURE_ADDRESS_WRAP; sampDesc.AddressW = D3D11_TEXTURE_ADDRESS_WRAP; sampDesc.MaxAnisotropy = D3D11_REQ_MAXANISOTROPY; sampDesc.ComparisonFunc = D3D11_COMPARISON_NEVER; sampDesc.MinLOD = 0; sampDesc.MaxLOD = D3D11_FLOAT32_MAX; hr = m_pD3DRender->GetDevice()->CreateSamplerState(&sampDesc, &m_pSamplerLinear); if (FAILED(hr)) { Error("CreateSamplerState %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } // // Create alpha blend state // We need alpha blending to support layer. // // BlendState for first layer. // Some VR apps (like SteamVR Home beta) submit the texture that alpha is zero on all pixels. // So we need to ignore alpha of first layer. D3D11_BLEND_DESC BlendDesc; ZeroMemory(&BlendDesc, sizeof(BlendDesc)); BlendDesc.AlphaToCoverageEnable = FALSE; BlendDesc.IndependentBlendEnable = FALSE; for (int i = 0; i < 8; i++) { BlendDesc.RenderTarget[i].BlendEnable = TRUE; BlendDesc.RenderTarget[i].SrcBlend = D3D11_BLEND_ONE; BlendDesc.RenderTarget[i].DestBlend = D3D11_BLEND_ZERO; BlendDesc.RenderTarget[i].BlendOp = D3D11_BLEND_OP_ADD; BlendDesc.RenderTarget[i].SrcBlendAlpha = D3D11_BLEND_ONE; BlendDesc.RenderTarget[i].DestBlendAlpha = D3D11_BLEND_ZERO; BlendDesc.RenderTarget[i].BlendOpAlpha = D3D11_BLEND_OP_ADD; BlendDesc.RenderTarget[i].RenderTargetWriteMask = D3D11_COLOR_WRITE_ENABLE_RED | D3D11_COLOR_WRITE_ENABLE_GREEN | D3D11_COLOR_WRITE_ENABLE_BLUE; } hr = m_pD3DRender->GetDevice()->CreateBlendState(&BlendDesc, &m_pBlendStateFirst); if (FAILED(hr)) { Error("CreateBlendState %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } // BleandState for other layers than first. BlendDesc.AlphaToCoverageEnable = FALSE; BlendDesc.IndependentBlendEnable = FALSE; for (int i = 0; i < 8; i++) { BlendDesc.RenderTarget[i].BlendEnable = TRUE; BlendDesc.RenderTarget[i].SrcBlend = D3D11_BLEND_SRC_ALPHA; BlendDesc.RenderTarget[i].DestBlend = D3D11_BLEND_INV_SRC_ALPHA; BlendDesc.RenderTarget[i].BlendOp = D3D11_BLEND_OP_ADD; BlendDesc.RenderTarget[i].SrcBlendAlpha = D3D11_BLEND_ONE; BlendDesc.RenderTarget[i].DestBlendAlpha = D3D11_BLEND_ZERO; BlendDesc.RenderTarget[i].BlendOpAlpha = D3D11_BLEND_OP_ADD; BlendDesc.RenderTarget[i].RenderTargetWriteMask = D3D11_COLOR_WRITE_ENABLE_ALL; } hr = m_pD3DRender->GetDevice()->CreateBlendState(&BlendDesc, &m_pBlendState); if (FAILED(hr)) { Error("CreateBlendState %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } m_pStagingTexture = compositionTexture; std::vector quadShaderCSO( QUAD_SHADER_CSO_PTR, QUAD_SHADER_CSO_PTR + QUAD_SHADER_CSO_LEN ); ComPtr quadVertexShader = CreateVertexShader(m_pD3DRender->GetDevice(), quadShaderCSO); enableColorCorrection = Settings::Instance().m_enableColorCorrection; if (enableColorCorrection) { std::vector colorCorrectionShaderCSO( COLOR_CORRECTION_CSO_PTR, COLOR_CORRECTION_CSO_PTR + COLOR_CORRECTION_CSO_LEN ); ComPtr colorCorrectedTexture = CreateTexture( m_pD3DRender->GetDevice(), Settings::Instance().m_renderWidth, Settings::Instance().m_renderHeight, Settings::Instance().m_enableHdr ? DXGI_FORMAT_R16G16B16A16_FLOAT : DXGI_FORMAT_R8G8B8A8_UNORM_SRGB ); struct ColorCorrection { float renderWidth; float renderHeight; float brightness; float contrast; float saturation; float gamma; float sharpening; float _align; }; ColorCorrection colorCorrectionStruct = { (float)Settings::Instance().m_renderWidth, (float)Settings::Instance().m_renderHeight, Settings::Instance().m_brightness, Settings::Instance().m_contrast + 1.f, Settings::Instance().m_saturation + 1.f, Settings::Instance().m_gamma, Settings::Instance().m_sharpening }; ComPtr colorCorrectionBuffer = CreateBuffer(m_pD3DRender->GetDevice(), colorCorrectionStruct); m_colorCorrectionPipeline = std::make_unique(m_pD3DRender->GetDevice()); m_colorCorrectionPipeline->Initialize( { m_pStagingTexture.Get() }, quadVertexShader.Get(), colorCorrectionShaderCSO, colorCorrectedTexture.Get(), colorCorrectionBuffer.Get() ); m_pStagingTexture = colorCorrectedTexture; } enableFFE = Settings::Instance().m_enableFoveatedEncoding; if (enableFFE) { m_ffr = std::make_unique(m_pD3DRender->GetDevice()); m_ffr->Initialize(m_pStagingTexture.Get()); m_pStagingTexture = m_ffr->GetOutputTexture(); } if (Settings::Instance().m_enableHdr) { std::vector yuv420ShaderCSO( RGBTOYUV420_CSO_PTR, RGBTOYUV420_CSO_PTR + RGBTOYUV420_CSO_LEN ); uint32_t texWidth, texHeight; GetEncodingResolution(&texWidth, &texHeight); ComPtr yuvTexture = CreateTexture( m_pD3DRender->GetDevice(), texWidth, texHeight, Settings::Instance().m_use10bitEncoder ? DXGI_FORMAT_P010 : DXGI_FORMAT_NV12 ); struct YUVParams { float offset[4]; float yCoeff[4]; float uCoeff[4]; float vCoeff[4]; float renderWidth; float renderHeight; float _padding0; float _padding1; }; // Bless this page for ending my stint of plugging in random values // from other projects: // https://kdashg.github.io/misc/colors/from-coeffs.html YUVParams paramStruct_bt2020_8bit_full = { { 0.0000000f, 0.5019608f, 0.5019608f, 0.0f }, // offset { 0.2627000f, 0.6780000f, 0.0593000f, 0.0f }, // yCoeff { -0.1390825f, -0.3589567f, 0.4980392f, 0.0f }, // uCoeff { 0.4980392f, -0.4579826f, -0.0400566f, 0.0f }, // vCoeff (float)texWidth, (float)texHeight, 0.0, 0.0 }; YUVParams paramStruct_bt2020_10bit_full = { { 0.0000000f, 0.5004888f, 0.5004888f, 0.0f }, // offset { 0.2627000f, 0.6780000f, 0.0593000f, 0.0f }, // yCoeff { -0.1394936f, -0.3600177f, 0.4995112f, 0.0f }, // uCoeff { 0.4995112f, -0.4593363f, -0.0401750f, 0.0f }, // vCoeff (float)texWidth, (float)texHeight, 0.0, 0.0 }; YUVParams paramStruct_bt2020_8bit_limited = { { 0.0627451f, 0.5019608f, 0.5019608f, 0.0f }, // offset { 0.2256129f, 0.5822824f, 0.0509282f, 0.0f }, // yCoeff { -0.1226554f, -0.3165603f, 0.4392157f, 0.0f }, // uCoeff { 0.4392157f, -0.4038902f, -0.0353255f, 0.0f }, // vCoeff (float)texWidth, (float)texHeight, 0.0, 0.0 }; YUVParams paramStruct_bt2020_10bit_limited = { { 0.0625611f, 0.5004888f, 0.5004888f, 0.0f }, // offset { 0.2249513f, 0.5805748f, 0.0507789f, 0.0f }, // yCoeff { -0.1222957f, -0.3156319f, 0.4379277f, 0.0f }, // uCoeff { 0.4379277f, -0.4027058f, -0.0352219f, 0.0f }, // vCoeff (float)texWidth, (float)texHeight, 0.0, 0.0 }; YUVParams& paramStruct = paramStruct_bt2020_8bit_full; if (Settings::Instance().m_use10bitEncoder) { paramStruct = paramStruct_bt2020_10bit_full; } else { paramStruct = paramStruct_bt2020_8bit_full; } ComPtr paramBuffer = CreateBuffer(m_pD3DRender->GetDevice(), paramStruct); m_yuvPipeline = std::make_unique(m_pD3DRender->GetDevice()); m_yuvPipeline->Initialize( { m_pStagingTexture.Get() }, quadVertexShader.Get(), yuv420ShaderCSO, yuvTexture.Get(), paramBuffer.Get() ); m_pStagingTexture = yuvTexture; } Debug("Staging Texture created\n"); return true; } void FrameRender::SetViewParams( vr::HmdRect2_t projLeft, vr::HmdMatrix34_t eyeToHeadLeft, vr::HmdRect2_t projRight, vr::HmdMatrix34_t eyeToHeadRight ) { m_viewProj[0] = projLeft; m_eyeToHead[0] = eyeToHeadLeft; m_viewProj[1] = projRight; m_eyeToHead[1] = eyeToHeadRight; } bool FrameRender::RenderFrame( ID3D11Texture2D* pTexture[][2], vr::VRTextureBounds_t bounds[][2], vr::HmdMatrix34_t poses[], int layerCount, bool recentering, const std::string& message, const std::string& debugText ) { // Set render target m_pD3DRender->GetContext()->OMSetRenderTargets(1, m_pRenderTargetView.GetAddressOf(), NULL); m_pD3DRender->GetContext()->OMSetDepthStencilState(m_depthStencilState.Get(), 0); // Clear the back buffer m_pD3DRender->GetContext()->ClearRenderTargetView( m_pRenderTargetView.Get(), DirectX::Colors::MidnightBlue ); // Overlay recentering texture on top of all layers. int recenterLayer = -1; if (recentering) { recenterLayer = layerCount; layerCount++; } // Set up our projection, HMD, and HMD-to-eye transforms once const auto nearZ = 0.001f; const auto farZ = 1.0f; DirectX::XMMATRIX projectionMatL = DirectX::XMMatrixPerspectiveOffCenterRH( m_viewProj[0].vTopLeft.v[0] * nearZ, m_viewProj[0].vBottomRight.v[0] * nearZ, -m_viewProj[0].vTopLeft.v[1] * nearZ, -m_viewProj[0].vBottomRight.v[1] * nearZ, nearZ, farZ ); DirectX::XMMATRIX projectionMatR = DirectX::XMMatrixPerspectiveOffCenterRH( m_viewProj[1].vTopLeft.v[0] * nearZ, m_viewProj[1].vBottomRight.v[0] * nearZ, -m_viewProj[1].vTopLeft.v[1] * nearZ, -m_viewProj[1].vBottomRight.v[1] * nearZ, nearZ, farZ ); DirectX::XMMATRIX hmdToEyeMatL = DirectX::XMMatrixInverse(nullptr, HmdMatrix_AsDxMatPosOnly(m_eyeToHead[0])); DirectX::XMMATRIX hmdToEyeMatR = DirectX::XMMatrixInverse(nullptr, HmdMatrix_AsDxMatPosOnly(m_eyeToHead[1])); DirectX::XMMATRIX hmdPoseForTargetTs = HmdMatrix_AsDxMatOrientOnly(poses[0]); // Set to HmdMatrix_AsDxMat to debug the rendering // I think the negative Y basis is a handedness thing? DirectX::XMMATRIX identityMat = DirectX::XMLoadFloat4x4(&_identityMat); for (int i = 0; i < layerCount; i++) { ID3D11Texture2D* textures[2]; vr::VRTextureBounds_t bound[2]; if (i == recenterLayer) { textures[0] = (ID3D11Texture2D*)m_recenterTexture.Get(); textures[1] = (ID3D11Texture2D*)m_recenterTexture.Get(); bound[0].uMin = bound[0].vMin = bound[1].uMin = bound[1].vMin = 0.0f; bound[0].uMax = bound[0].vMax = bound[1].uMax = bound[1].vMax = 1.0f; } else { textures[0] = pTexture[i][0]; textures[1] = pTexture[i][1]; bound[0] = bounds[i][0]; bound[1] = bounds[i][1]; } if (textures[0] == NULL || textures[1] == NULL) { Debug( "Ignore NULL layer. layer=%d/%d%s%s\n", i, layerCount, recentering ? L" (recentering)" : L"", !message.empty() ? L" (message)" : L"" ); continue; } D3D11_TEXTURE2D_DESC srcDesc; textures[0]->GetDesc(&srcDesc); D3D11_SHADER_RESOURCE_VIEW_DESC SRVDesc = {}; SRVDesc.Format = srcDesc.Format; SRVDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D; SRVDesc.Texture2D.MostDetailedMip = 0; SRVDesc.Texture2D.MipLevels = 1; ComPtr pShaderResourceView[2]; HRESULT hr = m_pD3DRender->GetDevice()->CreateShaderResourceView( textures[0], &SRVDesc, pShaderResourceView[0].ReleaseAndGetAddressOf() ); if (FAILED(hr)) { Error("CreateShaderResourceView %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } hr = m_pD3DRender->GetDevice()->CreateShaderResourceView( textures[1], &SRVDesc, pShaderResourceView[1].ReleaseAndGetAddressOf() ); if (FAILED(hr)) { Error("CreateShaderResourceView %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } if (i == 0) { m_pD3DRender->GetContext()->OMSetBlendState(m_pBlendStateFirst.Get(), NULL, 0xffffffff); } else { m_pD3DRender->GetContext()->OMSetBlendState(m_pBlendState.Get(), NULL, 0xffffffff); } uint32_t inputColorAdjust = 0; if (Settings::Instance().m_enableHdr) { if (SRVDesc.Format == DXGI_FORMAT_R8G8B8A8_UNORM_SRGB || SRVDesc.Format == DXGI_FORMAT_B8G8R8A8_UNORM_SRGB || SRVDesc.Format == DXGI_FORMAT_B8G8R8X8_UNORM_SRGB) { inputColorAdjust = 1; // do sRGB manually } if (Settings::Instance().m_forceHdrSrgbCorrection) { inputColorAdjust = 1; } if (Settings::Instance().m_clampHdrExtendedRange) { inputColorAdjust |= 0x10; // Clamp values to 0.0 to 1.0 } } else { if (SRVDesc.Format != DXGI_FORMAT_R8G8B8A8_UNORM_SRGB && SRVDesc.Format != DXGI_FORMAT_B8G8R8A8_UNORM_SRGB && SRVDesc.Format != DXGI_FORMAT_B8G8R8X8_UNORM_SRGB) { inputColorAdjust = 2; // undo sRGB? if (Settings::Instance().m_forceHdrSrgbCorrection) { inputColorAdjust = 0; } } if (Settings::Instance().m_clampHdrExtendedRange) { inputColorAdjust |= 0x10; // Clamp values to 0.0 to 1.0 } } // // Update uv-coordinates in vertex buffer according to bounds. // DirectX::XMMATRIX framePose = (i == recenterLayer) ? identityMat : HmdMatrix_AsDxMatOrientOnly(poses[i]); DirectX::XMMATRIX framePoseInv = DirectX::XMMatrixInverse(nullptr, framePose); // framePose is the position of the layer in space, ie an identity matrix // would place the quad perpendicular in the floor at 0,0,0 DirectX::XMMATRIX viewMatDiff = DirectX::XMMatrixInverse(nullptr, hmdPoseForTargetTs * framePoseInv); DirectX::XMMATRIX transformMatL = viewMatDiff * hmdToEyeMatL * projectionMatL; DirectX::XMMATRIX transformMatR = viewMatDiff * hmdToEyeMatR * projectionMatR; if (i == recenterLayer) { transformMatL = identityMat; transformMatR = identityMat; } const auto depth = 700.0f; const auto m = 1.0f; DirectX::XMFLOAT4 vertsL[4]; DirectX::XMFLOAT4 vertsR[4]; DirectX::XMVECTORF32 vertsL_VF32[4] = { { { { -1.0f * -m_viewProj[0].vTopLeft.v[0] * depth * m, 1.0f * -m_viewProj[0].vTopLeft.v[1] * depth * m, -depth, 1.0f } } }, { { { 1.0f * m_viewProj[0].vBottomRight.v[0] * depth * m, -1.0f * m_viewProj[0].vBottomRight.v[1] * depth * m, -depth, 1.0f } } }, { { { 1.0f * m_viewProj[0].vBottomRight.v[0] * depth * m, 1.0f * -m_viewProj[0].vTopLeft.v[1] * depth * m, -depth, 1.0f } } }, { { { -1.0f * -m_viewProj[0].vTopLeft.v[0] * depth * m, -1.0f * m_viewProj[0].vBottomRight.v[1] * depth * m, -depth, 1.0f } } } }; for (int i = 0; i < 4; i++) { DirectX::XMStoreFloat4( &vertsL[i], DirectX::XMVector3Transform(vertsL_VF32[i], transformMatL) ); } DirectX::XMVECTORF32 vertsR_VF32[4] = { { { { -1.0f * -m_viewProj[1].vTopLeft.v[0] * depth * m, 1.0f * -m_viewProj[1].vTopLeft.v[1] * depth * m, -depth, 1.0f } } }, { { { 1.0f * m_viewProj[1].vBottomRight.v[0] * depth * m, -1.0f * m_viewProj[1].vBottomRight.v[1] * depth * m, -depth, 1.0f } } }, { { { 1.0f * m_viewProj[1].vBottomRight.v[0] * depth * m, 1.0f * -m_viewProj[1].vTopLeft.v[1] * depth * m, -depth, 1.0f } } }, { { { -1.0f * -m_viewProj[1].vTopLeft.v[0] * depth * m, -1.0f * m_viewProj[1].vBottomRight.v[1] * depth * m, -depth, 1.0f } } } }; for (int i = 0; i < 4; i++) { DirectX::XMStoreFloat4( &vertsR[i], DirectX::XMVector3Transform(vertsR_VF32[i], transformMatR) ); } // We discard the z value because we never want any clipping, // but we do want the w value for perspective correction. SimpleVertex vertices[] = { // Left View { DirectX::XMFLOAT4(vertsL[0].x, vertsL[0].y, 0.5, vertsL[0].w), DirectX::XMFLOAT2(bound[0].uMin, bound[0].vMax), 0 + (inputColorAdjust * 2) }, { DirectX::XMFLOAT4(vertsL[1].x, vertsL[1].y, 0.5, vertsL[1].w), DirectX::XMFLOAT2(bound[0].uMax, bound[0].vMin), 0 + (inputColorAdjust * 2) }, { DirectX::XMFLOAT4(vertsL[2].x, vertsL[2].y, 0.5, vertsL[2].w), DirectX::XMFLOAT2(bound[0].uMax, bound[0].vMax), 0 + (inputColorAdjust * 2) }, { DirectX::XMFLOAT4(vertsL[3].x, vertsL[3].y, 0.5, vertsL[3].w), DirectX::XMFLOAT2(bound[0].uMin, bound[0].vMin), 0 + (inputColorAdjust * 2) }, // Right View { DirectX::XMFLOAT4(vertsR[0].x, vertsR[0].y, 0.5, vertsR[0].w), DirectX::XMFLOAT2(bound[1].uMin, bound[1].vMax), 1 + (inputColorAdjust * 2) }, { DirectX::XMFLOAT4(vertsR[1].x, vertsR[1].y, 0.5, vertsR[1].w), DirectX::XMFLOAT2(bound[1].uMax, bound[1].vMin), 1 + (inputColorAdjust * 2) }, { DirectX::XMFLOAT4(vertsR[2].x, vertsR[2].y, 0.5, vertsR[2].w), DirectX::XMFLOAT2(bound[1].uMax, bound[1].vMax), 1 + (inputColorAdjust * 2) }, { DirectX::XMFLOAT4(vertsR[3].x, vertsR[3].y, 0.5, vertsR[3].w), DirectX::XMFLOAT2(bound[1].uMin, bound[1].vMin), 1 + (inputColorAdjust * 2) }, }; D3D11_MAPPED_SUBRESOURCE mapped = { 0 }; hr = m_pD3DRender->GetContext()->Map( m_pVertexBuffer.Get(), 0, D3D11_MAP_WRITE_DISCARD, 0, &mapped ); if (FAILED(hr)) { Error("Map %p %ls\n", hr, GetErrorStr(hr).c_str()); return false; } memcpy(mapped.pData, vertices, sizeof(vertices)); m_pD3DRender->GetContext()->Unmap(m_pVertexBuffer.Get(), 0); // Set the input layout m_pD3DRender->GetContext()->IASetInputLayout(m_pVertexLayout.Get()); // // Set buffers // UINT stride = sizeof(SimpleVertex); UINT offset = 0; m_pD3DRender->GetContext()->IASetVertexBuffers( 0, 1, m_pVertexBuffer.GetAddressOf(), &stride, &offset ); m_pD3DRender->GetContext()->IASetIndexBuffer(m_pIndexBuffer.Get(), DXGI_FORMAT_R16_UINT, 0); m_pD3DRender->GetContext()->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST); m_pD3DRender->GetContext()->PSSetConstantBuffers( 0, 1, m_pFrameRenderCBuffer.GetAddressOf() ); // // Set shaders // m_pD3DRender->GetContext()->VSSetShader(m_pVertexShader.Get(), nullptr, 0); m_pD3DRender->GetContext()->PSSetShader(m_pPixelShader.Get(), nullptr, 0); ID3D11ShaderResourceView* shaderResourceView[2] = { pShaderResourceView[0].Get(), pShaderResourceView[1].Get() }; m_pD3DRender->GetContext()->PSSetShaderResources(0, 2, shaderResourceView); m_pD3DRender->GetContext()->PSSetSamplers(0, 1, m_pSamplerLinear.GetAddressOf()); // // Draw // // Left eye m_pD3DRender->GetContext()->RSSetViewports(1, &m_viewportL); m_pD3DRender->GetContext()->RSSetScissorRects(1, &m_scissorL); m_pD3DRender->GetContext()->DrawIndexed(VERTEX_INDEX_COUNT / 2, 0, 0); // Right eye m_pD3DRender->GetContext()->RSSetViewports(1, &m_viewportR); m_pD3DRender->GetContext()->RSSetScissorRects(1, &m_scissorR); m_pD3DRender->GetContext()->DrawIndexed( VERTEX_INDEX_COUNT / 2, (VERTEX_INDEX_COUNT / 2), 0 ); } // Restore full viewport/scissor rect for the rest m_pD3DRender->GetContext()->RSSetViewports(1, &m_viewport); m_pD3DRender->GetContext()->RSSetScissorRects(1, &m_scissor); if (enableColorCorrection) { m_colorCorrectionPipeline->Render(); } if (enableFFE) { m_ffr->Render(); } if (Settings::Instance().m_enableHdr) { m_yuvPipeline->Render(); } m_pD3DRender->GetContext()->Flush(); return true; } ComPtr FrameRender::GetTexture() { return m_pStagingTexture; } void FrameRender::GetEncodingResolution(uint32_t* width, uint32_t* height) { if (enableFFE) { m_ffr->GetOptimizedResolution(width, height); } else { *width = Settings::Instance().m_renderWidth; *height = Settings::Instance().m_renderHeight; } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/FrameRender.h ================================================ #pragma once #include #include #include #include #include #include #include #include #include #include #include #include #include "FFR.h" #include "alvr_server/openvr_driver_wrap.h" #include "d3d-render-utils/RenderPipelineYUV.h" #include "shared/d3drender.h" #define GPU_PRIORITY_VAL 7 using Microsoft::WRL::ComPtr; template class ComQIPtr : public ComPtr { public: inline ComQIPtr(IUnknown* unk) { this->ptr_ = nullptr; unk->QueryInterface(__uuidof(T), (void**)this->GetAddressOf()); } inline ComPtr& operator=(IUnknown* unk) { ComPtr::Clear(); unk->QueryInterface(__uuidof(T), (void**)this->GetAddressOf()); return *this; } }; class FrameRender { public: FrameRender(std::shared_ptr pD3DRender); virtual ~FrameRender(); bool Startup(); void SetViewParams( vr::HmdRect2_t projLeft, vr::HmdMatrix34_t eyeToHeadLeft, vr::HmdRect2_t projRight, vr::HmdMatrix34_t eyeToHeadRight ); bool RenderFrame( ID3D11Texture2D* pTexture[][2], vr::VRTextureBounds_t bounds[][2], vr::HmdMatrix34_t poses[], int layerCount, bool recentering, const std::string& message, const std::string& debugText ); void GetEncodingResolution(uint32_t* width, uint32_t* height); ComPtr GetTexture(); private: std::shared_ptr m_pD3DRender; ComPtr m_pStagingTexture; ComPtr m_pVertexShader; ComPtr m_pPixelShader; ComPtr m_pVertexLayout; ComPtr m_pVertexBuffer; ComPtr m_pIndexBuffer; ComPtr m_pFrameRenderCBuffer; ComPtr m_pSamplerLinear; ComPtr m_pRenderTargetView; ComPtr m_depthStencilState; D3D11_VIEWPORT m_viewportL, m_viewportR, m_viewport; D3D11_RECT m_scissorL, m_scissorR, m_scissor; ComPtr m_pBlendStateFirst; ComPtr m_pBlendState; ComPtr m_recenterTexture; ComPtr m_recenterResourceView; ComPtr m_messageBGTexture; ComPtr m_messageBGResourceView; vr::HmdRect2_t m_viewProj[2]; vr::HmdMatrix34_t m_eyeToHead[2]; struct SimpleVertex { DirectX::XMFLOAT4 Pos; DirectX::XMFLOAT2 Tex; uint32_t View; }; // Parameter for Draw method. 2-triangles for both eyes. static const int VERTEX_INDEX_COUNT = 12; std::unique_ptr m_colorCorrectionPipeline; bool enableColorCorrection; std::unique_ptr m_ffr; bool enableFFE; std::unique_ptr m_yuvPipeline; static bool SetGpuPriority(ID3D11Device* device) { typedef enum _D3DKMT_SCHEDULINGPRIORITYCLASS { D3DKMT_SCHEDULINGPRIORITYCLASS_IDLE, D3DKMT_SCHEDULINGPRIORITYCLASS_BELOW_NORMAL, D3DKMT_SCHEDULINGPRIORITYCLASS_NORMAL, D3DKMT_SCHEDULINGPRIORITYCLASS_ABOVE_NORMAL, D3DKMT_SCHEDULINGPRIORITYCLASS_HIGH, D3DKMT_SCHEDULINGPRIORITYCLASS_REALTIME } D3DKMT_SCHEDULINGPRIORITYCLASS; ComQIPtr dxgiDevice(device); if (!dxgiDevice) { Info("[GPU PRIO FIX] Failed to get IDXGIDevice\n"); return false; } HMODULE gdi32 = GetModuleHandleW(L"GDI32"); if (!gdi32) { Info("[GPU PRIO FIX] Failed to get GDI32\n"); return false; } NTSTATUS(WINAPI * d3dkmt_spspc)(HANDLE, D3DKMT_SCHEDULINGPRIORITYCLASS); d3dkmt_spspc = (decltype(d3dkmt_spspc) )GetProcAddress(gdi32, "D3DKMTSetProcessSchedulingPriorityClass"); if (!d3dkmt_spspc) { Info("[GPU PRIO FIX] Failed to get d3dkmt_spspc\n"); return false; } NTSTATUS status = d3dkmt_spspc(GetCurrentProcess(), D3DKMT_SCHEDULINGPRIORITYCLASS_REALTIME); if (status == 0xc0000022) { // STATUS_ACCESS_DENIED, see http://deusexmachina.uk/ntstatus.html Info( "[GPU PRIO FIX] Failed to set process (%d) priority class, please run ALVR as " "Administrator.\n", GetCurrentProcess() ); return false; } else if (status != 0) { Info( "[GPU PRIO FIX] Failed to set process (%d) priority class: %u\n", GetCurrentProcess(), status ); return false; } HRESULT hr = dxgiDevice->SetGPUThreadPriority(GPU_PRIORITY_VAL); if (FAILED(hr)) { Info("[GPU PRIO FIX] SetGPUThreadPriority failed\n"); return false; } Debug("[GPU PRIO FIX] D3D11 GPU priority setup success\n"); return true; } }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/NvCodecUtils.h ================================================ /* * Copyright 2017-2022 NVIDIA Corporation. All rights reserved. * * Please refer to the NVIDIA end user license agreement (EULA) associated * with this source code for terms and conditions that govern your use of * this software. Any use, reproduction, disclosure, or distribution of * this software and related documentation outside the terms of the EULA * is strictly prohibited. * */ //--------------------------------------------------------------------------- //! \file NvCodecUtils.h //! \brief Miscellaneous classes and error checking functions. //! //! Used by Transcode/Encode samples apps for reading input files, mutithreading, performance measurement or colorspace conversion while decoding. //--------------------------------------------------------------------------- #pragma once #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef __cuda_cuda_h__ inline bool check(CUresult e, int iLine, const char *szFile) { if (e != CUDA_SUCCESS) { const char *szErrName = NULL; cuGetErrorName(e, &szErrName); //LOG(FATAL) << "CUDA driver API error " << szErrName << " at line " << iLine << " in file " << szFile; return false; } return true; } #endif #ifdef __CUDA_RUNTIME_H__ inline bool check(cudaError_t e, int iLine, const char *szFile) { if (e != cudaSuccess) { //LOG(FATAL) << "CUDA runtime API error " << cudaGetErrorName(e) << " at line " << iLine << " in file " << szFile; return false; } return true; } #endif #ifdef _NV_ENCODEAPI_H_ inline bool check(NVENCSTATUS e, int iLine, const char *szFile) { const char *aszErrName[] = { "NV_ENC_SUCCESS", "NV_ENC_ERR_NO_ENCODE_DEVICE", "NV_ENC_ERR_UNSUPPORTED_DEVICE", "NV_ENC_ERR_INVALID_ENCODERDEVICE", "NV_ENC_ERR_INVALID_DEVICE", "NV_ENC_ERR_DEVICE_NOT_EXIST", "NV_ENC_ERR_INVALID_PTR", "NV_ENC_ERR_INVALID_EVENT", "NV_ENC_ERR_INVALID_PARAM", "NV_ENC_ERR_INVALID_CALL", "NV_ENC_ERR_OUT_OF_MEMORY", "NV_ENC_ERR_ENCODER_NOT_INITIALIZED", "NV_ENC_ERR_UNSUPPORTED_PARAM", "NV_ENC_ERR_LOCK_BUSY", "NV_ENC_ERR_NOT_ENOUGH_BUFFER", "NV_ENC_ERR_INVALID_VERSION", "NV_ENC_ERR_MAP_FAILED", "NV_ENC_ERR_NEED_MORE_INPUT", "NV_ENC_ERR_ENCODER_BUSY", "NV_ENC_ERR_EVENT_NOT_REGISTERD", "NV_ENC_ERR_GENERIC", "NV_ENC_ERR_INCOMPATIBLE_CLIENT_KEY", "NV_ENC_ERR_UNIMPLEMENTED", "NV_ENC_ERR_RESOURCE_REGISTER_FAILED", "NV_ENC_ERR_RESOURCE_NOT_REGISTERED", "NV_ENC_ERR_RESOURCE_NOT_MAPPED", }; if (e != NV_ENC_SUCCESS) { //LOG(FATAL) << "NVENC error " << aszErrName[e] << " at line " << iLine << " in file " << szFile; return false; } return true; } #endif #ifdef _WINERROR_ inline bool check(HRESULT e, int iLine, const char *szFile) { if (e != S_OK) { std::stringstream stream; stream << std::hex << std::uppercase << e; //LOG(FATAL) << "HRESULT error 0x" << stream.str() << " at line " << iLine << " in file " << szFile; return false; } return true; } #endif #if defined(__gl_h_) || defined(__GL_H__) inline bool check(GLenum e, int iLine, const char *szFile) { if (e != 0) { //LOG(ERROR) << "GLenum error " << e << " at line " << iLine << " in file " << szFile; return false; } return true; } #endif inline bool check(int e, int iLine, const char *szFile) { if (e < 0) { //LOG(ERROR) << "General error " << e << " at line " << iLine << " in file " << szFile; return false; } return true; } #define ck(call) check(call, __LINE__, __FILE__) #define MAKE_FOURCC( ch0, ch1, ch2, ch3 ) \ ( (uint32_t)(uint8_t)(ch0) | ( (uint32_t)(uint8_t)(ch1) << 8 ) | \ ( (uint32_t)(uint8_t)(ch2) << 16 ) | ( (uint32_t)(uint8_t)(ch3) << 24 ) ) /** * @brief Wrapper class around std::thread */ class NvThread { public: NvThread() = default; NvThread(const NvThread&) = delete; NvThread& operator=(const NvThread& other) = delete; NvThread(std::thread&& thread) : t(std::move(thread)) { } NvThread(NvThread&& thread) : t(std::move(thread.t)) { } NvThread& operator=(NvThread&& other) { t = std::move(other.t); return *this; } ~NvThread() { join(); } void join() { if (t.joinable()) { t.join(); } } private: std::thread t; }; #ifndef _WIN32 #define _stricmp strcasecmp #define _stat64 stat64 #endif /** * @brief Utility class to allocate buffer memory. Helps avoid I/O during the encode/decode loop in case of performance tests. */ class BufferedFileReader { public: /** * @brief Constructor function to allocate appropriate memory and copy file contents into it */ BufferedFileReader(const char *szFileName, bool bPartial = false) { struct _stat64 st; if (_stat64(szFileName, &st) != 0) { return; } nSize = st.st_size; while (nSize) { try { pBuf = new uint8_t[(size_t)nSize]; if (nSize != st.st_size) { //LOG(WARNING) << "File is too large - only " << std::setprecision(4) << 100.0 * nSize / st.st_size << "% is loaded"; } break; } catch(std::bad_alloc) { if (!bPartial) { //LOG(ERROR) << "Failed to allocate memory in BufferedReader"; return; } nSize = (uint32_t)(nSize * 0.9); } } std::ifstream fpIn(szFileName, std::ifstream::in | std::ifstream::binary); if (!fpIn) { //LOG(ERROR) << "Unable to open input file: " << szFileName; return; } std::streamsize nRead = fpIn.read(reinterpret_cast(pBuf), nSize).gcount(); fpIn.close(); assert(nRead == nSize); } ~BufferedFileReader() { if (pBuf) { delete[] pBuf; } } bool GetBuffer(uint8_t **ppBuf, uint64_t *pnSize) { if (!pBuf) { return false; } *ppBuf = pBuf; *pnSize = nSize; return true; } private: uint8_t *pBuf = NULL; uint64_t nSize = 0; }; /** * @brief Template class to facilitate color space conversion */ template class YuvConverter { public: YuvConverter(int nWidth, int nHeight) : nWidth(nWidth), nHeight(nHeight) { pQuad = new T[((nWidth + 1) / 2) * ((nHeight + 1) / 2)]; } ~YuvConverter() { delete[] pQuad; } void PlanarToUVInterleaved(T *pFrame, int nPitch = 0) { if (nPitch == 0) { nPitch = nWidth; } // sizes of source surface plane int nSizePlaneY = nPitch * nHeight; int nSizePlaneU = ((nPitch + 1) / 2) * ((nHeight + 1) / 2); int nSizePlaneV = nSizePlaneU; T *puv = pFrame + nSizePlaneY; if (nPitch == nWidth) { memcpy(pQuad, puv, nSizePlaneU * sizeof(T)); } else { for (int i = 0; i < (nHeight + 1) / 2; i++) { memcpy(pQuad + ((nWidth + 1) / 2) * i, puv + ((nPitch + 1) / 2) * i, ((nWidth + 1) / 2) * sizeof(T)); } } T *pv = puv + nSizePlaneU; for (int y = 0; y < (nHeight + 1) / 2; y++) { for (int x = 0; x < (nWidth + 1) / 2; x++) { puv[y * nPitch + x * 2] = pQuad[y * ((nWidth + 1) / 2) + x]; puv[y * nPitch + x * 2 + 1] = pv[y * ((nPitch + 1) / 2) + x]; } } } void UVInterleavedToPlanar(T *pFrame, int nPitch = 0) { if (nPitch == 0) { nPitch = nWidth; } // sizes of source surface plane int nSizePlaneY = nPitch * nHeight; int nSizePlaneU = ((nPitch + 1) / 2) * ((nHeight + 1) / 2); int nSizePlaneV = nSizePlaneU; T *puv = pFrame + nSizePlaneY, *pu = puv, *pv = puv + nSizePlaneU; // split chroma from interleave to planar for (int y = 0; y < (nHeight + 1) / 2; y++) { for (int x = 0; x < (nWidth + 1) / 2; x++) { pu[y * ((nPitch + 1) / 2) + x] = puv[y * nPitch + x * 2]; pQuad[y * ((nWidth + 1) / 2) + x] = puv[y * nPitch + x * 2 + 1]; } } if (nPitch == nWidth) { memcpy(pv, pQuad, nSizePlaneV * sizeof(T)); } else { for (int i = 0; i < (nHeight + 1) / 2; i++) { memcpy(pv + ((nPitch + 1) / 2) * i, pQuad + ((nWidth + 1) / 2) * i, ((nWidth + 1) / 2) * sizeof(T)); } } } private: T *pQuad; int nWidth, nHeight; }; /** * @brief Class for writing IVF format header for AV1 codec */ class IVFUtils { public: void WriteFileHeader(std::vector &vPacket, uint32_t nFourCC, uint32_t nWidth, uint32_t nHeight, uint32_t nFrameRateNum, uint32_t nFrameRateDen, uint32_t nFrameCnt) { char header[32]; header[0] = 'D'; header[1] = 'K'; header[2] = 'I'; header[3] = 'F'; mem_put_le16(header + 4, 0); // version mem_put_le16(header + 6, 32); // header size mem_put_le32(header + 8, nFourCC); // fourcc mem_put_le16(header + 12, nWidth); // width mem_put_le16(header + 14, nHeight); // height mem_put_le32(header + 16, nFrameRateNum); // rate mem_put_le32(header + 20, nFrameRateDen); // scale mem_put_le32(header + 24, nFrameCnt); // length mem_put_le32(header + 28, 0); // unused vPacket.insert(vPacket.end(), &header[0], &header[32]); } void WriteFrameHeader(std::vector &vPacket, size_t nFrameSize, int64_t pts) { char header[12]; mem_put_le32(header, (int)nFrameSize); mem_put_le32(header + 4, (int)(pts & 0xFFFFFFFF)); mem_put_le32(header + 8, (int)(pts >> 32)); vPacket.insert(vPacket.end(), &header[0], &header[12]); } private: static inline void mem_put_le32(void *vmem, int val) { unsigned char *mem = (unsigned char *)vmem; mem[0] = (unsigned char)((val >> 0) & 0xff); mem[1] = (unsigned char)((val >> 8) & 0xff); mem[2] = (unsigned char)((val >> 16) & 0xff); mem[3] = (unsigned char)((val >> 24) & 0xff); } static inline void mem_put_le16(void *vmem, int val) { unsigned char *mem = (unsigned char *)vmem; mem[0] = (unsigned char)((val >> 0) & 0xff); mem[1] = (unsigned char)((val >> 8) & 0xff); } }; /** * @brief Utility class to measure elapsed time in seconds between the block of executed code */ class StopWatch { public: void Start() { t0 = std::chrono::high_resolution_clock::now(); } double Stop() { return std::chrono::duration_cast(std::chrono::high_resolution_clock::now().time_since_epoch() - t0.time_since_epoch()).count() / 1.0e9; } private: std::chrono::high_resolution_clock::time_point t0; }; template class ConcurrentQueue { public: ConcurrentQueue() {} ConcurrentQueue(size_t size) : maxSize(size) {} ConcurrentQueue(const ConcurrentQueue&) = delete; ConcurrentQueue& operator=(const ConcurrentQueue&) = delete; void setSize(size_t s) { maxSize = s; } void push_back(const T& value) { // Do not use a std::lock_guard here. We will need to explicitly // unlock before notify_one as the other waiting thread will // automatically try to acquire mutex once it wakes up // (which will happen on notify_one) std::unique_lock lock(m_mutex); auto wasEmpty = m_List.empty(); while (full()) { m_cond.wait(lock); } m_List.push_back(value); if (wasEmpty && !m_List.empty()) { lock.unlock(); m_cond.notify_one(); } } T pop_front() { std::unique_lock lock(m_mutex); while (m_List.empty()) { m_cond.wait(lock); } auto wasFull = full(); T data = std::move(m_List.front()); m_List.pop_front(); if (wasFull && !full()) { lock.unlock(); m_cond.notify_one(); } return data; } T front() { std::unique_lock lock(m_mutex); while (m_List.empty()) { m_cond.wait(lock); } return m_List.front(); } size_t size() { std::unique_lock lock(m_mutex); return m_List.size(); } bool empty() { std::unique_lock lock(m_mutex); return m_List.empty(); } void clear() { std::unique_lock lock(m_mutex); m_List.clear(); } private: bool full() { if (m_List.size() == maxSize) return true; return false; } private: std::list m_List; std::mutex m_mutex; std::condition_variable m_cond; size_t maxSize; }; inline void CheckInputFile(const char *szInFilePath) { std::ifstream fpIn(szInFilePath, std::ios::in | std::ios::binary); if (fpIn.fail()) { std::ostringstream err; err << "Unable to open input file: " << szInFilePath << std::endl; throw std::invalid_argument(err.str()); } } inline void ValidateResolution(int nWidth, int nHeight) { if (nWidth <= 0 || nHeight <= 0) { std::ostringstream err; err << "Please specify positive non zero resolution as -s WxH. Current resolution is " << nWidth << "x" << nHeight << std::endl; throw std::invalid_argument(err.str()); } } template void Nv12ToColor32(uint8_t *dpNv12, int nNv12Pitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 0); template void Nv12ToColor64(uint8_t *dpNv12, int nNv12Pitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 0); template void P016ToColor32(uint8_t *dpP016, int nP016Pitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 4); template void P016ToColor64(uint8_t *dpP016, int nP016Pitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 4); template void YUV444ToColor32(uint8_t *dpYUV444, int nPitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 0); template void YUV444ToColor64(uint8_t *dpYUV444, int nPitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 0); template void YUV444P16ToColor32(uint8_t *dpYUV444, int nPitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 4); template void YUV444P16ToColor64(uint8_t *dpYUV444, int nPitch, uint8_t *dpBgra, int nBgraPitch, int nWidth, int nHeight, int iMatrix = 4); template void Nv12ToColorPlanar(uint8_t *dpNv12, int nNv12Pitch, uint8_t *dpBgrp, int nBgrpPitch, int nWidth, int nHeight, int iMatrix = 0); template void P016ToColorPlanar(uint8_t *dpP016, int nP016Pitch, uint8_t *dpBgrp, int nBgrpPitch, int nWidth, int nHeight, int iMatrix = 4); template void YUV444ToColorPlanar(uint8_t *dpYUV444, int nPitch, uint8_t *dpBgrp, int nBgrpPitch, int nWidth, int nHeight, int iMatrix = 0); template void YUV444P16ToColorPlanar(uint8_t *dpYUV444, int nPitch, uint8_t *dpBgrp, int nBgrpPitch, int nWidth, int nHeight, int iMatrix = 4); void Bgra64ToP016(uint8_t *dpBgra, int nBgraPitch, uint8_t *dpP016, int nP016Pitch, int nWidth, int nHeight, int iMatrix = 4); void ConvertUInt8ToUInt16(uint8_t *dpUInt8, uint16_t *dpUInt16, int nSrcPitch, int nDestPitch, int nWidth, int nHeight); void ConvertUInt16ToUInt8(uint16_t *dpUInt16, uint8_t *dpUInt8, int nSrcPitch, int nDestPitch, int nWidth, int nHeight); void ResizeNv12(unsigned char *dpDstNv12, int nDstPitch, int nDstWidth, int nDstHeight, unsigned char *dpSrcNv12, int nSrcPitch, int nSrcWidth, int nSrcHeight, unsigned char *dpDstNv12UV = nullptr); void ResizeP016(unsigned char *dpDstP016, int nDstPitch, int nDstWidth, int nDstHeight, unsigned char *dpSrcP016, int nSrcPitch, int nSrcWidth, int nSrcHeight, unsigned char *dpDstP016UV = nullptr); void ScaleYUV420(unsigned char *dpDstY, unsigned char* dpDstU, unsigned char* dpDstV, int nDstPitch, int nDstChromaPitch, int nDstWidth, int nDstHeight, unsigned char *dpSrcY, unsigned char* dpSrcU, unsigned char* dpSrcV, int nSrcPitch, int nSrcChromaPitch, int nSrcWidth, int nSrcHeight, bool bSemiplanar); #ifdef __cuda_cuda_h__ void ComputeCRC(uint8_t *pBuffer, uint32_t *crcValue, CUstream_st *outputCUStream); #endif ================================================ FILE: alvr/server_openvr/cpp/platform/win32/NvEncoder.cpp ================================================ /* * Copyright 2017-2022 NVIDIA Corporation. All rights reserved. * * Please refer to the NVIDIA end user license agreement (EULA) associated * with this source code for terms and conditions that govern your use of * this software. Any use, reproduction, disclosure, or distribution of * this software and related documentation outside the terms of the EULA * is strictly prohibited. * */ #include "NvEncoder.h" #ifndef _WIN32 #include static inline bool operator==(const GUID &guid1, const GUID &guid2) { return !memcmp(&guid1, &guid2, sizeof(GUID)); } static inline bool operator!=(const GUID &guid1, const GUID &guid2) { return !(guid1 == guid2); } #endif NvEncoder::NvEncoder(NV_ENC_DEVICE_TYPE eDeviceType, void *pDevice, uint32_t nWidth, uint32_t nHeight, NV_ENC_BUFFER_FORMAT eBufferFormat, uint32_t nExtraOutputDelay, bool bMotionEstimationOnly, bool bOutputInVideoMemory, bool bDX12Encode, bool bUseIVFContainer) : m_pDevice(pDevice), m_eDeviceType(eDeviceType), m_nWidth(nWidth), m_nHeight(nHeight), m_nMaxEncodeWidth(nWidth), m_nMaxEncodeHeight(nHeight), m_eBufferFormat(eBufferFormat), m_bMotionEstimationOnly(bMotionEstimationOnly), m_bOutputInVideoMemory(bOutputInVideoMemory), m_bIsDX12Encode(bDX12Encode), m_bUseIVFContainer(bUseIVFContainer), m_nExtraOutputDelay(nExtraOutputDelay), m_hEncoder(nullptr) { LoadNvEncApi(); if (!m_nvenc.nvEncOpenEncodeSession) { m_nEncoderBuffer = 0; NVENC_THROW_ERROR("EncodeAPI not found", NV_ENC_ERR_NO_ENCODE_DEVICE); } NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS encodeSessionExParams = { NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER }; encodeSessionExParams.device = m_pDevice; encodeSessionExParams.deviceType = m_eDeviceType; encodeSessionExParams.apiVersion = NVENCAPI_VERSION; void *hEncoder = NULL; NVENC_API_CALL(m_nvenc.nvEncOpenEncodeSessionEx(&encodeSessionExParams, &hEncoder)); m_hEncoder = hEncoder; } void NvEncoder::LoadNvEncApi() { #if defined(_WIN32) #if defined(_WIN64) HMODULE hModule = LoadLibrary(TEXT("nvEncodeAPI64.dll")); #else HMODULE hModule = LoadLibrary(TEXT("nvEncodeAPI.dll")); #endif #else void *hModule = dlopen("libnvidia-encode.so.1", RTLD_LAZY); #endif if (hModule == NULL) { NVENC_THROW_ERROR("NVENC library file is not found. Please ensure NV driver is installed", NV_ENC_ERR_NO_ENCODE_DEVICE); } m_hModule = hModule; typedef NVENCSTATUS(NVENCAPI *NvEncodeAPIGetMaxSupportedVersion_Type)(uint32_t*); #if defined(_WIN32) NvEncodeAPIGetMaxSupportedVersion_Type NvEncodeAPIGetMaxSupportedVersion = (NvEncodeAPIGetMaxSupportedVersion_Type)GetProcAddress(hModule, "NvEncodeAPIGetMaxSupportedVersion"); #else NvEncodeAPIGetMaxSupportedVersion_Type NvEncodeAPIGetMaxSupportedVersion = (NvEncodeAPIGetMaxSupportedVersion_Type)dlsym(hModule, "NvEncodeAPIGetMaxSupportedVersion"); #endif uint32_t version = 0; uint32_t currentVersion = (NVENCAPI_MAJOR_VERSION << 4) | NVENCAPI_MINOR_VERSION; NVENC_API_CALL(NvEncodeAPIGetMaxSupportedVersion(&version)); if (currentVersion > version) { NVENC_THROW_ERROR("Current Driver Version does not support this NvEncodeAPI version, please upgrade driver", NV_ENC_ERR_INVALID_VERSION); } typedef NVENCSTATUS(NVENCAPI *NvEncodeAPICreateInstance_Type)(NV_ENCODE_API_FUNCTION_LIST*); #if defined(_WIN32) NvEncodeAPICreateInstance_Type NvEncodeAPICreateInstance = (NvEncodeAPICreateInstance_Type)GetProcAddress(hModule, "NvEncodeAPICreateInstance"); #else NvEncodeAPICreateInstance_Type NvEncodeAPICreateInstance = (NvEncodeAPICreateInstance_Type)dlsym(hModule, "NvEncodeAPICreateInstance"); #endif if (!NvEncodeAPICreateInstance) { NVENC_THROW_ERROR("Cannot find NvEncodeAPICreateInstance() entry in NVENC library", NV_ENC_ERR_NO_ENCODE_DEVICE); } m_nvenc = { NV_ENCODE_API_FUNCTION_LIST_VER }; NVENC_API_CALL(NvEncodeAPICreateInstance(&m_nvenc)); } NvEncoder::~NvEncoder() { DestroyHWEncoder(); if (m_hModule) { #if defined(_WIN32) FreeLibrary((HMODULE)m_hModule); #else dlclose(m_hModule); #endif m_hModule = nullptr; } } void NvEncoder::CreateDefaultEncoderParams(NV_ENC_INITIALIZE_PARAMS* pIntializeParams, GUID codecGuid, GUID presetGuid, NV_ENC_TUNING_INFO tuningInfo) { if (!m_hEncoder) { NVENC_THROW_ERROR("Encoder Initialization failed", NV_ENC_ERR_NO_ENCODE_DEVICE); return; } if (pIntializeParams == nullptr || pIntializeParams->encodeConfig == nullptr) { NVENC_THROW_ERROR("pInitializeParams and pInitializeParams->encodeConfig can't be NULL", NV_ENC_ERR_INVALID_PTR); } memset(pIntializeParams->encodeConfig, 0, sizeof(NV_ENC_CONFIG)); auto pEncodeConfig = pIntializeParams->encodeConfig; memset(pIntializeParams, 0, sizeof(NV_ENC_INITIALIZE_PARAMS)); pIntializeParams->encodeConfig = pEncodeConfig; pIntializeParams->encodeConfig->version = NV_ENC_CONFIG_VER; pIntializeParams->version = NV_ENC_INITIALIZE_PARAMS_VER; pIntializeParams->encodeGUID = codecGuid; pIntializeParams->presetGUID = presetGuid; pIntializeParams->encodeWidth = m_nWidth; pIntializeParams->encodeHeight = m_nHeight; pIntializeParams->darWidth = m_nWidth; pIntializeParams->darHeight = m_nHeight; pIntializeParams->frameRateNum = 30; pIntializeParams->frameRateDen = 1; pIntializeParams->enablePTD = 1; pIntializeParams->reportSliceOffsets = 0; pIntializeParams->enableSubFrameWrite = 0; pIntializeParams->maxEncodeWidth = m_nWidth; pIntializeParams->maxEncodeHeight = m_nHeight; pIntializeParams->enableMEOnlyMode = m_bMotionEstimationOnly; pIntializeParams->enableOutputInVidmem = m_bOutputInVideoMemory; #if defined(_WIN32) if (!m_bOutputInVideoMemory) { pIntializeParams->enableEncodeAsync = GetCapabilityValue(codecGuid, NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT); } #endif NV_ENC_PRESET_CONFIG presetConfig = { NV_ENC_PRESET_CONFIG_VER, { NV_ENC_CONFIG_VER } }; m_nvenc.nvEncGetEncodePresetConfig(m_hEncoder, codecGuid, presetGuid, &presetConfig); memcpy(pIntializeParams->encodeConfig, &presetConfig.presetCfg, sizeof(NV_ENC_CONFIG)); pIntializeParams->encodeConfig->frameIntervalP = 1; pIntializeParams->encodeConfig->gopLength = NVENC_INFINITE_GOPLENGTH; pIntializeParams->encodeConfig->rcParams.rateControlMode = NV_ENC_PARAMS_RC_CONSTQP; if (!m_bMotionEstimationOnly) { pIntializeParams->tuningInfo = tuningInfo; NV_ENC_PRESET_CONFIG presetConfig = { NV_ENC_PRESET_CONFIG_VER, { NV_ENC_CONFIG_VER } }; m_nvenc.nvEncGetEncodePresetConfigEx(m_hEncoder, codecGuid, presetGuid, tuningInfo, &presetConfig); memcpy(pIntializeParams->encodeConfig, &presetConfig.presetCfg, sizeof(NV_ENC_CONFIG)); } else { m_encodeConfig.version = NV_ENC_CONFIG_VER; m_encodeConfig.rcParams.rateControlMode = NV_ENC_PARAMS_RC_CONSTQP; m_encodeConfig.rcParams.constQP = { 28, 31, 25 }; } if (pIntializeParams->encodeGUID == NV_ENC_CODEC_H264_GUID) { if (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444 || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT) { pIntializeParams->encodeConfig->encodeCodecConfig.h264Config.chromaFormatIDC = 3; } pIntializeParams->encodeConfig->encodeCodecConfig.h264Config.idrPeriod = pIntializeParams->encodeConfig->gopLength; } else if (pIntializeParams->encodeGUID == NV_ENC_CODEC_HEVC_GUID) { pIntializeParams->encodeConfig->encodeCodecConfig.hevcConfig.pixelBitDepthMinus8 = (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV420_10BIT || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT ) ? 2 : 0; if (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444 || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT) { pIntializeParams->encodeConfig->encodeCodecConfig.hevcConfig.chromaFormatIDC = 3; } pIntializeParams->encodeConfig->encodeCodecConfig.hevcConfig.idrPeriod = pIntializeParams->encodeConfig->gopLength; } else if (pIntializeParams->encodeGUID == NV_ENC_CODEC_AV1_GUID) { pIntializeParams->encodeConfig->encodeCodecConfig.av1Config.pixelBitDepthMinus8 = (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV420_10BIT) ? 2 : 0; pIntializeParams->encodeConfig->encodeCodecConfig.av1Config.inputPixelBitDepthMinus8 = (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV420_10BIT) ? 2 : 0; pIntializeParams->encodeConfig->encodeCodecConfig.av1Config.chromaFormatIDC = 1; pIntializeParams->encodeConfig->encodeCodecConfig.av1Config.idrPeriod = pIntializeParams->encodeConfig->gopLength; if (m_bOutputInVideoMemory) { pIntializeParams->encodeConfig->frameIntervalP = 1; } } if (m_bIsDX12Encode) { pIntializeParams->bufferFormat = m_eBufferFormat; } return; } void NvEncoder::CreateEncoder(const NV_ENC_INITIALIZE_PARAMS* pEncoderParams) { if (!m_hEncoder) { NVENC_THROW_ERROR("Encoder Initialization failed", NV_ENC_ERR_NO_ENCODE_DEVICE); } if (!pEncoderParams) { NVENC_THROW_ERROR("Invalid NV_ENC_INITIALIZE_PARAMS ptr", NV_ENC_ERR_INVALID_PTR); } if (pEncoderParams->encodeWidth == 0 || pEncoderParams->encodeHeight == 0) { NVENC_THROW_ERROR("Invalid encoder width and height", NV_ENC_ERR_INVALID_PARAM); } if (pEncoderParams->encodeGUID != NV_ENC_CODEC_H264_GUID && pEncoderParams->encodeGUID != NV_ENC_CODEC_HEVC_GUID && pEncoderParams->encodeGUID != NV_ENC_CODEC_AV1_GUID) { NVENC_THROW_ERROR("Invalid codec guid", NV_ENC_ERR_INVALID_PARAM); } if (pEncoderParams->encodeGUID == NV_ENC_CODEC_H264_GUID) { if (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV420_10BIT || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT) { NVENC_THROW_ERROR("10-bit format isn't supported by H264 encoder", NV_ENC_ERR_INVALID_PARAM); } } if (pEncoderParams->encodeGUID == NV_ENC_CODEC_AV1_GUID) { if (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444 || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT) { NVENC_THROW_ERROR("YUV444 format isn't supported by AV1 encoder", NV_ENC_ERR_INVALID_PARAM); } } // set other necessary params if not set yet if (pEncoderParams->encodeGUID == NV_ENC_CODEC_H264_GUID) { if ((m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444) && (pEncoderParams->encodeConfig->encodeCodecConfig.h264Config.chromaFormatIDC != 3)) { NVENC_THROW_ERROR("Invalid ChromaFormatIDC", NV_ENC_ERR_INVALID_PARAM); } } if (pEncoderParams->encodeGUID == NV_ENC_CODEC_HEVC_GUID) { bool yuv10BitFormat = (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV420_10BIT || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT) ? true : false; if (yuv10BitFormat && pEncoderParams->encodeConfig->encodeCodecConfig.hevcConfig.pixelBitDepthMinus8 != 2) { NVENC_THROW_ERROR("Invalid PixelBitdepth", NV_ENC_ERR_INVALID_PARAM); } if ((m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444 || m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV444_10BIT) && (pEncoderParams->encodeConfig->encodeCodecConfig.hevcConfig.chromaFormatIDC != 3)) { NVENC_THROW_ERROR("Invalid ChromaFormatIDC", NV_ENC_ERR_INVALID_PARAM); } } if (pEncoderParams->encodeGUID == NV_ENC_CODEC_AV1_GUID) { bool yuv10BitFormat = (m_eBufferFormat == NV_ENC_BUFFER_FORMAT_YUV420_10BIT) ? true : false; if (yuv10BitFormat && pEncoderParams->encodeConfig->encodeCodecConfig.av1Config.pixelBitDepthMinus8 != 2) { NVENC_THROW_ERROR("Invalid PixelBitdepth", NV_ENC_ERR_INVALID_PARAM); } if (pEncoderParams->encodeConfig->encodeCodecConfig.av1Config.chromaFormatIDC != 1) { NVENC_THROW_ERROR("Invalid ChromaFormatIDC", NV_ENC_ERR_INVALID_PARAM); } if (m_bOutputInVideoMemory && pEncoderParams->encodeConfig->frameIntervalP > 1) { NVENC_THROW_ERROR("Alt Ref frames not supported for AV1 in case of OutputInVideoMemory", NV_ENC_ERR_INVALID_PARAM); } } memcpy(&m_initializeParams, pEncoderParams, sizeof(m_initializeParams)); m_initializeParams.version = NV_ENC_INITIALIZE_PARAMS_VER; if (pEncoderParams->encodeConfig) { memcpy(&m_encodeConfig, pEncoderParams->encodeConfig, sizeof(m_encodeConfig)); m_encodeConfig.version = NV_ENC_CONFIG_VER; } else { NV_ENC_PRESET_CONFIG presetConfig = { NV_ENC_PRESET_CONFIG_VER, { NV_ENC_CONFIG_VER } }; if (!m_bMotionEstimationOnly) { m_nvenc.nvEncGetEncodePresetConfigEx(m_hEncoder, pEncoderParams->encodeGUID, pEncoderParams->presetGUID, pEncoderParams->tuningInfo, &presetConfig); memcpy(&m_encodeConfig, &presetConfig.presetCfg, sizeof(NV_ENC_CONFIG)); if (m_bOutputInVideoMemory && pEncoderParams->encodeGUID == NV_ENC_CODEC_AV1_GUID) { m_encodeConfig.frameIntervalP = 1; } } else { m_encodeConfig.version = NV_ENC_CONFIG_VER; m_encodeConfig.rcParams.rateControlMode = NV_ENC_PARAMS_RC_CONSTQP; m_encodeConfig.rcParams.constQP = { 28, 31, 25 }; } } m_initializeParams.encodeConfig = &m_encodeConfig; NVENC_API_CALL(m_nvenc.nvEncInitializeEncoder(m_hEncoder, &m_initializeParams)); m_bEncoderInitialized = true; m_nWidth = m_initializeParams.encodeWidth; m_nHeight = m_initializeParams.encodeHeight; m_nMaxEncodeWidth = m_initializeParams.maxEncodeWidth; m_nMaxEncodeHeight = m_initializeParams.maxEncodeHeight; m_nEncoderBuffer = m_encodeConfig.frameIntervalP + m_encodeConfig.rcParams.lookaheadDepth + m_nExtraOutputDelay; m_nOutputDelay = m_nEncoderBuffer - 1; if (!m_bOutputInVideoMemory) { m_vpCompletionEvent.resize(m_nEncoderBuffer, nullptr); } #if defined(_WIN32) for (uint32_t i = 0; i < m_vpCompletionEvent.size(); i++) { m_vpCompletionEvent[i] = CreateEvent(NULL, FALSE, FALSE, NULL); if (!m_bIsDX12Encode) { NV_ENC_EVENT_PARAMS eventParams = { NV_ENC_EVENT_PARAMS_VER }; eventParams.completionEvent = m_vpCompletionEvent[i]; m_nvenc.nvEncRegisterAsyncEvent(m_hEncoder, &eventParams); } } #endif m_vMappedInputBuffers.resize(m_nEncoderBuffer, nullptr); if (m_bMotionEstimationOnly) { m_vMappedRefBuffers.resize(m_nEncoderBuffer, nullptr); if (!m_bOutputInVideoMemory) { InitializeMVOutputBuffer(); } } else { if (!m_bOutputInVideoMemory && !m_bIsDX12Encode) { m_vBitstreamOutputBuffer.resize(m_nEncoderBuffer, nullptr); InitializeBitstreamBuffer(); } } AllocateInputBuffers(m_nEncoderBuffer); } void NvEncoder::DestroyEncoder() { if (!m_hEncoder) { return; } ReleaseInputBuffers(); DestroyHWEncoder(); } void NvEncoder::DestroyHWEncoder() { if (!m_hEncoder) { return; } #if defined(_WIN32) for (uint32_t i = 0; i < m_vpCompletionEvent.size(); i++) { if (m_vpCompletionEvent[i]) { if (!m_bIsDX12Encode) { NV_ENC_EVENT_PARAMS eventParams = { NV_ENC_EVENT_PARAMS_VER }; eventParams.completionEvent = m_vpCompletionEvent[i]; m_nvenc.nvEncUnregisterAsyncEvent(m_hEncoder, &eventParams); } CloseHandle(m_vpCompletionEvent[i]); } } m_vpCompletionEvent.clear(); #endif if (m_bMotionEstimationOnly) { DestroyMVOutputBuffer(); } else { if (!m_bIsDX12Encode) DestroyBitstreamBuffer(); } m_nvenc.nvEncDestroyEncoder(m_hEncoder); m_hEncoder = nullptr; m_bEncoderInitialized = false; } const NvEncInputFrame* NvEncoder::GetNextInputFrame() { int i = m_iToSend % m_nEncoderBuffer; return &m_vInputFrames[i]; } const NvEncInputFrame* NvEncoder::GetNextReferenceFrame() { int i = m_iToSend % m_nEncoderBuffer; return &m_vReferenceFrames[i]; } void NvEncoder::MapResources(uint32_t bfrIdx) { NV_ENC_MAP_INPUT_RESOURCE mapInputResource = { NV_ENC_MAP_INPUT_RESOURCE_VER }; mapInputResource.registeredResource = m_vRegisteredResources[bfrIdx]; NVENC_API_CALL(m_nvenc.nvEncMapInputResource(m_hEncoder, &mapInputResource)); m_vMappedInputBuffers[bfrIdx] = mapInputResource.mappedResource; if (m_bMotionEstimationOnly) { mapInputResource.registeredResource = m_vRegisteredResourcesForReference[bfrIdx]; NVENC_API_CALL(m_nvenc.nvEncMapInputResource(m_hEncoder, &mapInputResource)); m_vMappedRefBuffers[bfrIdx] = mapInputResource.mappedResource; } } void NvEncoder::EncodeFrame(std::vector> &vPacket, NV_ENC_PIC_PARAMS *pPicParams) { vPacket.clear(); if (!IsHWEncoderInitialized()) { NVENC_THROW_ERROR("Encoder device not found", NV_ENC_ERR_NO_ENCODE_DEVICE); } int bfrIdx = m_iToSend % m_nEncoderBuffer; MapResources(bfrIdx); NVENCSTATUS nvStatus = DoEncode(m_vMappedInputBuffers[bfrIdx], m_vBitstreamOutputBuffer[bfrIdx], pPicParams); if (nvStatus == NV_ENC_SUCCESS || nvStatus == NV_ENC_ERR_NEED_MORE_INPUT) { m_iToSend++; GetEncodedPacket(m_vBitstreamOutputBuffer, vPacket, true); } else { NVENC_THROW_ERROR("nvEncEncodePicture API failed", nvStatus); } } void NvEncoder::RunMotionEstimation(std::vector &mvData) { if (!m_hEncoder) { NVENC_THROW_ERROR("Encoder Initialization failed", NV_ENC_ERR_NO_ENCODE_DEVICE); return; } const uint32_t bfrIdx = m_iToSend % m_nEncoderBuffer; MapResources(bfrIdx); NVENCSTATUS nvStatus = DoMotionEstimation(m_vMappedInputBuffers[bfrIdx], m_vMappedRefBuffers[bfrIdx], m_vMVDataOutputBuffer[bfrIdx]); if (nvStatus == NV_ENC_SUCCESS) { m_iToSend++; std::vector> vPacket; GetEncodedPacket(m_vMVDataOutputBuffer, vPacket, true); if (vPacket.size() != 1) { NVENC_THROW_ERROR("GetEncodedPacket() doesn't return one (and only one) MVData", NV_ENC_ERR_GENERIC); } mvData = vPacket[0]; } else { NVENC_THROW_ERROR("nvEncEncodePicture API failed", nvStatus); } } void NvEncoder::GetSequenceParams(std::vector &seqParams) { uint8_t spsppsData[1024]; // Assume maximum spspps data is 1KB or less memset(spsppsData, 0, sizeof(spsppsData)); NV_ENC_SEQUENCE_PARAM_PAYLOAD payload = { NV_ENC_SEQUENCE_PARAM_PAYLOAD_VER }; uint32_t spsppsSize = 0; payload.spsppsBuffer = spsppsData; payload.inBufferSize = sizeof(spsppsData); payload.outSPSPPSPayloadSize = &spsppsSize; NVENC_API_CALL(m_nvenc.nvEncGetSequenceParams(m_hEncoder, &payload)); seqParams.clear(); seqParams.insert(seqParams.end(), &spsppsData[0], &spsppsData[spsppsSize]); } NVENCSTATUS NvEncoder::DoEncode(NV_ENC_INPUT_PTR inputBuffer, NV_ENC_OUTPUT_PTR outputBuffer, NV_ENC_PIC_PARAMS *pPicParams) { NV_ENC_PIC_PARAMS picParams = {}; if (pPicParams) { picParams = *pPicParams; } picParams.version = NV_ENC_PIC_PARAMS_VER; picParams.pictureStruct = NV_ENC_PIC_STRUCT_FRAME; picParams.inputBuffer = inputBuffer; picParams.bufferFmt = GetPixelFormat(); picParams.inputWidth = GetEncodeWidth(); picParams.inputHeight = GetEncodeHeight(); picParams.outputBitstream = outputBuffer; picParams.completionEvent = GetCompletionEvent(m_iToSend % m_nEncoderBuffer); NVENCSTATUS nvStatus = m_nvenc.nvEncEncodePicture(m_hEncoder, &picParams); return nvStatus; } void NvEncoder::SendEOS() { NV_ENC_PIC_PARAMS picParams = { NV_ENC_PIC_PARAMS_VER }; picParams.encodePicFlags = NV_ENC_PIC_FLAG_EOS; picParams.completionEvent = GetCompletionEvent(m_iToSend % m_nEncoderBuffer); NVENC_API_CALL(m_nvenc.nvEncEncodePicture(m_hEncoder, &picParams)); } void NvEncoder::EndEncode(std::vector> &vPacket) { vPacket.clear(); if (!IsHWEncoderInitialized()) { NVENC_THROW_ERROR("Encoder device not initialized", NV_ENC_ERR_ENCODER_NOT_INITIALIZED); } SendEOS(); GetEncodedPacket(m_vBitstreamOutputBuffer, vPacket, false); } void NvEncoder::GetEncodedPacket(std::vector &vOutputBuffer, std::vector> &vPacket, bool bOutputDelay) { unsigned i = 0; int iEnd = bOutputDelay ? m_iToSend - m_nOutputDelay : m_iToSend; for (; m_iGot < iEnd; m_iGot++) { WaitForCompletionEvent(m_iGot % m_nEncoderBuffer); NV_ENC_LOCK_BITSTREAM lockBitstreamData = { NV_ENC_LOCK_BITSTREAM_VER }; lockBitstreamData.outputBitstream = vOutputBuffer[m_iGot % m_nEncoderBuffer]; lockBitstreamData.doNotWait = false; NVENC_API_CALL(m_nvenc.nvEncLockBitstream(m_hEncoder, &lockBitstreamData)); uint8_t *pData = (uint8_t *)lockBitstreamData.bitstreamBufferPtr; if (vPacket.size() < i + 1) { vPacket.push_back(std::vector()); } vPacket[i].clear(); if ((m_initializeParams.encodeGUID == NV_ENC_CODEC_AV1_GUID) && (m_bUseIVFContainer)) { if (m_bWriteIVFFileHeader) { m_IVFUtils.WriteFileHeader(vPacket[i], MAKE_FOURCC('A', 'V', '0', '1'), m_initializeParams.encodeWidth, m_initializeParams.encodeHeight, m_initializeParams.frameRateNum, m_initializeParams.frameRateDen, 0xFFFF); m_bWriteIVFFileHeader = false; } m_IVFUtils.WriteFrameHeader(vPacket[i], lockBitstreamData.bitstreamSizeInBytes, lockBitstreamData.outputTimeStamp); } vPacket[i].insert(vPacket[i].end(), &pData[0], &pData[lockBitstreamData.bitstreamSizeInBytes]); i++; NVENC_API_CALL(m_nvenc.nvEncUnlockBitstream(m_hEncoder, lockBitstreamData.outputBitstream)); if (m_vMappedInputBuffers[m_iGot % m_nEncoderBuffer]) { NVENC_API_CALL(m_nvenc.nvEncUnmapInputResource(m_hEncoder, m_vMappedInputBuffers[m_iGot % m_nEncoderBuffer])); m_vMappedInputBuffers[m_iGot % m_nEncoderBuffer] = nullptr; } if (m_bMotionEstimationOnly && m_vMappedRefBuffers[m_iGot % m_nEncoderBuffer]) { NVENC_API_CALL(m_nvenc.nvEncUnmapInputResource(m_hEncoder, m_vMappedRefBuffers[m_iGot % m_nEncoderBuffer])); m_vMappedRefBuffers[m_iGot % m_nEncoderBuffer] = nullptr; } } } bool NvEncoder::Reconfigure(const NV_ENC_RECONFIGURE_PARAMS *pReconfigureParams) { NVENC_API_CALL(m_nvenc.nvEncReconfigureEncoder(m_hEncoder, const_cast(pReconfigureParams))); memcpy(&m_initializeParams, &(pReconfigureParams->reInitEncodeParams), sizeof(m_initializeParams)); if (pReconfigureParams->reInitEncodeParams.encodeConfig) { memcpy(&m_encodeConfig, pReconfigureParams->reInitEncodeParams.encodeConfig, sizeof(m_encodeConfig)); } m_nWidth = m_initializeParams.encodeWidth; m_nHeight = m_initializeParams.encodeHeight; m_nMaxEncodeWidth = m_initializeParams.maxEncodeWidth; m_nMaxEncodeHeight = m_initializeParams.maxEncodeHeight; return true; } NV_ENC_REGISTERED_PTR NvEncoder::RegisterResource(void *pBuffer, NV_ENC_INPUT_RESOURCE_TYPE eResourceType, int width, int height, int pitch, NV_ENC_BUFFER_FORMAT bufferFormat, NV_ENC_BUFFER_USAGE bufferUsage, NV_ENC_FENCE_POINT_D3D12* pInputFencePoint) { NV_ENC_REGISTER_RESOURCE registerResource = { NV_ENC_REGISTER_RESOURCE_VER }; registerResource.resourceType = eResourceType; registerResource.resourceToRegister = pBuffer; registerResource.width = width; registerResource.height = height; registerResource.pitch = pitch; registerResource.bufferFormat = bufferFormat; registerResource.bufferUsage = bufferUsage; registerResource.pInputFencePoint = pInputFencePoint; NVENC_API_CALL(m_nvenc.nvEncRegisterResource(m_hEncoder, ®isterResource)); return registerResource.registeredResource; } void NvEncoder::RegisterInputResources(std::vector inputframes, NV_ENC_INPUT_RESOURCE_TYPE eResourceType, int width, int height, int pitch, NV_ENC_BUFFER_FORMAT bufferFormat, bool bReferenceFrame) { for (uint32_t i = 0; i < inputframes.size(); ++i) { NV_ENC_REGISTERED_PTR registeredPtr = RegisterResource(inputframes[i], eResourceType, width, height, pitch, bufferFormat, NV_ENC_INPUT_IMAGE); std::vector _chromaOffsets; NvEncoder::GetChromaSubPlaneOffsets(bufferFormat, pitch, height, _chromaOffsets); NvEncInputFrame inputframe = {}; inputframe.inputPtr = (void *)inputframes[i]; inputframe.chromaOffsets[0] = 0; inputframe.chromaOffsets[1] = 0; for (uint32_t ch = 0; ch < _chromaOffsets.size(); ch++) { inputframe.chromaOffsets[ch] = _chromaOffsets[ch]; } inputframe.numChromaPlanes = NvEncoder::GetNumChromaPlanes(bufferFormat); inputframe.pitch = pitch; inputframe.chromaPitch = NvEncoder::GetChromaPitch(bufferFormat, pitch); inputframe.bufferFormat = bufferFormat; inputframe.resourceType = eResourceType; if (bReferenceFrame) { m_vRegisteredResourcesForReference.push_back(registeredPtr); m_vReferenceFrames.push_back(inputframe); } else { m_vRegisteredResources.push_back(registeredPtr); m_vInputFrames.push_back(inputframe); } } } void NvEncoder::FlushEncoder() { if (!m_bMotionEstimationOnly && !m_bOutputInVideoMemory) { // Incase of error it is possible for buffers still mapped to encoder. // flush the encoder queue and then unmapped it if any surface is still mapped try { std::vector> vPacket; EndEncode(vPacket); } catch (...) { } } } void NvEncoder::UnregisterInputResources() { FlushEncoder(); if (m_bMotionEstimationOnly) { for (uint32_t i = 0; i < m_vMappedRefBuffers.size(); ++i) { if (m_vMappedRefBuffers[i]) { m_nvenc.nvEncUnmapInputResource(m_hEncoder, m_vMappedRefBuffers[i]); } } } m_vMappedRefBuffers.clear(); for (uint32_t i = 0; i < m_vMappedInputBuffers.size(); ++i) { if (m_vMappedInputBuffers[i]) { m_nvenc.nvEncUnmapInputResource(m_hEncoder, m_vMappedInputBuffers[i]); } } m_vMappedInputBuffers.clear(); for (uint32_t i = 0; i < m_vRegisteredResources.size(); ++i) { if (m_vRegisteredResources[i]) { m_nvenc.nvEncUnregisterResource(m_hEncoder, m_vRegisteredResources[i]); } } m_vRegisteredResources.clear(); for (uint32_t i = 0; i < m_vRegisteredResourcesForReference.size(); ++i) { if (m_vRegisteredResourcesForReference[i]) { m_nvenc.nvEncUnregisterResource(m_hEncoder, m_vRegisteredResourcesForReference[i]); } } m_vRegisteredResourcesForReference.clear(); } void NvEncoder::WaitForCompletionEvent(int iEvent) { #if defined(_WIN32) // Check if we are in async mode. If not, don't wait for event; NV_ENC_CONFIG sEncodeConfig = { 0 }; NV_ENC_INITIALIZE_PARAMS sInitializeParams = { 0 }; sInitializeParams.encodeConfig = &sEncodeConfig; GetInitializeParams(&sInitializeParams); if (0U == sInitializeParams.enableEncodeAsync) { return; } #ifdef DEBUG WaitForSingleObject(m_vpCompletionEvent[iEvent], INFINITE); #else // wait for 20s which is infinite on terms of gpu time if (WaitForSingleObject(m_vpCompletionEvent[iEvent], 20000) == WAIT_FAILED) { NVENC_THROW_ERROR("Failed to encode frame", NV_ENC_ERR_GENERIC); } #endif #endif } uint32_t NvEncoder::GetWidthInBytes(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t width) { switch (bufferFormat) { case NV_ENC_BUFFER_FORMAT_NV12: case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: case NV_ENC_BUFFER_FORMAT_YUV444: return width; case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: return width * 2; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return width * 4; default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return 0; } } uint32_t NvEncoder::GetNumChromaPlanes(const NV_ENC_BUFFER_FORMAT bufferFormat) { switch (bufferFormat) { case NV_ENC_BUFFER_FORMAT_NV12: case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: return 1; case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: case NV_ENC_BUFFER_FORMAT_YUV444: case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: return 2; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return 0; default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return -1; } } uint32_t NvEncoder::GetChromaPitch(const NV_ENC_BUFFER_FORMAT bufferFormat,const uint32_t lumaPitch) { switch (bufferFormat) { case NV_ENC_BUFFER_FORMAT_NV12: case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: case NV_ENC_BUFFER_FORMAT_YUV444: case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: return lumaPitch; case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: return (lumaPitch + 1)/2; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return 0; default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return -1; } } void NvEncoder::GetChromaSubPlaneOffsets(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t pitch, const uint32_t height, std::vector& chromaOffsets) { chromaOffsets.clear(); switch (bufferFormat) { case NV_ENC_BUFFER_FORMAT_NV12: case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: chromaOffsets.push_back(pitch * height); return; case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: chromaOffsets.push_back(pitch * height); chromaOffsets.push_back(chromaOffsets[0] + (NvEncoder::GetChromaPitch(bufferFormat, pitch) * GetChromaHeight(bufferFormat, height))); return; case NV_ENC_BUFFER_FORMAT_YUV444: case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: chromaOffsets.push_back(pitch * height); chromaOffsets.push_back(chromaOffsets[0] + (pitch * height)); return; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return; default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return; } } uint32_t NvEncoder::GetChromaHeight(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t lumaHeight) { switch (bufferFormat) { case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: case NV_ENC_BUFFER_FORMAT_NV12: case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: return (lumaHeight + 1)/2; case NV_ENC_BUFFER_FORMAT_YUV444: case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: return lumaHeight; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return 0; default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return 0; } } uint32_t NvEncoder::GetChromaWidthInBytes(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t lumaWidth) { switch (bufferFormat) { case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: return (lumaWidth + 1) / 2; case NV_ENC_BUFFER_FORMAT_NV12: return lumaWidth; case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: return 2 * lumaWidth; case NV_ENC_BUFFER_FORMAT_YUV444: return lumaWidth; case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: return 2 * lumaWidth; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return 0; default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return 0; } } int NvEncoder::GetCapabilityValue(GUID guidCodec, NV_ENC_CAPS capsToQuery) { if (!m_hEncoder) { return 0; } NV_ENC_CAPS_PARAM capsParam = { NV_ENC_CAPS_PARAM_VER }; capsParam.capsToQuery = capsToQuery; int v; m_nvenc.nvEncGetEncodeCaps(m_hEncoder, guidCodec, &capsParam, &v); return v; } int NvEncoder::GetFrameSize() const { switch (GetPixelFormat()) { case NV_ENC_BUFFER_FORMAT_YV12: case NV_ENC_BUFFER_FORMAT_IYUV: case NV_ENC_BUFFER_FORMAT_NV12: return GetEncodeWidth() * (GetEncodeHeight() + (GetEncodeHeight() + 1) / 2); case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: return 2 * GetEncodeWidth() * (GetEncodeHeight() + (GetEncodeHeight() + 1) / 2); case NV_ENC_BUFFER_FORMAT_YUV444: return GetEncodeWidth() * GetEncodeHeight() * 3; case NV_ENC_BUFFER_FORMAT_YUV444_10BIT: return 2 * GetEncodeWidth() * GetEncodeHeight() * 3; case NV_ENC_BUFFER_FORMAT_ARGB: case NV_ENC_BUFFER_FORMAT_ARGB10: case NV_ENC_BUFFER_FORMAT_AYUV: case NV_ENC_BUFFER_FORMAT_ABGR: case NV_ENC_BUFFER_FORMAT_ABGR10: return 4 * GetEncodeWidth() * GetEncodeHeight(); default: NVENC_THROW_ERROR("Invalid Buffer format", NV_ENC_ERR_INVALID_PARAM); return 0; } } void NvEncoder::GetInitializeParams(NV_ENC_INITIALIZE_PARAMS *pInitializeParams) { if (!pInitializeParams || !pInitializeParams->encodeConfig) { NVENC_THROW_ERROR("Both pInitializeParams and pInitializeParams->encodeConfig can't be NULL", NV_ENC_ERR_INVALID_PTR); } NV_ENC_CONFIG *pEncodeConfig = pInitializeParams->encodeConfig; *pEncodeConfig = m_encodeConfig; *pInitializeParams = m_initializeParams; pInitializeParams->encodeConfig = pEncodeConfig; } void NvEncoder::InitializeBitstreamBuffer() { for (int i = 0; i < m_nEncoderBuffer; i++) { NV_ENC_CREATE_BITSTREAM_BUFFER createBitstreamBuffer = { NV_ENC_CREATE_BITSTREAM_BUFFER_VER }; NVENC_API_CALL(m_nvenc.nvEncCreateBitstreamBuffer(m_hEncoder, &createBitstreamBuffer)); m_vBitstreamOutputBuffer[i] = createBitstreamBuffer.bitstreamBuffer; } } void NvEncoder::DestroyBitstreamBuffer() { for (uint32_t i = 0; i < m_vBitstreamOutputBuffer.size(); i++) { if (m_vBitstreamOutputBuffer[i]) { m_nvenc.nvEncDestroyBitstreamBuffer(m_hEncoder, m_vBitstreamOutputBuffer[i]); } } m_vBitstreamOutputBuffer.clear(); } void NvEncoder::InitializeMVOutputBuffer() { for (int i = 0; i < m_nEncoderBuffer; i++) { NV_ENC_CREATE_MV_BUFFER createMVBuffer = { NV_ENC_CREATE_MV_BUFFER_VER }; NVENC_API_CALL(m_nvenc.nvEncCreateMVBuffer(m_hEncoder, &createMVBuffer)); m_vMVDataOutputBuffer.push_back(createMVBuffer.mvBuffer); } } void NvEncoder::DestroyMVOutputBuffer() { for (uint32_t i = 0; i < m_vMVDataOutputBuffer.size(); i++) { if (m_vMVDataOutputBuffer[i]) { m_nvenc.nvEncDestroyMVBuffer(m_hEncoder, m_vMVDataOutputBuffer[i]); } } m_vMVDataOutputBuffer.clear(); } NVENCSTATUS NvEncoder::DoMotionEstimation(NV_ENC_INPUT_PTR inputBuffer, NV_ENC_INPUT_PTR inputBufferForReference, NV_ENC_OUTPUT_PTR outputBuffer) { NV_ENC_MEONLY_PARAMS meParams = { NV_ENC_MEONLY_PARAMS_VER }; meParams.inputBuffer = inputBuffer; meParams.referenceFrame = inputBufferForReference; meParams.inputWidth = GetEncodeWidth(); meParams.inputHeight = GetEncodeHeight(); meParams.mvBuffer = outputBuffer; meParams.completionEvent = GetCompletionEvent(m_iToSend % m_nEncoderBuffer); NVENCSTATUS nvStatus = m_nvenc.nvEncRunMotionEstimationOnly(m_hEncoder, &meParams); return nvStatus; } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/NvEncoder.h ================================================ /* * Copyright 2017-2022 NVIDIA Corporation. All rights reserved. * * Please refer to the NVIDIA end user license agreement (EULA) associated * with this source code for terms and conditions that govern your use of * this software. Any use, reproduction, disclosure, or distribution of * this software and related documentation outside the terms of the EULA * is strictly prohibited. * */ #pragma once #include #include "alvr_server/nvEncodeAPI.h" #include #include #include #include #include #include #include "NvCodecUtils.h" /** * @brief Exception class for error reporting from NvEncodeAPI calls. */ class NVENCException : public std::exception { public: NVENCException(const std::string& errorStr, const NVENCSTATUS errorCode) : m_errorString(errorStr), m_errorCode(errorCode) {} virtual ~NVENCException() throw() {} virtual const char* what() const throw() { return m_errorString.c_str(); } NVENCSTATUS getErrorCode() const { return m_errorCode; } const std::string& getErrorString() const { return m_errorString; } static NVENCException makeNVENCException(const std::string& errorStr, const NVENCSTATUS errorCode, const std::string& functionName, const std::string& fileName, int lineNo); private: std::string m_errorString; NVENCSTATUS m_errorCode; }; inline NVENCException NVENCException::makeNVENCException(const std::string& errorStr, const NVENCSTATUS errorCode, const std::string& functionName, const std::string& fileName, int lineNo) { std::ostringstream errorLog; errorLog << functionName << " : " << errorStr << " at " << fileName << ":" << lineNo << std::endl; NVENCException exception(errorLog.str(), errorCode); return exception; } #define NVENC_THROW_ERROR( errorStr, errorCode ) \ do \ { \ throw NVENCException::makeNVENCException(errorStr, errorCode, __FUNCTION__, __FILE__, __LINE__); \ } while (0) #define NVENC_API_CALL( nvencAPI ) \ do \ { \ NVENCSTATUS errorCode = nvencAPI; \ if( errorCode != NV_ENC_SUCCESS) \ { \ std::ostringstream errorLog; \ errorLog << #nvencAPI << " returned error " << errorCode; \ throw NVENCException::makeNVENCException(errorLog.str(), errorCode, __FUNCTION__, __FILE__, __LINE__); \ } \ } while (0) struct NvEncInputFrame { void* inputPtr = nullptr; uint32_t chromaOffsets[2]; uint32_t numChromaPlanes; uint32_t pitch; uint32_t chromaPitch; NV_ENC_BUFFER_FORMAT bufferFormat; NV_ENC_INPUT_RESOURCE_TYPE resourceType; }; /** * @brief Shared base class for different encoder interfaces. */ class NvEncoder { public: /** * @brief This function is used to initialize the encoder session. * Application must call this function to initialize the encoder, before * starting to encode any frames. */ virtual void CreateEncoder(const NV_ENC_INITIALIZE_PARAMS* pEncodeParams); /** * @brief This function is used to destroy the encoder session. * Application must call this function to destroy the encoder session and * clean up any allocated resources. The application must call EndEncode() * function to get any queued encoded frames before calling DestroyEncoder(). */ virtual void DestroyEncoder(); /** * @brief This function is used to reconfigure an existing encoder session. * Application can use this function to dynamically change the bitrate, * resolution and other QOS parameters. If the application changes the * resolution, it must set NV_ENC_RECONFIGURE_PARAMS::forceIDR. */ bool Reconfigure(const NV_ENC_RECONFIGURE_PARAMS *pReconfigureParams); /** * @brief This function is used to get the next available input buffer. * Applications must call this function to obtain a pointer to the next * input buffer. The application must copy the uncompressed data to the * input buffer and then call EncodeFrame() function to encode it. */ const NvEncInputFrame* GetNextInputFrame(); /** * @brief This function is used to encode a frame. * Applications must call EncodeFrame() function to encode the uncompressed * data, which has been copied to an input buffer obtained from the * GetNextInputFrame() function. */ virtual void EncodeFrame(std::vector> &vPacket, NV_ENC_PIC_PARAMS *pPicParams = nullptr); /** * @brief This function to flush the encoder queue. * The encoder might be queuing frames for B picture encoding or lookahead; * the application must call EndEncode() to get all the queued encoded frames * from the encoder. The application must call this function before destroying * an encoder session. */ virtual void EndEncode(std::vector> &vPacket); /** * @brief This function is used to query hardware encoder capabilities. * Applications can call this function to query capabilities like maximum encode * dimensions, support for lookahead or the ME-only mode etc. */ int GetCapabilityValue(GUID guidCodec, NV_ENC_CAPS capsToQuery); /** * @brief This function is used to get the current device on which encoder is running. */ void *GetDevice() const { return m_pDevice; } /** * @brief This function is used to get the current device type which encoder is running. */ NV_ENC_DEVICE_TYPE GetDeviceType() const { return m_eDeviceType; } /** * @brief This function is used to get the current encode width. * The encode width can be modified by Reconfigure() function. */ int GetEncodeWidth() const { return m_nWidth; } /** * @brief This function is used to get the current encode height. * The encode height can be modified by Reconfigure() function. */ int GetEncodeHeight() const { return m_nHeight; } /** * @brief This function is used to get the current frame size based on pixel format. */ int GetFrameSize() const; /** * @brief This function is used to initialize config parameters based on * given codec and preset guids. * The application can call this function to get the default configuration * for a certain preset. The application can either use these parameters * directly or override them with application-specific settings before * using them in CreateEncoder() function. */ void CreateDefaultEncoderParams(NV_ENC_INITIALIZE_PARAMS* pIntializeParams, GUID codecGuid, GUID presetGuid, NV_ENC_TUNING_INFO tuningInfo = NV_ENC_TUNING_INFO_UNDEFINED); /** * @brief This function is used to get the current initialization parameters, * which had been used to configure the encoder session. * The initialization parameters are modified if the application calls * Reconfigure() function. */ void GetInitializeParams(NV_ENC_INITIALIZE_PARAMS *pInitializeParams); /** * @brief This function is used to run motion estimation * This is used to run motion estimation on a a pair of frames. The * application must copy the reference frame data to the buffer obtained * by calling GetNextReferenceFrame(), and copy the input frame data to * the buffer obtained by calling GetNextInputFrame() before calling the * RunMotionEstimation() function. */ void RunMotionEstimation(std::vector &mvData); /** * @brief This function is used to get an available reference frame. * Application must call this function to get a pointer to reference buffer, * to be used in the subsequent RunMotionEstimation() function. */ const NvEncInputFrame* GetNextReferenceFrame(); /** * @brief This function is used to get sequence and picture parameter headers. * Application can call this function after encoder is initialized to get SPS and PPS * nalus for the current encoder instance. The sequence header data might change when * application calls Reconfigure() function. */ void GetSequenceParams(std::vector &seqParams); /** * @brief NvEncoder class virtual destructor. */ virtual ~NvEncoder(); public: /** * @brief This a static function to get chroma offsets for YUV planar formats. */ static void GetChromaSubPlaneOffsets(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t pitch, const uint32_t height, std::vector& chromaOffsets); /** * @brief This a static function to get the chroma plane pitch for YUV planar formats. */ static uint32_t GetChromaPitch(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t lumaPitch); /** * @brief This a static function to get the number of chroma planes for YUV planar formats. */ static uint32_t GetNumChromaPlanes(const NV_ENC_BUFFER_FORMAT bufferFormat); /** * @brief This a static function to get the chroma plane width in bytes for YUV planar formats. */ static uint32_t GetChromaWidthInBytes(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t lumaWidth); /** * @brief This a static function to get the chroma planes height in bytes for YUV planar formats. */ static uint32_t GetChromaHeight(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t lumaHeight); /** * @brief This a static function to get the width in bytes for the frame. * For YUV planar format this is the width in bytes of the luma plane. */ static uint32_t GetWidthInBytes(const NV_ENC_BUFFER_FORMAT bufferFormat, const uint32_t width); /** * @brief This function returns the number of allocated buffers. */ uint32_t GetEncoderBufferCount() const { return m_nEncoderBuffer; } protected: /** * @brief NvEncoder class constructor. * NvEncoder class constructor cannot be called directly by the application. */ NvEncoder(NV_ENC_DEVICE_TYPE eDeviceType, void *pDevice, uint32_t nWidth, uint32_t nHeight, NV_ENC_BUFFER_FORMAT eBufferFormat, uint32_t nOutputDelay, bool bMotionEstimationOnly, bool bOutputInVideoMemory = false, bool bDX12Encode = false, bool bUseIVFContainer = true); /** * @brief This function is used to check if hardware encoder is properly initialized. */ bool IsHWEncoderInitialized() const { return m_hEncoder != NULL && m_bEncoderInitialized; } /** * @brief This function is used to register CUDA, D3D or OpenGL input buffers with NvEncodeAPI. * This is non public function and is called by derived class for allocating * and registering input buffers. */ void RegisterInputResources(std::vector inputframes, NV_ENC_INPUT_RESOURCE_TYPE eResourceType, int width, int height, int pitch, NV_ENC_BUFFER_FORMAT bufferFormat, bool bReferenceFrame = false); /** * @brief This function is used to unregister resources which had been previously registered for encoding * using RegisterInputResources() function. */ void UnregisterInputResources(); /** * @brief This function is used to register CUDA, D3D or OpenGL input or output buffers with NvEncodeAPI. */ NV_ENC_REGISTERED_PTR RegisterResource(void *pBuffer, NV_ENC_INPUT_RESOURCE_TYPE eResourceType, int width, int height, int pitch, NV_ENC_BUFFER_FORMAT bufferFormat, NV_ENC_BUFFER_USAGE bufferUsage = NV_ENC_INPUT_IMAGE, NV_ENC_FENCE_POINT_D3D12* pInputFencePoint = NULL); /** * @brief This function returns maximum width used to open the encoder session. * All encode input buffers are allocated using maximum dimensions. */ uint32_t GetMaxEncodeWidth() const { return m_nMaxEncodeWidth; } /** * @brief This function returns maximum height used to open the encoder session. * All encode input buffers are allocated using maximum dimensions. */ uint32_t GetMaxEncodeHeight() const { return m_nMaxEncodeHeight; } /** * @brief This function returns the completion event. */ void* GetCompletionEvent(uint32_t eventIdx) { return (m_vpCompletionEvent.size() == m_nEncoderBuffer) ? m_vpCompletionEvent[eventIdx] : nullptr; } /** * @brief This function returns the current pixel format. */ NV_ENC_BUFFER_FORMAT GetPixelFormat() const { return m_eBufferFormat; } /** * @brief This function is used to submit the encode commands to the * NVENC hardware. */ NVENCSTATUS DoEncode(NV_ENC_INPUT_PTR inputBuffer, NV_ENC_OUTPUT_PTR outputBuffer, NV_ENC_PIC_PARAMS *pPicParams); /** * @brief This function is used to submit the encode commands to the * NVENC hardware for ME only mode. */ NVENCSTATUS DoMotionEstimation(NV_ENC_INPUT_PTR inputBuffer, NV_ENC_INPUT_PTR inputBufferForReference, NV_ENC_OUTPUT_PTR outputBuffer); /** * @brief This function is used to map the input buffers to NvEncodeAPI. */ void MapResources(uint32_t bfrIdx); /** * @brief This function is used to wait for completion of encode command. */ void WaitForCompletionEvent(int iEvent); /** * @brief This function is used to send EOS to HW encoder. */ void SendEOS(); private: /** * @brief This is a private function which is used to check if there is any buffering done by encoder. * The encoder generally buffers data to encode B frames or for lookahead * or pipelining. */ bool IsZeroDelay() { return m_nOutputDelay == 0; } /** * @brief This is a private function which is used to load the encode api shared library. */ void LoadNvEncApi(); /** * @brief This is a private function which is used to get the output packets * from the encoder HW. * This is called by DoEncode() function. If there is buffering enabled, * this may return without any output data. */ void GetEncodedPacket(std::vector &vOutputBuffer, std::vector> &vPacket, bool bOutputDelay); /** * @brief This is a private function which is used to initialize the bitstream buffers. * This is only used in the encoding mode. */ void InitializeBitstreamBuffer(); /** * @brief This is a private function which is used to destroy the bitstream buffers. * This is only used in the encoding mode. */ void DestroyBitstreamBuffer(); /** * @brief This is a private function which is used to initialize MV output buffers. * This is only used in ME-only Mode. */ void InitializeMVOutputBuffer(); /** * @brief This is a private function which is used to destroy MV output buffers. * This is only used in ME-only Mode. */ void DestroyMVOutputBuffer(); /** * @brief This is a private function which is used to destroy HW encoder. */ void DestroyHWEncoder(); /** * @brief This function is used to flush the encoder queue. */ void FlushEncoder(); private: /** * @brief This is a pure virtual function which is used to allocate input buffers. * The derived classes must implement this function. */ virtual void AllocateInputBuffers(int32_t numInputBuffers) = 0; /** * @brief This is a pure virtual function which is used to destroy input buffers. * The derived classes must implement this function. */ virtual void ReleaseInputBuffers() = 0; protected: bool m_bMotionEstimationOnly = false; bool m_bOutputInVideoMemory = false; bool m_bIsDX12Encode = false; void *m_hEncoder = nullptr; NV_ENCODE_API_FUNCTION_LIST m_nvenc; NV_ENC_INITIALIZE_PARAMS m_initializeParams = {}; std::vector m_vInputFrames; std::vector m_vRegisteredResources; std::vector m_vReferenceFrames; std::vector m_vRegisteredResourcesForReference; std::vector m_vMappedInputBuffers; std::vector m_vMappedRefBuffers; std::vector m_vpCompletionEvent; int32_t m_iToSend = 0; int32_t m_iGot = 0; int32_t m_nEncoderBuffer = 0; int32_t m_nOutputDelay = 0; IVFUtils m_IVFUtils; bool m_bWriteIVFFileHeader = true; bool m_bUseIVFContainer = true; private: uint32_t m_nWidth; uint32_t m_nHeight; NV_ENC_BUFFER_FORMAT m_eBufferFormat; void *m_pDevice; NV_ENC_DEVICE_TYPE m_eDeviceType; NV_ENC_CONFIG m_encodeConfig = {}; bool m_bEncoderInitialized = false; uint32_t m_nExtraOutputDelay = 3; // To ensure encode and graphics can work in parallel, m_nExtraOutputDelay should be set to at least 1 std::vector m_vBitstreamOutputBuffer; std::vector m_vMVDataOutputBuffer; uint32_t m_nMaxEncodeWidth = 0; uint32_t m_nMaxEncodeHeight = 0; void* m_hModule = nullptr; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/NvEncoderD3D11.cpp ================================================ /* * Copyright 2017-2022 NVIDIA Corporation. All rights reserved. * * Please refer to the NVIDIA end user license agreement (EULA) associated * with this source code for terms and conditions that govern your use of * this software. Any use, reproduction, disclosure, or distribution of * this software and related documentation outside the terms of the EULA * is strictly prohibited. * */ #ifndef _WIN32 #include #endif #include "NvEncoderD3D11.h" #ifndef MAKEFOURCC #define MAKEFOURCC(a,b,c,d) (((unsigned int)a) | (((unsigned int)b)<< 8) | (((unsigned int)c)<<16) | (((unsigned int)d)<<24) ) #endif DXGI_FORMAT GetD3D11Format(NV_ENC_BUFFER_FORMAT eBufferFormat) { switch (eBufferFormat) { case NV_ENC_BUFFER_FORMAT_NV12: return DXGI_FORMAT_NV12; case NV_ENC_BUFFER_FORMAT_ARGB: return DXGI_FORMAT_B8G8R8A8_UNORM; case NV_ENC_BUFFER_FORMAT_ABGR: return DXGI_FORMAT_R8G8B8A8_UNORM; case NV_ENC_BUFFER_FORMAT_ABGR10: return DXGI_FORMAT_R8G8B8A8_UNORM; case NV_ENC_BUFFER_FORMAT_YUV420_10BIT: return DXGI_FORMAT_P010; default: return DXGI_FORMAT_UNKNOWN; } } NvEncoderD3D11::NvEncoderD3D11(ID3D11Device* pD3D11Device, uint32_t nWidth, uint32_t nHeight, NV_ENC_BUFFER_FORMAT eBufferFormat, uint32_t nExtraOutputDelay, bool bMotionEstimationOnly, bool bOutputInVideoMemory) : NvEncoder(NV_ENC_DEVICE_TYPE_DIRECTX, pD3D11Device, nWidth, nHeight, eBufferFormat, nExtraOutputDelay, bMotionEstimationOnly, bOutputInVideoMemory) { if (!pD3D11Device) { NVENC_THROW_ERROR("Bad d3d11device ptr", NV_ENC_ERR_INVALID_PTR); return; } if (GetD3D11Format(GetPixelFormat()) == DXGI_FORMAT_UNKNOWN) { NVENC_THROW_ERROR("Unsupported Buffer format", NV_ENC_ERR_INVALID_PARAM); } if (!m_hEncoder) { NVENC_THROW_ERROR("Encoder Initialization failed", NV_ENC_ERR_INVALID_DEVICE); } m_pD3D11Device = pD3D11Device; m_pD3D11Device->AddRef(); m_pD3D11Device->GetImmediateContext(&m_pD3D11DeviceContext); } NvEncoderD3D11::~NvEncoderD3D11() { ReleaseD3D11Resources(); } void NvEncoderD3D11::AllocateInputBuffers(int32_t numInputBuffers) { if (!IsHWEncoderInitialized()) { NVENC_THROW_ERROR("Encoder intialization failed", NV_ENC_ERR_ENCODER_NOT_INITIALIZED); } // for MEOnly mode we need to allocate seperate set of buffers for reference frame int numCount = m_bMotionEstimationOnly ? 2 : 1; for (int count = 0; count < numCount; count++) { std::vector inputFrames; for (int i = 0; i < numInputBuffers; i++) { ID3D11Texture2D *pInputTextures = NULL; D3D11_TEXTURE2D_DESC desc; ZeroMemory(&desc, sizeof(D3D11_TEXTURE2D_DESC)); desc.Width = GetMaxEncodeWidth(); desc.Height = GetMaxEncodeHeight(); desc.MipLevels = 1; desc.ArraySize = 1; desc.Format = GetD3D11Format(GetPixelFormat()); desc.SampleDesc.Count = 1; desc.Usage = D3D11_USAGE_DEFAULT; desc.BindFlags = D3D11_BIND_RENDER_TARGET; desc.CPUAccessFlags = 0; if (m_pD3D11Device->CreateTexture2D(&desc, NULL, &pInputTextures) != S_OK) { NVENC_THROW_ERROR("Failed to create d3d11textures", NV_ENC_ERR_OUT_OF_MEMORY); } inputFrames.push_back(pInputTextures); } RegisterInputResources(inputFrames, NV_ENC_INPUT_RESOURCE_TYPE_DIRECTX, GetMaxEncodeWidth(), GetMaxEncodeHeight(), 0, GetPixelFormat(), count == 1 ? true : false); } } void NvEncoderD3D11::ReleaseInputBuffers() { ReleaseD3D11Resources(); } void NvEncoderD3D11::ReleaseD3D11Resources() { if (!m_hEncoder) { return; } UnregisterInputResources(); for (uint32_t i = 0; i < m_vInputFrames.size(); ++i) { if (m_vInputFrames[i].inputPtr) { reinterpret_cast(m_vInputFrames[i].inputPtr)->Release(); } } m_vInputFrames.clear(); for (uint32_t i = 0; i < m_vReferenceFrames.size(); ++i) { if (m_vReferenceFrames[i].inputPtr) { reinterpret_cast(m_vReferenceFrames[i].inputPtr)->Release(); } } m_vReferenceFrames.clear(); if (m_pD3D11DeviceContext) { m_pD3D11DeviceContext->Release(); m_pD3D11DeviceContext = nullptr; } if (m_pD3D11Device) { m_pD3D11Device->Release(); m_pD3D11Device = nullptr; } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/NvEncoderD3D11.h ================================================ /* * Copyright 2017-2022 NVIDIA Corporation. All rights reserved. * * Please refer to the NVIDIA end user license agreement (EULA) associated * with this source code for terms and conditions that govern your use of * this software. Any use, reproduction, disclosure, or distribution of * this software and related documentation outside the terms of the EULA * is strictly prohibited. * */ #pragma once #include #include #include #include #include #include "NvEncoder.h" class NvEncoderD3D11 : public NvEncoder { public: NvEncoderD3D11(ID3D11Device* pD3D11Device, uint32_t nWidth, uint32_t nHeight, NV_ENC_BUFFER_FORMAT eBufferFormat, uint32_t nExtraOutputDelay = 3, bool bMotionEstimationOnly = false, bool bOPInVideoMemory = false); virtual ~NvEncoderD3D11(); protected: /** * @brief This function is used to release the input buffers allocated for encoding. * This function is an override of virtual function NvEncoder::ReleaseInputBuffers(). */ virtual void ReleaseInputBuffers() override; private: /** * @brief This function is used to allocate input buffers for encoding. * This function is an override of virtual function NvEncoder::AllocateInputBuffers(). * This function creates ID3D11Texture2D textures which is used to accept input data. * To obtain handle to input buffers application must call NvEncoder::GetNextInputFrame() */ virtual void AllocateInputBuffers(int32_t numInputBuffers) override; private: /** * @brief This is a private function to release ID3D11Texture2D textures used for encoding. */ void ReleaseD3D11Resources(); protected: ID3D11Device *m_pD3D11Device = nullptr; private: ID3D11DeviceContext* m_pD3D11DeviceContext = nullptr; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/OvrDirectModeComponent.cpp ================================================ #include "OvrDirectModeComponent.h" OvrDirectModeComponent::OvrDirectModeComponent( std::shared_ptr pD3DRender, std::shared_ptr poseHistory ) : m_pD3DRender(pD3DRender) , m_poseHistory(poseHistory) , m_submitLayer(0) { } void OvrDirectModeComponent::SetEncoder(std::shared_ptr pEncoder) { m_pEncoder = pEncoder; } /** Specific to Oculus compositor support, textures supplied must be created using this method. */ void OvrDirectModeComponent::CreateSwapTextureSet( uint32_t unPid, const SwapTextureSetDesc_t* pSwapTextureSetDesc, SwapTextureSet_t* pOutSwapTextureSet ) { Debug( "OvrDirectModeComponent::CreateSwapTextureSet pid=%d Format=%d %dx%d SampleCount=%d", unPid, pSwapTextureSetDesc->nFormat, pSwapTextureSetDesc->nWidth, pSwapTextureSetDesc->nHeight, pSwapTextureSetDesc->nSampleCount ); // HRESULT hr = D3D11CreateDevice(pAdapter, D3D_DRIVER_TYPE_HARDWARE, NULL, creationFlags, NULL, // 0, D3D11_SDK_VERSION, &pDevice, &eFeatureLevel, &pContext); D3D11_TEXTURE2D_DESC SharedTextureDesc = {}; DXGI_FORMAT format = (DXGI_FORMAT)pSwapTextureSetDesc->nFormat; SharedTextureDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET; if (format == DXGI_FORMAT_R32G8X24_TYPELESS || format == DXGI_FORMAT_R32_TYPELESS) { SharedTextureDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL; } SharedTextureDesc.ArraySize = 1; SharedTextureDesc.MipLevels = 1; SharedTextureDesc.SampleDesc.Count = pSwapTextureSetDesc->nSampleCount == 0 ? 1 : pSwapTextureSetDesc->nSampleCount; SharedTextureDesc.SampleDesc.Quality = 0; SharedTextureDesc.Usage = D3D11_USAGE_DEFAULT; SharedTextureDesc.Format = format; // Some(or all?) applications request larger texture than we specified in // GetRecommendedRenderTargetSize. But, we must create textures in requested size to prevent // cropped output. And then we must shrink texture to H.264 movie size. SharedTextureDesc.Width = pSwapTextureSetDesc->nWidth; SharedTextureDesc.Height = pSwapTextureSetDesc->nHeight; // SharedTextureDesc.MiscFlags = D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX | // D3D11_RESOURCE_MISC_SHARED_NTHANDLE; SharedTextureDesc.MiscFlags = D3D11_RESOURCE_MISC_SHARED; ProcessResource* processResource = new ProcessResource(); processResource->pid = unPid; for (int i = 0; i < 3; i++) { HRESULT hr = m_pD3DRender->GetDevice()->CreateTexture2D( &SharedTextureDesc, NULL, &processResource->textures[i] ); // LogDriver("texture%d %p res:%d %s", i, texture[i], hr, GetDxErrorStr(hr).c_str()); if (FAILED(hr)) { Error("CreateSwapTextureSet CreateTexture2D %p %ls", hr, GetErrorStr(hr).c_str()); delete processResource; break; } IDXGIResource* pResource; hr = processResource->textures[i]->QueryInterface( __uuidof(IDXGIResource), (void**)&pResource ); if (FAILED(hr)) { Error("CreateSwapTextureSet QueryInterface %p %ls", hr, GetErrorStr(hr).c_str()); delete processResource; break; } // LogDriver("QueryInterface %p res:%d %s", pResource, hr, GetDxErrorStr(hr).c_str()); hr = pResource->GetSharedHandle(&processResource->sharedHandles[i]); if (FAILED(hr)) { Error("CreateSwapTextureSet GetSharedHandle %p %ls", hr, GetErrorStr(hr).c_str()); delete processResource; pResource->Release(); break; } // LogDriver("GetSharedHandle %p res:%d %s", processResource->sharedHandles[i], hr, // GetDxErrorStr(hr).c_str()); m_handleMap.insert( std::make_pair(processResource->sharedHandles[i], std::make_pair(processResource, i)) ); pOutSwapTextureSet->rSharedTextureHandles[i] = (vr::SharedTextureHandle_t)processResource->sharedHandles[i]; pResource->Release(); Debug("Created Texture %d %p", i, processResource->sharedHandles[i]); } // m_processMap.insert(std::pair(unPid, processResource)); } /** Used to textures created using CreateSwapTextureSet. Only one of the set's handles needs to be * used to destroy the entire set. */ void OvrDirectModeComponent::DestroySwapTextureSet(vr::SharedTextureHandle_t sharedTextureHandle) { Debug("OvrDirectModeComponent::DestroySwapTextureSet %p", sharedTextureHandle); auto it = m_handleMap.find((HANDLE)sharedTextureHandle); if (it != m_handleMap.end()) { // Release all reference (a bit forcible) ProcessResource* p = it->second.first; m_handleMap.erase(p->sharedHandles[0]); m_handleMap.erase(p->sharedHandles[1]); m_handleMap.erase(p->sharedHandles[2]); delete p; } else { Debug("Requested to destroy not managing texture. handle:%p", sharedTextureHandle); } } /** Used to purge all texture sets for a given process. */ void OvrDirectModeComponent::DestroyAllSwapTextureSets(uint32_t unPid) { Debug("OvrDirectModeComponent::DestroyAllSwapTextureSets pid=%d", unPid); for (auto it = m_handleMap.begin(); it != m_handleMap.end();) { if (it->second.first->pid == unPid) { if (it->second.second == 0) { delete it->second.first; } m_handleMap.erase(it++); } else { ++it; } } } /** After Present returns, calls this to get the next index to use for rendering. */ void OvrDirectModeComponent::GetNextSwapTextureSetIndex( vr::SharedTextureHandle_t sharedTextureHandles[2], uint32_t (*pIndices)[2] ) { Debug("OvrDirectModeComponent::GetNextSwapTextureSetIndex"); (*pIndices)[0]++; (*pIndices)[0] %= 3; (*pIndices)[1]++; (*pIndices)[1] %= 3; } /** Call once per layer to draw for this frame. One shared texture handle per eye. Textures must * be created using CreateSwapTextureSet and should be alternated per frame. Call Present once all * layers have been submitted. */ void OvrDirectModeComponent::SubmitLayer(const SubmitLayerPerEye_t (&perEye)[2]) { Debug("OvrDirectModeComponent::SubmitLayer"); m_presentMutex.lock(); // mHmdPose is the same pose for both eyes, getting the eye view pose // requires some records keeping, unfortunately (m_eyeToHead) auto pPose = &perEye[0].mHmdPose; if (m_submitLayer == 0) { // Detect FrameIndex of submitted frame by pPose. // This is important part to achieve smooth headtracking. // We search for history of TrackingInfo and find the TrackingInfo which have nearest matrix // value. auto pose = m_poseHistory->GetBestPoseMatch(*pPose); if (pose) { // found the frameIndex m_prevTargetTimestampNs = m_targetTimestampNs; m_targetTimestampNs = pose->targetTimestampNs; m_prevFramePoseRotation = m_framePoseRotation; m_framePoseRotation.x = pose->motion.pose.orientation.x; m_framePoseRotation.y = pose->motion.pose.orientation.y; m_framePoseRotation.z = pose->motion.pose.orientation.z; m_framePoseRotation.w = pose->motion.pose.orientation.w; } else { m_targetTimestampNs = 0; m_framePoseRotation = HmdQuaternion_Init(0.0, 0.0, 0.0, 0.0); } } if (m_submitLayer < MAX_LAYERS) { m_submitLayers[m_submitLayer][0] = perEye[0]; m_submitLayers[m_submitLayer][1] = perEye[1]; m_submitLayer++; } else { Warn("Too many layers submitted!"); } // CopyTexture(); m_presentMutex.unlock(); } /** Submits queued layers for display. */ void OvrDirectModeComponent::Present(vr::SharedTextureHandle_t syncTexture) { Debug("OvrDirectModeComponent::Present"); m_presentMutex.lock(); ReportPresent(m_targetTimestampNs, 0); bool useMutex = true; IDXGIKeyedMutex* pKeyedMutex = NULL; uint32_t layerCount = m_submitLayer; m_submitLayer = 0; if (m_prevTargetTimestampNs == m_targetTimestampNs) { Debug("Discard duplicated frame. FrameIndex=%llu (Ignoring)", m_targetTimestampNs); // return; } ID3D11Texture2D* pSyncTexture = m_pD3DRender->GetSharedTexture((HANDLE)syncTexture); if (!pSyncTexture) { Warn("[VDispDvr] SyncTexture is NULL!"); m_presentMutex.unlock(); return; } if (useMutex) { // Access to shared texture must be wrapped in AcquireSync/ReleaseSync // to ensure the compositor has finished rendering to it before it gets used. // This enforces scheduling of work on the gpu between processes. if (SUCCEEDED(pSyncTexture->QueryInterface(__uuidof(IDXGIKeyedMutex), (void**)&pKeyedMutex) )) { // TODO: Reasonable timeout and timeout handling HRESULT hr = pKeyedMutex->AcquireSync(0, 10); if (hr != S_OK) { Debug( "[VDispDvr] ACQUIRESYNC FAILED!!! hr=%d %p %ls", hr, hr, GetErrorStr(hr).c_str() ); pKeyedMutex->Release(); m_presentMutex.unlock(); return; } } } CopyTexture(layerCount); if (useMutex) { if (pKeyedMutex) { pKeyedMutex->ReleaseSync(0); pKeyedMutex->Release(); } } ReportComposed(m_targetTimestampNs, 0); if (m_pEncoder) { m_pEncoder->NewFrameReady(); } m_presentMutex.unlock(); } void OvrDirectModeComponent::PostPresent() { Debug("OvrDirectModeComponent::PostPresent"); WaitForVSync(); } void OvrDirectModeComponent::CopyTexture(uint32_t layerCount) { uint64_t presentationTime = GetTimestampUs(); ID3D11Texture2D* pTexture[MAX_LAYERS][2]; ComPtr Texture[MAX_LAYERS][2]; vr::VRTextureBounds_t bounds[MAX_LAYERS][2]; vr::HmdMatrix34_t poses[MAX_LAYERS]; for (uint32_t i = 0; i < layerCount; i++) { // Find left eye texture. HANDLE leftEyeTexture = (HANDLE)m_submitLayers[i][0].hTexture; auto it = m_handleMap.find(leftEyeTexture); if (it == m_handleMap.end()) { // Ignore this layer. Debug( "Submitted texture is not found on HandleMap. eye=right layer=%d/%d Texture " "Handle=%p", i, layerCount, leftEyeTexture ); } else { Texture[i][0] = it->second.first->textures[it->second.second]; D3D11_TEXTURE2D_DESC desc; Texture[i][0]->GetDesc(&desc); // Find right eye texture. HANDLE rightEyeTexture = (HANDLE)m_submitLayers[i][1].hTexture; it = m_handleMap.find(rightEyeTexture); if (it == m_handleMap.end()) { // Ignore this layer Debug( "Submitted texture is not found on HandleMap. eye=left layer=%d/%d Texture " "Handle=%p", i, layerCount, rightEyeTexture ); Texture[i][0].Reset(); } else { Texture[i][1] = it->second.first->textures[it->second.second]; } } pTexture[i][0] = Texture[i][0].Get(); pTexture[i][1] = Texture[i][1].Get(); bounds[i][0] = m_submitLayers[i][0].bounds; bounds[i][1] = m_submitLayers[i][1].bounds; poses[i] = m_submitLayers[i][0].mHmdPose; } // This can go away, but is useful to see it as a separate packet on the gpu in traces. m_pD3DRender->GetContext()->Flush(); if (m_pEncoder) { // Wait for the encoder to be ready. This is important because the encoder thread // blocks on transmit which uses our shared d3d context (which is not thread safe). m_pEncoder->WaitForEncode(); std::string debugText; uint64_t submitFrameIndex = m_targetTimestampNs; // Copy entire texture to staging so we can read the pixels to send to remote device. m_pEncoder->CopyToStaging( pTexture, bounds, poses, layerCount, false, presentationTime, submitFrameIndex, "", debugText ); m_pD3DRender->GetContext()->Flush(); } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/OvrDirectModeComponent.h ================================================ #pragma once #include "CEncoder.h" #include "alvr_server/PoseHistory.h" #include "alvr_server/Utils.h" #include "alvr_server/openvr_driver_wrap.h" #include "alvr_server/Settings.h" #include class OvrDirectModeComponent : public vr::IVRDriverDirectModeComponent { public: OvrDirectModeComponent( std::shared_ptr pD3DRender, std::shared_ptr poseHistory ); void SetEncoder(std::shared_ptr pEncoder); /** Specific to Oculus compositor support, textures supplied must be created using this method. */ virtual void CreateSwapTextureSet( uint32_t unPid, const SwapTextureSetDesc_t* pSwapTextureSetDesc, SwapTextureSet_t* pOutSwapTextureSet ); /** Used to textures created using CreateSwapTextureSet. Only one of the set's handles needs to * be used to destroy the entire set. */ virtual void DestroySwapTextureSet(vr::SharedTextureHandle_t sharedTextureHandle); /** Used to purge all texture sets for a given process. */ virtual void DestroyAllSwapTextureSets(uint32_t unPid); /** After Present returns, calls this to get the next index to use for rendering. */ virtual void GetNextSwapTextureSetIndex( vr::SharedTextureHandle_t sharedTextureHandles[2], uint32_t (*pIndices)[2] ); /** Call once per layer to draw for this frame. One shared texture handle per eye. Textures * must be created using CreateSwapTextureSet and should be alternated per frame. Call Present * once all layers have been submitted. */ virtual void SubmitLayer(const SubmitLayerPerEye_t (&perEye)[2]); /** Submits queued layers for display. */ virtual void Present(vr::SharedTextureHandle_t syncTexture); /** Called after Present to allow driver to take more time until vsync after they've * successfully acquired the sync texture in Present.*/ virtual void PostPresent(); void CopyTexture(uint32_t layerCount); private: std::shared_ptr m_pD3DRender; std::shared_ptr m_pEncoder; std::shared_ptr m_poseHistory; // Resource for each process struct ProcessResource { ComPtr textures[3]; HANDLE sharedHandles[3]; uint32_t pid; }; std::map> m_handleMap; static const int MAX_LAYERS = 10; int m_submitLayer; SubmitLayerPerEye_t m_submitLayers[MAX_LAYERS][2]; vr::HmdQuaternion_t m_prevFramePoseRotation; vr::HmdQuaternion_t m_framePoseRotation; uint64_t m_targetTimestampNs; uint64_t m_prevTargetTimestampNs; std::mutex m_presentMutex; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoder.cpp ================================================ #include "VideoEncoder.h" ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoder.h ================================================ #pragma once #include "NvEncoderD3D11.h" #include "shared/d3drender.h" #include #include class VideoEncoder { public: virtual void Initialize() = 0; virtual void Shutdown() = 0; virtual void Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ) = 0; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderAMF.cpp ================================================ #include "VideoEncoderAMF.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #define AMF_THROW_IF(expr) \ { \ AMF_RESULT res = expr; \ if (res != AMF_OK) { \ throw MakeException("AMF Error %d. %s", res, L#expr); \ } \ } const wchar_t* VideoEncoderAMF::START_TIME_PROPERTY = L"StartTimeProperty"; const wchar_t* VideoEncoderAMF::FRAME_INDEX_PROPERTY = L"FrameIndexProperty"; AMFPipe::AMFPipe(amf::AMFComponentPtr src, AMFDataReceiver receiver) : m_amfComponentSrc(src) , m_receiver(receiver) { } AMFPipe::~AMFPipe() { Debug("AMFPipe::~AMFPipe() m_amfComponentSrc->Drain\n"); m_amfComponentSrc->Drain(); } void AMFPipe::doPassthrough(bool hasQueryTimeout, uint32_t timerResolution) { amf::AMFDataPtr data = nullptr; if (hasQueryTimeout) { AMF_RESULT res = m_amfComponentSrc->QueryOutput(&data); if (res == AMF_OK && data) { m_receiver(data); } else { Debug("Failed to get AMF component data. Last status: %d.\n", res); } } else { uint16_t timeout = 1000; // 1s timeout AMF_RESULT res = m_amfComponentSrc->QueryOutput(&data); timeBeginPeriod(timerResolution); while (!data && --timeout != 0) { amf_sleep(1); res = m_amfComponentSrc->QueryOutput(&data); } timeEndPeriod(timerResolution); if (data) { m_receiver(data); } else { Debug("Failed to get AMF component data. Last status: %d.\n", res); } } } AMFSolidPipe::AMFSolidPipe(amf::AMFComponentPtr src, amf::AMFComponentPtr dst) : AMFPipe(src, std::bind(&AMFSolidPipe::Passthrough, this, std::placeholders::_1)) , m_amfComponentDst(dst) { } void AMFSolidPipe::Passthrough(AMFDataPtr data) { auto res = m_amfComponentDst->SubmitInput(data); switch (res) { case AMF_OK: break; case AMF_INPUT_FULL: Debug("m_amfComponentDst->SubmitInput returns AMF_INPUT_FULL.\n"); break; case AMF_NEED_MORE_INPUT: Debug("m_amfComponentDst->SubmitInput returns AMF_NEED_MORE_INPUT.\n"); break; default: Debug("m_amfComponentDst->SubmitInput returns code %d.\n", res); break; } } AMFPipeline::AMFPipeline() : m_pipes() { TIMECAPS tc; m_timerResolution = timeGetDevCaps(&tc, sizeof(tc)) == TIMERR_NOERROR ? tc.wPeriodMin : 1; } AMFPipeline::~AMFPipeline() { for (auto& pipe : m_pipes) { delete pipe; } } void AMFPipeline::Connect(AMFPipePtr pipe) { m_pipes.emplace_back(pipe); } void AMFPipeline::Run(bool hasQueryTimeout) { for (auto& pipe : m_pipes) { pipe->doPassthrough(hasQueryTimeout, m_timerResolution); } } // // VideoEncoderAMF // VideoEncoderAMF::VideoEncoderAMF(std::shared_ptr d3dRender, int width, int height) : m_d3dRender(d3dRender) , m_codec(Settings::Instance().m_codec) , m_refreshRate(Settings::Instance().m_refreshRate) , m_renderWidth(width) , m_renderHeight(height) , m_bitrateInMBits(30) , m_surfaceFormat(amf::AMF_SURFACE_RGBA) , m_use10bit(Settings::Instance().m_use10bitEncoder) , m_hasQueryTimeout(false) { if (Settings::Instance().m_enableHdr) { // Bypass preprocessor and converters for HDR, since it will already be YUV m_surfaceFormat = m_use10bit ? amf::AMF_SURFACE_P010 : amf::AMF_SURFACE_NV12; } } VideoEncoderAMF::~VideoEncoderAMF() { } amf::AMFComponentPtr VideoEncoderAMF::MakeEncoder( amf::AMF_SURFACE_FORMAT inputFormat, int width, int height, int codec, int refreshRate, int bitrateInMbits ) { const wchar_t* pCodec; amf_int32 frameRateIn = refreshRate; amf_int64 bitRateIn = bitrateInMbits * 1'000'000L; // in bits switch (codec) { case ALVR_CODEC_H264: if (m_use10bit) { throw MakeException("H.264 10-bit encoding is not supported"); } pCodec = AMFVideoEncoderVCE_AVC; break; case ALVR_CODEC_HEVC: pCodec = AMFVideoEncoder_HEVC; break; case ALVR_CODEC_AV1: pCodec = AMFVideoEncoder_AV1; break; default: throw MakeException("Unsupported video encoding %d", codec); } amf::AMFComponentPtr amfEncoder; // Create encoder component. AMF_THROW_IF(g_AMFFactory.GetFactory()->CreateComponent(m_amfContext, pCodec, &amfEncoder)); switch (codec) { case ALVR_CODEC_H264: { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_USAGE, AMF_VIDEO_ENCODER_USAGE_ULTRA_LOW_LATENCY); switch (Settings::Instance().m_h264Profile) { case ALVR_H264_PROFILE_BASELINE: amfEncoder->SetProperty(AMF_VIDEO_ENCODER_PROFILE, AMF_VIDEO_ENCODER_PROFILE_BASELINE); break; case ALVR_H264_PROFILE_MAIN: amfEncoder->SetProperty(AMF_VIDEO_ENCODER_PROFILE, AMF_VIDEO_ENCODER_PROFILE_MAIN); break; default: case ALVR_H264_PROFILE_HIGH: amfEncoder->SetProperty(AMF_VIDEO_ENCODER_PROFILE, AMF_VIDEO_ENCODER_PROFILE_HIGH); break; } amfEncoder->SetProperty(AMF_VIDEO_ENCODER_PROFILE_LEVEL, 42); switch (Settings::Instance().m_rateControlMode) { case ALVR_CBR: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_CBR ); // Required for CBR to work correctly amfEncoder->SetProperty( AMF_VIDEO_ENCODER_FILLER_DATA_ENABLE, Settings::Instance().m_fillerData ); break; case ALVR_VBR: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR ); break; } switch (Settings::Instance().m_entropyCoding) { case ALVR_CABAC: amfEncoder->SetProperty(AMF_VIDEO_ENCODER_CABAC_ENABLE, AMF_VIDEO_ENCODER_CABAC); break; case ALVR_CAVLC: amfEncoder->SetProperty(AMF_VIDEO_ENCODER_CABAC_ENABLE, AMF_VIDEO_ENCODER_CALV); break; } amfEncoder->SetProperty(AMF_VIDEO_ENCODER_TARGET_BITRATE, bitRateIn); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_PEAK_BITRATE, bitRateIn); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_FRAMESIZE, ::AMFConstructSize(width, height)); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_FRAMERATE, ::AMFConstructRate(frameRateIn, 1)); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_B_PIC_PATTERN, 0); switch (Settings::Instance().m_encoderQualityPreset) { case ALVR_QUALITY: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_QUALITY_PRESET, AMF_VIDEO_ENCODER_QUALITY_PRESET_QUALITY ); break; case ALVR_BALANCED: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_QUALITY_PRESET, AMF_VIDEO_ENCODER_QUALITY_PRESET_BALANCED ); break; case ALVR_SPEED: default: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_QUALITY_PRESET, AMF_VIDEO_ENCODER_QUALITY_PRESET_SPEED ); break; } amf::AMFCapsPtr caps; if (amfEncoder->GetCaps(&caps) == AMF_OK) { caps->GetProperty(AMF_VIDEO_ENCODER_CAP_PRE_ANALYSIS, &m_hasPreAnalysis); caps->GetProperty(AMF_VIDEO_ENCODER_CAPS_QUERY_TIMEOUT_SUPPORT, &m_hasQueryTimeout); } if (Settings::Instance().m_enableAmfPreAnalysis) { if (!Settings::Instance().m_useAmfPreproc || Settings::Instance().m_use10bitEncoder) { Warn("Pre-analysis could not be enabled because \"Use preproc\" is not enabled or " "\"Reduce color banding\" is enabled."); } else if (m_hasPreAnalysis) { Warn("Enabling h264 pre-analysis. You may experience higher latency when this is " "enabled."); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_PRE_ANALYSIS_ENABLE, Settings::Instance().m_enableAmfPreAnalysis ); } else { Warn("Pre-analysis could not be enabled because your GPU does not support it for " "h264 encoding."); } } // Enable Full Range amfEncoder->SetProperty(AMF_VIDEO_ENCODER_FULL_RANGE_COLOR, true); // Use BT.2020 for HDR encoding, BT.709 otherwise. if (Settings::Instance().m_enableHdr) { // Also specify the input formats for HDR to prevent incorrect color conversions amfEncoder->SetProperty( AMF_VIDEO_ENCODER_INPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_INPUT_TRANSFER_CHARACTERISTIC, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 ); // sRGB amfEncoder->SetProperty( AMF_VIDEO_ENCODER_INPUT_COLOR_PRIMARIES, AMF_COLOR_PRIMARIES_BT2020 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_OUTPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_OUTPUT_TRANSFER_CHARACTERISTIC, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 ); // sRGB amfEncoder->SetProperty( AMF_VIDEO_ENCODER_OUTPUT_COLOR_PRIMARIES, AMF_COLOR_PRIMARIES_BT2020 ); } else { amfEncoder->SetProperty( AMF_VIDEO_ENCODER_OUTPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_OUTPUT_TRANSFER_CHARACTERISTIC, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 ); // sRGB amfEncoder->SetProperty( AMF_VIDEO_ENCODER_OUTPUT_COLOR_PRIMARIES, AMF_COLOR_PRIMARIES_BT709 ); } // No noticable performance difference and should improve subjective quality by allocating // more bits to smooth areas amfEncoder->SetProperty(AMF_VIDEO_ENCODER_ENABLE_VBAQ, Settings::Instance().m_enableVbaq); // May impact performance but improves quality in high-motion areas amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HIGH_MOTION_QUALITY_BOOST_ENABLE, Settings::Instance().m_enableAmfHmqb ); // Turns Off IDR/I Frames amfEncoder->SetProperty(AMF_VIDEO_ENCODER_IDR_PERIOD, 0); // Disable AUD to produce the same stream format as VideoEncoderNVENC. // FIXME: This option doesn't work in 22.10.3, but works in versions prior 22.5.1 amfEncoder->SetProperty(AMF_VIDEO_ENCODER_INSERT_AUD, false); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_VBV_BUFFER_SIZE, bitRateIn / frameRateIn * 1.1); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_MAX_NUM_REFRAMES, 0); if (m_hasQueryTimeout) { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_QUERY_TIMEOUT, 1000); // 1s timeout } } case ALVR_CODEC_HEVC: { amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_USAGE, AMF_VIDEO_ENCODER_HEVC_USAGE_ULTRA_LOW_LATENCY ); switch (Settings::Instance().m_rateControlMode) { case ALVR_CBR: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CBR ); // Required for CBR to work correctly amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_FILLER_DATA_ENABLE, Settings::Instance().m_fillerData ); break; case ALVR_VBR: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR ); break; } amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_TARGET_BITRATE, bitRateIn); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_PEAK_BITRATE, bitRateIn); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_FRAMESIZE, ::AMFConstructSize(width, height) ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_FRAMERATE, ::AMFConstructRate(frameRateIn, 1) ); switch (Settings::Instance().m_encoderQualityPreset) { case ALVR_QUALITY: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET, AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_QUALITY ); break; case ALVR_BALANCED: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET, AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_BALANCED ); break; case ALVR_SPEED: default: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET, AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_SPEED ); break; } if (m_use10bit) { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_COLOR_BIT_DEPTH, AMF_COLOR_BIT_DEPTH_10); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_PROFILE, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10 ); } else { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_COLOR_BIT_DEPTH, AMF_COLOR_BIT_DEPTH_8); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_PROFILE, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN ); } amf::AMFCapsPtr caps; if (amfEncoder->GetCaps(&caps) == AMF_OK) { caps->GetProperty(AMF_VIDEO_ENCODER_HEVC_CAP_PRE_ANALYSIS, &m_hasPreAnalysis); caps->GetProperty( AMF_VIDEO_ENCODER_CAPS_HEVC_QUERY_TIMEOUT_SUPPORT, &m_hasQueryTimeout ); } if (Settings::Instance().m_enableAmfPreAnalysis) { if (!Settings::Instance().m_useAmfPreproc || Settings::Instance().m_use10bitEncoder) { Warn("Pre-analysis could not be enabled because \"Use preproc\" is not enabled or " "\"Reduce color banding\" is enabled."); } else if (m_hasPreAnalysis) { Warn("Enabling HEVC pre-analysis. You may experience higher latency when this is " "enabled."); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_PRE_ANALYSIS_ENABLE, Settings::Instance().m_enableAmfPreAnalysis ); } else { Warn("Pre-analysis could not be enabled because your GPU does not support it for " "HEVC encoding."); } } // Enable Full Range amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE, AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE_FULL ); // Use BT.2020 for HDR encoding, BT.709 otherwise. if (Settings::Instance().m_enableHdr) { // Also specify the input formats for HDR to prevent incorrect color conversions amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_INPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_INPUT_TRANSFER_CHARACTERISTIC, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 ); // sRGB amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_INPUT_COLOR_PRIMARIES, AMF_COLOR_PRIMARIES_BT2020 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_OUTPUT_TRANSFER_CHARACTERISTIC, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 ); // sRGB amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PRIMARIES, AMF_COLOR_PRIMARIES_BT2020 ); } else { amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709 ); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_OUTPUT_TRANSFER_CHARACTERISTIC, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 ); // sRGB amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PRIMARIES, AMF_COLOR_PRIMARIES_BT709 ); } // No noticable performance difference and should improve subjective quality by allocating // more bits to smooth areas amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_ENABLE_VBAQ, Settings::Instance().m_enableVbaq ); // May impact performance but improves quality in high-motion areas amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_HIGH_MOTION_QUALITY_BOOST_ENABLE, Settings::Instance().m_enableAmfHmqb ); // Turns Off IDR/I Frames amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_NUM_GOPS_PER_IDR, 0); // Set infinite GOP length amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_GOP_SIZE, 0); // Disable AUD to produce the same stream format as VideoEncoderNVENC. // FIXME: This option doesn't work in 22.10.3, but works in versions prior 22.5.1 amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_INSERT_AUD, false); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_HEVC_VBV_BUFFER_SIZE, bitRateIn / frameRateIn * 1.1 ); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_MAX_NUM_REFRAMES, 0); if (m_hasQueryTimeout) { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_HEVC_QUERY_TIMEOUT, 1000); // 1s timeout } } case ALVR_CODEC_AV1: { amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_USAGE, AMF_VIDEO_ENCODER_AV1_USAGE_ULTRA_LOW_LATENCY ); switch (Settings::Instance().m_rateControlMode) { case ALVR_CBR: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_CBR ); // Required for CBR to work correctly amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_FILLER_DATA, Settings::Instance().m_fillerData ); break; case ALVR_VBR: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR ); break; } amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_TARGET_BITRATE, bitRateIn); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_PEAK_BITRATE, bitRateIn); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_FRAMESIZE, ::AMFConstructSize(width, height)); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_FRAMERATE, ::AMFConstructRate(frameRateIn, 1) ); switch (Settings::Instance().m_encoderQualityPreset) { case ALVR_QUALITY: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET, AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_QUALITY ); break; case ALVR_BALANCED: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET, AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_BALANCED ); break; case ALVR_SPEED: default: amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET, AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_SPEED ); break; } if (m_use10bit) { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_COLOR_BIT_DEPTH, AMF_COLOR_BIT_DEPTH_10); // There's no separate profile for 10-bit for AV1 (as of AMF v1.4.33). Assumedly MAIN // works fine for both. amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_PROFILE, AMF_VIDEO_ENCODER_AV1_PROFILE_MAIN ); } else { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_COLOR_BIT_DEPTH, AMF_COLOR_BIT_DEPTH_8); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_PROFILE, AMF_VIDEO_ENCODER_AV1_PROFILE_MAIN ); } // There is no VBAQ option for AV1. Instead it has CAQ (Content adaptive quantization) if (Settings::Instance().m_enableVbaq) { amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_AQ_MODE, AMF_VIDEO_ENCODER_AV1_AQ_MODE_CAQ ); } else { amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_AQ_MODE, AMF_VIDEO_ENCODER_AV1_AQ_MODE_NONE ); } amf::AMFCapsPtr caps; if (amfEncoder->GetCaps(&caps) == AMF_OK) { caps->GetProperty(AMF_VIDEO_ENCODER_AV1_CAP_PRE_ANALYSIS, &m_hasPreAnalysis); } if (Settings::Instance().m_enableAmfPreAnalysis) { if (!Settings::Instance().m_useAmfPreproc || Settings::Instance().m_use10bitEncoder) { Warn("Pre-analysis could not be enabled because \"Use preproc\" is not enabled or " "\"Reduce color banding\" is enabled."); } else if (m_hasPreAnalysis) { Warn("Enabling AV1 pre-analysis. You may experience higher latency when this is " "enabled."); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_PRE_ANALYSIS_ENABLE, Settings::Instance().m_enableAmfPreAnalysis ); } else { Warn("Pre-analysis could not be enabled because your GPU does not support it for " "AV1 encoding."); } } // Enable Full Range amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PROFILE, AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG ); // May impact performance but improves quality in high-motion areas amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_HIGH_MOTION_QUALITY_BOOST, Settings::Instance().m_enableAmfHmqb ); // Set infinite GOP length amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_GOP_SIZE, 0); amfEncoder->SetProperty( AMF_VIDEO_ENCODER_AV1_VBV_BUFFER_SIZE, bitRateIn / frameRateIn * 1.2 ); amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_MAX_NUM_REFRAMES, 0); // AV1 assumed always has support for query timeout. m_hasQueryTimeout = true; if (m_hasQueryTimeout) { amfEncoder->SetProperty(AMF_VIDEO_ENCODER_AV1_QUERY_TIMEOUT, 1000); // 1s timeout } } } Debug("Configured %s.\n", pCodec); AMF_THROW_IF(amfEncoder->Init(inputFormat, width, height)); Debug("Initialized %s.\n", pCodec); return amfEncoder; } amf::AMFComponentPtr VideoEncoderAMF::MakeConverter( amf::AMF_SURFACE_FORMAT inputFormat, int width, int height, amf::AMF_SURFACE_FORMAT outputFormat ) { amf::AMFComponentPtr amfConverter; AMF_THROW_IF( g_AMFFactory.GetFactory()->CreateComponent(m_amfContext, AMFVideoConverter, &amfConverter) ); AMF_THROW_IF(amfConverter->SetProperty(AMF_VIDEO_CONVERTER_MEMORY_TYPE, amf::AMF_MEMORY_DX11)); AMF_THROW_IF(amfConverter->SetProperty(AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, outputFormat)); AMF_THROW_IF(amfConverter->SetProperty( AMF_VIDEO_CONVERTER_OUTPUT_SIZE, ::AMFConstructSize(width, height) )); AMF_THROW_IF(amfConverter->Init(inputFormat, width, height)); Debug("Initialized %s.\n", AMFVideoConverter); return amfConverter; } amf::AMFComponentPtr VideoEncoderAMF::MakePreprocessor(amf::AMF_SURFACE_FORMAT inputFormat, int width, int height) { amf::AMFComponentPtr amfPreprocessor; AMF_THROW_IF( g_AMFFactory.GetFactory()->CreateComponent(m_amfContext, AMFPreProcessing, &amfPreprocessor) ); AMF_THROW_IF(amfPreprocessor->SetProperty(AMF_PP_ENGINE_TYPE, amf::AMF_MEMORY_DX11)); AMF_THROW_IF(amfPreprocessor->SetProperty( AMF_PP_ADAPTIVE_FILTER_STRENGTH, Settings::Instance().m_amfPreProcSigma )); AMF_THROW_IF(amfPreprocessor->SetProperty( AMF_PP_ADAPTIVE_FILTER_SENSITIVITY, Settings::Instance().m_amfPreProcTor )); AMF_THROW_IF(amfPreprocessor->Init(inputFormat, width, height)); Debug("Initialized %s.\n", AMFPreProcessing); return amfPreprocessor; } void VideoEncoderAMF::Initialize() { Debug("Initializing VideoEncoderAMF.\n"); AMF_THROW_IF(g_AMFFactory.Init()); AMF_THROW_IF(g_AMFFactory.GetFactory()->CreateContext(&m_amfContext)); AMF_THROW_IF(m_amfContext->InitDX11(m_d3dRender->GetDevice())); amf::AMF_SURFACE_FORMAT inFormat = m_surfaceFormat; if (Settings::Instance().m_enableHdr) { // Bypass preprocessor and converters for HDR, since it will already be YUV ; } else if (m_use10bit) { inFormat = amf::AMF_SURFACE_R10G10B10A2; m_amfComponents.emplace_back( MakeConverter(m_surfaceFormat, m_renderWidth, m_renderHeight, inFormat) ); } else { if (Settings::Instance().m_useAmfPreproc) { inFormat = amf::AMF_SURFACE_NV12; m_amfComponents.emplace_back( MakeConverter(m_surfaceFormat, m_renderWidth, m_renderHeight, inFormat) ); m_amfComponents.emplace_back(MakePreprocessor(inFormat, m_renderWidth, m_renderHeight)); } } m_amfComponents.emplace_back(MakeEncoder( inFormat, m_renderWidth, m_renderHeight, m_codec, m_refreshRate, m_bitrateInMBits )); m_pipeline = new AMFPipeline(); for (int i = 0; i < m_amfComponents.size() - 1; i++) { m_pipeline->Connect(new AMFSolidPipe(m_amfComponents[i], m_amfComponents[i + 1])); } m_pipeline->Connect(new AMFPipe( m_amfComponents.back(), std::bind(&VideoEncoderAMF::Receive, this, std::placeholders::_1) )); Debug("Successfully initialized VideoEncoderAMF.\n"); } void VideoEncoderAMF::Shutdown() { Debug("Shutting down VideoEncoderAMF.\n"); delete m_pipeline; for (auto& component : m_amfComponents) { component->Release(); delete component; } m_amfContext->Terminate(); m_amfContext = NULL; g_AMFFactory.Terminate(); if (fpOut) { fpOut.close(); } Debug("Successfully shutdown VideoEncoderAMF.\n"); } void VideoEncoderAMF::Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ) { amf::AMFSurfacePtr surface; // Surface is cached by AMF. auto params = GetDynamicEncoderParams(); if (params.updated) { amf_int64 bitRateIn = params.bitrate_bps / params.framerate * m_refreshRate; // in bps const amf_int64 maxRate = 1'000'000'000; if (bitRateIn > maxRate) { // Don't warn if we're within 1% (10 mbps) of the max bitrate because that case gets // triggered by rounding errors with the max supported rate if (bitRateIn > maxRate + maxRate / 100) { Warn("Set bitrate over max supported by AMF, clamping to 1000mbps"); } bitRateIn = maxRate; } if (m_codec == ALVR_CODEC_H264) { m_amfComponents.back()->SetProperty(AMF_VIDEO_ENCODER_TARGET_BITRATE, bitRateIn); m_amfComponents.back()->SetProperty(AMF_VIDEO_ENCODER_PEAK_BITRATE, bitRateIn); m_amfComponents.back()->SetProperty( AMF_VIDEO_ENCODER_VBV_BUFFER_SIZE, bitRateIn / m_refreshRate * 1.1 ); } else { m_amfComponents.back()->SetProperty(AMF_VIDEO_ENCODER_HEVC_TARGET_BITRATE, bitRateIn); m_amfComponents.back()->SetProperty(AMF_VIDEO_ENCODER_HEVC_PEAK_BITRATE, bitRateIn); m_amfComponents.back()->SetProperty( AMF_VIDEO_ENCODER_HEVC_VBV_BUFFER_SIZE, bitRateIn / m_refreshRate * 1.1 ); } if (Settings::Instance().m_amdBitrateCorruptionFix) { RequestIDR(); } } AMF_THROW_IF(m_amfContext->AllocSurface( amf::AMF_MEMORY_DX11, m_surfaceFormat, m_renderWidth, m_renderHeight, &surface )); ID3D11Texture2D* textureDX11 = (ID3D11Texture2D*)surface->GetPlaneAt(0)->GetNative( ); // no reference counting - do not Release() m_d3dRender->GetContext()->CopyResource(textureDX11, pTexture); amf_pts start_time = amf_high_precision_clock(); surface->SetProperty(START_TIME_PROPERTY, start_time); surface->SetProperty(FRAME_INDEX_PROPERTY, targetTimestampNs); ApplyFrameProperties(surface, insertIDR); m_amfComponents.front()->SubmitInput(surface); m_pipeline->Run(m_hasQueryTimeout); } void VideoEncoderAMF::Receive(AMFDataPtr data) { amf_pts current_time = amf_high_precision_clock(); amf_pts start_time = 0; uint64_t targetTimestampNs; data->GetProperty(START_TIME_PROPERTY, &start_time); data->GetProperty(FRAME_INDEX_PROPERTY, &targetTimestampNs); amf::AMFBufferPtr buffer(data); // query for buffer interface char* p = reinterpret_cast(buffer->GetNative()); int length = static_cast(buffer->GetSize()); if (fpOut) { fpOut.write(p, length); } uint64_t type; bool isIdr; if (m_codec == ALVR_CODEC_H264) { data->GetProperty(AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE, &type); isIdr = type == AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_IDR; } else { data->GetProperty(AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE, &type); isIdr = type == AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE_IDR; } ParseFrameNals(m_codec, reinterpret_cast(p), length, targetTimestampNs, isIdr); } void VideoEncoderAMF::ApplyFrameProperties(const amf::AMFSurfacePtr& surface, bool insertIDR) { switch (m_codec) { case ALVR_CODEC_H264: // FIXME: This option doesn't work in drivers 22.3.1 - 22.5.1, but works in 22.10.3 surface->SetProperty(AMF_VIDEO_ENCODER_INSERT_AUD, false); if (insertIDR) { Debug("Inserting IDR frame for H.264.\n"); surface->SetProperty(AMF_VIDEO_ENCODER_INSERT_SPS, true); surface->SetProperty(AMF_VIDEO_ENCODER_INSERT_PPS, true); surface->SetProperty( AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR ); } break; case ALVR_CODEC_HEVC: // FIXME: This option works with 22.10.3, but may not work with older drivers surface->SetProperty(AMF_VIDEO_ENCODER_HEVC_INSERT_AUD, false); if (insertIDR) { Debug("Inserting IDR frame for H.265.\n"); // Insert VPS,SPS,PPS // These options don't work properly on older AMD driver (Radeon Software 17.7, AMF // Runtime 1.4.4) Fixed in 18.9.2 & 1.4.9 surface->SetProperty(AMF_VIDEO_ENCODER_HEVC_INSERT_HEADER, true); surface->SetProperty( AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR ); } break; case ALVR_CODEC_AV1: if (insertIDR) { Debug("Inserting IDR frame for AV1.\n"); surface->SetProperty(AMF_VIDEO_ENCODER_AV1_FORCE_INSERT_SEQUENCE_HEADER, true); surface->SetProperty( AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_KEY ); } break; default: throw MakeException("Invalid video codec"); } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderAMF.h ================================================ #pragma once #include "VideoEncoder.h" #include "../../shared/amf/public/common/AMFFactory.h" #include "../../shared/amf/public/common/AMFSTL.h" #include "../../shared/amf/public/common/Thread.h" #include "../../shared/amf/public/include/components/PreProcessing.h" #include "../../shared/amf/public/include/components/VideoConverter.h" #include "../../shared/amf/public/include/components/VideoEncoderAV1.h" #include "../../shared/amf/public/include/components/VideoEncoderHEVC.h" #include "../../shared/amf/public/include/components/VideoEncoderVCE.h" typedef amf::AMFData* AMFDataPtr; typedef std::function AMFDataReceiver; class AMFPipeline; class AMFPipe { public: AMFPipe(amf::AMFComponentPtr src, AMFDataReceiver receiver); virtual ~AMFPipe(); void doPassthrough(bool hasQueryTimeout, uint32_t timerResolution); protected: amf::AMFComponentPtr m_amfComponentSrc; AMFDataReceiver m_receiver; }; typedef AMFPipe* AMFPipePtr; class AMFSolidPipe : public AMFPipe { public: AMFSolidPipe(amf::AMFComponentPtr src, amf::AMFComponentPtr dst); protected: void Passthrough(AMFDataPtr); amf::AMFComponentPtr m_amfComponentDst; }; class AMFPipeline { public: AMFPipeline(); ~AMFPipeline(); void Connect(AMFPipePtr pipe); void Run(bool hasQueryTimeout); protected: uint32_t m_timerResolution; std::vector m_pipes; }; typedef AMFPipeline* AMFPipelinePtr; // Video encoder for AMD VCE and VCN. class VideoEncoderAMF : public VideoEncoder { public: VideoEncoderAMF(std::shared_ptr pD3DRender, int width, int height); ~VideoEncoderAMF(); void Initialize(); void Shutdown(); void Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ); void Receive(AMFDataPtr data); private: static const wchar_t* START_TIME_PROPERTY; static const wchar_t* FRAME_INDEX_PROPERTY; amf::AMFComponentPtr MakeConverter( amf::AMF_SURFACE_FORMAT inputFormat, int width, int height, amf::AMF_SURFACE_FORMAT outputFormat ); amf::AMFComponentPtr MakePreprocessor(amf::AMF_SURFACE_FORMAT inputFormat, int width, int height); amf::AMFComponentPtr MakeEncoder( amf::AMF_SURFACE_FORMAT inputFormat, int width, int height, int codec, int refreshRate, int bitrateInMbits ); amf::AMFContextPtr m_amfContext; AMFPipelinePtr m_pipeline; std::vector m_amfComponents; std::ofstream fpOut; std::shared_ptr m_d3dRender; bool m_use10bit; amf::AMF_SURFACE_FORMAT m_surfaceFormat; int m_codec; int m_refreshRate; int m_renderWidth; int m_renderHeight; int m_bitrateInMBits; bool m_hasQueryTimeout; bool m_hasPreAnalysis; void ApplyFrameProperties(const amf::AMFSurfacePtr& surface, bool insertIDR); }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderNVENC.cpp ================================================ #include "VideoEncoderNVENC.h" #include "NvCodecUtils.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "alvr_server/Utils.h" VideoEncoderNVENC::VideoEncoderNVENC(std::shared_ptr pD3DRender, int width, int height) : m_pD3DRender(pD3DRender) , m_codec(Settings::Instance().m_codec) , m_refreshRate(Settings::Instance().m_refreshRate) , m_renderWidth(width) , m_renderHeight(height) , m_bitrateInMBits(30) { } VideoEncoderNVENC::~VideoEncoderNVENC() { } void VideoEncoderNVENC::Initialize() { // // Initialize Encoder // NV_ENC_BUFFER_FORMAT format = Settings::Instance().m_enableHdr ? NV_ENC_BUFFER_FORMAT_NV12 : NV_ENC_BUFFER_FORMAT_ABGR; if (Settings::Instance().m_use10bitEncoder) { format = Settings::Instance().m_enableHdr ? NV_ENC_BUFFER_FORMAT_YUV420_10BIT : NV_ENC_BUFFER_FORMAT_ABGR10; } Debug( "Initializing CNvEncoder. Width=%d Height=%d Format=%d\n", m_renderWidth, m_renderHeight, format ); try { m_NvNecoder = std::make_shared( m_pD3DRender->GetDevice(), m_renderWidth, m_renderHeight, format, 0 ); } catch (NVENCException e) { throw MakeException( "NvEnc NvEncoderD3D11 failed. Code=%d %hs\n", e.getErrorCode(), e.what() ); } NV_ENC_INITIALIZE_PARAMS initializeParams = { NV_ENC_INITIALIZE_PARAMS_VER }; NV_ENC_CONFIG encodeConfig = { NV_ENC_CONFIG_VER }; initializeParams.encodeConfig = &encodeConfig; FillEncodeConfig( initializeParams, m_refreshRate, m_renderWidth, m_renderHeight, m_bitrateInMBits * 1'000'000L ); try { m_NvNecoder->CreateEncoder(&initializeParams); } catch (NVENCException e) { if (e.getErrorCode() == NV_ENC_ERR_INVALID_PARAM) { throw MakeException( "This GPU does not support H.265 encoding. (NvEncoderCuda NV_ENC_ERR_INVALID_PARAM)" ); } throw MakeException("NvEnc CreateEncoder failed. Code=%d %hs", e.getErrorCode(), e.what()); } Debug("CNvEncoder is successfully initialized.\n"); } void VideoEncoderNVENC::Shutdown() { std::vector> vPacket; if (m_NvNecoder) m_NvNecoder->EndEncode(vPacket); for (std::vector& packet : vPacket) { if (fpOut) { fpOut.write(reinterpret_cast(packet.data()), packet.size()); } } if (m_NvNecoder) { m_NvNecoder->DestroyEncoder(); m_NvNecoder.reset(); } Debug("CNvEncoder::Shutdown\n"); if (fpOut) { fpOut.close(); } } void VideoEncoderNVENC::Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ) { auto params = GetDynamicEncoderParams(); if (params.updated) { m_bitrateInMBits = params.bitrate_bps / 1'000'000; NV_ENC_INITIALIZE_PARAMS initializeParams = { NV_ENC_INITIALIZE_PARAMS_VER }; NV_ENC_CONFIG encodeConfig = { NV_ENC_CONFIG_VER }; initializeParams.encodeConfig = &encodeConfig; FillEncodeConfig( initializeParams, params.framerate, m_renderWidth, m_renderHeight, m_bitrateInMBits * 1'000'000L ); NV_ENC_RECONFIGURE_PARAMS reconfigureParams = { NV_ENC_RECONFIGURE_PARAMS_VER }; reconfigureParams.reInitEncodeParams = initializeParams; m_NvNecoder->Reconfigure(&reconfigureParams); } std::vector> vPacket; const NvEncInputFrame* encoderInputFrame = m_NvNecoder->GetNextInputFrame(); ID3D11Texture2D* pInputTexture = reinterpret_cast(encoderInputFrame->inputPtr); m_pD3DRender->GetContext()->CopyResource(pInputTexture, pTexture); NV_ENC_PIC_PARAMS picParams = {}; if (insertIDR) { Debug("Inserting IDR frame.\n"); picParams.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR; } m_NvNecoder->EncodeFrame(vPacket, &picParams); for (std::vector& packet : vPacket) { uint8_t* buf = packet.data(); int len = (int)packet.size(); // NVENC's AV1 encoding includes a bunch of IVF wrapping, // so we need to strip it down to just the OBUs if (m_codec == ALVR_CODEC_AV1) { const uint8_t ivf_magic[4] = { 0x44, 0x4B, 0x49, 0x46 }; if (len >= 4 && !memcmp(buf, ivf_magic, 4)) { buf += 32; len -= 32; } if (len <= 12) { continue; } buf += 12; // skip past the IVF packet size header thing len -= 12; } if (len <= 0) { continue; } if (fpOut) { fpOut.write(reinterpret_cast(buf), len); } ParseFrameNals(m_codec, buf, len, targetTimestampNs, insertIDR); } } void VideoEncoderNVENC::FillEncodeConfig( NV_ENC_INITIALIZE_PARAMS& initializeParams, int refreshRate, int renderWidth, int renderHeight, uint64_t bitrate_bps ) { auto& encodeConfig = *initializeParams.encodeConfig; GUID encoderGUID; switch (m_codec) { case ALVR_CODEC_H264: encoderGUID = NV_ENC_CODEC_H264_GUID; break; case ALVR_CODEC_HEVC: encoderGUID = NV_ENC_CODEC_HEVC_GUID; break; case ALVR_CODEC_AV1: encoderGUID = NV_ENC_CODEC_AV1_GUID; break; } GUID qualityPreset; // See recommended NVENC settings for low-latency encoding. // https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-video-encoder-api-prog-guide/#recommended-nvenc-settings switch (Settings::Instance().m_nvencQualityPreset) { case 7: qualityPreset = NV_ENC_PRESET_P7_GUID; break; case 6: qualityPreset = NV_ENC_PRESET_P6_GUID; break; case 5: qualityPreset = NV_ENC_PRESET_P5_GUID; break; case 4: qualityPreset = NV_ENC_PRESET_P4_GUID; break; case 3: qualityPreset = NV_ENC_PRESET_P3_GUID; break; case 2: qualityPreset = NV_ENC_PRESET_P2_GUID; break; case 1: default: qualityPreset = NV_ENC_PRESET_P1_GUID; break; } NV_ENC_TUNING_INFO tuningPreset = static_cast(Settings::Instance().m_nvencTuningPreset); m_NvNecoder->CreateDefaultEncoderParams( &initializeParams, encoderGUID, qualityPreset, tuningPreset ); initializeParams.encodeWidth = initializeParams.darWidth = renderWidth; initializeParams.encodeHeight = initializeParams.darHeight = renderHeight; initializeParams.frameRateNum = refreshRate; initializeParams.frameRateDen = 1; if (Settings::Instance().m_nvencRefreshRate != -1) { initializeParams.frameRateNum = Settings::Instance().m_nvencRefreshRate; } initializeParams.enableWeightedPrediction = Settings::Instance().m_nvencEnableWeightedPrediction; // 16 is recommended when using reference frame invalidation. But it has caused bad visual // quality. Now, use 0 (use default). uint32_t maxNumRefFrames = 0; uint32_t gopLength = NVENC_INFINITE_GOPLENGTH; if (Settings::Instance().m_nvencMaxNumRefFrames != -1) { maxNumRefFrames = Settings::Instance().m_nvencMaxNumRefFrames; } if (Settings::Instance().m_nvencGopLength != -1) { gopLength = Settings::Instance().m_nvencGopLength; } switch (m_codec) { case ALVR_CODEC_H264: { auto& config = encodeConfig.encodeCodecConfig.h264Config; config.repeatSPSPPS = 1; config.enableIntraRefresh = Settings::Instance().m_nvencEnableIntraRefresh; if (Settings::Instance().m_nvencIntraRefreshPeriod != -1) { config.intraRefreshPeriod = Settings::Instance().m_nvencIntraRefreshPeriod; } if (Settings::Instance().m_nvencIntraRefreshCount != -1) { config.intraRefreshCnt = Settings::Instance().m_nvencIntraRefreshCount; } switch (Settings::Instance().m_entropyCoding) { case ALVR_CABAC: config.entropyCodingMode = NV_ENC_H264_ENTROPY_CODING_MODE_CABAC; break; case ALVR_CAVLC: config.entropyCodingMode = NV_ENC_H264_ENTROPY_CODING_MODE_CAVLC; break; } config.maxNumRefFrames = maxNumRefFrames; config.idrPeriod = gopLength; if (Settings::Instance().m_fillerData) { config.enableFillerDataInsertion = Settings::Instance().m_rateControlMode == ALVR_CBR; } config.h264VUIParameters.videoSignalTypePresentFlag = 1; config.h264VUIParameters.videoFormat = NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED; config.h264VUIParameters.videoFullRangeFlag = 1; config.h264VUIParameters.colourDescriptionPresentFlag = 1; if (Settings::Instance().m_enableHdr) { config.h264VUIParameters.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT2020; config.h264VUIParameters.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB; config.h264VUIParameters.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL; } else { config.h264VUIParameters.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709; config.h264VUIParameters.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB; config.h264VUIParameters.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT709; } } break; case ALVR_CODEC_HEVC: { auto& config = encodeConfig.encodeCodecConfig.hevcConfig; config.repeatSPSPPS = 1; config.enableIntraRefresh = Settings::Instance().m_nvencEnableIntraRefresh; if (Settings::Instance().m_nvencIntraRefreshPeriod != -1) { config.intraRefreshPeriod = Settings::Instance().m_nvencIntraRefreshPeriod; } if (Settings::Instance().m_nvencIntraRefreshCount != -1) { config.intraRefreshCnt = Settings::Instance().m_nvencIntraRefreshCount; } config.maxNumRefFramesInDPB = maxNumRefFrames; config.idrPeriod = gopLength; if (Settings::Instance().m_use10bitEncoder) { encodeConfig.encodeCodecConfig.hevcConfig.pixelBitDepthMinus8 = 2; } if (Settings::Instance().m_fillerData) { config.enableFillerDataInsertion = Settings::Instance().m_rateControlMode == ALVR_CBR; } config.hevcVUIParameters.videoSignalTypePresentFlag = 1; config.hevcVUIParameters.videoFormat = NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED; config.hevcVUIParameters.videoFullRangeFlag = 1; config.hevcVUIParameters.colourDescriptionPresentFlag = 1; if (Settings::Instance().m_enableHdr) { config.hevcVUIParameters.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT2020; config.hevcVUIParameters.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB; config.hevcVUIParameters.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL; } else { config.hevcVUIParameters.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709; config.hevcVUIParameters.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB; config.hevcVUIParameters.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT709; } } break; case ALVR_CODEC_AV1: { auto& config = encodeConfig.encodeCodecConfig.av1Config; config.repeatSeqHdr = 1; config.enableIntraRefresh = Settings::Instance().m_nvencEnableIntraRefresh; if (Settings::Instance().m_nvencIntraRefreshPeriod != -1) { config.intraRefreshPeriod = Settings::Instance().m_nvencIntraRefreshPeriod; } if (Settings::Instance().m_nvencIntraRefreshCount != -1) { config.intraRefreshCnt = Settings::Instance().m_nvencIntraRefreshCount; } config.maxNumRefFramesInDPB = maxNumRefFrames; config.idrPeriod = gopLength; if (Settings::Instance().m_use10bitEncoder) { config.pixelBitDepthMinus8 = 2; } if (Settings::Instance().m_fillerData) { config.enableBitstreamPadding = Settings::Instance().m_rateControlMode == ALVR_CBR; } config.chromaFormatIDC = 1; // 4:2:0, 4:4:4 currently not supported config.colorRange = 1; if (Settings::Instance().m_enableHdr) { config.colorPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT2020; config.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB; config.matrixCoefficients = NV_ENC_VUI_MATRIX_COEFFS_BT2020_NCL; } else { config.colorPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709; config.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_SRGB; config.matrixCoefficients = NV_ENC_VUI_MATRIX_COEFFS_BT709; } } break; } // Disable automatic IDR insertion by NVENC. We need to manually insert IDR when packet is // dropped if don't use reference frame invalidation. encodeConfig.gopLength = gopLength; encodeConfig.frameIntervalP = 1; if (Settings::Instance().m_nvencPFrameStrategy != -1) { encodeConfig.frameIntervalP = Settings::Instance().m_nvencPFrameStrategy; } switch (Settings::Instance().m_rateControlMode) { case ALVR_CBR: encodeConfig.rcParams.rateControlMode = NV_ENC_PARAMS_RC_CBR; break; case ALVR_VBR: encodeConfig.rcParams.rateControlMode = NV_ENC_PARAMS_RC_VBR; break; } encodeConfig.rcParams.multiPass = static_cast(Settings::Instance().m_nvencMultiPass); encodeConfig.rcParams.lowDelayKeyFrameScale = 1; if (Settings::Instance().m_nvencLowDelayKeyFrameScale != -1) { encodeConfig.rcParams.lowDelayKeyFrameScale = Settings::Instance().m_nvencLowDelayKeyFrameScale; } uint32_t maxFrameSize = static_cast(bitrate_bps / refreshRate); Debug("VideoEncoderNVENC: maxFrameSize=%d bits\n", maxFrameSize); encodeConfig.rcParams.vbvBufferSize = maxFrameSize * 1.1; encodeConfig.rcParams.vbvInitialDelay = maxFrameSize * 1.1; encodeConfig.rcParams.maxBitRate = static_cast(bitrate_bps); encodeConfig.rcParams.averageBitRate = static_cast(bitrate_bps); if (Settings::Instance().m_nvencAdaptiveQuantizationMode == SpatialAQ) { encodeConfig.rcParams.enableAQ = 1; } else if (Settings::Instance().m_nvencAdaptiveQuantizationMode == TemporalAQ) { encodeConfig.rcParams.enableTemporalAQ = 1; } if (Settings::Instance().m_nvencRateControlMode != -1) { encodeConfig.rcParams.rateControlMode = (NV_ENC_PARAMS_RC_MODE)Settings::Instance().m_nvencRateControlMode; } if (Settings::Instance().m_nvencRcBufferSize != -1) { encodeConfig.rcParams.vbvBufferSize = Settings::Instance().m_nvencRcBufferSize; } if (Settings::Instance().m_nvencRcInitialDelay != -1) { encodeConfig.rcParams.vbvInitialDelay = Settings::Instance().m_nvencRcInitialDelay; } if (Settings::Instance().m_nvencRcMaxBitrate != -1) { encodeConfig.rcParams.maxBitRate = Settings::Instance().m_nvencRcMaxBitrate; } if (Settings::Instance().m_nvencRcAverageBitrate != -1) { encodeConfig.rcParams.averageBitRate = Settings::Instance().m_nvencRcAverageBitrate; } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderNVENC.h ================================================ #pragma once #include "NvEncoderD3D11.h" #include "VideoEncoder.h" #include "shared/d3drender.h" #include enum AdaptiveQuantizationMode { SpatialAQ = 1, TemporalAQ = 2 }; // Video encoder for NVIDIA NvEnc. class VideoEncoderNVENC : public VideoEncoder { public: VideoEncoderNVENC(std::shared_ptr pD3DRender, int width, int height); ~VideoEncoderNVENC(); void Initialize(); void Shutdown(); void Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ); private: void FillEncodeConfig( NV_ENC_INITIALIZE_PARAMS& initializeParams, int refreshRate, int renderWidth, int renderHeight, uint64_t bitrate_bps ); std::ofstream fpOut; std::shared_ptr m_NvNecoder; std::shared_ptr m_pD3DRender; int m_codec; int m_refreshRate; int m_renderWidth; int m_renderHeight; int m_bitrateInMBits; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderSW.cpp ================================================ #ifdef ALVR_GPL #include "VideoEncoderSW.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "alvr_server/Utils.h" #include #include #include #include VideoEncoderSW::VideoEncoderSW(std::shared_ptr d3dRender, int width, int height) : m_d3dRender(d3dRender) , m_codec(ALVR_CODEC_H264) , m_refreshRate(Settings::Instance().m_refreshRate) , m_renderWidth(width) , m_renderHeight(height) , m_bitrateInMBits(30) { // #ifdef ALVR_DEBUG_LOG // av_log_set_level(AV_LOG_DEBUG); // av_log_set_callback(LibVALog); // Debug("Set FFMPEG/LibAV to debug logging"); // #endif } VideoEncoderSW::~VideoEncoderSW() { } void VideoEncoderSW::LibVALog(void* v, int level, const char* data, va_list va) { const char* prefix = "[libav]: "; std::stringstream sstream; sstream << prefix << data; vprintf(sstream.str().c_str(), va); } void VideoEncoderSW::Initialize() { int err; Debug("Initializing VideoEncoderSW.\n"); const auto& settings = Settings::Instance(); // Query codec AVCodecID codecId = ToFFMPEGCodec(m_codec); if (!codecId) throw MakeException("Invalid requested codec %d", m_codec); const AVCodec* codec = avcodec_find_encoder(codecId); if (codec == NULL) throw MakeException("Could not find codec id %d", codecId); // Initialize CodecContext m_codecContext = avcodec_alloc_context3(codec); if (m_codecContext == NULL) throw MakeException("Failed to allocate encoder id %d", codecId); // Set codec settings AVDictionary* opt = NULL; av_dict_set(&opt, "preset", "ultrafast", 0); av_dict_set(&opt, "tune", "zerolatency", 0); switch (settings.m_h264Profile) { case ALVR_H264_PROFILE_BASELINE: m_codecContext->profile = FF_PROFILE_H264_BASELINE; break; case ALVR_H264_PROFILE_MAIN: m_codecContext->profile = FF_PROFILE_H264_MAIN; break; default: case ALVR_H264_PROFILE_HIGH: m_codecContext->profile = FF_PROFILE_H264_HIGH; break; } switch (settings.m_entropyCoding) { case ALVR_CABAC: av_dict_set(&opt, "coder", "ac", 0); break; case ALVR_CAVLC: av_dict_set(&opt, "coder", "vlc", 0); break; } m_codecContext->width = m_renderWidth; m_codecContext->height = m_renderHeight; m_codecContext->time_base = AVRational { 1, (int)(1e9) }; m_codecContext->framerate = AVRational { settings.m_refreshRate, 1 }; m_codecContext->sample_aspect_ratio = AVRational { 1, 1 }; m_codecContext->pix_fmt = settings.m_use10bitEncoder ? AV_PIX_FMT_YUV420P10 : AV_PIX_FMT_YUV420P; m_codecContext->color_range = AVCOL_RANGE_JPEG; if (settings.m_enableHdr) { m_codecContext->color_primaries = AVCOL_PRI_BT2020; m_codecContext->color_trc = AVCOL_TRC_GAMMA22; m_codecContext->colorspace = AVCOL_SPC_BT2020_NCL; } else { m_codecContext->color_primaries = AVCOL_PRI_BT709; m_codecContext->color_trc = AVCOL_TRC_GAMMA22; m_codecContext->colorspace = AVCOL_SPC_BT709; } m_codecContext->max_b_frames = 0; m_codecContext->gop_size = 0; m_codecContext->bit_rate = m_bitrateInMBits * 1'000'000L; m_codecContext->rc_buffer_size = m_codecContext->bit_rate / settings.m_refreshRate * 1.1; switch (settings.m_rateControlMode) { case ALVR_CBR: if (settings.m_fillerData) { av_dict_set(&opt, "nal-hrd", "cbr", 0); } break; case ALVR_VBR: av_dict_set(&opt, "nal-hrd", "vbr", 0); break; } m_codecContext->rc_max_rate = m_codecContext->bit_rate; m_codecContext->thread_count = settings.m_swThreadCount; if ((err = avcodec_open2(m_codecContext, codec, &opt))) throw MakeException("Cannot open video encoder codec: %d", err); // Config transfer/encode frames m_transferredFrame = av_frame_alloc(); m_transferredFrame->buf[0] = av_buffer_alloc(1); m_encoderFrame = av_frame_alloc(); m_encoderFrame->width = m_codecContext->width; m_encoderFrame->height = m_codecContext->height; m_encoderFrame->format = m_codecContext->pix_fmt; if ((err = av_frame_get_buffer(m_encoderFrame, 0))) throw MakeException("Error when allocating encoder frame: %d", err); Debug("Successfully initialized VideoEncoderSW"); } void VideoEncoderSW::Shutdown() { Debug("Shutting down VideoEncoderSW.\n"); av_frame_free(&m_transferredFrame); av_frame_free(&m_encoderFrame); avcodec_free_context(&m_codecContext); sws_freeContext(m_scalerContext); m_scalerContext = nullptr; Debug("Successfully shutdown VideoEncoderSW.\n"); } void VideoEncoderSW::Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ) { // Handle bitrate changes auto params = GetDynamicEncoderParams(); if (params.updated) { m_codecContext->bit_rate = params.bitrate_bps; m_codecContext->framerate = AVRational { (int)params.framerate, 1 }; m_codecContext->rc_buffer_size = m_codecContext->bit_rate / params.framerate * 1.1; m_codecContext->rc_max_rate = m_codecContext->bit_rate; } // Setup staging texture if not defined yet; we can only define it here as we now have the // texture's size if (!m_stagingTex) { HRESULT hr = SetupStagingTexture(pTexture); if (FAILED(hr)) { Error("Failed to create staging texture: %p %ls", hr, GetErrorStr(hr).c_str()); return; } Debug("Success in creating staging texture"); } // Copy texture and map it to memory /// SteamVR crashes if the swapchain textures are set to staging, which is needed to be read by /// the CPU. Unless there's another solution we have to copy the texture every time, which is /// gonna be another performance hit. HRESULT hr = CopyTexture(pTexture); if (FAILED(hr)) { Error("Failed to copy texture to staging: %p %ls", hr, GetErrorStr(hr).c_str()); return; } // Debug("Success in mapping staging texture"); AVPixelFormat inputFormat = AV_PIX_FMT_RGBA; if (Settings::Instance().m_enableHdr) { inputFormat = Settings::Instance().m_use10bitEncoder ? AV_PIX_FMT_YUV420P10 : AV_PIX_FMT_YUV420P; } // Setup software scaler if not defined yet; we can only define it here as we now have the // texture's size if (!m_scalerContext) { m_scalerContext = sws_getContext( m_stagingTexDesc.Width, m_stagingTexDesc.Height, inputFormat, m_codecContext->width, m_codecContext->height, m_codecContext->pix_fmt, SWS_BILINEAR, NULL, NULL, NULL ); if (!m_scalerContext) { Error("Couldn't initialize SWScaler."); m_d3dRender->GetContext()->Unmap(m_stagingTex.Get(), 0); return; } Debug("Successfully initialized SWScaler."); } // We got the texture, populate tansferredFrame with data m_transferredFrame->width = m_stagingTexDesc.Width; m_transferredFrame->height = m_stagingTexDesc.Height; m_transferredFrame->data[0] = (uint8_t*)m_stagingTexMap.pData; m_transferredFrame->linesize[0] = m_stagingTexMap.RowPitch; m_transferredFrame->format = inputFormat; m_transferredFrame->pts = targetTimestampNs; // Use SWScaler for scaling if (sws_scale( m_scalerContext, m_transferredFrame->data, m_transferredFrame->linesize, 0, m_transferredFrame->height, m_encoderFrame->data, m_encoderFrame->linesize ) == 0) { Error("SWScale failed."); m_d3dRender->GetContext()->Unmap(m_stagingTex.Get(), 0); return; } // Debug("SWScale succeeded."); // Send frame for encoding m_encoderFrame->pict_type = insertIDR ? AV_PICTURE_TYPE_I : AV_PICTURE_TYPE_NONE; m_encoderFrame->pts = targetTimestampNs; int err; if ((err = avcodec_send_frame(m_codecContext, m_encoderFrame)) < 0) { Error("Encoding frame failed: err code %d", err); m_d3dRender->GetContext()->Unmap(m_stagingTex.Get(), 0); return; } // Debug("Send frame succeeded."); // Retrieve frames from encoding and send them until buffer is emptied while (true) { AVPacket* packet = av_packet_alloc(); err = avcodec_receive_packet(m_codecContext, packet); if (err != 0) { av_packet_free(&packet); break; } // Send encoded frame to client bool isIdr = (packet->flags & AV_PKT_FLAG_KEY) != 0; ParseFrameNals(m_codec, packet->data, packet->size, packet->pts, isIdr); // Debug("Sent encoded packet to client"); av_packet_free(&packet); } if (err == AVERROR(EINVAL)) { Error("Received encoded frame failed: err code %d", err); } // Unmap the copied texture and delete it m_d3dRender->GetContext()->Unmap(m_stagingTex.Get(), 0); } HRESULT VideoEncoderSW::SetupStagingTexture(ID3D11Texture2D* pTexture) { D3D11_TEXTURE2D_DESC desc; pTexture->GetDesc(&desc); m_stagingTexDesc.Width = desc.Width; m_stagingTexDesc.Height = desc.Height; m_stagingTexDesc.MipLevels = desc.MipLevels; m_stagingTexDesc.ArraySize = desc.ArraySize; m_stagingTexDesc.Format = desc.Format; m_stagingTexDesc.SampleDesc = desc.SampleDesc; m_stagingTexDesc.Usage = D3D11_USAGE_STAGING; m_stagingTexDesc.BindFlags = 0; m_stagingTexDesc.CPUAccessFlags = D3D11_CPU_ACCESS_READ; m_stagingTexDesc.MiscFlags = 0; return m_d3dRender->GetDevice()->CreateTexture2D(&m_stagingTexDesc, nullptr, &m_stagingTex); } HRESULT VideoEncoderSW::CopyTexture(ID3D11Texture2D* pTexture) { m_d3dRender->GetContext()->CopyResource(m_stagingTex.Get(), pTexture); return m_d3dRender->GetContext()->Map( m_stagingTex.Get(), 0, D3D11_MAP_READ, 0, &m_stagingTexMap ); } AVCodecID VideoEncoderSW::ToFFMPEGCodec(ALVR_CODEC codec) { switch (codec) { case ALVR_CODEC_H264: return AV_CODEC_ID_H264; case ALVR_CODEC_HEVC: return AV_CODEC_ID_HEVC; case ALVR_CODEC_AV1: Warn("AV1 is not supported. Using HEVC instead."); return AV_CODEC_ID_HEVC; default: return AV_CODEC_ID_NONE; } } #endif // ALVR_GPL ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderSW.h ================================================ #ifdef ALVR_GPL #pragma once #include #include "ALVR-common/packet_types.h" #include "VideoEncoder.h" #include "shared/d3drender.h" extern "C" { #include #include #include #include } using Microsoft::WRL::ComPtr; // Software video encoder using FFMPEG class VideoEncoderSW : public VideoEncoder { public: VideoEncoderSW(std::shared_ptr pD3DRender, int width, int height); ~VideoEncoderSW(); void Initialize(); void Shutdown(); static void LibVALog(void*, int level, const char* data, va_list va); AVCodecID ToFFMPEGCodec(ALVR_CODEC codec); void Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ); HRESULT SetupStagingTexture(ID3D11Texture2D* pTexture); HRESULT CopyTexture(ID3D11Texture2D* pTexture); private: std::shared_ptr m_d3dRender; AVCodecContext* m_codecContext; AVFrame *m_transferredFrame, *m_encoderFrame; SwsContext* m_scalerContext = nullptr; ComPtr m_stagingTex; D3D11_TEXTURE2D_DESC m_stagingTexDesc; D3D11_MAPPED_SUBRESOURCE m_stagingTexMap; ALVR_CODEC m_codec; int m_refreshRate; int m_renderWidth; int m_renderHeight; int m_bitrateInMBits; }; #endif // ALVR_GPL ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderVPL.cpp ================================================ #include "VideoEncoderVPL.h" #include "alvr_server/Logger.h" #include "alvr_server/Settings.h" #include "alvr_server/Utils.h" #define VPLVERSION(major, minor) (major << 16 | minor) #define MAJOR_API_VERSION_REQUIRED 2 #define MINOR_API_VERSION_REQUIRED 9 #define WAIT_100_MILLISECONDS 100 #define ALIGN16(value) (((value + 15) >> 4) << 4) #define ERROR_THROW(msg, ...) \ { \ Error("VPL: " msg "\n", __VA_ARGS__); \ throw MakeException("VPL: " msg, __VA_ARGS__); \ } #define VPL_LOG(fn, msg, ...) fn("VPL: " msg "\n", __VA_ARGS__) #define VPL_DEBUG(msg, ...) VPL_LOG(Debug, msg, __VA_ARGS__) #define VPL_WARN(msg, ...) VPL_LOG(Warn, msg, __VA_ARGS__) #define VPL_INFO(msg, ...) VPL_LOG(Info, msg, __VA_ARGS__) #define VERIFY(expr, msg) \ if (!(expr)) \ ERROR_THROW("%s. %s", msg, #expr) #define VPL_VERIFY(expr) \ { \ mfxStatus res = expr; \ if (res != MFX_ERR_NONE) \ ERROR_THROW("\"%s\" failed with %d", #expr, res); \ } VideoEncoderVPL::VideoEncoderVPL(std::shared_ptr pD3DRender, int width, int height) : m_pD3DRender(pD3DRender) , m_renderWidth(width) , m_renderHeight(height) , m_bitrateInMBits(30) { VPL_DEBUG("constructed"); } VideoEncoderVPL::~VideoEncoderVPL() { VPL_DEBUG("destructed"); } void VideoEncoderVPL::Initialize() { VPL_DEBUG("initialize"); ChooseParams(); InitTransferTex(); InitVpl(); InitVplEncode(); // Prepare output bitstream m_vplBitstream.MaxLength = m_renderWidth * m_renderHeight * 8; m_vplBitstream.Data = (mfxU8*)calloc(m_vplBitstream.MaxLength, sizeof(mfxU8)); } void VideoEncoderVPL::Shutdown() { VPL_DEBUG("shutdown"); MFXVideoENCODE_Close(m_vplSession); MFXClose(m_vplSession); if (m_vplBitstream.Data) free(m_vplBitstream.Data); if (m_vplLoader) MFXUnload(m_vplLoader); } void VideoEncoderVPL::Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ) { // VPL_DEBUG("transmit"); auto dynParams = GetDynamicEncoderParams(); if (dynParams.updated) { m_vplEncodeParams.mfx.TargetKbps = dynParams.bitrate_bps / 1000; MFXVideoENCODE_Reset(m_vplSession, &m_vplEncodeParams); } auto encSurface = VplImportTexture(pTexture); mfxEncodeCtrl encodeCtrl = {}; encodeCtrl.FrameType = insertIDR ? MFX_FRAMETYPE_IDR : 0; mfxStatus sts = MFX_ERR_NONE; mfxSyncPoint syncp = {}; bool isEncGoing = true; bool isDraining = false; while (isEncGoing) { isDraining = encSurface == nullptr; sts = MFXVideoENCODE_EncodeFrameAsync( m_vplSession, &encodeCtrl, encSurface, &m_vplBitstream, &syncp ); if (encSurface) { VPL_VERIFY(encSurface->FrameInterface->Release(encSurface)); encSurface = nullptr; } switch (sts) { case MFX_ERR_NONE: // MFX_ERR_NONE and syncp indicate output is available if (syncp) { // Encode output is not available on CPU until sync operation completes do { sts = MFXVideoCORE_SyncOperation(m_vplSession, syncp, WAIT_100_MILLISECONDS); if (MFX_ERR_NONE == sts) { ParseFrameNals( m_codec, reinterpret_cast( m_vplBitstream.Data + m_vplBitstream.DataOffset ), m_vplBitstream.DataLength, targetTimestampNs, insertIDR ); m_vplBitstream.DataLength = 0; } } while (sts == MFX_WRN_IN_EXECUTION); } break; case MFX_ERR_NOT_ENOUGH_BUFFER: ERROR_THROW("not enough buffer"); case MFX_ERR_MORE_DATA: if (isDraining == true) isEncGoing = false; break; case MFX_ERR_DEVICE_LOST: ERROR_THROW("device lost"); case MFX_WRN_DEVICE_BUSY: VPL_DEBUG("device busy"); break; default: ERROR_THROW("unknown encoding status %d", sts); } } } void VideoEncoderVPL::InitTransferTex() { D3D11_TEXTURE2D_DESC transferTexDesc = { UINT(m_renderWidth), UINT(m_renderHeight), 1, 1, m_dxColorFormat, { 1, 0 }, D3D11_USAGE_DEFAULT, D3D11_BIND_SHADER_RESOURCE, 0, D3D11_RESOURCE_MISC_SHARED }; HRESULT hr = m_pD3DRender->GetDevice()->CreateTexture2D(&transferTexDesc, nullptr, &m_transferTex); if (FAILED(hr)) ERROR_THROW("failed to create transfer texture HR=%p %ls", hr, GetErrorStr(hr).c_str()); } void VideoEncoderVPL::InitVpl() { m_vplLoader = MFXLoad(); VERIFY(m_vplLoader != nullptr, "MFXLoad failed -- is implementation in path?"); CheckVPLConfig(); VPL_VERIFY(MFXCreateSession(m_vplLoader, 0, &m_vplSession)); VPL_VERIFY( MFXVideoCORE_SetHandle(m_vplSession, MFX_HANDLE_D3D11_DEVICE, m_pD3DRender->GetDevice()) ); LogImplementationInfo(); // Get interface for ImportFrameSurface VPL_VERIFY(MFXGetMemoryInterface(m_vplSession, &m_vplMemoryInterface)); VERIFY(m_vplMemoryInterface != nullptr, "MFXGetMemoryInterface failed"); } void VideoEncoderVPL::InitVplEncode() { m_vplEncodeParams.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY; m_vplEncodeParams.mfx.LowPower = MFX_CODINGOPTION_ON; m_vplEncodeParams.AsyncDepth = 1; m_vplEncodeParams.mfx.CodecId = m_vplCodec; m_vplEncodeParams.mfx.CodecProfile = m_vplCodecProfile; m_vplEncodeParams.mfx.TargetUsage = m_vplQualityPreset; m_vplEncodeParams.mfx.TargetKbps = m_bitrateInMBits * 1000; m_vplEncodeParams.mfx.RateControlMethod = m_vplRateControlMode; m_vplEncodeParams.mfx.FrameInfo.FrameRateExtN = m_refreshRate; m_vplEncodeParams.mfx.FrameInfo.FrameRateExtD = 1; m_vplEncodeParams.mfx.FrameInfo.FourCC = m_vplColorFormat; m_vplEncodeParams.mfx.FrameInfo.ChromaFormat = m_vplChromaFormat; m_vplEncodeParams.mfx.FrameInfo.CropW = m_renderWidth; m_vplEncodeParams.mfx.FrameInfo.CropH = m_renderHeight; m_vplEncodeParams.mfx.FrameInfo.Width = ALIGN16(m_renderWidth); m_vplEncodeParams.mfx.FrameInfo.Height = ALIGN16(m_renderHeight); mfxStatus sts = MFXVideoENCODE_Query(m_vplSession, &m_vplEncodeParams, &m_vplEncodeParams); switch (sts) { case MFX_WRN_INCOMPATIBLE_VIDEO_PARAM: VPL_WARN("incompatible video params, auto-correcting"); break; case MFX_WRN_PARTIAL_ACCELERATION: VPL_WARN("partial acceleration"); break; case MFX_ERR_UNSUPPORTED: ERROR_THROW("query unsupported"); } // Initialize ENCODE VPL_VERIFY(MFXVideoENCODE_Init(m_vplSession, &m_vplEncodeParams)); } mfxFrameSurface1* VideoEncoderVPL::VplImportTexture(ID3D11Texture2D* texture) { m_pD3DRender->GetContext()->CopyResource(m_transferTex.p, texture); mfxSurfaceD3D11Tex2D extSurfD3D11 = {}; extSurfD3D11.SurfaceInterface.Header.SurfaceType = MFX_SURFACE_TYPE_D3D11_TEX2D; extSurfD3D11.SurfaceInterface.Header.SurfaceFlags = MFX_SURFACE_FLAG_IMPORT_SHARED | MFX_SURFACE_FLAG_IMPORT_COPY; extSurfD3D11.SurfaceInterface.Header.StructSize = sizeof(mfxSurfaceD3D11Tex2D); extSurfD3D11.texture2D = m_transferTex.p; mfxFrameSurface1* encSurface = nullptr; VPL_VERIFY(m_vplMemoryInterface->ImportFrameSurface( m_vplMemoryInterface, MFX_SURFACE_COMPONENT_ENCODE, &extSurfD3D11.SurfaceInterface.Header, &encSurface )); return encSurface; } void VideoEncoderVPL::ChooseParams() { Settings& s = Settings::Instance(); m_refreshRate = s.m_refreshRate; m_codec = s.m_codec; // h264 encoding is currently broken due to an encoding // error when forcing the idr frame type: // // mfxEncodeCtrl encodeCtrl = {}; // encodeCtrl.FrameType = MFX_FRAMETYPE_IDR; // // Results in error -15 (MFX_ERR_INVALID_VIDEO_PARAM) // when encoding a frame. if (m_codec == ALVR_CODEC_H264) { VPL_WARN("h264 codec currently unsupported, forcing HEVC"); m_codec = ALVR_CODEC_HEVC; } if (s.m_enableHdr) { if (s.m_use10bitEncoder) { m_dxColorFormat = DXGI_FORMAT_P010; m_vplColorFormat = MFX_FOURCC_P010; m_vplChromaFormat = MFX_CHROMAFORMAT_YUV420; } else { m_dxColorFormat = DXGI_FORMAT_NV12; m_vplColorFormat = MFX_FOURCC_NV12; m_vplChromaFormat = MFX_CHROMAFORMAT_YUV420; } } else { m_dxColorFormat = DXGI_FORMAT_R8G8B8A8_UNORM; m_vplColorFormat = MFX_FOURCC_BGR4; m_vplChromaFormat = MFX_CHROMAFORMAT_YUV444; } switch (m_codec) { case ALVR_CODEC_H264: m_vplCodec = MFX_CODEC_AVC; break; case ALVR_CODEC_HEVC: m_vplCodec = MFX_CODEC_HEVC; break; case ALVR_CODEC_AV1: m_vplCodec = MFX_CODEC_AV1; break; default: ERROR_THROW("unsupported video encoding %d", s.m_codec); } m_vplCodecProfile = MFX_PROFILE_UNKNOWN; if (m_codec == ALVR_CODEC_H264) { switch (s.m_h264Profile) { case ALVR_H264_PROFILE_BASELINE: m_vplCodecProfile = MFX_PROFILE_AVC_BASELINE; break; case ALVR_H264_PROFILE_MAIN: m_vplCodecProfile = MFX_PROFILE_AVC_MAIN; break; case ALVR_H264_PROFILE_HIGH: m_vplCodecProfile = MFX_PROFILE_AVC_HIGH; break; default: ERROR_THROW("unsupported h264 profile %d", s.m_h264Profile); } } if (s.m_use10bitEncoder) { switch (m_codec) { case ALVR_CODEC_H264: m_vplCodecProfile = MFX_PROFILE_AVC_HIGH10; break; case ALVR_CODEC_HEVC: m_vplCodecProfile = MFX_PROFILE_HEVC_MAIN10; break; } } switch (s.m_encoderQualityPreset) { case ALVR_QUALITY: m_vplQualityPreset = MFX_TARGETUSAGE_BEST_QUALITY; break; case ALVR_BALANCED: m_vplQualityPreset = MFX_TARGETUSAGE_BALANCED; break; case ALVR_SPEED: m_vplQualityPreset = MFX_TARGETUSAGE_BEST_SPEED; break; default: ERROR_THROW("invalid encoder quality preset"); } switch (s.m_rateControlMode) { case ALVR_CBR: m_vplRateControlMode = MFX_RATECONTROL_CBR; break; case ALVR_VBR: m_vplRateControlMode = MFX_RATECONTROL_VBR; break; default: ERROR_THROW("invalid rate control mode"); } } void VideoEncoderVPL::CheckVPLConfig() { mfxConfig cfg[5]; mfxVariant cfgVal[5]; // Implementation used must be the hardware implementation cfg[0] = MFXCreateConfig(m_vplLoader); VERIFY(cfg[0] != NULL, "MFXCreateConfig failed"); cfgVal[0].Type = MFX_VARIANT_TYPE_U32; cfgVal[0].Data.U32 = MFX_IMPL_TYPE_HARDWARE; VPL_VERIFY(MFXSetConfigFilterProperty(cfg[0], (mfxU8*)"mfxImplDescription.Impl", cfgVal[0])); // Implementation used must provide API version 2.9 or newer cfg[1] = MFXCreateConfig(m_vplLoader); VERIFY(NULL != cfg[1], "MFXCreateConfig failed") cfgVal[1].Type = MFX_VARIANT_TYPE_U32; cfgVal[1].Data.U32 = VPLVERSION(2, 9); VPL_VERIFY(MFXSetConfigFilterProperty( cfg[1], (mfxU8*)"mfxImplDescription.ApiVersion.Version", cfgVal[1] )); // Implementation used must be D3D11 acceleration mode cfg[2] = MFXCreateConfig(m_vplLoader); VERIFY(NULL != cfg[2], "MFXCreateConfig failed") cfgVal[2].Type = MFX_VARIANT_TYPE_U32; cfgVal[2].Data.U32 = MFX_ACCEL_MODE_VIA_D3D11; VPL_VERIFY( MFXSetConfigFilterProperty(cfg[2], (mfxU8*)"mfxImplDescription.AccelerationMode", cfgVal[2]) ); // Implementation used must be D3D11 surface sharing mode // Applying the 3 associated parameters (logical AND operation) using a single mfxConfig cfg[3] = MFXCreateConfig(m_vplLoader); VERIFY(NULL != cfg[3], "MFXCreateConfig failed") cfgVal[3].Type = MFX_VARIANT_TYPE_U32; cfgVal[3].Data.U32 = MFX_SURFACE_TYPE_D3D11_TEX2D; VPL_VERIFY(MFXSetConfigFilterProperty( cfg[3], (mfxU8*)"mfxSurfaceTypesSupported.surftype.SurfaceType", cfgVal[3] )); cfgVal[3].Data.U32 = MFX_SURFACE_COMPONENT_ENCODE; VPL_VERIFY(MFXSetConfigFilterProperty( cfg[3], (mfxU8*)"mfxSurfaceTypesSupported.surftype.surfcomp.SurfaceComponent", cfgVal[3] )); cfgVal[3].Data.U32 = MFX_SURFACE_FLAG_IMPORT_COPY; VPL_VERIFY(MFXSetConfigFilterProperty( cfg[3], (mfxU8*)"mfxSurfaceTypesSupported.surftype.surfcomp.SurfaceFlags", cfgVal[3] )); // Implementation must provide correct codec cfg[4] = MFXCreateConfig(m_vplLoader); VERIFY(NULL != cfg[4], "MFXCreateConfig failed") cfgVal[4].Type = MFX_VARIANT_TYPE_U32; cfgVal[4].Data.U32 = m_vplCodec; VPL_VERIFY(MFXSetConfigFilterProperty( cfg[4], (mfxU8*)"mfxImplDescription.mfxEncoderDescription.encoder.CodecID", cfgVal[4] )); } void VideoEncoderVPL::LogImplementationInfo() { mfxImplDescription* idesc = nullptr; mfxStatus sts; // Loads info about implementation at specified list location sts = MFXEnumImplementations(m_vplLoader, 0, MFX_IMPLCAPS_IMPLDESCSTRUCTURE, (mfxHDL*)&idesc); if (!idesc || (sts != MFX_ERR_NONE)) return; const char* accel_mode; switch (idesc->AccelerationMode) { case MFX_ACCEL_MODE_NA: accel_mode = "na"; break; case MFX_ACCEL_MODE_VIA_D3D9: accel_mode = "d3d9"; break; case MFX_ACCEL_MODE_VIA_D3D11: accel_mode = "d3d11"; break; case MFX_ACCEL_MODE_VIA_VAAPI: accel_mode = "vaapi"; break; case MFX_ACCEL_MODE_VIA_VAAPI_DRM_MODESET: accel_mode = "vaapi_drm_modeset"; break; case MFX_ACCEL_MODE_VIA_VAAPI_GLX: accel_mode = "vaapi_glx"; break; case MFX_ACCEL_MODE_VIA_VAAPI_X11: accel_mode = "vaapi_x11"; break; case MFX_ACCEL_MODE_VIA_VAAPI_WAYLAND: accel_mode = "vaapi_wayland"; break; case MFX_ACCEL_MODE_VIA_HDDLUNITE: accel_mode = "hddlunite"; break; default: accel_mode = "unknown"; break; } VPL_INFO( "using api version %hu.%hu on device %s", idesc->ApiVersion.Major, idesc->ApiVersion.Minor, idesc->Dev.DeviceID ); VPL_DEBUG("api version: %hu.%hu", idesc->ApiVersion.Major, idesc->ApiVersion.Minor); VPL_DEBUG("impl type: hw"); VPL_DEBUG("accel mode: %s", accel_mode); VPL_DEBUG("device id: %s", idesc->Dev.DeviceID); MFXDispReleaseImplDescription(m_vplLoader, idesc); #if (MFX_VERSION >= 2004) // Show implementation path, added in 2.4 API mfxHDL implPath = nullptr; sts = MFXEnumImplementations(m_vplLoader, 0, MFX_IMPLCAPS_IMPLPATH, &implPath); if (!implPath || (sts != MFX_ERR_NONE)) return; VPL_DEBUG("path: %s", reinterpret_cast(implPath)); MFXDispReleaseImplDescription(m_vplLoader, implPath); #endif } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/VideoEncoderVPL.h ================================================ #pragma once #include "VideoEncoder.h" #include "shared/d3drender.h" #include #include #include #include #include "vpl/mfx.h" #include "vpl/mfxjpeg.h" #include "vpl/mfxmemory.h" #include "vpl/mfxstructures.h" #include "vpl/mfxvideo.h" #if (MFX_VERSION >= 2000) #include "vpl/mfxdispatcher.h" #endif class VideoEncoderVPL : public VideoEncoder { public: VideoEncoderVPL(std::shared_ptr pD3DRender, int width, int height); ~VideoEncoderVPL(); void Initialize(); void Shutdown(); void Transmit( ID3D11Texture2D* pTexture, uint64_t presentationTime, uint64_t targetTimestampNs, bool insertIDR ); private: void CheckVPLConfig(); void ChooseParams(); void InitTransferTex(); void InitVpl(); void InitVplEncode(); mfxFrameSurface1* VplImportTexture(ID3D11Texture2D* texture); void LogImplementationInfo(); std::shared_ptr m_pD3DRender; int m_codec; int m_renderWidth; int m_renderHeight; int m_refreshRate; int m_bitrateInMBits; mfxU32 m_vplCodec; mfxU32 m_vplCodecProfile; mfxU32 m_vplColorFormat; mfxU32 m_vplChromaFormat; mfxU32 m_vplQualityPreset; mfxU32 m_vplRateControlMode; DXGI_FORMAT m_dxColorFormat; mfxVideoParam m_vplEncodeParams = {}; mfxLoader m_vplLoader = nullptr; mfxSession m_vplSession = nullptr; mfxBitstream m_vplBitstream = {}; mfxMemoryInterface* m_vplMemoryInterface = nullptr; CComPtr m_transferTex; }; ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/QuadVertexShader.hlsl ================================================ struct PixelType { float2 uv : TEXCOORD0; // TEXCOORD0 must be first if I don't want to define "position" in the pixel shader float4 position : SV_Position; }; //https://gamedev.stackexchange.com/questions/98283/how-do-i-draw-a-full-screen-quad-in-directx-11 PixelType main(uint vertexID : SV_VertexID) { PixelType pix; pix.uv = float2(vertexID & 1, vertexID >> 1); pix.position = float4((pix.uv.x - 0.5f) * 2, -(pix.uv.y - 0.5f) * 2, 0, 1); return pix; } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/RenderPipeline.cpp ================================================ #include "RenderPipeline.h" using namespace std::string_literals; namespace d3d_render_utils { RenderPipeline::RenderPipeline(ID3D11Device* device) { mDevice = device; mDevice->GetImmediateContext(&mImmediateContext); } void RenderPipeline::Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, ID3D11PixelShader* pixelShader, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer, bool enableAlphaBlend, bool overrideAlpha ) { mInputTextureViews.clear(); for (auto tex : inputTextures) { ID3D11ShaderResourceView* resourceView; OK_OR_THROW( mDevice->CreateShaderResourceView(tex, nullptr, &resourceView), "Failed to create input texture resosurce view." ); mInputTextureViews.push_back(resourceView); } OK_OR_THROW( mDevice->CreateShaderResourceView(renderTarget, nullptr, &mRenderTargetResourceView), "Failed to create render target resosurce view." ); OK_OR_THROW( mDevice->CreateRenderTargetView(renderTarget, nullptr, &mRenderTargetView), "Failed to create ID3D11RenderTargetView." ); D3D11_TEXTURE2D_DESC renderTargetDesc; renderTarget->GetDesc(&renderTargetDesc); mViewport.Width = (FLOAT)renderTargetDesc.Width; mViewport.Height = (FLOAT)renderTargetDesc.Height; mViewport.MinDepth = 0.0f; mViewport.MaxDepth = 1.0f; mViewport.TopLeftX = 0.0f; mViewport.TopLeftY = 0.0f; mGenerateMipmaps = renderTargetDesc.MipLevels != 1; mShaderBuffer = shaderBuffer; mVertexShader = quadVertexShader; mPixelShader = pixelShader; D3D11_BLEND_DESC blendDesc = { 0 }; blendDesc.RenderTarget[0].BlendEnable = enableAlphaBlend; blendDesc.RenderTarget[0].SrcBlend = (overrideAlpha ? D3D11_BLEND_ONE : D3D11_BLEND_SRC_ALPHA); blendDesc.RenderTarget[0].DestBlend = (overrideAlpha ? D3D11_BLEND_ZERO : D3D11_BLEND_INV_SRC_ALPHA); blendDesc.RenderTarget[0].BlendOp = D3D11_BLEND_OP_ADD; blendDesc.RenderTarget[0].SrcBlendAlpha = D3D11_BLEND_ONE; blendDesc.RenderTarget[0].DestBlendAlpha = D3D11_BLEND_ZERO; blendDesc.RenderTarget[0].BlendOpAlpha = D3D11_BLEND_OP_ADD; blendDesc.RenderTarget[0].RenderTargetWriteMask = (overrideAlpha ? D3D11_COLOR_WRITE_ENABLE_RED | D3D11_COLOR_WRITE_ENABLE_GREEN | D3D11_COLOR_WRITE_ENABLE_BLUE : D3D11_COLOR_WRITE_ENABLE_ALL); OK_OR_THROW( mDevice->CreateBlendState(&blendDesc, &mBlendState), "Failed to create blend state." ); } void RenderPipeline::Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, std::vector& pixelShaderCSO, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer, bool enableAlphaBlend, bool overrideAlpha ) { auto pixelShader = CreatePixelShader(mDevice.Get(), pixelShaderCSO); Initialize( inputTextures, quadVertexShader, pixelShader, renderTarget, shaderBuffer, enableAlphaBlend, overrideAlpha ); } void RenderPipeline::Render(ID3D11DeviceContext* otherContext) { ID3D11DeviceContext* context = otherContext != nullptr ? otherContext : mImmediateContext.Get(); context->OMSetRenderTargets(1, mRenderTargetView.GetAddressOf(), nullptr); context->RSSetViewports(1, &mViewport); context->OMSetBlendState(mBlendState.Get(), nullptr, 0xffffffff); if (mShaderBuffer != nullptr) { context->PSSetConstantBuffers(0, 1, mShaderBuffer.GetAddressOf()); } std::vector inputTextureViewPtrs; for (auto texView : mInputTextureViews) { inputTextureViewPtrs.push_back(texView.Get()); } context->PSSetShaderResources(0, (UINT)mInputTextureViews.size(), &inputTextureViewPtrs[0]); context->VSSetShader(mVertexShader.Get(), nullptr, 0); context->PSSetShader(mPixelShader.Get(), nullptr, 0); context->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLESTRIP); context->Draw(4, 0); if (mGenerateMipmaps) { context->GenerateMips(mRenderTargetResourceView.Get()); } } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/RenderPipeline.h ================================================ #pragma once #include "RenderUtils.h" namespace d3d_render_utils { class RenderPipeline { public: RenderPipeline(ID3D11Device* device); void Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, std::vector& pixelShaderCSO, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer = nullptr, bool enableAlphaBlend = false, bool overrideAlpha = false ); void Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, ID3D11PixelShader* pixelShader, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer = nullptr, bool enableAlphaBlend = false, bool overrideAlpha = false ); void Render(ID3D11DeviceContext* otherContext = nullptr); private: D3D11_VIEWPORT mViewport; bool mGenerateMipmaps; Microsoft::WRL::ComPtr mDevice; Microsoft::WRL::ComPtr mImmediateContext; std::vector> mInputTextureViews; Microsoft::WRL::ComPtr mRasterizerState; Microsoft::WRL::ComPtr mRenderTargetView; Microsoft::WRL::ComPtr mRenderTargetResourceView; Microsoft::WRL::ComPtr mShaderBuffer; Microsoft::WRL::ComPtr mVertexShader; Microsoft::WRL::ComPtr mPixelShader; Microsoft::WRL::ComPtr mBlendState; }; } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/RenderPipelineYUV.cpp ================================================ #include "RenderPipelineYUV.h" using namespace std::string_literals; namespace d3d_render_utils { RenderPipelineYUV::RenderPipelineYUV(ID3D11Device* device) { mDevice = device; mDevice->GetImmediateContext(&mImmediateContext); } void RenderPipelineYUV::Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, ID3D11PixelShader* pixelShader, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer ) { mInputTextureViews.clear(); for (auto tex : inputTextures) { ID3D11ShaderResourceView* resourceView; OK_OR_THROW( mDevice->CreateShaderResourceView(tex, nullptr, &resourceView), "Failed to create input texture resosurce view." ); mInputTextureViews.push_back(resourceView); } D3D11_TEXTURE2D_DESC renderTargetDesc; renderTarget->GetDesc(&renderTargetDesc); DXGI_FORMAT uvFormat = renderTargetDesc.Format == DXGI_FORMAT_NV12 ? DXGI_FORMAT_R8G8_UNORM : DXGI_FORMAT_R16G16_UNORM; DXGI_FORMAT yFormat = renderTargetDesc.Format == DXGI_FORMAT_NV12 ? DXGI_FORMAT_R8_UNORM : DXGI_FORMAT_R16_UNORM; // Create SRV for luminance (Y) plane D3D11_SHADER_RESOURCE_VIEW_DESC srvDescY = {}; srvDescY.Format = yFormat; srvDescY.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D; srvDescY.Texture2D.MostDetailedMip = 0; srvDescY.Texture2D.MipLevels = 1; ID3D11ShaderResourceView* pSRVY; OK_OR_THROW( mDevice->CreateShaderResourceView(renderTarget, &srvDescY, &mRenderTargetResourceViewY), "Failed to create render target resosurce view." ); // Create SRV for chrominance (UV) planes D3D11_SHADER_RESOURCE_VIEW_DESC srvDescUV = {}; srvDescUV.Format = uvFormat; srvDescUV.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D; srvDescUV.Texture2D.MostDetailedMip = 0; srvDescUV.Texture2D.MipLevels = 1; ID3D11ShaderResourceView* pSRVUV; OK_OR_THROW( mDevice->CreateShaderResourceView(renderTarget, &srvDescUV, &mRenderTargetResourceViewUV), "Failed to create render target resosurce view." ); // Create luminance (Y) render target view D3D11_RENDER_TARGET_VIEW_DESC rtvDescLuminance = {}; rtvDescLuminance.Format = yFormat; rtvDescLuminance.ViewDimension = D3D11_RTV_DIMENSION_TEXTURE2D; rtvDescLuminance.Texture2D.MipSlice = 0; ID3D11RenderTargetView* pRTVLuminance; OK_OR_THROW( mDevice->CreateRenderTargetView(renderTarget, &rtvDescLuminance, &mRenderTargetViewY), "Failed to create ID3D11RenderTargetView." ); // Create chrominance (UV) render target view D3D11_RENDER_TARGET_VIEW_DESC rtvDescChrominance = {}; rtvDescChrominance.Format = uvFormat; rtvDescChrominance.ViewDimension = D3D11_RTV_DIMENSION_TEXTURE2D; rtvDescChrominance.Texture2D.MipSlice = 0; ID3D11RenderTargetView* pRTVChrominance; OK_OR_THROW( mDevice->CreateRenderTargetView(renderTarget, &rtvDescChrominance, &mRenderTargetViewUV), "Failed to create ID3D11RenderTargetView." ); mViewportY.Width = (FLOAT)renderTargetDesc.Width; mViewportY.Height = (FLOAT)renderTargetDesc.Height; mViewportY.MinDepth = 0.0f; mViewportY.MaxDepth = 1.0f; mViewportY.TopLeftX = 0.0f; mViewportY.TopLeftY = 0.0f; mViewportUV.Width = (FLOAT)renderTargetDesc.Width; mViewportUV.Height = (FLOAT)renderTargetDesc.Height; mViewportUV.MinDepth = 0.0f; mViewportUV.MaxDepth = 1.0f; mViewportUV.TopLeftX = 0.0f; mViewportUV.TopLeftY = 0.0f; mGenerateMipmaps = renderTargetDesc.MipLevels != 1; mShaderBuffer = shaderBuffer; mVertexShader = quadVertexShader; mPixelShader = pixelShader; } void RenderPipelineYUV::Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, std::vector& pixelShaderCSO, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer ) { auto pixelShader = CreatePixelShader(mDevice.Get(), pixelShaderCSO); Initialize(inputTextures, quadVertexShader, pixelShader, renderTarget, shaderBuffer); } void RenderPipelineYUV::Render(ID3D11DeviceContext* otherContext) { ID3D11DeviceContext* context = otherContext != nullptr ? otherContext : mImmediateContext.Get(); ID3D11RenderTargetView* aRenderTargets[] = { mRenderTargetViewY.Get(), mRenderTargetViewUV.Get() }; D3D11_VIEWPORT aViewports[] = { mViewportY, mViewportUV }; ID3D11RenderTargetView* aRenderTargetsHack[] = { mRenderTargetViewY.Get() }; D3D11_VIEWPORT aViewportsHack[] = { mViewportY }; context->OMSetRenderTargets(2, aRenderTargets, nullptr); context->RSSetViewports(2, aViewports); if (mShaderBuffer != nullptr) { context->PSSetConstantBuffers(0, 1, mShaderBuffer.GetAddressOf()); } std::vector inputTextureViewPtrs; for (auto texView : mInputTextureViews) { inputTextureViewPtrs.push_back(texView.Get()); } context->PSSetShaderResources(0, (UINT)mInputTextureViews.size(), &inputTextureViewPtrs[0]); context->VSSetShader(mVertexShader.Get(), nullptr, 0); context->PSSetShader(mPixelShader.Get(), nullptr, 0); context->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLESTRIP); context->Draw(4, 0); // HACK: Whyyyyyy does the Y channel only render the top-left corner // with both render targets enabled???? context->OMSetRenderTargets(1, aRenderTargetsHack, nullptr); context->RSSetViewports(1, aViewportsHack); context->Draw(4, 0); if (mGenerateMipmaps) { context->GenerateMips(mRenderTargetResourceViewY.Get()); context->GenerateMips(mRenderTargetResourceViewUV.Get()); } } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/RenderPipelineYUV.h ================================================ #pragma once #include "RenderUtils.h" namespace d3d_render_utils { class RenderPipelineYUV { public: RenderPipelineYUV(ID3D11Device* device); void Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, std::vector& pixelShaderCSO, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer = nullptr ); void Initialize( std::vector inputTextures, ID3D11VertexShader* quadVertexShader, ID3D11PixelShader* pixelShader, ID3D11Texture2D* renderTarget, ID3D11Buffer* shaderBuffer = nullptr ); void Render(ID3D11DeviceContext* otherContext = nullptr); private: D3D11_VIEWPORT mViewportY; D3D11_VIEWPORT mViewportUV; bool mGenerateMipmaps; Microsoft::WRL::ComPtr mDevice; Microsoft::WRL::ComPtr mImmediateContext; std::vector> mInputTextureViews; Microsoft::WRL::ComPtr mRasterizerState; Microsoft::WRL::ComPtr mRenderTargetViewY; Microsoft::WRL::ComPtr mRenderTargetViewUV; Microsoft::WRL::ComPtr mRenderTargetResourceViewY; Microsoft::WRL::ComPtr mRenderTargetResourceViewUV; Microsoft::WRL::ComPtr mShaderBuffer; Microsoft::WRL::ComPtr mVertexShader; Microsoft::WRL::ComPtr mPixelShader; Microsoft::WRL::ComPtr mBlendState; }; } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/RenderUtils.cpp ================================================ #include "RenderUtils.h" #include using namespace std::string_literals; using Microsoft::WRL::ComPtr; namespace d3d_render_utils { void GetAdapterInfo(ID3D11Device* d3dDevice, int32_t& adapterIndex, std::wstring& adapterName) { ComPtr dxgiDevice; OK_OR_THROW(QUERY(d3dDevice, &dxgiDevice), "Failed to query DXGI device."); ComPtr adapter; OK_OR_THROW( dxgiDevice->GetParent(__uuidof(IDXGIAdapter), (void**)&adapter), "Failed to get DXGI adapter." ); ComPtr factory; OK_OR_THROW( adapter->GetParent(__uuidof(IDXGIFactory), (void**)&factory), "Failed to get DXGI factory." ); DXGI_ADAPTER_DESC adapterDesc; adapter->GetDesc(&adapterDesc); ComPtr enumeratedAdapter; for (UINT idx = 0; factory->EnumAdapters(idx, &enumeratedAdapter) != DXGI_ERROR_NOT_FOUND; idx++) { DXGI_ADAPTER_DESC enumeratedDesc; enumeratedAdapter->GetDesc(&enumeratedDesc); if (enumeratedDesc.AdapterLuid.HighPart == adapterDesc.AdapterLuid.HighPart && enumeratedDesc.AdapterLuid.LowPart == adapterDesc.AdapterLuid.LowPart) { adapterIndex = idx; adapterName = adapterDesc.Description; return; } } throw MakeException("No valid adapter found."); } ID3D11Device* CreateDevice(IDXGIAdapter* dxgiAdapter) { UINT creationFlags = 0; #if _DEBUG creationFlags |= D3D11_CREATE_DEVICE_DEBUG; #endif D3D_FEATURE_LEVEL featureLevel; ID3D11Device* device; ComPtr context; OK_OR_THROW( D3D11CreateDevice( dxgiAdapter, dxgiAdapter != nullptr ? D3D_DRIVER_TYPE_UNKNOWN : D3D_DRIVER_TYPE_HARDWARE, nullptr, creationFlags, nullptr, 0, D3D11_SDK_VERSION, &device, &featureLevel, &context ), "Failed to create D3D11 device!" ); if (featureLevel < D3D_FEATURE_LEVEL_11_0) { throw MakeException("DX11 level hardware required!"); } // todo: check if needed: ComPtr multithread; if (SUCCEEDED(QUERY(context, &multithread))) { multithread->SetMultithreadProtected(true); } else { Debug("Failed to get ID3D11Multithread interface. Ignore.\n"); } return device; } ID3D11Device* CreateDevice(uint32_t adapterIndex) { ComPtr factory; OK_OR_THROW( CreateDXGIFactory1(__uuidof(IDXGIFactory1), (void**)&factory), "Failed to create DXGIFactory1!" ); ComPtr adapter; OK_OR_THROW(factory->EnumAdapters(adapterIndex, &adapter), "Failed to create DXGIAdapter!"); return CreateDevice(adapter.Get()); } ID3D11Texture2D* CreateTexture( ID3D11Device* device, uint32_t width, uint32_t height, DXGI_FORMAT format, bool mipmaps, bool shared, UINT sampleCount ) { D3D11_TEXTURE2D_DESC desc = { 0 }; desc.Width = width; desc.Height = height; desc.Format = format; desc.SampleDesc.Count = sampleCount; desc.MipLevels = mipmaps ? 0 : 1; desc.MiscFlags = (shared ? D3D11_RESOURCE_MISC_SHARED : 0) | (mipmaps ? D3D11_RESOURCE_MISC_GENERATE_MIPS : 0); // D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX | D3D11_RESOURCE_MISC_SHARED_NTHANDLE desc.ArraySize = 1; desc.SampleDesc.Quality = 0; desc.Usage = D3D11_USAGE_DEFAULT; desc.BindFlags = D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE; desc.CPUAccessFlags = 0; ID3D11Texture2D* texture; OK_OR_THROW(device->CreateTexture2D(&desc, nullptr, &texture), "Failed to create texture."); return texture; } ID3D11Buffer* _CreateBuffer(ID3D11Device* device, const void* bufferData, size_t bufferSize, D3D11_USAGE usage) { D3D11_BUFFER_DESC bufferDesc = { 0 }; bufferDesc.Usage = usage; bufferDesc.ByteWidth = (UINT)bufferSize; bufferDesc.BindFlags = D3D11_BIND_CONSTANT_BUFFER; bufferDesc.StructureByteStride = 0; D3D11_SUBRESOURCE_DATA dataDesc = { 0 }; dataDesc.pSysMem = bufferData; ID3D11Buffer* buffer; OK_OR_THROW( device->CreateBuffer(&bufferDesc, bufferData != nullptr ? &dataDesc : nullptr, &buffer), "Failed to create D3D11 buffer." ); return buffer; } void UpdateBuffer(ID3D11DeviceContext* context, ID3D11Buffer* buffer, const void* bufferData) { context->UpdateSubresource(buffer, 0, nullptr, bufferData, 0, 0); } ID3D11VertexShader* CreateVertexShader(ID3D11Device* device, std::vector& vertexShaderCSO) { ID3D11VertexShader* vertexShader; OK_OR_THROW( device->CreateVertexShader( &vertexShaderCSO[0], vertexShaderCSO.size(), nullptr, &vertexShader ), "Failed to create vertex shader." ); return vertexShader; } ID3D11PixelShader* CreatePixelShader(ID3D11Device* device, std::vector& pixelShaderCSO) { ID3D11PixelShader* pixelShader; OK_OR_THROW( device->CreatePixelShader(&pixelShaderCSO[0], pixelShaderCSO.size(), nullptr, &pixelShader), "Failed to create pixel shader." ); return pixelShader; } ID3D11Texture2D* GetTextureFromHandle(ID3D11Device* device, HANDLE handle) { ID3D11Texture2D* texture; OK_OR_THROW( device->OpenSharedResource(handle, __uuidof(ID3D11Texture2D), (void**)&texture), "[VDispDvr] SyncTexture is NULL!" ); return texture; } HANDLE GetHandleFromTexture(ID3D11Texture2D* texture) { auto exceptMsg = "Failed to get handle from shared texture"; ComPtr resource; OK_OR_THROW(QUERY(texture, &resource), exceptMsg); HANDLE handle; OK_OR_THROW(resource->GetSharedHandle(&handle), exceptMsg); return handle; } void KeyedMutexSync( ID3D11Device* device, HANDLE handle, uint64_t timeout, std::function callback ) { ComPtr syncTexture = GetTextureFromHandle(device, handle); ComPtr keyedMutex; OK_OR_THROW(QUERY(syncTexture, &keyedMutex), "Failed to query mutex"); // TODO: Reasonable timeout and timeout handling OK_OR_THROW(keyedMutex->AcquireSync(0, (DWORD)timeout), "[VDispDvr] ACQUIRESYNC FAILED!!!"); callback(); keyedMutex->ReleaseSync(0); } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/d3d-render-utils/RenderUtils.h ================================================ #pragma once #pragma comment(lib, "dxgi.lib") #pragma comment(lib, "d3d11.lib") #include #include #include #include #include #include #include #include "alvr_server/Logger.h" #include "alvr_server/Utils.h" #define OK_OR_THROW(dxcall, msg) \ { \ HRESULT hr = dxcall; \ if (FAILED(hr)) \ throw MakeException("%ls HR=%p %ls", msg, hr, GetErrorStr(hr).c_str()); \ } #define QUERY(from, ppd3d) from->QueryInterface(__uuidof(*(ppd3d)), (void**)(ppd3d)) namespace d3d_render_utils { void GetAdapterInfo(ID3D11Device* d3dDevice, int32_t& adapterIndex, std::wstring& adapterName); ID3D11Device* CreateDevice(IDXGIAdapter* dxgiAdapter = nullptr); ID3D11Device* CreateDevice(uint32_t adapterIndex); ID3D11Texture2D* CreateTexture( ID3D11Device* device, uint32_t width, uint32_t height, DXGI_FORMAT format = DXGI_FORMAT_R8G8B8A8_UNORM, bool mipmaps = false, bool shared = false, UINT sampleCount = 1 ); ID3D11Buffer* _CreateBuffer(ID3D11Device* device, const void* bufferData, size_t bufferSize, D3D11_USAGE usage); template ID3D11Buffer* CreateBuffer(ID3D11Device* device, const T& bufferData, D3D11_USAGE usage = D3D11_USAGE_IMMUTABLE) { return _CreateBuffer(device, &bufferData, sizeof(T), usage); } void UpdateBuffer(ID3D11DeviceContext* context, ID3D11Buffer* buffer, const void* bufferData); ID3D11VertexShader* CreateVertexShader(ID3D11Device* device, std::vector& vertexShaderCSO); ID3D11PixelShader* CreatePixelShader(ID3D11Device* device, std::vector& pixelShaderCSO); ID3D11Texture2D* GetTextureFromHandle(ID3D11Device* device, HANDLE handle); HANDLE GetHandleFromTexture(ID3D11Texture2D* texture); void KeyedMutexSync( ID3D11Device* device, HANDLE sharedTextureHandle, uint64_t timeout, std::function acquiredSyncCallback ); } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/shared/d3drender.cpp ================================================ //===================== Copyright (c) Valve Corporation. All Rights Reserved. ====================== #include "d3drender.h" #include #include #pragma comment( lib, "dxgi.lib" ) #pragma comment( lib, "d3d11.lib" ) #pragma comment( lib, "rpcrt4.lib" ) #define Log( ... ) namespace { inline bool operator==( const LUID &A, const LUID &B ) { return A.HighPart == B.HighPart && A.LowPart == B.LowPart; } bool FindDXGIOutput( IDXGIFactory *pFactory, int32_t nWidth, int32_t nHeight, IDXGIAdapter **pOutAdapter, IDXGIOutput **pOutOutput, int32_t *pOutX, int32_t *pOutY ) { IDXGIAdapter *pDXGIAdapter; for ( UINT nAdapterIndex = 0; pFactory->EnumAdapters( nAdapterIndex, &pDXGIAdapter ) != DXGI_ERROR_NOT_FOUND; nAdapterIndex++ ) { IDXGIOutput *pDXGIOutput; for ( UINT nOutputIndex = 0; pDXGIAdapter->EnumOutputs( nOutputIndex, &pDXGIOutput ) != DXGI_ERROR_NOT_FOUND; nOutputIndex++ ) { DXGI_OUTPUT_DESC desc; pDXGIOutput->GetDesc( &desc ); //if ( desc.DesktopCoordinates.right - desc.DesktopCoordinates.left == nWidth && // desc.DesktopCoordinates.bottom - desc.DesktopCoordinates.top == nHeight ) //{ *pOutAdapter = pDXGIAdapter; *pOutOutput = pDXGIOutput; *pOutX = desc.DesktopCoordinates.left; *pOutY = desc.DesktopCoordinates.top; return true; //} pDXGIOutput->Release(); } pDXGIAdapter->Release(); } return false; } bool CreateDevice( IDXGIAdapter *pDXGIAdapter, ID3D11Device **pD3D11Device, ID3D11DeviceContext **pD3D11Context ) { UINT creationFlags = 0; #if _DEBUG creationFlags |= D3D11_CREATE_DEVICE_DEBUG; #endif D3D_FEATURE_LEVEL eFeatureLevel; HRESULT hRes = D3D11CreateDevice( pDXGIAdapter, D3D_DRIVER_TYPE_UNKNOWN, NULL, creationFlags, NULL, 0, D3D11_SDK_VERSION, pD3D11Device, &eFeatureLevel, pD3D11Context ); #if _DEBUG // CreateDevice fails on Win10 in debug if the Win10 SDK isn't installed. if ( pD3D11Device == NULL ) { hRes = D3D11CreateDevice( pDXGIAdapter, D3D_DRIVER_TYPE_UNKNOWN, NULL, 0, NULL, 0, D3D11_SDK_VERSION, pD3D11Device, &eFeatureLevel, pD3D11Context ); } #endif if ( FAILED( hRes ) ) { Log( "Failed to create D3D11 device! (err=%u)", hRes ); return false; } if ( eFeatureLevel < D3D_FEATURE_LEVEL_11_0 ) { Log( "DX11 level hardware required!" ); return false; } ID3D11Multithread *D3D11Multithread = NULL; HRESULT hr = (*pD3D11Context)->QueryInterface(__uuidof(ID3D11Multithread), (void **)&D3D11Multithread); if (SUCCEEDED(hr)) { Log("Successfully get ID3D11Multithread interface. We set SetMultithreadProtected(TRUE)"); D3D11Multithread->SetMultithreadProtected(TRUE); D3D11Multithread->Release(); } else { Log("Failed to get ID3D11Multithread interface. Ignore."); } return true; } class CEventHelper { public: CEventHelper() { UuidFromStringA( ( RPC_CSTR ) "8c8f13b1-60eb-4b6a-a433-de86104115ac", &guid ); EventRegister( &guid, nullptr, nullptr, &handle ); } REGHANDLE handle; GUID guid; }; CEventHelper s_eventHelper; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void EventWriteString( const wchar_t* pwchEvent ) { ::EventWriteString( s_eventHelper.handle, 0, 0, pwchEvent ); } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- CD3DRender::CD3DRender() : m_pDXGIFactory( NULL ) , m_pDXGIOutput( NULL ) , m_pDXGISwapChain( NULL ) , m_pD3D11Device( NULL ) , m_pD3D11Context( NULL ) , m_nDisplayWidth( 0 ) , m_nDisplayHeight( 0 ) , m_nDisplayX( 0 ) , m_nDisplayY( 0 ) { // Initialize DXGI { // Need to use DXGI 1.1 for shared texture support. IDXGIFactory1 *pDXGIFactory1; if ( FAILED( CreateDXGIFactory1( __uuidof( IDXGIFactory1 ), ( void ** )&pDXGIFactory1 ) ) ) { Log( "Failed to create DXGIFactory1!" ); return; } else if ( FAILED( pDXGIFactory1->QueryInterface( __uuidof( IDXGIFactory ), ( void ** )&m_pDXGIFactory ) ) ) { pDXGIFactory1->Release(); Log( "Failed to get DXGIFactory interface!" ); return; } pDXGIFactory1->Release(); } } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- CD3DRender::~CD3DRender() { SAFE_RELEASE( m_pDXGIFactory ); } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CD3DRender::GetDisplayPos( int32_t *pDisplayX, int32_t *pDisplayY ) { *pDisplayX = m_nDisplayX; *pDisplayY = m_nDisplayY; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CD3DRender::GetDisplaySize( uint32_t *pDisplayWidth, uint32_t *pDisplayHeight ) { *pDisplayWidth = m_nDisplayWidth; *pDisplayHeight = m_nDisplayHeight; } //-------------------------------------------------------------------------------------------------- // Purpose: Return the DXGI index and name of the adapter currently in use. //-------------------------------------------------------------------------------------------------- bool CD3DRender::GetAdapterInfo( int32_t *pAdapterIndex, std::wstring &adapterName ) { if ( m_pD3D11Device == NULL ) return false; bool bSuccess = false; IDXGIDevice *pDXGIDevice; if ( SUCCEEDED( m_pD3D11Device->QueryInterface( __uuidof( IDXGIDevice ), ( void ** )&pDXGIDevice ) ) ) { IDXGIAdapter *pDXGIAdapter; if ( SUCCEEDED( pDXGIDevice->GetParent( __uuidof( IDXGIAdapter ), ( void ** )&pDXGIAdapter ) ) ) { DXGI_ADAPTER_DESC adapterDesc; pDXGIAdapter->GetDesc( &adapterDesc ); IDXGIFactory *pDXGIFactory; if ( SUCCEEDED( pDXGIAdapter->GetParent( __uuidof( IDXGIFactory ), ( void ** )&pDXGIFactory ) ) ) { IDXGIAdapter *pEnumeratedAdapter; for ( UINT nAdapterIndex = 0; pDXGIFactory->EnumAdapters( nAdapterIndex, &pEnumeratedAdapter ) != DXGI_ERROR_NOT_FOUND; nAdapterIndex++ ) { DXGI_ADAPTER_DESC enumeratedDesc; pEnumeratedAdapter->GetDesc( &enumeratedDesc ); pEnumeratedAdapter->Release(); if ( enumeratedDesc.AdapterLuid == adapterDesc.AdapterLuid ) { if ( pAdapterIndex ) *pAdapterIndex = nAdapterIndex; adapterName = adapterDesc.Description; bSuccess = true; break; } } pDXGIFactory->Release(); } pDXGIAdapter->Release(); } pDXGIDevice->Release(); } return bSuccess; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- ID3D11Texture2D *CD3DRender::GetSharedTexture( HANDLE hSharedTexture ) { if ( !hSharedTexture ) return NULL; for ( SharedTextures_t::iterator it = m_SharedTextureCache.begin(); it != m_SharedTextureCache.end(); ++it ) { if ( it->m_hSharedTexture == hSharedTexture ) { return it->m_pTexture; } } ID3D11Texture2D *pTexture; if ( SUCCEEDED( m_pD3D11Device->OpenSharedResource( hSharedTexture, __uuidof( ID3D11Texture2D ), ( void ** )&pTexture ) ) ) { SharedTextureEntry_t entry { hSharedTexture, pTexture }; m_SharedTextureCache.push_back( entry ); return pTexture; } return NULL; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CD3DRender::Initialize( uint32_t nAdapterIndex ) { Shutdown(); if ( m_pDXGIFactory == NULL ) return false; IDXGIAdapter *pDXGIAdapter; if ( FAILED( m_pDXGIFactory->EnumAdapters( nAdapterIndex, &pDXGIAdapter ) ) ) return false; bool bSuccess = CreateDevice( pDXGIAdapter, &m_pD3D11Device, &m_pD3D11Context ); pDXGIAdapter->Release(); return bSuccess; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CD3DRender::Initialize( uint32_t nDisplayWidth, uint32_t nDisplayHeight ) { Shutdown(); if ( m_pDXGIFactory == NULL ) return false; m_nDisplayWidth = nDisplayWidth; m_nDisplayHeight = nDisplayHeight; IDXGIAdapter *pDXGIAdapter; if ( !FindDXGIOutput( m_pDXGIFactory, m_nDisplayWidth, m_nDisplayHeight, &pDXGIAdapter, &m_pDXGIOutput, &m_nDisplayX, &m_nDisplayY ) ) return false; bool bSuccess = CreateDevice( pDXGIAdapter, &m_pD3D11Device, &m_pD3D11Context ); pDXGIAdapter->Release(); return bSuccess; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CD3DRender::Shutdown() { SetFullscreen( FALSE ); SAFE_RELEASE( m_pD3D11Context ); SAFE_RELEASE( m_pD3D11Device ); SAFE_RELEASE( m_pDXGISwapChain ); SAFE_RELEASE( m_pDXGIOutput ); m_nDisplayWidth = 0; m_nDisplayHeight = 0; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CD3DRender::CreateSwapChain( HWND hWnd, const DXGI_RATIONAL &refreshRate ) { if ( !m_pD3D11Device ) return false; if ( !m_pDXGIOutput ) return false; if ( !m_pDXGIFactory ) return false; // Determine which video mode to use DXGI_MODE_DESC modeDesc; ZeroMemory( &modeDesc, sizeof( modeDesc ) ); modeDesc.Width = m_nDisplayWidth; modeDesc.Height = m_nDisplayHeight; modeDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM; modeDesc.ScanlineOrdering = DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED; modeDesc.Scaling = DXGI_MODE_SCALING_UNSPECIFIED; modeDesc.RefreshRate = refreshRate; DXGI_MODE_DESC modeOut = modeDesc; if ( FAILED( m_pDXGIOutput->FindClosestMatchingMode( &modeDesc, &modeOut, m_pD3D11Device ) ) ) { Log( "Failed to find closest matching mode!" ); return false; } // Create a fullscreen swap chain for the window DXGI_SWAP_CHAIN_DESC swapChainDesc; ZeroMemory( &swapChainDesc, sizeof( swapChainDesc ) ); swapChainDesc.BufferCount = 2; swapChainDesc.BufferDesc = modeOut; swapChainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT; swapChainDesc.OutputWindow = hWnd; swapChainDesc.SampleDesc.Count = 1; swapChainDesc.SampleDesc.Quality = 0; swapChainDesc.Windowed = TRUE; swapChainDesc.SwapEffect = DXGI_SWAP_EFFECT_DISCARD; swapChainDesc.Flags = DXGI_SWAP_CHAIN_FLAG_ALLOW_MODE_SWITCH; if ( FAILED( m_pDXGIFactory->CreateSwapChain( m_pD3D11Device, &swapChainDesc, &m_pDXGISwapChain ) ) ) { Log( "Failed to create swap chain!" ); return false; } m_pDXGIFactory->MakeWindowAssociation( swapChainDesc.OutputWindow, DXGI_MWA_NO_WINDOW_CHANGES | DXGI_MWA_NO_ALT_ENTER ); //!! Probably don't need this as long as we Flush after Present. IDXGIDevice1 *pDXGIDevice1; if ( SUCCEEDED( m_pD3D11Device->QueryInterface( __uuidof( IDXGIDevice1 ), ( void ** )&pDXGIDevice1 ) ) ) { pDXGIDevice1->SetGPUThreadPriority( 7 ); pDXGIDevice1->SetMaximumFrameLatency( 1 ); pDXGIDevice1->Release(); } return true; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CD3DRender::SetFullscreen( BOOL bFullscreen ) { if ( m_pDXGISwapChain && m_pDXGIOutput ) { // Bail if no change is necessary. BOOL bState; if ( SUCCEEDED( m_pDXGISwapChain->GetFullscreenState( &bState, NULL ) ) ) { if ( bState == bFullscreen ) return; } // Bail if not ready to go fullscreen yet. if ( bFullscreen ) { if ( m_pDXGISwapChain->Present( 0, DXGI_PRESENT_TEST ) != S_OK ) return; } if ( m_pDXGISwapChain->SetFullscreenState( bFullscreen, bFullscreen ? m_pDXGIOutput : NULL ) == S_OK ) { UpdateBuffers(); // WM_SIZE doesn't get sent since window was created at proper size } } } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CD3DRender::UpdateBuffers() { if ( m_pDXGISwapChain ) { DXGI_SWAP_CHAIN_DESC swapChainDesc; if ( SUCCEEDED( m_pDXGISwapChain->GetDesc( &swapChainDesc ) ) ) { m_pD3D11Context->ClearState(); //OMSetRenderTargets(0, NULL, NULL); m_pDXGISwapChain->ResizeBuffers( 0, ( UINT )m_nDisplayWidth, ( UINT )m_nDisplayHeight, swapChainDesc.BufferDesc.Format, swapChainDesc.Flags ); } } } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CD3DRender::GetAdapterLuid( int32_t nAdapterIndex, uint64_t *pAdapterLuid ) { bool bSuccess = false; if ( m_pDXGIFactory != NULL ) { IDXGIAdapter *pDXGIAdapter; if ( SUCCEEDED( m_pDXGIFactory->EnumAdapters( nAdapterIndex, &pDXGIAdapter ) ) ) { DXGI_ADAPTER_DESC adapterDesc; pDXGIAdapter->GetDesc( &adapterDesc ); pDXGIAdapter->Release(); *pAdapterLuid = *( uint64_t * )&adapterDesc.AdapterLuid; bSuccess = true; } } return bSuccess; } #define MIN(a, b) ((a) < (b) ? (a) : (b)) //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CD3DRender::CopyTextureData( BYTE *pDst, uint32_t nDstRowPitch, const BYTE *pSrc, uint32_t nSrcRowPitch, uint32_t nWidth, uint32_t nHeight, uint32_t nPitch ) { if ( nDstRowPitch == nSrcRowPitch ) { memcpy( pDst, pSrc, nSrcRowPitch * nHeight ); } else { uint32_t nMinRowPitch = MIN(nDstRowPitch, nSrcRowPitch); for ( uint32_t i = 0; i < nHeight; i++ ) { memcpy( pDst, pSrc, nMinRowPitch ); pDst += nDstRowPitch; pSrc += nSrcRowPitch; } } } ================================================ FILE: alvr/server_openvr/cpp/platform/win32/shared/d3drender.h ================================================ //===================== Copyright (c) Valve Corporation. All Rights Reserved. ====================== // // Helper class for working with D3D. // //================================================================================================== #pragma once #include #include #include #include void EventWriteString( const wchar_t* pwchEvent ); // gpuview event #define SAFE_RELEASE( x ) if ( x ) { ( x )->Release(); ( x ) = NULL; } class CD3DRender { public: CD3DRender(); ~CD3DRender(); bool Initialize( uint32_t nDisplayWidth, uint32_t nDisplayHeight ); bool Initialize( uint32_t nAdapterIndex ); void Shutdown(); void GetDisplayPos( int32_t *pDisplayX, int32_t *pDisplayY ); void GetDisplaySize( uint32_t *pDisplayWidth, uint32_t *pDisplayHeight ); bool GetAdapterInfo( int32_t *pAdapterIndex, std::wstring &adapterName ); ID3D11Texture2D *GetSharedTexture( HANDLE hSharedTexture ); bool CreateSwapChain( HWND hWnd, const DXGI_RATIONAL &refreshRate ); void SetFullscreen( BOOL bFullscreen ); void UpdateBuffers(); ID3D11Device *GetDevice() { return m_pD3D11Device; } ID3D11DeviceContext *GetContext() { return m_pD3D11Context; } IDXGISwapChain *GetSwapChain() { return m_pDXGISwapChain; } bool GetAdapterLuid( int32_t nAdapterIndex, uint64_t *pAdapterLuid ); static void CopyTextureData( BYTE *pDst, uint32_t nDstRowPitch, const BYTE *pSrc, uint32_t nSrcRowPitch, uint32_t nWidth, uint32_t nHeight, uint32_t nPitch ); private: IDXGIFactory *m_pDXGIFactory; IDXGIOutput *m_pDXGIOutput; IDXGISwapChain *m_pDXGISwapChain; ID3D11Device *m_pD3D11Device; ID3D11DeviceContext *m_pD3D11Context; uint32_t m_nDisplayWidth, m_nDisplayHeight; int32_t m_nDisplayX, m_nDisplayY; struct SharedTextureEntry_t { HANDLE m_hSharedTexture; ID3D11Texture2D *m_pTexture; }; typedef std::vector< SharedTextureEntry_t > SharedTextures_t; SharedTextures_t m_SharedTextureCache; }; ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/AMFFactory.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "AMFFactory.h" #include "Thread.h" #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wexit-time-destructors" #pragma clang diagnostic ignored "-Wglobal-constructors" #endif AMFFactoryHelper g_AMFFactory; #ifdef __clang__ #pragma clang diagnostic pop #endif #ifdef AMF_CORE_STATIC extern "C" { extern AMF_CORE_LINK AMF_RESULT AMF_CDECL_CALL AMFInit(amf_uint64 version, amf::AMFFactory **ppFactory); } #endif //------------------------------------------------------------------------------------------------- AMFFactoryHelper::AMFFactoryHelper() : m_hDLLHandle(NULL), m_pFactory(NULL), m_pDebug(NULL), m_pTrace(NULL), m_AMFRuntimeVersion(0), m_iRefCount(0) { } //------------------------------------------------------------------------------------------------- AMFFactoryHelper::~AMFFactoryHelper() { Terminate(); } //------------------------------------------------------------------------------------------------- AMF_RESULT AMFFactoryHelper::Init(const wchar_t* dllName) { dllName; #ifndef AMF_CORE_STATIC if (m_hDLLHandle != NULL) { amf_atomic_inc(&m_iRefCount); return AMF_OK; } const wchar_t* dllName_ = dllName == NULL ? AMF_DLL_NAME : dllName; #if defined (_WIN32) || defined (__APPLE__) m_hDLLHandle = amf_load_library(dllName_); #else m_hDLLHandle = amf_load_library1(dllName_, false); //load with local flags #endif if(m_hDLLHandle == NULL) { return AMF_FAIL; } AMFInit_Fn initFun = (AMFInit_Fn)::amf_get_proc_address(m_hDLLHandle, AMF_INIT_FUNCTION_NAME); if(initFun == NULL) { return AMF_FAIL; } AMF_RESULT res = initFun(AMF_FULL_VERSION, &m_pFactory); if(res != AMF_OK) { return res; } AMFQueryVersion_Fn versionFun = (AMFQueryVersion_Fn)::amf_get_proc_address(m_hDLLHandle, AMF_QUERY_VERSION_FUNCTION_NAME); if(versionFun == NULL) { return AMF_FAIL; } res = versionFun(&m_AMFRuntimeVersion); if(res != AMF_OK) { return res; } #else AMF_RESULT res = AMFInit(AMF_FULL_VERSION, &m_pFactory); if (res != AMF_OK) { return res; } m_AMFRuntimeVersion = AMF_FULL_VERSION; #endif m_pFactory->GetTrace(&m_pTrace); m_pFactory->GetDebug(&m_pDebug); amf_atomic_inc(&m_iRefCount); return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMFFactoryHelper::Terminate() { if(m_hDLLHandle != NULL) { amf_atomic_dec(&m_iRefCount); if(m_iRefCount == 0) { amf_free_library(m_hDLLHandle); m_hDLLHandle = NULL; m_pFactory= NULL; m_pDebug = NULL; m_pTrace = NULL; } } return AMF_OK; } //------------------------------------------------------------------------------------------------- amf::AMFFactory* AMFFactoryHelper::GetFactory() { return m_pFactory; } //------------------------------------------------------------------------------------------------- amf::AMFDebug* AMFFactoryHelper::GetDebug() { return m_pDebug; } //------------------------------------------------------------------------------------------------- amf::AMFTrace* AMFFactoryHelper::GetTrace() { return m_pTrace; } //------------------------------------------------------------------------------------------------- amf_uint64 AMFFactoryHelper::AMFQueryVersion() { return m_AMFRuntimeVersion; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMFFactoryHelper::LoadExternalComponent(amf::AMFContext* pContext, const wchar_t* dll, const char* function, void* reserved, amf::AMFComponent** ppComponent) { // check passed in parameters if (!pContext || !dll || !function) { return AMF_INVALID_ARG; } // check if DLL has already been loaded amf_handle hDll = NULL; for (std::vector::iterator it = m_extComponents.begin(); it != m_extComponents.end(); ++it) { #if defined(_WIN32) if (wcsicmp(it->m_DLL.c_str(), dll) == 0) // ignore case on Windows #elif defined(__linux) // Linux if (wcscmp(it->m_DLL.c_str(), dll) == 0) // case sensitive on Linux #endif { if (it->m_hDLLHandle != NULL) { hDll = it->m_hDLLHandle; amf_atomic_inc(&it->m_iRefCount); break; } return AMF_UNEXPECTED; } } // DLL wasn't loaded before so load it now and // add it to the internal list if (hDll == NULL) { ComponentHolder component; component.m_iRefCount = 0; component.m_hDLLHandle = NULL; component.m_DLL = dll; #if defined(_WIN32) || defined(__APPLE__) hDll = amf_load_library(dll); #else hDll = amf_load_library1(dll, false); //global flag set to true #endif if (hDll == NULL) return AMF_FAIL; // since LoadLibrary succeeded add the information // into the internal list so we can properly free // the DLL later on, even if we fail to get the // required information from it... component.m_hDLLHandle = hDll; amf_atomic_inc(&component.m_iRefCount); m_extComponents.push_back(component); } // look for function we want in the dll we just loaded typedef AMF_RESULT(AMF_CDECL_CALL *AMFCreateComponentFunc)(amf::AMFContext*, void* reserved, amf::AMFComponent**); AMFCreateComponentFunc initFn = (AMFCreateComponentFunc)::amf_get_proc_address(hDll, function); if (initFn == NULL) return AMF_FAIL; return initFn(pContext, reserved, ppComponent); } //------------------------------------------------------------------------------------------------- AMF_RESULT AMFFactoryHelper::UnLoadExternalComponent(const wchar_t* dll) { if (!dll) { return AMF_INVALID_ARG; } for (std::vector::iterator it = m_extComponents.begin(); it != m_extComponents.end(); ++it) { #if defined(_WIN32) if (wcsicmp(it->m_DLL.c_str(), dll) == 0) // ignore case on Windows #elif defined(__linux) // Linux if (wcscmp(it->m_DLL.c_str(), dll) == 0) // case sensitive on Linux #endif { if (it->m_hDLLHandle == NULL) { return AMF_UNEXPECTED; } amf_atomic_dec(&it->m_iRefCount); if (it->m_iRefCount == 0) { amf_free_library(it->m_hDLLHandle); m_extComponents.erase(it); } break; } } return AMF_OK; } //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/AMFFactory.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_AMFFactory_h #define AMF_AMFFactory_h #pragma once #include "../include/core/Factory.h" #include #include class AMFFactoryHelper { public: AMFFactoryHelper(); virtual ~AMFFactoryHelper(); AMF_RESULT Init(const wchar_t* dllName = NULL); AMF_RESULT Terminate(); AMF_RESULT LoadExternalComponent(amf::AMFContext* pContext, const wchar_t* dll, const char* function, void* reserved, amf::AMFComponent** ppComponent); AMF_RESULT UnLoadExternalComponent(const wchar_t* dll); amf::AMFFactory* GetFactory(); amf::AMFDebug* GetDebug(); amf::AMFTrace* GetTrace(); amf_uint64 AMFQueryVersion(); amf_handle GetAMFDLLHandle() { return m_hDLLHandle; } protected: struct ComponentHolder { amf_handle m_hDLLHandle; amf_long m_iRefCount; std::wstring m_DLL; ComponentHolder() { m_hDLLHandle = NULL; m_iRefCount = 0; } }; amf_handle m_hDLLHandle; amf::AMFFactory* m_pFactory; amf::AMFDebug* m_pDebug; amf::AMFTrace* m_pTrace; amf_uint64 m_AMFRuntimeVersion; amf_long m_iRefCount; std::vector m_extComponents; }; extern ::AMFFactoryHelper g_AMFFactory; #endif // AMF_AMFFactory_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/AMFMath.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #pragma once #include namespace amf { // right-handed system // +y is up // +x is to the right // -z is forward const float AMF_PI = 3.141592654f; const float AMF_1DIV2PI = 0.159154943f; const float AMF_2PI = 6.283185307f; const float AMF_PIDIV2 = 1.570796327f; const uint32_t AMF_PERMUTE_0X = 0; const uint32_t AMF_PERMUTE_0Y = 1; const uint32_t AMF_PERMUTE_0Z = 2; const uint32_t AMF_PERMUTE_0W = 3; const uint32_t AMF_PERMUTE_1X = 4; const uint32_t AMF_PERMUTE_1Y = 5; const uint32_t AMF_PERMUTE_1Z = 6; const uint32_t AMF_PERMUTE_1W = 7; const uint32_t AMF_SWIZZLE_X = 0; const uint32_t AMF_SWIZZLE_Y = 1; const uint32_t AMF_SWIZZLE_Z = 2; const uint32_t AMF_SWIZZLE_W = 3; //--------------------------------------------------------------------------------------------- class VectorPOD { public: float x; float y; float z; float w; // Vector():x(0),y(0),z(0),w(0){} // Vector(float _x, float _y, float _z, float _w ):x(_x),y(_y),z(_z),w(_w){} void Assign(float _x, float _y, float _z, float _w) { x= _x; y = _y; z = _z; w = _w; } inline VectorPOD& operator-=(const VectorPOD& other) { x -=other.x; y -=other.y; z -=other.z; w -=other.w; return *this; } inline VectorPOD operator-(const VectorPOD& other) const { VectorPOD vector; vector.x = x - other.x; vector.y = y - other.y; vector.z = z - other.z; vector.w = w - other.w; return vector; } inline VectorPOD& operator+=(const VectorPOD& other) { x +=other.x; y +=other.y; z +=other.z; w +=other.w; return *this; } inline VectorPOD operator+(const VectorPOD& other) const { VectorPOD vector; vector.x = x + other.x; vector.y = y + other.y; vector.z = z + other.z; vector.w = w + other.w; return vector; } inline VectorPOD operator*(const VectorPOD& other) const { VectorPOD vector; vector.x = x * other.x; vector.y = y * other.y; vector.z = z * other.z; vector.w = w * other.w; return vector; } inline VectorPOD operator*=(const VectorPOD& other) { x*=other.x; y*=other.y; z*=other.z; w*=other.w; return *this; } inline VectorPOD Swizzle(uint32_t E0, uint32_t E1, uint32_t E2, uint32_t E3) const { const uint32_t *aPtr = (const uint32_t* )(this); VectorPOD Result; uint32_t *pWork = (uint32_t*)(&Result); pWork[0] = aPtr[E0]; pWork[1] = aPtr[E1]; pWork[2] = aPtr[E2]; pWork[3] = aPtr[E3]; return Result; } /* inline VectorPOD& operator=(const VectorPOD& other) { Assign(other.x, other.y, other.z, other.w); return *this; } */ inline bool operator==(const VectorPOD& other) const { return x == other.x && y == other.y && z == other.z && w == other.w; } inline bool operator!=(const VectorPOD& other) const { return !operator==(other); } inline VectorPOD Dot3(const VectorPOD& vec) const { float fValue = x * vec.x + y * vec.y + z * vec.z; VectorPOD Result; Result.Assign(fValue, fValue, fValue, fValue); return Result; } inline VectorPOD Dot4(const VectorPOD& vec) const { float fValue = x * vec.x + y * vec.y + z * vec.z + w * vec.w; VectorPOD Result; Result.Assign(fValue, fValue, fValue, fValue); return Result; } inline VectorPOD LengthSq3() const { return Dot3(*this); } inline VectorPOD LengthSq4() const { return Dot4(*this); } inline VectorPOD Sqrt() const { VectorPOD Result; Result.x = sqrtf(x); Result.y = sqrtf(y); Result.z = sqrtf(z); Result.w = sqrtf(w); return Result; } inline VectorPOD Length3() const { VectorPOD Result; Result = LengthSq3(); Result = Result.Sqrt(); return Result; } inline VectorPOD Length4() const { VectorPOD Result; Result = LengthSq4(); Result = Result.Sqrt(); return Result; } inline VectorPOD Normalize3() const { float fLength; VectorPOD vResult; vResult = Length3(); fLength = vResult.x; // Prevent divide by zero if (fLength > 0) { fLength = 1.0f / fLength; } vResult.x = x * fLength; vResult.y = y * fLength; vResult.z = z * fLength; vResult.w = w * fLength; return vResult; } inline VectorPOD Cross3(const VectorPOD& vec) const { VectorPOD vResult; vResult.Assign( (y * vec.z) - (z * vec.y), (z * vec.x) - (x * vec.z), (x * vec.y) - (y * vec.x), 0.0f); return vResult; } inline VectorPOD Negate() const { VectorPOD Result; Result.x = -x; Result.y = -y; Result.z = -z; Result.w = -w; return Result; } inline VectorPOD operator-() const { return Negate(); } inline VectorPOD MergeXY(const VectorPOD& vec) const { VectorPOD Result; Result.x = x; Result.y = vec.x; Result.z = y; Result.w = vec.y; return Result; } inline VectorPOD MergeZW(const VectorPOD& vec) const { VectorPOD Result; Result.x = z; Result.y = vec.z; Result.z = w; Result.w = vec.w; return Result; } inline VectorPOD VectorPermute(const VectorPOD& vec, uint32_t PermuteX, uint32_t PermuteY, uint32_t PermuteZ, uint32_t PermuteW ) const { const uint32_t *aPtr[2]; aPtr[0] = (const uint32_t* )(this); aPtr[1] = (const uint32_t* )(&vec); VectorPOD Result; uint32_t *pWork = (uint32_t*)(&Result); const uint32_t i0 = PermuteX & 3; const uint32_t vi0 = PermuteX >> 2; pWork[0] = aPtr[vi0][i0]; const uint32_t i1 = PermuteY & 3; const uint32_t vi1 = PermuteY >> 2; pWork[1] = aPtr[vi1][i1]; const uint32_t i2 = PermuteZ & 3; const uint32_t vi2 = PermuteZ >> 2; pWork[2] = aPtr[vi2][i2]; const uint32_t i3 = PermuteW & 3; const uint32_t vi3 = PermuteW >> 2; pWork[3] = aPtr[vi3][i3]; return Result; } inline VectorPOD Reciprocal() { VectorPOD Result; Result.x = 1.f / x; Result.y = 1.f / y; Result.z = 1.f / z; Result.w = 1.f / w; return Result; } }; class Vector : public VectorPOD { public: Vector() { x = 0; y = 0; z = 0; w = 0; } Vector(float _x, float _y, float _z, float _w ) { x = _x; y = _y; z = _z; w = _w; } Vector(const VectorPOD& other) { operator=(other); } Vector& operator=(const VectorPOD& other) { x = other.x; y = other.y; z = other.z; w = other.w; return *this; } //--------------------------------------------------------------------------------------------- }; //--------------------------------------------------------------------------------------------- class Quaternion : public Vector { public: Quaternion(){} Quaternion(const Quaternion& other){Assign(other.x, other.y, other.z, other.w);} Quaternion(float pitch, float yaw, float roll) {FromEuler(pitch, yaw, roll);} Quaternion(float _x, float _y, float _z, float _w) {Assign(_x, _y, _z, _w);} inline void FromEuler(float pitch, float yaw, float roll) { float cy = cosf(yaw * 0.5f); float sy = sinf(yaw * 0.5f); float cr = cosf(roll * 0.5f); float sr = sinf(roll * 0.5f); float cp = cosf(pitch * 0.5f); float sp = sinf(pitch * 0.5f); w = cp * cr * cy + sp * sr * sy; x = cp * sr * cy - sp * cr * sy; y = cp * cr * sy + sp * sr * cy; z = sp * cr * cy - cp * sr * sy; } inline Quaternion& operator=(const Quaternion& other) { Assign(other.x, other.y, other.z, other.w); return *this; } inline bool operator==(const Quaternion& other) const { return x == other.x && y == other.y && z == other.z && w == other.w; } inline bool operator!=(const Quaternion& other) const { return !operator==(other); } inline Quaternion operator*(const Quaternion& other) const { return Quaternion( other.w * x + other.x * w + other.y * z - other.z * y, other.w * y - other.x * z + other.y * w + other.z * x, other.w * z + other.x * y - other.y * x + other.z * w, other.w * w - other.x * x - other.y * y - other.z * z ); } inline const Quaternion& RotateBy(const Quaternion& rotator) { *this = rotator * (*this); return *this; } inline Vector ToEulerAngles() const { float yaw, pitch, roll; #if 0 // roll (x-axis rotation) float sinr = 2.0f * (w * x + y * z); float cosr = 1.0f - 2.0f * (x * x + y * y); roll = atan2f(sinr, cosr); // pitch (y-axis rotation) float sinp = 2.0f * (w * y - z * x); if (fabsf(sinp) >= 1) pitch = copysignf(AMF_PIDIV2, sinp); // use 90 degrees if out of range else pitch = asinf(sinp); // yaw (z-axis rotation) float siny = 2.0f * (w * z + x * y); float cosy = 1.0f - 2.0f * (y * y + z * z); yaw = atan2f(siny, cosy); #else float sqw = w*w; float sqx = x*x; float sqy = y*y; float sqz = z*z; float unit = sqx + sqy + sqz + sqw; // if normalised is one, otherwise is correction factor float test = x*y + z*w; if (test > 0.499f * unit) { // singularity at north pole yaw = 2.0f * atan2(x, w); pitch = AMF_PIDIV2; roll = 0; }else if (test < -0.499f*unit) { // singularity at south pole yaw = -2.0f * atan2(x, w); pitch = -AMF_PIDIV2; roll = 0; } else { yaw = atan2(2.0f * (y*w - x*z), sqx - sqy - sqz + sqw); pitch = asin(2.0f * test / unit); roll = atan2(2.0f * (x*w - y * z), -sqx + sqy - sqz + sqw); } #endif return Vector(pitch, yaw, roll, 0); } inline Quaternion& operator-=(const Quaternion& other) { x -= other.x; y -= other.y; z -= other.z; w -= other.w; return *this; } inline Quaternion operator-(const Quaternion& other) const { Quaternion vector; vector.x = x - other.x; vector.y = y - other.y; vector.z = z - other.z; vector.w = w - other.w; return vector; } inline Quaternion& operator+=(const Quaternion& other) { x += other.x; y += other.y; z += other.z; w += other.w; return *this; } inline Quaternion operator+(const Quaternion& other) const { Quaternion vector; vector.x = x + other.x; vector.y = y + other.y; vector.z = z + other.z; vector.w = w + other.w; return vector; } inline Quaternion Conjugate() const { Quaternion result(-x, -y, -z, w); return result; } inline Vector DistanceAngles(const Quaternion& newValue) const { Vector diff; amf::Quaternion diffQ = newValue * Conjugate(); float len = diffQ.Length4().x; if (len <= 0.0005f) { diff = amf::Vector(2.0f * diffQ.x, 2.0f * diffQ.y, 2.0f * diffQ.z, 0); } else { float angle = 2.0f * atan2(len, diffQ.w); diff = amf::Vector(diffQ.x * angle / len, diffQ.y * angle / len, diffQ.z * angle / len, 0); } return diff; } }; inline void ScalarSinCos(float* pSin, float* pCos, float Value) { // Map Value to y in [-pi,pi], x = 2*pi*quotient + remainder. float quotient = AMF_1DIV2PI *Value; if (Value >= 0.0f) { quotient = (float)((int)(quotient + 0.5f)); } else { quotient = (float)((int)(quotient - 0.5f)); } float y = Value - AMF_2PI * quotient; // Map y to [-pi/2,pi/2] with sin(y) = sin(Value). float sign; if (y > AMF_PIDIV2) { y = AMF_PI - y; sign = -1.0f; } else if (y < -AMF_PIDIV2) { y = -AMF_PI - y; sign = -1.0f; } else { sign = +1.0f; } float y2 = y * y; // 11-degree minimax approximation *pSin = ( ( ( ( (-2.3889859e-08f * y2 + 2.7525562e-06f) * y2 - 0.00019840874f ) * y2 + 0.0083333310f ) * y2 - 0.16666667f ) * y2 + 1.0f ) * y; // 10-degree minimax approximation float p = ( ( ( ( -2.6051615e-07f * y2 + 2.4760495e-05f ) * y2 - 0.0013888378f ) * y2 + 0.041666638f ) * y2 - 0.5f ) * y2 + 1.0f; *pCos = sign*p; } //--------------------------------------------------------------------------------------------- class Matrix { public: union { float m[4][4]; VectorPOD r[4]; float k[16]; }; Matrix() {Identity();} Matrix(float *_m) { memcpy(m, _m, sizeof(m));} Matrix(float i0, float i1, float i2, float i3, float i4, float i5, float i6, float i7, float i8, float i9, float i10, float i11, float i12, float i13, float i14, float i15) { k[0] = i0; k[1] = i1; k[2] = i2; k[3] = i3; k[4] = i4; k[5] = i5; k[6] = i6; k[7] = i7; k[8] = i8; k[9] = i9; k[10] = i10; k[11] = i11; k[12] = i12; k[13] = i13; k[14] = i14; k[15] = i15; } inline void Identity() { k[0] = k[5] = k[10] = k[15] = 1.0f; k[1] = k[2] = k[3] = k[4] = k[6] = k[7] = k[8] = k[9] = k[11] = k[12] = k[13] = k[14] = 0.0f; } inline Matrix &operator=(const Matrix & other) { memcpy(m, other.m, sizeof(m)); return *this; } inline Matrix operator*(const Matrix& n) const { return Matrix( k[0]*n.k[0] + k[4]*n.k[1] + k[8]*n.k[2] + k[12]*n.k[3], k[1]*n.k[0] + k[5]*n.k[1] + k[9]*n.k[2] + k[13]*n.k[3], k[2]*n.k[0] + k[6]*n.k[1] + k[10]*n.k[2] + k[14]*n.k[3], k[3]*n.k[0] + k[7]*n.k[1] + k[11]*n.k[2] + k[15]*n.k[3], k[0]*n.k[4] + k[4]*n.k[5] + k[8]*n.k[6] + k[12]*n.k[7], k[1]*n.k[4] + k[5]*n.k[5] + k[9]*n.k[6] + k[13]*n.k[7], k[2]*n.k[4] + k[6]*n.k[5] + k[10]*n.k[6] + k[14]*n.k[7], k[3]*n.k[4] + k[7]*n.k[5] + k[11]*n.k[6] + k[15]*n.k[7], k[0]*n.k[8] + k[4]*n.k[9] + k[8]*n.k[10] + k[12]*n.k[11], k[1]*n.k[8] + k[5]*n.k[9] + k[9]*n.k[10] + k[13]*n.k[11], k[2]*n.k[8] + k[6]*n.k[9] + k[10]*n.k[10] + k[14]*n.k[11], k[3]*n.k[8] + k[7]*n.k[9] + k[11]*n.k[10] + k[15]*n.k[11], k[0]*n.k[12] + k[4]*n.k[13] + k[8]*n.k[14] + k[12]*n.k[15], k[1]*n.k[12] + k[5]*n.k[13] + k[9]*n.k[14] + k[13]*n.k[15], k[2]*n.k[12] + k[6]*n.k[13] + k[10]*n.k[14] + k[14]*n.k[15], k[3]*n.k[12] + k[7]*n.k[13] + k[11]*n.k[14] + k[15]*n.k[15]); } inline Matrix operator*=(const Matrix& other) { *this = *this * other; return *this; } inline bool operator==(const Matrix& other) const { return memcmp(this, &other, sizeof(*this)) == 0; } inline bool operator!=(const Matrix& other) const { return memcmp(this, &other, sizeof(*this)) != 0; } inline Vector operator*(const Vector& v) const { Vector Z(v.z, v.z, v.z, v.z); Vector Y(v.y, v.y, v.y, v.y); Vector X(v.x, v.x, v.x, v.x); Vector ret; ret = Z * r[2] + r[3]; ret = Y * r[1] + ret; ret = X * r[0] + ret; return ret; } void MatrixAffineTransformation(const Vector &Scaling, const Vector &RotationOrigin, const Vector &RotationQuaternion, const Vector &Translation) { // M = MScaling * Inverse(MRotationOrigin) * MRotation * MRotationOrigin * MTranslation; MatrixScalingFromVector(Scaling); Vector VRotationOrigin (RotationOrigin.x,RotationOrigin.y,RotationOrigin.z, 0); Matrix MRotation; MRotation.MatrixRotationQuaternion(RotationQuaternion); Vector VTranslation (Translation.x, Translation.y, Translation.z, 0); r[3] -= VRotationOrigin; *this *= MRotation; r[3] += VRotationOrigin; r[3] += VTranslation; } inline void MatrixScalingFromVector(const Vector& Scale) { m[0][0] = Scale.x; m[1][1] = Scale.y; m[2][2] = Scale.z; m[3][3] = 1.0f; } void MatrixRotationQuaternion(const Vector& Quaternion) { static const Vector Constant1110 = {1.0f, 1.0f, 1.0f, 0.0f}; Vector Q0 = Quaternion + Quaternion; Vector Q1 = Quaternion * Q0; Vector V0 = Q1.VectorPermute(Constant1110, AMF_PERMUTE_0Y, AMF_PERMUTE_0X, AMF_PERMUTE_0X, AMF_PERMUTE_1W); Vector V1 = Q1.VectorPermute(Constant1110, AMF_PERMUTE_0Z, AMF_PERMUTE_0Z, AMF_PERMUTE_0Y, AMF_PERMUTE_1W); Vector R0 = Constant1110 - V0; R0 = R0 - V1; V0 = Quaternion.Swizzle(AMF_SWIZZLE_X, AMF_SWIZZLE_X, AMF_SWIZZLE_Y, AMF_SWIZZLE_W); V1 = Q0.Swizzle(AMF_SWIZZLE_Z, AMF_SWIZZLE_Y, AMF_SWIZZLE_Z, AMF_SWIZZLE_W); V0 = V0 * V1; V1 = Vector(Quaternion.w, Quaternion.w, Quaternion.w, Quaternion.w); Vector V2 = Q0.Swizzle(AMF_SWIZZLE_Y, AMF_SWIZZLE_Z, AMF_SWIZZLE_X, AMF_SWIZZLE_W); V1 = V1 * V2; Vector R1 = V0 + V1; Vector R2 = V0 - V1; V0 = R1.VectorPermute(R2, AMF_PERMUTE_0Y, AMF_PERMUTE_1X, AMF_PERMUTE_1Y, AMF_PERMUTE_0Z); V1 = R1.VectorPermute(R2, AMF_PERMUTE_0X, AMF_PERMUTE_1Z, AMF_PERMUTE_0X, AMF_PERMUTE_1Z); r[0] = R0.VectorPermute(V0, AMF_PERMUTE_0X, AMF_PERMUTE_1X, AMF_PERMUTE_1Y, AMF_PERMUTE_0W); r[1] = R0.VectorPermute(V0, AMF_PERMUTE_1Z, AMF_PERMUTE_0Y, AMF_PERMUTE_1W, AMF_PERMUTE_0W); r[2] = R0.VectorPermute(V1, AMF_PERMUTE_1X, AMF_PERMUTE_1Y, AMF_PERMUTE_0Z, AMF_PERMUTE_0W); r[3] = Vector(0.0f, 0.0f, 0.0f, 1.0f); } inline void LookToLH(const Vector& EyePosition, const Vector& EyeDirection, const Vector& UpDirection) { Vector R2 = EyeDirection.Normalize3(); Vector R0 = UpDirection.Cross3(R2); R0 = R0.Normalize3(); Vector R1 = R2.Cross3(R0); Vector NegEyePosition = EyePosition.Negate(); Vector D0 = R0.Dot3(NegEyePosition); Vector D1 = R1.Dot3(NegEyePosition); Vector D2 = R2.Dot3(NegEyePosition); Matrix M; M.r[0] = Vector(R0.x, R0.y, R0.z, D0.w); M.r[1] = Vector(R1.x, R1.y, R1.z, D1.w); M.r[2] = Vector(R2.x, R2.y, R2.z, D2.w); M.r[3] = Vector(0.0f, 0.0f, 0.0f, 1.0f); *this = M.Transpose(); } inline Matrix Transpose() const { // Original matrix: // // m00m01m02m03 // m10m11m12m13 // m20m21m22m23 // m30m31m32m33 Matrix P; P.r[0] = r[0].MergeXY(r[2]); // m00m20m01m21 P.r[1] = r[1].MergeXY(r[3]); // m10m30m11m31 P.r[2] = r[0].MergeZW(r[2]); // m02m22m03m23 P.r[3] = r[1].MergeZW(r[3]); // m12m32m13m33 Matrix MT; MT.r[0] = P.r[0].MergeXY(P.r[1]); // m00m10m20m30 MT.r[1] = P.r[0].MergeZW(P.r[1]); // m01m11m21m31 MT.r[2] = P.r[2].MergeXY(P.r[3]); // m02m12m22m32 MT.r[3] = P.r[2].MergeZW(P.r[3]); // m03m13m23m33 return MT; } inline void LookAtLH(Vector& EyePosition, Vector& FocusPosition, Vector& UpDirection) { Vector EyeDirection = FocusPosition - EyePosition; LookToLH(EyePosition, EyeDirection, UpDirection); } inline void PerspectiveFovLH(float FovAngleY, float AspectRatio, float NearZ, float FarZ) { float SinFov; float CosFov; ScalarSinCos(&SinFov, &CosFov, 0.5f * FovAngleY); float Height = CosFov / SinFov; float Width = Height / AspectRatio; float fRange = FarZ / (FarZ-NearZ); m[0][0] = Width; m[0][1] = 0.0f; m[0][2] = 0.0f; m[0][3] = 0.0f; m[1][0] = 0.0f; m[1][1] = Height; m[1][2] = 0.0f; m[1][3] = 0.0f; m[2][0] = 0.0f; m[2][1] = 0.0f; m[2][2] = fRange; m[2][3] = 1.0f; m[3][0] = 0.0f; m[3][1] = 0.0f; m[3][2] = -fRange * NearZ; m[3][3] = 0.0f; } inline void RotationRollPitchYaw(float Pitch, float Yaw, float Roll) { Quaternion Q; Q.FromEuler( Pitch, Yaw, Roll); MatrixAffineTransformation(Vector(1.f, 1.f, 1.f, 0.f), Vector(), Q, Vector()); } inline Vector Determinant() { static const Vector Sign (1.0f, -1.0f, 1.0f, -1.0f); Vector V0(r[2].y, r[2].x, r[2].x, r[2].x); Vector V1(r[3].z, r[3].z, r[3].y, r[3].y); Vector V2(r[2].y, r[2].x, r[2].x, r[2].x); Vector V3(r[3].w, r[3].w, r[3].w, r[3].z); Vector V4(r[2].z, r[2].z, r[2].y, r[2].y); Vector V5(r[3].w, r[3].w, r[3].w, r[3].z); Vector P0 = V0 * V1; Vector P1 = V2 * V3; Vector P2 = V4 * V5; V0 = Vector(r[2].z, r[2].z, r[2].y, r[2].y); V1 = Vector(r[3].y, r[3].x, r[3].x, r[3].x); V2 = Vector(r[2].w, r[2].w, r[2].w, r[2].z); V3 = Vector(r[3].y, r[3].x, r[3].x, r[3].x); V4 = Vector(r[2].w, r[2].w, r[2].w, r[2].z); V5 = Vector(r[3].z, r[3].z, r[3].y, r[3].y); P0 -= V0 * V1; P1 -= V2 * V3; P2 -= V4 * V5; V0 = Vector(r[1].w, r[1].w, r[1].w, r[1].z); V1 = Vector(r[1].z, r[1].z, r[1].y, r[1].y); V2 = Vector(r[1].y, r[1].x, r[1].x, r[1].x); Vector S = r[0] * Sign; Vector R = V0 * P0; R -= V1 * P1; R += V2 * P2; return S.Dot4(R); } #define XM3RANKDECOMPOSE(a, b, c, x, y, z) \ if((x) < (y)) \ { \ if((y) < (z)) \ { \ (a) = 2; \ (b) = 1; \ (c) = 0; \ } \ else \ { \ (a) = 1; \ \ if((x) < (z)) \ { \ (b) = 2; \ (c) = 0; \ } \ else \ { \ (b) = 0; \ (c) = 2; \ } \ } \ } \ else \ { \ if((x) < (z)) \ { \ (a) = 2; \ (b) = 0; \ (c) = 1; \ } \ else \ { \ (a) = 0; \ \ if((y) < (z)) \ { \ (b) = 2; \ (c) = 1; \ } \ else \ { \ (b) = 1; \ (c) = 2; \ } \ } \ } #define XM3_DECOMP_EPSILON 0.0001f inline amf::Quaternion ConvertMatrixToQuat() { amf::Quaternion q; float r22 = m[2][2]; if (r22 <= 0.f) // x^2 + y^2 >= z^2 + w^2 { float dif10 = m[1][1] - m[0][0]; float omr22 = 1.f - r22; if (dif10 <= 0.f) // x^2 >= y^2 { float fourXSqr = omr22 - dif10; float inv4x = 0.5f / sqrtf(fourXSqr); q.x = fourXSqr*inv4x; q.y = (m[0][1] + m[1][0])*inv4x; q.z = (m[0][2] + m[2][0])*inv4x; q.w = (m[1][2] - m[2][1])*inv4x; } else // y^2 >= x^2 { float fourYSqr = omr22 + dif10; float inv4y = 0.5f / sqrtf(fourYSqr); q.x = (m[0][1] + m[1][0])*inv4y; q.y = fourYSqr*inv4y; q.z = (m[1][2] + m[2][1])*inv4y; q.w = (m[2][0] - m[0][2])*inv4y; } } else // z^2 + w^2 >= x^2 + y^2 { float sum10 = m[1][1] + m[0][0]; float opr22 = 1.f + r22; if (sum10 <= 0.f) // z^2 >= w^2 { float fourZSqr = opr22 - sum10; float inv4z = 0.5f / sqrtf(fourZSqr); q.x = (m[0][2] + m[2][0])*inv4z; q.y = (m[1][2] + m[2][1])*inv4z; q.z = fourZSqr*inv4z; q.w = (m[0][1] - m[1][0])*inv4z; } else // w^2 >= z^2 { float fourWSqr = opr22 + sum10; float inv4w = 0.5f / sqrtf(fourWSqr); q.x = (m[1][2] - m[2][1])*inv4w; q.y = (m[2][0] - m[0][2])*inv4w; q.z = (m[0][1] - m[1][0])*inv4w; q.w = fourWSqr*inv4w; } } return q; } inline bool DecomposeMatrix(amf::Quaternion &q, amf::Vector &p, amf::Vector &s) { static amf::Vector amfXMIdentityR0( 1.0f, 0.0f, 0.0f, 0.0f ); static amf::Vector amfXMIdentityR1( 0.0f, 1.0f, 0.0f, 0.0f ); static amf::Vector amfXMIdentityR2( 0.0f, 0.0f, 1.0f, 0.0f ); static const amf::VectorPOD *pvCanonicalBasis[3] = { &amfXMIdentityR0, &amfXMIdentityR1, &amfXMIdentityR2 }; // p.Assign(-m.m[0][3], -m.m[1][3], -m.m[2][3], 0); p = r[3]; amf::VectorPOD *ppvBasis[3]; amf::Matrix matTemp; ppvBasis[0] = &matTemp.r[0]; ppvBasis[1] = &matTemp.r[1]; ppvBasis[2] = &matTemp.r[2]; matTemp.r[0] = r[0]; matTemp.r[1] = r[1]; matTemp.r[2] = r[2]; matTemp.r[3] = amf::Vector(0.0f, 0.0f, 0.0f, 1.0f ); float *pfScales = (float*)&s; size_t a, b, c; pfScales[0] = ppvBasis[0][0].Length3().x; pfScales[1] = ppvBasis[1][0].Length3().x; pfScales[2] = ppvBasis[2][0].Length3().x; pfScales[3] = 0.f; XM3RANKDECOMPOSE(a, b, c, pfScales[0], pfScales[1], pfScales[2]) if(pfScales[a] < XM3_DECOMP_EPSILON) { ppvBasis[a][0] = pvCanonicalBasis[a][0]; } ppvBasis[a][0] = ppvBasis[a][0].Normalize3(); if(pfScales[b] < XM3_DECOMP_EPSILON) { size_t aa, bb, cc; float fAbsX, fAbsY, fAbsZ; fAbsX = fabsf(ppvBasis[a][0].x); fAbsY = fabsf(ppvBasis[a][0].y); fAbsZ = fabsf(ppvBasis[a][0].z); XM3RANKDECOMPOSE(aa, bb, cc, fAbsX, fAbsY, fAbsZ) ppvBasis[b][0] = ppvBasis[a][0].Cross3(pvCanonicalBasis[cc][0]); } ppvBasis[b][0] = ppvBasis[b][0].Normalize3(); if(pfScales[c] < XM3_DECOMP_EPSILON) { ppvBasis[c][0] = ppvBasis[a][0].Cross3(ppvBasis[b][0]); } ppvBasis[c][0] = ppvBasis[c][0].Normalize3(); float fDet = matTemp.Determinant().x; // use Kramer's rule to check for handedness of coordinate system if(fDet < 0.0f) { // switch coordinate system by negating the scale and inverting the basis vector on the x-axis pfScales[a] = -pfScales[a]; ppvBasis[a][0] = ppvBasis[a][0].Negate(); fDet = -fDet; } fDet -= 1.0f; fDet *= fDet; if(XM3_DECOMP_EPSILON < fDet) { // Non-SRT matrix encountered return false; } q = matTemp.ConvertMatrixToQuat(); return true; } inline Matrix Inverse(Vector *pDeterminant) { float A2323 = m[2][2] * m[3][3] - m[2][3] * m[3][2]; float A1323 = m[2][1] * m[3][3] - m[2][3] * m[3][1]; float A1223 = m[2][1] * m[3][2] - m[2][2] * m[3][1]; float A0323 = m[2][0] * m[3][3] - m[2][3] * m[3][0]; float A0223 = m[2][0] * m[3][2] - m[2][2] * m[3][0]; float A0123 = m[2][0] * m[3][1] - m[2][1] * m[3][0]; float A2313 = m[1][2] * m[3][3] - m[1][3] * m[3][2]; float A1313 = m[1][1] * m[3][3] - m[1][3] * m[3][1]; float A1213 = m[1][1] * m[3][2] - m[1][2] * m[3][1]; float A2312 = m[1][2] * m[2][3] - m[1][3] * m[2][2]; float A1312 = m[1][1] * m[2][3] - m[1][3] * m[2][1]; float A1212 = m[1][1] * m[2][2] - m[1][2] * m[2][1]; float A0313 = m[1][0] * m[3][3] - m[1][3] * m[3][0]; float A0213 = m[1][0] * m[3][2] - m[1][2] * m[3][0]; float A0312 = m[1][0] * m[2][3] - m[1][3] * m[2][0]; float A0212 = m[1][0] * m[2][2] - m[1][2] * m[2][0]; float A0113 = m[1][0] * m[3][1] - m[1][1] * m[3][0]; float A0112 = m[1][0] * m[2][1] - m[1][1] * m[2][0]; float det = m[0][0] * (m[1][1] * A2323 - m[1][2] * A1323 + m[1][3] * A1223) - m[0][1] * (m[1][0] * A2323 - m[1][2] * A0323 + m[1][3] * A0223) + m[0][2] * (m[1][0] * A1323 - m[1][1] * A0323 + m[1][3] * A0123) - m[0][3] * (m[1][0] * A1223 - m[1][1] * A0223 + m[1][2] * A0123); det = 1.0f / det; Matrix ret; ret.m[0][0] = det * (m[1][1] * A2323 - m[1][2] * A1323 + m[1][3] * A1223); ret.m[0][1] = det * -(m[0][1] * A2323 - m[0][2] * A1323 + m[0][3] * A1223); ret.m[0][2] = det * (m[0][1] * A2313 - m[0][2] * A1313 + m[0][3] * A1213); ret.m[0][3] = det * -(m[0][1] * A2312 - m[0][2] * A1312 + m[0][3] * A1212); ret.m[1][0] = det * -(m[1][0] * A2323 - m[1][2] * A0323 + m[1][3] * A0223); ret.m[1][1] = det * (m[0][0] * A2323 - m[0][2] * A0323 + m[0][3] * A0223); ret.m[1][2] = det * -(m[0][0] * A2313 - m[0][2] * A0313 + m[0][3] * A0213); ret.m[1][3] = det * (m[0][0] * A2312 - m[0][2] * A0312 + m[0][3] * A0212); ret.m[2][0] = det * (m[1][0] * A1323 - m[1][1] * A0323 + m[1][3] * A0123); ret.m[2][1] = det * -(m[0][0] * A1323 - m[0][1] * A0323 + m[0][3] * A0123); ret.m[2][2] = det * (m[0][0] * A1313 - m[0][1] * A0313 + m[0][3] * A0113); ret.m[2][3] = det * -(m[0][0] * A1312 - m[0][1] * A0312 + m[0][3] * A0112); ret.m[3][0] = det * -(m[1][0] * A1223 - m[1][1] * A0223 + m[1][2] * A0123); ret.m[3][1] = det * (m[0][0] * A1223 - m[0][1] * A0223 + m[0][2] * A0123); ret.m[3][2] = det * -(m[0][0] * A1213 - m[0][1] * A0213 + m[0][2] * A0113); ret.m[3][3] = det * (m[0][0] * A1212 - m[0][1] * A0212 + m[0][2] * A0112); if (pDeterminant != nullptr) { *pDeterminant = Vector(det,det,det,det); } return ret; } }; class Pose { public: typedef enum ValidityFlagBits { // validity flags PF_NONE = 0x0000, PF_ORIENTATION = 0x0001, PF_POSITION = 0x0002, PF_ORIENTATION_VELOCITY = 0x0004, PF_POSITION_VELOCITY = 0x0008, PF_ORIENTATION_ACCELERATION = 0x0010, PF_POSITION_ACCELERATION = 0x0020, PF_COORDINATES = PF_ORIENTATION | PF_POSITION, PF_VELOCITY = PF_ORIENTATION_VELOCITY | PF_POSITION_VELOCITY, PF_ACCELERATION = PF_ORIENTATION_ACCELERATION | PF_POSITION_ACCELERATION, } ValidityFlagBits; typedef uint32_t ValidityFlags; Pose() : m_ValidityFlags(PF_NONE){} Pose(const Pose &other) { *this = other; } Pose(const amf::Quaternion& orientation, const amf::Vector& position) { Set(orientation, position); } Pose(const amf::Quaternion& orientation, const amf::Vector& position, const amf::Vector& orientationVelocity, const amf::Vector& positionVelocity) { Set(orientation, position, orientationVelocity, positionVelocity); } Pose(const amf::Quaternion& orientation, const amf::Vector& position, const amf::Vector& orientationVelocity, const amf::Vector& positionVelocity, const amf::Vector& orientationAcceleration, const amf::Vector& positionAcceleration) { Set(orientation, position, orientationVelocity, positionVelocity, orientationAcceleration, positionAcceleration); } void Set(const amf::Quaternion& orientation, const amf::Vector& position) { m_Orientation = orientation; m_Position = position; m_ValidityFlags = PF_COORDINATES; } void Set(const amf::Quaternion& orientation, const amf::Vector& position, const amf::Vector& orientationVelocity, const amf::Vector& positionVelocity) { m_Orientation = orientation; m_Position = position; m_OrientationVelocity = orientationVelocity; m_PositionVelocity = positionVelocity; m_ValidityFlags = PF_COORDINATES | PF_VELOCITY; } void Set(const amf::Quaternion& orientation, const amf::Vector& position, const amf::Vector& orientationVelocity, const amf::Vector& positionVelocity, const amf::Vector& orientationAcceleration, const amf::Vector& positionAcceleration) { m_Orientation = orientation; m_Position = position; m_OrientationVelocity = orientationVelocity; m_PositionVelocity = positionVelocity; m_OrientationAcceleration = orientationAcceleration; m_PositionAcceleration = positionAcceleration; m_ValidityFlags = PF_COORDINATES | PF_VELOCITY | PF_ACCELERATION; } inline const amf::Quaternion& GetOrientation() const { return m_Orientation; } inline const amf::Vector& GetPosition() const { return m_Position; } inline const amf::Vector& GetOrientationVelocity() const { return m_OrientationVelocity; } inline const amf::Vector& GetPositionVelocity() const { return m_PositionVelocity; } inline const amf::Vector& GetOrientationAcceleration() const { return m_OrientationAcceleration; } inline const amf::Vector& GetPositionAcceleration() const { return m_PositionAcceleration; } inline ValidityFlags GetValidityFlags() const { return m_ValidityFlags; } inline void SetOrientation(const amf::Quaternion& orienation) { m_Orientation = orienation; m_ValidityFlags |= PF_ORIENTATION; } inline void SetPosition(const amf::Vector& position) { m_Position = position; m_ValidityFlags |= PF_POSITION; } inline void SetOrientationVelocity(const amf::Vector& orientationVelocity) { m_OrientationVelocity = orientationVelocity; m_ValidityFlags |= PF_ORIENTATION_VELOCITY; } inline void SetPositionVelocity(const amf::Vector& positionVelocity) { m_PositionVelocity = positionVelocity; m_ValidityFlags |= PF_POSITION_VELOCITY; } inline void SetOrientationAcceleration(const amf::Vector& orientationAcceleration) { m_OrientationAcceleration = orientationAcceleration; m_ValidityFlags |= PF_ORIENTATION_ACCELERATION; } inline void SetPositionAcceleration(const amf::Vector& positionAcceleration) { m_PositionAcceleration = positionAcceleration; m_ValidityFlags |= PF_POSITION_ACCELERATION; } protected: amf::Quaternion m_Orientation; amf::Vector m_Position; amf::Vector m_OrientationVelocity; amf::Vector m_PositionVelocity; amf::Vector m_OrientationAcceleration; amf::Vector m_PositionAcceleration; ValidityFlags m_ValidityFlags; }; //------------------------------------------------------------------------------------------------- template class AlphaFilter { public: AlphaFilter(T alpha) : m_Alpha(alpha), m_FilteredValue(0) { } T Apply(T value) { m_FilteredValue = m_FilteredValue + m_Alpha * (value - m_FilteredValue); return m_FilteredValue; } private: T m_Alpha; T m_FilteredValue; }; //------------------------------------------------------------------------------------------------- template class AlphaBetaFilter { public: AlphaBetaFilter(T alpha, T beta) : m_Alpha(alpha), m_Beta(beta), m_Value(0), m_PrevValue(0), m_Velocity(0) { } T Apply(T value, T dt) { m_PrevValue = m_Value; m_Value += m_Velocity * dt; T rk = value - m_Value; m_Value += m_Alpha * rk; m_Velocity += (m_Beta * rk) / dt; return m_Value; } inline T GetVelocity() const { return m_Velocity; } private: T m_Alpha, m_Beta; T m_Value, m_PrevValue, m_Velocity; }; //------------------------------------------------------------------------------------------------- template class ThresholdFilter { public: ThresholdFilter(T threshold) : m_Threshold(threshold) { } T Apply(T value) const { T result = value; if (std::abs(value) < m_Threshold) { result = T(0); } return result; } private: T m_Threshold; }; //------------------------------------------------------------------------------------------------- class Derivative { public: Derivative() {} inline static float Calculate(float newVal, float oldVal, float dt) { return (newVal - oldVal) / dt; } inline static float Calculate(float dx, float dt) { return dx / dt; } static amf::Vector Calculate(const amf::Vector& newVal, const amf::Vector& oldVal, float dt) { amf::Vector result; result.w = Calculate(newVal.w, oldVal.w, dt); result.x = Calculate(newVal.x, oldVal.x, dt); result.y = Calculate(newVal.y, oldVal.y, dt); result.z = Calculate(newVal.z, oldVal.z, dt); return result; } static amf::Vector Calculate(const amf::Vector& dx, float dt) { amf::Vector result; result.w = Calculate(dx.w, dt); result.x = Calculate(dx.x, dt); result.y = Calculate(dx.y, dt); result.z = Calculate(dx.z, dt); return result; } }; //--------------------------------------------------------------------------------------------- } // namespace amf ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/AMFSTL.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "AMFSTL.h" #include #include #include #include #include #include #include #include #include #include #if defined(__ANDROID__) #include #endif #if !defined(__APPLE__) && !defined(_WIN32) #include #endif #pragma warning(disable: 4996) #if defined(__linux) || defined(__APPLE__) extern "C" { extern int vscwprintf(const wchar_t* p_fmt, va_list p_args); extern int vscprintf(const char* p_fmt, va_list p_args); } #endif #ifdef _MSC_VER #define snprintf _snprintf #define vscprintf _vscprintf #define vscwprintf _vscwprintf // Count chars without writing to string #define vswprintf _vsnwprintf #endif using namespace amf; #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wexit-time-destructors" #pragma clang diagnostic ignored "-Wglobal-constructors" #endif static const amf_string AMF_FORBIDDEN_SYMBOLS = ":? %,;@&=+$<>#\""; static const amf_string AMF_FORBIDDEN_SYMBOLS_QUERY = ":? %,;@+$<>#\""; #ifdef __clang__ #pragma clang diagnostic pop #endif //MM: in question: unwise = "{" | "}" | "|" | "\" | "^" | "[" | "]" | "`" //---------------------------------------------------------------------------------------- // string conversaion //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_from_unicode_to_utf8(const amf_wstring& str) { amf_string result; if(0 == str.size()) { return result; } #if defined(_WIN32) _configthreadlocale(_ENABLE_PER_THREAD_LOCALE); #endif const wchar_t* pwBuff = str.c_str(); #if defined(_WIN32) int Utf8BuffSize = ::WideCharToMultiByte(CP_UTF8, 0, pwBuff, -1, NULL, 0, NULL, NULL); if(0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); Utf8BuffSize = ::WideCharToMultiByte(CP_UTF8, 0, pwBuff, -1, &result[0], Utf8BuffSize, NULL, NULL); Utf8BuffSize--; #elif defined(__ANDROID__) char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); int Utf8BuffSize = str.length(); if(0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); mbstate_t mbs; mbrlen(NULL, 0, &mbs); Utf8BuffSize = 0; for( int i = 0; i < str.length(); i++) { //MM TODO Android - not implemented //int written = wcrtomb(&result[Utf8BuffSize], pwBuff[i], &mbs); result[Utf8BuffSize] = (char)(pwBuff[i]); int written = 1; // temp replacement Utf8BuffSize += written; } setlocale(LC_CTYPE, old_locale); #else char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); int Utf8BuffSize = wcstombs(NULL, pwBuff, 0); if(0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); Utf8BuffSize = wcstombs(&result[0], pwBuff, Utf8BuffSize); setlocale(LC_CTYPE, old_locale); #endif result.resize(Utf8BuffSize); return result; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_from_utf8_to_unicode(const amf_string& str) { amf_wstring result; if(0 == str.size()) { return result; } #if defined(_WIN32) _configthreadlocale(_ENABLE_PER_THREAD_LOCALE); #endif const char* pUtf8Buff = str.c_str(); #if defined(_WIN32) int UnicodeBuffSize = ::MultiByteToWideChar(CP_UTF8, 0, pUtf8Buff, -1, NULL, 0); if(0 == UnicodeBuffSize) { return result; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); UnicodeBuffSize = ::MultiByteToWideChar(CP_UTF8, 0, pUtf8Buff, -1, &result[0], UnicodeBuffSize); UnicodeBuffSize--; #elif defined(__ANDROID__) //MM on android mbstowcs cannot be used to define length char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); mbstate_t mbs; mbrlen(NULL, 0, &mbs); int len = str.length(); const char* pt = pUtf8Buff; int UnicodeBuffSize = 0; while(len > 0) { size_t length = mbrlen (pt, len, &mbs); //MM TODO Android always return 1 if((length == 0) || (length > len)) { break; } UnicodeBuffSize++; len -= length; pt += length; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); mbrlen (NULL, 0, &mbs); len = str.length(); pt = pUtf8Buff; UnicodeBuffSize = 0; while(len > 0) { size_t length = mbrlen (pt, len, &mbs); if((length == 0) || (length > len)) { break; } mbrtowc(&result[UnicodeBuffSize], pt, length, &mbs); //MM TODO Android always return 1 char UnicodeBuffSize++; len -= length; pt += length; } setlocale(LC_CTYPE, old_locale); #else char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); int UnicodeBuffSize = mbstowcs(NULL, pUtf8Buff, 0); if(0 == UnicodeBuffSize) { return result; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); UnicodeBuffSize = mbstowcs(&result[0], pUtf8Buff, UnicodeBuffSize); setlocale(LC_CTYPE, old_locale); #endif result.resize(UnicodeBuffSize); return result; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_from_unicode_to_multibyte(const amf_wstring& str) { amf_string result; if(0 == str.size()) { return result; } const wchar_t* pwBuff = str.c_str(); #if defined(__ANDROID__) std::wstring_convert> converter; result.assign(converter.to_bytes(pwBuff).c_str()); /* int Utf8BuffSize = str.length(); if(0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); mbstate_t mbs; mbrlen(NULL, 0, &mbs); Utf8BuffSize = 0; for( int i = 0; i < str.length(); i++) { //MM TODO Android - not implemented //int written = wcrtomb(&result[Utf8BuffSize], pwBuff[i], &mbs); result[Utf8BuffSize] = (char)(pwBuff[i]); int written = 1; // temp replacement Utf8BuffSize += written; } result.resize(Utf8BuffSize); */ #else amf_size Utf8BuffSize = wcstombs(NULL, pwBuff, 0); if(static_cast(-1) == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); Utf8BuffSize = wcstombs(&result[0], pwBuff, Utf8BuffSize); result.resize(Utf8BuffSize); #endif return result; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_from_multibyte_to_unicode(const amf_string& str) { amf_wstring result; if(0 == str.size()) { return result; } const char* pUtf8Buff = str.c_str(); #if defined(__ANDROID__) //MM on android mbstowcs cannot be used to define length mbstate_t mbs; mbrlen(NULL, 0, &mbs); int len = str.length(); const char* pt = pUtf8Buff; int UnicodeBuffSize = 0; while(len > 0) { size_t length = mbrlen (pt, len, &mbs); //MM TODO Android always return 1 if((length == 0) || (length > len)) { break; } UnicodeBuffSize++; len -= length; pt += length; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); mbrlen (NULL, 0, &mbs); len = str.length(); pt = pUtf8Buff; UnicodeBuffSize = 0; while(len > 0) { size_t length = mbrlen (pt, len, &mbs); if((length == 0) || (length > len)) { break; } mbrtowc(&result[UnicodeBuffSize], pt, length, &mbs); //MM TODO Android always return 1 char UnicodeBuffSize++; len -= length; pt += length; } #else amf_size UnicodeBuffSize = mbstowcs(NULL, pUtf8Buff, 0); if(0 == UnicodeBuffSize) { return result; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); UnicodeBuffSize = mbstowcs(&result[0], pUtf8Buff, UnicodeBuffSize); #endif result.resize(UnicodeBuffSize); return result; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_from_string_to_hex_string(const amf_string& str) { amf_string ret; char buf[10]; for(int i = 0; i < (int)str.length(); i++) { sprintf(buf, "%02X", (unsigned char)str[i]); ret += buf; } return ret; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_from_hex_string_to_string(const amf_string& str) { amf_string ret; char buf[3] = { 0, 0, 0 }; for(int i = 0; i < (int)str.length(); i += 2) { buf[0] = str[i]; buf[1] = str[i + 1]; int tmp = 0; sscanf(buf, "%2X", &tmp); ret += (char)tmp; } return ret; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_string_to_lower(const amf_string& str) { std::locale loc; amf_string out = str.c_str(); size_t iLen = out.length(); for(size_t i = 0; i < iLen; i++) { out[i] = std::tolower (out[i], loc); } return out; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_string_to_lower(const amf_wstring& str) { std::locale loc; amf_wstring out = str.c_str(); size_t iLen = out.length(); for(size_t i = 0; i < iLen; i++) { out[i] = std::tolower (out[i], loc); } return out; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_string_to_upper(const amf_string& str) { std::locale loc; amf_string out = str.c_str(); size_t iLen = out.length(); for(size_t i = 0; i < iLen; i++) { out[i] = std::toupper (out[i], loc); } return out; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_string_to_upper(const amf_wstring& str) { std::locale loc; amf_wstring out = str.c_str(); size_t iLen = out.length(); for(size_t i = 0; i < iLen; i++) { out[i] = std::toupper (out[i], loc); } return out; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_convert_path_to_os_accepted_path(const amf_wstring& path) { amf_wstring result = path; amf_wstring::size_type pos = 0; while(pos != amf_string::npos) { pos = result.find(L'/', pos); if(pos == amf_wstring::npos) { break; } result[pos] = PATH_SEPARATOR_WCHAR; pos++; } return result; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_convert_path_to_url_accepted_path(const amf_wstring& path) { amf_wstring result = path; amf_wstring::size_type pos = 0; while(pos != amf_string::npos) { pos = result.find(L'\\', pos); if(pos == amf_wstring::npos) { break; } result[pos] = L'/'; pos++; } return result; } //---------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------------------ amf_string AMF_STD_CALL amf::amf_from_unicode_to_url_utf8(const amf_wstring& data, bool bQuery) // converts to UTF8 and replace fobidden symbols { amf_string converted = amf_from_unicode_to_utf8(amf_convert_path_to_url_accepted_path(data)); // convert all necessary symbols to hex amf_string Result; amf_size num = converted.length(); char buf[20]; for(amf_size i = 0; i < num; i++) { if((converted[i] <= 0x20) || (converted[i] >= 0x7F) || (bQuery && ( AMF_FORBIDDEN_SYMBOLS.find(converted[i]) != amf_string::npos) ) || (!bQuery && ( AMF_FORBIDDEN_SYMBOLS_QUERY.find(converted[i]) != amf_string::npos) )) { snprintf(buf, sizeof(buf), "%%%02X", (unsigned int)(unsigned char)converted[i]); } else { buf[0] = converted[i]; buf[1] = 0; } Result += buf; } return Result; } //------------------------------------------------------------------------------------------------------------ amf_wstring AMF_STD_CALL amf::amf_from_url_utf8_to_unicode(const amf_string& data) { amf_string Result; amf_string::size_type pos = 0; while(pos != amf_string::npos) { amf_string::size_type old_pos = pos; pos = data.find('%', pos); if(pos == amf_string::npos) { Result += data.substr(old_pos); break; } if(pos - old_pos > 0) { Result += data.substr(old_pos, pos - old_pos); } char buf[5] = { '0', 'x', 0, 0, 0 }; buf[2] = data[pos + 1]; buf[3] = data[pos + 2]; char* ret = NULL; Result += (char)strtol(buf, &ret, 16); pos += 3; } amf_wstring converted = amf_from_utf8_to_unicode(Result); return converted; } //---------------------------------------------------------------------------------------------- amf_size AMF_STD_CALL amf::amf_string_ci_find(const amf_wstring& left, const amf_wstring& right, amf_size off) { amf_wstring _left = amf_string_to_lower(left); amf_wstring _right = amf_string_to_lower(right); return _left.find(_right, off); } //---------------------------------------------------------------------------------------------- amf_size AMF_STD_CALL amf::amf_string_ci_rfind(const amf_wstring& left, const amf_wstring& right, amf_size off) { amf_wstring _left = amf_string_to_lower(left); amf_wstring _right = amf_string_to_lower(right); return _left.rfind(_right, off); } //---------------------------------------------------------------------------------------------- amf_int AMF_STD_CALL amf::amf_string_ci_compare(const amf_wstring& left, const amf_wstring& right) { amf_wstring _left = amf_string_to_lower(left); amf_wstring _right = amf_string_to_lower(right); return _left.compare(_right); } //---------------------------------------------------------------------------------------------- amf_int AMF_STD_CALL amf::amf_string_ci_compare(const amf_string& left, const amf_string& right) { amf_string _left = amf_string_to_lower(left); amf_string _right = amf_string_to_lower(right); return _left.compare(_right); } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_string_format(const wchar_t* format, ...) { va_list arglist; va_start(arglist, format); amf_wstring text = amf_string_formatVA(format, arglist); va_end(arglist); return text; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_string_format(const char* format, ...) { va_list arglist; va_start(arglist, format); amf_string text = amf_string_formatVA(format, arglist); va_end(arglist); return text; } //---------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf::amf_string_formatVA(const wchar_t* format, va_list args) { #if (defined(__linux) || defined(__APPLE__)) && (!defined(__ANDROID__)) //replace %s with %ls amf_wstring text(format); amf_wstring textReplaced; textReplaced.reserve(text.length() * 2); bool percentFlag = false; for(amf_wstring::iterator i = text.begin(); i != text.end(); ++i) { if(percentFlag && (*i == L's')) { textReplaced.push_back(L'l'); textReplaced.push_back(L's'); } else if(percentFlag && (*i == L'S')) { textReplaced.push_back(L's'); } else { textReplaced.push_back(*i); } percentFlag = (*i != L'%') ? false : !percentFlag; } format = textReplaced.c_str(); #endif //#if defined(__linux) va_list argcopy; #ifdef _WIN32 argcopy = args; #else va_copy(argcopy, args); #endif int size = vscwprintf(format, argcopy); va_end(argcopy); std::vector buf(size + 1); wchar_t* pBuf = &buf[0]; vswprintf(pBuf, size + 1, format, args); return pBuf; } //---------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf::amf_string_formatVA(const char* format, va_list args) { va_list argcopy; #ifdef _WIN32 argcopy = args; #else va_copy(argcopy, args); #endif int size = vscprintf(format, args); va_end(argcopy); std::vector buf(size + 1); char* pBuf = &buf[0]; vsnprintf(pBuf, size + 1, format, args); return pBuf; } #if (defined(__linux) || defined(__APPLE__)) && !defined(__ANDROID__) int vscprintf(const char* format, va_list argptr) { char* p_tmp_buf; size_t tmp_buf_size; FILE* fd = open_memstream(&p_tmp_buf, &tmp_buf_size); if(fd == 0) { return -1; } va_list arg_copy; va_copy(arg_copy, argptr); vfprintf(fd, format, arg_copy); va_end(arg_copy); fclose(fd); free(p_tmp_buf); return tmp_buf_size; } int vscwprintf(const wchar_t* format, va_list argptr) { wchar_t* p_tmp_buf; size_t tmp_buf_size; FILE* fd = open_wmemstream(&p_tmp_buf, &tmp_buf_size); if(fd == 0) { return -1; } va_list arg_copy; va_copy(arg_copy, argptr); vfwprintf(fd, format, argptr); va_end(arg_copy); fclose(fd); free(p_tmp_buf); return tmp_buf_size; } #endif //---------------------------------------------------------------------------------------- void* AMF_STD_CALL amf_alloc(size_t count) { return malloc(count); } //---------------------------------------------------------------------------------------- void AMF_STD_CALL amf_free(void* ptr) { free(ptr); } //---------------------------------------------------------------------------------------- void* AMF_STD_CALL amf_aligned_alloc(size_t count, size_t alignment) { #if defined(_WIN32) return _aligned_malloc(count, alignment); #elif defined (__APPLE__) void* p = nullptr; posix_memalign(&p, alignment, count); return p; #elif defined(__linux) return memalign(alignment, count); #endif } //---------------------------------------------------------------------------------------- void AMF_STD_CALL amf_aligned_free(void* ptr) { #if defined(_WIN32) return _aligned_free(ptr); #else return free(ptr); #endif } //---------------------------------------------------------------------------------------- #if defined (__ANDROID__) template static bool isOneOf(CHAR_T p_ch, const CHAR_T* p_set) { for (const CHAR_T* current = p_set; *current != 0; ++current) { if (*current == p_ch) return true; } return false; } static void processWidthAndPrecision(amf_string& p_fmt, va_list& p_args) { for (size_t i = 0; i < p_fmt.length(); i++) { if (p_fmt[i] == '*') { int value = va_arg(p_args, int); char valueString[64]; sprintf(valueString, "%d", value); p_fmt.replace(i, 1, valueString); } } } typedef size_t(*outputStreamDelegateW)(void* p_context, size_t p_offset, const wchar_t* p_stringToAdd, size_t p_length); static size_t amf_wprintfCore(outputStreamDelegateW p_outDelegate, void* p_context, const wchar_t* p_fmt, va_list p_args) { static const wchar_t formatSpecifiers[] = L"cCdiouxXeEfgGaAnpsSZ"; bool inFormat = false; const wchar_t* beginCurrentFormat = NULL; amf_wstring::size_type formatLength = 0; amf_wstring currentFormat; amf_wstring currentArgumentString; size_t totalCount = 0; for (const wchar_t* fmt = p_fmt; *fmt != L'\0'; ++fmt) { if (*fmt == L'%') { inFormat = !inFormat; if (inFormat) // Beginning of a format substring - fmt points at the opening % { beginCurrentFormat = fmt; // Save the pointer to the current format substring formatLength = 0; } else // This was a percent character %% - don't bother { beginCurrentFormat = NULL; } currentFormat.clear(); } if (inFormat) { ++formatLength; if (isOneOf(*fmt, formatSpecifiers)) { // end of the format specifier inFormat = false; currentFormat.assign(beginCurrentFormat, formatLength); // currentFormat now contains a modified format string for the current parameter amf_string currentFormatMB = amf_from_unicode_to_multibyte(currentFormat.c_str()); //processWidthAndPrecision(currentFormatMB, &p_args[0]); // This would extract additional arguments for width and precision and replace * with their values for (size_t i = 0; i < currentFormatMB.length(); i++) { if (currentFormatMB[i] == '*') { int value = va_arg(p_args, int); char valueString[64]; sprintf(valueString, "%d", value); currentFormatMB.replace(i, 1, valueString); } } switch (*fmt) { case L'c': { wchar_t ch; switch (*(fmt - 1)) { case L'h': ch = static_cast(va_arg(p_args, int)); break; case L'l': case L'w': ch = va_arg(p_args, unsigned int); break; default: ch = va_arg(p_args, unsigned int); // In a wchar_t version of printf %c means wchar_t } currentArgumentString = ch; } break; case L'C': { wchar_t ch; switch (*(fmt - 1)) { case L'h': ch = static_cast(va_arg(p_args, int)); break; case L'l': case L'w': ch = va_arg(p_args, unsigned int); break; default: ch = static_cast(va_arg(p_args, int)); // In a wchar_t version of printf %C means char } currentArgumentString = ch; } break; case L's': { const void* str = va_arg(p_args, const void*); if (str != NULL) { const wchar_t* str_wchar = nullptr; switch (*(fmt - 1)) { case L'h': currentArgumentString = amf_from_utf8_to_unicode(reinterpret_cast(str)); str_wchar = currentArgumentString.c_str(); break; case L'l': case L'w': currentArgumentString = str_wchar = reinterpret_cast(str); break; default: currentArgumentString = str_wchar = reinterpret_cast(str); } } else { currentArgumentString = L"(null)"; } } break; case L'S': { const void* str = va_arg(p_args, const void*); if (str != NULL) { switch (*(fmt - 1)) { case (wchar_t)'h': currentArgumentString = amf_from_utf8_to_unicode(reinterpret_cast(str)); break; case L'l': case L'w': currentArgumentString = reinterpret_cast(str); break; default: currentArgumentString = amf_from_utf8_to_unicode(reinterpret_cast(str)); } } else { currentArgumentString = L"(null)"; } } break; // All integer formats case L'i': case L'd': case L'u': case L'o': case L'x': case L'X': { char tempBuffer[64]; // 64 bytes should be enough for any numeric format switch (*(fmt - 1)) { case L'l': if (*(fmt - 2) == L'l') // long long { sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, long long)); } else { sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, long)); } break; case L'h': sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, int)); break; #ifdef _WIN32 case L'I': // I is Microsoft-specific #else case L'z': // z and t are C99-specific, but seem to be unsupported in VC case L't': #endif sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, size_t)); break; default: sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, int)); break; } currentArgumentString = amf_from_utf8_to_unicode(tempBuffer); } break; // All floating point formats case L'e': case L'E': case L'f': case L'g': case L'G': case L'a': case L'A': { char tempBuffer[64]; // 64 bytes should be enough for any numeric format switch (*(fmt - 1)) { case L'l': case L'L': sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, long double)); break; default: sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, double)); break; } currentArgumentString = amf_from_utf8_to_unicode(tempBuffer); } break; // Pointer case L'p': { char tempBuffer[64]; // 64 bytes should be enough for any numeric format sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, const void*)); currentArgumentString = amf_from_utf8_to_unicode(tempBuffer); } break; case L'n': { int* dest = va_arg(p_args, int*); *dest = static_cast(totalCount); currentArgumentString.clear(); } break; } size_t length = currentArgumentString.length(); if (p_outDelegate != NULL) // If destination buffer is NULL, just count the characters { p_outDelegate(p_context, totalCount, currentArgumentString.c_str(), length); } totalCount += length; } // if (isOneOf(*fmt, formatSpecifiers)) } // if (inFormat) else { // Just copy the character into the output buffer if (p_outDelegate != NULL) // If destination buffer is NULL, just count the characters { p_outDelegate(p_context, totalCount, fmt, 1); } ++totalCount; } } return totalCount; } typedef size_t(*outputStreamDelegate)(void* p_context, size_t p_offset, const char* p_stringToAdd, size_t p_length); static size_t amf_printfCore(outputStreamDelegate p_outDelegate, void* p_context, const char* p_fmt, va_list p_args) { static const char formatSpecifiers[] = "cCdiouxXeEfgGaAnpsSZ"; bool inFormat = false; const char* beginCurrentFormat = NULL; amf_string::size_type formatLength = 0; amf_string currentFormat; amf_string currentArgumentString; size_t totalCount = 0; for (const char* fmt = p_fmt; *fmt != '\0'; ++fmt) { if (*fmt == '%') { inFormat = !inFormat; if (inFormat) // Beginning of a format substring - fmt points at the opening % { beginCurrentFormat = fmt; // Save the pointer to the current format substring formatLength = 0; } else // This was a percent character %% - don't bother { beginCurrentFormat = NULL; } currentFormat.clear(); } if (inFormat) { ++formatLength; if (isOneOf(*fmt, formatSpecifiers)) { // end of the format specifier inFormat = false; currentFormat.assign(beginCurrentFormat, formatLength); // currentFormat now contains a modified format string for the current parameter amf_string currentFormatMB = currentFormat.c_str(); //processWidthAndPrecision(currentFormatMB, &p_args[0]); // This would extract additional arguments for width and precision and replace * with their values for (size_t i = 0; i < currentFormatMB.length(); i++) { if (currentFormatMB[i] == '*') { int value = va_arg(p_args, int); char valueString[64]; sprintf(valueString, "%d", value); currentFormatMB.replace(i, 1, valueString); } } switch (*fmt) { case 'c': { char ch; switch (*(fmt - 1)) { case 'h': ch = static_cast(va_arg(p_args, int)); break; case 'l': case 'w': ch = va_arg(p_args, unsigned int); break; default: ch = va_arg(p_args, unsigned int); // In a wchar_t version of printf %c means wchar_t } currentArgumentString = ch; } break; case 'C': { char ch; switch (*(fmt - 1)) { case 'h': ch = static_cast(va_arg(p_args, int)); break; case 'l': case 'w': ch = va_arg(p_args, unsigned int); break; default: ch = static_cast(va_arg(p_args, int)); // In a wchar_t version of printf %C means char } currentArgumentString = ch; } break; case 's': { const void* str = va_arg(p_args, const void*); if (str != NULL) { switch (*(fmt - 1)) { case 'h': currentArgumentString = reinterpret_cast(str); break; case 'l': case 'w': currentArgumentString = amf_from_unicode_to_utf8(reinterpret_cast(str)); break; default: currentArgumentString = reinterpret_cast(str); } } else { currentArgumentString = "(null)"; } } break; case L'S': { const void* str = va_arg(p_args, const void*); if (str != NULL) { switch (*(fmt - 1)) { case 'h': currentArgumentString = reinterpret_cast(str); break; case 'l': case 'w': currentArgumentString = amf_from_unicode_to_utf8(reinterpret_cast(str)); break; default: currentArgumentString = amf_from_unicode_to_utf8(reinterpret_cast(str)); } } else { currentArgumentString = "(null)"; } } break; // All integer formats case L'i': case L'd': case L'u': case L'o': case L'x': case L'X': { char tempBuffer[64]; // 64 bytes should be enough for any numeric format switch (*(fmt - 1)) { case L'l': if (*(fmt - 1) == L'l') // long long { sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, long long)); } else { sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, long)); } break; case L'h': sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, int)); break; #ifdef _WIN32 case L'I': // I is Microsoft-specific #else case L'z': // z and t are C99-specific, but seem to be unsupported in VC case L't': #endif sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, size_t)); break; default: sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, int)); break; } currentArgumentString = tempBuffer; } break; // All floating point formats case L'e': case L'E': case L'f': case L'g': case L'G': case L'a': case L'A': { char tempBuffer[64]; // 64 bytes should be enough for any numeric format switch (*(fmt - 1)) { case L'l': case L'L': sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, long double)); break; default: sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, double)); break; } currentArgumentString = tempBuffer; } break; // Pointer case L'p': { char tempBuffer[64]; // 64 bytes should be enough for any numeric format sprintf(tempBuffer, currentFormatMB.c_str(), va_arg(p_args, const void*)); currentArgumentString = tempBuffer; } break; case L'n': { int* dest = va_arg(p_args, int*); *dest = static_cast(totalCount); currentArgumentString.clear(); } break; } size_t length = currentArgumentString.length(); if (p_outDelegate != NULL) // If destination buffer is NULL, just count the characters { p_outDelegate(p_context, totalCount, currentArgumentString.c_str(), length); } totalCount += length; } // if (isOneOf(*fmt, formatSpecifiers)) } // if (inFormat) else { // Just copy the character into the output buffer if (p_outDelegate != NULL) // If destination buffer is NULL, just count the characters { p_outDelegate(p_context, totalCount, fmt, 1); } ++totalCount; } } return totalCount; } typedef struct { wchar_t* m_Buf; size_t m_Size; } MemBufferContextW; typedef struct { char* m_Buf; size_t m_Size; } MemBufferContext; static size_t writeToMem(void* p_context, size_t p_offset, const wchar_t* p_stringToAdd, size_t p_length) { wchar_t* buf = &(reinterpret_cast(p_context)->m_Buf[p_offset]); size_t bufSize = reinterpret_cast(p_context)->m_Size - 1; // -1 to accommodate a trailing '\0' for (int i = 0; i < p_length && i < bufSize; i++) { *buf++ = p_stringToAdd[i]; } return p_length; } static size_t writeToFile(void* p_context, size_t, const wchar_t* p_stringToAdd, size_t p_length) { return fwrite(p_stringToAdd, sizeof(wchar_t), p_length, reinterpret_cast(p_context)); } extern "C" { int vswprintf(wchar_t* p_buf, size_t p_size, const wchar_t* p_fmt, va_list p_args) { MemBufferContextW context = { p_buf, p_size }; int bytesWritten = (int)amf_wprintfCore(writeToMem, &context, p_fmt, p_args); p_buf[bytesWritten] = L'\0'; return bytesWritten; } int wsprintf(wchar_t* p_buf, const wchar_t* p_fmt, ...) { va_list argptr; va_start(argptr, p_fmt); return vswprintf(p_buf, static_cast(-1), p_fmt, argptr); } int swprintf(wchar_t* p_buf, size_t p_size, const wchar_t* p_fmt, ...) { va_list argptr; va_start(argptr, p_fmt); return vswprintf(p_buf, p_size, p_fmt, argptr); } int vfwprintf(FILE* p_stream, const wchar_t* p_fmt, va_list p_args) { return (int)amf_wprintfCore(writeToFile, p_stream, p_fmt, p_args); } int fwprintf(FILE* p_stream, const wchar_t* p_fmt, ...) { va_list argptr; va_start(argptr, p_fmt); return vfwprintf(p_stream, p_fmt, argptr); } #if !defined(__APPLE__) int vscwprintf(const wchar_t* p_fmt, va_list p_args) { return (int)amf_wprintfCore(NULL, NULL, p_fmt, p_args); } int vscprintf(const char* p_fmt, va_list p_args) { return (int)amf_printfCore(NULL, NULL, p_fmt, p_args); } #endif } #endif //-------------------------------------------------------------------------------- // Mac doens't have _wcsicmp(0 - poor man implementation //-------------------------------------------------------------------------------- #ifdef __APPLE__ extern "C" { int _wcsicmp(const wchar_t* s1, const wchar_t* s2) { amf_wstring low_s1 = s1; amf_wstring low_s2 = s2; std::transform(low_s1.begin(), low_s1.end(), low_s1.begin(), ::tolower); std::transform(low_s2.begin(), low_s2.end(), low_s2.begin(), ::tolower); return wcscmp(low_s1.c_str(), low_s2.c_str()); } } #endif ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/AMFSTL.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_AMFSTL_h #define AMF_AMFSTL_h #pragma once #if defined(__GNUC__) //disable gcc warinings on STL code #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Weffc++" #include //default stl allocator #else #include //default stl allocator #endif #include #include #include #include #include #include #include #include #include "../include/core/Interface.h" #if defined(__cplusplus) extern "C" { #endif // allocator void* AMF_STD_CALL amf_alloc(amf_size count); void AMF_STD_CALL amf_free(void* ptr); void* AMF_STD_CALL amf_aligned_alloc(size_t count, size_t alignment); void AMF_STD_CALL amf_aligned_free(void* ptr); #if defined(__cplusplus) } #endif namespace amf { #pragma warning(push) #pragma warning(disable: 4996) // was declared deprecated //------------------------------------------------------------------------------------------------- // STL allocator redefined - will allocate all memory in "C" runtime of Common.DLL //------------------------------------------------------------------------------------------------- template class amf_allocator : public std::allocator<_Ty> { public: amf_allocator() : std::allocator<_Ty>() {} amf_allocator(const amf_allocator<_Ty>& rhs) : std::allocator<_Ty>(rhs) {} template amf_allocator(const amf_allocator<_Other>& rhs) : std::allocator<_Ty>(rhs) {} template struct rebind // convert an allocator<_Ty> to an allocator <_Other> { typedef amf_allocator<_Other> other; }; void deallocate(_Ty* const _Ptr, const size_t _Count) { _Count; amf_free((void*)_Ptr); } _Ty* allocate(const size_t _Count, const void* = static_cast(0)) { // allocate array of _Count el ements return static_cast<_Ty*>(amf_alloc(_Count * sizeof(_Ty))); } }; //------------------------------------------------------------------------------------------------- // STL container templates with changed memory allocation //------------------------------------------------------------------------------------------------- template class amf_vector : public std::vector<_Ty, amf_allocator<_Ty> > { public: typedef std::vector<_Ty, amf_allocator<_Ty> > _base; amf_vector() : _base() {} explicit amf_vector(size_t _Count) : _base(_Count) {} //MM GCC has strange compile error. to get around replaced size_type with size_t amf_vector(size_t _Count, const _Ty& _Val) : _base(_Count,_Val) {} }; template class amf_list : public std::list<_Ty, amf_allocator<_Ty> > {}; template class amf_deque : public std::deque<_Ty, amf_allocator<_Ty> > {}; template class amf_queue : public std::queue<_Ty, amf_deque<_Ty> > {}; template > class amf_map : public std::map<_Kty, _Ty, _Pr, amf_allocator> > {}; template > class amf_set : public std::set<_Kty, _Pr, amf_allocator<_Kty> > {}; template class amf_limited_deque : public amf_deque<_Ty> // circular queue of pointers to blocks { public: typedef amf_deque<_Ty> _base; amf_limited_deque(size_t size_limit) : _base(), _size_limit(size_limit) { // construct empty deque } size_t size_limit() { return _size_limit; } void set_size_limit(size_t size_limit) { _size_limit = size_limit; while(_base::size() > _size_limit) { _base::pop_front(); } } _Ty push_front(const _Ty& _Val) { // insert element at beginning _Ty ret; if(_size_limit > 0) { _base::push_front(_Val); if(_base::size() > _size_limit) { ret = _base::back(); _base::pop_back(); } } return ret; } void push_front_ex(const _Ty& _Val) { // insert element at beginning _base::push_front(_Val); } _Ty push_back(const _Ty& _Val) { // insert element at beginning _Ty ret; if(_size_limit > 0) { _base::push_back(_Val); if(_base::size() > _size_limit) { ret = _base::front(); _base::pop_front(); } } return ret; } protected: size_t _size_limit; }; #pragma warning(pop) //--------------------------------------------------------------- #if defined(__GNUC__) //disable gcc warinings on STL code #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Weffc++" #endif template class AMFInterfacePtr_TAdapted : public AMFInterfacePtr_T<_Interf> { public: AMFInterfacePtr_TAdapted* operator&() { return this; } AMFInterfacePtr_TAdapted() : AMFInterfacePtr_T<_Interf>() {} AMFInterfacePtr_TAdapted(_Interf* pOther) : AMFInterfacePtr_T<_Interf>(pOther) {} AMFInterfacePtr_TAdapted(const AMFInterfacePtr_T<_Interf>& other) : AMFInterfacePtr_T<_Interf>(other) {} }; template class amf_vector > : public std::vector, amf_allocator > > { public: typedef AMFInterfacePtr_T<_Interf>& reference; typedef std::vector, amf_allocator > > baseclass; reference operator[](size_t n) { return baseclass::operator[](n); } }; template class amf_deque > : public std::deque, amf_allocator > > {}; template class amf_list > : public std::list, amf_allocator > > {}; #if defined(__GNUC__) // restore gcc warnings #pragma GCC diagnostic pop #endif } //------------------------------------------------------------------------------------------------- // string classes //------------------------------------------------------------------------------------------------- typedef std::basic_string, amf::amf_allocator > amf_string; typedef std::basic_string, amf::amf_allocator > amf_wstring; template std::size_t amf_string_hash(TAmfString const& s) noexcept { #if defined(_WIN64) || defined(__x86_64__) constexpr size_t fnvOffsetBasis = 14695981039346656037ULL; constexpr size_t fnvPrime = 1099511628211ULL; #else // defined(_WIN64) || defined(__x86_64__) constexpr size_t fnvOffsetBasis = 2166136261U; constexpr size_t fnvPrime = 16777619U; #endif // defined(_WIN64) || defined(__x86_64__) const unsigned char* const pStr = reinterpret_cast(s.c_str()); const size_t count = s.size() * sizeof(typename TAmfString::value_type); size_t value = fnvOffsetBasis; for (size_t i = 0; i < count; ++i) { value ^= static_cast(pStr[i]); value *= fnvPrime; } return value; } template<> struct std::hash { std::size_t operator()(amf_wstring const& s) const noexcept { return amf_string_hash(s); } }; template<> struct std::hash { std::size_t operator()(amf_string const& s) const noexcept { return amf_string_hash(s); } }; namespace amf { //------------------------------------------------------------------------------------------------- // string conversion //------------------------------------------------------------------------------------------------- amf_string AMF_STD_CALL amf_from_unicode_to_utf8(const amf_wstring& str); amf_wstring AMF_STD_CALL amf_from_utf8_to_unicode(const amf_string& str); amf_string AMF_STD_CALL amf_from_unicode_to_multibyte(const amf_wstring& str); amf_wstring AMF_STD_CALL amf_from_multibyte_to_unicode(const amf_string& str); amf_string AMF_STD_CALL amf_from_string_to_hex_string(const amf_string& str); amf_string AMF_STD_CALL amf_from_hex_string_to_string(const amf_string& str); amf_string AMF_STD_CALL amf_string_to_lower(const amf_string& str); amf_wstring AMF_STD_CALL amf_string_to_lower(const amf_wstring& str); amf_string AMF_STD_CALL amf_string_to_upper(const amf_string& str); amf_wstring AMF_STD_CALL amf_string_to_upper(const amf_wstring& str); amf_string AMF_STD_CALL amf_from_unicode_to_url_utf8(const amf_wstring& data, bool bQuery = false); // converts to UTF8 and replace fobidden symbols amf_wstring AMF_STD_CALL amf_from_url_utf8_to_unicode(const amf_string& data); amf_wstring AMF_STD_CALL amf_convert_path_to_os_accepted_path(const amf_wstring& path); amf_wstring AMF_STD_CALL amf_convert_path_to_url_accepted_path(const amf_wstring& path); //------------------------------------------------------------------------------------------------- // string helpers //------------------------------------------------------------------------------------------------- amf_wstring AMF_STD_CALL amf_string_format(const wchar_t* format, ...); amf_string AMF_STD_CALL amf_string_format(const char* format, ...); amf_wstring AMF_STD_CALL amf_string_formatVA(const wchar_t* format, va_list args); amf_string AMF_STD_CALL amf_string_formatVA(const char* format, va_list args); amf_int AMF_STD_CALL amf_string_ci_compare(const amf_wstring& left, const amf_wstring& right); amf_int AMF_STD_CALL amf_string_ci_compare(const amf_string& left, const amf_string& right); amf_size AMF_STD_CALL amf_string_ci_find(const amf_wstring& left, const amf_wstring& right, amf_size off = 0); amf_size AMF_STD_CALL amf_string_ci_rfind(const amf_wstring& left, const amf_wstring& right, amf_size off = amf_wstring::npos); //------------------------------------------------------------------------------------------------- } // namespace amf #if defined(__GNUC__) // restore gcc warnings #pragma GCC diagnostic pop #endif #endif // AMF_AMFSTL_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/ByteArray.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_ByteArray_h #define AMF_ByteArray_h #pragma once #include "../include/core/Platform.h" #define INIT_ARRAY_SIZE 1024 #define ARRAY_MAX_SIZE (1LL << 60LL) // extremely large maximum size //------------------------------------------------------------------------ class AMFByteArray { protected: amf_uint8 *m_pData; amf_size m_iSize; amf_size m_iMaxSize; public: AMFByteArray() : m_pData(0), m_iSize(0), m_iMaxSize(0) { } AMFByteArray(const AMFByteArray &other) : m_pData(0), m_iSize(0), m_iMaxSize(0) { *this = other; } AMFByteArray(amf_size num) : m_pData(0), m_iSize(0), m_iMaxSize(0) { SetSize(num); } virtual ~AMFByteArray() { if (m_pData != 0) { delete[] m_pData; } } void SetSize(amf_size num) { if (num == m_iSize) { return; } if (num < m_iSize) { memset(m_pData + num, 0, m_iMaxSize - num); } else if (num > m_iMaxSize) { // This is done to prevent the following error from surfacing // for the pNewData allocation on some compilers: // -Werror=alloc-size-larger-than= amf_size newSize = (num / INIT_ARRAY_SIZE) * INIT_ARRAY_SIZE + INIT_ARRAY_SIZE; if (newSize > ARRAY_MAX_SIZE) { return; } m_iMaxSize = newSize; amf_uint8 *pNewData = new amf_uint8[m_iMaxSize]; memset(pNewData, 0, m_iMaxSize); if (m_pData != NULL) { memcpy(pNewData, m_pData, m_iSize); delete[] m_pData; } m_pData = pNewData; } m_iSize = num; } void Copy(const AMFByteArray &old) { if (m_iMaxSize < old.m_iSize) { m_iMaxSize = old.m_iMaxSize; if (m_pData != NULL) { delete[] m_pData; } m_pData = new amf_uint8[m_iMaxSize]; memset(m_pData, 0, m_iMaxSize); } memcpy(m_pData, old.m_pData, old.m_iSize); m_iSize = old.m_iSize; } amf_uint8 operator[] (amf_size iPos) const { return m_pData[iPos]; } amf_uint8& operator[] (amf_size iPos) { return m_pData[iPos]; } AMFByteArray& operator=(const AMFByteArray &other) { SetSize(other.GetSize()); if (GetSize() > 0) { memcpy(GetData(), other.GetData(), GetSize()); } return *this; } amf_uint8 *GetData() const { return m_pData; } amf_size GetSize() const { return m_iSize; } }; #endif // AMF_ByteArray_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/CPUCaps.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2019 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include #include #include #include #include #include #if defined(_WIN32) #include #else #include #endif class InstructionSet { // forward declarations class InstructionSet_Internal; public: // getters static std::string Vendor(void) { return CPU_Rep.vendor_; } static std::string Brand(void) { return CPU_Rep.brand_; } static bool SSE3(void) { return CPU_Rep.f_1_ECX_[0]; } static bool PCLMULQDQ(void) { return CPU_Rep.f_1_ECX_[1]; } static bool MONITOR(void) { return CPU_Rep.f_1_ECX_[3]; } static bool SSSE3(void) { return CPU_Rep.f_1_ECX_[9]; } static bool FMA(void) { return CPU_Rep.f_1_ECX_[12]; } static bool CMPXCHG16B(void) { return CPU_Rep.f_1_ECX_[13]; } static bool SSE41(void) { return CPU_Rep.f_1_ECX_[19]; } static bool SSE42(void) { return CPU_Rep.f_1_ECX_[20]; } static bool MOVBE(void) { return CPU_Rep.f_1_ECX_[22]; } static bool POPCNT(void) { return CPU_Rep.f_1_ECX_[23]; } static bool AES(void) { return CPU_Rep.f_1_ECX_[25]; } static bool XSAVE(void) { return CPU_Rep.f_1_ECX_[26]; } static bool OSXSAVE(void) { return CPU_Rep.f_1_ECX_[27]; } static bool AVX(void) { return CPU_Rep.f_1_ECX_[28]; } static bool F16C(void) { return CPU_Rep.f_1_ECX_[29]; } static bool RDRAND(void) { return CPU_Rep.f_1_ECX_[30]; } static bool MSR(void) { return CPU_Rep.f_1_EDX_[5]; } static bool CX8(void) { return CPU_Rep.f_1_EDX_[8]; } static bool SEP(void) { return CPU_Rep.f_1_EDX_[11]; } static bool CMOV(void) { return CPU_Rep.f_1_EDX_[15]; } static bool CLFSH(void) { return CPU_Rep.f_1_EDX_[19]; } static bool MMX(void) { return CPU_Rep.f_1_EDX_[23]; } static bool FXSR(void) { return CPU_Rep.f_1_EDX_[24]; } static bool SSE(void) { return CPU_Rep.f_1_EDX_[25]; } static bool SSE2(void) { return CPU_Rep.f_1_EDX_[26]; } static bool FSGSBASE(void) { return CPU_Rep.f_7_EBX_[0]; } static bool BMI1(void) { return CPU_Rep.f_7_EBX_[3]; } static bool HLE(void) { return CPU_Rep.isIntel_ && CPU_Rep.f_7_EBX_[4]; } static bool AVX2(void) { return CPU_Rep.f_7_EBX_[5]; } static bool BMI2(void) { return CPU_Rep.f_7_EBX_[8]; } static bool ERMS(void) { return CPU_Rep.f_7_EBX_[9]; } static bool INVPCID(void) { return CPU_Rep.f_7_EBX_[10]; } static bool RTM(void) { return CPU_Rep.isIntel_ && CPU_Rep.f_7_EBX_[11]; } static bool AVX512F(void) { return CPU_Rep.f_7_EBX_[16]; } static bool RDSEED(void) { return CPU_Rep.f_7_EBX_[18]; } static bool ADX(void) { return CPU_Rep.f_7_EBX_[19]; } static bool AVX512PF(void) { return CPU_Rep.f_7_EBX_[26]; } static bool AVX512ER(void) { return CPU_Rep.f_7_EBX_[27]; } static bool AVX512CD(void) { return CPU_Rep.f_7_EBX_[28]; } static bool SHA(void) { return CPU_Rep.f_7_EBX_[29]; } static bool AVX512BW(void) { return CPU_Rep.f_7_EBX_[30]; } static bool AVX512VL(void) { return CPU_Rep.f_7_EBX_[31]; } static bool PREFETCHWT1(void) { return CPU_Rep.f_7_ECX_[0]; } static bool LAHF(void) { return CPU_Rep.f_81_ECX_[0]; } static bool LZCNT(void) { return CPU_Rep.isIntel_ && CPU_Rep.f_81_ECX_[5]; } static bool ABM(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_ECX_[5]; } static bool SSE4a(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_ECX_[6]; } static bool XOP(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_ECX_[11]; } static bool TBM(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_ECX_[21]; } static bool SYSCALL(void) { return CPU_Rep.isIntel_ && CPU_Rep.f_81_EDX_[11]; } static bool MMXEXT(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_EDX_[22]; } static bool RDTSCP(void) { return CPU_Rep.isIntel_ && CPU_Rep.f_81_EDX_[27]; } static bool _3DNOWEXT(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_EDX_[30]; } static bool _3DNOW(void) { return CPU_Rep.isAMD_ && CPU_Rep.f_81_EDX_[31]; } private: static const InstructionSet_Internal CPU_Rep; class InstructionSet_Internal { protected: void GetCpuID ( int32_t registers[4], //out int32_t functionID, int32_t subfunctionID = 0 ) { #ifdef _WIN32 if(!subfunctionID) { __cpuid((int *)registers, (int)functionID); } else { __cpuidex((int *)registers, (int)functionID, subfunctionID); } #else asm volatile ( "cpuid": "=a" (registers[0]), "=b" (registers[1]), "=c" (registers[2]), "=d" (registers[3]): "a" (functionID), "c" (subfunctionID) ); #endif } public: InstructionSet_Internal() : nIds_( 0 ), nExIds_( 0 ), isIntel_( false ), isAMD_( false ), f_1_ECX_( 0 ), f_1_EDX_( 0 ), f_7_EBX_( 0 ), f_7_ECX_( 0 ), f_81_ECX_( 0 ), f_81_EDX_( 0 ) { //int cpuInfo[4] = {-1}; std::array cpui; // Calling __cpuid with 0x0 as the function_id argument // gets the number of the highest valid function ID. //todo: verify //__cpuid(cpui.data(), 0); GetCpuID(cpui.data(), 0); nIds_ = cpui[0]; for (int i = 0; i <= nIds_; ++i) { //todo: verify //__cpuidex(cpui.data(), i, 0); GetCpuID(cpui.data(), i, 0); data_.push_back(cpui); } // Capture vendor string char vendor[0x20]; std::memset(vendor, 0, sizeof(vendor)); *reinterpret_cast(vendor) = data_[0][1]; *reinterpret_cast(vendor + 4) = data_[0][3]; *reinterpret_cast(vendor + 8) = data_[0][2]; vendor_ = vendor; if (vendor_ == "GenuineIntel") { isIntel_ = true; } else if (vendor_ == "AuthenticAMD") { isAMD_ = true; } // load bitset with flags for function 0x00000001 if (nIds_ >= 1) { f_1_ECX_ = data_[1][2]; f_1_EDX_ = data_[1][3]; } // load bitset with flags for function 0x00000007 if (nIds_ >= 7) { f_7_EBX_ = data_[7][1]; f_7_ECX_ = data_[7][2]; } // Calling __cpuid with 0x80000000 as the function_id argument // gets the number of the highest valid extended ID. //todo: verify //__cpuid(cpui.data(), 0x80000000); GetCpuID(cpui.data(), 0x80000000); nExIds_ = cpui[0]; char brand[0x40]; memset(brand, 0, sizeof(brand)); for (int i = 0x80000000; i <= nExIds_; ++i) { //todo: verify //__cpuidex(cpui.data(), i, 0); GetCpuID(cpui.data(), i, 0); extdata_.push_back(cpui); } // load bitset with flags for function 0x80000001 if (nExIds_ >= 0x80000001) { f_81_ECX_ = extdata_[1][2]; f_81_EDX_ = extdata_[1][3]; } // Interpret CPU brand string if reported if (nExIds_ >= 0x80000004) { memcpy(brand, extdata_[2].data(), sizeof(cpui)); memcpy(brand + 16, extdata_[3].data(), sizeof(cpui)); memcpy(brand + 32, extdata_[4].data(), sizeof(cpui)); brand_ = brand; } }; virtual ~InstructionSet_Internal() { int i = 0; ++i; } int nIds_; int nExIds_; std::string vendor_; std::string brand_; bool isIntel_; bool isAMD_; std::bitset<32> f_1_ECX_; std::bitset<32> f_1_EDX_; std::bitset<32> f_7_EBX_; std::bitset<32> f_7_ECX_; std::bitset<32> f_81_ECX_; std::bitset<32> f_81_EDX_; std::vector> data_; std::vector> extdata_; }; }; ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/CurrentTimeImpl.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "CurrentTimeImpl.h" namespace amf { //------------------------------------------------------------------------------------------------- AMFCurrentTimeImpl::AMFCurrentTimeImpl() : m_timeOfFirstCall(-1) { } //------------------------------------------------------------------------------------------------- AMFCurrentTimeImpl::~AMFCurrentTimeImpl() { m_timeOfFirstCall = -1; } //------------------------------------------------------------------------------------------------- amf_pts AMF_STD_CALL AMFCurrentTimeImpl::Get() { amf::AMFLock lock(&m_sync); // We want pts time to start at 0 and subsequent // times to be relative to that if (m_timeOfFirstCall < 0) { m_timeOfFirstCall = amf_high_precision_clock(); return 0; } return (amf_high_precision_clock() - m_timeOfFirstCall); // In nanoseconds } //------------------------------------------------------------------------------------------------- void AMF_STD_CALL AMFCurrentTimeImpl::Reset() { m_timeOfFirstCall = -1; } } ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/CurrentTimeImpl.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_CurrentTimeImpl_h #define AMF_CurrentTimeImpl_h #include "../include/core/CurrentTime.h" #include "InterfaceImpl.h" #include "Thread.h" namespace amf { class AMFCurrentTimeImpl : public AMFInterfaceImpl { public: AMFCurrentTimeImpl(); ~AMFCurrentTimeImpl(); AMF_BEGIN_INTERFACE_MAP AMF_INTERFACE_ENTRY(AMFCurrentTime) AMF_END_INTERFACE_MAP virtual amf_pts AMF_STD_CALL Get(); virtual void AMF_STD_CALL Reset(); private: amf_pts m_timeOfFirstCall; mutable AMFCriticalSection m_sync; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFCurrentTimePtr; //----------------------------------------------------------------------------------------------} } #endif // AMF_CurrentTimeImpl_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/DataStream.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file DataStream.h * @brief AMFDataStream declaration *************************************************************************************************** */ #ifndef AMF_DataStream_h #define AMF_DataStream_h #pragma once #include "../include/core/Interface.h" namespace amf { // currently supports only // file:// // memory:// // eventually can be extended with: // rtsp:// // rtmp:// // http:// // etc //---------------------------------------------------------------------------------------------- enum AMF_STREAM_OPEN { AMFSO_READ = 0, AMFSO_WRITE = 1, AMFSO_READ_WRITE = 2, AMFSO_APPEND = 3, }; //---------------------------------------------------------------------------------------------- enum AMF_FILE_SHARE { AMFFS_EXCLUSIVE = 0, AMFFS_SHARE_READ = 1, AMFFS_SHARE_WRITE = 2, AMFFS_SHARE_READ_WRITE = 3, }; //---------------------------------------------------------------------------------------------- enum AMF_SEEK_ORIGIN { AMF_SEEK_BEGIN = 0, AMF_SEEK_CURRENT = 1, AMF_SEEK_END = 2, }; //---------------------------------------------------------------------------------------------- // AMFDataStream interface //---------------------------------------------------------------------------------------------- class AMF_NO_VTABLE AMFDataStream : public AMFInterface { public: AMF_DECLARE_IID(0xdb08fe70, 0xb743, 0x4c26, 0xb2, 0x77, 0xa5, 0xc8, 0xe8, 0x14, 0xda, 0x4) // interface virtual AMF_RESULT AMF_STD_CALL Open(const wchar_t* pFileUrl, AMF_STREAM_OPEN eOpenType, AMF_FILE_SHARE eShareType) = 0; virtual AMF_RESULT AMF_STD_CALL Close() = 0; virtual AMF_RESULT AMF_STD_CALL Read(void* pData, amf_size iSize, amf_size* pRead) = 0; virtual AMF_RESULT AMF_STD_CALL Write(const void* pData, amf_size iSize, amf_size* pWritten) = 0; virtual AMF_RESULT AMF_STD_CALL Seek(AMF_SEEK_ORIGIN eOrigin, amf_int64 iPosition, amf_int64* pNewPosition) = 0; virtual AMF_RESULT AMF_STD_CALL GetPosition(amf_int64* pPosition) = 0; virtual AMF_RESULT AMF_STD_CALL GetSize(amf_int64* pSize) = 0; virtual bool AMF_STD_CALL IsSeekable() = 0; static AMF_RESULT AMF_STD_CALL OpenDataStream(const wchar_t* pFileUrl, AMF_STREAM_OPEN eOpenType, AMF_FILE_SHARE eShareType, AMFDataStream** str); }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFDataStreamPtr; //---------------------------------------------------------------------------------------------- } //namespace amf #endif // AMF_DataStream_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/DataStreamFactory.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "DataStream.h" #include "DataStreamMemory.h" #include "DataStreamFile.h" #include "TraceAdapter.h" #include using namespace amf; //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL amf::AMFDataStream::OpenDataStream(const wchar_t* pFileUrl, AMF_STREAM_OPEN eOpenType, AMF_FILE_SHARE eShareType, AMFDataStream** str) { AMF_RETURN_IF_FALSE(pFileUrl != NULL, AMF_INVALID_ARG); AMF_RESULT res = AMF_NOT_SUPPORTED; std::wstring url(pFileUrl); std::wstring protocol; std::wstring path; std::wstring::size_type found_pos = url.find(L"://", 0); if(found_pos != std::wstring::npos) { protocol = url.substr(0, found_pos); path = url.substr(found_pos + 3); } else { protocol = L"file"; path = url; } AMFDataStreamPtr ptr = NULL; if(protocol == L"file") { ptr = new AMFDataStreamFileImpl; res = AMF_OK; } if(protocol == L"memory") { ptr = new AMFDataStreamMemoryImpl(); res = AMF_OK; } if( res == AMF_OK ) { res = ptr->Open(path.c_str(), eOpenType, eShareType); if( res != AMF_OK ) { return res; } *str = ptr.Detach(); return AMF_OK; } return res; } //------------------------------------------------------------------------------------------------- ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/DataStreamFile.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "TraceAdapter.h" #include "DataStreamFile.h" #pragma warning(disable: 4996) #if defined(_WIN32) #include #endif #include #include #include #if defined(_WIN32) #define amf_close _close #define amf_read _read #define amf_write _write #define amf_seek64 _lseeki64 #elif defined(__linux)// Linux #include #define amf_close close #define amf_read read #define amf_write write #define amf_seek64 lseek64 #elif defined(__APPLE__) #include #define amf_close close #define amf_read read #define amf_write write #define amf_seek64 lseek #endif using namespace amf; #define AMF_FACILITY L"AMFDataStreamFileImpl" #define AMF_FILE_PROTOCOL L"file" //------------------------------------------------------------------------------------------------- AMFDataStreamFileImpl::AMFDataStreamFileImpl() : m_iFileDescriptor(-1), m_Path() {} //------------------------------------------------------------------------------------------------- AMFDataStreamFileImpl::~AMFDataStreamFileImpl() { Close(); } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::Close() { AMF_RESULT err = AMF_OK; if(m_iFileDescriptor != -1) { const int status = amf_close(m_iFileDescriptor); if(status != 0) { err = AMF_FAIL; } m_iFileDescriptor = -1; } return err; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::Read(void* pData, amf_size iSize, amf_size* pRead) { AMF_RETURN_IF_FALSE(m_iFileDescriptor != -1, AMF_FILE_NOT_OPEN, L"Read() - File not open"); AMF_RESULT err = AMF_OK; int ready = amf_read(m_iFileDescriptor, pData, (amf_uint)iSize); if(pRead != NULL) { *pRead = ready; } if(ready == 0) // eof { err = AMF_EOF; } else if(ready == -1) { err = AMF_FAIL; } return err; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::Write(const void* pData, amf_size iSize, amf_size* pWritten) { AMF_RETURN_IF_FALSE(m_iFileDescriptor != -1, AMF_FILE_NOT_OPEN, L"Write() - File not Open"); AMF_RESULT err = AMF_OK; amf_uint32 written = amf_write(m_iFileDescriptor, pData, (amf_uint)iSize); if(pWritten != NULL) { *pWritten = written; } if(written != iSize) // check errors { err = AMF_FAIL; } return err; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::Seek(AMF_SEEK_ORIGIN eOrigin, amf_int64 iPosition, amf_int64* pNewPosition) { AMF_RETURN_IF_FALSE(m_iFileDescriptor != -1, AMF_FILE_NOT_OPEN, L"Seek() - File not Open"); int org = 0; switch(eOrigin) { case AMF_SEEK_BEGIN: org = SEEK_SET; break; case AMF_SEEK_CURRENT: org = SEEK_CUR; break; case AMF_SEEK_END: org = SEEK_END; break; } amf_int64 new_pos = 0; new_pos = amf_seek64(m_iFileDescriptor, iPosition, org); if(new_pos == -1L) // check errors { return AMF_FAIL; } if(pNewPosition != NULL) { *pNewPosition = new_pos; } return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::GetPosition(amf_int64* pPosition) { AMF_RETURN_IF_FALSE(pPosition != NULL, AMF_INVALID_POINTER); AMF_RETURN_IF_FALSE(m_iFileDescriptor != -1, AMF_FILE_NOT_OPEN, L"GetPosition() - File not Open"); *pPosition = amf_seek64(m_iFileDescriptor, 0, SEEK_CUR); if(*pPosition == -1L) { return AMF_FAIL; } return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::GetSize(amf_int64* pSize) { AMF_RETURN_IF_FALSE(pSize != NULL, AMF_INVALID_POINTER); AMF_RETURN_IF_FALSE(m_iFileDescriptor != -1, AMF_FILE_NOT_OPEN, L"GetSize() - File not open"); amf_int64 cur_pos = amf_seek64(m_iFileDescriptor, 0, SEEK_CUR); *pSize = amf_seek64(m_iFileDescriptor, 0, SEEK_END); amf_seek64(m_iFileDescriptor, cur_pos, SEEK_SET); return AMF_OK; } //------------------------------------------------------------------------------------------------- bool AMF_STD_CALL AMFDataStreamFileImpl::IsSeekable() { return true; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamFileImpl::Open(const wchar_t* pFilePath, AMF_STREAM_OPEN eOpenType, AMF_FILE_SHARE eShareType) { if(m_iFileDescriptor != -1) { Close(); } AMF_RETURN_IF_FALSE(pFilePath != NULL, AMF_INVALID_ARG); m_Path = pFilePath; #if defined(_WIN32) int access = _O_BINARY; #else int access = 0; #endif switch(eOpenType) { case AMFSO_READ: access |= O_RDONLY; break; case AMFSO_WRITE: access |= O_CREAT | O_TRUNC | O_WRONLY; break; case AMFSO_READ_WRITE: access |= O_CREAT | O_TRUNC | O_RDWR; break; case AMFSO_APPEND: access |= O_CREAT | O_APPEND | O_RDWR; break; } #ifdef _WIN32 int shflag = 0; switch(eShareType) { case AMFFS_EXCLUSIVE: shflag = _SH_DENYRW; break; case AMFFS_SHARE_READ: shflag = _SH_DENYWR; break; case AMFFS_SHARE_WRITE: shflag = _SH_DENYRD; break; case AMFFS_SHARE_READ_WRITE: shflag = _SH_DENYNO; break; } #endif #ifdef O_BINARY access |= O_BINARY; #endif #ifdef _WIN32 m_iFileDescriptor = _wsopen(m_Path.c_str(), access, shflag, 0666); #else amf_string str = amf_from_unicode_to_utf8(m_Path); m_iFileDescriptor = open(str.c_str(), access, 0666); #endif if(m_iFileDescriptor == -1) { return AMF_FAIL; } return AMF_OK; } //------------------------------------------------------------------------------------------------- ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/DataStreamFile.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_DataStreamFile_h #define AMF_DataStreamFile_h #pragma once #include "DataStream.h" #include "InterfaceImpl.h" #include "AMFSTL.h" #include namespace amf { class AMFDataStreamFileImpl : public AMFInterfaceImpl { public: AMFDataStreamFileImpl(); virtual ~AMFDataStreamFileImpl(); // interface virtual AMF_RESULT AMF_STD_CALL Close(); virtual AMF_RESULT AMF_STD_CALL Read(void* pData, amf_size iSize, amf_size* pRead); virtual AMF_RESULT AMF_STD_CALL Write(const void* pData, amf_size iSize, amf_size* pWritten); virtual AMF_RESULT AMF_STD_CALL Seek(AMF_SEEK_ORIGIN eOrigin, amf_int64 iPosition, amf_int64* pNewPosition); virtual AMF_RESULT AMF_STD_CALL GetPosition(amf_int64* pPosition); virtual AMF_RESULT AMF_STD_CALL GetSize(amf_int64* pSize); virtual bool AMF_STD_CALL IsSeekable(); // local // aways pass full URL just in case virtual AMF_RESULT AMF_STD_CALL Open(const wchar_t* pFilePath, AMF_STREAM_OPEN eOpenType, AMF_FILE_SHARE eShareType); protected: int m_iFileDescriptor; amf_wstring m_Path; }; } //namespace amf #endif // AMF_DataStreamFile_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/DataStreamMemory.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "Thread.h" #include "TraceAdapter.h" #include "DataStreamMemory.h" using namespace amf; #define AMF_FACILITY L"AMFDataStreamMemoryImpl" //------------------------------------------------------------------------------------------------- AMFDataStreamMemoryImpl::AMFDataStreamMemoryImpl() : m_pMemory(NULL), m_uiMemorySize(0), m_uiAllocatedSize(0), m_pos(0) {} //------------------------------------------------------------------------------------------------- AMFDataStreamMemoryImpl::~AMFDataStreamMemoryImpl() { Close(); } //------------------------------------------------------------------------------------------------- // interface //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamMemoryImpl::Close() { if(m_pMemory != NULL) { amf_virtual_free(m_pMemory); } m_pMemory = NULL, m_uiMemorySize = 0, m_uiAllocatedSize = 0, m_pos = 0; return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMFDataStreamMemoryImpl::Realloc(amf_size iSize) { if(iSize > m_uiMemorySize) { amf_uint8* pNewMemory = (amf_uint8*)amf_virtual_alloc(iSize); if(pNewMemory == NULL) { return AMF_OUT_OF_MEMORY; } m_uiAllocatedSize = iSize; if(m_pMemory != NULL) { memcpy(pNewMemory, m_pMemory, m_uiMemorySize); amf_virtual_free(m_pMemory); } m_pMemory = pNewMemory; } m_uiMemorySize = iSize; if(m_pos > m_uiMemorySize) { m_pos = m_uiMemorySize; } return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamMemoryImpl::Read(void* pData, amf_size iSize, amf_size* pRead) { AMF_RETURN_IF_FALSE(pData != NULL, AMF_INVALID_POINTER, L"Read() - pData==NULL"); AMF_RETURN_IF_FALSE(m_pMemory != NULL, AMF_NOT_INITIALIZED, L"Read() - Stream is not allocated"); amf_size toRead = AMF_MIN(iSize, m_uiMemorySize - m_pos); memcpy(pData, m_pMemory + m_pos, toRead); m_pos += toRead; if(pRead != NULL) { *pRead = toRead; } return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamMemoryImpl::Write(const void* pData, amf_size iSize, amf_size* pWritten) { AMF_RETURN_IF_FALSE(pData != NULL, AMF_INVALID_POINTER, L"Write() - pData==NULL"); AMF_RETURN_IF_FAILED(Realloc(m_pos + iSize), L"Write() - Stream is not allocated"); amf_size toWrite = AMF_MIN(iSize, m_uiMemorySize - m_pos); memcpy(m_pMemory + m_pos, pData, toWrite); m_pos += toWrite; if(pWritten != NULL) { *pWritten = toWrite; } return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamMemoryImpl::Seek(AMF_SEEK_ORIGIN eOrigin, amf_int64 iPosition, amf_int64* pNewPosition) { switch(eOrigin) { case AMF_SEEK_BEGIN: m_pos = (amf_size)iPosition; break; case AMF_SEEK_CURRENT: m_pos += (amf_size)iPosition; break; case AMF_SEEK_END: m_pos = m_uiMemorySize - (amf_size)iPosition; break; } if(m_pos > m_uiMemorySize) { m_pos = m_uiMemorySize; } if(pNewPosition != NULL) { *pNewPosition = m_pos; } return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamMemoryImpl::GetPosition(amf_int64* pPosition) { AMF_RETURN_IF_FALSE(pPosition != NULL, AMF_INVALID_POINTER, L"GetPosition() - pPosition==NULL"); *pPosition = m_pos; return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT AMF_STD_CALL AMFDataStreamMemoryImpl::GetSize(amf_int64* pSize) { AMF_RETURN_IF_FALSE(pSize != NULL, AMF_INVALID_POINTER, L"GetPosition() - pSize==NULL"); *pSize = m_uiMemorySize; return AMF_OK; } //------------------------------------------------------------------------------------------------- bool AMF_STD_CALL AMFDataStreamMemoryImpl::IsSeekable() { return true; } //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/DataStreamMemory.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_DataStreamMemory_h #define AMF_DataStreamMemory_h #pragma once #include "DataStream.h" #include "InterfaceImpl.h" namespace amf { class AMFDataStreamMemoryImpl : public AMFInterfaceImpl { public: AMFDataStreamMemoryImpl(); virtual ~AMFDataStreamMemoryImpl(); // interface virtual AMF_RESULT AMF_STD_CALL Open(const wchar_t* /*pFileUrl*/, AMF_STREAM_OPEN /*eOpenType*/, AMF_FILE_SHARE /*eShareType*/) { //pFileUrl; //eOpenType; //eShareType; return AMF_OK; } virtual AMF_RESULT AMF_STD_CALL Close(); virtual AMF_RESULT AMF_STD_CALL Read(void* pData, amf_size iSize, amf_size* pRead); virtual AMF_RESULT AMF_STD_CALL Write(const void* pData, amf_size iSize, amf_size* pWritten); virtual AMF_RESULT AMF_STD_CALL Seek(AMF_SEEK_ORIGIN eOrigin, amf_int64 iPosition, amf_int64* pNewPosition); virtual AMF_RESULT AMF_STD_CALL GetPosition(amf_int64* pPosition); virtual AMF_RESULT AMF_STD_CALL GetSize(amf_int64* pSize); virtual bool AMF_STD_CALL IsSeekable(); protected: AMF_RESULT Realloc(amf_size iSize); amf_uint8* m_pMemory; amf_size m_uiMemorySize; amf_size m_uiAllocatedSize; amf_size m_pos; private: AMFDataStreamMemoryImpl(const AMFDataStreamMemoryImpl&); AMFDataStreamMemoryImpl& operator=(const AMFDataStreamMemoryImpl&); }; } //namespace amf #endif // AMF_DataStreamMemory_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/IOCapsImpl.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "IOCapsImpl.h" namespace amf { //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// AMFIOCapsImpl::SurfaceFormat::SurfaceFormat() : m_Format(AMF_SURFACE_UNKNOWN), m_Native(false) { } AMFIOCapsImpl::SurfaceFormat::SurfaceFormat(AMF_SURFACE_FORMAT format, amf_bool native) : m_Format(format), m_Native(native) { } //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// AMFIOCapsImpl::MemoryType::MemoryType() : m_Type(AMF_MEMORY_UNKNOWN), m_Native(false) { } AMFIOCapsImpl::MemoryType::MemoryType(AMF_MEMORY_TYPE type, amf_bool native) : m_Type(type), m_Native(native) { } //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// AMFIOCapsImpl::AMFIOCapsImpl() : m_MinWidth(-1), m_MaxWidth(-1), m_MinHeight(-1), m_MaxHeight(-1), m_VertAlign(-1), m_InterlacedSupported(false) { } AMFIOCapsImpl::AMFIOCapsImpl(amf_int32 minWidth, amf_int32 maxWidth, amf_int32 minHeight, amf_int32 maxHeight, amf_int32 vertAlign, amf_bool interlacedSupport, amf_int32 numOfNativeFormats, const AMF_SURFACE_FORMAT* nativeFormats, amf_int32 numOfNonNativeFormats, const AMF_SURFACE_FORMAT* nonNativeFormats, amf_int32 numOfNativeMemTypes, const AMF_MEMORY_TYPE* nativeMemTypes, amf_int32 numOfNonNativeMemTypes, const AMF_MEMORY_TYPE* nonNativeMemTypes) { m_MinWidth = minWidth; m_MaxWidth = maxWidth; m_MinHeight = minHeight; m_MaxHeight = maxHeight; m_VertAlign = vertAlign; m_InterlacedSupported = interlacedSupport; PopulateSurfaceFormats(numOfNativeFormats, nativeFormats, true); PopulateSurfaceFormats(numOfNonNativeFormats, nonNativeFormats, false); PopulateMemoryTypes(numOfNativeMemTypes, nativeMemTypes, true); PopulateMemoryTypes(numOfNonNativeMemTypes, nonNativeMemTypes, false); } void AMFIOCapsImpl::PopulateSurfaceFormats(amf_int32 numOfFormats, const AMF_SURFACE_FORMAT* formats, amf_bool native) { if (formats != NULL) { for (amf_int32 i = 0; i < numOfFormats; i++) { bool found = false; for(amf_size exists_idx = 0; exists_idx < m_SurfaceFormats.size(); exists_idx++) { if(m_SurfaceFormats[exists_idx].GetFormat() == formats[i]) { found = true; } } if(!found) { m_SurfaceFormats.push_back(SurfaceFormat(formats[i], native)); } } } } void AMFIOCapsImpl::PopulateMemoryTypes(amf_int32 numOfTypes, const AMF_MEMORY_TYPE* memTypes, amf_bool native) { if (memTypes != NULL) { for (amf_int32 i = 0; i < numOfTypes; i++) { bool found = false; for(amf_size exists_idx = 0; exists_idx < m_MemoryTypes.size(); exists_idx++) { if(m_MemoryTypes[exists_idx].GetType() == memTypes[i]) { found = true; } } if(!found) { m_MemoryTypes.push_back(MemoryType(memTypes[i], native)); } } } } // Get supported resolution ranges in pixels/lines: void AMF_STD_CALL AMFIOCapsImpl::GetWidthRange(amf_int32* minWidth, amf_int32* maxWidth) const { if (minWidth != NULL) { *minWidth = m_MinWidth; } if (maxWidth != NULL) { *maxWidth = m_MaxWidth; } } void AMF_STD_CALL AMFIOCapsImpl::GetHeightRange(amf_int32* minHeight, amf_int32* maxHeight) const { if (minHeight != NULL) { *minHeight = m_MinHeight; } if (maxHeight != NULL) { *maxHeight = m_MaxHeight; } } // Get memory alignment in lines: // Vertical aligmnent should be multiples of this number amf_int32 AMF_STD_CALL AMFIOCapsImpl::GetVertAlign() const { return m_VertAlign; } // Enumerate supported surface pixel formats: amf_int32 AMF_STD_CALL AMFIOCapsImpl::GetNumOfFormats() const { return (amf_int32)m_SurfaceFormats.size(); } AMF_RESULT AMF_STD_CALL AMFIOCapsImpl::GetFormatAt(amf_int32 index, AMF_SURFACE_FORMAT* format, bool* native) const { if (index >= 0 && index < static_cast(m_SurfaceFormats.size())) { SurfaceFormat curFormat(m_SurfaceFormats.at(index)); if (format != NULL) { *format = curFormat.GetFormat(); } if (native != NULL) { *native = curFormat.IsNative(); } return AMF_OK; } else { return AMF_INVALID_ARG; } } // Enumerate supported surface formats: amf_int32 AMF_STD_CALL AMFIOCapsImpl::GetNumOfMemoryTypes() const { return (amf_int32)m_MemoryTypes.size(); } AMF_RESULT AMF_STD_CALL AMFIOCapsImpl::GetMemoryTypeAt(amf_int32 index, AMF_MEMORY_TYPE* memType, bool* native) const { if (index >= 0 && index < static_cast(m_MemoryTypes.size())) { MemoryType curType(m_MemoryTypes.at(index)); if (memType != NULL) { *memType = curType.GetType(); } if (native != NULL) { *native = curType.IsNative(); } return AMF_OK; } else { return AMF_INVALID_ARG; } } // interlaced support: amf_bool AMF_STD_CALL AMFIOCapsImpl::IsInterlacedSupported() const { return m_InterlacedSupported; } void AMFIOCapsImpl::SetResolution(amf_int32 minWidth, amf_int32 maxWidth, amf_int32 minHeight, amf_int32 maxHeight) { m_MinWidth = minWidth; m_MaxWidth = maxWidth; m_MinHeight = minHeight; m_MaxHeight = maxHeight; } void AMFIOCapsImpl::SetVertAlign(amf_int32 vertAlign) { m_VertAlign = vertAlign; } void AMFIOCapsImpl::SetInterlacedSupport(amf_bool interlaced) { m_InterlacedSupported = interlaced; } } ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/IOCapsImpl.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_IOCapsImpl_h #define AMF_IOCapsImpl_h #pragma once #include "InterfaceImpl.h" #include "../include/components/ComponentCaps.h" #include namespace amf { class AMFIOCapsImpl : public AMFInterfaceImpl { protected: class SurfaceFormat { public: typedef std::vector Collection; public: SurfaceFormat(); SurfaceFormat(AMF_SURFACE_FORMAT format, amf_bool native); inline AMF_SURFACE_FORMAT GetFormat() const throw() { return m_Format; } inline amf_bool IsNative() const throw() { return m_Native; } private: AMF_SURFACE_FORMAT m_Format; amf_bool m_Native; }; class MemoryType { public: typedef std::vector Collection; public: MemoryType(); MemoryType(AMF_MEMORY_TYPE type, amf_bool native); inline AMF_MEMORY_TYPE GetType() const throw() { return m_Type; } inline amf_bool IsNative() const throw() { return m_Native; } private: AMF_MEMORY_TYPE m_Type; amf_bool m_Native; }; struct Resolution { amf_int32 m_Width; amf_int32 m_Height; }; protected: AMFIOCapsImpl(); AMFIOCapsImpl(amf_int32 minWidth, amf_int32 maxWidth, amf_int32 minHeight, amf_int32 maxHeight, amf_int32 vertAlign, amf_bool interlacedSupport, amf_int32 numOfNativeFormats, const AMF_SURFACE_FORMAT* nativeFormats, amf_int32 numOfNonNativeFormats, const AMF_SURFACE_FORMAT* nonNativeFormats, amf_int32 numOfNativeMemTypes, const AMF_MEMORY_TYPE* nativeMemTypes, amf_int32 numOfNonNativeMemTypes, const AMF_MEMORY_TYPE* nonNativeMemTypes); public: // Get supported resolution ranges in pixels/lines: virtual void AMF_STD_CALL GetWidthRange(amf_int32* minWidth, amf_int32* maxWidth) const; virtual void AMF_STD_CALL GetHeightRange(amf_int32* minHeight, amf_int32* maxHeight) const; // Get memory alignment in lines: // Vertical aligmnent should be multiples of this number virtual amf_int32 AMF_STD_CALL GetVertAlign() const; // Enumerate supported surface pixel formats: virtual amf_int32 AMF_STD_CALL GetNumOfFormats() const; virtual AMF_RESULT AMF_STD_CALL GetFormatAt(amf_int32 index, AMF_SURFACE_FORMAT* format, amf_bool* native) const; // Enumerate supported surface formats: virtual amf_int32 AMF_STD_CALL GetNumOfMemoryTypes() const; virtual AMF_RESULT AMF_STD_CALL GetMemoryTypeAt(amf_int32 index, AMF_MEMORY_TYPE* memType, amf_bool* native) const; // interlaced support: virtual amf_bool AMF_STD_CALL IsInterlacedSupported() const; protected: void SetResolution(amf_int32 minWidth, amf_int32 maxWidth, amf_int32 minHeight, amf_int32 maxHeight); void SetVertAlign(amf_int32 alignment); void SetInterlacedSupport(amf_bool interlaced); void PopulateSurfaceFormats(amf_int32 numOfFormats, const AMF_SURFACE_FORMAT* formats, amf_bool native); void PopulateMemoryTypes(amf_int32 numOfTypes, const AMF_MEMORY_TYPE* memTypes, amf_bool native); protected: amf_int32 m_MinWidth; amf_int32 m_MaxWidth; amf_int32 m_MinHeight; amf_int32 m_MaxHeight; amf_int32 m_VertAlign; amf_bool m_InterlacedSupported; SurfaceFormat::Collection m_SurfaceFormats; MemoryType::Collection m_MemoryTypes; }; } #endif // AMF_IOCapsImpl_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/InterfaceImpl.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_InterfaceImpl_h #define AMF_InterfaceImpl_h #pragma once #include "../include/core/Interface.h" #include "Thread.h" #pragma warning(disable : 4511) namespace amf { #define AMF_BEGIN_INTERFACE_MAP \ virtual AMF_RESULT AMF_STD_CALL QueryInterface(const amf::AMFGuid & interfaceID, void** ppInterface) \ { \ AMF_RESULT err = AMF_NO_INTERFACE; \ #define AMF_INTERFACE_ENTRY(T) \ if(AMFCompareGUIDs(interfaceID, T::IID())) \ { \ *ppInterface = (void*)static_cast(this); \ this->Acquire(); \ err = AMF_OK; \ } \ else \ #define AMF_INTERFACE_ENTRY_THIS(T, _TI) \ if(AMFCompareGUIDs(interfaceID, T::IID())) \ { \ *ppInterface = (void*)static_cast(static_cast<_TI*>(this)); \ this->Acquire(); \ err = AMF_OK; \ } \ else \ #define AMF_INTERFACE_MULTI_ENTRY(T) \ if(AMFCompareGUIDs(interfaceID, T::IID())) \ { \ *ppInterface = (void*)static_cast(this); \ AcquireInternal(); \ err = AMF_OK; \ } \ else \ #define AMF_INTERFACE_CHAIN_ENTRY(T) \ if(static_cast(*this).T::QueryInterface(interfaceID, ppInterface) == AMF_OK) \ {err = AMF_OK;} \ else \ //good as an example but we should not use aggregate pattern without big reason - very hard to debug #define AMF_INTERFACE_AGREGATED_ENTRY(T, _Ptr) \ if(AMFCompareGUIDs(interfaceID, T::IID())) \ { \ T* ptr = static_cast(_Ptr); \ *ppInterface = (void*)ptr; \ ptr->Acquire(); \ err = AMF_OK; \ } \ else \ #define AMF_INTERFACE_CHAIN_AGREGATED_ENTRY(T, _Ptr) \ if(err = static_cast(_Ptr)->QueryInterface(interfaceID, ppInterface)) { \ } \ else \ #define AMF_END_INTERFACE_MAP \ {} \ return err; \ } \ //--------------------------------------------------------------- class AMFInterfaceBase { protected: amf_long m_refCount; virtual ~AMFInterfaceBase() #if __GNUC__ == 11 //WORKAROUND for gcc-11 bug __attribute__ ((noinline)) #endif {} public: AMFInterfaceBase() : m_refCount(0) {} virtual amf_long AMF_STD_CALL AcquireInternal() { amf_long newVal = amf_atomic_inc(&m_refCount); return newVal; } virtual amf_long AMF_STD_CALL ReleaseInternal() { amf_long newVal = amf_atomic_dec(&m_refCount); if(newVal == 0) { delete this; } return newVal; } virtual amf_long AMF_STD_CALL RefCountInternal() { return m_refCount; } }; //--------------------------------------------------------------- template class AMFInterfaceImpl : public _Base, public AMFInterfaceBase { protected: virtual ~AMFInterfaceImpl() {} public: AMFInterfaceImpl(_Param1 param1, _Param2 param2, _Param3 param3) : _Base(param1, param2, param3) {} AMFInterfaceImpl(_Param1 param1, _Param2 param2) : _Base(param1, param2) {} AMFInterfaceImpl(_Param1 param1) : _Base(param1) {} AMFInterfaceImpl() {} virtual amf_long AMF_STD_CALL Acquire() { return AMFInterfaceBase::AcquireInternal(); } virtual amf_long AMF_STD_CALL Release() { return AMFInterfaceBase::ReleaseInternal(); } virtual amf_long AMF_STD_CALL RefCount() { return AMFInterfaceBase::RefCountInternal(); } AMF_BEGIN_INTERFACE_MAP AMF_INTERFACE_ENTRY(AMFInterface) AMF_INTERFACE_ENTRY(_Base) AMF_END_INTERFACE_MAP }; //--------------------------------------------------------------- template class AMFInterfaceMultiImpl : public _Base { protected: virtual ~AMFInterfaceMultiImpl() {} public: AMFInterfaceMultiImpl(_Param1 param1, _Param2 param2, _Param3 param3, _Param4 param4, _Param5 param5, _Param6 param6) : _Base(param1, param2, param3, param4, param5, param6) {} AMFInterfaceMultiImpl(_Param1 param1, _Param2 param2, _Param3 param3, _Param4 param4, _Param5 param5) : _Base(param1, param2, param3, param4, param5) {} AMFInterfaceMultiImpl(_Param1 param1, _Param2 param2, _Param3 param3, _Param4 param4) : _Base(param1, param2, param3, param4) {} AMFInterfaceMultiImpl(_Param1 param1, _Param2 param2, _Param3 param3) : _Base(param1, param2, param3) {} AMFInterfaceMultiImpl(_Param1 param1, _Param2 param2) : _Base(param1, param2) {} AMFInterfaceMultiImpl(_Param1 param1) : _Base(param1) {} AMFInterfaceMultiImpl() {} virtual amf_long AMF_STD_CALL Acquire() { return AMFInterfaceBase::AcquireInternal(); } virtual amf_long AMF_STD_CALL Release() { return AMFInterfaceBase::ReleaseInternal(); } virtual amf_long AMF_STD_CALL RefCount() { return AMFInterfaceBase::RefCountInternal(); } AMF_BEGIN_INTERFACE_MAP AMF_INTERFACE_ENTRY_THIS(AMFInterface, _BaseInterface) AMF_INTERFACE_CHAIN_ENTRY(_Base) AMF_END_INTERFACE_MAP }; } // namespace amf #endif // AMF_InterfaceImpl_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/ObservableImpl.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file ObservableImpl.h * @brief AMFObservableImpl common template declaration *************************************************************************************************** */ #ifndef AMF_ObservableImpl_h #define AMF_ObservableImpl_h #pragma once #include "Thread.h" #include namespace amf { template class AMFObservableImpl { private: typedef std::list ObserversList; ObserversList m_observers; public: AMFObservableImpl() : m_observers() {} virtual ~AMFObservableImpl() { assert(m_observers.size() == 0); } virtual void AMF_STD_CALL AddObserver(Observer* pObserver) { if (pObserver == nullptr) { return; } amf_bool found = false; AMFLock lock(&m_sc); for (typename ObserversList::iterator it = m_observers.begin(); it != m_observers.end(); it++) { if (*it == pObserver) { found = true; break; } } if (found == false) { m_observers.push_back(pObserver); } } virtual void AMF_STD_CALL RemoveObserver(Observer* pObserver) { AMFLock lock(&m_sc); m_observers.remove(pObserver); } protected: void AMF_STD_CALL ClearObservers() { AMFLock lock(&m_sc); m_observers.clear(); } void AMF_STD_CALL NotifyObservers(void (AMF_STD_CALL Observer::* pEvent)()) { ObserversList tempList; { AMFLock lock(&m_sc); tempList = m_observers; } for (typename ObserversList::iterator it = tempList.begin(); it != tempList.end(); ++it) { Observer* pObserver = *it; (pObserver->*pEvent)(); } } template void AMF_STD_CALL NotifyObservers(void (AMF_STD_CALL Observer::* pEvent)(TArg0), TArg0 arg0) { ObserversList tempList; { AMFLock lock(&m_sc); tempList = m_observers; } for (typename ObserversList::iterator it = tempList.begin(); it != tempList.end(); ++it) { Observer* pObserver = *it; (pObserver->*pEvent)(arg0); } } template void AMF_STD_CALL NotifyObservers(void (AMF_STD_CALL Observer::* pEvent)(TArg0, TArg1), TArg0 arg0, TArg1 arg1) { ObserversList tempList; { AMFLock lock(&m_sc); tempList = m_observers; } for (typename ObserversList::iterator it = tempList.begin(); it != tempList.end(); it++) { Observer* pObserver = *it; (pObserver->*pEvent)(arg0, arg1); } } private: AMFCriticalSection m_sc; }; } #endif //AMF_ObservableImpl_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/PropertyStorageExImpl.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include #include "PropertyStorageExImpl.h" #include "PropertyStorageImpl.h" #include "TraceAdapter.h" #pragma warning(disable: 4996) using namespace amf; #define AMF_FACILITY L"AMFPropertyStorageExImpl" #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wexit-time-destructors" #pragma clang diagnostic ignored "-Wglobal-constructors" #endif amf::AMFCriticalSection amf::ms_csAMFPropertyStorageExImplMaps; #ifdef __clang__ #pragma clang diagnostic pop #endif //------------------------------------------------------------------------------------------------- AMF_RESULT amf::CastVariantToAMFProperty(amf::AMFVariantStruct* pDest, const amf::AMFVariantStruct* pSrc, amf::AMF_VARIANT_TYPE eType, amf::AMF_PROPERTY_CONTENT_TYPE /*contentType*/, const amf::AMFEnumDescriptionEntry* pEnumDescription) { AMF_RETURN_IF_INVALID_POINTER(pDest); AMF_RESULT err = AMF_OK; switch (eType) { case AMF_VARIANT_INTERFACE: if (pSrc->type == eType) { err = AMFVariantCopy(pDest, pSrc); } else { pDest->type = AMF_VARIANT_INTERFACE; pDest->pInterface = nullptr; } break; case AMF_VARIANT_INT64: { if(pEnumDescription) { const AMFEnumDescriptionEntry* pEnumDescriptionCache = pEnumDescription; err = AMFVariantChangeType(pDest, pSrc, AMF_VARIANT_INT64); bool found = false; if(err == AMF_OK) { //mean numeric came. validating while(pEnumDescriptionCache->name) { if(pEnumDescriptionCache->value == AMFVariantGetInt64(pDest)) { AMFVariantAssignInt64(pDest, pEnumDescriptionCache->value); found = true; break; } pEnumDescriptionCache++; } err = found ? AMF_OK : AMF_INVALID_ARG; } if(!found) { pEnumDescriptionCache = pEnumDescription; err = AMFVariantChangeType(pDest, pSrc, AMF_VARIANT_WSTRING); if(err == AMF_OK) { //string came. validating and assigning numeric found = false; while(pEnumDescriptionCache->name) { if(amf_wstring(pEnumDescriptionCache->name) == AMFVariantGetWString(pDest)) { AMFVariantAssignInt64(pDest, pEnumDescriptionCache->value); found = true; break; } pEnumDescriptionCache++; } err = found ? AMF_OK : AMF_INVALID_ARG; } } } else { err = AMFVariantChangeType(pDest, pSrc, AMF_VARIANT_INT64); } } break; default: err = AMFVariantChangeType(pDest, pSrc, eType); break; } return err; } //------------------------------------------------------------------------------------------------- AMFPropertyInfoImpl::AMFPropertyInfoImpl(const wchar_t* name, const wchar_t* desc, AMF_VARIANT_TYPE type, AMF_PROPERTY_CONTENT_TYPE contentType, AMFVariantStruct defaultValue, AMFVariantStruct minValue, AMFVariantStruct maxValue, bool allowChangeInRuntime, const AMFEnumDescriptionEntry* pEnumDescription) : m_name(), m_desc() { AMF_PROPERTY_ACCESS_TYPE accessTypeTmp = allowChangeInRuntime ? AMF_PROPERTY_ACCESS_FULL : AMF_PROPERTY_ACCESS_READ_WRITE; Init(name, desc, type, contentType, defaultValue, minValue, maxValue, accessTypeTmp, pEnumDescription); } //------------------------------------------------------------------------------------------------- AMFPropertyInfoImpl::AMFPropertyInfoImpl(const wchar_t* name, const wchar_t* desc, AMF_VARIANT_TYPE type, AMF_PROPERTY_CONTENT_TYPE contentType, AMFVariantStruct defaultValue, AMFVariantStruct minValue, AMFVariantStruct maxValue, AMF_PROPERTY_ACCESS_TYPE accessType, const AMFEnumDescriptionEntry* pEnumDescription) : m_name(), m_desc() { Init(name, desc, type, contentType, defaultValue, minValue, maxValue, accessType, pEnumDescription); } //------------------------------------------------------------------------------------------------- AMFPropertyInfoImpl::AMFPropertyInfoImpl() : m_name(), m_desc() { AMFVariantInit(&this->defaultValue); AMFVariantInit(&this->minValue); AMFVariantInit(&this->maxValue); name = L""; desc = L""; type = AMF_VARIANT_EMPTY; contentType = AMF_PROPERTY_CONTENT_TYPE(-1); accessType = AMF_PROPERTY_ACCESS_FULL; } //------------------------------------------------------------------------------------------------- void AMFPropertyInfoImpl::Init(const wchar_t* name_, const wchar_t* desc_, AMF_VARIANT_TYPE type_, AMF_PROPERTY_CONTENT_TYPE contentType_, AMFVariantStruct defaultValue_, AMFVariantStruct minValue_, AMFVariantStruct maxValue_, AMF_PROPERTY_ACCESS_TYPE accessType_, const AMFEnumDescriptionEntry* pEnumDescription_) { m_name = name_; name = m_name.c_str(); m_desc = desc_; desc = m_desc.c_str(); type = type_; contentType = contentType_; accessType = accessType_; AMFVariantInit(&defaultValue); AMFVariantInit(&minValue); AMFVariantInit(&maxValue); pEnumDescription = pEnumDescription_; switch(type) { case AMF_VARIANT_BOOL: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignBool(&defaultValue, false); } } break; case AMF_VARIANT_RECT: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignRect(&defaultValue, AMFConstructRect(0, 0, 0, 0)); } } break; case AMF_VARIANT_SIZE: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignSize(&defaultValue, AMFConstructSize(0, 0)); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignSize(&minValue, AMFConstructSize(INT_MIN, INT_MIN)); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignSize(&maxValue, AMFConstructSize(INT_MAX, INT_MAX)); } } break; case AMF_VARIANT_POINT: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignPoint(&defaultValue, AMFConstructPoint(0, 0)); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignPoint(&minValue, AMFConstructPoint(INT_MIN, INT_MIN)); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignPoint(&maxValue, AMFConstructPoint(INT_MAX, INT_MAX)); } } break; case AMF_VARIANT_RATE: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignRate(&defaultValue, AMFConstructRate(0, 0)); } if (CastVariantToAMFProperty(&this->minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignRate(&this->minValue, AMFConstructRate(0, 1)); } if (CastVariantToAMFProperty(&this->maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignRate(&this->maxValue, AMFConstructRate(INT_MAX, INT_MAX)); } } break; case AMF_VARIANT_RATIO: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignRatio(&defaultValue, AMFConstructRatio(0, 0)); } } break; case AMF_VARIANT_COLOR: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignColor(&defaultValue, AMFConstructColor(0, 0, 0, 255)); } } break; case AMF_VARIANT_INT64: { if(pEnumDescription) { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignInt64(&defaultValue, pEnumDescription->value); } } else //AMF_PROPERTY_CONTENT_DEFAULT { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignInt64(&defaultValue, 0); } if(CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignInt64(&minValue, INT_MIN); } if(CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignInt64(&maxValue, INT_MAX); } } } break; case AMF_VARIANT_DOUBLE: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignDouble(&defaultValue, 0); } if(CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignDouble(&minValue, DBL_MIN); } if(CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignDouble(&maxValue, DBL_MAX); } } break; case AMF_VARIANT_STRING: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignString(&maxValue, ""); } } break; case AMF_VARIANT_WSTRING: { if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignWString(&maxValue, L""); } } break; case AMF_VARIANT_INTERFACE: if(CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignWString(&maxValue, L""); } break; case AMF_VARIANT_FLOAT: { if (CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloat(&defaultValue, 0); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloat(&minValue, FLT_MIN); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloat(&maxValue, FLT_MAX); } } break; case AMF_VARIANT_FLOAT_SIZE: { if (CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatSize(&defaultValue, AMFConstructFloatSize(0, 0)); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatSize(&minValue, AMFConstructFloatSize(FLT_MIN, FLT_MIN)); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatSize(&maxValue, AMFConstructFloatSize(FLT_MAX, FLT_MAX)); } } break; case AMF_VARIANT_FLOAT_POINT2D: { if (CastVariantToAMFProperty(&defaultValue, &defaultValue, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatPoint2D(&defaultValue, AMFConstructFloatPoint2D(0, 0)); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatPoint2D(&minValue, AMFConstructFloatPoint2D(FLT_MIN, FLT_MIN)); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatPoint2D(&maxValue, AMFConstructFloatPoint2D(FLT_MAX, FLT_MAX)); } } break; case AMF_VARIANT_FLOAT_POINT3D: { if (CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatPoint3D(&defaultValue, AMFConstructFloatPoint3D(0, 0, 0)); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatPoint3D(&minValue, AMFConstructFloatPoint3D(FLT_MIN, FLT_MIN, FLT_MIN)); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatPoint3D(&maxValue, AMFConstructFloatPoint3D(FLT_MAX, FLT_MAX, FLT_MAX)); } } break; case AMF_VARIANT_FLOAT_VECTOR4D: { if (CastVariantToAMFProperty(&defaultValue, &defaultValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatVector4D(&defaultValue, AMFConstructFloatVector4D(0, 0, 0, 0)); } if (CastVariantToAMFProperty(&minValue, &minValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatVector4D(&minValue, AMFConstructFloatVector4D(FLT_MIN, FLT_MIN, FLT_MIN, FLT_MIN)); } if (CastVariantToAMFProperty(&maxValue, &maxValue_, type, contentType, pEnumDescription) != AMF_OK) { AMFVariantAssignFloatVector4D(&maxValue, AMFConstructFloatVector4D(FLT_MAX, FLT_MAX, FLT_MAX, FLT_MAX)); } } break; default: break; } value = defaultValue; } AMFPropertyInfoImpl::AMFPropertyInfoImpl(const AMFPropertyInfoImpl& propertyInfo) : AMFPropertyInfo(), m_name(), m_desc() { Init(propertyInfo.name, propertyInfo.desc, propertyInfo.type, propertyInfo.contentType, propertyInfo.defaultValue, propertyInfo.minValue, propertyInfo.maxValue, propertyInfo.accessType, propertyInfo.pEnumDescription); } //------------------------------------------------------------------------------------------------- AMFPropertyInfoImpl& AMFPropertyInfoImpl::operator=(const AMFPropertyInfoImpl& propertyInfo) { // store name and desc inside instance in m_sName and m_sDesc recpectively; // m_pName and m_pDesc are pointed to our local copies this->m_name = propertyInfo.name; this->m_desc = propertyInfo.desc; this->name = m_name.c_str(); this->desc = m_desc.c_str(); this->type = propertyInfo.type; this->contentType = propertyInfo.contentType; this->accessType = propertyInfo.accessType; AMFVariantCopy(&this->defaultValue, &propertyInfo.defaultValue); AMFVariantCopy(&this->minValue, &propertyInfo.minValue); AMFVariantCopy(&this->maxValue, &propertyInfo.maxValue); this->pEnumDescription = propertyInfo.pEnumDescription; this->value = propertyInfo.value; this->userModified = propertyInfo.userModified; return *this; } //------------------------------------------------------------------------------------------------- AMFPropertyInfoImpl::~AMFPropertyInfoImpl() { AMFVariantClear(&this->defaultValue); AMFVariantClear(&this->minValue); AMFVariantClear(&this->maxValue); } //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- //------------------------------------------------------------------------------------------------- ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/PropertyStorageExImpl.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // ///------------------------------------------------------------------------- /// @file PropertyStorageExImpl.h /// @brief AMFPropertyStorageExImpl header ///------------------------------------------------------------------------- #ifndef AMF_PropertyStorageExImpl_h #define AMF_PropertyStorageExImpl_h #pragma once #include "../include/core/PropertyStorageEx.h" #include "Thread.h" #include "InterfaceImpl.h" #include "ObservableImpl.h" #include "TraceAdapter.h" #include #include #include namespace amf { AMF_RESULT CastVariantToAMFProperty(AMFVariantStruct* pDest, const AMFVariantStruct* pSrc, AMF_VARIANT_TYPE eType, AMF_PROPERTY_CONTENT_TYPE contentType, const AMFEnumDescriptionEntry* pEnumDescription = 0); //--------------------------------------------------------------------------------------------- class AMFPropertyInfoImpl : public AMFPropertyInfo { private: amf_wstring m_name; amf_wstring m_desc; void Init(const wchar_t* name, const wchar_t* desc, AMF_VARIANT_TYPE type, AMF_PROPERTY_CONTENT_TYPE contentType, AMFVariantStruct defaultValue, AMFVariantStruct minValue, AMFVariantStruct maxValue, AMF_PROPERTY_ACCESS_TYPE accessType, const AMFEnumDescriptionEntry* pEnumDescription); public: AMFVariant value; amf_bool userModified = false; public: AMFPropertyInfoImpl(const wchar_t* name, const wchar_t* desc, AMF_VARIANT_TYPE type, AMF_PROPERTY_CONTENT_TYPE contentType, AMFVariantStruct defaultValue, AMFVariantStruct minValue, AMFVariantStruct maxValue, bool allowChangeInRuntime, const AMFEnumDescriptionEntry* pEnumDescription); AMFPropertyInfoImpl(const wchar_t* name, const wchar_t* desc, AMF_VARIANT_TYPE type, AMF_PROPERTY_CONTENT_TYPE contentType, AMFVariantStruct defaultValue, AMFVariantStruct minValue, AMFVariantStruct maxValue, AMF_PROPERTY_ACCESS_TYPE accessType, const AMFEnumDescriptionEntry* pEnumDescription); AMFPropertyInfoImpl(); AMFPropertyInfoImpl(const AMFPropertyInfoImpl& propertyInfo); AMFPropertyInfoImpl& operator=(const AMFPropertyInfoImpl& propertyInfo); virtual ~AMFPropertyInfoImpl(); virtual void OnPropertyChanged() { } }; typedef amf_map > PropertyInfoMap; //--------------------------------------------------------------------------------------------- template class AMFPropertyStorageExImpl : public _TBase, public AMFObservableImpl { protected: PropertyInfoMap m_PropertiesInfo; public: AMFPropertyStorageExImpl() { } virtual ~AMFPropertyStorageExImpl() { } // interface access AMF_BEGIN_INTERFACE_MAP AMF_INTERFACE_ENTRY(AMFPropertyStorage) AMF_INTERFACE_ENTRY(AMFPropertyStorageEx) AMF_END_INTERFACE_MAP using _TBase::GetProperty; using _TBase::SetProperty; // interface //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL Clear() { ResetDefaultValues(); return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL AddTo(AMFPropertyStorage* pDest, bool overwrite, bool /*deep*/) const { AMF_RETURN_IF_INVALID_POINTER(pDest); if (pDest != this) { for (PropertyInfoMap::const_iterator it = m_PropertiesInfo.begin(); it != m_PropertiesInfo.end(); it++) { if (!overwrite && pDest->HasProperty(it->first.c_str())) { continue; } AMF_RESULT err = pDest->SetProperty(it->first.c_str(), it->second->value); if (err != AMF_INVALID_ARG) // not validated - skip it { AMF_RETURN_IF_FAILED(err, L"AddTo() - failed to copy property=%s", it->first.c_str()); } } } return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL CopyTo(AMFPropertyStorage* pDest, bool deep) const { AMF_RETURN_IF_INVALID_POINTER(pDest); if (pDest != this) { pDest->Clear(); return AddTo(pDest, true, deep); } return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL SetProperty(const wchar_t* name, AMFVariantStruct value) { AMF_RETURN_IF_INVALID_POINTER(name); const AMFPropertyInfo* pParamInfo = NULL; AMF_RESULT err = GetPropertyInfo(name, &pParamInfo); if (err != AMF_OK) { return err; } if (pParamInfo && !pParamInfo->AllowedWrite()) { return AMF_ACCESS_DENIED; } return SetPrivateProperty(name, value); } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL GetProperty(const wchar_t* name, AMFVariantStruct* pValue) const { AMF_RETURN_IF_INVALID_POINTER(name); AMF_RETURN_IF_INVALID_POINTER(pValue); const AMFPropertyInfo* pParamInfo = NULL; AMF_RESULT err = GetPropertyInfo(name, &pParamInfo); if (err != AMF_OK) { return err; } if (pParamInfo && !pParamInfo->AllowedRead()) { return AMF_ACCESS_DENIED; } return GetPrivateProperty(name, pValue); } //------------------------------------------------------------------------------------------------- virtual bool AMF_STD_CALL HasProperty(const wchar_t* name) const { const AMFPropertyInfo* pParamInfo = NULL; AMF_RESULT err = GetPropertyInfo(name, &pParamInfo); return (err != AMF_OK) ? false : true; } //------------------------------------------------------------------------------------------------- virtual amf_size AMF_STD_CALL GetPropertyCount() const { return m_PropertiesInfo.size(); } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL GetPropertyAt(amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue) const { AMF_RETURN_IF_INVALID_POINTER(name); AMF_RETURN_IF_INVALID_POINTER(pValue); AMF_RETURN_IF_FALSE(nameSize != 0, AMF_INVALID_ARG); AMF_RETURN_IF_FALSE(index < m_PropertiesInfo.size(), AMF_INVALID_ARG); PropertyInfoMap::const_iterator found = m_PropertiesInfo.begin(); for (amf_size i = 0; i < index; i++) { found++; } size_t copySize = AMF_MIN(nameSize-1, found->first.length()); memcpy(name, found->first.c_str(), copySize * sizeof(wchar_t)); name[copySize] = 0; AMFVariantCopy(pValue, &found->second->value); return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual amf_size AMF_STD_CALL GetPropertiesInfoCount() const { return m_PropertiesInfo.size(); } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL GetPropertyInfo(amf_size szInd, const AMFPropertyInfo** ppParamInfo) const { AMF_RETURN_IF_INVALID_POINTER(ppParamInfo); AMF_RETURN_IF_FALSE(szInd < m_PropertiesInfo.size(), AMF_INVALID_ARG); PropertyInfoMap::const_iterator it = m_PropertiesInfo.begin(); for (; szInd > 0; --szInd) { it++; } *ppParamInfo = it->second.get(); return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL GetPropertyInfo(const wchar_t* name, const AMFPropertyInfo** ppParamInfo) const { AMF_RETURN_IF_INVALID_POINTER(name); AMF_RETURN_IF_INVALID_POINTER(ppParamInfo); PropertyInfoMap::const_iterator it = m_PropertiesInfo.find(name); if (it != m_PropertiesInfo.end()) { *ppParamInfo = it->second.get(); return AMF_OK; } return AMF_NOT_FOUND; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL ValidateProperty(const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated) const { AMF_RETURN_IF_INVALID_POINTER(name); AMF_RETURN_IF_INVALID_POINTER(pOutValidated); AMF_RESULT err = AMF_OK; const AMFPropertyInfo* pParamInfo = NULL; AMF_RETURN_IF_FAILED(GetPropertyInfo(name, &pParamInfo), L"Property=%s", name); AMF_RETURN_IF_FAILED(CastVariantToAMFProperty(pOutValidated, &value, pParamInfo->type, pParamInfo->contentType, pParamInfo->pEnumDescription), L"Property=%s", name); switch(pParamInfo->type) { case AMF_VARIANT_INT64: if((pParamInfo->minValue.type != AMF_VARIANT_EMPTY && AMFVariantGetInt64(pOutValidated) < AMFVariantGetInt64(&pParamInfo->minValue)) || (pParamInfo->maxValue.type != AMF_VARIANT_EMPTY && AMFVariantGetInt64(pOutValidated) > AMFVariantGetInt64(&pParamInfo->maxValue)) ) { err = AMF_OUT_OF_RANGE; } break; case AMF_VARIANT_DOUBLE: if((AMFVariantGetDouble(pOutValidated) < AMFVariantGetDouble(&pParamInfo->minValue)) || (AMFVariantGetDouble(pOutValidated) > AMFVariantGetDouble(&pParamInfo->maxValue)) ) { err = AMF_OUT_OF_RANGE; } break; case AMF_VARIANT_FLOAT: if ((AMFVariantGetFloat(pOutValidated) < AMFVariantGetFloat(&pParamInfo->minValue)) || (AMFVariantGetFloat(pOutValidated) > AMFVariantGetFloat(&pParamInfo->maxValue))) { err = AMF_OUT_OF_RANGE; } break; case AMF_VARIANT_RATE: { // NOTE: denominator can't be 0 const AMFRate& validatedSize = AMFVariantGetRate(pOutValidated); AMFRate minSize = AMFConstructRate(0, 1); AMFRate maxSize = AMFConstructRate(INT_MAX, INT_MAX); if (pParamInfo->minValue.type != AMF_VARIANT_EMPTY) { minSize = AMFVariantGetRate(&pParamInfo->minValue); } if (pParamInfo->maxValue.type != AMF_VARIANT_EMPTY) { maxSize = AMFVariantGetRate(&pParamInfo->maxValue); } if (validatedSize.num < minSize.num || validatedSize.num > maxSize.num || validatedSize.den < minSize.den || validatedSize.den > maxSize.den) { err = AMF_OUT_OF_RANGE; } } break; case AMF_VARIANT_SIZE: { AMFSize validatedSize = AMFVariantGetSize(pOutValidated); AMFSize minSize = AMFConstructSize(0, 0); AMFSize maxSize = AMFConstructSize(INT_MAX, INT_MAX); if (pParamInfo->minValue.type != AMF_VARIANT_EMPTY) { minSize = AMFVariantGetSize(&pParamInfo->minValue); } if (pParamInfo->maxValue.type != AMF_VARIANT_EMPTY) { maxSize = AMFVariantGetSize(&pParamInfo->maxValue); } if (validatedSize.width < minSize.width || validatedSize.height < minSize.height || validatedSize.width > maxSize.width || validatedSize.height > maxSize.height) { err = AMF_OUT_OF_RANGE; } } break; case AMF_VARIANT_FLOAT_SIZE: { AMFFloatSize validatedSize = AMFVariantGetFloatSize(pOutValidated); AMFFloatSize minSize = AMFConstructFloatSize(0, 0); AMFFloatSize maxSize = AMFConstructFloatSize(FLT_MIN, FLT_MAX); if (pParamInfo->minValue.type != AMF_VARIANT_EMPTY) { minSize = AMFVariantGetFloatSize(&pParamInfo->minValue); } if (pParamInfo->maxValue.type != AMF_VARIANT_EMPTY) { maxSize = AMFVariantGetFloatSize(&pParamInfo->maxValue); } if (validatedSize.width < minSize.width || validatedSize.height < minSize.height || validatedSize.width > maxSize.width || validatedSize.height > maxSize.height) { err = AMF_OUT_OF_RANGE; } } break; default: // GK: Clang issues a warning when not every value of an enum is handled in a switch-case break; } return err; } //------------------------------------------------------------------------------------------------- virtual void AMF_STD_CALL OnPropertyChanged(const wchar_t* /*name*/){ } //------------------------------------------------------------------------------------------------- virtual void AMF_STD_CALL AddObserver(AMFPropertyStorageObserver* pObserver) { AMFObservableImpl::AddObserver(pObserver); } //------------------------------------------------------------------------------------------------- virtual void AMF_STD_CALL RemoveObserver(AMFPropertyStorageObserver* pObserver) { AMFObservableImpl::RemoveObserver(pObserver); } //------------------------------------------------------------------------------------------------- protected: //------------------------------------------------------------------------------------------------- AMF_RESULT SetAccessType(const wchar_t* name, AMF_PROPERTY_ACCESS_TYPE accessType) { AMF_RETURN_IF_INVALID_POINTER(name); PropertyInfoMap::iterator found = m_PropertiesInfo.find(name); AMF_RETURN_IF_FALSE(found != m_PropertiesInfo.end(), AMF_NOT_FOUND); if (found->second->accessType == accessType) { return AMF_OK; } found->second->accessType = accessType; OnPropertyChanged(name); NotifyObservers(&AMFPropertyStorageObserver::OnPropertyChanged, name); return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT SetPrivateProperty(const wchar_t* name, AMFVariantStruct value) { AMF_RETURN_IF_INVALID_POINTER(name); AMFVariant validatedValue; AMF_RESULT validateResult = ValidateProperty(name, value, &validatedValue); if (validateResult != AMF_OK) { return validateResult; } PropertyInfoMap::iterator found = m_PropertiesInfo.find(name); if (found == m_PropertiesInfo.end()) { return AMF_NOT_FOUND; } if (found->second->value == validatedValue) { return AMF_OK; } found->second->value = validatedValue; found->second->OnPropertyChanged(); OnPropertyChanged(name); NotifyObservers(&AMFPropertyStorageObserver::OnPropertyChanged, name); return AMF_OK; } //------------------------------------------------------------------------------------------------- AMF_RESULT GetPrivateProperty(const wchar_t* name, AMFVariantStruct* pValue) const { AMF_RETURN_IF_INVALID_POINTER(name); AMF_RETURN_IF_INVALID_POINTER(pValue); PropertyInfoMap::const_iterator found = m_PropertiesInfo.find(name); if (found != m_PropertiesInfo.end()) { AMFVariantCopy(pValue, &found->second->value); return AMF_OK; } // NOTE: needed for internal components that don't automatically // expose their properties in the main map... const AMFPropertyInfo* pParamInfo; if (GetPropertyInfo(name, &pParamInfo) == AMF_OK) { AMFVariantCopy(pValue, &pParamInfo->defaultValue); return AMF_OK; } return AMF_NOT_FOUND; } //------------------------------------------------------------------------------------------------- template AMF_RESULT AMF_STD_CALL SetPrivateProperty(const wchar_t* name, const _T& value) { AMF_RESULT err = SetPrivateProperty(name, static_cast(AMFVariant(value))); return err; } //------------------------------------------------------------------------------------------------- template AMF_RESULT AMF_STD_CALL GetPrivateProperty(const wchar_t* name, _T* pValue) const { AMFVariant var; AMF_RESULT err = GetPrivateProperty(name, static_cast(&var)); if(err == AMF_OK) { *pValue = static_cast<_T>(var); } return err; } //------------------------------------------------------------------------------------------------- bool HasPrivateProperty(const wchar_t* name) const { return m_PropertiesInfo.find(name) != m_PropertiesInfo.end(); } //------------------------------------------------------------------------------------------------- bool IsRuntimeChange(const wchar_t* name) const { PropertyInfoMap::const_iterator it = m_PropertiesInfo.find(name); return (it != m_PropertiesInfo.end()) ? it->second->AllowedChangeInRuntime() : false; } //------------------------------------------------------------------------------------------------- void ResetDefaultValues() { // copy defaults to property storage for (PropertyInfoMap::iterator it = m_PropertiesInfo.begin(); it != m_PropertiesInfo.end(); ++it) { AMFPropertyInfoImpl* info = it->second.get(); info->value = info->defaultValue; info->userModified = false; } } //------------------------------------------------------------------------------------------------- private: AMFPropertyStorageExImpl(const AMFPropertyStorageExImpl&); AMFPropertyStorageExImpl& operator=(const AMFPropertyStorageExImpl&); }; extern AMFCriticalSection ms_csAMFPropertyStorageExImplMaps; //--------------------------------------------------------------------------------------------- #define AMFPrimitivePropertyInfoMapBegin \ { \ amf::AMFPropertyInfoImpl* s_PropertiesInfo[] = \ { #define AMFPrimitivePropertyInfoMapEnd \ }; \ for (amf_size i = 0; i < sizeof(s_PropertiesInfo) / sizeof(s_PropertiesInfo[0]); ++i) \ { \ amf::AMFPropertyInfoImpl* pPropInfo = s_PropertiesInfo[i]; \ m_PropertiesInfo[pPropInfo->name].reset(pPropInfo); \ } \ } #define AMFPropertyInfoBool(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_BOOL, 0, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoEnum(_name, _desc, _defaultValue, pEnumDescription, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_INT64, 0, amf::AMFVariant(amf_int64(_defaultValue)), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, pEnumDescription) #define AMFPropertyInfoInt64(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_INT64, 0, amf::AMFVariant(amf_int64(_defaultValue)), \ amf::AMFVariant(amf_int64(_minValue)), amf::AMFVariant(amf_int64(_maxValue)), _AccessType, 0) #define AMFPropertyInfoDouble(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_DOUBLE, 0, amf::AMFVariant(amf_double(_defaultValue)), \ amf::AMFVariant(amf_double(_minValue)), amf::AMFVariant(amf_double(_maxValue)), _AccessType, 0) #define AMFPropertyInfoFloat(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_FLOAT, 0, amf::AMFVariant(amf_float(_defaultValue)), \ amf::AMFVariant(amf_float(_minValue)), amf::AMFVariant(amf_float(_maxValue)), _AccessType, 0) #define AMFPropertyInfoRect(_name, _desc, defaultLeft, defaultTop, defaultRight, defaultBottom, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_RECT, 0, amf::AMFVariant(AMFConstructRect(defaultLeft, defaultTop, defaultRight, defaultBottom)), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoPoint(_name, _desc, defaultX, defaultY, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_POINT, 0, amf::AMFVariant(AMFConstructPoint(defaultX, defaultY)), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoSize(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_SIZE, 0, amf::AMFVariant(AMFSize(_defaultValue)), \ amf::AMFVariant(AMFSize(_minValue)), amf::AMFVariant(AMFSize(_maxValue)), _AccessType, 0) #define AMFPropertyInfoFloatSize(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_FLOAT_SIZE, 0, amf::AMFVariant(AMFFloatSize(_defaultValue)), \ amf::AMFVariant(AMFFloatSize(_minValue)), amf::AMFVariant(AMFFloatSize(_maxValue)), _AccessType, 0) #define AMFPropertyInfoRate(_name, _desc, defaultNum, defaultDen, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_RATE, 0, amf::AMFVariant(AMFConstructRate(defaultNum, defaultDen)), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoRateEx(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_RATE, 0, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(_minValue), amf::AMFVariant(_maxValue), _AccessType, 0) #define AMFPropertyInfoRatio(_name, _desc, defaultNum, defaultDen, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_RATIO, 0, amf::AMFVariant(AMFConstructRatio(defaultNum, defaultDen)), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoColor(_name, _desc, defaultR, defaultG, defaultB, defaultA, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_COLOR, 0, amf::AMFVariant(AMFConstructColor(defaultR, defaultG, defaultB, defaultA)), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoString(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_STRING, 0, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoWString(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_WSTRING, 0, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoInterface(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_INTERFACE, 0, amf::AMFVariant(amf::AMFInterfacePtr(_defaultValue)), \ amf::AMFVariant(amf::AMFInterfacePtr()), amf::AMFVariant(amf::AMFInterfacePtr()), _AccessType, 0) #define AMFPropertyInfoXML(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_STRING, AMF_PROPERTY_CONTENT_XML, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoPath(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_WSTRING, AMF_PROPERTY_CONTENT_FILE_OPEN_PATH, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoSavePath(_name, _desc, _defaultValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_WSTRING, AMF_PROPERTY_CONTENT_FILE_SAVE_PATH, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(), amf::AMFVariant(), _AccessType, 0) #define AMFPropertyInfoFloatVector4D(_name, _desc, _defaultValue, _minValue, _maxValue, _AccessType) \ new amf::AMFPropertyInfoImpl(_name, _desc, amf::AMF_VARIANT_FLOAT_VECTOR4D, 0, amf::AMFVariant(_defaultValue), \ amf::AMFVariant(_minValue), amf::AMFVariant(_maxValue), _AccessType, 0) } // namespace amf #endif // #ifndef AMF_PropertyStorageExImpl_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/PropertyStorageImpl.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // ///------------------------------------------------------------------------- /// @file PropertyStorageImpl.h /// @brief AMFPropertyStorageImpl header ///------------------------------------------------------------------------- #ifndef AMF_PropertyStorageImpl_h #define AMF_PropertyStorageImpl_h #pragma once #include "../include/core/PropertyStorage.h" #include "Thread.h" #include "InterfaceImpl.h" #include "ObservableImpl.h" #include "TraceAdapter.h" namespace amf { //--------------------------------------------------------------------------------------------- template class AMFPropertyStorageImpl : public _TBase, public AMFObservableImpl { public: //------------------------------------------------------------------------------------------------- AMFPropertyStorageImpl() : m_PropertyValues() { } //------------------------------------------------------------------------------------------------- virtual ~AMFPropertyStorageImpl() { } //------------------------------------------------------------------------------------------------- // interface access AMF_BEGIN_INTERFACE_MAP AMF_INTERFACE_ENTRY(AMFPropertyStorage) AMF_END_INTERFACE_MAP //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL SetProperty(const wchar_t* pName, AMFVariantStruct value) { AMF_RETURN_IF_INVALID_POINTER(pName); m_PropertyValues[pName] = value; OnPropertyChanged(pName); NotifyObservers(&AMFPropertyStorageObserver::OnPropertyChanged, pName); return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL GetProperty(const wchar_t* pName, AMFVariantStruct* pValue) const { AMF_RETURN_IF_INVALID_POINTER(pName); AMF_RETURN_IF_INVALID_POINTER(pValue); amf_wstring name(pName); amf_map::const_iterator found = m_PropertyValues.find(name); if(found != m_PropertyValues.end()) { AMFVariantCopy(pValue, &found->second); return AMF_OK; } return AMF_NOT_FOUND; } //------------------------------------------------------------------------------------------------- virtual bool AMF_STD_CALL HasProperty(const wchar_t* pName) const { AMF_ASSERT(pName != NULL); return m_PropertyValues.find(pName) != m_PropertyValues.end(); } //------------------------------------------------------------------------------------------------- virtual amf_size AMF_STD_CALL GetPropertyCount() const { return m_PropertyValues.size(); } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL GetPropertyAt(amf_size index, wchar_t* pName, amf_size nameSize, AMFVariantStruct* pValue) const { AMF_RETURN_IF_INVALID_POINTER(pName); AMF_RETURN_IF_INVALID_POINTER(pValue); AMF_RETURN_IF_FALSE(nameSize != 0, AMF_INVALID_ARG); amf_map::const_iterator found = m_PropertyValues.begin(); if(found == m_PropertyValues.end()) { return AMF_INVALID_ARG; } for( amf_size i = 0; i < index; i++) { found++; if(found == m_PropertyValues.end()) { return AMF_INVALID_ARG; } } size_t copySize = AMF_MIN(nameSize-1, found->first.length()); memcpy(pName, found->first.c_str(), copySize * sizeof(wchar_t)); pName[copySize] = 0; AMFVariantCopy(pValue, &found->second); return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL Clear() { m_PropertyValues.clear(); return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL AddTo(AMFPropertyStorage* pDest, bool overwrite, bool /*deep*/) const { AMF_RETURN_IF_INVALID_POINTER(pDest); AMF_RESULT err = AMF_OK; amf_map::const_iterator it = m_PropertyValues.begin(); for(; it != m_PropertyValues.end(); it++) { if(!HasProperty(it->first.c_str())) // ignore properties which aren't accessible { continue; } if(!overwrite) { if(pDest->HasProperty(it->first.c_str())) { continue; } } { err = pDest->SetProperty(it->first.c_str(), it->second); } if(err == AMF_ACCESS_DENIED) { continue; } AMF_RETURN_IF_FAILED(err, L"AddTo() - failed to copy property=%s", it->first.c_str()); } return AMF_OK; } //------------------------------------------------------------------------------------------------- virtual AMF_RESULT AMF_STD_CALL CopyTo(AMFPropertyStorage* pDest, bool deep) const { AMF_RETURN_IF_INVALID_POINTER(pDest); if(pDest != this) { pDest->Clear(); return AddTo(pDest, true, deep); } else { return AMF_OK; } } //------------------------------------------------------------------------------------------------- virtual void AMF_STD_CALL OnPropertyChanged(const wchar_t* /*name*/) { } //------------------------------------------------------------------------------------------------- virtual void AMF_STD_CALL AddObserver(AMFPropertyStorageObserver* pObserver) { AMFObservableImpl::AddObserver(pObserver); } //------------------------------------------------------------------------------------------------- virtual void AMF_STD_CALL RemoveObserver(AMFPropertyStorageObserver* pObserver) { AMFObservableImpl::RemoveObserver(pObserver); } //------------------------------------------------------------------------------------------------- protected: //------------------------------------------------------------------------------------------------- amf_map m_PropertyValues; }; //--------------------------------------------------------------------------------------------- //--------------------------------------------------------------------------------------------- } #endif // AMF_PropertyStorageImpl_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/Thread.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #if defined(_WIN32) #include #else #include #endif #include "Thread.h" #if defined(METRO_APP) #include #include #endif namespace amf { //---------------------------------------------------------------------------- AMFEvent::AMFEvent(bool bInitiallyOwned, bool bManualReset, const wchar_t* pName) : m_hSyncObject() { m_hSyncObject = amf_create_event(bInitiallyOwned, bManualReset, pName); } //---------------------------------------------------------------------------- AMFEvent::~AMFEvent() { amf_delete_event(m_hSyncObject); } //---------------------------------------------------------------------------- bool AMFEvent::Lock(amf_ulong ulTimeout) { return amf_wait_for_event(m_hSyncObject, ulTimeout); } //---------------------------------------------------------------------------- bool AMFEvent::LockTimeout(amf_ulong ulTimeout) { return amf_wait_for_event_timeout(m_hSyncObject, ulTimeout); } //---------------------------------------------------------------------------- bool AMFEvent::Unlock() { return true; } //---------------------------------------------------------------------------- bool AMFEvent::SetEvent() { return amf_set_event(m_hSyncObject); } //---------------------------------------------------------------------------- bool AMFEvent::ResetEvent() { return amf_reset_event(m_hSyncObject); } //---------------------------------------------------------------------------- //---------------------------------------------------------------------------- AMFMutex::AMFMutex(bool bInitiallyOwned, const wchar_t* pName #if defined(_WIN32) , bool bOpenExistent #endif ):m_hSyncObject() { #if defined(_WIN32) if(bOpenExistent) { m_hSyncObject = amf_open_mutex(pName); } else #else //#pragma message AMF_TODO("Open mutex!!! missing functionality in Linux!!!") #endif { m_hSyncObject = amf_create_mutex(bInitiallyOwned, pName); } } //---------------------------------------------------------------------------- AMFMutex::~AMFMutex() { if(m_hSyncObject) { amf_delete_mutex(m_hSyncObject); } } //---------------------------------------------------------------------------- bool AMFMutex::Lock(amf_ulong ulTimeout) { if(m_hSyncObject) { return amf_wait_for_mutex(m_hSyncObject, ulTimeout); } else { return false; } } //---------------------------------------------------------------------------- bool AMFMutex::Unlock() { if(m_hSyncObject) { return amf_release_mutex(m_hSyncObject); } else { return false; } } //---------------------------------------------------------------------------- bool AMFMutex::IsValid() { return m_hSyncObject != NULL; } //---------------------------------------------------------------------------- //---------------------------------------------------------------------------- AMFCriticalSection::AMFCriticalSection() : m_Sect() { m_Sect = amf_create_critical_section(); } //---------------------------------------------------------------------------- AMFCriticalSection::~AMFCriticalSection() { amf_delete_critical_section(m_Sect); } //---------------------------------------------------------------------------- bool AMFCriticalSection::Lock(amf_ulong ulTimeout) { return (ulTimeout != AMF_INFINITE) ? amf_wait_critical_section(m_Sect, ulTimeout) : amf_enter_critical_section(m_Sect); } //---------------------------------------------------------------------------- bool AMFCriticalSection::Unlock() { return amf_leave_critical_section(m_Sect); } //---------------------------------------------------------------------------- AMFSemaphore::AMFSemaphore(amf_long iInitCount, amf_long iMaxCount, const wchar_t* pName) : m_hSemaphore(NULL) { Create(iInitCount, iMaxCount, pName); } //---------------------------------------------------------------------------- AMFSemaphore::~AMFSemaphore() { amf_delete_semaphore(m_hSemaphore); } //---------------------------------------------------------------------------- bool AMFSemaphore::Create(amf_long iInitCount, amf_long iMaxCount, const wchar_t* pName) { if(m_hSemaphore != NULL) // delete old one { amf_delete_semaphore(m_hSemaphore); m_hSemaphore = NULL; } if(iMaxCount > 0) { m_hSemaphore = amf_create_semaphore(iInitCount, iMaxCount, pName); } return true; } //---------------------------------------------------------------------------- bool AMFSemaphore::Lock(amf_ulong ulTimeout) { return amf_wait_for_semaphore(m_hSemaphore, ulTimeout); } //---------------------------------------------------------------------------- bool AMFSemaphore::Unlock() { amf_long iOldCount = 0; return amf_release_semaphore(m_hSemaphore, 1, &iOldCount); } //---------------------------------------------------------------------------- AMFLock::AMFLock(AMFSyncBase* pBase, amf_ulong ulTimeout) : m_pBase(pBase), m_bLocked() { m_bLocked = Lock(ulTimeout); } //---------------------------------------------------------------------------- AMFLock::~AMFLock() { if (IsLocked() == true) { Unlock(); } } //---------------------------------------------------------------------------- bool AMFLock::Lock(amf_ulong ulTimeout) { if(m_pBase == NULL) { return false; } m_bLocked = m_pBase->Lock(ulTimeout); return m_bLocked; } //---------------------------------------------------------------------------- bool AMFLock::Unlock() { if(m_pBase == NULL) { return false; } const bool unlockSucceeded = m_pBase->Unlock(); m_bLocked = m_bLocked && (unlockSucceeded == false); return unlockSucceeded; } //---------------------------------------------------------------------------- bool AMFLock::IsLocked() { return m_bLocked; } //---------------------------------------------------------------------------- #if defined(METRO_APP) using namespace Platform; using namespace Windows::Foundation; using namespace Windows::UI::Xaml; using namespace Windows::UI::Xaml::Controls; using namespace Windows::UI::Xaml::Navigation; class AMFThreadObj { Windows::Foundation::IAsyncAction^ m_AsyncAction; AMFEvent m_StopEvent; AMFThread* m_pOwner; public: AMFThreadObj(AMFThread* owner); virtual ~AMFThreadObj(); virtual bool Start(); virtual bool RequestStop(); virtual bool WaitForStop(); virtual bool StopRequested(); // this is executed in the thread and overloaded by implementor virtual void Run() { m_pOwner->Run(); } virtual bool Init(){ return m_pOwner->Init(); } virtual bool Terminate(){ return m_pOwner->Terminate();} }; AMFThreadObj::AMFThreadObj(AMFThread* owner) : m_StopEvent(true, true), m_pOwner(owner) {} AMFThreadObj::~AMFThreadObj() {} bool AMFThreadObj::Start() { auto workItemDelegate = [this](IAsyncAction ^ workItem) { if( !this->Init() ) { return; } this->Run(); this->Terminate(); this->m_AsyncAction = nullptr; if( this->StopRequested() ) { this->m_StopEvent.SetEvent(); } }; Windows::System::Threading::WorkItemPriority WorkPriority; WorkPriority = Windows::System::Threading::WorkItemPriority::Normal; auto workItemHandler = ref new Windows::System::Threading::WorkItemHandler(workItemDelegate); m_AsyncAction = Windows::System::Threading::ThreadPool::RunAsync(workItemHandler, WorkPriority); return true; } bool AMFThreadObj::RequestStop() { if( m_AsyncAction == nullptr ) { return true; } m_StopEvent.ResetEvent(); return true; } bool AMFThreadObj::WaitForStop() { if( m_AsyncAction == nullptr ) { return true; } return m_StopEvent.Lock(); } bool AMFThreadObj::StopRequested() { return !m_StopEvent.Lock(0); } bool AMFThreadObj::IsRunning() { return m_AsyncAction != nullptr; } void amf::ExitThread() {} //#endif//#if defined(METRO_APP) //#if defined(_WIN32) #elif defined(_WIN32) // _WIN32 and METRO_APP defines are not mutually exclusive class AMFThreadObj { AMFThread* m_pOwner; uintptr_t m_pThread; AMFEvent m_StopEvent; AMFCriticalSection m_Lock; public: // this is called by owner AMFThreadObj(AMFThread* owner); virtual ~AMFThreadObj(); virtual bool Start(); virtual bool RequestStop(); virtual bool WaitForStop(); virtual bool StopRequested(); virtual bool IsRunning(); protected: static void AMF_CDECL_CALL AMFThreadProc(void* pThis); // this is executed in the thread and overloaded by implementor virtual void Run() { m_pOwner->Run(); } virtual bool Init() { return m_pOwner->Init(); } virtual bool Terminate() { return m_pOwner->Terminate(); } }; //---------------------------------------------------------------------------- AMFThreadObj::AMFThreadObj(AMFThread* owner) : m_pOwner(owner), m_pThread(uintptr_t(-1)), m_StopEvent(true, true) {} //---------------------------------------------------------------------------- AMFThreadObj::~AMFThreadObj() { // RequestStop(); // WaitForStop(); } //---------------------------------------------------------------------------- void AMF_CDECL_CALL AMFThreadObj::AMFThreadProc(void* pThis) { AMFThreadObj* pT = (AMFThreadObj*)pThis; if(!pT->Init()) { return; } pT->Run(); pT->Terminate(); pT->m_pThread = uintptr_t(-1); if(pT->StopRequested()) { pT->m_StopEvent.SetEvent(); // signal to stop that we just finished } } //---------------------------------------------------------------------------- bool AMFThreadObj::Start() { if(m_pThread != (uintptr_t)-1L) { return true; } AMFLock lock(&m_Lock); m_pThread = _beginthread(AMFThreadProc, 0, (void* )this); return m_pThread != (uintptr_t)-1L; } //---------------------------------------------------------------------------- bool AMFThreadObj::RequestStop() { if(m_pThread == (uintptr_t)-1L) { return true; } m_StopEvent.ResetEvent(); return true; } //---------------------------------------------------------------------------- bool AMFThreadObj::WaitForStop() { AMFLock lock(&m_Lock); if(m_pThread == (uintptr_t)-1L) { return true; } bool stopped = m_StopEvent.Lock(); m_pThread = (uintptr_t)-1L; return stopped; } //---------------------------------------------------------------------------- bool AMFThreadObj::StopRequested() { return !m_StopEvent.Lock(0); } bool AMFThreadObj::IsRunning() { return m_pThread != (uintptr_t)-1L; } //---------------------------------------------------------------------------- void ExitThread() { _endthread(); } #endif //#if defined(_WIN32) #if defined(__linux) || defined(__APPLE__) class AMFThreadObj { public: AMFThreadObj(AMFThread* owner); virtual ~AMFThreadObj(); virtual bool Start(); virtual bool RequestStop(); virtual bool WaitForStop(); virtual bool StopRequested(); virtual bool IsRunning(); // this is executed in the thread and overloaded by implementor virtual void Run() { m_pOwner->Run(); } virtual bool Init(){ return m_pOwner->Init(); } virtual bool Terminate(){ return m_pOwner->Terminate();} private: AMFThread* m_pOwner; pthread_t m_hThread; bool m_bStopRequested; bool m_bRunning; bool m_bInternalRunning; //used to detect thread auto-exit case and make join in Start AMFCriticalSection m_Lock; AMFThreadObj(const AMFThreadObj&); AMFThreadObj& operator=(const AMFThreadObj&); static void* AMF_CDECL_CALL AMFThreadProc(void* pThis); }; AMFThreadObj::AMFThreadObj(AMFThread* owner) : m_pOwner(owner), m_bStopRequested(false), m_bRunning(false), m_bInternalRunning(false) { } AMFThreadObj::~AMFThreadObj() { RequestStop(); WaitForStop(); } void* AMF_CDECL_CALL AMFThreadObj::AMFThreadProc(void* pThis) { AMFThreadObj* pT = (AMFThreadObj*)pThis; if(!pT->Init()) { return 0; } pT->Run(); pT->Terminate(); pT->m_bStopRequested = false; pT->m_bInternalRunning = false; return 0; } bool AMFThreadObj::Start() { bool result = true; if(m_bRunning == true && m_bInternalRunning == false) { pthread_join(m_hThread, 0); m_bRunning = false; m_bStopRequested = false; } if (IsRunning() == false) { WaitForStop(); AMFLock lock(&m_Lock); if (pthread_create(&m_hThread, 0, AMFThreadProc, (void*)this) == 0) { m_bRunning = true; m_bInternalRunning = true; } else { result = false; } } return result; } bool AMFThreadObj::RequestStop() { AMFLock lock(&m_Lock); if (IsRunning() == false) { return true; } m_bStopRequested = true; return true; } bool AMFThreadObj::WaitForStop() { AMFLock lock(&m_Lock); if (IsRunning() == true) { pthread_join(m_hThread, 0); m_bRunning = false; } m_bStopRequested = false; return true; } bool AMFThreadObj::StopRequested() { return m_bStopRequested; } bool AMFThreadObj::IsRunning() { return m_bRunning && m_bInternalRunning; } void ExitThread() { pthread_exit(0); } #endif //#if defined(__linux) AMFThread::AMFThread() : m_thread() { m_thread = new AMFThreadObj(this); } AMFThread::~AMFThread() { delete m_thread; } bool AMFThread::Start() { return m_thread->Start(); } bool AMFThread::RequestStop() { return m_thread->RequestStop(); } bool AMFThread::WaitForStop() { return m_thread->WaitForStop(); } bool AMFThread::StopRequested() { return m_thread->StopRequested(); } bool AMFThread::IsRunning() const { return m_thread->IsRunning(); } } //namespace ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/Thread.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Thread_h #define AMF_Thread_h #pragma once #include #include #include #include "../include/core/Platform.h" #ifndef _WIN32 #include #endif extern "C" { // threads #define AMF_INFINITE (0xFFFFFFFF) // Infinite ulTimeout // threads: atomic amf_long AMF_CDECL_CALL amf_atomic_inc(amf_long* X); amf_long AMF_CDECL_CALL amf_atomic_dec(amf_long* X); // threads: critical section amf_handle AMF_CDECL_CALL amf_create_critical_section(); bool AMF_CDECL_CALL amf_delete_critical_section(amf_handle cs); bool AMF_CDECL_CALL amf_enter_critical_section(amf_handle cs); bool AMF_CDECL_CALL amf_wait_critical_section(amf_handle cs, amf_ulong ulTimeout); bool AMF_CDECL_CALL amf_leave_critical_section(amf_handle cs); // threads: event amf_handle AMF_CDECL_CALL amf_create_event(bool bInitiallyOwned, bool bManualReset, const wchar_t* pName); bool AMF_CDECL_CALL amf_delete_event(amf_handle hevent); bool AMF_CDECL_CALL amf_set_event(amf_handle hevent); bool AMF_CDECL_CALL amf_reset_event(amf_handle hevent); bool AMF_CDECL_CALL amf_wait_for_event(amf_handle hevent, amf_ulong ulTimeout); bool AMF_CDECL_CALL amf_wait_for_event_timeout(amf_handle hevent, amf_ulong ulTimeout); // threads: mutex amf_handle AMF_CDECL_CALL amf_create_mutex(bool bInitiallyOwned, const wchar_t* pName); #if defined(_WIN32) amf_handle AMF_CDECL_CALL amf_open_mutex(const wchar_t* pName); #endif bool AMF_CDECL_CALL amf_delete_mutex(amf_handle hmutex); bool AMF_CDECL_CALL amf_wait_for_mutex(amf_handle hmutex, amf_ulong ulTimeout); bool AMF_CDECL_CALL amf_release_mutex(amf_handle hmutex); // threads: semaphore amf_handle AMF_CDECL_CALL amf_create_semaphore(amf_long iInitCount, amf_long iMaxCount, const wchar_t* pName); bool AMF_CDECL_CALL amf_delete_semaphore(amf_handle hsemaphore); bool AMF_CDECL_CALL amf_wait_for_semaphore(amf_handle hsemaphore, amf_ulong ulTimeout); bool AMF_CDECL_CALL amf_release_semaphore(amf_handle hsemaphore, amf_long iCount, amf_long* iOldCount); // threads: delay void AMF_CDECL_CALL amf_sleep(amf_ulong delay); amf_pts AMF_CDECL_CALL amf_high_precision_clock(); // in 100 of nanosec void AMF_CDECL_CALL amf_increase_timer_precision(); void AMF_CDECL_CALL amf_restore_timer_precision(); amf_handle AMF_CDECL_CALL amf_load_library(const wchar_t* filename); amf_handle AMF_CDECL_CALL amf_load_library1(const wchar_t* filename, bool bGlobal); void* AMF_CDECL_CALL amf_get_proc_address(amf_handle module, const char* procName); int AMF_CDECL_CALL amf_free_library(amf_handle module); #if defined(__APPLE__) amf_uint64 AMF_STD_CALL get_current_thread_id(); #else amf_uint32 AMF_STD_CALL get_current_thread_id(); #endif #if !defined(METRO_APP) // virtual memory void* AMF_CDECL_CALL amf_virtual_alloc(amf_size size); void AMF_CDECL_CALL amf_virtual_free(void* ptr); #else #define amf_virtual_alloc amf_alloc #define amf_virtual_free amf_free #endif // cpu #if defined(_WIN32) || (__linux__) amf_int32 AMF_STD_CALL amf_get_cpu_cores(); #endif } namespace amf { //---------------------------------------------------------------- class AMF_NO_VTABLE AMFSyncBase { public: virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE) = 0; virtual bool Unlock() = 0; }; //---------------------------------------------------------------- class AMFEvent : public AMFSyncBase { private: amf_handle m_hSyncObject; AMFEvent(const AMFEvent&); AMFEvent& operator=(const AMFEvent&); public: AMFEvent(bool bInitiallyOwned = false, bool bManualReset = false, const wchar_t* pName = NULL); virtual ~AMFEvent(); virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE); virtual bool LockTimeout(amf_ulong ulTimeout = AMF_INFINITE); virtual bool Unlock(); bool SetEvent(); bool ResetEvent(); amf_handle GetNative() { return m_hSyncObject; } }; //---------------------------------------------------------------- class AMFMutex : public AMFSyncBase { private: amf_handle m_hSyncObject; AMFMutex(const AMFMutex&); AMFMutex& operator=(const AMFMutex&); public: AMFMutex(bool bInitiallyOwned = false, const wchar_t* pName = NULL #if defined(_WIN32) , bool bOpenExistent = false #endif ); virtual ~AMFMutex(); virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE); virtual bool Unlock(); bool IsValid(); }; //---------------------------------------------------------------- class AMFCriticalSection : public AMFSyncBase { private: amf_handle m_Sect; AMFCriticalSection(const AMFCriticalSection&); AMFCriticalSection& operator=(const AMFCriticalSection&); public: AMFCriticalSection(); virtual ~AMFCriticalSection(); virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE); virtual bool Unlock(); }; //---------------------------------------------------------------- class AMFSemaphore : public AMFSyncBase { private: amf_handle m_hSemaphore; AMFSemaphore(const AMFSemaphore&); AMFSemaphore& operator=(const AMFSemaphore&); public: AMFSemaphore(amf_long iInitCount, amf_long iMaxCount, const wchar_t* pName = NULL); virtual ~AMFSemaphore(); virtual bool Create(amf_long iInitCount, amf_long iMaxCount, const wchar_t* pName = NULL); virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE); virtual bool Unlock(); }; //---------------------------------------------------------------- class AMFLock { private: AMFSyncBase* m_pBase; bool m_bLocked; AMFLock(const AMFLock&); AMFLock& operator=(const AMFLock&); public: AMFLock(AMFSyncBase* pBase, amf_ulong ulTimeout = AMF_INFINITE); ~AMFLock(); bool Lock(amf_ulong ulTimeout = AMF_INFINITE); bool Unlock(); bool IsLocked(); }; //---------------------------------------------------------------- class AMFReadWriteSync { private: struct ReadWriteResources { // max threads reading concurrently const int m_maxReadThreads; AMFSemaphore m_readSemaphore; AMFCriticalSection m_writeCriticalSection; ReadWriteResources() : m_maxReadThreads(10), m_readSemaphore(m_maxReadThreads, m_maxReadThreads), m_writeCriticalSection() { } }; class ReadSync : public AMFSyncBase { private: ReadSync(const ReadSync&); ReadSync& operator=(const ReadSync&); ReadWriteResources& m_resources; public: ReadSync(ReadWriteResources& resources) : m_resources(resources) { } virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE) { return m_resources.m_readSemaphore.Lock(ulTimeout); } virtual bool Unlock() { return m_resources.m_readSemaphore.Unlock(); } }; class WriteSync : public AMFSyncBase { private: WriteSync(const WriteSync&); WriteSync& operator=(const WriteSync&); ReadWriteResources& m_resources; public: WriteSync(ReadWriteResources& resources) : m_resources(resources) { } /// waits passed timeout for other writers; wait readers for infinite virtual bool Lock(amf_ulong ulTimeout = AMF_INFINITE) { if(!m_resources.m_writeCriticalSection.Lock(ulTimeout)) { return false; } for(int i = 0; i < m_resources.m_maxReadThreads; i++) { m_resources.m_readSemaphore.Lock(); } return true; } virtual bool Unlock() { // there is windows function to release N times by one call - could be optimize later for(int i = 0; i < m_resources.m_maxReadThreads; i++) { m_resources.m_readSemaphore.Unlock(); } return m_resources.m_writeCriticalSection.Unlock(); } }; private: ReadWriteResources m_resources; ReadSync m_readSync; WriteSync m_writeSync; public: AMFReadWriteSync() : m_resources(), m_readSync(m_resources), m_writeSync(m_resources) { } AMFSyncBase* GetReadSync() { return &m_readSync; } AMFSyncBase* GetWriteSync() { return &m_writeSync; } }; //---------------------------------------------------------------- class AMFThreadObj; class AMFThread { friend class AMFThreadObj; public: AMFThread(); virtual ~AMFThread(); virtual bool Start(); virtual bool RequestStop(); virtual bool WaitForStop(); virtual bool StopRequested(); virtual bool IsRunning() const; protected: // this is executed in the thread and overloaded by implementor virtual void Run() = 0; virtual bool Init() { return true; } virtual bool Terminate() { return true; } private: AMFThreadObj* m_thread; AMFThread(const AMFThread&); AMFThread& operator=(const AMFThread&); }; void ExitThread(); //---------------------------------------------------------------- template class AMFQueue { protected: class ItemData { public: T data; amf_ulong ulID; amf_long ulPriority; ItemData() : data(), ulID(), ulPriority(){} }; typedef std::list< ItemData > QueueList; QueueList m_Queue; AMFCriticalSection m_cSect; AMFEvent m_SomethingInQueueEvent; AMFSemaphore m_QueueSizeSem; amf_int32 m_iQueueSize; bool InternalGet(amf_ulong& ulID, T& item) { AMFLock lock(&m_cSect); if(!m_Queue.empty()) // something to get { ItemData& itemdata = m_Queue.front(); ulID = itemdata.ulID; item = itemdata.data; m_Queue.pop_front(); m_QueueSizeSem.Unlock(); if(m_Queue.empty()) { m_SomethingInQueueEvent.ResetEvent(); } return true; } return false; } public: AMFQueue(amf_int32 iQueueSize = 0) : m_Queue(), m_cSect(), m_SomethingInQueueEvent(false, false), m_QueueSizeSem(iQueueSize, iQueueSize > 0 ? iQueueSize + 1 : 0), m_iQueueSize(iQueueSize) {} virtual ~AMFQueue(){} virtual bool SetQueueSize(amf_int32 iQueueSize) { bool success = m_QueueSizeSem.Create(iQueueSize, iQueueSize > 0 ? iQueueSize + 1 : 0); if(success) { m_iQueueSize = iQueueSize; } return success; } virtual amf_int32 GetQueueSize() { return m_iQueueSize; } virtual bool Add(amf_ulong ulID, const T& item, amf_long ulPriority = 0, amf_ulong ulTimeout = AMF_INFINITE) { if(m_QueueSizeSem.Lock(ulTimeout) == false) { return false; } { AMFLock lock(&m_cSect); ItemData itemdata; itemdata.ulID = ulID; itemdata.data = item; itemdata.ulPriority = ulPriority; typename QueueList::iterator iter = m_Queue.end(); for(; iter != m_Queue.begin(); ) { iter--; if(ulPriority <= (iter->ulPriority)) { iter++; break; } } m_Queue.insert(iter, itemdata); m_SomethingInQueueEvent.SetEvent(); // this will set all waiting threads - some of them get data, some of them not } return true; } virtual bool Get(amf_ulong& ulID, T& item, amf_ulong ulTimeout) { if(InternalGet(ulID, item)) // try right away { return true; } // wait for queue if(m_SomethingInQueueEvent.Lock(ulTimeout)) { return InternalGet(ulID, item); } return false; } virtual void Clear() { bool bValue = true; while(bValue) { amf_ulong ulID; T item; bValue = InternalGet(ulID, item); } } virtual amf_size GetSize() { AMFLock lock(&m_cSect); return m_Queue.size(); } }; //---------------------------------------------------------------- template class AMFQueueThread : public AMFThread { private: AMFQueueThread(const AMFQueueThread&); AMFQueueThread& operator=(const AMFQueueThread&); protected: AMFQueue* m_pInQueue; AMFQueue* m_pOutQueue; AMFMutex m_mutexInProcess; ///< This mutex shows other threads that the thread function allocates ///< some objects on stack and it is unsafe state. To manipulate objects owned by descendant classes ///< client must lock this mutex by calling BlockProcessing member function. When client finished its work ///< corresponding UnblockProcessing member function call must be done. bool m_blockProcessingRequested; AMFCriticalSection m_csBlockingRequest; public: AMFQueueThread(AMFQueue* pInQueue, AMFQueue* pOutQueue) : m_pInQueue(pInQueue), m_pOutQueue(pOutQueue), m_mutexInProcess(), m_blockProcessingRequested(false), m_csBlockingRequest() {} virtual bool Process(amf_ulong& ulID, inT& inData, outT& outData) = 0; virtual void BlockProcessing() { AMFLock lock(&m_csBlockingRequest); m_blockProcessingRequested = true; m_mutexInProcess.Lock(); } virtual void UnblockProcessing() { AMFLock lock(&m_csBlockingRequest); m_mutexInProcess.Unlock(); m_blockProcessingRequested = false; } virtual bool IsPaused() { return false; } virtual void OnHaveOutput() {} virtual void OnIdle() {} virtual void Run() { bool bStop = false; while(!bStop) { { AMFLock lock(&m_mutexInProcess); inT inData; amf_ulong ulID = 0; bool callProcess = true; if(m_pInQueue != NULL) { amf_ulong waitTimeout = 5; bool validInput = m_pInQueue->Get(ulID, inData, waitTimeout); // Pulse to check Stop from time to time if(StopRequested()) { bStop = true; } if(!validInput) { callProcess = false; } } if(!bStop && callProcess) { outT outData; bool validOutput = Process(ulID, inData, outData); if(StopRequested()) { bStop = true; } if(!bStop && (m_pOutQueue != NULL) && validOutput) { m_pOutQueue->Add(ulID, outData); OnHaveOutput(); } } else { OnIdle(); } } if(StopRequested()) { bStop = true; } #if defined(__linux) || defined(__APPLE__) ///< HACK ///< This amf_sleep(0) is required to emulate windows mutex behavior. ///< In Windows release mutext causes some other waiting thread is receiving ownership of mutex. ///< In Linux it is not true. ///< Without sleep AMFLock destructor releases mutex but immediately on next cycle AMFLock constructor tries to lock ///< the mutex and now system have two threads waiting for the mutex. ///< Using some random logic system decides who will be unlocked. ///< This thread may win during several seconds it looks like pipeline is hang. ///< amf_sleep call causes waiting thread is becoming unlocked. if(m_blockProcessingRequested) { amf_sleep(0); } #endif } } }; //---------------------------------------------------------------- template class AMFQueueThreadPipeline { private: AMFQueueThreadPipeline(const AMFQueueThreadPipeline&); AMFQueueThreadPipeline& operator=(const AMFQueueThreadPipeline&); public: AMFQueue* m_pInQueue; AMFQueue* m_pOutQueue; std::vector<_Thread*> m_ThreadPool; AMFQueueThreadPipeline(AMFQueue* pInQueue, AMFQueue* pOutQueue) : m_pInQueue(pInQueue), m_pOutQueue(pOutQueue), m_ThreadPool() {} virtual ~AMFQueueThreadPipeline() { Stop(); } void Start(int iNumberOfThreads, ThreadParam param) { if((long)m_ThreadPool.size() >= iNumberOfThreads) { Stop(); //temporary to remove stopped threads. need callback from thread to clean pool //return; } size_t initialSize = m_ThreadPool.size(); for(size_t i = initialSize; i < (size_t)iNumberOfThreads; i++) { _Thread* pThread = new _Thread(m_pInQueue, m_pOutQueue, param); m_ThreadPool.push_back(pThread); pThread->Start(); } } void RequestStop() { long num = (long)m_ThreadPool.size(); for(long i = 0; i < num; i++) { m_ThreadPool[i]->RequestStop(); } } void BlockProcessing() { long num = (long)m_ThreadPool.size(); for(long i = 0; i < num; i++) { m_ThreadPool[i]->BlockProcessing(); } } void UnblockProcessing() { long num = (long)m_ThreadPool.size(); for(long i = 0; i < num; i++) { m_ThreadPool[i]->UnblockProcessing(); } } void WaitForStop() { long num = (long)m_ThreadPool.size(); for(long i = 0; i < num; i++) { _Thread* pThread = m_ThreadPool[i]; pThread->WaitForStop(); delete pThread; } m_ThreadPool.clear(); } void Stop() { RequestStop(); WaitForStop(); } }; //---------------------------------------------------------------- class AMFPreciseWaiter { public: AMFPreciseWaiter() : m_WaitEvent(), m_bCancel(false) {} virtual ~AMFPreciseWaiter() {} amf_pts Wait(amf_pts waittime) { if (waittime < 0) { return 0; } m_bCancel = false; amf_pts start = amf_high_precision_clock(); amf_pts waited = 0; int count = 0; while(!m_bCancel) { count++; if(!m_WaitEvent.LockTimeout(1)) { break; } waited = amf_high_precision_clock() - start; if(waited >= waittime) { break; } } return waited; } amf_pts WaitEx(amf_pts waittime) { m_bCancel = false; amf_pts start = amf_high_precision_clock(); amf_pts waited = 0; int count = 0; while (!m_bCancel && waited < waittime) { if (waittime - waited < 2 * AMF_SECOND / 1000)// last 2 ms burn CPU for precision { for (int i = 0; i < 1000; i++) { count++; #ifdef _WIN32 YieldProcessor(); #endif } } else if (!m_WaitEvent.LockTimeout(1)) { break; } waited = amf_high_precision_clock() - start; } return waited; } void Cancel() { m_bCancel = true; } protected: AMFEvent m_WaitEvent; bool m_bCancel; }; //---------------------------------------------------------------- } // namespace amf #endif // AMF_Thread_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/TraceAdapter.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "../include/core/Factory.h" #include "Thread.h" #include "TraceAdapter.h" #pragma warning(disable: 4251) #pragma warning(disable: 4996) using namespace amf; #if defined(AMF_CORE_STATIC) || defined(AMF_RUNTIME) || defined(AMF_LITE) extern "C" { extern AMF_CORE_LINK AMF_RESULT AMF_CDECL_CALL AMFInit(amf_uint64 version, amf::AMFFactory **ppFactory); } #else #include "AMFFactory.h" #endif //------------------------------------------------------------------------------------------------ static AMFTrace *s_pTrace = NULL; //------------------------------------------------------------------------------------------------ static AMFTrace *GetTrace() { if (s_pTrace == NULL) { #if defined(AMF_CORE_STATIC) || defined(AMF_RUNTIME) || defined(AMF_LITE) AMFFactory *pFactory = NULL; AMFInit(AMF_FULL_VERSION, &pFactory); pFactory->GetTrace(&s_pTrace); #else s_pTrace = g_AMFFactory.GetTrace(); if (s_pTrace == nullptr) { g_AMFFactory.Init(); // last resort, should not happen s_pTrace = g_AMFFactory.GetTrace(); g_AMFFactory.Terminate(); } #endif } return s_pTrace; } //------------------------------------------------------------------------------------------------ static AMFDebug *s_pDebug = NULL; //------------------------------------------------------------------------------------------------ static AMFDebug *GetDebug() { if (s_pDebug == NULL) { #if defined(AMF_CORE_STATIC) || defined(AMF_RUNTIME) || defined(AMF_LITE) AMFFactory *pFactory = NULL; AMFInit(AMF_FULL_VERSION, &pFactory); pFactory->GetDebug(&s_pDebug); #else s_pDebug = g_AMFFactory.GetDebug(); if (s_pDebug == nullptr) { g_AMFFactory.Init(); // last resort, should not happen s_pDebug = g_AMFFactory.GetDebug(); g_AMFFactory.Terminate(); } #endif } return s_pDebug; } //------------------------------------------------------------------------------------------------ AMF_RESULT AMF_CDECL_CALL amf::AMFSetCustomDebugger(AMFDebug *pDebugger) { s_pDebug = pDebugger; return AMF_OK; } //------------------------------------------------------------------------------------------------ AMF_RESULT AMF_CDECL_CALL amf::AMFSetCustomTracer(AMFTrace *pTracer) { s_pTrace = pTracer; return AMF_OK; } //------------------------------------------------------------------------------------------------ AMF_RESULT AMF_CDECL_CALL amf::AMFTraceEnableAsync(bool enable) { return GetTrace()->TraceEnableAsync(enable); } //------------------------------------------------------------------------------------------------ AMF_RESULT AMF_CDECL_CALL amf::AMFTraceFlush() { return GetTrace()->TraceFlush(); } //------------------------------------------------------------------------------------------------ void AMF_CDECL_CALL amf::AMFTraceW(const wchar_t* src_path, amf_int32 line, amf_int32 level, const wchar_t* scope, amf_int32 countArgs, const wchar_t* format, ...) // if countArgs <= 0 -> no args, formatting could be optimized then { if(countArgs <= 0) { GetTrace()->Trace(src_path, line, level, scope, format, NULL); } else { va_list vl; va_start(vl, format); GetTrace()->Trace(src_path, line, level, scope, format, &vl); va_end(vl); } } //------------------------------------------------------------------------------------------------ AMF_RESULT AMF_CDECL_CALL amf::AMFTraceSetPath(const wchar_t* path) { return GetTrace()->SetPath(path); } //------------------------------------------------------------------------------------------------ AMF_RESULT AMF_CDECL_CALL amf::AMFTraceGetPath(wchar_t* path, amf_size* pSize) { return GetTrace()->GetPath(path, pSize); } //------------------------------------------------------------------------------------------------ bool AMF_CDECL_CALL amf::AMFTraceEnableWriter(const wchar_t* writerID, bool enable) { return GetTrace()->EnableWriter(writerID, enable); } //------------------------------------------------------------------------------------------------ bool AMF_CDECL_CALL amf::AMFTraceWriterEnabled(const wchar_t* writerID) { return GetTrace()->WriterEnabled(writerID); } //------------------------------------------------------------------------------------------------ amf_int32 AMF_CDECL_CALL amf::AMFTraceSetGlobalLevel(amf_int32 level) { return GetTrace()->SetGlobalLevel(level); } //------------------------------------------------------------------------------------------------ amf_int32 AMF_CDECL_CALL amf::AMFTraceGetGlobalLevel() { return GetTrace()->GetGlobalLevel(); } //------------------------------------------------------------------------------------------------ amf_int32 AMF_CDECL_CALL amf::AMFTraceSetWriterLevel(const wchar_t* writerID, amf_int32 level) { return GetTrace()->SetWriterLevel(writerID, level); } //------------------------------------------------------------------------------------------------ amf_int32 AMF_CDECL_CALL amf::AMFTraceGetWriterLevel(const wchar_t* writerID) { return GetTrace()->GetWriterLevel(writerID); } //------------------------------------------------------------------------------------------------ amf_int32 AMF_CDECL_CALL amf::AMFTraceSetWriterLevelForScope(const wchar_t* writerID, const wchar_t* scope, amf_int32 level) { return GetTrace()->SetWriterLevelForScope(writerID, scope, level); } //------------------------------------------------------------------------------------------------ amf_int32 AMF_CDECL_CALL amf::AMFTraceGetWriterLevelForScope(const wchar_t* writerID, const wchar_t* scope) { return GetTrace()->GetWriterLevelForScope(writerID, scope); } //------------------------------------------------------------------------------------------------ void AMF_CDECL_CALL amf::AMFTraceRegisterWriter(const wchar_t* writerID, AMFTraceWriter* pWriter) { GetTrace()->RegisterWriter(writerID, pWriter, true); } void AMF_CDECL_CALL amf::AMFTraceUnregisterWriter(const wchar_t* writerID) { GetTrace()->UnregisterWriter(writerID); } #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wexit-time-destructors" #pragma clang diagnostic ignored "-Wglobal-constructors" #endif void AMF_CDECL_CALL amf::AMFTraceEnterScope() { GetTrace()->Indent(1); } amf_uint32 AMF_CDECL_CALL AMFTraceGetScopeDepth() { return GetTrace()->GetIndentation(); } void AMF_CDECL_CALL amf::AMFTraceExitScope() { GetTrace()->Indent(-1); } void AMF_CDECL_CALL amf::AMFAssertsEnable(bool enable) { GetDebug()->AssertsEnable(enable); } bool AMF_CDECL_CALL amf::AMFAssertsEnabled() { return GetDebug()->AssertsEnabled(); } amf_wstring AMF_CDECL_CALL amf::AMFFormatResult(AMF_RESULT result) { return amf::amf_string_format(L"AMF_ERROR %d : %s: ", result, GetTrace()->GetResultText(result)); } const wchar_t* AMF_STD_CALL amf::AMFGetResultText(AMF_RESULT res) { return GetTrace()->GetResultText(res); } const wchar_t* AMF_STD_CALL amf::AMFSurfaceGetFormatName(const AMF_SURFACE_FORMAT eSurfaceFormat) { return GetTrace()->SurfaceGetFormatName(eSurfaceFormat); } AMF_SURFACE_FORMAT AMF_STD_CALL amf::AMFSurfaceGetFormatByName(const wchar_t* pwName) { return GetTrace()->SurfaceGetFormatByName(pwName); } const wchar_t* AMF_STD_CALL amf::AMFGetMemoryTypeName(const AMF_MEMORY_TYPE memoryType) { return GetTrace()->GetMemoryTypeName(memoryType); } AMF_MEMORY_TYPE AMF_STD_CALL amf::AMFGetMemoryTypeByName(const wchar_t* name) { return GetTrace()->GetMemoryTypeByName(name); } ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/TraceAdapter.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // ///------------------------------------------------------------------------- /// @file TraceAdapter.h /// @brief AMFTrace interface ///------------------------------------------------------------------------- #ifndef AMF_TraceAdapter_h #define AMF_TraceAdapter_h #pragma once #include "../include/core/Debug.h" #include "../include/core/Trace.h" #include "../include/core/Result.h" #include "../common/AMFFactory.h" #include "AMFSTL.h" #ifndef WIN32 #include #endif #include //----------------------------------- // Visual Studio memory leak report #if defined(WIN32) && defined(_DEBUG) && defined(CRTDBG) #include #if !defined(METRO_APP) #ifdef _DEBUG #define DEBUG_NEW new(_NORMAL_BLOCK, __FILE__, __LINE__) #define new DEBUG_NEW #endif #endif #endif //----------------------------------- #if defined(_DEBUG) && defined(__linux) #include #include #endif namespace amf { /** ******************************************************************************* * AMFTraceEnableAsync * * @brief * Enable or disable async mode * * There are 2 modes trace can work in: * Synchronous - every Trace call immediately goes to writers: console, windows, file, ... * Asynchronous - trace message go to thread local queues; separate thread passes them to writes * Asynchronous mode offers no synchronization between working threads which are writing traces * and high performance. * Asynchronous mode is not enabled always as that dedicated thread (started in Media SDK module) cannot be * terminated safely. See msdn ExitProcess description: it terminates all threads without notifications. * ExitProcess is called after exit from main() -> before module static variables destroyed and before atexit * notifiers are called -> no way to finish trace dedicated thread. * * Therefore here is direct enable of asynchronous mode. * AMFTraceEnableAsync(true) increases internal asynchronous counter by 1; AMFTraceEnableAsync(false) decreases by 1 * when counter becomes > 0 mode - switches to async; when becomes 0 - switches to sync * * Tracer must be switched to sync mode before quit application, otherwise async writing thread will be force terminated by OS (at lease Windows) * See MSDN ExitProcess article for details. ******************************************************************************* */ extern "C" { AMF_RESULT AMF_CDECL_CALL AMFTraceEnableAsync(bool enable); /** ******************************************************************************* * AMFDebugSetDebugger * * @brief * it is used to set a local debugger, or set NULL to remove * ******************************************************************************* */ AMF_RESULT AMF_CDECL_CALL AMFSetCustomDebugger(AMFDebug *pDebugger); /** ******************************************************************************* * AMFTraceSetTracer * * @brief * it is used to set a local tracer, or set NULL to remove * ******************************************************************************* */ AMF_RESULT AMF_CDECL_CALL AMFSetCustomTracer(AMFTrace *pTrace); /** ******************************************************************************* * AMFTraceFlush * * @brief * Enforce trace writers flush * ******************************************************************************* */ AMF_RESULT AMF_CDECL_CALL AMFTraceFlush(); /** ******************************************************************************* * EXPAND * * @brief * Auxilary Macro used to evaluate __VA_ARGS__ from 1 macro argument into list of them * * It is needed for COUNT_ARGS macro * ******************************************************************************* */ #define EXPAND(x) x /** ******************************************************************************* * GET_TENTH_ARG * * @brief * Auxilary Macro for COUNT_ARGS macro * ******************************************************************************* */ #define GET_TENTH_ARG(a, b, c, d, e, f, g, h, i, j, name, ...) name /** ******************************************************************************* * COUNT_ARGS * * @brief * Macro returns number of arguments actually passed into it * * COUNT_ARGS macro works ok for 1..10 arguments * It is needed to distinguish macro call with optional parameters and without them ******************************************************************************* */ #define COUNT_ARGS(...) EXPAND(GET_TENTH_ARG(__VA_ARGS__, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1)) /** ******************************************************************************* * AMFTraceW * * @brief * General trace function with all possible parameters ******************************************************************************* */ void AMF_CDECL_CALL AMFTraceW(const wchar_t* src_path, amf_int32 line, amf_int32 level, const wchar_t* scope, amf_int32 countArgs, const wchar_t* format, ...); /** ******************************************************************************* * AMF_UNICODE * * @brief * Macro to convert string constant into wide char string constant * * Auxilary AMF_UNICODE_ macro is needed as otherwise it is not possible to use AMF_UNICODE(__FILE__) * Microsoft macro _T also uses 2 passes to accomplish that ******************************************************************************* */ #define AMF_UNICODE(s) AMF_UNICODE_(s) #define AMF_UNICODE_(s) L ## s /** ******************************************************************************* * AMFTrace * * @brief * Most general macro for trace, incapsulates passing source file and line ******************************************************************************* */ #define AMFTrace(level, scope, /*format, */...) amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, level, scope, COUNT_ARGS(__VA_ARGS__) - 1, __VA_ARGS__) /** ******************************************************************************* * AMFTraceError * * @brief * Shortened macro to trace exactly error. * * Similar macroses are: AMFTraceWarning, AMFTraceInfo, AMFTraceDebug ******************************************************************************* */ #define AMFTraceError(scope, /*format, */...) amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, AMF_TRACE_ERROR, scope, COUNT_ARGS(__VA_ARGS__) - 1, __VA_ARGS__) #define AMFTraceWarning(scope, /*format, */...) amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, AMF_TRACE_WARNING, scope, COUNT_ARGS(__VA_ARGS__) - 1, __VA_ARGS__) #define AMFTraceInfo(scope, /*format, */...) amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, AMF_TRACE_INFO, scope, COUNT_ARGS(__VA_ARGS__) - 1, __VA_ARGS__) #define AMFTraceDebug(scope, /*format, */...) amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, AMF_TRACE_DEBUG, scope, COUNT_ARGS(__VA_ARGS__) - 1, __VA_ARGS__) /** ******************************************************************************* * AMFDebugHitEvent * * @brief * Designed to determine how many are specific events take place ******************************************************************************* */ void AMF_CDECL_CALL AMFDebugHitEvent(const wchar_t* scope, const wchar_t* eventName); /** ******************************************************************************* * AMFDebugGetEventsCount * * @brief * Designed to acquire counter of events reported by call AMFDebugHitEvent ******************************************************************************* */ amf_int64 AMF_CDECL_CALL AMFDebugGetEventsCount(const wchar_t* scope, const wchar_t* eventName); /** ******************************************************************************* * AMFAssertsEnabled * * @brief * Returns bool values indicating if asserts were enabled or not ******************************************************************************* */ bool AMF_CDECL_CALL AMFAssertsEnabled(); /** ******************************************************************************* * AMFTraceEnterScope * * @brief * Increase trace indentation value by 1 * * Indentation value is thread specific ******************************************************************************* */ void AMF_CDECL_CALL AMFTraceEnterScope(); /** ******************************************************************************* * AMFTraceExitScope * * @brief * Decrease trace indentation value by 1 * * Indentation value is thread specific ******************************************************************************* */ void AMF_CDECL_CALL AMFTraceExitScope(); /** ******************************************************************************* * AMF_FACILITY * * @brief * Default value for AMF_FACILITY, this NULL leads to generate facility from source file name * * This AMF_FACILITY could be overloaded locally with #define AMF_FACILITY L"LocalScope" ******************************************************************************* */ static const wchar_t* AMF_FACILITY = NULL; } //extern "C" /** ******************************************************************************* * AMFDebugBreak * * @brief * Macro for switching to debug of application ******************************************************************************* */ #if defined(_DEBUG) #if defined(_WIN32) #define AMFDebugBreak {if(amf::AMFAssertsEnabled()) {__debugbreak();} \ } //{ } #elif defined(__linux) // #define AMFDebugBreak ((void)0) #define AMFDebugBreak {if(amf::AMFAssertsEnabled() && ptrace(PTRACE_TRACEME, 0, 1, 0) < 0) {raise(SIGTRAP);} \ }//{ } #elif defined(__APPLE__) #define AMFDebugBreak {if(amf::AMFAssertsEnabled()) {assert(0);} \ } #endif #else #define AMFDebugBreak #endif /** ******************************************************************************* * __FormatMessage * * @brief * Auxilary function to select from 2 messages and preformat message if any arguments are specified ******************************************************************************* */ inline amf_wstring __FormatMessage(int /*argsCount*/, const wchar_t* expression) { return amf_wstring(expression); // the only expression is provided - return this one } inline amf_wstring __FormatMessage(int argsCount, const wchar_t* /*expression*/, const wchar_t* message, ...) { // this version of __FormatMessage for case when descriptive message is provided with optional args if(argsCount <= 0) { return amf_wstring(message); } else { va_list arglist; va_start(arglist, message); amf_wstring result = amf::amf_string_formatVA(message, arglist); va_end(arglist); return result; } } /** ******************************************************************************* * AMF_FIRST_VALUE * * @brief * Auxilary macro: extracts first argument from the list ******************************************************************************* */ #define AMF_FIRST_VALUE(x, ...) x /** ******************************************************************************* * AMF_BASE_RETURN * * @brief * Base generic macro: checks expression for success, if failed: trace error, debug break and return an error * * return_result is a parameter to return to upper level, could be hard-coded or * specified exp_res what means pass inner level error ******************************************************************************* */ #define AMF_BASE_RETURN(exp, exp_type, check_func, format_prefix, level, scope, return_result/*(could be exp_res)*/, /* optional message args*/ ...) \ { \ exp_type exp_res = (exp_type)(exp); \ if(!check_func(exp_res)) \ { \ amf_wstring message = format_prefix(exp_res) + amf::__FormatMessage(COUNT_ARGS(__VA_ARGS__) - 2, __VA_ARGS__); \ EXPAND(amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, level, scope, 0, message.c_str()) ); \ AMFDebugBreak; \ return return_result; \ } \ } /** ******************************************************************************* * AMF_BASE_ASSERT * * @brief * Base generic macro: checks expression for success, if failed: trace error, debug break ******************************************************************************* */ #define AMF_BASE_ASSERT(exp, exp_type, check_func, format_prefix, level, scope, return_result/*(could be exp_res)*/, /*optional message, optional message args*/ ...) \ { \ exp_type exp_res = (exp_type)(exp); \ if(!check_func(exp_res)) \ { \ amf_wstring message = format_prefix(exp_res) + amf::__FormatMessage(COUNT_ARGS(__VA_ARGS__) - 2, __VA_ARGS__); \ EXPAND(amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, level, scope, 0, message.c_str()) ); \ AMFDebugBreak; \ } \ } /** ******************************************************************************* * AMF_BASE_CALL * * @brief * Macro supporting cascade call function returning AMF_RESULT from another * * return_result is a parameter to return to upper level, could be hard-coded or * specified exp_res what means pass inner level error ******************************************************************************* */ #define AMF_BASE_CALL(exp, exp_type, check_func, format_prefix, level, scope, return_result/*(could be exp_res)*/, /*optional message, optional message args*/ ...) \ { \ amf_wstring function_name = amf::__FormatMessage(COUNT_ARGS(__VA_ARGS__) - 2, __VA_ARGS__); \ amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, AMF_TRACE_DEBUG, scope, 0, function_name.c_str()); \ amf::AMFTraceEnterScope(); \ exp_type exp_res = (exp_type)(exp); \ amf::AMFTraceExitScope(); \ if(!check_func(exp_res)) \ { \ amf_wstring message = format_prefix(exp_res) + function_name; \ EXPAND(amf::AMFTraceW(AMF_UNICODE(__FILE__), __LINE__, level, scope, 0, message.c_str()) ); \ AMFDebugBreak; \ return return_result; \ } \ } /** ******************************************************************************* * AMFCheckExpression * * @brief * Checks if result succeeds ******************************************************************************* */ inline bool AMFCheckExpression(int result) { return result != 0; } /** ******************************************************************************* * AMFFormatAssert * * @brief * Returns default assertion message ******************************************************************************* */ inline amf_wstring AMFFormatAssert(int result) { return result ? amf_wstring() : amf_wstring(L"Assertion failed:"); } /** ******************************************************************************* * AMFOpenCLSucceeded * * @brief * Checks cl_status for success ******************************************************************************* */ inline bool AMFOpenCLSucceeded(int result) { return result == 0; } /** ******************************************************************************* * AMFFormatOpenCLError * * @brief * Formats open CL error ******************************************************************************* */ inline amf_wstring AMFFormatOpenCLError(int result) { return amf::amf_string_format(L"OpenCL failed, error = %d:", result); } /** ******************************************************************************* * AMFResultIsOK * * @brief * Checks if AMF_RESULT is OK ******************************************************************************* */ inline bool AMFResultIsOK(AMF_RESULT result) { return result == AMF_OK; } /** ******************************************************************************* * AMFSucceeded * * @brief * Checks if AMF_RESULT is succeeded ******************************************************************************* */ inline bool AMFSucceeded(AMF_RESULT result) { return result == AMF_OK || result == AMF_REPEAT; } /** ******************************************************************************* * AMFFormatResult * * @brief * Formats AMF_RESULT into descriptive string ******************************************************************************* */ amf_wstring AMF_CDECL_CALL AMFFormatResult(AMF_RESULT result); /** ******************************************************************************* * AMFHResultSucceded * * @brief * Checks if HRESULT succeeded ******************************************************************************* */ inline bool AMFHResultSucceded(HRESULT result) { return SUCCEEDED(result); } /** ******************************************************************************* * AMFFormatHResult * * @brief * Formats HRESULT into descriptive string ******************************************************************************* */ inline amf_wstring AMFFormatHResult(HRESULT result) { return amf::amf_string_format(L"COM failed, HR = %0X:", result); } /** ******************************************************************************* * AMFVkResultSucceeded * * @brief * Checks if VkResult succeeded ******************************************************************************* */ inline bool AMFVkResultSucceeded(int result) { return result == 0; } /** ******************************************************************************* * AMFFormatVkResult * * @brief * Formats VkResult into descriptive string ******************************************************************************* */ inline amf_wstring AMFFormatVkResult(int result) { return amf::amf_string_format(L"Vulkan failed, VkResult = %d:", result); } /** ******************************************************************************* * AMF_CALL * * @brief * Macro to call AMF_RESULT returning function from AMF_RESULT returning function * * It does: * 1) Trace (level == debug) function name (or message if specified) * 2) Indent trace * 3) Call function * 4) Unindent trace * 5) Checks its result * 6) If not OK trace error, switch to debugger (if asserts enabled) and return that error code to upper level * * Use cases: * A) AMF_CALL(Init("Name")); // trace expression itself * B) AMF_CALL(Init("Name"), L"Initialize resources"); // trace desciptive message * C) AMF_CALL(Init(name), L"Initialize resources with %s", name); // trace descriptive message with aditional arguments from runtime ******************************************************************************* */ #define AMF_CALL(exp, ... /*optional format, args*/) AMF_BASE_CALL(exp, AMF_RESULT, amf::AMFResultIsOK, amf::AMFFormatResult, AMF_TRACE_ERROR, AMF_FACILITY, exp_res, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * AMF_ASSERT_OK * * @brief * Checks expression == AMF_OK, otherwise trace error and debug break * * Could be used: A) with just expression B) with optinal descriptive message C) message + args for printf ******************************************************************************* */ #define AMF_ASSERT_OK(exp, ... /*optional format, args*/) AMF_BASE_ASSERT(exp, AMF_RESULT, amf::AMFResultIsOK, amf::AMFFormatResult, AMF_TRACE_ERROR, AMF_FACILITY, AMF_FAIL, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * AMF_ASSERT * * @brief * Checks expression != 0, otherwise trace error and debug break * * Could be used: A) with just expression B) with optinal descriptive message C) message + args for printf ******************************************************************************* */ #define AMF_ASSERT(exp, ...) AMF_BASE_ASSERT(exp, int, amf::AMFCheckExpression, amf::AMFFormatAssert, AMF_TRACE_ERROR, AMF_FACILITY, AMF_FAIL, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * AMF_RETURN_IF_FAILED * * @brief * Checks expression != 0, otherwise trace error, debug break and return that error to upper level * * Could be used: A) with just expression B) with optinal descriptive message C) message + args for printf ******************************************************************************* */ #define AMF_RETURN_IF_FAILED(exp, ...) AMF_BASE_RETURN(exp, AMF_RESULT, amf::AMFResultIsOK, amf::AMFFormatResult, AMF_TRACE_ERROR, AMF_FACILITY, exp_res, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * ASSERT_RETURN_IF_CL_FAILED * * @brief * Checks cl error is ok, otherwise trace error, debug break and return that error to upper level * * Could be used: A) with just expression B) with optinal descriptive message C) message + args for printf ******************************************************************************* */ #define ASSERT_RETURN_IF_CL_FAILED(exp, /*optional format, args,*/...) AMF_BASE_RETURN(exp, int, amf::AMFOpenCLSucceeded, amf::AMFFormatOpenCLError, AMF_TRACE_ERROR, AMF_FACILITY, AMF_OPENCL_FAILED, L###exp, ##__VA_ARGS__) #define AMF_RETURN_IF_CL_FAILED(exp, /*optional format, args,*/...) AMF_BASE_RETURN(exp, int, amf::AMFOpenCLSucceeded, amf::AMFFormatOpenCLError, AMF_TRACE_ERROR, AMF_FACILITY, AMF_OPENCL_FAILED, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * ASSERT_RETURN_IF_HR_FAILED * * @brief * Obsolete macro: Checks HRESULT if succeeded, otherwise trace error, debug break and return specified error to upper level * * Other macroses below are also obsolete ******************************************************************************* */ #define ASSERT_RETURN_IF_HR_FAILED(exp, reterr, /*optional format, args,*/...) AMF_BASE_RETURN(exp, HRESULT, amf::AMFHResultSucceded, amf::AMFFormatHResult, AMF_TRACE_ERROR, AMF_FACILITY, reterr, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * ASSERT_RETURN_IF_VK_FAILED * * @brief * Checks VkResult if succeeded, otherwise trace error, debug break and return specified error to upper level * * Could be used: A) with just expression B) with optinal descriptive message C) message + args for printf ******************************************************************************* */ #define ASSERT_RETURN_IF_VK_FAILED(exp, reterr, /*optional format, args,*/...) AMF_BASE_RETURN(exp, int, amf::AMFVkResultSucceeded, amf::AMFFormatVkResult, AMF_TRACE_ERROR, AMF_FACILITY, reterr, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * AMF_RETURN_IF_FALSE * * @brief * Checks expression != 0, otherwise trace error, debug break and return that error to upper level * * Could be used: A) with just expression B) with optinal descriptive message C) message + args for printf ******************************************************************************* */ #define AMF_RETURN_IF_FALSE(exp, ret_value, /*optional message,*/ ...) AMF_BASE_RETURN(exp, int, amf::AMFCheckExpression, amf::AMFFormatAssert, AMF_TRACE_ERROR, AMF_FACILITY, ret_value, L###exp, ##__VA_ARGS__) /** ******************************************************************************* * AMF_RETURN_IF_INVALID_POINTER * * @brief * Checks ptr != NULL, otherwise trace error, debug break and return that error to upper level * ******************************************************************************* */ #define AMF_RETURN_IF_INVALID_POINTER(ptr, /*optional message,*/ ...) AMF_BASE_RETURN(ptr != NULL, int, amf::AMFCheckExpression, amf::AMFFormatAssert, AMF_TRACE_ERROR, AMF_FACILITY, AMF_INVALID_POINTER, L"invalid pointer : " L###ptr, ##__VA_ARGS__) /** ******************************************************************************* * AMFTestEventObserver * * @brief * Interface to subscribe on test events ******************************************************************************* */ extern "C" { /** ******************************************************************************* * AMFTraceSetPath * * @brief * Set Trace path * * Returns AMF_OK if succeeded ******************************************************************************* */ AMF_RESULT AMF_CDECL_CALL AMFTraceSetPath(const wchar_t* path); /** ******************************************************************************* * AMFTraceGetPath * * @brief * Get Trace path * * Returns AMF_OK if succeeded ******************************************************************************* */ AMF_RESULT AMF_CDECL_CALL AMFTraceGetPath( wchar_t* path, ///< [out] buffer able to hold *pSize symbols; path is copied there, at least part fitting the buffer, always terminator is copied amf_size* pSize ///< [in, out] size of buffer, returned needed size of buffer including zero terminator ); /** ******************************************************************************* * AMFTraceEnableWriter * * @brief * Disable trace to registered writer * * Returns previous state ******************************************************************************* */ bool AMF_CDECL_CALL AMFTraceEnableWriter(const wchar_t* writerID, bool enable); /** ******************************************************************************* * AMFTraceWriterEnabled * * @brief * Return flag if writer enabled ******************************************************************************* */ bool AMF_CDECL_CALL AMFTraceWriterEnabled(const wchar_t* writerID); /** ******************************************************************************* * AMFTraceSetGlobalLevel * * @brief * Sets trace level for writer and scope * * Returns previous setting ******************************************************************************* */ amf_int32 AMF_CDECL_CALL AMFTraceSetGlobalLevel(amf_int32 level); /** ******************************************************************************* * AMFTraceGetGlobalLevel * * @brief * Returns global level ******************************************************************************* */ amf_int32 AMF_CDECL_CALL AMFTraceGetGlobalLevel(); /** ******************************************************************************* * AMFTraceSetWriterLevel * * @brief * Sets trace level for writer * * Returns previous setting ******************************************************************************* */ amf_int32 AMF_CDECL_CALL AMFTraceSetWriterLevel(const wchar_t* writerID, amf_int32 level); /** ******************************************************************************* * AMFTraceGetWriterLevel * * @brief * Gets trace level for writer ******************************************************************************* */ amf_int32 AMF_CDECL_CALL AMFTraceGetWriterLevel(const wchar_t* writerID); /** ******************************************************************************* * AMFTraceSetWriterLevelForScope * * @brief * Sets trace level for writer and scope * * Returns previous setting ******************************************************************************* */ amf_int32 AMF_CDECL_CALL AMFTraceSetWriterLevelForScope(const wchar_t* writerID, const wchar_t* scope, amf_int32 level); /** ******************************************************************************* * AMFTraceGetWriterLevelForScope * * @brief * Gets trace level for writer and scope ******************************************************************************* */ amf_int32 AMF_CDECL_CALL AMFTraceGetWriterLevelForScope(const wchar_t* writerID, const wchar_t* scope); /** ******************************************************************************* * AMFTraceRegisterWriter * * @brief * Register custom trace writer * ******************************************************************************* */ void AMF_CDECL_CALL AMFTraceRegisterWriter(const wchar_t* writerID, AMFTraceWriter* pWriter); /** ******************************************************************************* * AMFTraceUnregisterWriter * * @brief * Register custom trace writer * ******************************************************************************* */ void AMF_CDECL_CALL AMFTraceUnregisterWriter(const wchar_t* writerID); /* ******************************************************************************* * AMFAssertsEnable * * @brief * Enable asserts in checks * ******************************************************************************* */ void AMF_CDECL_CALL AMFAssertsEnable(bool enable); /** ******************************************************************************* * AMFAssertsEnabled * * @brief * Returns true if asserts in checks enabled * ******************************************************************************* */ bool AMF_CDECL_CALL AMFAssertsEnabled(); const wchar_t* AMF_STD_CALL AMFGetResultText(AMF_RESULT res); const wchar_t* AMF_STD_CALL AMFSurfaceGetFormatName(const AMF_SURFACE_FORMAT eSurfaceFormat); AMF_SURFACE_FORMAT AMF_STD_CALL AMFSurfaceGetFormatByName(const wchar_t* pwName); const wchar_t* AMF_STD_CALL AMFGetMemoryTypeName(const AMF_MEMORY_TYPE memoryType); AMF_MEMORY_TYPE AMF_STD_CALL AMFGetMemoryTypeByName(const wchar_t* name); } //extern "C" } // namespace amf #endif // AMF_TraceAdapter_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/common/Windows/ThreadWindows.cpp ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #include "../Thread.h" #ifdef _WIN32 #include #include #include //---------------------------------------------------------------------------------------- // threading //---------------------------------------------------------------------------------------- amf_long AMF_CDECL_CALL amf_atomic_inc(amf_long* X) { return InterlockedIncrement((long*)X); } //---------------------------------------------------------------------------------------- amf_long AMF_CDECL_CALL amf_atomic_dec(amf_long* X) { return InterlockedDecrement((long*)X); } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_create_critical_section() { CRITICAL_SECTION* cs = new CRITICAL_SECTION; #if defined(METRO_APP) ::InitializeCriticalSectionEx(cs, 0, CRITICAL_SECTION_NO_DEBUG_INFO); #else ::InitializeCriticalSection(cs); #endif return (amf_handle)cs; // in Win32 - no errors } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_delete_critical_section(amf_handle cs) { if(cs == NULL) { return false; } ::DeleteCriticalSection((CRITICAL_SECTION*)cs); delete (CRITICAL_SECTION*)cs; return true; // in Win32 - no errors } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_enter_critical_section(amf_handle cs) { if(cs == NULL) { return false; } ::EnterCriticalSection((CRITICAL_SECTION*)cs); return true; // in Win32 - no errors } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_wait_critical_section(amf_handle cs, amf_ulong ulTimeout) { if(cs == NULL) { return false; } while (true) { const BOOL success = ::TryEnterCriticalSection((CRITICAL_SECTION*)cs); if (success == TRUE) { return true; // in Win32 - no errors } if (ulTimeout == 0) { return false; } amf_sleep(1); ulTimeout--; } return false; } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_leave_critical_section(amf_handle cs) { if(cs == NULL) { return false; } ::LeaveCriticalSection((CRITICAL_SECTION*)cs); return true; // in Win32 - no errors } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_create_event(bool bInitiallyOwned, bool bManualReset, const wchar_t* pName) { #if defined(METRO_APP) DWORD flags = ((bManualReset) ? CREATE_EVENT_MANUAL_RESET : 0) | ((bInitiallyOwned) ? CREATE_EVENT_INITIAL_SET : 0); return ::CreateEventEx(NULL, pName, flags, STANDARD_RIGHTS_ALL | EVENT_MODIFY_STATE); #else return ::CreateEventW(NULL, bManualReset == true, bInitiallyOwned == true, pName); #endif } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_delete_event(amf_handle hevent) { if(hevent == NULL) { return false; } return ::CloseHandle(hevent) != FALSE; } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_set_event(amf_handle hevent) { if(hevent == NULL) { return false; } return ::SetEvent(hevent) != FALSE; } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_reset_event(amf_handle hevent) { if(hevent == NULL) { return false; } return ::ResetEvent(hevent) != FALSE; } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_wait_for_event(amf_handle hevent, amf_ulong ulTimeout) { if(hevent == NULL) { return false; } #if defined(METRO_APP) return ::WaitForSingleObjectEx(hevent, ulTimeout, FALSE) == WAIT_OBJECT_0; #else return ::WaitForSingleObject(hevent, ulTimeout) == WAIT_OBJECT_0; #endif } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_wait_for_event_timeout(amf_handle hevent, amf_ulong ulTimeout) { if(hevent == NULL) { return false; } DWORD ret; #if defined(METRO_APP) ret = ::WaitForSingleObjectEx(hevent, ulTimeout, FALSE); #else ret = ::WaitForSingleObject(hevent, ulTimeout); #endif return ret == WAIT_OBJECT_0 || ret == WAIT_TIMEOUT; } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_create_mutex(bool bInitiallyOwned, const wchar_t* pName) { #if defined(METRO_APP) DWORD flags = (bInitiallyOwned) ? CREATE_MUTEX_INITIAL_OWNER : 0; return ::CreateMutexEx(NULL, pName, flags, STANDARD_RIGHTS_ALL); #else return ::CreateMutexW(NULL, bInitiallyOwned == true, pName); #endif } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_open_mutex(const wchar_t* pName) { return ::OpenMutexW(MUTEX_ALL_ACCESS, FALSE, pName); } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_delete_mutex(amf_handle hmutex) { if(hmutex == NULL) { return false; } return ::CloseHandle(hmutex) != FALSE; } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_wait_for_mutex(amf_handle hmutex, amf_ulong ulTimeout) { if(hmutex == NULL) { return false; } #if defined(METRO_APP) return ::WaitForSingleObjectEx(hmutex, ulTimeout, FALSE) == WAIT_OBJECT_0; #else return ::WaitForSingleObject(hmutex, ulTimeout) == WAIT_OBJECT_0; #endif } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_release_mutex(amf_handle hmutex) { if(hmutex == NULL) { return false; } return ::ReleaseMutex(hmutex) != FALSE; } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_create_semaphore(amf_long iInitCount, amf_long iMaxCount, const wchar_t* pName) { if(iMaxCount == NULL || iInitCount > iMaxCount) { return NULL; } #if defined(METRO_APP) return ::CreateSemaphoreEx(NULL, iInitCount, iMaxCount, pName, 0, STANDARD_RIGHTS_ALL | SEMAPHORE_MODIFY_STATE); #else return ::CreateSemaphoreW(NULL, iInitCount, iMaxCount, pName); #endif } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_delete_semaphore(amf_handle hsemaphore) { if(hsemaphore == NULL) { return false; } return ::CloseHandle(hsemaphore) != FALSE; } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_wait_for_semaphore(amf_handle hsemaphore, amf_ulong timeout) { if(hsemaphore == NULL) { return true; } #if defined(METRO_APP) return ::WaitForSingleObjectEx(hsemaphore, timeout, false) == WAIT_OBJECT_0; #else return ::WaitForSingleObject(hsemaphore, timeout) == WAIT_OBJECT_0; #endif } //---------------------------------------------------------------------------------------- bool AMF_CDECL_CALL amf_release_semaphore(amf_handle hsemaphore, amf_long iCount, amf_long* iOldCount) { if(hsemaphore == NULL) { return false; } return ::ReleaseSemaphore(hsemaphore, iCount, iOldCount) != FALSE; } //------------------------------------------------------------------------------ void AMF_CDECL_CALL amf_sleep(amf_ulong delay) { #if defined(METRO_APP) Concurrency::wait(delay); #else Sleep(delay); #endif } //---------------------------------------------------------------------------------------- amf_pts AMF_CDECL_CALL amf_high_precision_clock() { static int state = 0; static LARGE_INTEGER Frequency; static LARGE_INTEGER StartCount; static amf_pts offset = 0; if(state == 0) { if(QueryPerformanceFrequency(&Frequency)) { state = 1; QueryPerformanceCounter(&StartCount); } else { state = 2; } } if(state == 1) { LARGE_INTEGER PerformanceCount; if(QueryPerformanceCounter(&PerformanceCount)) { amf_pts elapsed = static_cast((PerformanceCount.QuadPart - StartCount.QuadPart) * 10000000LL / Frequency.QuadPart); // periodically reset StartCount in order to avoid overflow if (elapsed > (3600LL * AMF_SECOND)) { offset += elapsed; StartCount = PerformanceCount; return offset; } else { return offset + elapsed; } } } #if defined(METRO_APP) return GetTickCount64() * 10; #else return GetTickCount() * 10; #endif } //------------------------------------------------------------------------------------------------- #pragma comment (lib, "winmm.lib") static amf_uint32 timerPrecision = 1; void AMF_CDECL_CALL amf_increase_timer_precision() { #if !defined(METRO_APP) while (timeBeginPeriod(timerPrecision) == TIMERR_NOCANDO) { ++timerPrecision; } /* typedef NTSTATUS (CALLBACK * NTSETTIMERRESOLUTION)(IN ULONG DesiredTime,IN BOOLEAN SetResolution,OUT PULONG ActualTime); typedef NTSTATUS (CALLBACK * NTQUERYTIMERRESOLUTION)(OUT PULONG MaximumTime,OUT PULONG MinimumTime,OUT PULONG CurrentTime); HINSTANCE hNtDll = LoadLibrary(L"NTDLL.dll"); if(hNtDll != NULL) { ULONG MinimumResolution=0; ULONG MaximumResolution=0; ULONG ActualResolution=0; NTQUERYTIMERRESOLUTION NtQueryTimerResolution = (NTQUERYTIMERRESOLUTION)GetProcAddress(hNtDll, "NtQueryTimerResolution"); NTSETTIMERRESOLUTION NtSetTimerResolution = (NTSETTIMERRESOLUTION)GetProcAddress(hNtDll, "NtSetTimerResolution"); if(NtQueryTimerResolution != NULL && NtSetTimerResolution != NULL) { NtQueryTimerResolution (&MinimumResolution, &MaximumResolution, &ActualResolution); if(MaximumResolution != 0) { NtSetTimerResolution (MaximumResolution, TRUE, &ActualResolution); NtQueryTimerResolution (&MinimumResolution, &MaximumResolution, &ActualResolution); // if call NtQueryTimerResolution() again it will return the same values but precision is actually increased } } FreeLibrary(hNtDll); } */ #endif } void AMF_CDECL_CALL amf_restore_timer_precision() { #if !defined(METRO_APP) timeEndPeriod(timerPrecision); #endif } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_load_library1(const wchar_t* filename, bool /*bGlobal*/) { return amf_load_library(filename); } //---------------------------------------------------------------------------------------- amf_handle AMF_CDECL_CALL amf_load_library(const wchar_t* filename) { #if defined(METRO_APP) return LoadPackagedLibrary(filename, 0); #else return ::LoadLibraryExW(filename, NULL, LOAD_LIBRARY_SEARCH_USER_DIRS | LOAD_LIBRARY_SEARCH_APPLICATION_DIR | LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | LOAD_LIBRARY_SEARCH_SYSTEM32); #endif } //---------------------------------------------------------------------------------------- void* AMF_CDECL_CALL amf_get_proc_address(amf_handle module, const char* procName) { return (void*)::GetProcAddress((HMODULE)module, procName); } //---------------------------------------------------------------------------------------- int AMF_CDECL_CALL amf_free_library(amf_handle module) { return ::FreeLibrary((HMODULE)module)==TRUE; } #if !defined(METRO_APP) //---------------------------------------------------------------------------------------- // memory //---------------------------------------------------------------------------------------- void* AMF_CDECL_CALL amf_virtual_alloc(size_t size) { return VirtualAlloc(NULL, size, MEM_COMMIT, PAGE_READWRITE); } //---------------------------------------------------------------------------------------- void AMF_CDECL_CALL amf_virtual_free(void* ptr) { VirtualFree(ptr, NULL, MEM_RELEASE); } #endif //#if !defined(METRO_APP) //---------------------------------------------------------------------------------------- // cpu //---------------------------------------------------------------------------------------- amf_int32 AMF_STD_CALL amf_get_cpu_cores() { //query the number of CPU HW cores DWORD len = 0; GetLogicalProcessorInformation(NULL, &len); amf_uint32 count = len / sizeof(SYSTEM_LOGICAL_PROCESSOR_INFORMATION); std::unique_ptr pBuffer(new SYSTEM_LOGICAL_PROCESSOR_INFORMATION[count]); if (pBuffer) { GetLogicalProcessorInformation(pBuffer.get(), &len); count = len / sizeof(SYSTEM_LOGICAL_PROCESSOR_INFORMATION); amf_int32 iCores = 0; for (amf_uint32 idx = 0; idx < count; idx++) { if (pBuffer[idx].Relationship == RelationProcessorCore) { iCores++; } } return iCores; } return 1; } //---------------------------------------------------------------------------------------- //---------------------------------------------------------------------------------------- #endif // _WIN32 ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/Ambisonic2SRenderer.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // interface declaration; Ambisonic to Stereo Renderer //------------------------------------------------------------------------------------------------- #ifndef AMF_Ambisonic2SRenderer_h #define AMF_Ambisonic2SRenderer_h #pragma once #include "public/include/components/Component.h" #define AMFAmbisonic2SRendererHW L"AMFAmbisonic2SRenderer" enum AMF_AMBISONIC2SRENDERER_MODE_ENUM { AMF_AMBISONIC2SRENDERER_MODE_SIMPLE = 0, AMF_AMBISONIC2SRENDERER_MODE_HRTF_AMD0 = 1, AMF_AMBISONIC2SRENDERER_MODE_HRTF_MIT1 = 2, }; // static properties #define AMF_AMBISONIC2SRENDERER_IN_AUDIO_SAMPLE_RATE L"InSampleRate" // amf_int64 (default = 0) #define AMF_AMBISONIC2SRENDERER_IN_AUDIO_CHANNELS L"InChannels" // amf_int64 (only = 4) #define AMF_AMBISONIC2SRENDERER_IN_AUDIO_SAMPLE_FORMAT L"InSampleFormat" // amf_int64(AMF_AUDIO_FORMAT) (default = AMFAF_FLTP) #define AMF_AMBISONIC2SRENDERER_OUT_AUDIO_CHANNELS L"OutChannels" // amf_int64 (only = 2 - stereo) #define AMF_AMBISONIC2SRENDERER_OUT_AUDIO_SAMPLE_FORMAT L"OutSampleFormat" // amf_int64(AMF_AUDIO_FORMAT) (only = AMFAF_FLTP) #define AMF_AMBISONIC2SRENDERER_OUT_AUDIO_CHANNEL_LAYOUT L"OutChannelLayout" // amf_int64 (only = 3 - defalut stereo L R) #define AMF_AMBISONIC2SRENDERER_MODE L"StereoMode" //TODO: AMF_AMBISONIC2SRENDERER_MODE_ENUM(default=AMF_AMBISONIC2SRENDERER_MODE_HRTF) // dynamic properties #define AMF_AMBISONIC2SRENDERER_W L"w" //amf_int64 (default=0) #define AMF_AMBISONIC2SRENDERER_X L"x" //amf_int64 (default=1) #define AMF_AMBISONIC2SRENDERER_Y L"y" //amf_int64 (default=2) #define AMF_AMBISONIC2SRENDERER_Z L"z" //amf_int64 (default=3) #define AMF_AMBISONIC2SRENDERER_THETA L"Theta" //double (default=0.0) #define AMF_AMBISONIC2SRENDERER_PHI L"Phi" //double (default=0.0) #define AMF_AMBISONIC2SRENDERER_RHO L"Rho" //double (default=0.0) extern "C" { AMF_RESULT AMF_CDECL_CALL AMFCreateComponentAmbisonic(amf::AMFContext* pContext, void* reserved, amf::AMFComponent** ppComponent); } #endif //#ifndef AMF_Ambisonic2SRenderer_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/AudioCapture.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // Audio session interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_AudioCapture_h #define AMF_AudioCapture_h #pragma once #include "Component.h" // Set to capture from either a microphone or desktop #define AUDIOCAPTURE_SOURCE L"AudioCaptureSource" // amf_bool true for microphone, false for desktop; // In the case of capturing a microphone, the AUDIOCAPTURE_DEVICE_ACTIVE property // can be set to -1 so that the active input devices are looked up. If the initialization // is successful then the AUDIOCAPTURE_DEVICE_NAME and AUDIOCAPTURE_DEVICE_COUNT // properties will be set. #define AUDIOCAPTURE_DEVICE_ACTIVE L"AudioCaptureDeviceActive" // amf_int64 #define AUDIOCAPTURE_DEVICE_COUNT L"AudioCaptureDeviceCount" // amf_int64 #define AUDIOCAPTURE_DEVICE_NAME L"AudioCaptureDeviceName" // String // Codec used for audio capture #define AUDIOCAPTURE_CODEC L"AudioCaptureCodec" // amf_int64, AV_CODEC_ID_PCM_F32LE // Sample rate used for audio capture #define AUDIOCAPTURE_SAMPLERATE L"AudioCaptureSampleRate" // amf_int64, 44100 in samples // Sample count used for audio capture #define AUDIOCAPTURE_SAMPLES L"AudioCaptureSampleCount" // amf_int64, 1024 // Bitrate used for audio capture #define AUDIOCAPTURE_BITRATE L"AudioCaptureBitRate" // amf_int64, in bits // Channel count used for audio capture #define AUDIOCAPTURE_CHANNELS L"AudioCaptureChannelCount" // amf_int64, 2 // Channel layout used for audio capture #define AUDIOCAPTURE_CHANNEL_LAYOUT L"AudioCaptureChannelLayout" // amf_int64, AMF_AUDIO_CHANNEL_LAYOUT // Format used for audio capture #define AUDIOCAPTURE_FORMAT L"AudioCaptureFormat" // amf_int64, AMFAF_U8 // Block alignment #define AUDIOCAPTURE_BLOCKALIGN L"AudioCaptureBlockAlign" // amf_int64, bytes // Audio frame size #define AUDIOCAPTURE_FRAMESIZE L"AudioCaptureFrameSize" // amf_int64, bytes // Audio low latency state #define AUDIOCAPTURE_LOWLATENCY L"AudioCaptureLowLatency" // amf_int64; // Optional interface that provides current time #define AUDIOCAPTURE_CURRENT_TIME_INTERFACE L"CurrentTimeInterface" // interface to current time object extern "C" { // Component that allows the recording of inputs such as microphones or the audio that is being // rendered. The direction that is captured is controlled by the AUDIOCAPTURE_CAPTURE property // AMF_RESULT AMF_CDECL_CALL AMFCreateComponentAudioCapture(amf::AMFContext* pContext, amf::AMFComponent** ppComponent); } #endif // #ifndef AMF_AudioCapture_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/Capture.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // Capture interface declaration //------------------------------------------------------------------------------------------------- #ifndef __Capture_h__ #define __Capture_h__ #pragma once #include "../../../public/include/components/Component.h" typedef enum AMF_CAPTURE_DEVICE_TYPE_ENUM { AMF_CAPTURE_DEVICE_UNKNOWN = 0, AMF_CAPTURE_DEVICE_MEDIAFOUNDATION = 1, AMF_CAPTURE_DEVICE_WASAPI = 2, AMF_CAPTURE_DEVICE_SDI = 3, AMF_CAPTURE_DEVICE_SCREEN_DUPLICATION = 4, } AMF_CAPTURE_DEVICE_TYPE_ENUM; // device properties #define AMF_CAPTURE_DEVICE_TYPE L"DeviceType" // amf_int64( AMF_CAPTURE_DEVICE_TYPE_ENUM ) #define AMF_CAPTURE_DEVICE_NAME L"DeviceName" // wchar_t* : name of the device #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMFCaptureDevice interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFCaptureDevice : public AMFComponentEx { public: AMF_DECLARE_IID (0x5bfd1b17, 0x9f2a, 0x43c4, 0x9c, 0xdd, 0x2c, 0x3, 0x88, 0x43, 0xb5, 0xf3) virtual AMF_RESULT AMF_STD_CALL Start() = 0; virtual AMF_RESULT AMF_STD_CALL Stop() = 0; // TODO add callback interface for disconnected / lost / changed device notification }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFCaptureDevicePtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFCaptureDevice, 0x5bfd1b17, 0x9f2a, 0x43c4, 0x9c, 0xdd, 0x2c, 0x3, 0x88, 0x43, 0xb5, 0xf3) typedef struct AMFCaptureDeviceVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFCaptureDevice* pThis); amf_long (AMF_STD_CALL *Release)(AMFCaptureDevice* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFCaptureDevice* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFCaptureDevice* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFCaptureDevice* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFCaptureDevice* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFCaptureDevice* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFCaptureDevice* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFCaptureDevice* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFCaptureDevice* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFCaptureDevice* pThis, AMFPropertyStorageObserver* pObserver); // AMFPropertyStorageEx interface amf_size (AMF_STD_CALL *GetPropertiesInfoCount)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyInfoAt)(AMFCaptureDevice* pThis, amf_size index, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *GetPropertyInfo)(AMFCaptureDevice* pThis, const wchar_t* name, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *ValidateProperty)(AMFCaptureDevice* pThis, const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated); // AMFComponent interface AMF_RESULT (AMF_STD_CALL *Init)(AMFCaptureDevice* pThis, AMF_SURFACE_FORMAT format,amf_int32 width,amf_int32 height); AMF_RESULT (AMF_STD_CALL *ReInit)(AMFCaptureDevice* pThis, amf_int32 width,amf_int32 height); AMF_RESULT (AMF_STD_CALL *Terminate)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *Drain)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *Flush)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *SubmitInput)(AMFCaptureDevice* pThis, AMFData* pData); AMF_RESULT (AMF_STD_CALL *QueryOutput)(AMFCaptureDevice* pThis, AMFData** ppData); AMFContext* (AMF_STD_CALL *GetContext)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *SetOutputDataAllocatorCB)(AMFCaptureDevice* pThis, AMFDataAllocatorCB* callback); AMF_RESULT (AMF_STD_CALL *GetCaps)(AMFCaptureDevice* pThis, AMFCaps** ppCaps); AMF_RESULT (AMF_STD_CALL *Optimize)(AMFCaptureDevice* pThis, AMFComponentOptimizationCallback* pCallback); // AMFComponentEx interface amf_int32 (AMF_STD_CALL *GetInputCount)(AMFCaptureDevice* pThis); amf_int32 (AMF_STD_CALL *GetOutputCount)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *GetInput)(AMFCaptureDevice* pThis, amf_int32 index, AMFInput** ppInput); AMF_RESULT (AMF_STD_CALL *GetOutput)(AMFCaptureDevice* pThis, amf_int32 index, AMFOutput** ppOutput); // AMFCaptureDevice interface AMF_RESULT (AMF_STD_CALL *Start)(AMFCaptureDevice* pThis); AMF_RESULT (AMF_STD_CALL *Stop)(AMFCaptureDevice* pThis); } AMFCaptureVtbl; struct AMFCapture { const AMFCaptureVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFCaptureManager interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFCaptureManager : public AMFInterface { public: AMF_DECLARE_IID ( 0xf64d2f0d, 0xad16, 0x4ce7, 0x80, 0x5f, 0xa1, 0xe7, 0x3b, 0x0, 0xf4, 0x28) virtual AMF_RESULT AMF_STD_CALL Update() = 0; virtual amf_int32 AMF_STD_CALL GetDeviceCount() = 0; virtual AMF_RESULT AMF_STD_CALL GetDevice(amf_int32 index,AMFCaptureDevice **pDevice) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFCaptureManagerPtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFCaptureManager, 0xf64d2f0d, 0xad16, 0x4ce7, 0x80, 0x5f, 0xa1, 0xe7, 0x3b, 0x0, 0xf4, 0x28) typedef struct AMFCaptureManagerVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFCaptureManager* pThis); amf_long (AMF_STD_CALL *Release)(AMFCaptureManager* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFCaptureManager* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFCaptureManager interface AMF_RESULT (AMF_STD_CALL *Update)((AMFCaptureManager* pThis); amf_int32 (AMF_STD_CALL *GetDeviceCount)(AMFCaptureManager* pThis); AMF_RESULT (AMF_STD_CALL *GetDevice)(AMFCaptureManager* pThis, amf_int32 index,AMFCaptureDevice **pDevice); } AMFCaptureManagerVtbl; struct AMFCaptureManager { const AMFCaptureManagerVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace #endif extern "C" { AMF_RESULT AMF_CDECL_CALL AMFCreateCaptureManager(amf::AMFContext* pContext, amf::AMFCaptureManager** ppManager); } #endif // __Capture_h__ ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/ChromaKey.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file ChromaKey.h * @brief AMFChromaKey interface declaration *************************************************************************************************** */ #ifndef __AMFChromaKey_h__ #define __AMFChromaKey_h__ #pragma once #include "public/include/components/Component.h" #define AMFChromaKey L"AMFChromaKey" // static properties #define AMF_CHROMAKEY_COLOR L"ChromaKeyColor" // amf_uint64 (default=0x992A1E), YUV Green key Color #define AMF_CHROMAKEY_COLOR_EX L"ChromaKeyColorEX" // amf_uint64 (default=0), YUV Green key Color, secondary #define AMF_CHROMAKEY_RANGE_MIN L"ChromaKeyRangeMin" // amf_uint64 (default=20) color tolerance low, 0~255 #define AMF_CHROMAKEY_RANGE_MAX L"ChromaKeyRangeMax" // amf_uint64 (default=22) color tolerance high, 0~255 #define AMF_CHROMAKEY_RANGE_EXT L"ChromaKeyRangeExt" // amf_uint64 (default=40) color tolerance extended, 0~255 #define AMF_CHROMAKEY_SPILL_MODE L"ChromaKeySpillMode" // amf_uint64 (default=0) spill suppression mode #define AMF_CHROMAKEY_RANGE_SPILL L"ChromaKeyRangeSpill" // amf_uint64 (default=5) spill suppression threshold #define AMF_CHROMAKEY_LUMA_LOW L"ChromaKeyLumaLow" // amf_uint64 (default=16) minimum luma value for processing #define AMF_CHROMAKEY_INPUT_COUNT L"InputCount" // amf_uint64 (default=2) number of inputs #define AMF_CHROMAKEY_COLOR_POS L"KeyColorPos" // amf_uint64 (default=0) key color position from the surface #define AMF_CHROMAKEY_OUT_FORMAT L"ChromaKeyOutFormat" // amf_uint64 (default=RGBA) output format #define AMF_CHROMAKEY_MEMORY_TYPE L"ChromaKeyMemoryType" // amf_uint64 (default=DX11) mmeory type #define AMF_CHROMAKEY_COLOR_ADJ L"ChromaKeyColorAdj" // amf_uint64 (default=0) endble color adjustment #define AMF_CHROMAKEY_COLOR_ADJ_THRE L"ChromaKeyColorAdjThre" // amf_uint64 (default=0) color adjustment threshold #define AMF_CHROMAKEY_COLOR_ADJ_THRE2 L"ChromaKeyColorAdjThre2" // amf_uint64 (default=0) color adjustment threshold #define AMF_CHROMAKEY_BYPASS L"ChromaKeyBypass" // amf_uint64 (default=0) disable chromakey #define AMF_CHROMAKEY_EDGE L"ChromaKeyEdge" // amf_uint64 (default=0) endble edge detection #define AMF_CHROMAKEY_BOKEH L"ChromaKeyBokeh" // amf_uint64 (default=0) endble background bokeh #define AMF_CHROMAKEY_BOKEH_RADIUS L"ChromaKeyBokehRadius" // amf_uint64 (default=7) background bokeh radius #define AMF_CHROMAKEY_DEBUG L"ChromaKeyDebug" // amf_uint64 (default=0) endble debug mode #define AMF_CHROMAKEY_POSX L"ChromaKeyPosX" // amf_uint64 (default=0) positionX #define AMF_CHROMAKEY_POSY L"ChromaKeyPosY" // amf_uint64 (default=0) positionY extern "C" { AMF_RESULT AMF_CDECL_CALL AMFCreateComponentChromaKey(amf::AMFContext* pContext, amf::AMFComponentEx** ppComponent); } #endif //#ifndef __AMFChromaKey_h__ ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/ColorSpace.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // Color Spacedeclaration //------------------------------------------------------------------------------------------------- #ifndef AMF_ColorSpace_h #define AMF_ColorSpace_h #pragma once #include "../core/Platform.h" // YUV <--> RGB conversion matrix with range typedef enum AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM { AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN =-1, AMF_VIDEO_CONVERTER_COLOR_PROFILE_601 = 0, // studio range AMF_VIDEO_CONVERTER_COLOR_PROFILE_709 = 1, // studio range AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020 = 2, // studio range AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG = 3, // full range 601 // AMF_VIDEO_CONVERTER_COLOR_PROFILE_G22_BT709 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709, // AMF_VIDEO_CONVERTER_COLOR_PROFILE_G10_SCRGB = 4, // AMF_VIDEO_CONVERTER_COLOR_PROFILE_G10_BT709 = 5, // AMF_VIDEO_CONVERTER_COLOR_PROFILE_G10_BT2020 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020, // AMF_VIDEO_CONVERTER_COLOR_PROFILE_G2084_BT2020 = 6, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG, // full range AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709 = 7, // full range AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020 = 8, // full range AMF_VIDEO_CONVERTER_COLOR_PROFILE_COUNT } AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM; typedef enum AMF_COLOR_PRIMARIES_ENUM // as in VUI color_primaries AVC and HEVC { AMF_COLOR_PRIMARIES_UNDEFINED = 0, AMF_COLOR_PRIMARIES_BT709 = 1, AMF_COLOR_PRIMARIES_UNSPECIFIED = 2, AMF_COLOR_PRIMARIES_RESERVED = 3, AMF_COLOR_PRIMARIES_BT470M = 4, AMF_COLOR_PRIMARIES_BT470BG = 5, AMF_COLOR_PRIMARIES_SMPTE170M = 6, AMF_COLOR_PRIMARIES_SMPTE240M = 7, AMF_COLOR_PRIMARIES_FILM = 8, AMF_COLOR_PRIMARIES_BT2020 = 9, AMF_COLOR_PRIMARIES_SMPTE428 = 10, AMF_COLOR_PRIMARIES_SMPTE431 = 11, AMF_COLOR_PRIMARIES_SMPTE432 = 12, AMF_COLOR_PRIMARIES_JEDEC_P22 = 22, AMF_COLOR_PRIMARIES_CCCS = 1000, // Common Composition Color Space or scRGB } AMF_COLOR_PRIMARIES_ENUM; typedef enum AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM // as in VUI transfer_characteristic AVC and HEVC { AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED = 0, AMF_COLOR_TRANSFER_CHARACTERISTIC_BT709 = 1, //BT709 AMF_COLOR_TRANSFER_CHARACTERISTIC_UNSPECIFIED = 2, AMF_COLOR_TRANSFER_CHARACTERISTIC_RESERVED = 3, AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 = 4, //BT470_M AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA28 = 5, //BT470 AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE170M = 6, //BT601 AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE240M = 7, //SMPTE 240M AMF_COLOR_TRANSFER_CHARACTERISTIC_LINEAR = 8, AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG = 9, //LOG10 AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG_SQRT = 10,//LOG10 SQRT AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_4 = 11, AMF_COLOR_TRANSFER_CHARACTERISTIC_BT1361_ECG = 12, AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_1 = 13, AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_10 = 14, //BT709 AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_12 = 15, //BT709 AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE2084 = 16, //PQ AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE428 = 17, AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67 = 18, //HLG } AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM; typedef enum AMF_COLOR_BIT_DEPTH_ENUM { AMF_COLOR_BIT_DEPTH_UNDEFINED = 0, AMF_COLOR_BIT_DEPTH_8 = 8, AMF_COLOR_BIT_DEPTH_10 = 10, } AMF_COLOR_BIT_DEPTH_ENUM; typedef struct AMFHDRMetadata { amf_uint16 redPrimary[2]; // normalized to 50000 amf_uint16 greenPrimary[2]; // normalized to 50000 amf_uint16 bluePrimary[2]; // normalized to 50000 amf_uint16 whitePoint[2]; // normalized to 50000 amf_uint32 maxMasteringLuminance; // normalized to 10000 amf_uint32 minMasteringLuminance; // normalized to 10000 amf_uint16 maxContentLightLevel; // nit value amf_uint16 maxFrameAverageLightLevel; // nit value } AMFHDRMetadata; typedef enum AMF_COLOR_RANGE_ENUM { AMF_COLOR_RANGE_UNDEFINED = 0, AMF_COLOR_RANGE_STUDIO = 1, AMF_COLOR_RANGE_FULL = 2, } AMF_COLOR_RANGE_ENUM; // these properties can be set on input or outout surface // IDs are the same as in decoder properties // can be used to dynamically pass color data between components: // Decoder, Capture, Encoder. Presenter etc. #define AMF_VIDEO_COLOR_TRANSFER_CHARACTERISTIC L"ColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 Section 7.2 See ColorSpace.h for enum #define AMF_VIDEO_COLOR_PRIMARIES L"ColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 Section 7.1 See ColorSpace.h for enum #define AMF_VIDEO_COLOR_RANGE L"ColorRange" // amf_int64(AMF_COLOR_RANGE_ENUM) default = AMF_COLOR_RANGE_UNDEFINED #define AMF_VIDEO_COLOR_HDR_METADATA L"HdrMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL #endif //#ifndef AMF_ColorSpace_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/Component.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file Component.h * @brief AMFComponent interface declaration *************************************************************************************************** */ #ifndef AMF_Component_h #define AMF_Component_h #pragma once #include "../core/Data.h" #include "../core/PropertyStorageEx.h" #include "../core/Surface.h" #include "../core/Context.h" #include "ComponentCaps.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMFDataAllocatorCB interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFDataAllocatorCB : public AMFInterface { public: AMF_DECLARE_IID(0x4bf46198, 0x8b7b, 0x49d0, 0xaa, 0x72, 0x48, 0xd4, 0x7, 0xce, 0x24, 0xc5 ) virtual AMF_RESULT AMF_STD_CALL AllocBuffer(AMF_MEMORY_TYPE type, amf_size size, AMFBuffer** ppBuffer) = 0; virtual AMF_RESULT AMF_STD_CALL AllocSurface(AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32 vPitch, AMFSurface** ppSurface) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFDataAllocatorCBPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFDataAllocatorCB, 0x4bf46198, 0x8b7b, 0x49d0, 0xaa, 0x72, 0x48, 0xd4, 0x7, 0xce, 0x24, 0xc5 ) typedef struct AMFDataAllocatorCB AMFDataAllocatorCB; typedef struct AMFDataAllocatorCBVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFDataAllocatorCB* pThis); amf_long (AMF_STD_CALL *Release)(AMFDataAllocatorCB* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFDataAllocatorCB* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFDataAllocatorCB interface AMF_RESULT (AMF_STD_CALL *AllocBuffer)(AMFDataAllocatorCB* pThis, AMF_MEMORY_TYPE type, amf_size size, AMFBuffer** ppBuffer); AMF_RESULT (AMF_STD_CALL *AllocSurface)(AMFDataAllocatorCB* pThis, AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32 vPitch, AMFSurface** ppSurface); } AMFDataAllocatorCBVtbl; struct AMFDataAllocatorCB { const AMFDataAllocatorCBVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFComponentOptimizationCallback { public: virtual AMF_RESULT AMF_STD_CALL OnComponentOptimizationProgress(amf_uint percent) = 0; }; #else // #if defined(__cplusplus) typedef struct AMFComponentOptimizationCallback AMFComponentOptimizationCallback; typedef struct AMFComponentOptimizationCallbackVtbl { // AMFDataAllocatorCB interface AMF_RESULT (AMF_STD_CALL *OnComponentOptimizationProgress)(AMFComponentOptimizationCallback* pThis, amf_uint percent); } AMFComponentOptimizationCallbackVtbl; struct AMFComponentOptimizationCallback { const AMFComponentOptimizationCallbackVtbl *pVtbl; }; #endif //#if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFComponent interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFComponent : public AMFPropertyStorageEx { public: AMF_DECLARE_IID(0x8b51e5e4, 0x455d, 0x4034, 0xa7, 0x46, 0xde, 0x1b, 0xed, 0xc3, 0xc4, 0x6) virtual AMF_RESULT AMF_STD_CALL Init(AMF_SURFACE_FORMAT format,amf_int32 width,amf_int32 height) = 0; virtual AMF_RESULT AMF_STD_CALL ReInit(amf_int32 width,amf_int32 height) = 0; virtual AMF_RESULT AMF_STD_CALL Terminate() = 0; virtual AMF_RESULT AMF_STD_CALL Drain() = 0; virtual AMF_RESULT AMF_STD_CALL Flush() = 0; virtual AMF_RESULT AMF_STD_CALL SubmitInput(AMFData* pData) = 0; virtual AMF_RESULT AMF_STD_CALL QueryOutput(AMFData** ppData) = 0; virtual AMFContext* AMF_STD_CALL GetContext() = 0; virtual AMF_RESULT AMF_STD_CALL SetOutputDataAllocatorCB(AMFDataAllocatorCB* callback) = 0; virtual AMF_RESULT AMF_STD_CALL GetCaps(AMFCaps** ppCaps) = 0; virtual AMF_RESULT AMF_STD_CALL Optimize(AMFComponentOptimizationCallback* pCallback) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFComponentPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFComponent, 0x8b51e5e4, 0x455d, 0x4034, 0xa7, 0x46, 0xde, 0x1b, 0xed, 0xc3, 0xc4, 0x6) typedef struct AMFComponent AMFComponent; typedef struct AMFComponentVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFComponent* pThis); amf_long (AMF_STD_CALL *Release)(AMFComponent* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFComponent* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFComponent* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFComponent* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFComponent* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFComponent* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFComponent* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFComponent* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFComponent* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFComponent* pThis, AMFPropertyStorageObserver* pObserver); // AMFPropertyStorageEx interface amf_size (AMF_STD_CALL *GetPropertiesInfoCount)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyInfoAt)(AMFComponent* pThis, amf_size index, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *GetPropertyInfo)(AMFComponent* pThis, const wchar_t* name, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *ValidateProperty)(AMFComponent* pThis, const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated); // AMFComponent interface AMF_RESULT (AMF_STD_CALL *Init)(AMFComponent* pThis, AMF_SURFACE_FORMAT format,amf_int32 width,amf_int32 height); AMF_RESULT (AMF_STD_CALL *ReInit)(AMFComponent* pThis, amf_int32 width,amf_int32 height); AMF_RESULT (AMF_STD_CALL *Terminate)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *Drain)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *Flush)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *SubmitInput)(AMFComponent* pThis, AMFData* pData); AMF_RESULT (AMF_STD_CALL *QueryOutput)(AMFComponent* pThis, AMFData** ppData); AMFContext* (AMF_STD_CALL *GetContext)(AMFComponent* pThis); AMF_RESULT (AMF_STD_CALL *SetOutputDataAllocatorCB)(AMFComponent* pThis, AMFDataAllocatorCB* callback); AMF_RESULT (AMF_STD_CALL *GetCaps)(AMFComponent* pThis, AMFCaps** ppCaps); AMF_RESULT (AMF_STD_CALL *Optimize)(AMFComponent* pThis, AMFComponentOptimizationCallback* pCallback); } AMFComponentVtbl; struct AMFComponent { const AMFComponentVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFInput interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFInput : public AMFPropertyStorageEx { public: AMF_DECLARE_IID(0x1181eee7, 0x95f2, 0x434a, 0x9b, 0x96, 0xea, 0x55, 0xa, 0xa7, 0x84, 0x89) virtual AMF_RESULT AMF_STD_CALL SubmitInput(AMFData* pData) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFInputPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFInput, 0x1181eee7, 0x95f2, 0x434a, 0x9b, 0x96, 0xea, 0x55, 0xa, 0xa7, 0x84, 0x89) typedef struct AMFInput AMFInput; typedef struct AMFInputVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFInput* pThis); amf_long (AMF_STD_CALL *Release)(AMFInput* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFInput* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFInput* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFInput* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFInput* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFInput* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFInput* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFInput* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFInput* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFInput* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFInput* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFInput* pThis, AMFPropertyStorageObserver* pObserver); // AMFPropertyStorageEx interface amf_size (AMF_STD_CALL *GetPropertiesInfoCount)(AMFInput* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyInfoAt)(AMFInput* pThis, amf_size index, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *GetPropertyInfo)(AMFInput* pThis, const wchar_t* name, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *ValidateProperty)(AMFInput* pThis, const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated); // AMFInput interface AMF_RESULT (AMF_STD_CALL *SubmitInput)(AMFInput* pThis, AMFData* pData); } AMFInputVtbl; struct AMFInput { const AMFInputVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFOutput interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFOutput : public AMFPropertyStorageEx { public: AMF_DECLARE_IID(0x86a8a037, 0x912c, 0x4698, 0xb0, 0x46, 0x7, 0x5a, 0x1f, 0xac, 0x6b, 0x97) virtual AMF_RESULT AMF_STD_CALL QueryOutput(AMFData** ppData) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFOutputPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFOutput, 0x86a8a037, 0x912c, 0x4698, 0xb0, 0x46, 0x7, 0x5a, 0x1f, 0xac, 0x6b, 0x97) typedef struct AMFOutput AMFOutput; typedef struct AMFOutputVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFOutput* pThis); amf_long (AMF_STD_CALL *Release)(AMFOutput* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFOutput* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFOutput* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFOutput* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFOutput* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFOutput* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFOutput* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFOutput* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFOutput* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFOutput* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFOutput* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFOutput* pThis, AMFPropertyStorageObserver* pObserver); // AMFPropertyStorageEx interface amf_size (AMF_STD_CALL *GetPropertiesInfoCount)(AMFOutput* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyInfoAt)(AMFOutput* pThis, amf_size index, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *GetPropertyInfo)(AMFOutput* pThis, const wchar_t* name, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *ValidateProperty)(AMFOutput* pThis, const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated); // AMFOutput interface AMF_RESULT (AMF_STD_CALL *QueryOutput)(AMFOutput* pThis, AMFData** ppData); } AMFOutputVtbl; struct AMFOutput { const AMFOutputVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFComponent interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFComponentEx : public AMFComponent { public: AMF_DECLARE_IID(0xfda792af, 0x8712, 0x44df, 0x8e, 0xa0, 0xdf, 0xfa, 0xad, 0x2c, 0x80, 0x93) virtual amf_int32 AMF_STD_CALL GetInputCount() = 0; virtual amf_int32 AMF_STD_CALL GetOutputCount() = 0; virtual AMF_RESULT AMF_STD_CALL GetInput(amf_int32 index, AMFInput** ppInput) = 0; virtual AMF_RESULT AMF_STD_CALL GetOutput(amf_int32 index, AMFOutput** ppOutput) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFComponentExPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFComponentEx, 0xfda792af, 0x8712, 0x44df, 0x8e, 0xa0, 0xdf, 0xfa, 0xad, 0x2c, 0x80, 0x93) typedef struct AMFComponentEx AMFComponentEx; typedef struct AMFComponentExVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFComponentEx* pThis); amf_long (AMF_STD_CALL *Release)(AMFComponentEx* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFComponentEx* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFComponentEx* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFComponentEx* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFComponentEx* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFComponentEx* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFComponentEx* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFComponentEx* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFComponentEx* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFComponentEx* pThis, AMFPropertyStorageObserver* pObserver); // AMFPropertyStorageEx interface amf_size (AMF_STD_CALL *GetPropertiesInfoCount)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyInfoAt)(AMFComponentEx* pThis, amf_size index, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *GetPropertyInfo)(AMFComponentEx* pThis, const wchar_t* name, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *ValidateProperty)(AMFComponentEx* pThis, const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated); // AMFComponent interface AMF_RESULT (AMF_STD_CALL *Init)(AMFComponentEx* pThis, AMF_SURFACE_FORMAT format,amf_int32 width,amf_int32 height); AMF_RESULT (AMF_STD_CALL *ReInit)(AMFComponentEx* pThis, amf_int32 width,amf_int32 height); AMF_RESULT (AMF_STD_CALL *Terminate)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *Drain)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *Flush)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *SubmitInput)(AMFComponentEx* pThis, AMFData* pData); AMF_RESULT (AMF_STD_CALL *QueryOutput)(AMFComponentEx* pThis, AMFData** ppData); AMFContext* (AMF_STD_CALL *GetContext)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *SetOutputDataAllocatorCB)(AMFComponentEx* pThis, AMFDataAllocatorCB* callback); AMF_RESULT (AMF_STD_CALL *GetCaps)(AMFComponentEx* pThis, AMFCaps** ppCaps); AMF_RESULT (AMF_STD_CALL *Optimize)(AMFComponentEx* pThis, AMFComponentOptimizationCallback* pCallback); // AMFComponentEx interface amf_int32 (AMF_STD_CALL *GetInputCount)(AMFComponentEx* pThis); amf_int32 (AMF_STD_CALL *GetOutputCount)(AMFComponentEx* pThis); AMF_RESULT (AMF_STD_CALL *GetInput)(AMFComponentEx* pThis, amf_int32 index, AMFInput** ppInput); AMF_RESULT (AMF_STD_CALL *GetOutput)(AMFComponentEx* pThis, amf_int32 index, AMFOutput** ppOutput); } AMFComponentExVtbl; struct AMFComponentEx { const AMFComponentExVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace #endif typedef enum AMF_STREAM_TYPE_ENUM { AMF_STREAM_UNKNOWN = 0, AMF_STREAM_VIDEO = 1, AMF_STREAM_AUDIO = 2, AMF_STREAM_DATA = 3, } AMF_STREAM_TYPE_ENUM; typedef enum AMF_STREAM_CODEC_ID_ENUM // matched codecs from VideoDecoxcderUVD.h { AMF_STREAM_CODEC_ID_UNKNOWN = 0, AMF_STREAM_CODEC_ID_MPEG2 = 1, // AMFVideoDecoderUVD_MPEG2 AMF_STREAM_CODEC_ID_MPEG4 = 2, // AMFVideoDecoderUVD_MPEG4 AMF_STREAM_CODEC_ID_WMV3 = 3, // AMFVideoDecoderUVD_WMV3 AMF_STREAM_CODEC_ID_VC1 = 4, // AMFVideoDecoderUVD_VC1 AMF_STREAM_CODEC_ID_H264_AVC = 5, // AMFVideoDecoderUVD_H264_AVC AMF_STREAM_CODEC_ID_H264_MVC = 6, // AMFVideoDecoderUVD_H264_MVC AMF_STREAM_CODEC_ID_H264_SVC = 7, // AMFVideoDecoderUVD_H264_SVC AMF_STREAM_CODEC_ID_MJPEG = 8, // AMFVideoDecoderUVD_MJPEG AMF_STREAM_CODEC_ID_H265_HEVC = 9, // AMFVideoDecoderHW_H265_HEVC AMF_STREAM_CODEC_ID_H265_MAIN10 = 10, // AMFVideoDecoderHW_H265_MAIN10 AMF_STREAM_CODEC_ID_VP9 = 11, // AMFVideoDecoderHW_VP9 AMF_STREAM_CODEC_ID_VP9_10BIT = 12, // AMFVideoDecoderHW_VP9_10BIT AMF_STREAM_CODEC_ID_AV1 = 13, // AMFVideoDecoderHW_AV1 AMF_STREAM_CODEC_ID_AV1_12BIT = 14, // AMFVideoDecoderHW_AV1_12BIT } AMF_STREAM_CODEC_ID_ENUM; // common stream properties #define AMF_STREAM_TYPE L"StreamType" // amf_int64( AMF_STREAM_TYPE_ENUM ) #define AMF_STREAM_ENABLED L"Enabled" // bool( default = false ) #define AMF_STREAM_CODEC_ID L"CodecID" // amf_int64(Video: AMF_STREAM_CODEC_ID_ENUM, Audio: AVCodecID) (default = 0 - uncompressed) #define AMF_STREAM_BIT_RATE L"BitRate" // amf_int64 (default = codec->bit_rate) #define AMF_STREAM_EXTRA_DATA L"ExtraData" // interface to AMFBuffer - as is from FFMPEG // video stream properties #define AMF_STREAM_VIDEO_MEMORY_TYPE L"VideoMemoryType" // amf_int64(AMF_MEMORY_TYPE); default = AMF_MEMORY_DX11 #define AMF_STREAM_VIDEO_FORMAT L"VideoFormat" // amf_int64(AMF_SURFACE_FORMAT); default = AMF_SURFACE_NV12 (used if AMF_STREAM_CODEC_ID == 0) #define AMF_STREAM_VIDEO_FRAME_RATE L"VideoFrameRate" // AMFRate; default = (30,1) - video frame rate #define AMF_STREAM_VIDEO_FRAME_SIZE L"VideoFrameSize" // AMFSize; default = (1920,1080) - video frame rate #define AMF_STREAM_VIDEO_SURFACE_POOL L"VideoSurfacePool" // amf_int64; default = 5, number of allocated output surfaces //TODO support interlaced frames // audio stream properties #define AMF_STREAM_AUDIO_FORMAT L"AudioFormat" // amf_int64(AMF_AUDIO_FORMAT); default = AMFAF_S16 #define AMF_STREAM_AUDIO_SAMPLE_RATE L"AudioSampleRate" // amf_int64; default = 48000 #define AMF_STREAM_AUDIO_CHANNELS L"AudioChannels" // amf_int64; default = 2 #define AMF_STREAM_AUDIO_CHANNEL_LAYOUT L"AudioChannelLayout" // amf_int64 (default = codec->channel_layout) #define AMF_STREAM_AUDIO_BLOCK_ALIGN L"AudioBlockAlign" // amf_int64 (default = codec->block_align) #define AMF_STREAM_AUDIO_FRAME_SIZE L"AudioFrameSize" // amf_int64 (default = codec->frame_size) #endif //#ifndef AMF_Component_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/ComponentCaps.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_ComponentCaps_h #define AMF_ComponentCaps_h #pragma once #include "../core/Interface.h" #include "../core/PropertyStorage.h" #include "../core/Surface.h" #if defined(__cplusplus) namespace amf { #endif typedef enum AMF_ACCELERATION_TYPE { AMF_ACCEL_NOT_SUPPORTED = -1, AMF_ACCEL_HARDWARE, AMF_ACCEL_GPU, AMF_ACCEL_SOFTWARE } AMF_ACCELERATION_TYPE; //---------------------------------------------------------------------------------------------- // AMFIOCaps interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFIOCaps : public AMFInterface { public: // Get supported resolution ranges in pixels/lines: virtual void AMF_STD_CALL GetWidthRange(amf_int32* minWidth, amf_int32* maxWidth) const = 0; virtual void AMF_STD_CALL GetHeightRange(amf_int32* minHeight, amf_int32* maxHeight) const = 0; // Get memory alignment in lines: Vertical aligmnent should be multiples of this number virtual amf_int32 AMF_STD_CALL GetVertAlign() const = 0; // Enumerate supported surface pixel formats virtual amf_int32 AMF_STD_CALL GetNumOfFormats() const = 0; virtual AMF_RESULT AMF_STD_CALL GetFormatAt(amf_int32 index, AMF_SURFACE_FORMAT* format, amf_bool* native) const = 0; // Enumerate supported memory types virtual amf_int32 AMF_STD_CALL GetNumOfMemoryTypes() const = 0; virtual AMF_RESULT AMF_STD_CALL GetMemoryTypeAt(amf_int32 index, AMF_MEMORY_TYPE* memType, amf_bool* native) const = 0; virtual amf_bool AMF_STD_CALL IsInterlacedSupported() const = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFIOCapsPtr; #else // #if defined(__cplusplus) typedef struct AMFIOCaps AMFIOCaps; typedef struct AMFIOCapsVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFIOCaps* pThis); amf_long (AMF_STD_CALL *Release)(AMFIOCaps* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFIOCaps* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFIOCaps interface // Get supported resolution ranges in pixels/lines: void (AMF_STD_CALL *GetWidthRange)(AMFIOCaps* pThis, amf_int32* minWidth, amf_int32* maxWidth); void (AMF_STD_CALL *GetHeightRange)(AMFIOCaps* pThis, amf_int32* minHeight, amf_int32* maxHeight); // Get memory alignment in lines: Vertical aligmnent should be multiples of this number amf_int32 (AMF_STD_CALL *GetVertAlign)(AMFIOCaps* pThis); // Enumerate supported surface pixel formats amf_int32 (AMF_STD_CALL *GetNumOfFormats)(AMFIOCaps* pThis); AMF_RESULT (AMF_STD_CALL *GetFormatAt)(AMFIOCaps* pThis, amf_int32 index, AMF_SURFACE_FORMAT* format, amf_bool* native); // Enumerate supported memory types amf_int32 (AMF_STD_CALL *GetNumOfMemoryTypes)(AMFIOCaps* pThis); AMF_RESULT (AMF_STD_CALL *GetMemoryTypeAt)(AMFIOCaps* pThis, amf_int32 index, AMF_MEMORY_TYPE* memType, amf_bool* native); amf_bool (AMF_STD_CALL *IsInterlacedSupported)(AMFIOCaps* pThis); } AMFIOCapsVtbl; struct AMFIOCaps { const AMFIOCapsVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFCaps interface - base interface for every h/w module supported by Capability Manager //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFCaps : public AMFPropertyStorage { public: virtual AMF_ACCELERATION_TYPE AMF_STD_CALL GetAccelerationType() const = 0; virtual AMF_RESULT AMF_STD_CALL GetInputCaps(AMFIOCaps** input) = 0; virtual AMF_RESULT AMF_STD_CALL GetOutputCaps(AMFIOCaps** output) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFCapsPtr; #else // #if defined(__cplusplus) typedef struct AMFCaps AMFCaps; typedef struct AMFCapsVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFCaps* pThis); amf_long (AMF_STD_CALL *Release)(AMFCaps* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFCaps* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFCaps* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFCaps* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFCaps* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFCaps* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFCaps* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFCaps* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFCaps* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFCaps* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFCaps* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFCaps* pThis, AMFPropertyStorageObserver* pObserver); // AMFCaps interface AMF_ACCELERATION_TYPE (AMF_STD_CALL *GetAccelerationType)(AMFCaps* pThis); AMF_RESULT (AMF_STD_CALL *GetInputCaps)(AMFCaps* pThis, AMFIOCaps** input); AMF_RESULT (AMF_STD_CALL *GetOutputCaps)(AMFCaps* pThis, AMFIOCaps** output); } AMFCapsVtbl; struct AMFCaps { const AMFCapsVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) } #endif #endif //#ifndef AMF_ComponentCaps_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/CursorCapture.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // Cursor capture interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_CursorCapture_h #define AMF_CursorCapture_h #pragma once namespace amf { class AMFCursorCapture : public AMFInterface { public: AMF_DECLARE_IID(0x166efa1a, 0x19b8, 0x42f2, 0x86, 0x0f, 0x56, 0x69, 0xca, 0x7a, 0x85, 0x4c) virtual AMF_RESULT AMF_STD_CALL AcquireCursor(amf::AMFSurface** pSurface) = 0; virtual AMF_RESULT AMF_STD_CALL Reset() = 0; }; typedef AMFInterfacePtr_T AMFCursorCapturePtr; } #endif // #ifndef AMF_CursorCapture_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/DisplayCapture.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // Desktop duplication interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_DisplayCapture_h #define AMF_DisplayCapture_h #pragma once #include "Component.h" extern "C" { // To create capture component with Desktop Duplication API use this function AMF_RESULT AMF_CDECL_CALL AMFCreateComponentDisplayCapture(amf::AMFContext* pContext, void* reserved, amf::AMFComponent** ppComponent); } // To create AMD Direct Capture component use this component ID with AMFFactory::CreateComponent() #define AMFDisplayCapture L"AMFDisplayCapture" // Static properties // typedef enum AMF_DISPLAYCAPTURE_MODE_ENUM { AMF_DISPLAYCAPTURE_MODE_KEEP_FRAMERATE = 0, // capture component maintains the frame rate and returns current visible surface AMF_DISPLAYCAPTURE_MODE_WAIT_FOR_PRESENT = 1, // capture component waits for flip (present) event AMF_DISPLAYCAPTURE_MODE_GET_CURRENT_SURFACE = 2, // returns current visible surface immediately } AMF_DISPLAYCAPTURE_MODE_ENUM; #define AMF_DISPLAYCAPTURE_MONITOR_INDEX L"MonitorIndex" // amf_int64, default = 0, Index of the display monitor; is determined by using EnumAdapters() in DXGI. #define AMF_DISPLAYCAPTURE_MODE L"CaptureMode" // amf_int64(AMF_DISPLAYCAPTURE_MODE_ENUM), default = AMF_DISPLAYCAPTURE_MODE_FRAMERATE, controls wait logic #define AMF_DISPLAYCAPTURE_FRAMERATE L"FrameRate" // AMFRate, default = (0, 1) Capture framerate, if 0 - capture rate will be driven by flip event from fullscreen app or DWM #define AMF_DISPLAYCAPTURE_CURRENT_TIME_INTERFACE L"CurrentTimeInterface" // AMFInterface(AMFCurrentTime) Optional interface object for providing timestamps. #define AMF_DISPLAYCAPTURE_FORMAT L"CurrentFormat" // amf_int64(AMF_SURFACE_FORMAT) Capture format - read-only #define AMF_DISPLAYCAPTURE_RESOLUTION L"Resolution" // AMFSize - screen resolution - read-only #define AMF_DISPLAYCAPTURE_DUPLICATEOUTPUT L"DuplicateOutput" // amf_bool, default = false, output AMF surface is a copy of captured #define AMF_DISPLAYCAPTURE_DESKTOP_RECT L"DesktopRect" // AMFRect - rect of the capture desktop - read-only #define AMF_DISPLAYCAPTURE_ENABLE_DIRTY_RECTS L"EnableDirtyRects" // amf_bool, default = false, enable dirty rectangles attached to output as AMF_DISPLAYCAPTURE_DIRTY_RECTS #define AMF_DISPLAYCAPTURE_DRAW_DIRTY_RECTS L"DrawDirtyRects" // amf_bool, default = false, copies capture output and draws dirty rectangles with red - for debugging only #define AMF_DISPLAYCAPTURE_ROTATION L"Rotation" // amf_int64(AMF_ROTATION_ENUM); default = AMF_ROTATION_NONE, monitor rotation state // Properties that can be set on output AMFSurface #define AMF_DISPLAYCAPTURE_DIRTY_RECTS L"DirtyRects" // AMFInterface*(AMFBuffer*) - array of AMFRect(s) #define AMF_DISPLAYCAPTURE_FRAME_INDEX L"FrameIndex" // amf_int64; default = 0, index of presented frame since capture started #define AMF_DISPLAYCAPTURE_FRAME_FLIP_TIMESTAMP L"FlipTimesamp" // amf_int64; default = 0, flip timestmap of presented frame // see Surface.h //#define AMF_SURFACE_ROTATION L"Rotation" // amf_int64(AMF_ROTATION_ENUM); default = AMF_ROTATION_NONE, can be set on surfaces - the same value as AMF_DISPLAYCAPTURE_ROTATION #endif // #ifndef AMF_DisplayCapture_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGAudioConverter.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // AMFFAudioConverterFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_AudioConverterFFMPEG_h #define AMF_AudioConverterFFMPEG_h #pragma once #define FFMPEG_AUDIO_CONVERTER L"AudioConverterFFMPEG" #define AUDIO_CONVERTER_IN_AUDIO_BIT_RATE L"In_BitRate" // amf_int64 (default = 128000) #define AUDIO_CONVERTER_IN_AUDIO_SAMPLE_RATE L"In_SampleRate" // amf_int64 (default = 0) #define AUDIO_CONVERTER_IN_AUDIO_CHANNELS L"In_Channels" // amf_int64 (default = 2) #define AUDIO_CONVERTER_IN_AUDIO_SAMPLE_FORMAT L"In_SampleFormat" // amf_int64 (default = AMFAF_UNKNOWN) (AMF_AUDIO_FORMAT) #define AUDIO_CONVERTER_IN_AUDIO_CHANNEL_LAYOUT L"In_ChannelLayout" // amf_int64 (default = 0) #define AUDIO_CONVERTER_IN_AUDIO_BLOCK_ALIGN L"In_BlockAlign" // amf_int64 (default = 0) #define AUDIO_CONVERTER_OUT_AUDIO_BIT_RATE L"Out_BitRate" // amf_int64 (default = 128000) #define AUDIO_CONVERTER_OUT_AUDIO_SAMPLE_RATE L"Out_SampleRate" // amf_int64 (default = 0) #define AUDIO_CONVERTER_OUT_AUDIO_CHANNELS L"Out_Channels" // amf_int64 (default = 2) #define AUDIO_CONVERTER_OUT_AUDIO_SAMPLE_FORMAT L"Out_SampleFormat" // amf_int64 (default = AMFAF_UNKNOWN) (AMF_AUDIO_FORMAT) #define AUDIO_CONVERTER_OUT_AUDIO_CHANNEL_LAYOUT L"Out_ChannelLayout" // amf_int64 (default = 0) #define AUDIO_CONVERTER_OUT_AUDIO_BLOCK_ALIGN L"Out_BlockAlign" // amf_int64 (default = 0) #endif //#ifndef AMF_AudioConverterFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGAudioDecoder.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // AudioDecoderFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_AudioDecoderFFMPEG_h #define AMF_AudioDecoderFFMPEG_h #pragma once #define FFMPEG_AUDIO_DECODER L"AudioDecoderFFMPEG" #define AUDIO_DECODER_ENABLE_DEBUGGING L"EnableDebug" // bool (default = false) - trace some debug information if set to true #define AUDIO_DECODER_ENABLE_DECODING L"EnableDecoding" // bool (default = true) - if false, component will not decode anything #define AUDIO_DECODER_IN_AUDIO_CODEC_ID L"In_CodecID" // amf_int64 (default = AV_CODEC_ID_NONE) - FFMPEG codec ID #define AUDIO_DECODER_IN_AUDIO_BIT_RATE L"In_BitRate" // amf_int64 (default = 128000) #define AUDIO_DECODER_IN_AUDIO_EXTRA_DATA L"In_ExtraData" // interface to AMFBuffer #define AUDIO_DECODER_IN_AUDIO_SAMPLE_RATE L"In_SampleRate" // amf_int64 (default = 0) #define AUDIO_DECODER_IN_AUDIO_CHANNELS L"In_Channels" // amf_int64 (default = 2) #define AUDIO_DECODER_IN_AUDIO_SAMPLE_FORMAT L"In_SampleFormat" // amf_int64 (default = AMFAF_UNKNOWN) (AMF_AUDIO_FORMAT) #define AUDIO_DECODER_IN_AUDIO_CHANNEL_LAYOUT L"In_ChannelLayout" // amf_int64 (default = 0) #define AUDIO_DECODER_IN_AUDIO_BLOCK_ALIGN L"In_BlockAlign" // amf_int64 (default = 0) #define AUDIO_DECODER_IN_AUDIO_FRAME_SIZE L"In_FrameSize" // amf_int64 (default = 0) #define AUDIO_DECODER_IN_AUDIO_SEEK_POSITION L"In_SeekPosition" // amf_int64 (default = 0) #define AUDIO_DECODER_OUT_AUDIO_BIT_RATE L"Out_BitRate" // amf_int64 (default = 128000) #define AUDIO_DECODER_OUT_AUDIO_SAMPLE_RATE L"Out_SampleRate" // amf_int64 (default = 0) #define AUDIO_DECODER_OUT_AUDIO_CHANNELS L"Out_Channels" // amf_int64 (default = 2) #define AUDIO_DECODER_OUT_AUDIO_SAMPLE_FORMAT L"Out_SampleFormat" // amf_int64 (default = AMFAF_UNKNOWN) (AMF_AUDIO_FORMAT) #define AUDIO_DECODER_OUT_AUDIO_CHANNEL_LAYOUT L"Out_ChannelLayout" // amf_int64 (default = 0) #define AUDIO_DECODER_OUT_AUDIO_BLOCK_ALIGN L"Out_BlockAlign" // amf_int64 (default = 0) #endif //#ifndef AMF_AudioDecoderFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGAudioEncoder.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // AudioEncoderFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_AudioEncoderFFMPEG_h #define AMF_AudioEncoderFFMPEG_h #pragma once #define FFMPEG_AUDIO_ENCODER L"AudioEncoderFFMPEG" #define AUDIO_ENCODER_ENABLE_DEBUGGING L"EnableDebug" // bool (default = false) - trace some debug information if set to true #define AUDIO_ENCODER_ENABLE_ENCODING L"EnableEncoding" // bool (default = true) - if false, component will not encode anything #define AUDIO_ENCODER_AUDIO_CODEC_ID L"CodecID" // amf_int64 (default = AV_CODEC_ID_NONE) - FFMPEG codec ID #define AUDIO_ENCODER_IN_AUDIO_SAMPLE_RATE L"In_SampleRate" // amf_int64 (default = 44100) #define AUDIO_ENCODER_IN_AUDIO_CHANNELS L"In_Channels" // amf_int64 (default = 2) #define AUDIO_ENCODER_IN_AUDIO_SAMPLE_FORMAT L"In_SampleFormat" // amf_int64 (default = AMFAF_S16) (AMF_AUDIO_FORMAT) #define AUDIO_ENCODER_IN_AUDIO_CHANNEL_LAYOUT L"In_ChannelLayout" // amf_int64 (default = 3) #define AUDIO_ENCODER_IN_AUDIO_BLOCK_ALIGN L"In_BlockAlign" // amf_int64 (default = 0) #define AUDIO_ENCODER_OUT_AUDIO_BIT_RATE L"Out_BitRate" // amf_int64 (default = 128000) #define AUDIO_ENCODER_OUT_AUDIO_EXTRA_DATA L"Out_ExtraData" // interface to AMFBuffer #define AUDIO_ENCODER_OUT_AUDIO_SAMPLE_RATE L"Out_SampleRate" // amf_int64 (default = 44100) #define AUDIO_ENCODER_OUT_AUDIO_CHANNELS L"Out_Channels" // amf_int64 (default = 2) #define AUDIO_ENCODER_OUT_AUDIO_SAMPLE_FORMAT L"Out_SampleFormat" // amf_int64 (default = AMFAF_S16) (AMF_AUDIO_FORMAT) #define AUDIO_ENCODER_OUT_AUDIO_CHANNEL_LAYOUT L"Out_ChannelLayout" // amf_int64 (default = 0) #define AUDIO_ENCODER_OUT_AUDIO_BLOCK_ALIGN L"Out_BlockAlign" // amf_int64 (default = 0) #define AUDIO_ENCODER_OUT_AUDIO_FRAME_SIZE L"Out_FrameSize" // amf_int64 (default = 0) #endif //#ifndef AMF_AudioEncoderFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGComponents.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // FFMPEG components definitions //------------------------------------------------------------------------------------------------- #ifndef AMF_ComponentsFFMPEG_h #define AMF_ComponentsFFMPEG_h #pragma once #if defined(_WIN32) #if defined(_M_AMD64) #define FFMPEG_DLL_NAME L"amf-component-ffmpeg64.dll" #else #define FFMPEG_DLL_NAME L"amf-component-ffmpeg32.dll" #endif #elif defined(__linux) #define FFMPEG_DLL_NAME L"amf-component-ffmpeg.so" #endif #endif //#ifndef AMF_ComponentsFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGEncoderAV1.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // HEVCEncoderFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_AV1EncoderFFMPEG_h #define AMF_AV1EncoderFFMPEG_h #pragma once #define FFMPEG_ENCODER_AV1 L"AV1EncoderFFMPEG" #endif //#ifndef AMF_HEVCEncoderFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGEncoderH264.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; H264/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // H264EncoderFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_H264EncoderFFMPEG_h #define AMF_H264EncoderFFMPEG_h #pragma once #define FFMPEG_ENCODER_H264 L"H264EncoderFFMPEG" #endif //#ifndef AMF_H264EncoderFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGEncoderHEVC.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // HEVCEncoderFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_HEVCEncoderFFMPEG_h #define AMF_HEVCEncoderFFMPEG_h #pragma once #define FFMPEG_ENCODER_HEVC L"HEVCEncoderFFMPEG" #endif //#ifndef AMF_HEVCEncoderFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGFileDemuxer.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // DemuxerFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_FileDemuxerFFMPEG_h #define AMF_FileDemuxerFFMPEG_h #pragma once #define FFMPEG_DEMUXER L"DemuxerFFMPEG" // component properties #define FFMPEG_DEMUXER_PATH L"Path" // string - the file to open #define FFMPEG_DEMUXER_URL L"Url" // string - the stream url to open #define FFMPEG_DEMUXER_START_FRAME L"StartFrame" // amf_int64 (default = 0) #define FFMPEG_DEMUXER_FRAME_COUNT L"FramesNumber" // amf_int64 (default = 0) #define FFMPEG_DEMUXER_DURATION L"Duration" // amf_int64 (default = 0) #define FFMPEG_DEMUXER_CHECK_MVC L"CheckMVC" // bool (default = true) //#define FFMPEG_DEMUXER_SYNC_AV L"SyncAV" // bool (default = false) #define FFMPEG_DEMUXER_INDIVIDUAL_STREAM_MODE L"StreamMode" // bool (default = true) #define FFMPEG_DEMUXER_LISTEN L"Listen" // bool (default = false) // for common, video and audio properties see Component.h // video stream properties #define FFMPEG_DEMUXER_VIDEO_PIXEL_ASPECT_RATIO L"PixelAspectRatio" // double (default = calculated) #define FFMPEG_DEMUXER_VIDEO_CODEC L"FFmpegCodec" // enum (from source) // buffer properties #define FFMPEG_DEMUXER_BUFFER_TYPE L"BufferType" // amf_int64 ( AMF_STREAM_TYPE_ENUM ) #define FFMPEG_DEMUXER_BUFFER_STREAM_INDEX L"BufferStreamIndexType" // amf_int64 ( stream index ) #endif //#ifndef AMF_FileDemuxerFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGFileMuxer.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // MuxerFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_FileMuxerFFMPEG_h #define AMF_FileMuxerFFMPEG_h #pragma once #define FFMPEG_MUXER L"MuxerFFMPEG" // component properties #define FFMPEG_MUXER_PATH L"Path" // string - the file to open #define FFMPEG_MUXER_URL L"Url" // string - the stream url to open #define FFMPEG_MUXER_LISTEN L"Listen" // bool (default = false) #define FFMPEG_MUXER_ENABLE_VIDEO L"EnableVideo" // bool (default = true) #define FFMPEG_MUXER_ENABLE_AUDIO L"EnableAudio" // bool (default = false) #define FFMPEG_MUXER_CURRENT_TIME_INTERFACE L"CurrentTimeInterface" #define FFMPEG_MUXER_VIDEO_ROTATION L"VideoRotation" // amf_int64 (0, 90, 180, 270, default = 0) #define FFMPEG_MUXER_USAGE_IS_TRIM L"UsageIsTrim" // bool (default = false) #endif //#ifndef AMF_FileMuxerFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FFMPEGVideoDecoder.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // VideoDecoderFFMPEG interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoDecoderFFMPEG_h #define AMF_VideoDecoderFFMPEG_h #pragma once #define FFMPEG_VIDEO_DECODER L"VideoDecoderFFMPEG" #define VIDEO_DECODER_ENABLE_DECODING L"EnableDecoding" // bool (default = true) - if false, component will not decode anything #define VIDEO_DECODER_CODEC_ID L"CodecID" // amf_int64 (AMF_STREAM_CODEC_ID_ENUM) codec ID #define VIDEO_DECODER_EXTRA_DATA L"ExtraData" // interface to AMFBuffer #define VIDEO_DECODER_RESOLUTION L"Resolution" // AMFSize #define VIDEO_DECODER_BITRATE L"BitRate" // amf_int64 (default = 0) #define VIDEO_DECODER_FRAMERATE L"FrameRate" // AMFRate #define VIDEO_DECODER_SEEK_POSITION L"SeekPosition" // amf_int64 (default = 0) #define VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC L"ColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 7.2 #endif //#ifndef AMF_VideoDecoderFFMPEG_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/FRC.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMFFRC_h #define AMFFRC_h #pragma once #define AMFFRC L"AMFFRC" // Select rendering API for FRC enum AMF_FRC_ENGINE { FRC_ENGINE_OFF = 0, FRC_ENGINE_DX12 = 1, FRC_ENGINE_OPENCL = 2, }; // Select present mode for FRC enum AMF_FRC_MODE_TYPE { FRC_OFF = 0, FRC_ON, FRC_ONLY_INTERPOLATED, FRC_x2_PRESENT, TOTAL_FRC_MODES }; enum AMF_FRC_SNAPSHOT_MODE_TYPE { FRC_SNAPSHOT_OFF = 0, FRC_SNAPSHOT_LOAD, FRC_SNAPSHOT_STORE, FRC_SNAPSHOT_REGRESSION_TEST, FRC_SNAPSHOT_STORE_NO_PADDING, TOTAL_FRC_SNAPSHOT_MODES }; enum AMF_FRC_PROFILE { FRC_PROFILE_LOW = 0, FRC_PROFILE_HIGH = 1, FRC_PROFILE_SUPER = 2, TOTAL_FRC_PROFILES }; enum AMF_FRC_MV_SEARCH_MODE { FRC_MV_SEARCH_NATIVE = 0, FRC_MV_SEARCH_PERFORMANCE = 1, TOTAL_FRC_MV_SEARCH_MODES }; #define AMF_FRC_ENGINE_TYPE L"FRCEngineType" // AMF_MEMORY_TYPE (DX12, OPENCL, default : DX12)" - determines how the object is initialized and what kernels to use #define AMF_FRC_OUTPUT_SIZE L"FRCSOutputSize" // AMFSize - output scaling width/hieight #define AMF_FRC_MODE L"FRCMode" // FRC mode (0-off, 1-on (call at x2 source FPS), 2-only interpolated, 3-x2 Present) #define AMF_FRC_ENABLE_FALLBACK L"FRCEnableFallback" // FRC enable fallback mode #define AMF_FRC_INDICATOR L"FRCIndicator" // bool (default : false) #define AMF_FRC_PROFILE L"FRCProfile" // FRC profile #define AMF_FRC_MV_SEARCH_MODE L"FRCMVSEARCHMODE" // FRC MV search mode #endif //#ifndef AMFFRC_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/HQScaler.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMFHQScaler_h #define AMFHQScaler_h #pragma once #define AMFHQScaler L"AMFHQScaler" // various types of algorithms supported by the high-quality scaler enum AMF_HQ_SCALER_ALGORITHM_ENUM { AMF_HQ_SCALER_ALGORITHM_BILINEAR = 0, AMF_HQ_SCALER_ALGORITHM_BICUBIC = 1, AMF_HQ_SCALER_ALGORITHM_FSR = 2, // deprecated AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_0 = 2, AMF_HQ_SCALER_ALGORITHM_POINT = 3, AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 = 4, }; // PA object properties #define AMF_HQ_SCALER_ALGORITHM L"HQScalerAlgorithm" // amf_int64(AMF_HQ_SCALER_ALGORITHM_ENUM) (Bi-linear, Bi-cubic, RCAS, Auto)" - determines which scaling algorithm will be used // auto will chose best option between algorithms available #define AMF_HQ_SCALER_ENGINE_TYPE L"HQScalerEngineType" // AMF_MEMORY_TYPE (DX11, DX12, OPENCL, VULKAN default : DX11)" - determines how the object is initialized and what kernels to use #define AMF_HQ_SCALER_OUTPUT_SIZE L"HQSOutputSize" // AMFSize - output scaling width/hieight #define AMF_HQ_SCALER_KEEP_ASPECT_RATIO L"KeepAspectRatio" // bool (default=false) Keep aspect ratio if scaling. #define AMF_HQ_SCALER_FILL L"Fill" // bool (default=false) fill area out of ROI. #define AMF_HQ_SCALER_FILL_COLOR L"FillColor" // AMFColor #define AMF_HQ_SCALER_FROM_SRGB L"FromSRGB" // bool (default=true) Convert to SRGB. #define AMF_HQ_SCALER_SHARPNESS L"HQScalerSharpness" // Float in the range of [0.0, 2.0] #define AMF_HQ_SCALER_FRAME_RATE L"HQScalerFrameRate" // Frame rate (off, 15, 30, 60) #endif //#ifndef AMFHQScaler_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/MediaSource.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_MediaSource_h #define AMF_MediaSource_h #pragma once #include "public/include/core/Interface.h" namespace amf { enum AMF_SEEK_TYPE { AMF_SEEK_PREV = 0, // nearest packet before pts AMF_SEEK_NEXT = 1, // nearest packet after pts AMF_SEEK_PREV_KEYFRAME = 2, // nearest keyframe packet before pts AMF_SEEK_NEXT_KEYFRAME = 3, // nearest keyframe packet after pts }; //---------------------------------------------------------------------------------------------- // media source interface. //---------------------------------------------------------------------------------------------- class AMFMediaSource : public AMFInterface { public: AMF_DECLARE_IID(0xb367695a, 0xdbd0, 0x4430, 0x95, 0x3b, 0xbc, 0x7d, 0xbd, 0x2a, 0xa7, 0x66) // interface virtual AMF_RESULT AMF_STD_CALL Seek(amf_pts pos, AMF_SEEK_TYPE seekType, amf_int32 whichStream) = 0; virtual amf_pts AMF_STD_CALL GetPosition() = 0; virtual amf_pts AMF_STD_CALL GetDuration() = 0; virtual void AMF_STD_CALL SetMinPosition(amf_pts pts) = 0; virtual amf_pts AMF_STD_CALL GetMinPosition() = 0; virtual void AMF_STD_CALL SetMaxPosition(amf_pts pts) = 0; virtual amf_pts AMF_STD_CALL GetMaxPosition() = 0; virtual amf_uint64 AMF_STD_CALL GetFrameFromPts(amf_pts pts) = 0; virtual amf_pts AMF_STD_CALL GetPtsFromFrame(amf_uint64 frame) = 0; virtual bool AMF_STD_CALL SupportFramesAccess() = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFMediaSourcePtr; } //namespace amf #endif //#ifndef AMF_MediaSource_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/PreAnalysis.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2019 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMFPreAnalysis_h #define AMFPreAnalysis_h #pragma once #define AMFPreAnalysis L"AMFPreAnalysis" enum AMF_PA_SCENE_CHANGE_DETECTION_SENSITIVITY_ENUM { AMF_PA_SCENE_CHANGE_DETECTION_SENSITIVITY_LOW = 0, AMF_PA_SCENE_CHANGE_DETECTION_SENSITIVITY_MEDIUM = 1, AMF_PA_SCENE_CHANGE_DETECTION_SENSITIVITY_HIGH = 2 }; enum AMF_PA_STATIC_SCENE_DETECTION_SENSITIVITY_ENUM { AMF_PA_STATIC_SCENE_DETECTION_SENSITIVITY_LOW = 0, AMF_PA_STATIC_SCENE_DETECTION_SENSITIVITY_MEDIUM = 1, AMF_PA_STATIC_SCENE_DETECTION_SENSITIVITY_HIGH = 2 }; enum AMF_PA_ACTIVITY_TYPE_ENUM { AMF_PA_ACTIVITY_Y = 0, AMF_PA_ACTIVITY_YUV = 1 }; enum AMF_PA_CAQ_STRENGTH_ENUM { AMF_PA_CAQ_STRENGTH_LOW = 0, AMF_PA_CAQ_STRENGTH_MEDIUM = 1, AMF_PA_CAQ_STRENGTH_HIGH = 2 }; // Perceptual adaptive quantization mode enum AMF_PA_PAQ_MODE_ENUM { AMF_PA_PAQ_MODE_NONE = 0, AMF_PA_PAQ_MODE_CAQ = 1 }; // Temporal adaptive quantization mode enum AMF_PA_TAQ_MODE_ENUM { AMF_PA_TAQ_MODE_NONE = 0, AMF_PA_TAQ_MODE_1 = 1, AMF_PA_TAQ_MODE_2 = 2 }; enum AMF_PA_HIGH_MOTION_QUALITY_BOOST_MODE_ENUM { AMF_PA_HIGH_MOTION_QUALITY_BOOST_MODE_NONE = 0, //default AMF_PA_HIGH_MOTION_QUALITY_BOOST_MODE_AUTO = 1 }; // PA object properties #define AMF_PA_ENGINE_TYPE L"PAEngineType" // AMF_MEMORY_TYPE (Host, DX11, OpenCL, Vulkan, Auto default : UNKNOWN (Auto))" - determines how the object is initialized and what kernels to use // by default it is Auto (DX11, OpenCL and Vulkan are currently available) #define AMF_PA_SCENE_CHANGE_DETECTION_ENABLE L"PASceneChangeDetectionEnable" // bool (default : True) - Enable Scene Change Detection GPU algorithm #define AMF_PA_SCENE_CHANGE_DETECTION_SENSITIVITY L"PASceneChangeDetectionSensitivity" // AMF_PA_SCENE_CHANGE_DETECTION_SENSITIVITY_ENUM (default : Medium) - Scene Change Detection Sensitivity #define AMF_PA_STATIC_SCENE_DETECTION_ENABLE L"PAStaticSceneDetectionEnable" // bool (default : False) - Enable Skip Detection GPU algorithm #define AMF_PA_STATIC_SCENE_DETECTION_SENSITIVITY L"PAStaticSceneDetectionSensitivity" // AMF_PA_STATIC_SCENE_DETECTION_SENSITIVITY_ENUM (default : High) - Allowable absolute difference between pixels (sample counts) #define AMF_PA_FRAME_SAD_ENABLE L"PAFrameSadEnable" // bool (default : True) - Enable Frame SAD algorithm #define AMF_PA_ACTIVITY_TYPE L"PAActivityType" // AMF_PA_ACTIVITY_TYPE_ENUM (default : Calculate on Y) - Block activity calculation mode #define AMF_PA_LTR_ENABLE L"PALongTermReferenceEnable" // bool (default : False) - Enable Automatic Long Term Reference frame management #define AMF_PA_LOOKAHEAD_BUFFER_DEPTH L"PALookAheadBufferDepth" // amf_uint64 (default : 0) Values: [0, MAX_LOOKAHEAD_DEPTH] - PA lookahead buffer size #define AMF_PA_PAQ_MODE L"PAPerceptualAQMode" // AMF_PA_PAQ_MODE_ENUM (default : AMF_PA_PAQ_MODE_NONE) - Perceptual AQ mode #define AMF_PA_TAQ_MODE L"PATemporalAQMode" // AMF_PA_TAQ_MODE_ENUM (default: AMF_PA_TAQ_MODE_NONE) - Temporal AQ mode #define AMF_PA_HIGH_MOTION_QUALITY_BOOST_MODE L"PAHighMotionQualityBoostMode" // AMF_PA_HIGH_MOTION_QUALITY_BOOST_MODE_ENUM (default: None) - High motion quality boost mode /////////////////////////////////////////// // the following properties are available // only through the Encoder - trying to // access/set them when PA is standalone // will fail #define AMF_PA_INITIAL_QP_AFTER_SCENE_CHANGE L"PAInitialQPAfterSceneChange" // amf_uint64 (default : 0) Values: [0, 51] - Base QP to be used immediately after scene change. If this value is not set, PA will choose a proper QP value #define AMF_PA_MAX_QP_BEFORE_FORCE_SKIP L"PAMaxQPBeforeForceSkip" // amf_uint64 (default : 35) Values: [0, 51] - When a static scene is detected, a skip frame is inserted only if the previous encoded frame average QP <= this value #define AMF_PA_CAQ_STRENGTH L"PACAQStrength" // AMF_PA_CAQ_STRENGTH_ENUM (default : Medium) - Content Adaptive Quantization (CAQ) strength ////////////////////////////////////////////////// // properties set by PA on output buffer interface in standalone mode #define AMF_PA_ACTIVITY_MAP L"PAActivityMap" // AMFInterface* -> AMFSurface*; Values: int32 - When PA is standalone, there will be a 2D Activity map generated for each frame #define AMF_PA_SCENE_CHANGE_DETECT L"PASceneChangeDetect" // bool - True/False - available if AMF_PA_SCENE_CHANGE_DETECTION_ENABLE was set to True when PA is standalone #define AMF_PA_STATIC_SCENE_DETECT L"PAStaticSceneDetect" // bool - True/False - available if AMF_PA_STATIC_SCENE_DETECTION_ENABLE was set to True when PA is standalone #endif //#ifndef AMFPreAnalysis_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/PreProcessing.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2020 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMFPreProcessing_h #define AMFPreProcessing_h #pragma once #define AMFPreProcessing L"AMFPreProcessing" // Pre-processing object properties #define AMF_PP_ENGINE_TYPE L"PPEngineType" // AMF_MEMORY_TYPE (Host, DX11, OPENCL, Auto default : OPENCL) - determines how the object is initialized and what kernels to use // by default it is OpenCL (Host, DX11 and OpenCL are currently available) // add a property that will determine the output format // by default we output in the same format as input // but in some cases we might need to change the output // format to be different than input #define AMF_PP_OUTPUT_MEMORY_TYPE L"PPOutputFormat" // AMF_MEMORY_TYPE (Host, DX11, OPENCL default : Unknown) - determines format of frame going out #define AMF_PP_ADAPTIVE_FILTER_STRENGTH L"PPAdaptiveFilterStrength" // int (default : 4) - strength: 0 - 10: the higher the value, the stronger the filtering #define AMF_PP_ADAPTIVE_FILTER_SENSITIVITY L"PPAdaptiveFilterSensitivity" // int (default : 4) - sensitivity: 0 - 10: the lower the value, the more sensitive to edge (preserve more details) // Encoder parameters used for adaptive filtering #define AMF_PP_TARGET_BITRATE L"PPTargetBitrate" // int64 (default: 2000000) - target bit rate #define AMF_PP_FRAME_RATE L"PPFrameRate" // AMFRate (default: 30, 1) - frame rate #define AMF_PP_ADAPTIVE_FILTER_ENABLE L"PPAdaptiveFilterEnable" // bool (default: false) - turn on/off adaptive filtering #endif //#ifndef AMFPreProcessing_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/SupportedCodecs.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // An interface available on some components to provide information on supported input and output codecs //------------------------------------------------------------------------------------------------- #ifndef AMF_SupportedCodecs_h #define AMF_SupportedCodecs_h #pragma once #include "public/include/core/Interface.h" //properties on the returned AMFPropertyStorage #define SUPPORTEDCODEC_ID L"CodecId" //amf_int64 #define SUPPORTEDCODEC_SAMPLERATE L"SampleRate" //amf_int32 namespace amf { class AMFSupportedCodecs : public AMFInterface { public: AMF_DECLARE_IID(0xc1003a83, 0x7934, 0x408a, 0x95, 0x5b, 0xc4, 0xdd, 0x85, 0x9d, 0xf5, 0x61) //call with increasing values until it returns AMF_OUT_OF_RANGE virtual AMF_RESULT AMF_STD_CALL GetInputCodecAt(amf_size index, AMFPropertyStorage** codec) const = 0; virtual AMF_RESULT AMF_STD_CALL GetOutputCodecAt(amf_size index, AMFPropertyStorage** codec) const = 0; }; typedef AMFInterfacePtr_T AMFSupportedCodecsPtr; } #endif ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VQEnhancer.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMFVQEnhancer_h #define AMFVQEnhancer_h #pragma once #define VE_FCR_DEFAULT_ATTENUATION 0.1 #define AMFVQEnhancer L"AMFVQEnhancer" #define AMF_VIDEO_ENHANCER_ENGINE_TYPE L"AMF_VIDEI_ENHANCER_ENGINE_TYPE" // AMF_MEMORY_TYPE (DX11, DX12, OPENCL, VULKAN default : DX11)" - determines how the object is initialized and what kernels to use #define AMF_VIDEO_ENHANCER_OUTPUT_SIZE L"AMF_VIDEO_ENHANCER_OUTPUT_SIZE" // AMFSize #define AMF_VE_FCR_ATTENUATION L"AMF_VE_FCR_ATTENUATION" // Float in the range of [0.02, 0.4], default : 0.1 #define AMF_VE_FCR_RADIUS L"AMF_VE_FCR_RADIUS" // int in the range of [1, 4] #define AMF_VE_FCR_SPLIT_VIEW L"AMF_VE_FCR_SPLIT_VIEW" // FCR View split window #endif //#ifndef AMFVQEnhancer_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoCapture.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // ZCamLive interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoCapture_h #define AMF_VideoCapture_h #pragma once #define VIDEOCAP_DEVICE_COUNT L"VideoCapDeviceCount" // amf_int64, (default=2), number of video capture devices #define VIDEOCAP_DEVICE_NAME L"VideoCapDeviceName" // WString, (default=""), name of the video capture device #define VIDEOCAP_DEVICE_ACTIVE L"VideoCapDeviceActive" // WString, (default=""), name of the selected video capture device #define VIDEOCAP_CODEC L"CodecID" // WString (default = "AMFVideoDecoderUVD_H264_AVC"), UVD codec ID #define VIDEOCAP_FRAMESIZE L"FrameSize" // AMFSize, (default=AMFConstructSize(1920, 1080)), frame size in pixels extern "C" { AMF_RESULT AMF_CDECL_CALL AMFCreateComponentVideoCapture(amf::AMFContext* pContext, amf::AMFComponentEx** ppComponent); } #endif // AMF_VideoCapture_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoConverter.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // AMFFVideoConverter interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoConverter_h #define AMF_VideoConverter_h #pragma once #include "Component.h" #include "ColorSpace.h" #define AMFVideoConverter L"AMFVideoConverter" enum AMF_VIDEO_CONVERTER_SCALE_ENUM { AMF_VIDEO_CONVERTER_SCALE_INVALID = -1, AMF_VIDEO_CONVERTER_SCALE_BILINEAR = 0, AMF_VIDEO_CONVERTER_SCALE_BICUBIC = 1 }; enum AMF_VIDEO_CONVERTER_TONEMAPPING_ENUM { AMF_VIDEO_CONVERTER_TONEMAPPING_COPY = 0, AMF_VIDEO_CONVERTER_TONEMAPPING_AMD = 1, AMF_VIDEO_CONVERTER_TONEMAPPING_LINEAR = 2, AMF_VIDEO_CONVERTER_TONEMAPPING_GAMMA = 3, AMF_VIDEO_CONVERTER_TONEMAPPING_REINHARD = 4, AMF_VIDEO_CONVERTER_TONEMAPPING_2390 = 5, }; #define AMF_VIDEO_CONVERTER_OUTPUT_FORMAT L"OutputFormat" // Values : AMF_SURFACE_NV12 or AMF_SURFACE_BGRA or AMF_SURFACE_YUV420P #define AMF_VIDEO_CONVERTER_MEMORY_TYPE L"MemoryType" // Values : AMF_MEMORY_DX11 or AMF_MEMORY_DX9 or AMF_MEMORY_UNKNOWN (get from input type) #define AMF_VIDEO_CONVERTER_COMPUTE_DEVICE L"ComputeDevice" // Values : AMF_MEMORY_COMPUTE_FOR_DX9 enumeration #define AMF_VIDEO_CONVERTER_OUTPUT_SIZE L"OutputSize" // AMFSize (default=0,0) width in pixels. default means no scaling #define AMF_VIDEO_CONVERTER_OUTPUT_RECT L"OutputRect" // AMFRect (default=0, 0, 0, 0) rectangle in pixels. default means no rect #define AMF_VIDEO_CONVERTER_SCALE L"ScaleType" // amf_int64(AMF_VIDEO_CONVERTER_SCALE_ENUM); default = AMF_VIDEO_CONVERTER_SCALE_BILINEAR #define AMF_VIDEO_CONVERTER_FORCE_OUTPUT_SURFACE_SIZE L"ForceOutputSurfaceSize" // bool (default=false) Force output size from output surface #define AMF_VIDEO_CONVERTER_KEEP_ASPECT_RATIO L"KeepAspectRatio" // bool (default=false) Keep aspect ratio if scaling. #define AMF_VIDEO_CONVERTER_FILL L"Fill" // bool (default=false) fill area out of ROI. #define AMF_VIDEO_CONVERTER_FILL_COLOR L"FillColor" // AMFColor //------------------------------------------------------------------------------------------------- // SDR color conversion //------------------------------------------------------------------------------------------------- #define AMF_VIDEO_CONVERTER_COLOR_PROFILE L"ColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO #define AMF_VIDEO_CONVERTER_LINEAR_RGB L"LinearRGB" // bool (default=false) Convert to/from linear RGB instead of sRGB using AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC or by default AMF_VIDEO_CONVERTER_TRANSFER_CHARACTERISTIC //------------------------------------------------------------------------------------------------- // HDR color conversion //------------------------------------------------------------------------------------------------- // AMF_VIDEO_CONVERTER_COLOR_PROFILE is used to define color space conversion // HDR data - can be set on converter or respectively on input and output surfaces (output surface via custom allocator) // if present, HDR_METADATA primary color overwrites COLOR_PRIMARIES // these properties can be set on converter component to configure input and output // these properties overwrite properties set on surface - see below #define AMF_VIDEO_CONVERTER_INPUT_TRANSFER_CHARACTERISTIC L"InputTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 7.2 See ColorSpace.h for enum #define AMF_VIDEO_CONVERTER_INPUT_COLOR_PRIMARIES L"InputColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 7.1 See ColorSpace.h for enum #define AMF_VIDEO_CONVERTER_INPUT_COLOR_RANGE L"InputColorRange" // amf_int64(AMF_COLOR_RANGE_ENUM) default = AMF_COLOR_RANGE_UNDEFINED #define AMF_VIDEO_CONVERTER_INPUT_HDR_METADATA L"InputHdrMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL #define AMF_VIDEO_CONVERTER_INPUT_TONEMAPPING L"InputTonemapping" // amf_int64(AMF_VIDEO_CONVERTER_TONEMAPPING_ENUM) default = AMF_VIDEO_CONVERTER_TONEMAPPING_LINEAR #define AMF_VIDEO_CONVERTER_OUTPUT_TRANSFER_CHARACTERISTIC L"OutputTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 7.2 See ColorSpace.h for enum #define AMF_VIDEO_CONVERTER_OUTPUT_COLOR_PRIMARIES L"OutputColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 7.1 See ColorSpace.h for enum #define AMF_VIDEO_CONVERTER_OUTPUT_COLOR_RANGE L"OutputColorRange" // amf_int64(AMF_COLOR_RANGE_ENUM) default = AMF_COLOR_RANGE_UNDEFINED #define AMF_VIDEO_CONVERTER_OUTPUT_HDR_METADATA L"OutputHdrMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL #define AMF_VIDEO_CONVERTER_OUTPUT_TONEMAPPING L"OutputTonemapping" // amf_int64(AMF_VIDEO_CONVERTER_TONEMAPPING_ENUM) default = AMF_VIDEO_CONVERTER_TONEMAPPING_AMD // these properties can be set on input or outout surface See ColorSpace.h // the same as decoder properties set on input surface - see below //#define AMF_VIDEO_COLOR_TRANSFER_CHARACTERISTIC L"ColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 7.2 See ColorSpace.h for enum //#define AMF_VIDEO_COLOR_PRIMARIES L"ColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 7.1 See ColorSpace.h for enum //#define AMF_VIDEO_COLOR_RANGE L"ColorRange" // amf_int64(AMF_COLOR_RANGE_ENUM) default = AMF_COLOR_RANGE_UNDEFINED //#define AMF_VIDEO_COLOR_HDR_METADATA L"HdrMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL // If decoder properties can be set on input see VideoDecoder.h // AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC // AMF_VIDEO_DECODER_COLOR_PRIMARIES // AMF_VIDEO_DECODER_COLOR_RANGE // AMF_VIDEO_DECODER_HDR_METADATA #define AMF_VIDEO_CONVERTER_USE_DECODER_HDR_METADATA L"UseDecoderHDRMetadata" // bool (default=true) enables use of decoder / surface input color properties above #endif //#ifndef AMF_VideoConverter_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoDecoderUVD.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // VideoDecoderUVD interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoDecoderUVD_h #define AMF_VideoDecoderUVD_h #pragma once #include "Component.h" #include "ColorSpace.h" #define AMFVideoDecoderUVD_MPEG2 L"AMFVideoDecoderUVD_MPEG2" #define AMFVideoDecoderUVD_MPEG4 L"AMFVideoDecoderUVD_MPEG4" #define AMFVideoDecoderUVD_WMV3 L"AMFVideoDecoderUVD_WMV3" #define AMFVideoDecoderUVD_VC1 L"AMFVideoDecoderUVD_VC1" #define AMFVideoDecoderUVD_H264_AVC L"AMFVideoDecoderUVD_H264_AVC" #define AMFVideoDecoderUVD_H264_MVC L"AMFVideoDecoderUVD_H264_MVC" #define AMFVideoDecoderUVD_H264_SVC L"AMFVideoDecoderUVD_H264_SVC" #define AMFVideoDecoderUVD_MJPEG L"AMFVideoDecoderUVD_MJPEG" #define AMFVideoDecoderHW_H265_HEVC L"AMFVideoDecoderHW_H265_HEVC" #define AMFVideoDecoderHW_H265_MAIN10 L"AMFVideoDecoderHW_H265_MAIN10" #define AMFVideoDecoderHW_VP9 L"AMFVideoDecoderHW_VP9" #define AMFVideoDecoderHW_VP9_10BIT L"AMFVideoDecoderHW_VP9_10BIT" #define AMFVideoDecoderHW_AV1 L"AMFVideoDecoderHW_AV1" #define AMFVideoDecoderHW_AV1_12BIT L"AMFVideoDecoderHW_AV1_12BIT" enum AMF_VIDEO_DECODER_MODE_ENUM { AMF_VIDEO_DECODER_MODE_REGULAR = 0, // DPB delay is based on number of reference frames + 1 (from SPS) AMF_VIDEO_DECODER_MODE_COMPLIANT, // DPB delay is based on profile - up to 16 AMF_VIDEO_DECODER_MODE_LOW_LATENCY, // DPB delay is 0. Expect stream with no reordering in P-Frames or B-Frames. B-frames can be present as long as they do not introduce any frame re-ordering }; enum AMF_TIMESTAMP_MODE_ENUM { AMF_TS_PRESENTATION = 0, // default. decoder will preserve timestamps from input to output AMF_TS_SORT, // decoder will resort PTS list AMF_TS_DECODE // timestamps reflect decode order - decoder will reuse them }; #define AMF_VIDEO_DECODER_SURFACE_COPY L"SurfaceCopy" // amf_bool; default = false; return output surfaces as a copy #define AMF_VIDEO_DECODER_EXTRADATA L"ExtraData" // AMFInterface* -> AMFBuffer* - AVCC - size length + SPS/PPS; or as Annex B. Optional if stream is Annex B #define AMF_VIDEO_DECODER_FRAME_RATE L"FrameRate" // amf_double; default = 0.0, optional property to restore duration in the output if needed #define AMF_TIMESTAMP_MODE L"TimestampMode" // amf_int64(AMF_TIMESTAMP_MODE_ENUM) - default AMF_TS_PRESENTATION - how input timestamps are treated // dynamic/adaptive resolution change #define AMF_VIDEO_DECODER_ADAPTIVE_RESOLUTION_CHANGE L"AdaptiveResolutionChange" // amf_bool; default = false; reuse allocated surfaces if new resolution is smaller #define AMF_VIDEO_DECODER_ALLOC_SIZE L"AllocSize" // AMFSize; default (1920,1088); size of allocated surface if AdaptiveResolutionChange is true #define AMF_VIDEO_DECODER_CURRENT_SIZE L"CurrentSize" // AMFSize; default = (0,0); current size of the video // reference frame management #define AMF_VIDEO_DECODER_REORDER_MODE L"ReorderMode" // amf_int64(AMF_VIDEO_DECODER_MODE_ENUM); default = AMF_VIDEO_DECODER_MODE_REGULAR; defines number of surfaces in DPB list. #define AMF_VIDEO_DECODER_SURFACE_POOL_SIZE L"SurfacePoolSize" // amf_int64; number of surfaces in the decode pool = DPB list size + number of surfaces for presentation #define AMF_VIDEO_DECODER_DPB_SIZE L"DPBSize" // amf_int64; minimum number of surfaces for reordering #define AMF_VIDEO_DECODER_DEFAULT_SURFACES_FOR_TRANSIT 5 // if AMF_VIDEO_DECODER_SURFACE_POOL_SIZE is 0 , AMF_VIDEO_DECODER_SURFACE_POOL_SIZE=AMF_VIDEO_DECODER_DEFAULT_SURFACES_FOR_TRANSIT+AMF_VIDEO_DECODER_DPB_SIZE // Decoder capabilities - exposed in AMFCaps interface #define AMF_VIDEO_DECODER_CAP_NUM_OF_STREAMS L"NumOfStreams" // amf_int64; maximum number of decode streams supported // metadata information: can be set on output surface // Properties could be set on surface based on HDR SEI or VUI header #define AMF_VIDEO_DECODER_COLOR_TRANSFER_CHARACTERISTIC L"ColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 7.2 #define AMF_VIDEO_DECODER_COLOR_PRIMARIES L"ColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 7.1 #define AMF_VIDEO_DECODER_HDR_METADATA L"HdrMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL /////// AMF_VIDEO_DECODER_FULL_RANGE_COLOR deprecated, use AMF_VIDEO_DECODER_COLOR_RANGE #define AMF_VIDEO_DECODER_FULL_RANGE_COLOR L"FullRangeColor" // bool; default = false; false = studio range, true = full range /////// #define AMF_VIDEO_DECODER_COLOR_RANGE L"ColorRange" // amf_int64(AMF_COLOR_RANGE_ENUM) default = AMF_COLOR_RANGE_UNDEFINED // can be set on output surface if YUV outout or on component to overwrite VUI #define AMF_VIDEO_DECODER_COLOR_PROFILE L"ColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO // properties to be set on decoder if internal converter is used #define AMF_VIDEO_DECODER_OUTPUT_TRANSFER_CHARACTERISTIC L"OutColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_DECODER_OUTPUT_COLOR_PRIMARIES L"OutputColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 7.1 See ColorSpace.h for enum #define AMF_VIDEO_DECODER_OUTPUT_HDR_METADATA L"OutHDRMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL #define AMF_VIDEO_DECODER_LOW_LATENCY L"LowLatencyDecode" // amf_bool; default = false; true = low latency decode, false = regular decode #if defined(__ANDROID__) #define AMF_VIDEO_DECODER_NATIVEWINDOW L"AndroidNativeWindow" // amf_int64; default = 0; pointer to native window #endif //__ANDROID__ #if defined(__APPLE__) #define AMF_VIDEO_DECODER_NATIVEWINDOW L"AppleNativeWindow" // amf_int64; default = 0; pointer to native window #endif //__APPLE__ #define AMF_VIDEO_DECODER_ENABLE_SMART_ACCESS_VIDEO L"EnableDecoderSmartAccessVideo" // amf_bool; default = false; true = enables smart access video feature #define AMF_VIDEO_DECODER_SKIP_TRANSFER_SMART_ACCESS_VIDEO L"SkipTransferSmartAccessVideo" // amf_bool; default = false; true = keeps output on GPU where it ran #define AMF_VIDEO_DECODER_CAP_SUPPORT_SMART_ACCESS_VIDEO L"SupportSmartAccessVideo" // amf_bool; returns true if system supports SmartAccess Video #define AMF_VIDEO_DECODER_SURFACE_CPU L"SurfaceCpu" // amf_bool. default = false, true = hint to decoder that output will be consumed on cpu #define AMF_VIDEO_DECODER_INSTANCE_INDEX L"DecoderInstance" // amf_int64; selected HW instance idx #define AMF_VIDEO_DECODER_CAP_NUM_OF_HW_INSTANCES L"NumOfHwDecoderInstances" // amf_int64 number of HW decoder instances #endif //#ifndef AMF_VideoDecoderUVD_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoEncoderAV1.h ================================================ // // Copyright (c) 2021-2022 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // VideoEncoderHW_AV1 interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoEncoderAV1_h #define AMF_VideoEncoderAV1_h #pragma once #include "Component.h" #include "ColorSpace.h" #include "PreAnalysis.h" #define AMFVideoEncoder_AV1 L"AMFVideoEncoderHW_AV1" enum AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_NONE = 0, // No encoding latency requirement. Encoder will balance encoding time and power consumption. AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_POWER_SAVING_REAL_TIME = 1, // Try the best to finish encoding a frame within 1/framerate sec. This mode may cause more power consumption AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_REAL_TIME = 2, // Try the best to finish encoding a frame within 1/(2 x framerate) sec. This mode will cause more power consumption than POWER_SAVING_REAL_TIME AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_LOWEST_LATENCY = 3 // Encoding as fast as possible. This mode causes highest power consumption. }; enum AMF_VIDEO_ENCODER_AV1_USAGE_ENUM { AMF_VIDEO_ENCODER_AV1_USAGE_TRANSCODING = 0, AMF_VIDEO_ENCODER_AV1_USAGE_ULTRA_LOW_LATENCY = 2, AMF_VIDEO_ENCODER_AV1_USAGE_LOW_LATENCY = 1, AMF_VIDEO_ENCODER_AV1_USAGE_WEBCAM = 3, AMF_VIDEO_ENCODER_AV1_USAGE_HIGH_QUALITY = 4, AMF_VIDEO_ENCODER_AV1_USAGE_LOW_LATENCY_HIGH_QUALITY = 5 }; enum AMF_VIDEO_ENCODER_AV1_PROFILE_ENUM { AMF_VIDEO_ENCODER_AV1_PROFILE_MAIN = 1 }; enum AMF_VIDEO_ENCODER_AV1_LEVEL_ENUM { AMF_VIDEO_ENCODER_AV1_LEVEL_2_0 = 0, AMF_VIDEO_ENCODER_AV1_LEVEL_2_1 = 1, AMF_VIDEO_ENCODER_AV1_LEVEL_2_2 = 2, AMF_VIDEO_ENCODER_AV1_LEVEL_2_3 = 3, AMF_VIDEO_ENCODER_AV1_LEVEL_3_0 = 4, AMF_VIDEO_ENCODER_AV1_LEVEL_3_1 = 5, AMF_VIDEO_ENCODER_AV1_LEVEL_3_2 = 6, AMF_VIDEO_ENCODER_AV1_LEVEL_3_3 = 7, AMF_VIDEO_ENCODER_AV1_LEVEL_4_0 = 8, AMF_VIDEO_ENCODER_AV1_LEVEL_4_1 = 9, AMF_VIDEO_ENCODER_AV1_LEVEL_4_2 = 10, AMF_VIDEO_ENCODER_AV1_LEVEL_4_3 = 11, AMF_VIDEO_ENCODER_AV1_LEVEL_5_0 = 12, AMF_VIDEO_ENCODER_AV1_LEVEL_5_1 = 13, AMF_VIDEO_ENCODER_AV1_LEVEL_5_2 = 14, AMF_VIDEO_ENCODER_AV1_LEVEL_5_3 = 15, AMF_VIDEO_ENCODER_AV1_LEVEL_6_0 = 16, AMF_VIDEO_ENCODER_AV1_LEVEL_6_1 = 17, AMF_VIDEO_ENCODER_AV1_LEVEL_6_2 = 18, AMF_VIDEO_ENCODER_AV1_LEVEL_6_3 = 19, AMF_VIDEO_ENCODER_AV1_LEVEL_7_0 = 20, AMF_VIDEO_ENCODER_AV1_LEVEL_7_1 = 21, AMF_VIDEO_ENCODER_AV1_LEVEL_7_2 = 22, AMF_VIDEO_ENCODER_AV1_LEVEL_7_3 = 23 }; enum AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_ENUM { AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_UNKNOWN = -1, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_CONSTANT_QP = 0, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR = 1, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_PEAK_CONSTRAINED_VBR = 2, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_CBR = 3, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_QUALITY_VBR = 4, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_HIGH_QUALITY_VBR = 5, AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_HIGH_QUALITY_CBR = 6 }; enum AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_64X16_ONLY = 1, AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_64X16_1080P_CODED_1082 = 2, AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_NO_RESTRICTIONS = 3 }; enum AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_ENUM { AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_NONE = 0, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_KEY = 1, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_INTRA_ONLY = 2, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_SWITCH = 3, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_SHOW_EXISTING = 4 }; enum AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_ENUM { AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_KEY = 0, AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_INTRA_ONLY = 1, AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_INTER = 2, AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_SWITCH = 3, AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_SHOW_EXISTING = 4 }; enum AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_ENUM { AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_HIGH_QUALITY = 0, AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_QUALITY = 30, AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_BALANCED = 70, AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_SPEED = 100 }; enum AMF_VIDEO_ENCODER_AV1_HEADER_INSERTION_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_HEADER_INSERTION_MODE_NONE = 0, AMF_VIDEO_ENCODER_AV1_HEADER_INSERTION_MODE_GOP_ALIGNED = 1, AMF_VIDEO_ENCODER_AV1_HEADER_INSERTION_MODE_KEY_FRAME_ALIGNED = 2 }; enum AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE_NONE = 0, AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE_FIXED_INTERVAL = 1 }; enum AMF_VIDEO_ENCODER_AV1_CDEF_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_CDEF_DISABLE = 0, AMF_VIDEO_ENCODER_AV1_CDEF_ENABLE_DEFAULT = 1 }; enum AMF_VIDEO_ENCODER_AV1_CDF_FRAME_END_UPDATE_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_CDF_FRAME_END_UPDATE_MODE_DISABLE = 0, AMF_VIDEO_ENCODER_AV1_CDF_FRAME_END_UPDATE_MODE_ENABLE_DEFAULT = 1 }; enum AMF_VIDEO_ENCODER_AV1_AQ_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_AQ_MODE_NONE = 0, AMF_VIDEO_ENCODER_AV1_AQ_MODE_CAQ = 1 // Content adaptive quantization mode }; enum AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE__DISABLED = 0, AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE__GOP_ALIGNED = 1, AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE__CONTINUOUS = 2 }; enum AMF_VIDEO_ENCODER_AV1_LTR_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_LTR_MODE_RESET_UNUSED = 0, AMF_VIDEO_ENCODER_AV1_LTR_MODE_KEEP_UNUSED = 1 }; enum AMF_VIDEO_ENCODER_AV1_OUTPUT_MODE_ENUM { AMF_VIDEO_ENCODER_AV1_OUTPUT_MODE_FRAME = 0, AMF_VIDEO_ENCODER_AV1_OUTPUT_MODE_TILE = 1 }; enum AMF_VIDEO_ENCODER_AV1_OUTPUT_BUFFER_TYPE_ENUM { AMF_VIDEO_ENCODER_AV1_OUTPUT_BUFFER_TYPE_FRAME = 0, AMF_VIDEO_ENCODER_AV1_OUTPUT_BUFFER_TYPE_TILE = 1, AMF_VIDEO_ENCODER_AV1_OUTPUT_BUFFER_TYPE_TILE_LAST = 2 }; // *** Static properties - can be set only before Init() *** // Encoder Engine Settings #define AMF_VIDEO_ENCODER_AV1_ENCODER_INSTANCE_INDEX L"Av1EncoderInstanceIndex" // amf_int64; default = 0; selected HW instance idx. The number of instances is queried by using AMF_VIDEO_ENCODER_AV1_CAP_NUM_OF_HW_INSTANCES #define AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE L"Av1EncodingLatencyMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_ENUM); default = depends on USAGE; The encoding latency mode. #define AMF_VIDEO_ENCODER_AV1_QUERY_TIMEOUT L"Av1QueryTimeout" // amf_int64; default = 0 (no wait); timeout for QueryOutput call in ms. // Usage Settings #define AMF_VIDEO_ENCODER_AV1_USAGE L"Av1Usage" // amf_int64(AMF_VIDEO_ENCODER_AV1_USAGE_ENUM); default = N/A; Encoder usage. fully configures parameter set. // Session Configuration #define AMF_VIDEO_ENCODER_AV1_FRAMESIZE L"Av1FrameSize" // AMFSize; default = 0,0; Frame size #define AMF_VIDEO_ENCODER_AV1_COLOR_BIT_DEPTH L"Av1ColorBitDepth" // amf_int64(AMF_COLOR_BIT_DEPTH_ENUM); default = AMF_COLOR_BIT_DEPTH_8 #define AMF_VIDEO_ENCODER_AV1_PROFILE L"Av1Profile" // amf_int64(AMF_VIDEO_ENCODER_AV1_PROFILE_ENUM) ; default = depends on USAGE; the codec profile of the coded bitstream #define AMF_VIDEO_ENCODER_AV1_LEVEL L"Av1Level" // amf_int64 (AMF_VIDEO_ENCODER_AV1_LEVEL_ENUM); default = depends on USAGE; the codec level of the coded bitstream #define AMF_VIDEO_ENCODER_AV1_TILES_PER_FRAME L"Av1NumTilesPerFrame" // amf_int64; default = 1; Number of tiles Per Frame. This is treated as suggestion. The actual number of tiles might be different due to compliance or encoder limitation. #define AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET L"Av1QualityPreset" // amf_int64(AMF_VIDEO_ENCODER_AV1_QUALITY_PRESET_ENUM); default = depends on USAGE; Quality Preset // Codec Configuration #define AMF_VIDEO_ENCODER_AV1_SCREEN_CONTENT_TOOLS L"Av1ScreenContentTools" // bool; default = depends on USAGE; If true, allow enabling screen content tools by AMF_VIDEO_ENCODER_AV1_PALETTE_MODE and AMF_VIDEO_ENCODER_AV1_FORCE_INTEGER_MV; if false, all screen content tools are disabled. #define AMF_VIDEO_ENCODER_AV1_ORDER_HINT L"Av1OrderHint" // bool; default = depends on USAGE; If true, code order hint; if false, don't code order hint #define AMF_VIDEO_ENCODER_AV1_FRAME_ID L"Av1FrameId" // bool; default = depends on USAGE; If true, code frame id; if false, don't code frame id #define AMF_VIDEO_ENCODER_AV1_TILE_GROUP_OBU L"Av1TileGroupObu" // bool; default = depends on USAGE; If true, code FrameHeaderObu + TileGroupObu and each TileGroupObu contains one tile; if false, code FrameObu. #define AMF_VIDEO_ENCODER_AV1_CDEF_MODE L"Av1CdefMode" // amd_int64(AMF_VIDEO_ENCODER_AV1_CDEF_MODE_ENUM); default = depends on USAGE; Cdef mode #define AMF_VIDEO_ENCODER_AV1_ERROR_RESILIENT_MODE L"Av1ErrorResilientMode" // bool; default = depends on USAGE; If true, enable error resilient mode; if false, disable error resilient mode // Rate Control and Quality Enhancement #define AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD L"Av1RateControlMethod" // amf_int64(AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_METHOD_ENUM); default = depends on USAGE; Rate Control Method #define AMF_VIDEO_ENCODER_AV1_QVBR_QUALITY_LEVEL L"Av1QvbrQualityLevel" // amf_int64; default = 23; QVBR quality level; range = 1-51 #define AMF_VIDEO_ENCODER_AV1_INITIAL_VBV_BUFFER_FULLNESS L"Av1InitialVBVBufferFullness" // amf_int64; default = depends on USAGE; Initial VBV Buffer Fullness 0=0% 64=100% // Alignment Mode Configuration #define AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE L"Av1AlignmentMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_ENUM); default = AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_64X16_ONLY; Alignment Mode. #define AMF_VIDEO_ENCODER_AV1_PRE_ANALYSIS_ENABLE L"Av1EnablePreAnalysis" // bool; default = depends on USAGE; If true, enables the pre-analysis module. Refer to AMF Video PreAnalysis API reference for more details. If false, disable the pre-analysis module. #define AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_PREENCODE L"Av1RateControlPreEncode" // bool; default = depends on USAGE; If true, enables pre-encode assist in rate control; if false, disables pre-encode assist in rate control. #define AMF_VIDEO_ENCODER_AV1_HIGH_MOTION_QUALITY_BOOST L"Av1HighMotionQualityBoost" // bool; default = depends on USAGE; If true, enable high motion quality boost mode; if false, disable high motion quality boost mode. #define AMF_VIDEO_ENCODER_AV1_AQ_MODE L"Av1AQMode" // amd_int64(AMF_VIDEO_ENCODER_AV1_AQ_MODE_ENUM); default = depends on USAGE; AQ mode // Picture Management Configuration #define AMF_VIDEO_ENCODER_AV1_MAX_NUM_TEMPORAL_LAYERS L"Av1MaxNumOfTemporalLayers" // amf_int64; default = depends on USAGE; Max number of temporal layers might be enabled. The maximum value can be queried from AMF_VIDEO_ENCODER_AV1_CAP_MAX_NUM_TEMPORAL_LAYERS #define AMF_VIDEO_ENCODER_AV1_MAX_LTR_FRAMES L"Av1MaxNumLTRFrames" // amf_int64; default = depends on USAGE; Max number of LTR frames. The maximum value can be queried from AMF_VIDEO_ENCODER_AV1_CAP_MAX_NUM_LTR_FRAMES #define AMF_VIDEO_ENCODER_AV1_LTR_MODE L"Av1LTRMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_LTR_MODE_ENUM); default = AMF_VIDEO_ENCODER_AV1_LTR_MODE_RESET_UNUSED; remove/keep unused LTRs (not specified in property AMF_VIDEO_ENCODER_AV1_FORCE_LTR_REFERENCE_BITFIELD) #define AMF_VIDEO_ENCODER_AV1_MAX_NUM_REFRAMES L"Av1MaxNumRefFrames" // amf_int64; default = 1; Maximum number of reference frames // color conversion #define AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA L"Av1InHDRMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL // Miscellaneous #define AMF_VIDEO_ENCODER_AV1_EXTRA_DATA L"Av1ExtraData" // AMFInterface* - > AMFBuffer*; buffer to retrieve coded sequence header #define AMF_VIDEO_ENCODER_AV1_ENABLE_SMART_ACCESS_VIDEO L"Av1EnableEncoderSmartAccessVideo" // amf_bool; default = false; true = enables smart access video feature #define AMF_VIDEO_ENCODER_AV1_INPUT_QUEUE_SIZE L"Av1InputQueueSize" // amf_int64; default 16; Set amf input queue size // Tile Output #define AMF_VIDEO_ENCODER_AV1_OUTPUT_MODE L"AV1OutputMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_OUTPUT_MODE_ENUM); default = AMF_VIDEO_ENCODER_AV1_OUTPUT_MODE_FRAME - defines encoder output mode // *** Dynamic properties - can be set anytime *** // Codec Configuration #define AMF_VIDEO_ENCODER_AV1_PALETTE_MODE L"Av1PaletteMode" // bool; default = depends on USAGE; If true, enable palette mode; if false, disable palette mode. Valid only when AMF_VIDEO_ENCODER_AV1_SCREEN_CONTENT_TOOLS is true. #define AMF_VIDEO_ENCODER_AV1_FORCE_INTEGER_MV L"Av1ForceIntegerMv" // bool; default = depends on USAGE; If true, enable force integer MV; if false, disable force integer MV. Valid only when AMF_VIDEO_ENCODER_AV1_SCREEN_CONTENT_TOOLS is true. #define AMF_VIDEO_ENCODER_AV1_CDF_UPDATE L"Av1CdfUpdate" // bool; default = depends on USAGE; If true, enable CDF update; if false, disable CDF update. #define AMF_VIDEO_ENCODER_AV1_CDF_FRAME_END_UPDATE_MODE L"Av1CdfFrameEndUpdateMode" // amd_int64(AMF_VIDEO_ENCODER_AV1_CDF_FRAME_END_UPDATE_MODE_ENUM); default = depends on USAGE; CDF frame end update mode // Rate Control and Quality Enhancement #define AMF_VIDEO_ENCODER_AV1_VBV_BUFFER_SIZE L"Av1VBVBufferSize" // amf_int64; default = depends on USAGE; VBV Buffer Size in bits #define AMF_VIDEO_ENCODER_AV1_FRAMERATE L"Av1FrameRate" // AMFRate; default = depends on usage; Frame Rate #define AMF_VIDEO_ENCODER_AV1_ENFORCE_HRD L"Av1EnforceHRD" // bool; default = depends on USAGE; If true, enforce HRD; if false, HRD is not enforced. #define AMF_VIDEO_ENCODER_AV1_FILLER_DATA L"Av1FillerData" // bool; default = depends on USAGE; If true, code filler data when needed; if false, don't code filler data. #define AMF_VIDEO_ENCODER_AV1_TARGET_BITRATE L"Av1TargetBitrate" // amf_int64; default = depends on USAGE; Target bit rate in bits #define AMF_VIDEO_ENCODER_AV1_PEAK_BITRATE L"Av1PeakBitrate" // amf_int64; default = depends on USAGE; Peak bit rate in bits #define AMF_VIDEO_ENCODER_AV1_MAX_COMPRESSED_FRAME_SIZE L"Av1MaxCompressedFrameSize" // amf_int64; default = 0; Max compressed frame Size in bits. 0 - no limit #define AMF_VIDEO_ENCODER_AV1_MIN_Q_INDEX_INTRA L"Av1MinQIndex_Intra" // amf_int64; default = depends on USAGE; Min QIndex for intra frames; range = 1-255 #define AMF_VIDEO_ENCODER_AV1_MAX_Q_INDEX_INTRA L"Av1MaxQIndex_Intra" // amf_int64; default = depends on USAGE; Max QIndex for intra frames; range = 1-255 #define AMF_VIDEO_ENCODER_AV1_MIN_Q_INDEX_INTER L"Av1MinQIndex_Inter" // amf_int64; default = depends on USAGE; Min QIndex for inter frames; range = 1-255 #define AMF_VIDEO_ENCODER_AV1_MAX_Q_INDEX_INTER L"Av1MaxQIndex_Inter" // amf_int64; default = depends on USAGE; Max QIndex for inter frames; range = 1-255 #define AMF_VIDEO_ENCODER_AV1_Q_INDEX_INTRA L"Av1QIndex_Intra" // amf_int64; default = depends on USAGE; intra-frame QIndex; range = 1-255 #define AMF_VIDEO_ENCODER_AV1_Q_INDEX_INTER L"Av1QIndex_Inter" // amf_int64; default = depends on USAGE; inter-frame QIndex; range = 1-255 #define AMF_VIDEO_ENCODER_AV1_RATE_CONTROL_SKIP_FRAME L"Av1RateControlSkipFrameEnable" // bool; default = depends on USAGE; If true, rate control may code skip frame when needed; if false, rate control will not code skip frame. // Picture Management Configuration #define AMF_VIDEO_ENCODER_AV1_GOP_SIZE L"Av1GOPSize" // amf_int64; default = depends on USAGE; GOP Size (distance between automatically inserted key frames). If 0, key frame will be inserted at first frame only. Note that GOP may be interrupted by AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE. #define AMF_VIDEO_ENCODER_AV1_INTRA_PERIOD L"Av1IntraPeriod" // amf_int64; default = 0; Intra period in frames. #define AMF_VIDEO_ENCODER_AV1_HEADER_INSERTION_MODE L"Av1HeaderInsertionMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_HEADER_INSERTION_MODE_ENUM); default = depends on USAGE; sequence header insertion mode #define AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE L"Av1SwitchFrameInsertionMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE_ENUM); default = depends on USAGE; switch frame insertin mode #define AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INTERVAL L"Av1SwitchFrameInterval" // amf_int64; default = depends on USAGE; the interval between two inserted switch frames. Valid only when AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE is AMF_VIDEO_ENCODER_AV1_SWITCH_FRAME_INSERTION_MODE_FIXED_INTERVAL. #define AMF_VIDEO_ENCODER_AV1_NUM_TEMPORAL_LAYERS L"Av1NumTemporalLayers" // amf_int64; default = depends on USAGE; Number of temporal layers. Can be changed at any time but the change is only applied when encoding next base layer frame. #define AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE L"Av1IntraRefreshMode" // amf_int64(AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE_ENUM); default AMF_VIDEO_ENCODER_AV1_INTRA_REFRESH_MODE__DISABLED #define AMF_VIDEO_ENCODER_AV1_INTRAREFRESH_STRIPES L"Av1IntraRefreshNumOfStripes" // amf_int64; default = N/A; Valid only when intra refresh is enabled. // color conversion #define AMF_VIDEO_ENCODER_AV1_INPUT_COLOR_PROFILE L"Av1InputColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO by size #define AMF_VIDEO_ENCODER_AV1_INPUT_TRANSFER_CHARACTERISTIC L"Av1InputColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 section 7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_ENCODER_AV1_INPUT_COLOR_PRIMARIES L"Av1InputColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 section 7.1 See ColorSpace.h for enum #define AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PROFILE L"Av1OutputColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO by size #define AMF_VIDEO_ENCODER_AV1_OUTPUT_TRANSFER_CHARACTERISTIC L"Av1OutputColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 ?7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_ENCODER_AV1_OUTPUT_COLOR_PRIMARIES L"Av1OutputColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 section 7.1 See ColorSpace.h for enum // Frame encode parameters #define AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE L"Av1ForceFrameType" // amf_int64(AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_ENUM); default = AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_NONE; generate particular frame type #define AMF_VIDEO_ENCODER_AV1_FORCE_INSERT_SEQUENCE_HEADER L"Av1ForceInsertSequenceHeader" // bool; default = false; If true, force insert sequence header with current frame; #define AMF_VIDEO_ENCODER_AV1_MARK_CURRENT_WITH_LTR_INDEX L"Av1MarkCurrentWithLTRIndex" // amf_int64; default = N/A; Mark current frame with LTR index #define AMF_VIDEO_ENCODER_AV1_FORCE_LTR_REFERENCE_BITFIELD L"Av1ForceLTRReferenceBitfield" // amf_int64; default = 0; force LTR bit-field #define AMF_VIDEO_ENCODER_AV1_ROI_DATA L"Av1ROIData" // 2D AMFSurface, surface format: AMF_SURFACE_GRAY32 #define AMF_VIDEO_ENCODER_AV1_PSNR_FEEDBACK L"Av1PSNRFeedback" // amf_bool; default = false; Signal encoder to calculate PSNR score #define AMF_VIDEO_ENCODER_AV1_SSIM_FEEDBACK L"Av1SSIMFeedback" // amf_bool; default = false; Signal encoder to calculate SSIM score #define AMF_VIDEO_ENCODER_AV1_STATISTICS_FEEDBACK L"Av1StatisticsFeedback" // amf_bool; default = false; Signal encoder to collect and feedback encoder statistics #define AMF_VIDEO_ENCODER_AV1_BLOCK_Q_INDEX_FEEDBACK L"Av1BlockQIndexFeedback" // amf_bool; default = false; Signal encoder to collect and feedback block level QIndex values // Encode output parameters #define AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE L"Av1OutputFrameType" // amf_int64(AMF_VIDEO_ENCODER_AV1_OUTPUT_FRAME_TYPE_ENUM); default = N/A #define AMF_VIDEO_ENCODER_AV1_OUTPUT_MARKED_LTR_INDEX L"Av1MarkedLTRIndex" // amf_int64; default = N/A; Marked LTR index #define AMF_VIDEO_ENCODER_AV1_OUTPUT_REFERENCED_LTR_INDEX_BITFIELD L"Av1ReferencedLTRIndexBitfield" // amf_int64; default = N/A; referenced LTR bit-field #define AMF_VIDEO_ENCODER_AV1_OUTPUT_BUFFER_TYPE L"AV1OutputBufferType" // amf_int64(AMF_VIDEO_ENCODER_AV1_OUTPUT_BUFFER_TYPE_ENUM); encoder output buffer type #define AMF_VIDEO_ENCODER_AV1_RECONSTRUCTED_PICTURE L"Av1ReconstructedPicture" // AMFInterface(AMFSurface); returns reconstructed picture as an AMFSurface attached to the output buffer as property AMF_VIDEO_ENCODER_RECONSTRUCTED_PICTURE of AMFInterface type #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PSNR_Y L"Av1PSNRY" // double; PSNR Y #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PSNR_U L"Av1PSNRU" // double; PSNR U #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PSNR_V L"Av1PSNRV" // double; PSNR V #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PSNR_ALL L"Av1PSNRALL" // double; PSNR All #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SSIM_Y L"Av1SSIMY" // double; SSIM Y #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SSIM_U L"Av1SSIMU" // double; SSIM U #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SSIM_V L"Av1SSIMV" // double; SSIM V #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SSIM_ALL L"Av1SSIMALL" // double; SSIM ALL // Encoder statistics feedback #define AMF_VIDEO_ENCODER_AV1_STATISTIC_FRAME_Q_INDEX L"Av1StatisticsFeedbackFrameQIndex" // amf_int64; Rate control base frame/initial QIndex #define AMF_VIDEO_ENCODER_AV1_STATISTIC_AVERAGE_Q_INDEX L"Av1StatisticsFeedbackAvgQIndex" // amf_int64; Average QIndex of all encoded SBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped SBs. #define AMF_VIDEO_ENCODER_AV1_STATISTIC_MAX_Q_INDEX L"Av1StatisticsFeedbackMaxQIndex" // amf_int64; Max QIndex among all encoded SBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped SBs. #define AMF_VIDEO_ENCODER_AV1_STATISTIC_MIN_Q_INDEX L"Av1StatisticsFeedbackMinQIndex" // amf_int64; Min QIndex among all encoded SBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped SBs. #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PIX_NUM_INTRA L"Av1StatisticsFeedbackPixNumIntra" // amf_int64; Number of the intra encoded pixels #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PIX_NUM_INTER L"Av1StatisticsFeedbackPixNumInter" // amf_int64; Number of the inter encoded pixels #define AMF_VIDEO_ENCODER_AV1_STATISTIC_PIX_NUM_SKIP L"Av1StatisticsFeedbackPixNumSkip" // amf_int64; Number of the skip mode pixels #define AMF_VIDEO_ENCODER_AV1_STATISTIC_BITCOUNT_RESIDUAL L"Av1StatisticsFeedbackBitcountResidual" // amf_int64; The bit count that corresponds to residual data #define AMF_VIDEO_ENCODER_AV1_STATISTIC_BITCOUNT_MOTION L"Av1StatisticsFeedbackBitcountMotion" // amf_int64; The bit count that corresponds to motion vectors #define AMF_VIDEO_ENCODER_AV1_STATISTIC_BITCOUNT_INTER L"Av1StatisticsFeedbackBitcountInter" // amf_int64; The bit count that are assigned to inter SBs #define AMF_VIDEO_ENCODER_AV1_STATISTIC_BITCOUNT_INTRA L"Av1StatisticsFeedbackBitcountIntra" // amf_int64; The bit count that are assigned to intra SBs #define AMF_VIDEO_ENCODER_AV1_STATISTIC_BITCOUNT_ALL_MINUS_HEADER L"Av1StatisticsFeedbackBitcountAllMinusHeader" // amf_int64; The bit count of the bitstream excluding header #define AMF_VIDEO_ENCODER_AV1_STATISTIC_MV_X L"Av1StatisticsFeedbackMvX" // amf_int64; Accumulated absolute values of horizontal MV's #define AMF_VIDEO_ENCODER_AV1_STATISTIC_MV_Y L"Av1StatisticsFeedbackMvY" // amf_int64; Accumulated absolute values of vertical MV's #define AMF_VIDEO_ENCODER_AV1_STATISTIC_RD_COST_FINAL L"Av1StatisticsFeedbackRdCostFinal" // amf_int64; Frame level final RD cost for full encoding #define AMF_VIDEO_ENCODER_AV1_STATISTIC_RD_COST_INTRA L"Av1StatisticsFeedbackRdCostIntra" // amf_int64; Frame level intra RD cost for full encoding #define AMF_VIDEO_ENCODER_AV1_STATISTIC_RD_COST_INTER L"Av1StatisticsFeedbackRdCostInter" // amf_int64; Frame level inter RD cost for full encoding #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SAD_FINAL L"Av1StatisticsFeedbackSadFinal" // amf_int64; Frame level final SAD for full encoding #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SAD_INTRA L"Av1StatisticsFeedbackSadIntra" // amf_int64; Frame level intra SAD for full encoding #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SAD_INTER L"Av1StatisticsFeedbackSadInter" // amf_int64; Frame level inter SAD for full encoding #define AMF_VIDEO_ENCODER_AV1_STATISTIC_SSE L"Av1StatisticsFeedbackSSE" // amf_int64; Frame level SSE (only calculated for AV1) #define AMF_VIDEO_ENCODER_AV1_STATISTIC_VARIANCE L"Av1StatisticsFeedbackVariance" // amf_int64; Frame level variance for full encoding // Encoder block level feedback #define AMF_VIDEO_ENCODER_AV1_BLOCK_Q_INDEX_MAP L"Av1BlockQIndexMap" // AMFInterface(AMFSurface); AMFSurface of format AMF_SURFACE_GRAY32 containing block level QIndex values // AV1 Encoder capabilities - exposed in AMFCaps interface #define AMF_VIDEO_ENCODER_AV1_CAP_NUM_OF_HW_INSTANCES L"Av1CapNumOfHwInstances" // amf_int64; default = N/A; number of HW encoder instances #define AMF_VIDEO_ENCODER_AV1_CAP_MAX_THROUGHPUT L"Av1CapMaxThroughput" // amf_int64; default = N/A; MAX throughput for AV1 encoder in MB (16 x 16 pixel) #define AMF_VIDEO_ENCODER_AV1_CAP_REQUESTED_THROUGHPUT L"Av1CapRequestedThroughput" // amf_int64; default = N/A; Currently total requested throughput for AV1 encode in MB (16 x 16 pixel) #define AMF_VIDEO_ENCODER_AV1_CAP_COLOR_CONVERSION L"Av1CapColorConversion" // amf_int64(AMF_ACCELERATION_TYPE); default = N/A; type of supported color conversion. #define AMF_VIDEO_ENCODER_AV1_CAP_PRE_ANALYSIS L"Av1PreAnalysis" // amf_bool - pre analysis module is available for AV1 UVE encoder, n/a for the other encoders #define AMF_VIDEO_ENCODER_AV1_CAP_MAX_BITRATE L"Av1MaxBitrate" // amf_int64; default = N/A; Maximum bit rate in bits #define AMF_VIDEO_ENCODER_AV1_CAP_MAX_PROFILE L"Av1MaxProfile" // amf_int64(AMF_VIDEO_ENCODER_AV1_PROFILE_ENUM); default = N/A; max value of code profile #define AMF_VIDEO_ENCODER_AV1_CAP_MAX_LEVEL L"Av1MaxLevel" // amf_int64(AMF_VIDEO_ENCODER_AV1_LEVEL_ENUM); default = N/A; max value of codec level #define AMF_VIDEO_ENCODER_AV1_CAP_MAX_NUM_TEMPORAL_LAYERS L"Av1CapMaxNumTemporalLayers" // amf_int64; default = N/A; The cap of maximum number of temporal layers #define AMF_VIDEO_ENCODER_AV1_CAP_MAX_NUM_LTR_FRAMES L"Av1CapMaxNumLTRFrames" // amf_int64; default = N/A; The cap of maximum number of LTR frames. This value is calculated based on current value of AMF_VIDEO_ENCODER_AV1_MAX_NUM_TEMPORAL_LAYERS. #define AMF_VIDEO_ENCODER_AV1_CAP_SUPPORT_TILE_OUTPUT L"AV1SupportTileOutput" // amf_bool; if tile output is supported #define AMF_VIDEO_ENCODER_AV1_CAP_SUPPORT_SMART_ACCESS_VIDEO L"Av1EncoderSupportSmartAccessVideo" // amf_bool; returns true if system supports SmartAccess Video #endif //#ifndef AMF_VideoEncoderAV1_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoEncoderHEVC.h ================================================ // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // VideoEncoderHW_HEVC interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoEncoderHEVC_h #define AMF_VideoEncoderHEVC_h #pragma once #include "Component.h" #include "ColorSpace.h" #include "PreAnalysis.h" #define AMFVideoEncoder_HEVC L"AMFVideoEncoderHW_HEVC" enum AMF_VIDEO_ENCODER_HEVC_USAGE_ENUM { AMF_VIDEO_ENCODER_HEVC_USAGE_TRANSCONDING = 0, // kept for backwards compatability AMF_VIDEO_ENCODER_HEVC_USAGE_TRANSCODING = 0, // fixed typo AMF_VIDEO_ENCODER_HEVC_USAGE_ULTRA_LOW_LATENCY, AMF_VIDEO_ENCODER_HEVC_USAGE_LOW_LATENCY, AMF_VIDEO_ENCODER_HEVC_USAGE_WEBCAM, AMF_VIDEO_ENCODER_HEVC_USAGE_HIGH_QUALITY, AMF_VIDEO_ENCODER_HEVC_USAGE_LOW_LATENCY_HIGH_QUALITY }; enum AMF_VIDEO_ENCODER_HEVC_PROFILE_ENUM { AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN = 1, AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN_10 = 2 }; enum AMF_VIDEO_ENCODER_HEVC_TIER_ENUM { AMF_VIDEO_ENCODER_HEVC_TIER_MAIN = 0, AMF_VIDEO_ENCODER_HEVC_TIER_HIGH = 1 }; enum AMF_VIDEO_ENCODER_LEVEL_ENUM { AMF_LEVEL_1 = 30, AMF_LEVEL_2 = 60, AMF_LEVEL_2_1 = 63, AMF_LEVEL_3 = 90, AMF_LEVEL_3_1 = 93, AMF_LEVEL_4 = 120, AMF_LEVEL_4_1 = 123, AMF_LEVEL_5 = 150, AMF_LEVEL_5_1 = 153, AMF_LEVEL_5_2 = 156, AMF_LEVEL_6 = 180, AMF_LEVEL_6_1 = 183, AMF_LEVEL_6_2 = 186 }; enum AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_ENUM { AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_UNKNOWN = -1, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CONSTANT_QP = 0, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_PEAK_CONSTRAINED_VBR, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_CBR, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_QUALITY_VBR, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_HIGH_QUALITY_VBR, AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_HIGH_QUALITY_CBR }; enum AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_ENUM { AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_NONE = 0, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_SKIP, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P }; enum AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE_ENUM { AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE_IDR, AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE_I, AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE_P }; enum AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_ENUM { AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_QUALITY = 0, AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_BALANCED = 5, AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_SPEED = 10 }; enum AMF_VIDEO_ENCODER_HEVC_HEADER_INSERTION_MODE_ENUM { AMF_VIDEO_ENCODER_HEVC_HEADER_INSERTION_MODE_NONE = 0, AMF_VIDEO_ENCODER_HEVC_HEADER_INSERTION_MODE_GOP_ALIGNED, AMF_VIDEO_ENCODER_HEVC_HEADER_INSERTION_MODE_IDR_ALIGNED }; enum AMF_VIDEO_ENCODER_HEVC_PICTURE_TRANSFER_MODE_ENUM { AMF_VIDEO_ENCODER_HEVC_PICTURE_TRANSFER_MODE_OFF = 0, AMF_VIDEO_ENCODER_HEVC_PICTURE_TRANSFER_MODE_ON }; enum AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE { AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE_STUDIO = 0, AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE_FULL = 1 }; enum AMF_VIDEO_ENCODER_HEVC_LTR_MODE_ENUM { AMF_VIDEO_ENCODER_HEVC_LTR_MODE_RESET_UNUSED = 0, AMF_VIDEO_ENCODER_HEVC_LTR_MODE_KEEP_UNUSED }; enum AMF_VIDEO_ENCODER_HEVC_OUTPUT_MODE_ENUM { AMF_VIDEO_ENCODER_HEVC_OUTPUT_MODE_FRAME = 0, AMF_VIDEO_ENCODER_HEVC_OUTPUT_MODE_SLICE = 1 }; enum AMF_VIDEO_ENCODER_HEVC_OUTPUT_BUFFER_TYPE_ENUM { AMF_VIDEO_ENCODER_HEVC_OUTPUT_BUFFER_TYPE_FRAME = 0, AMF_VIDEO_ENCODER_HEVC_OUTPUT_BUFFER_TYPE_SLICE = 1, AMF_VIDEO_ENCODER_HEVC_OUTPUT_BUFFER_TYPE_SLICE_LAST = 2 }; // Static properties - can be set before Init() #define AMF_VIDEO_ENCODER_HEVC_INSTANCE_INDEX L"HevcEncoderInstance" // amf_int64; selected instance idx #define AMF_VIDEO_ENCODER_HEVC_FRAMESIZE L"HevcFrameSize" // AMFSize; default = 0,0; Frame size #define AMF_VIDEO_ENCODER_HEVC_USAGE L"HevcUsage" // amf_int64(AMF_VIDEO_ENCODER_HEVC_USAGE_ENUM); default = N/A; Encoder usage type. fully configures parameter set. #define AMF_VIDEO_ENCODER_HEVC_PROFILE L"HevcProfile" // amf_int64(AMF_VIDEO_ENCODER_HEVC_PROFILE_ENUM) ; default = AMF_VIDEO_ENCODER_HEVC_PROFILE_MAIN; #define AMF_VIDEO_ENCODER_HEVC_TIER L"HevcTier" // amf_int64(AMF_VIDEO_ENCODER_HEVC_TIER_ENUM) ; default = AMF_VIDEO_ENCODER_HEVC_TIER_MAIN; #define AMF_VIDEO_ENCODER_HEVC_PROFILE_LEVEL L"HevcProfileLevel" // amf_int64 (AMF_VIDEO_ENCODER_LEVEL_ENUM, default depends on HW capabilities); #define AMF_VIDEO_ENCODER_HEVC_MAX_LTR_FRAMES L"HevcMaxOfLTRFrames" // amf_int64; default = 0; Max number of LTR frames #define AMF_VIDEO_ENCODER_HEVC_LTR_MODE L"HevcLTRMode" // amf_int64(AMF_VIDEO_ENCODER_HEVC_LTR_MODE_ENUM); default = AMF_VIDEO_ENCODER_HEVC_LTR_MODE_RESET_UNUSED; remove/keep unused LTRs (not specified in property AMF_VIDEO_ENCODER_HEVC_FORCE_LTR_REFERENCE_BITFIELD) #define AMF_VIDEO_ENCODER_HEVC_MAX_NUM_REFRAMES L"HevcMaxNumRefFrames" // amf_int64; default = 1; Maximum number of reference frames #define AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET L"HevcQualityPreset" // amf_int64(AMF_VIDEO_ENCODER_HEVC_QUALITY_PRESET_ENUM); default = depends on USAGE; Quality Preset #define AMF_VIDEO_ENCODER_HEVC_EXTRADATA L"HevcExtraData" // AMFInterface* - > AMFBuffer*; SPS/PPS buffer - read-only #define AMF_VIDEO_ENCODER_HEVC_ASPECT_RATIO L"HevcAspectRatio" // AMFRatio; default = 1, 1 #define AMF_VIDEO_ENCODER_HEVC_LOWLATENCY_MODE L"LowLatencyInternal" // bool; default = false, enables low latency mode #define AMF_VIDEO_ENCODER_HEVC_PRE_ANALYSIS_ENABLE L"HevcEnablePreAnalysis" // bool; default = false; enables the pre-analysis module. Currently only works in AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_PEAK_CONSTRAINED_VBR mode. Refer to AMF Video PreAnalysis API reference for more details. #define AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE L"HevcNominalRange" // amf_int64(AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE); default = amf_int64(AMF_VIDEO_ENCODER_HEVC_NOMINAL_RANGE_STUDIO); property is bool but amf_int64 also works for backward compatibility. #define AMF_VIDEO_ENCODER_HEVC_MAX_NUM_TEMPORAL_LAYERS L"HevcMaxNumOfTemporalLayers" // amf_int64; default = 1; Max number of temporal layers. // Picture control properties #define AMF_VIDEO_ENCODER_HEVC_NUM_GOPS_PER_IDR L"HevcGOPSPerIDR" // amf_int64; default = 1; The frequency to insert IDR as start of a GOP. 0 means no IDR will be inserted. #define AMF_VIDEO_ENCODER_HEVC_GOP_SIZE L"HevcGOPSize" // amf_int64; default = 60; GOP Size, in frames #define AMF_VIDEO_ENCODER_HEVC_DE_BLOCKING_FILTER_DISABLE L"HevcDeBlockingFilter" // bool; default = depends on USAGE; De-blocking Filter #define AMF_VIDEO_ENCODER_HEVC_SLICES_PER_FRAME L"HevcSlicesPerFrame" // amf_int64; default = 1; Number of slices Per Frame #define AMF_VIDEO_ENCODER_HEVC_HEADER_INSERTION_MODE L"HevcHeaderInsertionMode" // amf_int64(AMF_VIDEO_ENCODER_HEVC_HEADER_INSERTION_MODE_ENUM); default = NONE #define AMF_VIDEO_ENCODER_HEVC_INTRA_REFRESH_NUM_CTBS_PER_SLOT L"HevcIntraRefreshCTBsNumberPerSlot" // amf_int64; default = depends on USAGE; Intra Refresh CTBs Number Per Slot in 64x64 CTB // Rate control properties #define AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD L"HevcRateControlMethod" // amf_int64(AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD_ENUM); default = depends on USAGE; Rate Control Method #define AMF_VIDEO_ENCODER_HEVC_QVBR_QUALITY_LEVEL L"HevcQvbrQualityLevel" // amf_int64; default = 23; QVBR quality level; range = 1-51 #define AMF_VIDEO_ENCODER_HEVC_VBV_BUFFER_SIZE L"HevcVBVBufferSize" // amf_int64; default = depends on USAGE; VBV Buffer Size in bits #define AMF_VIDEO_ENCODER_HEVC_INITIAL_VBV_BUFFER_FULLNESS L"HevcInitialVBVBufferFullness" // amf_int64; default = 64; Initial VBV Buffer Fullness 0=0% 64=100% #define AMF_VIDEO_ENCODER_HEVC_ENABLE_VBAQ L"HevcEnableVBAQ" // // bool; default = depends on USAGE; Enable auto VBAQ #define AMF_VIDEO_ENCODER_HEVC_HIGH_MOTION_QUALITY_BOOST_ENABLE L"HevcHighMotionQualityBoostEnable"// bool; default = depends on USAGE; Enable High motion quality boost mode #define AMF_VIDEO_ENCODER_HEVC_PREENCODE_ENABLE L"HevcRateControlPreAnalysisEnable" // bool; default = depends on USAGE; enables pre-encode assisted rate control #define AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_PREANALYSIS_ENABLE L"HevcRateControlPreAnalysisEnable" // bool; default = depends on USAGE; enables pre-encode assisted rate control. Deprecated, please use AMF_VIDEO_ENCODER_PREENCODE_ENABLE instead. #ifdef _MSC_VER #ifndef __clang__ #pragma deprecated("AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_PREANALYSIS_ENABLE") #endif #endif // Motion estimation #define AMF_VIDEO_ENCODER_HEVC_MOTION_HALF_PIXEL L"HevcHalfPixel" // bool; default= true; Half Pixel #define AMF_VIDEO_ENCODER_HEVC_MOTION_QUARTERPIXEL L"HevcQuarterPixel" // bool; default= true; Quarter Pixel // color conversion #define AMF_VIDEO_ENCODER_HEVC_COLOR_BIT_DEPTH L"HevcColorBitDepth" // amf_int64(AMF_COLOR_BIT_DEPTH_ENUM); default = AMF_COLOR_BIT_DEPTH_8 #define AMF_VIDEO_ENCODER_HEVC_INPUT_COLOR_PROFILE L"HevcInColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO by size #define AMF_VIDEO_ENCODER_HEVC_INPUT_TRANSFER_CHARACTERISTIC L"HevcInColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 section 7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_ENCODER_HEVC_INPUT_COLOR_PRIMARIES L"HevcInColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 section 7.1 See ColorSpace.h for enum #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PROFILE L"HevcOutColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO by size #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_TRANSFER_CHARACTERISTIC L"HevcOutColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 ?7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PRIMARIES L"HevcOutColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 section 7.1 See ColorSpace.h for enum // Slice output #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_MODE L"HevcOutputMode" // amf_int64(AMF_VIDEO_ENCODER_HEVC_OUTPUT_MODE_ENUM); default = AMF_VIDEO_ENCODER_HEVC_OUTPUT_MODE_FRAME - defines encoder output mode // Dynamic properties - can be set at any time // Rate control properties #define AMF_VIDEO_ENCODER_HEVC_FRAMERATE L"HevcFrameRate" // AMFRate; default = depends on usage; Frame Rate #define AMF_VIDEO_ENCODER_HEVC_ENFORCE_HRD L"HevcEnforceHRD" // bool; default = depends on USAGE; Enforce HRD #define AMF_VIDEO_ENCODER_HEVC_FILLER_DATA_ENABLE L"HevcFillerDataEnable" // bool; default = depends on USAGE; Enforce HRD #define AMF_VIDEO_ENCODER_HEVC_TARGET_BITRATE L"HevcTargetBitrate" // amf_int64; default = depends on USAGE; Target bit rate in bits #define AMF_VIDEO_ENCODER_HEVC_PEAK_BITRATE L"HevcPeakBitrate" // amf_int64; default = depends on USAGE; Peak bit rate in bits #define AMF_VIDEO_ENCODER_HEVC_MAX_AU_SIZE L"HevcMaxAUSize" // amf_int64; default = 60; Max AU Size in bits #define AMF_VIDEO_ENCODER_HEVC_MIN_QP_I L"HevcMinQP_I" // amf_int64; default = depends on USAGE; Min QP; range = #define AMF_VIDEO_ENCODER_HEVC_MAX_QP_I L"HevcMaxQP_I" // amf_int64; default = depends on USAGE; Max QP; range = #define AMF_VIDEO_ENCODER_HEVC_MIN_QP_P L"HevcMinQP_P" // amf_int64; default = depends on USAGE; Min QP; range = #define AMF_VIDEO_ENCODER_HEVC_MAX_QP_P L"HevcMaxQP_P" // amf_int64; default = depends on USAGE; Max QP; range = #define AMF_VIDEO_ENCODER_HEVC_QP_I L"HevcQP_I" // amf_int64; default = 26; P-frame QP; range = 0-51 #define AMF_VIDEO_ENCODER_HEVC_QP_P L"HevcQP_P" // amf_int64; default = 26; P-frame QP; range = 0-51 #define AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_SKIP_FRAME_ENABLE L"HevcRateControlSkipFrameEnable" // bool; default = depends on USAGE; Rate Control Based Frame Skip // color conversion #define AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA L"HevcInHDRMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL //#define AMF_VIDEO_ENCODER_HEVC_OUTPUT_HDR_METADATA L"HevcOutHDRMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL // SVC #define AMF_VIDEO_ENCODER_HEVC_NUM_TEMPORAL_LAYERS L"HevcNumOfTemporalLayers" // amf_int64; default = 1; Number of temporal layers. Can be changed at any time but the change is only applied when encoding next base layer frame. // DPB management #define AMF_VIDEO_ENCODER_HEVC_PICTURE_TRANSFER_MODE L"HevcPicTransferMode" // amf_int64(AMF_VIDEO_ENCODER_HEVC_PICTURE_TRANSFER_MODE_ENUM); default = AMF_VIDEO_ENCODER_HEVC_PICTURE_TRANSFER_MODE_OFF - whether to exchange reference/reconstructed pic between encoder and application // misc #define AMF_VIDEO_ENCODER_HEVC_QUERY_TIMEOUT L"HevcQueryTimeout" // amf_int64; default = 0 (no wait); timeout for QueryOutput call in ms. #define AMF_VIDEO_ENCODER_HEVC_MEMORY_TYPE L"HevcEncoderMemoryType" // amf_int64(AMF_MEMORY_TYPE) , default is AMF_MEMORY_UNKNOWN, Values : AMF_MEMORY_DX11, AMF_MEMORY_DX9, AMF_MEMORY_UNKNOWN (auto) #define AMF_VIDEO_ENCODER_HEVC_ENABLE_SMART_ACCESS_VIDEO L"HevcEnableEncoderSmartAccessVideo" // amf_bool; default = false; true = enables smart access video feature #define AMF_VIDEO_ENCODER_HEVC_INPUT_QUEUE_SIZE L"HevcInputQueueSize" // amf_int64; default 16; Set amf input queue size // Per-submission properties - can be set on input surface interface #define AMF_VIDEO_ENCODER_HEVC_END_OF_SEQUENCE L"HevcEndOfSequence" // bool; default = false; generate end of sequence #define AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE L"HevcForcePictureType" // amf_int64(AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_ENUM); default = AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_NONE; generate particular picture type #define AMF_VIDEO_ENCODER_HEVC_INSERT_AUD L"HevcInsertAUD" // bool; default = false; insert AUD #define AMF_VIDEO_ENCODER_HEVC_INSERT_HEADER L"HevcInsertHeader" // bool; default = false; insert header(SPS, PPS, VPS) #define AMF_VIDEO_ENCODER_HEVC_MARK_CURRENT_WITH_LTR_INDEX L"HevcMarkCurrentWithLTRIndex" // amf_int64; default = N/A; Mark current frame with LTR index #define AMF_VIDEO_ENCODER_HEVC_FORCE_LTR_REFERENCE_BITFIELD L"HevcForceLTRReferenceBitfield"// amf_int64; default = 0; force LTR bit-field #define AMF_VIDEO_ENCODER_HEVC_ROI_DATA L"HevcROIData" // 2D AMFSurface, surface format: AMF_SURFACE_GRAY32 #define AMF_VIDEO_ENCODER_HEVC_REFERENCE_PICTURE L"HevcReferencePicture" // AMFInterface(AMFSurface); surface used for frame injection #define AMF_VIDEO_ENCODER_HEVC_PSNR_FEEDBACK L"HevcPSNRFeedback" // amf_bool; default = false; Signal encoder to calculate PSNR score #define AMF_VIDEO_ENCODER_HEVC_SSIM_FEEDBACK L"HevcSSIMFeedback" // amf_bool; default = false; Signal encoder to calculate SSIM score #define AMF_VIDEO_ENCODER_HEVC_STATISTICS_FEEDBACK L"HevcStatisticsFeedback" // amf_bool; default = false; Signal encoder to collect and feedback encoder statistics #define AMF_VIDEO_ENCODER_HEVC_BLOCK_QP_FEEDBACK L"HevcBlockQpFeedback" // amf_bool; default = false; Signal encoder to collect and feedback block level QP values // Properties set by encoder on output buffer interface #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE L"HevcOutputDataType" // amf_int64(AMF_VIDEO_ENCODER_HEVC_OUTPUT_DATA_TYPE_ENUM); default = N/A #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_MARKED_LTR_INDEX L"HevcMarkedLTRIndex" // amf_int64; default = -1; Marked LTR index #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_REFERENCED_LTR_INDEX_BITFIELD L"HevcReferencedLTRIndexBitfield"// amf_int64; default = 0; referenced LTR bit-field #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_TEMPORAL_LAYER L"HevcOutputTemporalLayer" // amf_int64; Temporal layer #define AMF_VIDEO_ENCODER_HEVC_OUTPUT_BUFFER_TYPE L"HevcOutputBufferType" // amf_int64(AMF_VIDEO_ENCODER_HEVC_OUTPUT_BUFFER_TYPE_ENUM); encoder output buffer type #define AMF_VIDEO_ENCODER_HEVC_RECONSTRUCTED_PICTURE L"HevcReconstructedPicture" // AMFInterface(AMFSurface); returns reconstructed picture as an AMFSurface attached to the output buffer as property AMF_VIDEO_ENCODER_RECONSTRUCTED_PICTURE of AMFInterface type #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PSNR_Y L"PSNRY" // double; PSNR Y #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PSNR_U L"PSNRU" // double; PSNR U #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PSNR_V L"PSNRV" // double; PSNR V #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PSNR_ALL L"PSNRALL" // double; PSNR All #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SSIM_Y L"SSIMY" // double; SSIM Y #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SSIM_U L"SSIMU" // double; SSIM U #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SSIM_V L"SSIMV" // double; SSIM V #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SSIM_ALL L"SSIMALL" // double; SSIM ALL // Encoder statistics feedback #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_FRAME_QP L"HevcStatisticsFeedbackFrameQP" // amf_int64; Rate control base frame/initial QP #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_AVERAGE_QP L"HevcStatisticsFeedbackAvgQP" // amf_int64; Average QP of all encoded CTBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped CTBs. #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_MAX_QP L"HevcStatisticsFeedbackMaxQP" // amf_int64; Max QP among all encoded CTBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped CTBs. #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_MIN_QP L"HevcStatisticsFeedbackMinQP" // amf_int64; Min QP among all encoded CTBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped CTBs. #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PIX_NUM_INTRA L"HevcStatisticsFeedbackPixNumIntra" // amf_int64; Number of the intra encoded pixels #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PIX_NUM_INTER L"HevcStatisticsFeedbackPixNumInter" // amf_int64; Number of the inter encoded pixels #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_PIX_NUM_SKIP L"HevcStatisticsFeedbackPixNumSkip" // amf_int64; Number of the skip mode pixels #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_BITCOUNT_RESIDUAL L"HevcStatisticsFeedbackBitcountResidual" // amf_int64; The bit count that corresponds to residual data #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_BITCOUNT_MOTION L"HevcStatisticsFeedbackBitcountMotion" // amf_int64; The bit count that corresponds to motion vectors #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_BITCOUNT_INTER L"HevcStatisticsFeedbackBitcountInter" // amf_int64; The bit count that are assigned to inter CTBs #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_BITCOUNT_INTRA L"HevcStatisticsFeedbackBitcountIntra" // amf_int64; The bit count that are assigned to intra CTBs #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_BITCOUNT_ALL_MINUS_HEADER L"HevcStatisticsFeedbackBitcountAllMinusHeader" // amf_int64; The bit count of the bitstream excluding header #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_MV_X L"HevcStatisticsFeedbackMvX" // amf_int64; Accumulated absolute values of horizontal MV's #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_MV_Y L"HevcStatisticsFeedbackMvY" // amf_int64; Accumulated absolute values of vertical MV's #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_RD_COST_FINAL L"HevcStatisticsFeedbackRdCostFinal" // amf_int64; Frame level final RD cost for full encoding #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_RD_COST_INTRA L"HevcStatisticsFeedbackRdCostIntra" // amf_int64; Frame level intra RD cost for full encoding #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_RD_COST_INTER L"HevcStatisticsFeedbackRdCostInter" // amf_int64; Frame level inter RD cost for full encoding #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SAD_FINAL L"HevcStatisticsFeedbackSadFinal" // amf_int64; Frame level final SAD for full encoding #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SAD_INTRA L"HevcStatisticsFeedbackSadIntra" // amf_int64; Frame level intra SAD for full encoding #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_SAD_INTER L"HevcStatisticsFeedbackSadInter" // amf_int64; Frame level inter SAD for full encoding #define AMF_VIDEO_ENCODER_HEVC_STATISTIC_VARIANCE L"HevcStatisticsFeedbackVariance" // amf_int64; Frame level variance for full encoding // Encoder block level feedback #define AMF_VIDEO_ENCODER_HEVC_BLOCK_QP_MAP L"HevcBlockQpMap" // AMFInterface(AMFSurface); AMFSurface of format AMF_SURFACE_GRAY32 containing block level QP values // HEVC Encoder capabilities - exposed in AMFCaps interface #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_BITRATE L"HevcMaxBitrate" // amf_int64; Maximum bit rate in bits #define AMF_VIDEO_ENCODER_HEVC_CAP_NUM_OF_STREAMS L"HevcNumOfStreams" // amf_int64; maximum number of encode streams supported #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_PROFILE L"HevcMaxProfile" // amf_int64(AMF_VIDEO_ENCODER_HEVC_PROFILE_ENUM) #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_TIER L"HevcMaxTier" // amf_int64(AMF_VIDEO_ENCODER_HEVC_TIER_ENUM) maximum profile tier #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_LEVEL L"HevcMaxLevel" // amf_int64 maximum profile level #define AMF_VIDEO_ENCODER_HEVC_CAP_MIN_REFERENCE_FRAMES L"HevcMinReferenceFrames" // amf_int64 minimum number of reference frames #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_REFERENCE_FRAMES L"HevcMaxReferenceFrames" // amf_int64 maximum number of reference frames #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_TEMPORAL_LAYERS L"HevcMaxTemporalLayers" // amf_int64 maximum number of temporal layers #define AMF_VIDEO_ENCODER_HEVC_CAP_NUM_OF_HW_INSTANCES L"HevcNumOfHwInstances" // amf_int64 number of HW encoder instances #define AMF_VIDEO_ENCODER_HEVC_CAP_COLOR_CONVERSION L"HevcColorConversion" // amf_int64(AMF_ACCELERATION_TYPE) - type of supported color conversion. default AMF_ACCEL_GPU #define AMF_VIDEO_ENCODER_HEVC_CAP_PRE_ANALYSIS L"HevcPreAnalysis" // amf_bool - pre analysis module is available for HEVC UVE encoder, n/a for the other encoders #define AMF_VIDEO_ENCODER_HEVC_CAP_ROI L"HevcROIMap" // amf_bool - ROI map support is available for HEVC UVE encoder, n/a for the other encoders #define AMF_VIDEO_ENCODER_HEVC_CAP_MAX_THROUGHPUT L"HevcMaxThroughput" // amf_int64 - MAX throughput for HEVC encoder in MB (16 x 16 pixel) #define AMF_VIDEO_ENCODER_HEVC_CAP_REQUESTED_THROUGHPUT L"HevcRequestedThroughput" // amf_int64 - Currently total requested throughput for HEVC encode in MB (16 x 16 pixel) #define AMF_VIDEO_ENCODER_HEVC_CAP_QUERY_TIMEOUT_SUPPORT L"HevcQueryTimeoutSupport" // amf_bool - Timeout supported for QueryOutout call #define AMF_VIDEO_ENCODER_CAPS_HEVC_QUERY_TIMEOUT_SUPPORT L"HevcQueryTimeoutSupport" // amf_bool - Timeout supported for QueryOutout call (Deprecated, please use AMF_VIDEO_ENCODER_HEVC_CAP_QUERY_TIMEOUT_SUPPORT instead) #define AMF_VIDEO_ENCODER_HEVC_CAP_SUPPORT_SLICE_OUTPUT L"HevcSupportSliceOutput" // amf_bool - if slice output is supported #define AMF_VIDEO_ENCODER_HEVC_CAP_SUPPORT_SMART_ACCESS_VIDEO L"HevcEncoderSupportSmartAccessVideo" // amf_bool; returns true if system supports SmartAccess Video #endif //#ifndef AMF_VideoEncoderHEVC_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoEncoderVCE.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // AMFVideoEncoderHW_AVC interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_VideoEncoderVCE_h #define AMF_VideoEncoderVCE_h #pragma once #include "Component.h" #include "ColorSpace.h" #include "PreAnalysis.h" #define AMFVideoEncoderVCE_AVC L"AMFVideoEncoderVCE_AVC" #define AMFVideoEncoderVCE_SVC L"AMFVideoEncoderVCE_SVC" enum AMF_VIDEO_ENCODER_USAGE_ENUM { AMF_VIDEO_ENCODER_USAGE_TRANSCONDING = 0, // kept for backwards compatability AMF_VIDEO_ENCODER_USAGE_TRANSCODING = 0, // fixed typo AMF_VIDEO_ENCODER_USAGE_ULTRA_LOW_LATENCY, AMF_VIDEO_ENCODER_USAGE_LOW_LATENCY, AMF_VIDEO_ENCODER_USAGE_WEBCAM, AMF_VIDEO_ENCODER_USAGE_HIGH_QUALITY, AMF_VIDEO_ENCODER_USAGE_LOW_LATENCY_HIGH_QUALITY }; enum AMF_VIDEO_ENCODER_PROFILE_ENUM { AMF_VIDEO_ENCODER_PROFILE_UNKNOWN = 0, AMF_VIDEO_ENCODER_PROFILE_BASELINE = 66, AMF_VIDEO_ENCODER_PROFILE_MAIN = 77, AMF_VIDEO_ENCODER_PROFILE_HIGH = 100, AMF_VIDEO_ENCODER_PROFILE_CONSTRAINED_BASELINE = 256, AMF_VIDEO_ENCODER_PROFILE_CONSTRAINED_HIGH = 257 }; enum AMF_VIDEO_ENCODER_H264_LEVEL_ENUM { AMF_H264_LEVEL__1 = 10, AMF_H264_LEVEL__1_1 = 11, AMF_H264_LEVEL__1_2 = 12, AMF_H264_LEVEL__1_3 = 13, AMF_H264_LEVEL__2 = 20, AMF_H264_LEVEL__2_1 = 21, AMF_H264_LEVEL__2_2 = 22, AMF_H264_LEVEL__3 = 30, AMF_H264_LEVEL__3_1 = 31, AMF_H264_LEVEL__3_2 = 32, AMF_H264_LEVEL__4 = 40, AMF_H264_LEVEL__4_1 = 41, AMF_H264_LEVEL__4_2 = 42, AMF_H264_LEVEL__5 = 50, AMF_H264_LEVEL__5_1 = 51, AMF_H264_LEVEL__5_2 = 52, AMF_H264_LEVEL__6 = 60, AMF_H264_LEVEL__6_1 = 61, AMF_H264_LEVEL__6_2 = 62 }; enum AMF_VIDEO_ENCODER_SCANTYPE_ENUM { AMF_VIDEO_ENCODER_SCANTYPE_PROGRESSIVE = 0, AMF_VIDEO_ENCODER_SCANTYPE_INTERLACED }; enum AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_ENUM { AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_UNKNOWN = -1, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_CONSTANT_QP = 0, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_CBR, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_PEAK_CONSTRAINED_VBR, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_LATENCY_CONSTRAINED_VBR, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_QUALITY_VBR, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_HIGH_QUALITY_VBR, AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_HIGH_QUALITY_CBR }; enum AMF_VIDEO_ENCODER_QUALITY_PRESET_ENUM { AMF_VIDEO_ENCODER_QUALITY_PRESET_BALANCED = 0, AMF_VIDEO_ENCODER_QUALITY_PRESET_SPEED, AMF_VIDEO_ENCODER_QUALITY_PRESET_QUALITY }; enum AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_ENUM { AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_NONE = 0, AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_FRAME, AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_TOP_FIELD, AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_BOTTOM_FIELD }; enum AMF_VIDEO_ENCODER_PICTURE_TYPE_ENUM { AMF_VIDEO_ENCODER_PICTURE_TYPE_NONE = 0, AMF_VIDEO_ENCODER_PICTURE_TYPE_SKIP, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR, AMF_VIDEO_ENCODER_PICTURE_TYPE_I, AMF_VIDEO_ENCODER_PICTURE_TYPE_P, AMF_VIDEO_ENCODER_PICTURE_TYPE_B }; enum AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_ENUM { AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_IDR, AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_I, AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_P, AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_B }; enum AMF_VIDEO_ENCODER_PREENCODE_MODE_ENUM { AMF_VIDEO_ENCODER_PREENCODE_DISABLED = 0, AMF_VIDEO_ENCODER_PREENCODE_ENABLED = 1, }; enum AMF_VIDEO_ENCODER_CODING_ENUM { AMF_VIDEO_ENCODER_UNDEFINED = 0, // BASELINE = CALV; MAIN, HIGH = CABAC AMF_VIDEO_ENCODER_CABAC, AMF_VIDEO_ENCODER_CALV, }; enum AMF_VIDEO_ENCODER_PICTURE_TRANSFER_MODE_ENUM { AMF_VIDEO_ENCODER_PICTURE_TRANSFER_MODE_OFF = 0, AMF_VIDEO_ENCODER_PICTURE_TRANSFER_MODE_ON }; enum AMF_VIDEO_ENCODER_LTR_MODE_ENUM { AMF_VIDEO_ENCODER_LTR_MODE_RESET_UNUSED = 0, AMF_VIDEO_ENCODER_LTR_MODE_KEEP_UNUSED }; enum AMF_VIDEO_ENCODER_OUTPUT_MODE_ENUM { AMF_VIDEO_ENCODER_OUTPUT_MODE_FRAME = 0, AMF_VIDEO_ENCODER_OUTPUT_MODE_SLICE = 1 }; enum AMF_VIDEO_ENCODER_OUTPUT_BUFFER_TYPE_ENUM { AMF_VIDEO_ENCODER_OUTPUT_BUFFER_TYPE_FRAME = 0, AMF_VIDEO_ENCODER_OUTPUT_BUFFER_TYPE_SLICE = 1, AMF_VIDEO_ENCODER_OUTPUT_BUFFER_TYPE_SLICE_LAST = 2 }; // Static properties - can be set before Init() #define AMF_VIDEO_ENCODER_INSTANCE_INDEX L"EncoderInstance" // amf_int64; selected HW instance idx #define AMF_VIDEO_ENCODER_FRAMESIZE L"FrameSize" // AMFSize; default = 0,0; Frame size #define AMF_VIDEO_ENCODER_EXTRADATA L"ExtraData" // AMFInterface* - > AMFBuffer*; SPS/PPS buffer in Annex B format - read-only #define AMF_VIDEO_ENCODER_USAGE L"Usage" // amf_int64(AMF_VIDEO_ENCODER_USAGE_ENUM); default = N/A; Encoder usage type. fully configures parameter set. #define AMF_VIDEO_ENCODER_PROFILE L"Profile" // amf_int64(AMF_VIDEO_ENCODER_PROFILE_ENUM) ; default = AMF_VIDEO_ENCODER_PROFILE_MAIN; H264 profile #define AMF_VIDEO_ENCODER_PROFILE_LEVEL L"ProfileLevel" // amf_int64(AMF_VIDEO_ENCODER_H264_LEVEL_ENUM); default = AMF_H264_LEVEL__4_2; H264 level #define AMF_VIDEO_ENCODER_MAX_LTR_FRAMES L"MaxOfLTRFrames" // amf_int64; default = 0; Max number of LTR frames #define AMF_VIDEO_ENCODER_LTR_MODE L"LTRMode" // amf_int64(AMF_VIDEO_ENCODER_LTR_MODE_ENUM); default = AMF_VIDEO_ENCODER_LTR_MODE_RESET_UNUSED; remove/keep unused LTRs (not specified in property AMF_VIDEO_ENCODER_FORCE_LTR_REFERENCE_BITFIELD) #define AMF_VIDEO_ENCODER_SCANTYPE L"ScanType" // amf_int64(AMF_VIDEO_ENCODER_SCANTYPE_ENUM); default = AMF_VIDEO_ENCODER_SCANTYPE_PROGRESSIVE; indicates input stream type #define AMF_VIDEO_ENCODER_MAX_NUM_REFRAMES L"MaxNumRefFrames" // amf_int64; Maximum number of reference frames #define AMF_VIDEO_ENCODER_MAX_CONSECUTIVE_BPICTURES L"MaxConsecutiveBPictures" // amf_int64; Maximum number of consecutive B Pictures #define AMF_VIDEO_ENCODER_ADAPTIVE_MINIGOP L"AdaptiveMiniGOP" // bool; default = false; Disable/Enable Adaptive MiniGOP #define AMF_VIDEO_ENCODER_ASPECT_RATIO L"AspectRatio" // AMFRatio; default = 1, 1 #define AMF_VIDEO_ENCODER_FULL_RANGE_COLOR L"FullRangeColor" // bool; default = false; inidicates that YUV input is (0,255) #define AMF_VIDEO_ENCODER_LOWLATENCY_MODE L"LowLatencyInternal" // bool; default = false, enables low latency mode and POC mode 2 in the encoder #define AMF_VIDEO_ENCODER_PRE_ANALYSIS_ENABLE L"EnablePreAnalysis" // bool; default = false; enables the pre-analysis module. Currently only works in AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_PEAK_CONSTRAINED_VBR mode. Refer to AMF Video PreAnalysis API reference for more details. #define AMF_VIDEO_ENCODER_PREENCODE_ENABLE L"RateControlPreanalysisEnable" // amf_int64(AMF_VIDEO_ENCODER_PREENCODE_MODE_ENUM); default = AMF_VIDEO_ENCODER_PREENCODE_DISABLED; enables pre-encode assisted rate control #define AMF_VIDEO_ENCODER_RATE_CONTROL_PREANALYSIS_ENABLE L"RateControlPreanalysisEnable" // amf_int64(AMF_VIDEO_ENCODER_PREENCODE_MODE_ENUM); default = AMF_VIDEO_ENCODER_PREENCODE_DISABLED; enables pre-encode assisted rate control. Deprecated, please use AMF_VIDEO_ENCODER_PREENCODE_ENABLE instead. #define AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD L"RateControlMethod" // amf_int64(AMF_VIDEO_ENCODER_RATE_CONTROL_METHOD_ENUM); default = depends on USAGE; Rate Control Method #define AMF_VIDEO_ENCODER_QVBR_QUALITY_LEVEL L"QvbrQualityLevel" // amf_int64; default = 23; QVBR quality level; range = 1-51 #define AMF_VIDEO_ENCODER_MAX_NUM_TEMPORAL_LAYERS L"MaxNumOfTemporalLayers" // amf_int64; default = 1; Max number of temporal layers. #if !defined(__GNUC__) && !defined(__clang__) #pragma deprecated("AMF_VIDEO_ENCODER_RATE_CONTROL_PREANALYSIS_ENABLE") #endif // Quality preset property #define AMF_VIDEO_ENCODER_QUALITY_PRESET L"QualityPreset" // amf_int64(AMF_VIDEO_ENCODER_QUALITY_PRESET_ENUM); default = depends on USAGE; Quality Preset // color conversion #define AMF_VIDEO_ENCODER_COLOR_BIT_DEPTH L"ColorBitDepth" // amf_int64(AMF_COLOR_BIT_DEPTH_ENUM); default = AMF_COLOR_BIT_DEPTH_8 #define AMF_VIDEO_ENCODER_INPUT_COLOR_PROFILE L"InColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO by size #define AMF_VIDEO_ENCODER_INPUT_TRANSFER_CHARACTERISTIC L"InColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 ?7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_ENCODER_INPUT_COLOR_PRIMARIES L"InColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 Section 7.1 See ColorSpace.h for enum #define AMF_VIDEO_ENCODER_INPUT_HDR_METADATA L"InHDRMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL #define AMF_VIDEO_ENCODER_OUTPUT_COLOR_PROFILE L"OutColorProfile" // amf_int64(AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM); default = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN - mean AUTO by size #define AMF_VIDEO_ENCODER_OUTPUT_TRANSFER_CHARACTERISTIC L"OutColorTransferChar" // amf_int64(AMF_COLOR_TRANSFER_CHARACTERISTIC_ENUM); default = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, ISO/IEC 23001-8_2013 Section 7.2 See VideoDecoderUVD.h for enum #define AMF_VIDEO_ENCODER_OUTPUT_COLOR_PRIMARIES L"OutColorPrimaries" // amf_int64(AMF_COLOR_PRIMARIES_ENUM); default = AMF_COLOR_PRIMARIES_UNDEFINED, ISO/IEC 23001-8_2013 Section 7.1 See ColorSpace.h for enum #define AMF_VIDEO_ENCODER_OUTPUT_HDR_METADATA L"OutHDRMetadata" // AMFBuffer containing AMFHDRMetadata; default NULL // Slice output #define AMF_VIDEO_ENCODER_OUTPUT_MODE L"OutputMode" // amf_int64(AMF_VIDEO_ENCODER_OUTPUT_MODE_ENUM); default = AMF_VIDEO_ENCODER_OUTPUT_MODE_FRAME - defines encoder output mode // Dynamic properties - can be set at any time // Rate control properties #define AMF_VIDEO_ENCODER_FRAMERATE L"FrameRate" // AMFRate; default = depends on usage; Frame Rate #define AMF_VIDEO_ENCODER_B_PIC_DELTA_QP L"BPicturesDeltaQP" // amf_int64; default = depends on USAGE; B-picture Delta #define AMF_VIDEO_ENCODER_REF_B_PIC_DELTA_QP L"ReferenceBPicturesDeltaQP"// amf_int64; default = depends on USAGE; Reference B-picture Delta #define AMF_VIDEO_ENCODER_ENFORCE_HRD L"EnforceHRD" // bool; default = depends on USAGE; Enforce HRD #define AMF_VIDEO_ENCODER_FILLER_DATA_ENABLE L"FillerDataEnable" // bool; default = false; Filler Data Enable #define AMF_VIDEO_ENCODER_ENABLE_VBAQ L"EnableVBAQ" // bool; default = depends on USAGE; Enable VBAQ #define AMF_VIDEO_ENCODER_HIGH_MOTION_QUALITY_BOOST_ENABLE L"HighMotionQualityBoostEnable"// bool; default = depends on USAGE; Enable High motion quality boost mode #define AMF_VIDEO_ENCODER_VBV_BUFFER_SIZE L"VBVBufferSize" // amf_int64; default = depends on USAGE; VBV Buffer Size in bits #define AMF_VIDEO_ENCODER_INITIAL_VBV_BUFFER_FULLNESS L"InitialVBVBufferFullness" // amf_int64; default = 64; Initial VBV Buffer Fullness 0=0% 64=100% #define AMF_VIDEO_ENCODER_MAX_AU_SIZE L"MaxAUSize" // amf_int64; default = 0; Max AU Size in bits #define AMF_VIDEO_ENCODER_MIN_QP L"MinQP" // amf_int64; default = depends on USAGE; Min QP; range = 0-51 #define AMF_VIDEO_ENCODER_MAX_QP L"MaxQP" // amf_int64; default = depends on USAGE; Max QP; range = 0-51 #define AMF_VIDEO_ENCODER_QP_I L"QPI" // amf_int64; default = 22; I-frame QP; range = 0-51 #define AMF_VIDEO_ENCODER_QP_P L"QPP" // amf_int64; default = 22; P-frame QP; range = 0-51 #define AMF_VIDEO_ENCODER_QP_B L"QPB" // amf_int64; default = 22; B-frame QP; range = 0-51 #define AMF_VIDEO_ENCODER_TARGET_BITRATE L"TargetBitrate" // amf_int64; default = depends on USAGE; Target bit rate in bits #define AMF_VIDEO_ENCODER_PEAK_BITRATE L"PeakBitrate" // amf_int64; default = depends on USAGE; Peak bit rate in bits #define AMF_VIDEO_ENCODER_RATE_CONTROL_SKIP_FRAME_ENABLE L"RateControlSkipFrameEnable" // bool; default = depends on USAGE; Rate Control Based Frame Skip // Picture control properties #define AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING L"HeaderInsertionSpacing" // amf_int64; default = depends on USAGE; Header Insertion Spacing; range 0-1000 #define AMF_VIDEO_ENCODER_B_PIC_PATTERN L"BPicturesPattern" // amf_int64; default = 0; B-picture Pattern (number of B-Frames) #define AMF_VIDEO_ENCODER_DE_BLOCKING_FILTER L"DeBlockingFilter" // bool; default = depends on USAGE; De-blocking Filter #define AMF_VIDEO_ENCODER_B_REFERENCE_ENABLE L"BReferenceEnable" // bool; default = true; Enable Refrence to B-frames #define AMF_VIDEO_ENCODER_IDR_PERIOD L"IDRPeriod" // amf_int64; default = depends on USAGE; IDR Period in frames #define AMF_VIDEO_ENCODER_INTRA_PERIOD L"IntraPeriod" // amf_int64; default = 0; Intra period in frames #define AMF_VIDEO_ENCODER_INTRA_REFRESH_NUM_MBS_PER_SLOT L"IntraRefreshMBsNumberPerSlot" // amf_int64; default = depends on USAGE; Intra Refresh MBs Number Per Slot in Macroblocks #define AMF_VIDEO_ENCODER_SLICES_PER_FRAME L"SlicesPerFrame" // amf_int64; default = 1; Number of slices Per Frame #define AMF_VIDEO_ENCODER_CABAC_ENABLE L"CABACEnable" // amf_int64(AMF_VIDEO_ENCODER_CODING_ENUM) default = AMF_VIDEO_ENCODER_UNDEFINED // Motion estimation #define AMF_VIDEO_ENCODER_MOTION_HALF_PIXEL L"HalfPixel" // bool; default= true; Half Pixel #define AMF_VIDEO_ENCODER_MOTION_QUARTERPIXEL L"QuarterPixel" // bool; default= true; Quarter Pixel // SVC #define AMF_VIDEO_ENCODER_NUM_TEMPORAL_ENHANCMENT_LAYERS L"NumOfTemporalEnhancmentLayers" // amf_int64; default = 1; range = 1-MaxTemporalLayers; Number of temporal Layers (SVC) // DPB management #define AMF_VIDEO_ENCODER_PICTURE_TRANSFER_MODE L"PicTransferMode" // amf_int64(AMF_VIDEO_ENCODER_PICTURE_TRANSFER_MODE_ENUM); default = AMF_VIDEO_ENCODER_PICTURE_TRANSFER_MODE_OFF - whether to exchange reference/reconstructed pic between encoder and application // misc #define AMF_VIDEO_ENCODER_QUERY_TIMEOUT L"QueryTimeout" // amf_int64; default = 0 (no wait); timeout for QueryOutput call in ms. #define AMF_VIDEO_ENCODER_MEMORY_TYPE L"EncoderMemoryType" // amf_int64(AMF_MEMORY_TYPE) , default is AMF_MEMORY_UNKNOWN, Values : AMF_MEMORY_DX11, AMF_MEMORY_DX9, AMF_MEMORY_VULKAN or AMF_MEMORY_UNKNOWN (auto) #define AMF_VIDEO_ENCODER_ENABLE_SMART_ACCESS_VIDEO L"EnableEncoderSmartAccessVideo" // amf_bool; default = false; true = enables smart access video feature #define AMF_VIDEO_ENCODER_INPUT_QUEUE_SIZE L"InputQueueSize" // amf_int64; default 16; Set amf input queue size // Per-submission properties - can be set on input surface interface #define AMF_VIDEO_ENCODER_END_OF_SEQUENCE L"EndOfSequence" // bool; default = false; generate end of sequence #define AMF_VIDEO_ENCODER_END_OF_STREAM L"EndOfStream" // bool; default = false; generate end of stream #define AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE L"ForcePictureType" // amf_int64(AMF_VIDEO_ENCODER_PICTURE_TYPE_ENUM); default = AMF_VIDEO_ENCODER_PICTURE_TYPE_NONE; generate particular picture type #define AMF_VIDEO_ENCODER_INSERT_AUD L"InsertAUD" // bool; default = false; insert AUD #define AMF_VIDEO_ENCODER_INSERT_SPS L"InsertSPS" // bool; default = false; insert SPS #define AMF_VIDEO_ENCODER_INSERT_PPS L"InsertPPS" // bool; default = false; insert PPS #define AMF_VIDEO_ENCODER_PICTURE_STRUCTURE L"PictureStructure" // amf_int64(AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_ENUM); default = AMF_VIDEO_ENCODER_PICTURE_STRUCTURE_FRAME; indicate picture type #define AMF_VIDEO_ENCODER_MARK_CURRENT_WITH_LTR_INDEX L"MarkCurrentWithLTRIndex" // //amf_int64; default = N/A; Mark current frame with LTR index #define AMF_VIDEO_ENCODER_FORCE_LTR_REFERENCE_BITFIELD L"ForceLTRReferenceBitfield"// amf_int64; default = 0; force LTR bit-field #define AMF_VIDEO_ENCODER_ROI_DATA L"ROIData" // 2D AMFSurface, surface format: AMF_SURFACE_GRAY32 #define AMF_VIDEO_ENCODER_REFERENCE_PICTURE L"ReferencePicture" // AMFInterface(AMFSurface); surface used for frame injection #define AMF_VIDEO_ENCODER_PSNR_FEEDBACK L"PSNRFeedback" // amf_bool; default = false; Signal encoder to calculate PSNR score #define AMF_VIDEO_ENCODER_SSIM_FEEDBACK L"SSIMFeedback" // amf_bool; default = false; Signal encoder to calculate SSIM score #define AMF_VIDEO_ENCODER_STATISTICS_FEEDBACK L"StatisticsFeedback" // amf_bool; default = false; Signal encoder to collect and feedback statistics #define AMF_VIDEO_ENCODER_BLOCK_QP_FEEDBACK L"BlockQpFeedback" // amf_bool; default = false; Signal encoder to collect and feedback block level QP values // properties set by encoder on output buffer interface #define AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE L"OutputDataType" // amf_int64(AMF_VIDEO_ENCODER_OUTPUT_DATA_TYPE_ENUM); default = N/A #define AMF_VIDEO_ENCODER_OUTPUT_MARKED_LTR_INDEX L"MarkedLTRIndex" //amf_int64; default = -1; Marked LTR index #define AMF_VIDEO_ENCODER_OUTPUT_REFERENCED_LTR_INDEX_BITFIELD L"ReferencedLTRIndexBitfield" // amf_int64; default = 0; referenced LTR bit-field #define AMF_VIDEO_ENCODER_OUTPUT_TEMPORAL_LAYER L"OutputTemporalLayer" // amf_int64; Temporal layer #define AMF_VIDEO_ENCODER_OUTPUT_BUFFER_TYPE L"OutputBufferType" // amf_int64(AMF_VIDEO_ENCODER_OUTPUT_BUFFER_TYPE_ENUM); encoder output buffer type #define AMF_VIDEO_ENCODER_PRESENTATION_TIME_STAMP L"PresentationTimeStamp" // amf_int64; Presentation time stamp (PTS) #define AMF_VIDEO_ENCODER_RECONSTRUCTED_PICTURE L"ReconstructedPicture" // AMFInterface(AMFSurface); returns reconstructed picture as an AMFSurface attached to the output buffer as property AMF_VIDEO_ENCODER_RECONSTRUCTED_PICTURE of AMFInterface type #define AMF_VIDEO_ENCODER_STATISTIC_PSNR_Y L"PSNRY" // double; PSNR Y #define AMF_VIDEO_ENCODER_STATISTIC_PSNR_U L"PSNRU" // double; PSNR U #define AMF_VIDEO_ENCODER_STATISTIC_PSNR_V L"PSNRV" // double; PSNR V #define AMF_VIDEO_ENCODER_STATISTIC_PSNR_ALL L"PSNRALL" // double; PSNR All #define AMF_VIDEO_ENCODER_STATISTIC_SSIM_Y L"SSIMY" // double; SSIM Y #define AMF_VIDEO_ENCODER_STATISTIC_SSIM_U L"SSIMU" // double; SSIM U #define AMF_VIDEO_ENCODER_STATISTIC_SSIM_V L"SSIMV" // double; SSIM V #define AMF_VIDEO_ENCODER_STATISTIC_SSIM_ALL L"SSIMALL" // double; SSIM ALL // Encoder statistics feedback #define AMF_VIDEO_ENCODER_STATISTIC_FRAME_QP L"StatisticsFeedbackFrameQP" // amf_int64; Rate control base frame/initial QP #define AMF_VIDEO_ENCODER_STATISTIC_AVERAGE_QP L"StatisticsFeedbackAvgQP" // amf_int64; Average calculated QP of all encoded MBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped MBs. #define AMF_VIDEO_ENCODER_STATISTIC_MAX_QP L"StatisticsFeedbackMaxQP" // amf_int64; Max calculated QP among all encoded MBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped MBs. #define AMF_VIDEO_ENCODER_STATISTIC_MIN_QP L"StatisticsFeedbackMinQP" // amf_int64; Min calculated QP among all encoded MBs in a picture. Value may be different from the one reported by bitstream analyzer when there are skipped MBs. #define AMF_VIDEO_ENCODER_STATISTIC_PIX_NUM_INTRA L"StatisticsFeedbackPixNumIntra" // amf_int64; Number of the intra encoded pixels #define AMF_VIDEO_ENCODER_STATISTIC_PIX_NUM_INTER L"StatisticsFeedbackPixNumInter" // amf_int64; Number of the inter encoded pixels #define AMF_VIDEO_ENCODER_STATISTIC_PIX_NUM_SKIP L"StatisticsFeedbackPixNumSkip" // amf_int64; Number of the skip mode pixels #define AMF_VIDEO_ENCODER_STATISTIC_BITCOUNT_RESIDUAL L"StatisticsFeedbackBitcountResidual" // amf_int64; The bit count that corresponds to residual data #define AMF_VIDEO_ENCODER_STATISTIC_BITCOUNT_MOTION L"StatisticsFeedbackBitcountMotion" // amf_int64; The bit count that corresponds to motion vectors #define AMF_VIDEO_ENCODER_STATISTIC_BITCOUNT_INTER L"StatisticsFeedbackBitcountInter" // amf_int64; The bit count that are assigned to inter MBs #define AMF_VIDEO_ENCODER_STATISTIC_BITCOUNT_INTRA L"StatisticsFeedbackBitcountIntra" // amf_int64; The bit count that are assigned to intra MBs #define AMF_VIDEO_ENCODER_STATISTIC_BITCOUNT_ALL_MINUS_HEADER L"StatisticsFeedbackBitcountAllMinusHeader" // amf_int64; The bit count of the bitstream excluding header #define AMF_VIDEO_ENCODER_STATISTIC_MV_X L"StatisticsFeedbackMvX" // amf_int64; Accumulated absolute values of horizontal MV's #define AMF_VIDEO_ENCODER_STATISTIC_MV_Y L"StatisticsFeedbackMvY" // amf_int64; Accumulated absolute values of vertical MV's #define AMF_VIDEO_ENCODER_STATISTIC_RD_COST_FINAL L"StatisticsFeedbackRdCostFinal" // amf_int64; Frame level final RD cost for full encoding #define AMF_VIDEO_ENCODER_STATISTIC_RD_COST_INTRA L"StatisticsFeedbackRdCostIntra" // amf_int64; Frame level intra RD cost for full encoding #define AMF_VIDEO_ENCODER_STATISTIC_RD_COST_INTER L"StatisticsFeedbackRdCostInter" // amf_int64; Frame level inter RD cost for full encoding #define AMF_VIDEO_ENCODER_STATISTIC_SATD_FINAL L"StatisticsFeedbackSatdFinal" // amf_int64; Frame level final SATD for full encoding #define AMF_VIDEO_ENCODER_STATISTIC_SATD_INTRA L"StatisticsFeedbackSatdIntra" // amf_int64; Frame level intra SATD for full encoding #define AMF_VIDEO_ENCODER_STATISTIC_SATD_INTER L"StatisticsFeedbackSatdInter" // amf_int64; Frame level inter SATD for full encoding #define AMF_VIDEO_ENCODER_STATISTIC_VARIANCE L"StatisticsFeedbackVariance" // amf_int64; Frame level variance for full encoding // Encoder block level feedback #define AMF_VIDEO_ENCODER_BLOCK_QP_MAP L"BlockQpMap" // AMFInterface(AMFSurface); AMFSurface of format AMF_SURFACE_GRAY32 containing block level QP values #define AMF_VIDEO_ENCODER_HDCP_COUNTER L"HDCPCounter" // const void* // Properties for multi-instance cloud gaming #define AMF_VIDEO_ENCODER_MAX_INSTANCES L"EncoderMaxInstances" // deprecated. amf_int64; default = 1; max number of encoder instances #define AMF_VIDEO_ENCODER_MULTI_INSTANCE_MODE L"MultiInstanceMode" // deprecated. bool; default = false; #define AMF_VIDEO_ENCODER_CURRENT_QUEUE L"MultiInstanceCurrentQueue"// deprecated. amf_int64; default = 0; // VCE Encoder capabilities - exposed in AMFCaps interface #define AMF_VIDEO_ENCODER_CAP_MAX_BITRATE L"MaxBitrate" // amf_int64; Maximum bit rate in bits #define AMF_VIDEO_ENCODER_CAP_NUM_OF_STREAMS L"NumOfStreams" // amf_int64; maximum number of encode streams supported #define AMF_VIDEO_ENCODER_CAP_MAX_PROFILE L"MaxProfile" // AMF_VIDEO_ENCODER_PROFILE_ENUM #define AMF_VIDEO_ENCODER_CAP_MAX_LEVEL L"MaxLevel" // amf_int64 maximum profile level #define AMF_VIDEO_ENCODER_CAP_BFRAMES L"BFrames" // bool is B-Frames supported #define AMF_VIDEO_ENCODER_CAP_MIN_REFERENCE_FRAMES L"MinReferenceFrames" // amf_int64 minimum number of reference frames #define AMF_VIDEO_ENCODER_CAP_MAX_REFERENCE_FRAMES L"MaxReferenceFrames" // amf_int64 maximum number of reference frames #define AMF_VIDEO_ENCODER_CAP_MAX_TEMPORAL_LAYERS L"MaxTemporalLayers" // amf_int64 maximum number of temporal layers #define AMF_VIDEO_ENCODER_CAP_FIXED_SLICE_MODE L"FixedSliceMode" // bool is fixed slice mode supported #define AMF_VIDEO_ENCODER_CAP_NUM_OF_HW_INSTANCES L"NumOfHwInstances" // amf_int64 number of HW encoder instances #define AMF_VIDEO_ENCODER_CAP_COLOR_CONVERSION L"ColorConversion" // amf_int64(AMF_ACCELERATION_TYPE) - type of supported color conversion. default AMF_ACCEL_GPU #define AMF_VIDEO_ENCODER_CAP_PRE_ANALYSIS L"PreAnalysis" // amf_bool - pre analysis module is available for H264 UVE encoder, n/a for the other encoders #define AMF_VIDEO_ENCODER_CAP_ROI L"ROIMap" // amf_bool - ROI map support is available for H264 UVE encoder, n/a for the other encoders #define AMF_VIDEO_ENCODER_CAP_MAX_THROUGHPUT L"MaxThroughput" // amf_int64 - MAX throughput for H264 encoder in MB (16 x 16 pixel) #define AMF_VIDEO_ENCODER_CAP_REQUESTED_THROUGHPUT L"RequestedThroughput" // amf_int64 - Currently total requested throughput for H264 encoder in MB (16 x 16 pixel) #define AMF_VIDEO_ENCODER_CAPS_QUERY_TIMEOUT_SUPPORT L"QueryTimeoutSupport" // amf_bool - Timeout supported for QueryOutout call (Deprecated, please use AMF_VIDEO_ENCODER_CAP_QUERY_TIMEOUT_SUPPORT ) #define AMF_VIDEO_ENCODER_CAP_QUERY_TIMEOUT_SUPPORT L"QueryTimeoutSupport" // amf_bool - Timeout supported for QueryOutout call #define AMF_VIDEO_ENCODER_CAP_SUPPORT_SLICE_OUTPUT L"SupportSliceOutput" // amf_bool - if slice output is supported #define AMF_VIDEO_ENCODER_CAP_SUPPORT_SMART_ACCESS_VIDEO L"EncoderSupportSmartAccessVideo" // amf_bool; returns true if system supports SmartAccess Video #endif //#ifndef AMF_VideoEncoderVCE_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/VideoStitch.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file VideoStitch.h * @brief AMFVideoStitch interface declaration *************************************************************************************************** */ #ifndef AMF_VideoStitch_h #define AMF_VideoStitch_h #pragma once #include "public/include/components/Component.h" #define AMFVideoStitch L"AMFVideoStitch" //Component name // static properties #define AMF_VIDEO_STITCH_OUTPUT_FORMAT L"OutputFormat" // Values, AMF_SURFACE_BGRA or AMF_SURFACE_RGBA #define AMF_VIDEO_STITCH_MEMORY_TYPE L"MemoryType" // Values, only AMF_MEMORY_DX11 is supported for now. #define AMF_VIDEO_STITCH_OUTPUT_SIZE L"OutputSize" // AMFSize, (width, height) in pixels. default= (0,0), will be the same size as input. #define AMF_VIDEO_STITCH_INPUTCOUNT L"InputCount" // amf_uint64, number of camera inputs. // individual camera direction and location #define AMF_VIDEO_CAMERA_ANGLE_PITCH L"CameraPitch" // double, in radians, default = 0, camera pitch orientation #define AMF_VIDEO_CAMERA_ANGLE_YAW L"CameraYaw" // double, in radians, default = 0, camera yaw orientation #define AMF_VIDEO_CAMERA_ANGLE_ROLL L"CameraRoll" // double, in radians, default = 0, camera roll orientation #define AMF_VIDEO_CAMERA_OFFSET_X L"CameraOffsetX" // double, in pixels, default = 0, X offset of camera center of the lens from the center of the rig. #define AMF_VIDEO_CAMERA_OFFSET_Y L"CameraOffsetY" // double, in pixels, default = 0, Y offset of camera center of the lens from the center of the rig. #define AMF_VIDEO_CAMERA_OFFSET_Z L"CameraOffsetZ" // double, in pixels, default = 0, Z offset of camera center of the lens from the center of the rig. #define AMF_VIDEO_CAMERA_HFOV L"CameraHFOV" // double, in radians, default = PI, - horizontal field of view #define AMF_VIDEO_CAMERA_SCALE L"CameraScale" // double, default = 1, scale coeff // lens correction parameters #define AMF_VIDEO_STITCH_LENS_CORR_K1 L"LensK1" // double, default = 0. #define AMF_VIDEO_STITCH_LENS_CORR_K2 L"LensK2" // double, default = 0. #define AMF_VIDEO_STITCH_LENS_CORR_K3 L"LensK3" // double, default = 0. #define AMF_VIDEO_STITCH_LENS_CORR_OFFX L"LensOffX" // double, default = 0. #define AMF_VIDEO_STITCH_LENS_CORR_OFFY L"LensOffY" // double, default = 0. #define AMF_VIDEO_STITCH_CROP L"Crop" //AMFRect, in pixels default = (0,0,0,0). #define AMF_VIDEO_STITCH_LENS_MODE L"LensMode" // Values, AMF_VIDEO_STITCH_LENS_CORR_MODE_ENUM, (default = AMF_VIDEO_STITCH_LENS_CORR_MODE_RADIAL) #define AMF_VIDEO_STITCH_OUTPUT_MODE L"OutputMode" // AMF_VIDEO_STITCH_OUTPUT_MODE_ENUM (default=AMF_VIDEO_STITCH_OUTPUT_MODE_PREVIEW) #define AMF_VIDEO_STITCH_COMBINED_SOURCE L"CombinedSource" // bool, (default=false) video sources are combined in one stream #define AMF_VIDEO_STITCH_COMPUTE_DEVICE L"ComputeDevice" // amf_int64(AMF_MEMORY_TYPE) Values, AMF_MEMORY_DX11, AMF_MEMORY_COMPUTE_FOR_DX11, AMF_MEMORY_OPENCL //for debug #define AMF_VIDEO_STITCH_WIRE_RENDER L"Wire" // bool (default=false) reder wireframe //view angle #define AMF_VIDEO_STITCH_VIEW_ROTATE_X L"AngleX" // double, in radians, default = 0 - delta from current position / automatilcally reset to 0 inside SetProperty() call #define AMF_VIDEO_STITCH_VIEW_ROTATE_Y L"AngleY" // double, in radians, default = 0 - delta from current position / automatilcally reset to 0 inside SetProperty() call #define AMF_VIDEO_STITCH_VIEW_ROTATE_Z L"AngleZ" // double, in radians, default = 0 - delta from current position / automatilcally reset to 0 inside SetProperty() call #define AMF_VIDEO_STITCH_COLOR_BALANCE L"ColorBalance" // bool (default=true) enables color balance //lens mode enum AMF_VIDEO_STITCH_LENS_ENUM { AMF_VIDEO_STITCH_LENS_RECTILINEAR = 0, //rect linear lens AMF_VIDEO_STITCH_LENS_FISHEYE_FULLFRAME = 1, //fisheye full frame AMF_VIDEO_STITCH_LENS_FISHEYE_CIRCULAR = 2, //fisheye, circular }; //Output Mode enum AMF_VIDEO_STITCH_OUTPUT_MODE_ENUM { AMF_VIDEO_STITCH_OUTPUT_MODE_PREVIEW = 0, //preview mode AMF_VIDEO_STITCH_OUTPUT_MODE_EQUIRECTANGULAR = 1, //equirectangular mode AMF_VIDEO_STITCH_OUTPUT_MODE_CUBEMAP = 2, //cubemap mode AMF_VIDEO_STITCH_OUTPUT_MODE_LAST = AMF_VIDEO_STITCH_OUTPUT_MODE_CUBEMAP, }; //audio mode enum AMF_VIDEO_STITCH_AUDIO_MODE_ENUM { AMF_VIDEO_STITCH_AUDIO_MODE_NONE = 0, //no audio AMF_VIDEO_STITCH_AUDIO_MODE_VIDEO = 1, //using audio from video stream AMF_VIDEO_STITCH_AUDIO_MODE_FILE = 2, //using audio from file AMF_VIDEO_STITCH_AUDIO_MODE_CAPTURE = 3, //using audio from capture device AMF_VIDEO_STITCH_AUDIO_MODE_INVALID = -1, //invalid }; #if defined(_M_AMD64) #define STITCH_DLL_NAME L"amf-stitch-64.dll" #else #define STITCH_DLL_NAME L"amf-stitch-32.dll" #endif #endif //#ifndef AMF_VideoStitch_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/components/ZCamLiveStream.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // //------------------------------------------------------------------------------------------------- // ZCamLive interface declaration //------------------------------------------------------------------------------------------------- #ifndef AMF_ZCamLiveStream_h #define AMF_ZCamLiveStream_h #pragma once #define ZCAMLIVE_STREAMCOUNT L"StreamCount" // amf_int64 (default = 4), number of streams #define ZCAMLIVE_VIDEO_FRAMESIZE L"FrameSize" // AMFSize (default = AMFConstructSize(2704, 1520)), frame size #define ZCAMLIVE_VIDEO_FRAMERATE L"FrameRate" // AMFRate (default = 30.0), video frame rate #define ZCAMLIVE_VIDEO_BIT_RATE L"BitRate" // amf_int64 (default = 3000000), video bitrate #define ZCAMLIVE_STREAM_ACTIVE_CAMERA L"ActiveCamera" // amf_int64 (default = -1, all the cameras), the index of the camera to capture #define ZCAMLIVE_STREAM_FRAMECOUNT L"FrameCount" // amf_int64 (default = 0), number of frames captured #define ZCAMLIVE_CODEC_ID L"CodecID" // WString (default = "AMFVideoDecoderUVD_H264_AVC"), UVD codec ID #define ZCAMLIVE_VIDEO_MODE L"VideoMode" // Enum (default = 0, 2K7P30), ZCam mode #define ZCAMLIVE_AUDIO_MODE L"AudioMode" // Enum (default = 0, Silent) - Audio mode #define ZCAMLIVE_LOWLATENCY L"LowLatency" // amf_int64 (default = 1, LowLatency), low latency flag #define ZCAMLIVE_IP_0 L"ZCamIP_00" // WString, IP address of the #1 stream, default "10.98.32.1" #define ZCAMLIVE_IP_1 L"ZCamIP_01" // WString, IP address of the #2 stream, default "10.98.32.2" #define ZCAMLIVE_IP_2 L"ZCamIP_02" // WString, IP address of the #3 stream, default "10.98.32.3" #define ZCAMLIVE_IP_3 L"ZCamIP_03" // WString, IP address of the #4 stream, default "10.98.32.4" //Camera live capture Mode enum CAMLIVE_MODE_ENUM { CAMLIVE_MODE_ZCAM_1080P24 = 0, //1920x1080, 24FPS CAMLIVE_MODE_ZCAM_1080P30, //1920x1080, 30FPS CAMLIVE_MODE_ZCAM_1080P60, //1920x1080, 60FPS CAMLIVE_MODE_ZCAM_2K7P24, //2704x1520, 24FPS CAMLIVE_MODE_ZCAM_2K7P30, //2704x1520, 24FPS CAMLIVE_MODE_ZCAM_2K7P60, //2704x1520, 24FPS CAMLIVE_MODE_ZCAM_2544P24, //3392x2544, 24FPS CAMLIVE_MODE_ZCAM_2544P30, //3392x2544, 24FPS CAMLIVE_MODE_ZCAM_2544P60, //3392x2544, 24FPS CAMLIVE_MODE_THETAS, //Ricoh TheataS CAMLIVE_MODE_THETAV, //Ricoh TheataV CAMLIVE_MODE_INVALID = -1, }; enum CAM_AUDIO_MODE_ENUM { CAM_AUDIO_MODE_NONE = 0, //None CAM_AUDIO_MODE_SILENT, //Silent audio CAM_AUDIO_MODE_CAMERA //Capture from camera, not supported yet }; extern "C" { AMF_RESULT AMF_CDECL_CALL AMFCreateComponentZCamLiveStream(amf::AMFContext* pContext, amf::AMFComponentEx** ppComponent); } #endif // AMF_ZCamLiveStream_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/AudioBuffer.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_AudioBuffer_h #define AMF_AudioBuffer_h #pragma once #include "Data.h" #if defined(_MSC_VER) #pragma warning( push ) #pragma warning(disable : 4263) #pragma warning(disable : 4264) #endif #if defined(__cplusplus) namespace amf { #endif typedef enum AMF_AUDIO_FORMAT { AMFAF_UNKNOWN =-1, AMFAF_U8 = 0, // amf_uint8 AMFAF_S16 = 1, // amf_int16 AMFAF_S32 = 2, // amf_int32 AMFAF_FLT = 3, // amf_float AMFAF_DBL = 4, // amf_double AMFAF_U8P = 5, // amf_uint8 AMFAF_S16P = 6, // amf_int16 AMFAF_S32P = 7, // amf_int32 AMFAF_FLTP = 8, // amf_float AMFAF_DBLP = 9, // amf_double AMFAF_FIRST = AMFAF_U8, AMFAF_LAST = AMFAF_DBLP, } AMF_AUDIO_FORMAT; typedef enum AMF_AUDIO_CHANNEL_LAYOUT { AMFACL_SPEAKER_FRONT_LEFT = 0x1, AMFACL_SPEAKER_FRONT_RIGHT = 0x2, AMFACL_SPEAKER_FRONT_CENTER = 0x4, AMFACL_SPEAKER_LOW_FREQUENCY = 0x8, AMFACL_SPEAKER_BACK_LEFT = 0x10, AMFACL_SPEAKER_BACK_RIGHT = 0x20, AMFACL_SPEAKER_FRONT_LEFT_OF_CENTER = 0x40, AMFACL_SPEAKER_FRONT_RIGHT_OF_CENTER = 0x80, AMFACL_SPEAKER_BACK_CENTER = 0x100, AMFACL_SPEAKER_SIDE_LEFT = 0x200, AMFACL_SPEAKER_SIDE_RIGHT = 0x400, AMFACL_SPEAKER_TOP_CENTER = 0x800, AMFACL_SPEAKER_TOP_FRONT_LEFT = 0x1000, AMFACL_SPEAKER_TOP_FRONT_CENTER = 0x2000, AMFACL_SPEAKER_TOP_FRONT_RIGHT = 0x4000, AMFACL_SPEAKER_TOP_BACK_LEFT = 0x8000, AMFACL_SPEAKER_TOP_BACK_CENTER = 0x10000, AMFACL_SPEAKER_TOP_BACK_RIGHT = 0x20000 } AMF_AUDIO_CHANNEL_LAYOUT; // get the most common layout for a given number of speakers inline int GetDefaultChannelLayout(int channels) { switch (channels) { case 1: return (AMFACL_SPEAKER_FRONT_CENTER); case 2: return (AMFACL_SPEAKER_FRONT_LEFT | AMFACL_SPEAKER_FRONT_RIGHT); case 4: return (AMFACL_SPEAKER_FRONT_LEFT | AMFACL_SPEAKER_FRONT_RIGHT | AMFACL_SPEAKER_BACK_LEFT | AMFACL_SPEAKER_BACK_RIGHT); case 6: return (AMFACL_SPEAKER_FRONT_LEFT | AMFACL_SPEAKER_FRONT_RIGHT | AMFACL_SPEAKER_FRONT_CENTER | AMFACL_SPEAKER_LOW_FREQUENCY | AMFACL_SPEAKER_BACK_LEFT | AMFACL_SPEAKER_BACK_RIGHT); case 8: return (AMFACL_SPEAKER_FRONT_LEFT | AMFACL_SPEAKER_FRONT_RIGHT | AMFACL_SPEAKER_FRONT_CENTER | AMFACL_SPEAKER_LOW_FREQUENCY | AMFACL_SPEAKER_BACK_LEFT | AMFACL_SPEAKER_BACK_RIGHT | AMFACL_SPEAKER_FRONT_LEFT_OF_CENTER | AMFACL_SPEAKER_FRONT_RIGHT_OF_CENTER); } return AMFACL_SPEAKER_FRONT_LEFT | AMFACL_SPEAKER_FRONT_RIGHT; } //---------------------------------------------------------------------------------------------- // AMFAudioBufferObserver interface - callback //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMFAudioBuffer; class AMF_NO_VTABLE AMFAudioBufferObserver { public: virtual void AMF_STD_CALL OnBufferDataRelease(AMFAudioBuffer* pBuffer) = 0; }; #else // #if defined(__cplusplus) typedef struct AMFAudioBuffer AMFAudioBuffer; typedef struct AMFAudioBufferObserver AMFAudioBufferObserver; typedef struct AMFAudioBufferObserverVtbl { void (AMF_STD_CALL *OnBufferDataRelease)(AMFAudioBufferObserver* pThis, AMFAudioBuffer* pBuffer); } AMFAudioBufferObserverVtbl; struct AMFAudioBufferObserver { const AMFAudioBufferObserverVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AudioBuffer interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFAudioBuffer : public AMFData { public: AMF_DECLARE_IID(0x2212ff8, 0x6107, 0x430b, 0xb6, 0x3c, 0xc7, 0xe5, 0x40, 0xe5, 0xf8, 0xeb) virtual amf_int32 AMF_STD_CALL GetSampleCount() = 0; virtual amf_int32 AMF_STD_CALL GetSampleRate() = 0; virtual amf_int32 AMF_STD_CALL GetChannelCount() = 0; virtual AMF_AUDIO_FORMAT AMF_STD_CALL GetSampleFormat() = 0; virtual amf_int32 AMF_STD_CALL GetSampleSize() = 0; virtual amf_uint32 AMF_STD_CALL GetChannelLayout() = 0; virtual void* AMF_STD_CALL GetNative() = 0; virtual amf_size AMF_STD_CALL GetSize() = 0; // Observer management #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Woverloaded-virtual" #endif virtual void AMF_STD_CALL AddObserver(AMFAudioBufferObserver* pObserver) = 0; virtual void AMF_STD_CALL RemoveObserver(AMFAudioBufferObserver* pObserver) = 0; #ifdef __clang__ #pragma clang diagnostic pop #endif }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFAudioBufferPtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFAudioBuffer, 0x2212ff8, 0x6107, 0x430b, 0xb6, 0x3c, 0xc7, 0xe5, 0x40, 0xe5, 0xf8, 0xeb) typedef struct AMFAudioBufferVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFAudioBuffer* pThis); amf_long (AMF_STD_CALL *Release)(AMFAudioBuffer* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFAudioBuffer* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFAudioBuffer* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFAudioBuffer* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFAudioBuffer* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFAudioBuffer* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFAudioBuffer* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFAudioBuffer* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFAudioBuffer* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFAudioBuffer* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFAudioBuffer* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFAudioBuffer* pThis, AMFPropertyStorageObserver* pObserver); // AMFData interface AMF_MEMORY_TYPE (AMF_STD_CALL *GetMemoryType)(AMFAudioBuffer* pThis); AMF_RESULT (AMF_STD_CALL *Duplicate)(AMFAudioBuffer* pThis, AMF_MEMORY_TYPE type, AMFData** ppData); AMF_RESULT (AMF_STD_CALL *Convert)(AMFAudioBuffer* pThis, AMF_MEMORY_TYPE type); // optimal interop if possilble. Copy through host memory if needed AMF_RESULT (AMF_STD_CALL *Interop)(AMFAudioBuffer* pThis, AMF_MEMORY_TYPE type); // only optimal interop if possilble. No copy through host memory for GPU objects AMF_DATA_TYPE (AMF_STD_CALL *GetDataType)(AMFAudioBuffer* pThis); amf_bool (AMF_STD_CALL *IsReusable)(AMFAudioBuffer* pThis); void (AMF_STD_CALL *SetPts)(AMFAudioBuffer* pThis, amf_pts pts); amf_pts (AMF_STD_CALL *GetPts)(AMFAudioBuffer* pThis); void (AMF_STD_CALL *SetDuration)(AMFAudioBuffer* pThis, amf_pts duration); amf_pts (AMF_STD_CALL *GetDuration)(AMFAudioBuffer* pThis); // AMFAudioBuffer interface amf_int32 (AMF_STD_CALL *GetSampleCount)(AMFAudioBuffer* pThis); amf_int32 (AMF_STD_CALL *GetSampleRate)(AMFAudioBuffer* pThis); amf_int32 (AMF_STD_CALL *GetChannelCount)(AMFAudioBuffer* pThis); AMF_AUDIO_FORMAT (AMF_STD_CALL *GetSampleFormat)(AMFAudioBuffer* pThis); amf_int32 (AMF_STD_CALL *GetSampleSize)(AMFAudioBuffer* pThis); amf_uint32 (AMF_STD_CALL *GetChannelLayout)(AMFAudioBuffer* pThis); void* (AMF_STD_CALL *GetNative)(AMFAudioBuffer* pThis); amf_size (AMF_STD_CALL *GetSize)(AMFAudioBuffer* pThis); // Observer management void (AMF_STD_CALL *AddObserver_AudioBuffer)(AMFAudioBuffer* pThis, AMFAudioBufferObserver* pObserver); void (AMF_STD_CALL *RemoveObserver_AudioBuffer)(AMFAudioBuffer* pThis, AMFAudioBufferObserver* pObserver); } AMFAudioBufferVtbl; struct AMFAudioBuffer { const AMFAudioBufferVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace #endif #if defined(_MSC_VER) #pragma warning( pop ) #endif #endif //#ifndef AMF_AudioBuffer_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Buffer.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Buffer_h #define AMF_Buffer_h #pragma once #include "Data.h" #if defined(_MSC_VER) #pragma warning( push ) #pragma warning(disable : 4263) #pragma warning(disable : 4264) #endif #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMF_BUFFER_USAGE translates to D3D11_BIND_FLAG or VkBufferUsageFlagBits // bit mask //---------------------------------------------------------------------------------------------- typedef enum AMF_BUFFER_USAGE_BITS { // D3D11 D3D12 Vulkan AMF_BUFFER_USAGE_DEFAULT = 0x80000000, // D3D11_USAGE_STAGING, VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT AMF_BUFFER_USAGE_NONE = 0x00000000, // 0 , D3D12_RESOURCE_FLAG_NONE, 0 AMF_BUFFER_USAGE_CONSTANT = 0x00000001, // D3D11_BIND_CONSTANT_BUFFER, VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT AMF_BUFFER_USAGE_SHADER_RESOURCE = 0x00000002, // D3D11_BIND_SHADER_RESOURCE, D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET, VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT AMF_BUFFER_USAGE_UNORDERED_ACCESS = 0x00000004, // D3D11_BIND_UNORDERED_ACCESS, D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS, VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT AMF_BUFFER_USAGE_TRANSFER_SRC = 0x00000008, // VK_BUFFER_USAGE_TRANSFER_SRC_BIT AMF_BUFFER_USAGE_TRANSFER_DST = 0x00000010, // VK_BUFFER_USAGE_TRANSFER_DST_BIT AMF_BUFFER_USAGE_NOSYNC = 0x00000020, // no fence (AMFFenceGUID) created no semaphore (AMFVulkanSync::hSemaphore) created AMF_BUFFER_USAGE_DECODER_SRC = 0x00000040, // VK_BUFFER_USAGE_VIDEO_DECODE_SRC_BIT_KHR } AMF_BUFFER_USAGE_BITS; typedef amf_flags AMF_BUFFER_USAGE; //---------------------------------------------------------------------------------------------- //---------------------------------------------------------------------------------------------- // AMFBufferObserver interface - callback //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMFBuffer; class AMF_NO_VTABLE AMFBufferObserver { public: virtual void AMF_STD_CALL OnBufferDataRelease(AMFBuffer* pBuffer) = 0; }; #else // #if defined(__cplusplus) typedef struct AMFBuffer AMFBuffer; typedef struct AMFBufferObserver AMFBufferObserver; typedef struct AMFBufferObserverVtbl { void (AMF_STD_CALL *OnBufferDataRelease)(AMFBufferObserver* pThis, AMFBuffer* pBuffer); } AMFBufferObserverVtbl; struct AMFBufferObserver { const AMFBufferObserverVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFBuffer interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFBuffer : public AMFData { public: AMF_DECLARE_IID(0xb04b7248, 0xb6f0, 0x4321, 0xb6, 0x91, 0xba, 0xa4, 0x74, 0xf, 0x9f, 0xcb) virtual AMF_RESULT AMF_STD_CALL SetSize(amf_size newSize) = 0; virtual amf_size AMF_STD_CALL GetSize() = 0; virtual void* AMF_STD_CALL GetNative() = 0; // Observer management #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Woverloaded-virtual" #endif virtual void AMF_STD_CALL AddObserver(AMFBufferObserver* pObserver) = 0; virtual void AMF_STD_CALL RemoveObserver(AMFBufferObserver* pObserver) = 0; #ifdef __clang__ #pragma clang diagnostic pop #endif }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFBufferPtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFBuffer, 0xb04b7248, 0xb6f0, 0x4321, 0xb6, 0x91, 0xba, 0xa4, 0x74, 0xf, 0x9f, 0xcb) typedef struct AMFBufferVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFBuffer* pThis); amf_long (AMF_STD_CALL *Release)(AMFBuffer* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFBuffer* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFBuffer* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFBuffer* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFBuffer* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFBuffer* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFBuffer* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFBuffer* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFBuffer* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFBuffer* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFBuffer* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFBuffer* pThis, AMFPropertyStorageObserver* pObserver); // AMFData interface AMF_MEMORY_TYPE (AMF_STD_CALL *GetMemoryType)(AMFBuffer* pThis); AMF_RESULT (AMF_STD_CALL *Duplicate)(AMFBuffer* pThis, AMF_MEMORY_TYPE type, AMFData** ppData); AMF_RESULT (AMF_STD_CALL *Convert)(AMFBuffer* pThis, AMF_MEMORY_TYPE type); // optimal interop if possilble. Copy through host memory if needed AMF_RESULT (AMF_STD_CALL *Interop)(AMFBuffer* pThis, AMF_MEMORY_TYPE type); // only optimal interop if possilble. No copy through host memory for GPU objects AMF_DATA_TYPE (AMF_STD_CALL *GetDataType)(AMFBuffer* pThis); amf_bool (AMF_STD_CALL *IsReusable)(AMFBuffer* pThis); void (AMF_STD_CALL *SetPts)(AMFBuffer* pThis, amf_pts pts); amf_pts (AMF_STD_CALL *GetPts)(AMFBuffer* pThis); void (AMF_STD_CALL *SetDuration)(AMFBuffer* pThis, amf_pts duration); amf_pts (AMF_STD_CALL *GetDuration)(AMFBuffer* pThis); // AMFBuffer interface AMF_RESULT (AMF_STD_CALL *SetSize)(AMFBuffer* pThis, amf_size newSize); amf_size (AMF_STD_CALL *GetSize)(AMFBuffer* pThis); void* (AMF_STD_CALL *GetNative)(AMFBuffer* pThis); // Observer management void (AMF_STD_CALL *AddObserver_Buffer)(AMFBuffer* pThis, AMFBufferObserver* pObserver); void (AMF_STD_CALL *RemoveObserver_Buffer)(AMFBuffer* pThis, AMFBufferObserver* pObserver); } AMFBufferVtbl; struct AMFBuffer { const AMFBufferVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace #endif #if defined(_MSC_VER) #pragma warning( pop ) #endif #endif //#ifndef AMF_Buffer_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Compute.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file Compute.h * @brief AMFCompute interface declaration *************************************************************************************************** */ #ifndef AMF_Compute_h #define AMF_Compute_h #pragma once #include "Buffer.h" #include "Surface.h" #if defined(__cplusplus) namespace amf { #endif typedef amf_uint64 AMF_KERNEL_ID; //---------------------------------------------------------------------------------------------- // enumerations for plane conversion //---------------------------------------------------------------------------------------------- typedef enum AMF_CHANNEL_ORDER { AMF_CHANNEL_ORDER_INVALID = 0, AMF_CHANNEL_ORDER_R = 1, AMF_CHANNEL_ORDER_RG = 2, AMF_CHANNEL_ORDER_BGRA = 3, AMF_CHANNEL_ORDER_RGBA = 4, AMF_CHANNEL_ORDER_ARGB = 5, AMF_CHANNEL_ORDER_YUY2 = 6, } AMF_CHANNEL_ORDER; //---------------------------------------------------------------------------------------------- typedef enum AMF_CHANNEL_TYPE { AMF_CHANNEL_INVALID = 0, AMF_CHANNEL_UNSIGNED_INT8 = 1, AMF_CHANNEL_UNSIGNED_INT32 = 2, AMF_CHANNEL_UNORM_INT8 = 3, AMF_CHANNEL_UNORM_INT16 = 4, AMF_CHANNEL_SNORM_INT16 = 5, AMF_CHANNEL_FLOAT = 6, AMF_CHANNEL_FLOAT16 = 7, AMF_CHANNEL_UNSIGNED_INT16 = 8, AMF_CHANNEL_UNORM_INT_101010 = 9, } AMF_CHANNEL_TYPE; //---------------------------------------------------------------------------------------------- #define AMF_STRUCTURED_BUFFER_FORMAT L"StructuredBufferFormat" // amf_int64(AMF_CHANNEL_TYPE), default - AMF_CHANNEL_UNSIGNED_INT32; to be set on AMFBuffer objects #if defined(_WIN32) AMF_WEAK GUID AMFStructuredBufferFormatGUID = { 0x90c5d674, 0xe90, 0x4181, {0xbd, 0xef, 0x26, 0x13, 0xc1, 0xdf, 0xa3, 0xbd} }; // UINT(DXGI_FORMAT), default - DXGI_FORMAT_R32_UINT; to be set on ID3D11Buffer or ID3D11Texture2D objects when used natively #endif //---------------------------------------------------------------------------------------------- // enumeration argument type //---------------------------------------------------------------------------------------------- typedef enum AMF_ARGUMENT_ACCESS_TYPE { AMF_ARGUMENT_ACCESS_READ = 0, AMF_ARGUMENT_ACCESS_WRITE = 1, AMF_ARGUMENT_ACCESS_READWRITE = 2, AMF_ARGUMENT_ACCESS_READWRITE_MASK = 0xFFFF, //Sampler parameters AMF_ARGUMENT_SAMPLER_LINEAR = 0x10000000, AMF_ARGUMENT_SAMPLER_NORM_COORD = 0x20000000, AMF_ARGUMENT_SAMPLER_POINT = 0x40000000, AMF_ARGUMENT_SAMPLER_MASK = 0xFFFF0000, } AMF_ARGUMENT_ACCESS_TYPE; //---------------------------------------------------------------------------------------------- // AMFComputeKernel interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFComputeKernel : public AMFInterface { public: AMF_DECLARE_IID(0x94815701, 0x6c84, 0x4ba6, 0xa9, 0xfe, 0xe9, 0xad, 0x40, 0xf8, 0x8, 0x8) virtual void* AMF_STD_CALL GetNative() = 0; virtual const wchar_t* AMF_STD_CALL GetIDName() = 0; virtual AMF_RESULT AMF_STD_CALL SetArgPlaneNative(amf_size index, void* pPlane, AMF_ARGUMENT_ACCESS_TYPE eAccess) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgBufferNative(amf_size index, void* pBuffer, AMF_ARGUMENT_ACCESS_TYPE eAccess) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgPlane(amf_size index, AMFPlane* pPlane, AMF_ARGUMENT_ACCESS_TYPE eAccess) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgBuffer(amf_size index, AMFBuffer* pBuffer, AMF_ARGUMENT_ACCESS_TYPE eAccess) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgInt32(amf_size index, amf_int32 data) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgInt64(amf_size index, amf_int64 data) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgFloat(amf_size index, amf_float data) = 0; virtual AMF_RESULT AMF_STD_CALL SetArgBlob(amf_size index, amf_size dataSize, const void* pData) = 0; virtual AMF_RESULT AMF_STD_CALL GetCompileWorkgroupSize(amf_size workgroupSize[3]) = 0; virtual AMF_RESULT AMF_STD_CALL Enqueue(amf_size dimension, amf_size globalOffset[3], amf_size globalSize[3], amf_size localSize[3]) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFComputeKernelPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFComputeKernel, 0x94815701, 0x6c84, 0x4ba6, 0xa9, 0xfe, 0xe9, 0xad, 0x40, 0xf8, 0x8, 0x8) typedef struct AMFComputeKernel AMFComputeKernel; typedef struct AMFComputeKernelVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFComputeKernel* pThis); amf_long (AMF_STD_CALL *Release)(AMFComputeKernel* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFComputeKernel* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFComputeKernel interface } AMFComputeKernelVtbl; struct AMFComputeKernel { const AMFComputeKernelVtbl *pVtbl; }; #endif //#if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFComputeSyncPoint interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFComputeSyncPoint : public AMFInterface { public: AMF_DECLARE_IID(0x66f33fe6, 0xaae, 0x4e65, 0xba, 0x3, 0xea, 0x8b, 0xa3, 0x60, 0x11, 0x2) virtual amf_bool AMF_STD_CALL IsCompleted() = 0; virtual void AMF_STD_CALL Wait() = 0; }; typedef AMFInterfacePtr_T AMFComputeSyncPointPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFComputeSyncPoint, 0x66f33fe6, 0xaae, 0x4e65, 0xba, 0x3, 0xea, 0x8b, 0xa3, 0x60, 0x11, 0x2) typedef struct AMFComputeSyncPoint AMFComputeSyncPoint; typedef struct AMFComputeSyncPointVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFComputeSyncPoint* pThis); amf_long (AMF_STD_CALL *Release)(AMFComputeSyncPoint* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFComputeSyncPoint* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFComputeSyncPoint interface amf_bool (AMF_STD_CALL *IsCompleted)(AMFComputeSyncPoint* pThis); void (AMF_STD_CALL *Wait)(AMFComputeSyncPoint* pThis); } AMFComputeSyncPointVtbl; struct AMFComputeSyncPoint { const AMFComputeSyncPointVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFCompute interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFCompute : public AMFInterface { public: AMF_DECLARE_IID(0x3846233a, 0x3f43, 0x443f, 0x8a, 0x45, 0x75, 0x22, 0x11, 0xa9, 0xfb, 0xd5) virtual AMF_MEMORY_TYPE AMF_STD_CALL GetMemoryType() = 0; virtual void* AMF_STD_CALL GetNativeContext() = 0; virtual void* AMF_STD_CALL GetNativeDeviceID() = 0; virtual void* AMF_STD_CALL GetNativeCommandQueue() = 0; virtual AMF_RESULT AMF_STD_CALL GetKernel(AMF_KERNEL_ID kernelID, AMFComputeKernel** kernel) = 0; virtual AMF_RESULT AMF_STD_CALL PutSyncPoint(AMFComputeSyncPoint** ppSyncPoint) = 0; virtual AMF_RESULT AMF_STD_CALL FinishQueue() = 0; virtual AMF_RESULT AMF_STD_CALL FlushQueue() = 0; virtual AMF_RESULT AMF_STD_CALL FillPlane(AMFPlane *pPlane, const amf_size origin[3], const amf_size region[3], const void* pColor) = 0; virtual AMF_RESULT AMF_STD_CALL FillBuffer(AMFBuffer* pBuffer, amf_size dstOffset, amf_size dstSize, const void* pSourcePattern, amf_size patternSize) = 0; virtual AMF_RESULT AMF_STD_CALL ConvertPlaneToBuffer(AMFPlane *pSrcPlane, AMFBuffer** ppDstBuffer) = 0; virtual AMF_RESULT AMF_STD_CALL CopyBuffer(AMFBuffer* pSrcBuffer, amf_size srcOffset, amf_size size, AMFBuffer* pDstBuffer, amf_size dstOffset) = 0; virtual AMF_RESULT AMF_STD_CALL CopyPlane(AMFPlane *pSrcPlane, const amf_size srcOrigin[3], const amf_size region[3], AMFPlane *pDstPlane, const amf_size dstOrigin[3]) = 0; virtual AMF_RESULT AMF_STD_CALL CopyBufferToHost(AMFBuffer* pSrcBuffer, amf_size srcOffset, amf_size size, void* pDest, amf_bool blocking) = 0; virtual AMF_RESULT AMF_STD_CALL CopyBufferFromHost(const void* pSource, amf_size size, AMFBuffer* pDstBuffer, amf_size dstOffsetInBytes, amf_bool blocking) = 0; virtual AMF_RESULT AMF_STD_CALL CopyPlaneToHost(AMFPlane *pSrcPlane, const amf_size origin[3], const amf_size region[3], void* pDest, amf_size dstPitch, amf_bool blocking) = 0; virtual AMF_RESULT AMF_STD_CALL CopyPlaneFromHost(void* pSource, const amf_size origin[3], const amf_size region[3], amf_size srcPitch, AMFPlane *pDstPlane, amf_bool blocking) = 0; virtual AMF_RESULT AMF_STD_CALL ConvertPlaneToPlane(AMFPlane* pSrcPlane, AMFPlane** ppDstPlane, AMF_CHANNEL_ORDER order, AMF_CHANNEL_TYPE type) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFComputePtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFCompute, 0x3846233a, 0x3f43, 0x443f, 0x8a, 0x45, 0x75, 0x22, 0x11, 0xa9, 0xfb, 0xd5) typedef struct AMFCompute AMFCompute; typedef struct AMFComputeVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFCompute* pThis); amf_long (AMF_STD_CALL *Release)(AMFCompute* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFCompute* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFCompute interface AMF_MEMORY_TYPE (AMF_STD_CALL *GetMemoryType)(AMFCompute* pThis); void* (AMF_STD_CALL *GetNativeContext)(AMFCompute* pThis); void* (AMF_STD_CALL *GetNativeDeviceID)(AMFCompute* pThis); void* (AMF_STD_CALL *GetNativeCommandQueue)(AMFCompute* pThis); AMF_RESULT (AMF_STD_CALL *GetKernel)(AMFCompute* pThis, AMF_KERNEL_ID kernelID, AMFComputeKernel** kernel); AMF_RESULT (AMF_STD_CALL *PutSyncPoint)(AMFCompute* pThis, AMFComputeSyncPoint** ppSyncPoint); AMF_RESULT (AMF_STD_CALL *FinishQueue)(AMFCompute* pThis); AMF_RESULT (AMF_STD_CALL *FlushQueue)(AMFCompute* pThis); AMF_RESULT (AMF_STD_CALL *FillPlane)(AMFCompute* pThis, AMFPlane *pPlane, const amf_size origin[3], const amf_size region[3], const void* pColor); AMF_RESULT (AMF_STD_CALL *FillBuffer)(AMFCompute* pThis, AMFBuffer* pBuffer, amf_size dstOffset, amf_size dstSize, const void* pSourcePattern, amf_size patternSize); AMF_RESULT (AMF_STD_CALL *ConvertPlaneToBuffer)(AMFCompute* pThis, AMFPlane *pSrcPlane, AMFBuffer** ppDstBuffer); AMF_RESULT (AMF_STD_CALL *CopyBuffer)(AMFCompute* pThis, AMFBuffer* pSrcBuffer, amf_size srcOffset, amf_size size, AMFBuffer* pDstBuffer, amf_size dstOffset); AMF_RESULT (AMF_STD_CALL *CopyPlane)(AMFCompute* pThis, AMFPlane *pSrcPlane, const amf_size srcOrigin[3], const amf_size region[3], AMFPlane *pDstPlane, const amf_size dstOrigin[3]); AMF_RESULT (AMF_STD_CALL *CopyBufferToHost)(AMFCompute* pThis, AMFBuffer* pSrcBuffer, amf_size srcOffset, amf_size size, void* pDest, amf_bool blocking); AMF_RESULT (AMF_STD_CALL *CopyBufferFromHost)(AMFCompute* pThis, const void* pSource, amf_size size, AMFBuffer* pDstBuffer, amf_size dstOffsetInBytes, amf_bool blocking); AMF_RESULT (AMF_STD_CALL *CopyPlaneToHost)(AMFCompute* pThis, AMFPlane *pSrcPlane, const amf_size origin[3], const amf_size region[3], void* pDest, amf_size dstPitch, amf_bool blocking); AMF_RESULT (AMF_STD_CALL *CopyPlaneFromHost)(AMFCompute* pThis, void* pSource, const amf_size origin[3], const amf_size region[3], amf_size srcPitch, AMFPlane *pDstPlane, amf_bool blocking); AMF_RESULT (AMF_STD_CALL *ConvertPlaneToPlane)(AMFCompute* pThis, AMFPlane* pSrcPlane, AMFPlane** ppDstPlane, AMF_CHANNEL_ORDER order, AMF_CHANNEL_TYPE type); } AMFComputeVtbl; struct AMFCompute { const AMFComputeVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFPrograms interface - singleton //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFPrograms { public: virtual AMF_RESULT AMF_STD_CALL RegisterKernelSourceFile(AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, const wchar_t* filepath, const char* options) = 0; virtual AMF_RESULT AMF_STD_CALL RegisterKernelSource(AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options) = 0; virtual AMF_RESULT AMF_STD_CALL RegisterKernelBinary(AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options) = 0; virtual AMF_RESULT AMF_STD_CALL RegisterKernelSource1(AMF_MEMORY_TYPE eMemoryType, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options) = 0; virtual AMF_RESULT AMF_STD_CALL RegisterKernelBinary1(AMF_MEMORY_TYPE eMemoryType, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options) = 0; }; #else // #if defined(__cplusplus) typedef struct AMFPrograms AMFPrograms; typedef struct AMFProgramsVtbl { AMF_RESULT (AMF_STD_CALL *RegisterKernelSourceFile)(AMFPrograms* pThis, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, const wchar_t* filepath, const char* options); AMF_RESULT (AMF_STD_CALL *RegisterKernelSource)(AMFPrograms* pThis, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options); AMF_RESULT (AMF_STD_CALL *RegisterKernelBinary)(AMFPrograms* pThis, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options); AMF_RESULT (AMF_STD_CALL *RegisterKernelSource1)(AMFPrograms* pThis, AMF_MEMORY_TYPE eMemoryType, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options); AMF_RESULT (AMF_STD_CALL *RegisterKernelBinary1)(AMFPrograms* pThis, AMF_MEMORY_TYPE eMemoryType, AMF_KERNEL_ID* pKernelID, const wchar_t* kernelid_name, const char* kernelName, amf_size dataSize, const amf_uint8* data, const char* options); } AMFProgramsVtbl; struct AMFPrograms { const AMFProgramsVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace amf #endif #endif // AMF_Compute_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/ComputeFactory.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_ComputeFactory_h #define AMF_ComputeFactory_h #pragma once #include "Compute.h" #if defined(__cplusplus) namespace amf { #endif // compute device audio capabilities accessed via GetProperties() from AMFComputeDevice #define AMF_DEVICE_NAME L"DeviceName" // char*, string, device name #define AMF_DRIVER_VERSION_NAME L"DriverVersion" // char*, string, driver version #define AMF_AUDIO_CONVOLUTION_MAX_STREAMS L"ConvolutionMaxStreams" // amf_int64, maximum number of audio streams supported in realtime #define AMF_AUDIO_CONVOLUTION_LENGTH L"ConvolutionLength" // amf_int64, length of convolution in samples #define AMF_AUDIO_CONVOLUTION_BUFFER_SIZE L"ConvolutionBufferSize" // amf_int64, buffer size in samples #define AMF_AUDIO_CONVOLUTION_SAMPLE_RATE L"ConvolutionSampleRate" // amf_int64, sample rate #if defined(__cplusplus) class AMF_NO_VTABLE AMFComputeDevice : public AMFPropertyStorage { public: AMF_DECLARE_IID(0xb79d7cf6, 0x2c5c, 0x4deb, 0xb8, 0x96, 0xa2, 0x9e, 0xbe, 0xa6, 0xe3, 0x97) virtual void* AMF_STD_CALL GetNativePlatform() = 0; virtual void* AMF_STD_CALL GetNativeDeviceID() = 0; virtual void* AMF_STD_CALL GetNativeContext() = 0; virtual AMF_RESULT AMF_STD_CALL CreateCompute(void *reserved, AMFCompute **ppCompute) = 0; virtual AMF_RESULT AMF_STD_CALL CreateComputeEx(void* pCommandQueue, AMFCompute **ppCompute) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFComputeDevicePtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFComputeDevice, 0xb79d7cf6, 0x2c5c, 0x4deb, 0xb8, 0x96, 0xa2, 0x9e, 0xbe, 0xa6, 0xe3, 0x97) typedef struct AMFComputeDevice AMFComputeDevice; typedef struct AMFComputeDeviceVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFComputeDevice* pThis); amf_long (AMF_STD_CALL *Release)(AMFComputeDevice* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFComputeDevice* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFComputeDevice* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFComputeDevice* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFComputeDevice* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFComputeDevice* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFComputeDevice* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFComputeDevice* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFComputeDevice* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFComputeDevice* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFComputeDevice* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFComputeDevice* pThis, AMFPropertyStorageObserver* pObserver); // AMFComputeDevice interface void* (AMF_STD_CALL *GetNativePlatform)(AMFComputeDevice* pThis); void* (AMF_STD_CALL *GetNativeDeviceID)(AMFComputeDevice* pThis); void* (AMF_STD_CALL *GetNativeContext)(AMFComputeDevice* pThis); AMF_RESULT (AMF_STD_CALL *CreateCompute)(AMFComputeDevice* pThis, void *reserved, AMFCompute **ppCompute); AMF_RESULT (AMF_STD_CALL *CreateComputeEx)(AMFComputeDevice* pThis, void* pCommandQueue, AMFCompute **ppCompute); } AMFComputeDeviceVtbl; struct AMFComputeDevice { const AMFComputeDeviceVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFComputeFactory : public AMFInterface { public: AMF_DECLARE_IID(0xe3c24bd7, 0x2d83, 0x416c, 0x8c, 0x4e, 0xfd, 0x13, 0xca, 0x86, 0xf4, 0xd0) virtual amf_int32 AMF_STD_CALL GetDeviceCount() = 0; virtual AMF_RESULT AMF_STD_CALL GetDeviceAt(amf_int32 index, AMFComputeDevice **ppDevice) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFComputeFactoryPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFComputeFactory, 0xe3c24bd7, 0x2d83, 0x416c, 0x8c, 0x4e, 0xfd, 0x13, 0xca, 0x86, 0xf4, 0xd0) typedef struct AMFComputeFactory AMFComputeFactory; typedef struct AMFComputeFactoryVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFComputeFactory* pThis); amf_long (AMF_STD_CALL *Release)(AMFComputeFactory* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFComputeFactory* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFComputeFactory interface amf_int32 (AMF_STD_CALL *GetDeviceCount)(AMFComputeFactory* pThis); AMF_RESULT (AMF_STD_CALL *GetDeviceAt)(AMFComputeFactory* pThis, amf_int32 index, AMFComputeDevice **ppDevice); } AMFComputeFactoryVtbl; struct AMFComputeFactory { const AMFComputeFactoryVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) } // namespace amf #endif #endif // AMF_ComputeFactory_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Context.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Context_h #define AMF_Context_h #pragma once #include "Buffer.h" #include "AudioBuffer.h" #include "Surface.h" #include "Compute.h" #include "ComputeFactory.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMFContext interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFContext : public AMFPropertyStorage { public: AMF_DECLARE_IID(0xa76a13f0, 0xd80e, 0x4fcc, 0xb5, 0x8, 0x65, 0xd0, 0xb5, 0x2e, 0xd9, 0xee) // Cleanup virtual AMF_RESULT AMF_STD_CALL Terminate() = 0; // DX9 virtual AMF_RESULT AMF_STD_CALL InitDX9(void* pDX9Device) = 0; virtual void* AMF_STD_CALL GetDX9Device(AMF_DX_VERSION dxVersionRequired = AMF_DX9) = 0; virtual AMF_RESULT AMF_STD_CALL LockDX9() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockDX9() = 0; class AMFDX9Locker; // DX11 virtual AMF_RESULT AMF_STD_CALL InitDX11(void* pDX11Device, AMF_DX_VERSION dxVersionRequired = AMF_DX11_0) = 0; virtual void* AMF_STD_CALL GetDX11Device(AMF_DX_VERSION dxVersionRequired = AMF_DX11_0) = 0; virtual AMF_RESULT AMF_STD_CALL LockDX11() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockDX11() = 0; class AMFDX11Locker; // OpenCL virtual AMF_RESULT AMF_STD_CALL InitOpenCL(void* pCommandQueue = NULL) = 0; virtual void* AMF_STD_CALL GetOpenCLContext() = 0; virtual void* AMF_STD_CALL GetOpenCLCommandQueue() = 0; virtual void* AMF_STD_CALL GetOpenCLDeviceID() = 0; virtual AMF_RESULT AMF_STD_CALL GetOpenCLComputeFactory(AMFComputeFactory **ppFactory) = 0; // advanced compute - multiple queries virtual AMF_RESULT AMF_STD_CALL InitOpenCLEx(AMFComputeDevice *pDevice) = 0; virtual AMF_RESULT AMF_STD_CALL LockOpenCL() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockOpenCL() = 0; class AMFOpenCLLocker; // OpenGL virtual AMF_RESULT AMF_STD_CALL InitOpenGL(amf_handle hOpenGLContext, amf_handle hWindow, amf_handle hDC) = 0; virtual amf_handle AMF_STD_CALL GetOpenGLContext() = 0; virtual amf_handle AMF_STD_CALL GetOpenGLDrawable() = 0; virtual AMF_RESULT AMF_STD_CALL LockOpenGL() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockOpenGL() = 0; class AMFOpenGLLocker; // XV - Linux virtual AMF_RESULT AMF_STD_CALL InitXV(void* pXVDevice) = 0; virtual void* AMF_STD_CALL GetXVDevice() = 0; virtual AMF_RESULT AMF_STD_CALL LockXV() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockXV() = 0; class AMFXVLocker; // Gralloc - Android virtual AMF_RESULT AMF_STD_CALL InitGralloc(void* pGrallocDevice) = 0; virtual void* AMF_STD_CALL GetGrallocDevice() = 0; virtual AMF_RESULT AMF_STD_CALL LockGralloc() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockGralloc() = 0; class AMFGrallocLocker; // Allocation virtual AMF_RESULT AMF_STD_CALL AllocBuffer(AMF_MEMORY_TYPE type, amf_size size, AMFBuffer** ppBuffer) = 0; virtual AMF_RESULT AMF_STD_CALL AllocSurface(AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMFSurface** ppSurface) = 0; virtual AMF_RESULT AMF_STD_CALL AllocAudioBuffer(AMF_MEMORY_TYPE type, AMF_AUDIO_FORMAT format, amf_int32 samples, amf_int32 sampleRate, amf_int32 channels, AMFAudioBuffer** ppAudioBuffer) = 0; // Wrap existing objects virtual AMF_RESULT AMF_STD_CALL CreateBufferFromHostNative(void* pHostBuffer, amf_size size, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromHostNative(AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32 vPitch, void* pData, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromDX9Native(void* pDX9Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromDX11Native(void* pDX11Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromOpenGLNative(AMF_SURFACE_FORMAT format, amf_handle hGLTextureID, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromGrallocNative(amf_handle hGrallocSurface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromOpenCLNative(AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, void** pClPlanes, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateBufferFromOpenCLNative(void* pCLBuffer, amf_size size, AMFBuffer** ppBuffer) = 0; // Access to AMFCompute interface - AMF_MEMORY_OPENCL, AMF_MEMORY_COMPUTE_FOR_DX9, AMF_MEMORY_COMPUTE_FOR_DX11 are currently supported virtual AMF_RESULT AMF_STD_CALL GetCompute(AMF_MEMORY_TYPE eMemType, AMFCompute** ppCompute) = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFContextPtr; //---------------------------------------------------------------------------------------------- // AMFContext1 interface //---------------------------------------------------------------------------------------------- class AMF_NO_VTABLE AMFContext1 : public AMFContext { public: AMF_DECLARE_IID(0xd9e9f868, 0x6220, 0x44c6, 0xa2, 0x2f, 0x7c, 0xd6, 0xda, 0xc6, 0x86, 0x46) virtual AMF_RESULT AMF_STD_CALL CreateBufferFromDX11Native(void* pHostBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL AllocBufferEx(AMF_MEMORY_TYPE type, amf_size size, AMF_BUFFER_USAGE usage, AMF_MEMORY_CPU_ACCESS access, AMFBuffer** ppBuffer) = 0; virtual AMF_RESULT AMF_STD_CALL AllocSurfaceEx(AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMF_SURFACE_USAGE usage, AMF_MEMORY_CPU_ACCESS access, AMFSurface** ppSurface) = 0; // Vulkan - Windows, Linux virtual AMF_RESULT AMF_STD_CALL InitVulkan(void* pVulkanDevice) = 0; virtual void* AMF_STD_CALL GetVulkanDevice() = 0; virtual AMF_RESULT AMF_STD_CALL LockVulkan() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockVulkan() = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromVulkanNative(void* pVulkanImage, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateBufferFromVulkanNative(void* pVulkanBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL GetVulkanDeviceExtensions(amf_size *pCount, const char **ppExtensions) = 0; class AMFVulkanLocker; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFContext1Ptr; class AMF_NO_VTABLE AMFContext2 : public AMFContext1 { public: AMF_DECLARE_IID(0x726241d3, 0xbd46, 0x4e90, 0x99, 0x68, 0x93, 0xe0, 0x7e, 0xa2, 0x98, 0x4d) // DX12 virtual AMF_RESULT AMF_STD_CALL InitDX12(void* pDX11Device, AMF_DX_VERSION dxVersionRequired = AMF_DX12) = 0; virtual void* AMF_STD_CALL GetDX12Device(AMF_DX_VERSION dxVersionRequired = AMF_DX12) = 0; virtual AMF_RESULT AMF_STD_CALL LockDX12() = 0; virtual AMF_RESULT AMF_STD_CALL UnlockDX12() = 0; virtual AMF_RESULT AMF_STD_CALL CreateSurfaceFromDX12Native(void* pResourceTexture, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0; virtual AMF_RESULT AMF_STD_CALL CreateBufferFromDX12Native(void* pResourceBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver) = 0; class AMFDX12Locker; }; typedef AMFInterfacePtr_T AMFContext2Ptr; #else typedef struct AMFContext AMFContext; AMF_DECLARE_IID(AMFContext, 0xa76a13f0, 0xd80e, 0x4fcc, 0xb5, 0x8, 0x65, 0xd0, 0xb5, 0x2e, 0xd9, 0xee) typedef struct AMFContextVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFContext* pThis); amf_long (AMF_STD_CALL *Release)(AMFContext* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFContext* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFInterface AMFPropertyStorage AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFContext* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFContext* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFContext* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFContext* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFContext* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFContext* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFContext* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFContext* pThis, AMFPropertyStorageObserver* pObserver); // AMFContext interface // Cleanup AMF_RESULT (AMF_STD_CALL *Terminate)(AMFContext* pThis); // DX9 AMF_RESULT (AMF_STD_CALL *InitDX9)(AMFContext* pThis, void* pDX9Device); void* (AMF_STD_CALL *GetDX9Device)(AMFContext* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX9)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX9)(AMFContext* pThis); // DX11 AMF_RESULT (AMF_STD_CALL *InitDX11)(AMFContext* pThis, void* pDX11Device, AMF_DX_VERSION dxVersionRequired); void* (AMF_STD_CALL *GetDX11Device)(AMFContext* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX11)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX11)(AMFContext* pThis); // OpenCL AMF_RESULT (AMF_STD_CALL *InitOpenCL)(AMFContext* pThis, void* pCommandQueue); void* (AMF_STD_CALL *GetOpenCLContext)(AMFContext* pThis); void* (AMF_STD_CALL *GetOpenCLCommandQueue)(AMFContext* pThis); void* (AMF_STD_CALL *GetOpenCLDeviceID)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *GetOpenCLComputeFactory)(AMFContext* pThis, AMFComputeFactory **ppFactory); // advanced compute - multiple queries AMF_RESULT (AMF_STD_CALL *InitOpenCLEx)(AMFContext* pThis, AMFComputeDevice *pDevice); AMF_RESULT (AMF_STD_CALL *LockOpenCL)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *UnlockOpenCL)(AMFContext* pThis); // OpenGL AMF_RESULT (AMF_STD_CALL *InitOpenGL)(AMFContext* pThis, amf_handle hOpenGLContext, amf_handle hWindow, amf_handle hDC); amf_handle (AMF_STD_CALL *GetOpenGLContext)(AMFContext* pThis); amf_handle (AMF_STD_CALL *GetOpenGLDrawable)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *LockOpenGL)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *UnlockOpenGL)(AMFContext* pThis); // XV - Linux AMF_RESULT (AMF_STD_CALL *InitXV)(AMFContext* pThis, void* pXVDevice); void* (AMF_STD_CALL *GetXVDevice)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *LockXV)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *UnlockXV)(AMFContext* pThis); // Gralloc - Android AMF_RESULT (AMF_STD_CALL *InitGralloc)(AMFContext* pThis, void* pGrallocDevice); void* (AMF_STD_CALL *GetGrallocDevice)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *LockGralloc)(AMFContext* pThis); AMF_RESULT (AMF_STD_CALL *UnlockGralloc)(AMFContext* pThis); // Allocation AMF_RESULT (AMF_STD_CALL *AllocBuffer)(AMFContext* pThis, AMF_MEMORY_TYPE type, amf_size size, AMFBuffer** ppBuffer); AMF_RESULT (AMF_STD_CALL *AllocSurface)(AMFContext* pThis, AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMFSurface** ppSurface); AMF_RESULT (AMF_STD_CALL *AllocAudioBuffer)(AMFContext* pThis, AMF_MEMORY_TYPE type, AMF_AUDIO_FORMAT format, amf_int32 samples, amf_int32 sampleRate, amf_int32 channels, AMFAudioBuffer** ppAudioBuffer); // Wrap existing objects AMF_RESULT (AMF_STD_CALL *CreateBufferFromHostNative)(AMFContext* pThis, void* pHostBuffer, amf_size size, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromHostNative)(AMFContext* pThis, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32 vPitch, void* pData, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX9Native)(AMFContext* pThis, void* pDX9Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX11Native)(AMFContext* pThis, void* pDX11Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromOpenGLNative)(AMFContext* pThis, AMF_SURFACE_FORMAT format, amf_handle hGLTextureID, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromGrallocNative)(AMFContext* pThis, amf_handle hGrallocSurface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromOpenCLNative)(AMFContext* pThis, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, void** pClPlanes, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateBufferFromOpenCLNative)(AMFContext* pThis, void* pCLBuffer, amf_size size, AMFBuffer** ppBuffer); // Access to AMFCompute interface - AMF_MEMORY_OPENCL, AMF_MEMORY_COMPUTE_FOR_DX9, AMF_MEMORY_COMPUTE_FOR_DX11 are currently supported AMF_RESULT (AMF_STD_CALL *GetCompute)(AMFContext* pThis, AMF_MEMORY_TYPE eMemType, AMFCompute** ppCompute); } AMFContextVtbl; struct AMFContext { const AMFContextVtbl *pVtbl; }; typedef struct AMFContext1 AMFContext1; AMF_DECLARE_IID(AMFContext1, 0xd9e9f868, 0x6220, 0x44c6, 0xa2, 0x2f, 0x7c, 0xd6, 0xda, 0xc6, 0x86, 0x46) typedef struct AMFContext1Vtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFContext1* pThis); amf_long (AMF_STD_CALL *Release)(AMFContext1* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFContext1* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFInterface AMFPropertyStorage AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFContext1* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFContext1* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFContext1* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFContext1* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFContext1* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFContext1* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFContext1* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFContext1* pThis, AMFPropertyStorageObserver* pObserver); // AMFContext interface // Cleanup AMF_RESULT (AMF_STD_CALL *Terminate)(AMFContext1* pThis); // DX9 AMF_RESULT (AMF_STD_CALL *InitDX9)(AMFContext1* pThis, void* pDX9Device); void* (AMF_STD_CALL *GetDX9Device)(AMFContext1* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX9)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX9)(AMFContext1* pThis); // DX11 AMF_RESULT (AMF_STD_CALL *InitDX11)(AMFContext1* pThis, void* pDX11Device, AMF_DX_VERSION dxVersionRequired); void* (AMF_STD_CALL *GetDX11Device)(AMFContext1* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX11)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX11)(AMFContext1* pThis); // OpenCL AMF_RESULT (AMF_STD_CALL *InitOpenCL)(AMFContext1* pThis, void* pCommandQueue); void* (AMF_STD_CALL *GetOpenCLContext)(AMFContext1* pThis); void* (AMF_STD_CALL *GetOpenCLCommandQueue)(AMFContext1* pThis); void* (AMF_STD_CALL *GetOpenCLDeviceID)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *GetOpenCLComputeFactory)(AMFContext1* pThis, AMFComputeFactory **ppFactory); // advanced compute - multiple queries AMF_RESULT (AMF_STD_CALL *InitOpenCLEx)(AMFContext1* pThis, AMFComputeDevice *pDevice); AMF_RESULT (AMF_STD_CALL *LockOpenCL)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockOpenCL)(AMFContext1* pThis); // OpenGL AMF_RESULT (AMF_STD_CALL *InitOpenGL)(AMFContext1* pThis, amf_handle hOpenGLContext, amf_handle hWindow, amf_handle hDC); amf_handle (AMF_STD_CALL *GetOpenGLContext)(AMFContext1* pThis); amf_handle (AMF_STD_CALL *GetOpenGLDrawable)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *LockOpenGL)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockOpenGL)(AMFContext1* pThis); // XV - Linux AMF_RESULT (AMF_STD_CALL *InitXV)(AMFContext1* pThis, void* pXVDevice); void* (AMF_STD_CALL *GetXVDevice)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *LockXV)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockXV)(AMFContext1* pThis); // Gralloc - Android AMF_RESULT (AMF_STD_CALL *InitGralloc)(AMFContext1* pThis, void* pGrallocDevice); void* (AMF_STD_CALL *GetGrallocDevice)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *LockGralloc)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockGralloc)(AMFContext1* pThis); // Allocation AMF_RESULT (AMF_STD_CALL *AllocBuffer)(AMFContext1* pThis, AMF_MEMORY_TYPE type, amf_size size, AMFBuffer** ppBuffer); AMF_RESULT (AMF_STD_CALL *AllocSurface)(AMFContext1* pThis, AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMFSurface** ppSurface); AMF_RESULT (AMF_STD_CALL *AllocAudioBuffer)(AMFContext1* pThis, AMF_MEMORY_TYPE type, AMF_AUDIO_FORMAT format, amf_int32 samples, amf_int32 sampleRate, amf_int32 channels, AMFAudioBuffer** ppAudioBuffer); // Wrap existing objects AMF_RESULT (AMF_STD_CALL *CreateBufferFromHostNative)(AMFContext1* pThis, void* pHostBuffer, amf_size size, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromHostNative)(AMFContext1* pThis, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32 vPitch, void* pData, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX9Native)(AMFContext1* pThis, void* pDX9Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX11Native)(AMFContext1* pThis, void* pDX11Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromOpenGLNative)(AMFContext1* pThis, AMF_SURFACE_FORMAT format, amf_handle hGLTextureID, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromGrallocNative)(AMFContext1* pThis, amf_handle hGrallocSurface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromOpenCLNative)(AMFContext1* pThis, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, void** pClPlanes, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateBufferFromOpenCLNative)(AMFContext1* pThis, void* pCLBuffer, amf_size size, AMFBuffer** ppBuffer); // Access to AMFCompute interface - AMF_MEMORY_OPENCL, AMF_MEMORY_COMPUTE_FOR_DX9, AMF_MEMORY_COMPUTE_FOR_DX11 are currently supported AMF_RESULT (AMF_STD_CALL *GetCompute)(AMFContext1* pThis, AMF_MEMORY_TYPE eMemType, AMFCompute** ppCompute); // AMFContext1 interface AMF_RESULT (AMF_STD_CALL *CreateBufferFromDX11Native)(AMFContext1* pThis, void* pHostBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *AllocBufferEx)(AMFContext1* pThis, AMF_MEMORY_TYPE type, amf_size size, AMF_BUFFER_USAGE usage, AMF_MEMORY_CPU_ACCESS access, AMFBuffer** ppBuffer); AMF_RESULT (AMF_STD_CALL *AllocSurfaceEx)(AMFContext1* pThis, AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMF_SURFACE_USAGE usage, AMF_MEMORY_CPU_ACCESS access, AMFSurface** ppSurface); // Vulkan - Windows, Linux AMF_RESULT (AMF_STD_CALL *InitVulkan)(AMFContext1* pThis, void* pVulkanDevice); void* (AMF_STD_CALL *GetVulkanDevice)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *LockVulkan)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *UnlockVulkan)(AMFContext1* pThis); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromVulkanNative)(AMFContext1* pThis, void* pVulkanImage, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateBufferFromVulkanNative)(AMFContext1* pThis, void* pVulkanBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *GetVulkanDeviceExtensions)(AMFContext1* pThis, amf_size *pCount, const char **ppExtensions); } AMFContext1Vtbl; struct AMFContext1 { const AMFContext1Vtbl *pVtbl; }; typedef struct AMFContext2 AMFContext2; AMF_DECLARE_IID(AMFContext2, 0xd9e9f868, 0x6220, 0x44c6, 0xa2, 0x2f, 0x7c, 0xd6, 0xda, 0xc6, 0x86, 0x46) typedef struct AMFContext2Vtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFContext2* pThis); amf_long (AMF_STD_CALL *Release)(AMFContext2* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFContext2* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFInterface AMFPropertyStorage AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFContext2* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFContext2* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFContext2* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFContext2* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFContext2* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFContext2* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFContext2* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFContext2* pThis, AMFPropertyStorageObserver* pObserver); // AMFContext interface // Cleanup AMF_RESULT (AMF_STD_CALL *Terminate)(AMFContext2* pThis); // DX9 AMF_RESULT (AMF_STD_CALL *InitDX9)(AMFContext2* pThis, void* pDX9Device); void* (AMF_STD_CALL *GetDX9Device)(AMFContext2* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX9)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX9)(AMFContext2* pThis); // DX11 AMF_RESULT (AMF_STD_CALL *InitDX11)(AMFContext2* pThis, void* pDX11Device, AMF_DX_VERSION dxVersionRequired); void* (AMF_STD_CALL *GetDX11Device)(AMFContext2* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX11)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX11)(AMFContext2* pThis); // OpenCL AMF_RESULT (AMF_STD_CALL *InitOpenCL)(AMFContext2* pThis, void* pCommandQueue); void* (AMF_STD_CALL *GetOpenCLContext)(AMFContext2* pThis); void* (AMF_STD_CALL *GetOpenCLCommandQueue)(AMFContext2* pThis); void* (AMF_STD_CALL *GetOpenCLDeviceID)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *GetOpenCLComputeFactory)(AMFContext2* pThis, AMFComputeFactory **ppFactory); // advanced compute - multiple queries AMF_RESULT (AMF_STD_CALL *InitOpenCLEx)(AMFContext2* pThis, AMFComputeDevice *pDevice); AMF_RESULT (AMF_STD_CALL *LockOpenCL)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockOpenCL)(AMFContext2* pThis); // OpenGL AMF_RESULT (AMF_STD_CALL *InitOpenGL)(AMFContext2* pThis, amf_handle hOpenGLContext, amf_handle hWindow, amf_handle hDC); amf_handle (AMF_STD_CALL *GetOpenGLContext)(AMFContext2* pThis); amf_handle (AMF_STD_CALL *GetOpenGLDrawable)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *LockOpenGL)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockOpenGL)(AMFContext2* pThis); // XV - Linux AMF_RESULT (AMF_STD_CALL *InitXV)(AMFContext2* pThis, void* pXVDevice); void* (AMF_STD_CALL *GetXVDevice)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *LockXV)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockXV)(AMFContext2* pThis); // Gralloc - Android AMF_RESULT (AMF_STD_CALL *InitGralloc)(AMFContext2* pThis, void* pGrallocDevice); void* (AMF_STD_CALL *GetGrallocDevice)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *LockGralloc)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockGralloc)(AMFContext2* pThis); // Allocation AMF_RESULT (AMF_STD_CALL *AllocBuffer)(AMFContext2* pThis, AMF_MEMORY_TYPE type, amf_size size, AMFBuffer** ppBuffer); AMF_RESULT (AMF_STD_CALL *AllocSurface)(AMFContext2* pThis, AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMFSurface** ppSurface); AMF_RESULT (AMF_STD_CALL *AllocAudioBuffer)(AMFContext2* pThis, AMF_MEMORY_TYPE type, AMF_AUDIO_FORMAT format, amf_int32 samples, amf_int32 sampleRate, amf_int32 channels, AMFAudioBuffer** ppAudioBuffer); // Wrap existing objects AMF_RESULT (AMF_STD_CALL *CreateBufferFromHostNative)(AMFContext2* pThis, void* pHostBuffer, amf_size size, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromHostNative)(AMFContext2* pThis, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32 vPitch, void* pData,AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX9Native)(AMFContext2* pThis, void* pDX9Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX11Native)(AMFContext2* pThis, void* pDX11Surface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromOpenGLNative)(AMFContext2* pThis, AMF_SURFACE_FORMAT format, amf_handle hGLTextureID, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromGrallocNative)(AMFContext2* pThis, amf_handle hGrallocSurface, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromOpenCLNative)(AMFContext2* pThis, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, void** pClPlanes, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateBufferFromOpenCLNative)(AMFContext2* pThis, void* pCLBuffer, amf_size size, AMFBuffer** ppBuffer); // Access to AMFCompute interface - AMF_MEMORY_OPENCL, AMF_MEMORY_COMPUTE_FOR_DX9, AMF_MEMORY_COMPUTE_FOR_DX11 are currently supported AMF_RESULT (AMF_STD_CALL *GetCompute)(AMFContext2* pThis, AMF_MEMORY_TYPE eMemType, AMFCompute** ppCompute); // AMFContext1 interface AMF_RESULT (AMF_STD_CALL *CreateBufferFromDX11Native)(AMFContext2* pThis, void* pHostBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *AllocBufferEx)(AMFContext2* pThis, AMF_MEMORY_TYPE type, amf_size size, AMF_BUFFER_USAGE usage, AMF_MEMORY_CPU_ACCESS access, AMFBuffer** ppBuffer); AMF_RESULT (AMF_STD_CALL *AllocSurfaceEx)(AMFContext2* pThis, AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT format, amf_int32 width, amf_int32 height, AMF_SURFACE_USAGE usage, AMF_MEMORY_CPU_ACCESS access, AMFSurface** ppSurface); // Vulkan - Windows, Linux AMF_RESULT (AMF_STD_CALL *InitVulkan)(AMFContext2* pThis, void* pVulkanDevice); void* (AMF_STD_CALL *GetVulkanDevice)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *LockVulkan)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockVulkan)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromVulkanNative)(AMFContext2* pThis, void* pVulkanImage, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateBufferFromVulkanNative)(AMFContext2* pThis, void* pVulkanBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); AMF_RESULT (AMF_STD_CALL *GetVulkanDeviceExtensions)(AMFContext2* pThis, amf_size *pCount, const char **ppExtensions); // AMFContext2 interface AMF_RESULT (AMF_STD_CALL *InitDX12)(AMFContext2* pThis, void* pDX11Device, AMF_DX_VERSION dxVersionRequired); void* (AMF_STD_CALL *GetDX12Device)(AMFContext2* pThis, AMF_DX_VERSION dxVersionRequired); AMF_RESULT (AMF_STD_CALL *LockDX12)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *UnlockDX12)(AMFContext2* pThis); AMF_RESULT (AMF_STD_CALL *CreateSurfaceFromDX12Native)(AMFContext2* pThis, void* pResourceTexture, AMFSurface** ppSurface, AMFSurfaceObserver* pObserver); AMF_RESULT (AMF_STD_CALL *CreateBufferFromDX12Native)(AMFContext2* pThis, void* pResourceBuffer, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver); } AMFContext2Vtbl; struct AMFContext2 { const AMFContext2Vtbl *pVtbl; }; #endif #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // Lockers //---------------------------------------------------------------------------------------------- class AMFContext::AMFDX9Locker { public: AMFDX9Locker() : m_Context(NULL) {} AMFDX9Locker(AMFContext* resources) : m_Context(NULL) { Lock(resources); } ~AMFDX9Locker() { if(m_Context != NULL) { m_Context->UnlockDX9(); } } void Lock(AMFContext* resources) { if(m_Context != NULL) { m_Context->UnlockDX9(); } m_Context = resources; if(m_Context != NULL) { m_Context->LockDX9(); } } protected: AMFContext* m_Context; private: AMFDX9Locker(const AMFDX9Locker&); AMFDX9Locker& operator=(const AMFDX9Locker&); }; //---------------------------------------------------------------------------------------------- class AMFContext::AMFDX11Locker { public: AMFDX11Locker() : m_Context(NULL) {} AMFDX11Locker(AMFContext* resources) : m_Context(NULL) { Lock(resources); } ~AMFDX11Locker() { if(m_Context != NULL) { m_Context->UnlockDX11(); } } void Lock(AMFContext* resources) { if(m_Context != NULL) { m_Context->UnlockDX11(); } m_Context = resources; if(m_Context != NULL) { m_Context->LockDX11(); } } protected: AMFContext* m_Context; private: AMFDX11Locker(const AMFDX11Locker&); AMFDX11Locker& operator=(const AMFDX11Locker&); }; //---------------------------------------------------------------------------------------------- class AMFContext::AMFOpenCLLocker { public: AMFOpenCLLocker() : m_Context(NULL) {} AMFOpenCLLocker(AMFContext* resources) : m_Context(NULL) { Lock(resources); } ~AMFOpenCLLocker() { if(m_Context != NULL) { m_Context->UnlockOpenCL(); } } void Lock(AMFContext* resources) { if(m_Context != NULL) { m_Context->UnlockOpenCL(); } m_Context = resources; if(m_Context != NULL) { m_Context->LockOpenCL(); } } protected: AMFContext* m_Context; private: AMFOpenCLLocker(const AMFOpenCLLocker&); AMFOpenCLLocker& operator=(const AMFOpenCLLocker&); }; //---------------------------------------------------------------------------------------------- class AMFContext::AMFOpenGLLocker { public: AMFOpenGLLocker(AMFContext* pContext) : m_pContext(pContext), m_GLLocked(false) { if(m_pContext != NULL) { if(m_pContext->LockOpenGL() == AMF_OK) { m_GLLocked = true; } } } ~AMFOpenGLLocker() { if(m_GLLocked) { m_pContext->UnlockOpenGL(); } } private: AMFContext* m_pContext; amf_bool m_GLLocked; ///< AMFOpenGLLocker can be called when OpenGL is not initialized yet ///< in this case don't call UnlockOpenGL AMFOpenGLLocker(const AMFOpenGLLocker&); AMFOpenGLLocker& operator=(const AMFOpenGLLocker&); }; //---------------------------------------------------------------------------------------------- class AMFContext::AMFXVLocker { public: AMFXVLocker() : m_pContext(NULL) {} AMFXVLocker(AMFContext* pContext) : m_pContext(NULL) { Lock(pContext); } ~AMFXVLocker() { if(m_pContext != NULL) { m_pContext->UnlockXV(); } } void Lock(AMFContext* pContext) { if((pContext != NULL) && (pContext->GetXVDevice() != NULL)) { m_pContext = pContext; m_pContext->LockXV(); } } protected: AMFContext* m_pContext; private: AMFXVLocker(const AMFXVLocker&); AMFXVLocker& operator=(const AMFXVLocker&); }; //---------------------------------------------------------------------------------------------- class AMFContext::AMFGrallocLocker { public: AMFGrallocLocker() : m_pContext(NULL) {} AMFGrallocLocker(AMFContext* pContext) : m_pContext(NULL) { Lock(pContext); } ~AMFGrallocLocker() { if(m_pContext != NULL) { m_pContext->UnlockGralloc(); } } void Lock(AMFContext* pContext) { if((pContext != NULL) && (pContext->GetGrallocDevice() != NULL)) { m_pContext = pContext; m_pContext->LockGralloc(); } } protected: AMFContext* m_pContext; private: AMFGrallocLocker(const AMFGrallocLocker&); AMFGrallocLocker& operator=(const AMFGrallocLocker&); }; //---------------------------------------------------------------------------------------------- class AMFContext1::AMFVulkanLocker { public: AMFVulkanLocker() : m_pContext(NULL) {} AMFVulkanLocker(AMFContext1* pContext) : m_pContext(NULL) { Lock(pContext); } ~AMFVulkanLocker() { if(m_pContext != NULL) { m_pContext->UnlockVulkan(); } } void Lock(AMFContext1* pContext) { if((pContext != NULL) && (pContext->GetVulkanDevice() != NULL)) { m_pContext = pContext; m_pContext->LockVulkan(); } } protected: AMFContext1* m_pContext; private: AMFVulkanLocker(const AMFVulkanLocker&); AMFVulkanLocker& operator=(const AMFVulkanLocker&); }; //---------------------------------------------------------------------------------------------- class AMFContext2::AMFDX12Locker { public: AMFDX12Locker() : m_Context(NULL) {} AMFDX12Locker(AMFContext2* resources) : m_Context(NULL) { Lock(resources); } ~AMFDX12Locker() { if (m_Context != NULL) { m_Context->UnlockDX12(); } } void Lock(AMFContext2* resources) { if (m_Context != NULL) { m_Context->UnlockDX12(); } m_Context = resources; if (m_Context != NULL) { m_Context->LockDX12(); } } protected: AMFContext2* m_Context; private: AMFDX12Locker(const AMFDX12Locker&); AMFDX12Locker& operator=(const AMFDX12Locker&); }; //---------------------------------------------------------------------------------------------- //---------------------------------------------------------------------------------------------- //---------------------------------------------------------------------------------------------- #endif #if defined(__cplusplus) } #endif enum AMF_CONTEXT_DEVICETYPE_ENUM { AMF_CONTEXT_DEVICE_TYPE_GPU = 0, AMF_CONTEXT_DEVICE_TYPE_CPU }; #define AMF_CONTEXT_DEVICE_TYPE L"AMF_Context_DeviceType" //Value type: amf_int64; Values : AMF_CONTEXT_DEVICE_TYPE_GPU for GPU (default) , AMF_CONTEXT_DEVICE_TYPE_CPU for CPU. #endif //#ifndef AMF_Context_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/CurrentTime.h ================================================ // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_CurrentTime_h #define AMF_CurrentTime_h #include "Platform.h" #include "Interface.h" namespace amf { // Current time interface class. This interface object can be passed // as a property to components requiring synchronized timing. The // implementation is: // - first call to Get() starts time and returns 0 // - subsequent calls to Get() returns values relative to 0 // - Reset() puts time back at 0 at next Get() call // class AMF_NO_VTABLE AMFCurrentTime : public AMFInterface { public: virtual amf_pts AMF_STD_CALL Get() = 0; virtual void AMF_STD_CALL Reset() = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFCurrentTimePtr; //----------------------------------------------------------------------------------------------} } #endif // AMF_CurrentTime_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/D3D12AMF.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef __D3D12AMF_h__ #define __D3D12AMF_h__ #pragma once #include "Platform.h" #if defined(_WIN32)||(defined(__linux) && defined(AMF_WSL)) #define AMFDX12_NUMBER_OF_DESCRYPTOR_HEAPS L"NumberOfDescryptorHeaps" // amf_int64, default is 4, to be set on AMFContext // syncronization properties set via SetPrivateData() AMF_WEAK GUID AMFResourceStateGUID = { 0x452da9bf, 0x4ad7, 0x47a5, { 0xa6, 0x9b, 0x96, 0xd3, 0x23, 0x76, 0xf2, 0xf3 } }; // Current resource state value (D3D12_RESOURCE_STATES ), sizeof(UINT), set on ID3D12Resource AMF_WEAK GUID AMFFenceGUID = { 0x910a7928, 0x57bd, 0x4b04, { 0x91, 0xa3, 0xe7, 0xb8, 0x04, 0x12, 0xcd, 0xa5 } }; // IUnknown (ID3D12Fence), set on ID3D12Resource syncronization fence for this resource AMF_WEAK GUID AMFFenceValueGUID = { 0x62a693d3, 0xbb4a, 0x46c9, { 0xa5, 0x04, 0x9a, 0x8e, 0x97, 0xbf, 0xf0, 0x56 } }; // The last value to wait on the fence from AMFFenceGUID; sizeof(UINT64), set on ID3D12Fence AMF_WEAK GUID AMFFenceD3D11GUID = { 0xdffdf6e0, 0x85e0, 0x4645, { 0x9d, 0x7, 0xe6, 0x4a, 0x19, 0x6b, 0xc9, 0xbf } }; // IUnknown (ID3D11Fence) OpenSharedFence for interop AMF_WEAK GUID AMFFenceValueD3D11GUID = { 0x86581b71, 0x699f, 0x484b, { 0xb8, 0x75, 0x24, 0xda, 0x49, 0x8a, 0x74, 0xcf } }; // last value to wait on in d3d11 AMF_WEAK GUID AMFSharedHandleFenceGUID = { 0xca60dcc8, 0x76d1, 0x4088, 0xad, 0xd, 0x97, 0x71, 0xe7, 0xb0, 0x92, 0x49 }; // ID3D12Fence shared handle for D3D11 interop #endif #endif // __D3D12AMF_h__ ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Data.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Data_h #define AMF_Data_h #pragma once #include "PropertyStorage.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- typedef enum AMF_DATA_TYPE { AMF_DATA_BUFFER = 0, AMF_DATA_SURFACE = 1, AMF_DATA_AUDIO_BUFFER = 2, AMF_DATA_USER = 1000, // all extensions will be AMF_DATA_USER+i } AMF_DATA_TYPE; //---------------------------------------------------------------------------------------------- typedef enum AMF_MEMORY_TYPE { AMF_MEMORY_UNKNOWN = 0, AMF_MEMORY_HOST = 1, AMF_MEMORY_DX9 = 2, AMF_MEMORY_DX11 = 3, AMF_MEMORY_OPENCL = 4, AMF_MEMORY_OPENGL = 5, AMF_MEMORY_XV = 6, AMF_MEMORY_GRALLOC = 7, AMF_MEMORY_COMPUTE_FOR_DX9 = 8, // deprecated, the same as AMF_MEMORY_OPENCL AMF_MEMORY_COMPUTE_FOR_DX11 = 9, // deprecated, the same as AMF_MEMORY_OPENCL AMF_MEMORY_VULKAN = 10, AMF_MEMORY_DX12 = 11, } AMF_MEMORY_TYPE; //---------------------------------------------------------------------------------------------- typedef enum AMF_DX_VERSION { AMF_DX9 = 90, AMF_DX9_EX = 91, AMF_DX11_0 = 110, AMF_DX11_1 = 111, AMF_DX12 = 120, } AMF_DX_VERSION; //---------------------------------------------------------------------------------------------- // AMF_MEMORY_CPU_ACCESS translates to D3D11_CPU_ACCESS_FLAG or VkImageUsageFlags // bit mask //---------------------------------------------------------------------------------------------- typedef enum AMF_MEMORY_CPU_ACCESS_BITS { // D3D11 D3D12 Vulkan AMF_MEMORY_CPU_DEFAULT = 0x80000000, // 0 , D3D12_HEAP_TYPE_DEFAULT , VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT AMF_MEMORY_CPU_NONE = 0x00000000, // 0 , D3D12_HEAP_TYPE_DEFAULT , AMF_MEMORY_CPU_READ = 0x00000001, // D3D11_CPU_ACCESS_READ , D3D12_HEAP_TYPE_READBACK, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT AMF_MEMORY_CPU_WRITE = 0x00000002, // D3D11_CPU_ACCESS_WRITE, D3D12_HEAP_TYPE_UPLOAD , VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT AMF_MEMORY_CPU_LOCAL = 0x00000004, // , D3D12_HEAP_TYPE_DEFAULT , VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT AMF_MEMORY_CPU_PINNED = 0x00000008, // , , VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT_KHR } AMF_MEMORY_CPU_ACCESS_BITS; typedef amf_flags AMF_MEMORY_CPU_ACCESS; //---------------------------------------------------------------------------------------------- // AMFData interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFData : public AMFPropertyStorage { public: AMF_DECLARE_IID(0xa1159bf6, 0x9104, 0x4107, 0x8e, 0xaa, 0xc5, 0x3d, 0x5d, 0xba, 0xc5, 0x11) virtual AMF_MEMORY_TYPE AMF_STD_CALL GetMemoryType() = 0; virtual AMF_RESULT AMF_STD_CALL Duplicate(AMF_MEMORY_TYPE type, AMFData** ppData) = 0; virtual AMF_RESULT AMF_STD_CALL Convert(AMF_MEMORY_TYPE type) = 0; // optimal interop if possilble. Copy through host memory if needed virtual AMF_RESULT AMF_STD_CALL Interop(AMF_MEMORY_TYPE type) = 0; // only optimal interop if possilble. No copy through host memory for GPU objects virtual AMF_DATA_TYPE AMF_STD_CALL GetDataType() = 0; virtual amf_bool AMF_STD_CALL IsReusable() = 0; virtual void AMF_STD_CALL SetPts(amf_pts pts) = 0; virtual amf_pts AMF_STD_CALL GetPts() = 0; virtual void AMF_STD_CALL SetDuration(amf_pts duration) = 0; virtual amf_pts AMF_STD_CALL GetDuration() = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFDataPtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) typedef struct AMFData AMFData; AMF_DECLARE_IID(AMFData, 0xa1159bf6, 0x9104, 0x4107, 0x8e, 0xaa, 0xc5, 0x3d, 0x5d, 0xba, 0xc5, 0x11) typedef struct AMFDataVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFData* pThis); amf_long (AMF_STD_CALL *Release)(AMFData* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFData* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFData* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFData* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFData* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFData* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFData* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFData* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFData* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFData* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFData* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFData* pThis, AMFPropertyStorageObserver* pObserver); // AMFData interface AMF_MEMORY_TYPE (AMF_STD_CALL *GetMemoryType)(AMFData* pThis); AMF_RESULT (AMF_STD_CALL *Duplicate)(AMFData* pThis, AMF_MEMORY_TYPE type, AMFData** ppData); AMF_RESULT (AMF_STD_CALL *Convert)(AMFData* pThis, AMF_MEMORY_TYPE type); // optimal interop if possilble. Copy through host memory if needed AMF_RESULT (AMF_STD_CALL *Interop)(AMFData* pThis, AMF_MEMORY_TYPE type); // only optimal interop if possilble. No copy through host memory for GPU objects AMF_DATA_TYPE (AMF_STD_CALL *GetDataType)(AMFData* pThis); amf_bool (AMF_STD_CALL *IsReusable)(AMFData* pThis); void (AMF_STD_CALL *SetPts)(AMFData* pThis, amf_pts pts); amf_pts (AMF_STD_CALL *GetPts)(AMFData* pThis); void (AMF_STD_CALL *SetDuration)(AMFData* pThis, amf_pts duration); amf_pts (AMF_STD_CALL *GetDuration)(AMFData* pThis); } AMFDataVtbl; struct AMFData { const AMFDataVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace #endif #endif //#ifndef AMF_Data_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Debug.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Debug_h #define AMF_Debug_h #pragma once #include "Platform.h" #include "Result.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMFDebug interface - singleton //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFDebug { public: virtual void AMF_STD_CALL EnablePerformanceMonitor(amf_bool enable) = 0; virtual amf_bool AMF_STD_CALL PerformanceMonitorEnabled() = 0; virtual void AMF_STD_CALL AssertsEnable(amf_bool enable) = 0; virtual amf_bool AMF_STD_CALL AssertsEnabled() = 0; }; #else // #if defined(__cplusplus) typedef struct AMFDebug AMFDebug; typedef struct AMFDebugVtbl { // AMFDebug interface void (AMF_STD_CALL *EnablePerformanceMonitor)(AMFDebug* pThis, amf_bool enable); amf_bool (AMF_STD_CALL *PerformanceMonitorEnabled)(AMFDebug* pThis); void (AMF_STD_CALL *AssertsEnable)(AMFDebug* pThis, amf_bool enable); amf_bool (AMF_STD_CALL *AssertsEnabled)(AMFDebug* pThis); } AMFDebugVtbl; struct AMFDebug { const AMFDebugVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) } #endif #endif // AMF_Debug_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Dump.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Dump_h #define AMF_Dump_h #pragma once #include "Platform.h" #include "Result.h" #include "Interface.h" #if defined(__cplusplus) namespace amf { #endif #if defined(__cplusplus) class AMF_NO_VTABLE AMFDump : public AMFInterface { public: AMF_DECLARE_IID(0x75366ad4, 0x504c, 0x430b, 0xbb, 0xe2, 0xad, 0x21, 0x82, 0x8, 0xf, 0x72); virtual const wchar_t* AMF_STD_CALL GetDumpBasePath() const = 0; // Get application dump base path virtual AMF_RESULT AMF_STD_CALL SetDumpBasePath(const wchar_t* path) = 0; // Set application dump base path // Enable/disable input and/or output stream dumps virtual bool AMF_STD_CALL IsInputDumpEnabled() const = 0; virtual AMF_RESULT AMF_STD_CALL EnableInputDump(bool enabled) = 0; virtual const wchar_t* AMF_STD_CALL GetInputDumpFullName() const = 0; // Get full name of dump file // Enable/disable input and/or output stream dumps virtual bool AMF_STD_CALL IsOutputDumpEnabled() const = 0; virtual AMF_RESULT AMF_STD_CALL EnableOutputDump(bool enabled) = 0; virtual const wchar_t* AMF_STD_CALL GetOutputDumpFullName() const = 0; // Get full name of dump file // When enabled, each new application session will create a subfolder with a time stamp in the base path tree (disabled by default) virtual bool AMF_STD_CALL IsPerSessionDumpEnabled() const = 0; virtual void AMF_STD_CALL EnablePerSessionDump(bool enabled) = 0; }; typedef AMFInterfacePtr_T AMFDumpPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFDump, 0x75366ad4, 0x504c, 0x430b, 0xbb, 0xe2, 0xad, 0x21, 0x82, 0x8, 0xf, 0x72); typedef struct AMFDump AMFDump; typedef struct AMFDumpVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFDump* pThis); amf_long (AMF_STD_CALL *Release)(AMFDump* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFDump* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFDump interface const wchar_t* (AMF_STD_CALL *GetDumpBasePath)(AMFDump* pThis) const; // Get application dump base path AMF_RESULT (AMF_STD_CALL *SetDumpBasePath)(AMFDump* pThis, const wchar_t* path); // Set application dump base path // Enable/disable input and/or output stream dumps bool (AMF_STD_CALL *IsInputDumpEnabled)(AMFDump* pThis) const; AMF_RESULT (AMF_STD_CALL *EnableInputDump)(AMFDump* pThis, bool enabled); const wchar_t* (AMF_STD_CALL *GetInputDumpFullName)(AMFDump* pThis) const; // Get full name of dump file // Enable/disable input and/or output stream dumps bool (AMF_STD_CALL *IsOutputDumpEnabled)(AMFDump* pThis) const; AMF_RESULT (AMF_STD_CALL *EnableOutputDump)(AMFDump* pThis, bool enabled); const wchar_t* (AMF_STD_CALL *GetOutputDumpFullName)(AMFDump* pThis) const; // Get full name of dump file // When enabled, each new application session will create a subfolder with a time stamp in the base path tree (disabled by default) bool (AMF_STD_CALL *IsPerSessionDumpEnabled)(AMFDump* pThis) const; void (AMF_STD_CALL *EnablePerSessionDump)(AMFDump* pThis, bool enabled); } AMFDumpVtbl; struct AMFDump { const AMFDumpVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace #endif #endif //AMF_Dump_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Factory.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Factory_h #define AMF_Factory_h #pragma once #include "Platform.h" #include "Version.h" #include "Result.h" #include "Context.h" #include "Debug.h" #include "Trace.h" #include "Compute.h" #include "../components/Component.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMFFactory interface - singleton //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFFactory { public: virtual AMF_RESULT AMF_STD_CALL CreateContext(AMFContext** ppContext) = 0; virtual AMF_RESULT AMF_STD_CALL CreateComponent(AMFContext* pContext, const wchar_t* id, AMFComponent** ppComponent) = 0; virtual AMF_RESULT AMF_STD_CALL SetCacheFolder(const wchar_t* path) = 0; virtual const wchar_t* AMF_STD_CALL GetCacheFolder() = 0; virtual AMF_RESULT AMF_STD_CALL GetDebug(AMFDebug** ppDebug) = 0; virtual AMF_RESULT AMF_STD_CALL GetTrace(AMFTrace** ppTrace) = 0; virtual AMF_RESULT AMF_STD_CALL GetPrograms(AMFPrograms** ppPrograms) = 0; }; #else typedef struct AMFFactory AMFFactory; typedef struct AMFFactoryVtbl { AMF_RESULT (AMF_STD_CALL *CreateContext)(AMFFactory* pThis, AMFContext** ppContext); AMF_RESULT (AMF_STD_CALL *CreateComponent)(AMFFactory* pThis, AMFContext* pContext, const wchar_t* id, AMFComponent** ppComponent); AMF_RESULT (AMF_STD_CALL *SetCacheFolder)(AMFFactory* pThis, const wchar_t* path); const wchar_t* (AMF_STD_CALL *GetCacheFolder)(AMFFactory* pThis); AMF_RESULT (AMF_STD_CALL *GetDebug)(AMFFactory* pThis, AMFDebug** ppDebug); AMF_RESULT (AMF_STD_CALL *GetTrace)(AMFFactory* pThis, AMFTrace** ppTrace); AMF_RESULT (AMF_STD_CALL *GetPrograms)(AMFFactory* pThis, AMFPrograms** ppPrograms); } AMFFactoryVtbl; struct AMFFactory { const AMFFactoryVtbl *pVtbl; }; #endif #if defined(__cplusplus) } #endif //---------------------------------------------------------------------------------------------- // DLL entry points //---------------------------------------------------------------------------------------------- #define AMF_INIT_FUNCTION_NAME "AMFInit" #define AMF_QUERY_VERSION_FUNCTION_NAME "AMFQueryVersion" #if defined(__cplusplus) extern "C" { typedef AMF_RESULT (AMF_CDECL_CALL *AMFInit_Fn)(amf_uint64 version, amf::AMFFactory **ppFactory); typedef AMF_RESULT (AMF_CDECL_CALL *AMFQueryVersion_Fn)(amf_uint64 *pVersion); } #else typedef AMF_RESULT (AMF_CDECL_CALL *AMFInit_Fn)(amf_uint64 version, AMFFactory **ppFactory); typedef AMF_RESULT (AMF_CDECL_CALL *AMFQueryVersion_Fn)(amf_uint64 *pVersion); #endif #if defined(_WIN32) #if defined(_M_AMD64) #define AMF_DLL_NAME L"amfrt64.dll" #define AMF_DLL_NAMEA "amfrt64.dll" #else #define AMF_DLL_NAME L"amfrt32.dll" #define AMF_DLL_NAMEA "amfrt32.dll" #endif #elif defined(__ANDROID__) && !defined(AMF_ANDROID_ENCODER) #define AMF_DLL_NAME L"libamf.so" #define AMF_DLL_NAMEA "libamf.so" #elif defined(__APPLE__) #define AMF_DLL_NAME L"libamfrt.framework/libamfrt" #define AMF_DLL_NAMEA "libamfrt.framework/libamfrt" #elif defined(__linux__) #if defined(__x86_64__) || defined(__aarch64__) #define AMF_DLL_NAME L"libamfrt64.so.1" #define AMF_DLL_NAMEA "libamfrt64.so.1" #else #define AMF_DLL_NAME L"libamfrt32.so.1" #define AMF_DLL_NAMEA "libamfrt32.so.1" #endif #endif //---------------------------------------------------------------------------------------------- #endif // AMF_Factory_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Interface.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Interface_h #define AMF_Interface_h #pragma once #include "Result.h" #if defined(__cplusplus) namespace amf { #endif #if defined(__cplusplus) #define AMF_DECLARE_IID(_data1, _data2, _data3, _data41, _data42, _data43, _data44, _data45, _data46, _data47, _data48) \ static AMF_INLINE const amf::AMFGuid IID() \ { \ amf::AMFGuid uid = {_data1, _data2, _data3, _data41, _data42, _data43, _data44, _data45, _data46, _data47, _data48}; \ return uid; \ } #else #define AMF_DECLARE_IID(name, _data1, _data2, _data3, _data41, _data42, _data43, _data44, _data45, _data46, _data47, _data48) \ AMF_INLINE static const AMFGuid IID_##name(void) \ { \ AMFGuid uid = {_data1, _data2, _data3, _data41, _data42, _data43, _data44, _data45, _data46, _data47, _data48}; \ return uid; \ } #endif //------------------------------------------------------------------------ // AMFInterface interface - base class for all AMF interfaces //------------------------------------------------------------------------ #if defined(__cplusplus) class AMF_NO_VTABLE AMFInterface { public: AMF_DECLARE_IID(0x9d872f34, 0x90dc, 0x4b93, 0xb6, 0xb2, 0x6c, 0xa3, 0x7c, 0x85, 0x25, 0xdb) virtual amf_long AMF_STD_CALL Acquire() = 0; virtual amf_long AMF_STD_CALL Release() = 0; virtual AMF_RESULT AMF_STD_CALL QueryInterface(const AMFGuid& interfaceID, void** ppInterface) = 0; }; #else AMF_DECLARE_IID(AMFInterface, 0x9d872f34, 0x90dc, 0x4b93, 0xb6, 0xb2, 0x6c, 0xa3, 0x7c, 0x85, 0x25, 0xdb) typedef struct AMFInterface AMFInterface; typedef struct AMFInterfaceVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFInterface* pThis); amf_long (AMF_STD_CALL *Release)(AMFInterface* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFInterface* pThis, const struct AMFGuid *interfaceID, void** ppInterface); } AMFInterfaceVtbl; struct AMFInterface { const AMFInterfaceVtbl *pVtbl; }; #endif //------------------------------------------------------------------------ // template for AMF smart pointer //------------------------------------------------------------------------ #if defined(__cplusplus) template class AMFInterfacePtr_T { private: _Interf* m_pInterf; void InternalAcquire() { if(m_pInterf != NULL) { m_pInterf->Acquire(); } } void InternalRelease() { if(m_pInterf != NULL) { m_pInterf->Release(); } } public: AMFInterfacePtr_T() : m_pInterf(NULL) {} AMFInterfacePtr_T(const AMFInterfacePtr_T<_Interf>& p) : m_pInterf(p.m_pInterf) { InternalAcquire(); } AMFInterfacePtr_T(_Interf* pInterface) : m_pInterf(pInterface) { InternalAcquire(); } template explicit AMFInterfacePtr_T(const AMFInterfacePtr_T<_OtherInterf>& cp) : m_pInterf(NULL) { void* pInterf = NULL; if((cp == NULL) || (cp->QueryInterface(_Interf::IID(), &pInterf) != AMF_OK)) { pInterf = NULL; } m_pInterf = static_cast<_Interf*>(pInterf); } template explicit AMFInterfacePtr_T(_OtherInterf* cp) : m_pInterf(NULL) { void* pInterf = NULL; if((cp == NULL) || (cp->QueryInterface(_Interf::IID(), &pInterf) != AMF_OK)) { pInterf = NULL; } m_pInterf = static_cast<_Interf*>(pInterf); } ~AMFInterfacePtr_T() { InternalRelease(); } AMFInterfacePtr_T& operator=(_Interf* pInterface) { if(m_pInterf != pInterface) { _Interf* pOldInterface = m_pInterf; m_pInterf = pInterface; InternalAcquire(); if(pOldInterface != NULL) { pOldInterface->Release(); } } return *this; } AMFInterfacePtr_T& operator=(const AMFInterfacePtr_T<_Interf>& cp) { return operator=(cp.m_pInterf); } void Attach(_Interf* pInterface) { InternalRelease(); m_pInterf = pInterface; } _Interf* Detach() { _Interf* const pOld = m_pInterf; m_pInterf = NULL; return pOld; } void Release() { InternalRelease(); m_pInterf = NULL; } operator _Interf*() const { return m_pInterf; } _Interf& operator*() const { return *m_pInterf; } // Returns the address of the interface pointer contained in this // class. This is required for initializing from C-style factory function to // avoid getting an incorrect ref count at the beginning. _Interf** operator&() { InternalRelease(); m_pInterf = 0; return &m_pInterf; } _Interf* operator->() const { return m_pInterf; } bool operator==(const AMFInterfacePtr_T<_Interf>& p) { return (m_pInterf == p.m_pInterf); } bool operator==(_Interf* p) { return (m_pInterf == p); } bool operator!=(const AMFInterfacePtr_T<_Interf>& p) { return !(operator==(p)); } bool operator!=(_Interf* p) { return !(operator==(p)); } _Interf* GetPtr() { return m_pInterf; } const _Interf* GetPtr() const { return m_pInterf; } }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFInterfacePtr; //---------------------------------------------------------------------------------------------- #endif #if defined(__cplusplus) } #endif #endif //#ifndef AMF_Interface_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Plane.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Plane_h #define AMF_Plane_h #pragma once #include "Interface.h" #if defined(__cplusplus) namespace amf { #endif //--------------------------------------------------------------------------------------------- typedef enum AMF_PLANE_TYPE { AMF_PLANE_UNKNOWN = 0, AMF_PLANE_PACKED = 1, // for all packed formats: BGRA, YUY2, etc AMF_PLANE_Y = 2, AMF_PLANE_UV = 3, AMF_PLANE_U = 4, AMF_PLANE_V = 5, } AMF_PLANE_TYPE; //--------------------------------------------------------------------------------------------- // AMFPlane interface //--------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFPlane : public AMFInterface { public: AMF_DECLARE_IID(0xbede1aa6, 0xd8fa, 0x4625, 0x94, 0x65, 0x6c, 0x82, 0xc4, 0x37, 0x71, 0x2e) virtual AMF_PLANE_TYPE AMF_STD_CALL GetType() = 0; virtual void* AMF_STD_CALL GetNative() = 0; virtual amf_int32 AMF_STD_CALL GetPixelSizeInBytes() = 0; virtual amf_int32 AMF_STD_CALL GetOffsetX() = 0; virtual amf_int32 AMF_STD_CALL GetOffsetY() = 0; virtual amf_int32 AMF_STD_CALL GetWidth() = 0; virtual amf_int32 AMF_STD_CALL GetHeight() = 0; virtual amf_int32 AMF_STD_CALL GetHPitch() = 0; virtual amf_int32 AMF_STD_CALL GetVPitch() = 0; virtual bool AMF_STD_CALL IsTiled() = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFPlanePtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFPlane, 0xbede1aa6, 0xd8fa, 0x4625, 0x94, 0x65, 0x6c, 0x82, 0xc4, 0x37, 0x71, 0x2e) typedef struct AMFPlane AMFPlane; typedef struct AMFPlaneVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFPlane* pThis); amf_long (AMF_STD_CALL *Release)(AMFPlane* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFPlane* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPlane interface AMF_PLANE_TYPE (AMF_STD_CALL *GetType)(AMFPlane* pThis); void* (AMF_STD_CALL *GetNative)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetPixelSizeInBytes)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetOffsetX)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetOffsetY)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetWidth)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetHeight)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetHPitch)(AMFPlane* pThis); amf_int32 (AMF_STD_CALL *GetVPitch)(AMFPlane* pThis); amf_bool (AMF_STD_CALL *IsTiled)(AMFPlane* pThis); } AMFPlaneVtbl; struct AMFPlane { const AMFPlaneVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } // namespace amf #endif #endif //#ifndef AMF_Plane_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Platform.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Platform_h #define AMF_Platform_h #pragma once //---------------------------------------------------------------------------------------------- // export declaration //---------------------------------------------------------------------------------------------- #if defined(_WIN32) #if defined(AMF_CORE_STATIC) #define AMF_CORE_LINK #else #if defined(AMF_CORE_EXPORTS) #define AMF_CORE_LINK __declspec(dllexport) #else #define AMF_CORE_LINK __declspec(dllimport) #endif #endif #elif defined(__linux) #if defined(AMF_CORE_EXPORTS) #define AMF_CORE_LINK __attribute__((visibility("default"))) #else #define AMF_CORE_LINK #endif #else #define AMF_CORE_LINK #endif // #ifdef _WIN32 #if defined(_DEBUG) && !defined(DEBUG) // prevents other headers to #define DEBUG without assigned value what causes failure in PAL build #define DEBUG 1 #endif #define AMF_MACRO_STRING2(x) #x #define AMF_MACRO_STRING(x) AMF_MACRO_STRING2(x) #define AMF_TODO(_todo) (__FILE__ "(" AMF_MACRO_STRING(__LINE__) "): TODO: "_todo) #if defined(__GNUC__) || defined(__clang__) #define AMF_ALIGN(n) __attribute__((aligned(n))) #elif defined(_MSC_VER) || defined(__INTEL_COMPILER) #define AMF_ALIGN(n) __declspec(align(n)) #else #define AMF_ALIGN(n) // #error Need to define AMF_ALIGN #endif #ifndef _WIN32 typedef signed int HRESULT; #define SUCCEEDED(hr) (((HRESULT)(hr)) >= 0) #define FAILED(hr) (((HRESULT)(hr)) < 0) #endif #include #include #include #if defined(_WIN32) #ifndef NOMINMAX #define NOMINMAX #endif #define AMF_STD_CALL __stdcall #define AMF_CDECL_CALL __cdecl #define AMF_FAST_CALL __fastcall #if defined(__GNUC__) || defined(__clang__) #define AMF_INLINE inline #define AMF_FORCEINLINE inline #else #define AMF_INLINE __inline #define AMF_FORCEINLINE __forceinline #endif #define AMF_NO_VTABLE __declspec(novtable) #define AMFPRId64 "I64d" #define LPRId64 L"I64d" #define AMFPRIud64 "Iu64d" #define LPRIud64 L"Iu64d" #define AMFPRIx64 "I64x" #define LPRIx64 L"I64x" #else // !WIN32 - Linux and Mac #define AMF_STD_CALL #define AMF_CDECL_CALL #define AMF_FAST_CALL #if defined(__GNUC__) || defined(__clang__) #define AMF_INLINE inline #define AMF_FORCEINLINE inline #else #define AMF_INLINE __inline__ #define AMF_FORCEINLINE __inline__ #endif #define AMF_NO_VTABLE #if !defined(AMFPRId64) #define AMFPRId64 "lld" #define LPRId64 L"lld" #define AMFPRIud64 "ulld" #define LPRIud64 L"ulld" #define AMFPRIx64 "llx" #define LPRIx64 L"llx" #endif #endif // WIN32 #if defined(_WIN32) #define AMF_WEAK __declspec( selectany ) #elif defined (__GNUC__) || defined (__GCC__) || defined(__clang__)//GCC or CLANG #define AMF_WEAK __attribute__((weak)) #endif #define amf_countof(x) (sizeof(x) / sizeof(x[0])) //------------------------------------------------------------------------------------------------- // basic data types //------------------------------------------------------------------------------------------------- typedef int64_t amf_int64; typedef int32_t amf_int32; typedef int16_t amf_int16; typedef int8_t amf_int8; typedef uint64_t amf_uint64; typedef uint32_t amf_uint32; typedef uint16_t amf_uint16; typedef uint8_t amf_uint8; typedef size_t amf_size; typedef void* amf_handle; typedef double amf_double; typedef float amf_float; typedef void amf_void; #if defined(__cplusplus) typedef bool amf_bool; #else typedef amf_uint8 amf_bool; #define true 1 #define false 0 #endif typedef long amf_long; typedef int amf_int; typedef unsigned long amf_ulong; typedef unsigned int amf_uint; typedef amf_int64 amf_pts; // in 100 nanosecs typedef amf_uint32 amf_flags; #define AMF_SECOND 10000000L // 1 second in 100 nanoseconds #define AMF_MILLISECOND (AMF_SECOND / 1000) #define AMF_MICROSECOND (AMF_MILLISECOND / 1000) #define AMF_MIN(a, b) ((a) < (b) ? (a) : (b)) #define AMF_MAX(a, b) ((a) > (b) ? (a) : (b)) #define AMF_CLAMP(x, a, b) (AMF_MIN(AMF_MAX(x, a), b)) #define AMF_BITS_PER_BYTE 8 #if defined(_WIN32) #define PATH_SEPARATOR_WSTR L"\\" #define PATH_SEPARATOR_WCHAR L'\\' #elif defined(__linux) || defined(__APPLE__) // Linux & Apple #define PATH_SEPARATOR_WSTR L"/" #define PATH_SEPARATOR_WCHAR L'/' #endif typedef struct AMFRect { amf_int32 left; amf_int32 top; amf_int32 right; amf_int32 bottom; #if defined(__cplusplus) bool operator==(const AMFRect& other) const { return left == other.left && top == other.top && right == other.right && bottom == other.bottom; } AMF_INLINE bool operator!=(const AMFRect& other) const { return !operator==(other); } amf_int32 Width() const { return right - left; } amf_int32 Height() const { return bottom - top; } #endif } AMFRect; static AMF_INLINE struct AMFRect AMFConstructRect(amf_int32 left, amf_int32 top, amf_int32 right, amf_int32 bottom) { struct AMFRect object = {left, top, right, bottom}; return object; } typedef struct AMFSize { amf_int32 width; amf_int32 height; #if defined(__cplusplus) bool operator==(const AMFSize& other) const { return width == other.width && height == other.height; } AMF_INLINE bool operator!=(const AMFSize& other) const { return !operator==(other); } #endif } AMFSize; static AMF_INLINE struct AMFSize AMFConstructSize(amf_int32 width, amf_int32 height) { struct AMFSize object = {width, height}; return object; } typedef struct AMFPoint { amf_int32 x; amf_int32 y; #if defined(__cplusplus) bool operator==(const AMFPoint& other) const { return x == other.x && y == other.y; } AMF_INLINE bool operator!=(const AMFPoint& other) const { return !operator==(other); } #endif } AMFPoint; static AMF_INLINE struct AMFPoint AMFConstructPoint(amf_int32 x, amf_int32 y) { struct AMFPoint object = { x, y }; return object; } typedef struct AMFFloatPoint2D { amf_float x; amf_float y; #if defined(__cplusplus) bool operator==(const AMFFloatPoint2D& other) const { return x == other.x && y == other.y; } AMF_INLINE bool operator!=(const AMFFloatPoint2D& other) const { return !operator==(other); } #endif } AMFFloatPoint2D; static AMF_INLINE struct AMFFloatPoint2D AMFConstructFloatPoint2D(amf_float x, amf_float y) { struct AMFFloatPoint2D object = {x, y}; return object; } typedef struct AMFFloatSize { amf_float width; amf_float height; #if defined(__cplusplus) bool operator==(const AMFFloatSize& other) const { return width == other.width && height == other.height; } AMF_INLINE bool operator!=(const AMFFloatSize& other) const { return !operator==(other); } #endif } AMFFloatSize; static AMF_INLINE struct AMFFloatSize AMFConstructFloatSize(amf_float w, amf_float h) { struct AMFFloatSize object = { w, h }; return object; } typedef struct AMFFloatPoint3D { amf_float x; amf_float y; amf_float z; #if defined(__cplusplus) bool operator==(const AMFFloatPoint3D& other) const { return x == other.x && y == other.y && z == other.z; } AMF_INLINE bool operator!=(const AMFFloatPoint3D& other) const { return !operator==(other); } #endif } AMFFloatPoint3D; static AMF_INLINE struct AMFFloatPoint3D AMFConstructFloatPoint3D(amf_float x, amf_float y, amf_float z) { struct AMFFloatPoint3D object = { x, y, z }; return object; } typedef struct AMFFloatVector4D { amf_float x; amf_float y; amf_float z; amf_float w; #if defined(__cplusplus) bool operator==(const AMFFloatVector4D& other) const { return x == other.x && y == other.y && z == other.z && w == other.w; } AMF_INLINE bool operator!=(const AMFFloatVector4D& other) const { return !operator==(other); } #endif } AMFFloatVector4D; static AMF_INLINE struct AMFFloatVector4D AMFConstructFloatVector4D(amf_float x, amf_float y, amf_float z, amf_float w) { struct AMFFloatVector4D object = { x, y, z, w }; return object; } typedef struct AMFRate { amf_uint32 num; amf_uint32 den; #if defined(__cplusplus) bool operator==(const AMFRate& other) const { return num == other.num && den == other.den; } AMF_INLINE bool operator!=(const AMFRate& other) const { return !operator==(other); } #endif } AMFRate; static AMF_INLINE struct AMFRate AMFConstructRate(amf_uint32 num, amf_uint32 den) { struct AMFRate object = {num, den}; return object; } typedef struct AMFRatio { amf_uint32 num; amf_uint32 den; #if defined(__cplusplus) bool operator==(const AMFRatio& other) const { return num == other.num && den == other.den; } AMF_INLINE bool operator!=(const AMFRatio& other) const { return !operator==(other); } #endif } AMFRatio; static AMF_INLINE struct AMFRatio AMFConstructRatio(amf_uint32 num, amf_uint32 den) { struct AMFRatio object = {num, den}; return object; } #pragma pack(push, 1) #if defined(_MSC_VER) #pragma warning( push ) #pragma warning(disable : 4200) #pragma warning(disable : 4201) #endif typedef struct AMFColor { union { struct { amf_uint8 r; amf_uint8 g; amf_uint8 b; amf_uint8 a; }; amf_uint32 rgba; }; #if defined(__cplusplus) bool operator==(const AMFColor& other) const { return r == other.r && g == other.g && b == other.b && a == other.a; } AMF_INLINE bool operator!=(const AMFColor& other) const { return !operator==(other); } #endif } AMFColor; #if defined(_MSC_VER) #pragma warning( pop ) #endif #pragma pack(pop) static AMF_INLINE struct AMFColor AMFConstructColor(amf_uint8 r, amf_uint8 g, amf_uint8 b, amf_uint8 a) { struct AMFColor object; object.r = r; object.g = g; object.b = b; object.a = a; return object; } #if defined(_WIN32) #include #if defined(__cplusplus) extern "C" { #endif // allocator static AMF_INLINE void* AMF_CDECL_CALL amf_variant_alloc(amf_size count) { return CoTaskMemAlloc(count); } static AMF_INLINE void AMF_CDECL_CALL amf_variant_free(void* ptr) { CoTaskMemFree(ptr); } #if defined(__cplusplus) } #endif #else // defined(_WIN32) #include #if defined(__cplusplus) extern "C" { #endif // allocator static AMF_INLINE void* AMF_CDECL_CALL amf_variant_alloc(amf_size count) { return malloc(count); } static AMF_INLINE void AMF_CDECL_CALL amf_variant_free(void* ptr) { free(ptr); } #if defined(__cplusplus) } #endif #endif // defined(_WIN32) #if defined(__cplusplus) namespace amf { #endif typedef struct AMFGuid { amf_uint32 data1; amf_uint16 data2; amf_uint16 data3; amf_uint8 data41; amf_uint8 data42; amf_uint8 data43; amf_uint8 data44; amf_uint8 data45; amf_uint8 data46; amf_uint8 data47; amf_uint8 data48; #if defined(__cplusplus) AMFGuid(amf_uint32 _data1, amf_uint16 _data2, amf_uint16 _data3, amf_uint8 _data41, amf_uint8 _data42, amf_uint8 _data43, amf_uint8 _data44, amf_uint8 _data45, amf_uint8 _data46, amf_uint8 _data47, amf_uint8 _data48) : data1 (_data1), data2 (_data2), data3 (_data3), data41(_data41), data42(_data42), data43(_data43), data44(_data44), data45(_data45), data46(_data46), data47(_data47), data48(_data48) {} bool operator==(const AMFGuid& other) const { return data1 == other.data1 && data2 == other.data2 && data3 == other.data3 && data41 == other.data41 && data42 == other.data42 && data43 == other.data43 && data44 == other.data44 && data45 == other.data45 && data46 == other.data46 && data47 == other.data47 && data48 == other.data48; } AMF_INLINE bool operator!=(const AMFGuid& other) const { return !operator==(other); } #endif } AMFGuid; #if defined(__cplusplus) static AMF_INLINE bool AMFCompareGUIDs(const AMFGuid& guid1, const AMFGuid& guid2) { return guid1 == guid2; } #else static AMF_INLINE amf_bool AMFCompareGUIDs(const struct AMFGuid guid1, const struct AMFGuid guid2) { return memcmp(&guid1, &guid2, sizeof(guid1)) == 0; } #endif #if defined(__cplusplus) } #endif #if defined(__APPLE__) //#include #define media_status_t int #define ANativeWindow void #define JNIEnv void #define jobject int #define JavaVM void #endif #endif //#ifndef AMF_Platform_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/PropertyStorage.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_PropertyStorage_h #define AMF_PropertyStorage_h #pragma once #include "Variant.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // AMFPropertyStorageObserver interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFPropertyStorageObserver { public: virtual void AMF_STD_CALL OnPropertyChanged(const wchar_t* name) = 0; }; #else //#if defined(__cplusplus) typedef struct AMFPropertyStorageObserver AMFPropertyStorageObserver; typedef struct AMFPropertyStorageObserverVtbl { void (AMF_STD_CALL *OnPropertyChanged)(AMFPropertyStorageObserver *pThis, const wchar_t* name); } AMFPropertyStorageObserverVtbl; struct AMFPropertyStorageObserver { const AMFPropertyStorageObserverVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFPropertyStorage interface //---------------------------------------------------------------------------------------------- class AMF_NO_VTABLE AMFPropertyStorage : public AMFInterface { public: AMF_DECLARE_IID(0xc7cec05b, 0xcfb9, 0x48af, 0xac, 0xe3, 0xf6, 0x8d, 0xf8, 0x39, 0x5f, 0xe3) virtual AMF_RESULT AMF_STD_CALL SetProperty(const wchar_t* name, AMFVariantStruct value) = 0; virtual AMF_RESULT AMF_STD_CALL GetProperty(const wchar_t* name, AMFVariantStruct* pValue) const = 0; virtual amf_bool AMF_STD_CALL HasProperty(const wchar_t* name) const = 0; virtual amf_size AMF_STD_CALL GetPropertyCount() const = 0; virtual AMF_RESULT AMF_STD_CALL GetPropertyAt(amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue) const = 0; virtual AMF_RESULT AMF_STD_CALL Clear() = 0; virtual AMF_RESULT AMF_STD_CALL AddTo(AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep) const= 0; virtual AMF_RESULT AMF_STD_CALL CopyTo(AMFPropertyStorage* pDest, amf_bool deep) const = 0; virtual void AMF_STD_CALL AddObserver(AMFPropertyStorageObserver* pObserver) = 0; virtual void AMF_STD_CALL RemoveObserver(AMFPropertyStorageObserver* pObserver) = 0; template AMF_RESULT AMF_STD_CALL SetProperty(const wchar_t* name, const _T& value); template AMF_RESULT AMF_STD_CALL GetProperty(const wchar_t* name, _T* pValue) const; template AMF_RESULT AMF_STD_CALL GetPropertyString(const wchar_t* name, _T* pValue) const; template AMF_RESULT AMF_STD_CALL GetPropertyWString(const wchar_t* name, _T* pValue) const; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFPropertyStoragePtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) typedef struct AMFPropertyStorage AMFPropertyStorage; AMF_DECLARE_IID(AMFPropertyStorage, 0xc7cec05b, 0xcfb9, 0x48af, 0xac, 0xe3, 0xf6, 0x8d, 0xf8, 0x39, 0x5f, 0xe3) typedef struct AMFPropertyStorageVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFPropertyStorage* pThis); amf_long (AMF_STD_CALL *Release)(AMFPropertyStorage* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFPropertyStorage* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFPropertyStorage* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFPropertyStorage* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFPropertyStorage* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFPropertyStorage* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFPropertyStorage* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFPropertyStorage* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFPropertyStorage* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFPropertyStorage* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFPropertyStorage* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFPropertyStorage* pThis, AMFPropertyStorageObserver* pObserver); } AMFPropertyStorageVtbl; struct AMFPropertyStorage { const AMFPropertyStorageVtbl *pVtbl; }; #define AMF_ASSIGN_PROPERTY_DATA(res, varType, pThis, name, val ) \ { \ AMFVariantStruct var = {0}; \ AMFVariantAssign##varType(&var, val); \ res = pThis->pVtbl->SetProperty(pThis, name, var ); \ } #define AMF_QUERY_INTERFACE(res, from, InterfaceTypeTo, to) \ { \ AMFGuid guid_##InterfaceTypeTo = IID_##InterfaceTypeTo(); \ res = from->pVtbl->QueryInterface(from, &guid_##InterfaceTypeTo, (void**)&to); \ } #define AMF_ASSIGN_PROPERTY_INTERFACE(res, pThis, name, val) \ { \ AMFInterface *amf_interface; \ AMFVariantStruct var; \ res = AMFVariantInit(&var); \ if (res == AMF_OK) \ { \ AMF_QUERY_INTERFACE(res, val, AMFInterface, amf_interface)\ if (res == AMF_OK) \ { \ res = AMFVariantAssignInterface(&var, amf_interface); \ amf_interface->pVtbl->Release(amf_interface); \ if (res == AMF_OK) \ { \ res = pThis->pVtbl->SetProperty(pThis, name, var); \ } \ } \ AMFVariantClear(&var); \ } \ } #define AMF_GET_PROPERTY_INTERFACE(res, pThis, name, TargetType, val) \ { \ AMFVariantStruct var; \ res = AMFVariantInit(&var); \ if (res != AMF_OK) \ { \ res = pThis->pVtbl->GetProperty(pThis, name, &var); \ if (res == AMF_OK) \ { \ if (var.type == AMF_VARIANT_INTERFACE && AMFVariantInterface(&var)) \ { \ AMF_QUERY_INTERFACE(res, AMFVariantInterface(&var), TargetType, val); \ } \ else \ { \ res = AMF_INVALID_DATA_TYPE; \ } \ } \ } \ AMFVariantClear(&var); \ } #define AMF_ASSIGN_PROPERTY_TYPE(res, varType, dataType , pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, varType, pThis, name, (dataType)val) #define AMF_ASSIGN_PROPERTY_INT64(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_TYPE(res, Int64, amf_int64, pThis, name, val) #define AMF_ASSIGN_PROPERTY_DOUBLE(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_TYPE(res, Double, amf_double, pThis, name, val) #define AMF_ASSIGN_PROPERTY_BOOL(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_TYPE(res, Bool, amf_bool, pThis, name, val) #define AMF_ASSIGN_PROPERTY_RECT(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, Rect, pThis, name, &val) #define AMF_ASSIGN_PROPERTY_SIZE(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, Size, pThis, name, &val) #define AMF_ASSIGN_PROPERTY_POINT(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, Point, pThis, name, &val) #define AMF_ASSIGN_PROPERTY_RATE(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, Rate, pThis, name, &val) #define AMF_ASSIGN_PROPERTY_RATIO(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, Ratio, pThis, name, &val) #define AMF_ASSIGN_PROPERTY_COLOR(res, pThis, name, val ) AMF_ASSIGN_PROPERTY_DATA(res, Color, pThis, name, &val) #endif // #if defined(__cplusplus) #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // template methods implementations //---------------------------------------------------------------------------------------------- template inline AMF_RESULT AMF_STD_CALL AMFPropertyStorage::SetProperty(const wchar_t* name, const _T& value) { AMF_RESULT err = SetProperty(name, static_cast(AMFVariant(value))); return err; } //---------------------------------------------------------------------------------------------- template inline AMF_RESULT AMF_STD_CALL AMFPropertyStorage::GetProperty(const wchar_t* name, _T* pValue) const { AMFVariant var; AMF_RESULT err = GetProperty(name, static_cast(&var)); if(err == AMF_OK) { *pValue = static_cast<_T>(var); } return err; } //---------------------------------------------------------------------------------------------- template inline AMF_RESULT AMF_STD_CALL AMFPropertyStorage::GetPropertyString(const wchar_t* name, _T* pValue) const { AMFVariant var; AMF_RESULT err = GetProperty(name, static_cast(&var)); if(err == AMF_OK) { *pValue = var.ToString().c_str(); } return err; } //---------------------------------------------------------------------------------------------- template inline AMF_RESULT AMF_STD_CALL AMFPropertyStorage::GetPropertyWString(const wchar_t* name, _T* pValue) const { AMFVariant var; AMF_RESULT err = GetProperty(name, static_cast(&var)); if(err == AMF_OK) { *pValue = var.ToWString().c_str(); } return err; } //---------------------------------------------------------------------------------------------- template<> inline AMF_RESULT AMF_STD_CALL AMFPropertyStorage::GetProperty(const wchar_t* name, AMFInterface** ppValue) const { AMFVariant var; AMF_RESULT err = GetProperty(name, static_cast(&var)); if(err == AMF_OK) { *ppValue = static_cast(var); } if(*ppValue) { (*ppValue)->Acquire(); } return err; } #endif // #if defined(__cplusplus) #if defined(__cplusplus) } //namespace amf #endif #endif // #ifndef AMF_PropertyStorage_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/PropertyStorageEx.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_PropertyStorageEx_h #define AMF_PropertyStorageEx_h #pragma once #include "PropertyStorage.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- typedef enum AMF_PROPERTY_CONTENT_ENUM { AMF_PROPERTY_CONTENT_DEFAULT = 0, AMF_PROPERTY_CONTENT_XML, // m_eType is AMF_VARIANT_STRING AMF_PROPERTY_CONTENT_FILE_OPEN_PATH, // m_eType AMF_VARIANT_WSTRING AMF_PROPERTY_CONTENT_FILE_SAVE_PATH, // m_eType AMF_VARIANT_WSTRING AMF_PROPERTY_CONTENT_INTEGER_ARRAY, // m_eType AMF_VARIANT_INTERFACE AMF_PROPERTY_CONTENT_FLOAT_ARRAY // m_eType AMF_VARIANT_INTERFACE } AMF_PROPERTY_CONTENT_ENUM; //---------------------------------------------------------------------------------------------- typedef enum AMF_PROPERTY_ACCESS_TYPE { AMF_PROPERTY_ACCESS_PRIVATE = 0, AMF_PROPERTY_ACCESS_READ = 0x1, AMF_PROPERTY_ACCESS_WRITE = 0x2, AMF_PROPERTY_ACCESS_READ_WRITE = (AMF_PROPERTY_ACCESS_READ | AMF_PROPERTY_ACCESS_WRITE), AMF_PROPERTY_ACCESS_WRITE_RUNTIME = 0x4, AMF_PROPERTY_ACCESS_FULL = 0xFF, AMF_PROPERTY_ACCESS_NON_PERSISTANT = 0x4000, AMF_PROPERTY_ACCESS_NON_PERSISTANT_READ = (AMF_PROPERTY_ACCESS_NON_PERSISTANT | AMF_PROPERTY_ACCESS_READ), AMF_PROPERTY_ACCESS_NON_PERSISTANT_READ_WRITE = (AMF_PROPERTY_ACCESS_NON_PERSISTANT | AMF_PROPERTY_ACCESS_READ_WRITE), AMF_PROPERTY_ACCESS_NON_PERSISTANT_FULL = (AMF_PROPERTY_ACCESS_NON_PERSISTANT | AMF_PROPERTY_ACCESS_FULL), AMF_PROPERTY_ACCESS_INVALID = 0x8000 } AMF_PROPERTY_ACCESS_TYPE; //---------------------------------------------------------------------------------------------- typedef struct AMFEnumDescriptionEntry { amf_int value; const wchar_t* name; } AMFEnumDescriptionEntry; //---------------------------------------------------------------------------------------------- typedef amf_uint32 AMF_PROPERTY_CONTENT_TYPE; typedef struct AMFPropertyInfo { const wchar_t* name; const wchar_t* desc; AMF_VARIANT_TYPE type; AMF_PROPERTY_CONTENT_TYPE contentType; AMFVariantStruct defaultValue; AMFVariantStruct minValue; AMFVariantStruct maxValue; AMF_PROPERTY_ACCESS_TYPE accessType; const AMFEnumDescriptionEntry* pEnumDescription; #if defined(__cplusplus) AMFPropertyInfo() : name(NULL), desc(NULL), type(), contentType(), defaultValue(), minValue(), maxValue(), accessType(AMF_PROPERTY_ACCESS_FULL), pEnumDescription(NULL) {} AMFPropertyInfo(const AMFPropertyInfo& propery) : name(propery.name), desc(propery.desc), type(propery.type), contentType(propery.contentType), defaultValue(propery.defaultValue), minValue(propery.minValue), maxValue(propery.maxValue), accessType(propery.accessType), pEnumDescription(propery.pEnumDescription) {} virtual ~AMFPropertyInfo(){} amf_bool AMF_STD_CALL AllowedRead() const { return (accessType & AMF_PROPERTY_ACCESS_READ) != 0; } amf_bool AMF_STD_CALL AllowedWrite() const { return (accessType & AMF_PROPERTY_ACCESS_WRITE) != 0; } amf_bool AMF_STD_CALL AllowedChangeInRuntime() const { return (accessType & AMF_PROPERTY_ACCESS_WRITE_RUNTIME) != 0; } AMFPropertyInfo& operator=(const AMFPropertyInfo& propery) { name = propery.name; desc = propery.desc; type = propery.type; contentType = propery.contentType; defaultValue = propery.defaultValue; minValue = propery.minValue; maxValue = propery.maxValue; accessType = propery.accessType; pEnumDescription = propery.pEnumDescription; return *this; } #endif // #if defined(__cplusplus) } AMFPropertyInfo; //---------------------------------------------------------------------------------------------- // AMFPropertyStorageEx interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFPropertyStorageEx : public AMFPropertyStorage { public: AMF_DECLARE_IID(0x16b8958d, 0xe943, 0x4a33, 0xa3, 0x5a, 0x88, 0x5a, 0xd8, 0x28, 0xf2, 0x67) virtual amf_size AMF_STD_CALL GetPropertiesInfoCount() const = 0; virtual AMF_RESULT AMF_STD_CALL GetPropertyInfo(amf_size index, const AMFPropertyInfo** ppInfo) const = 0; virtual AMF_RESULT AMF_STD_CALL GetPropertyInfo(const wchar_t* name, const AMFPropertyInfo** ppInfo) const = 0; virtual AMF_RESULT AMF_STD_CALL ValidateProperty(const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated) const = 0; }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFPropertyStorageExPtr; #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFPropertyStorageEx, 0x16b8958d, 0xe943, 0x4a33, 0xa3, 0x5a, 0x88, 0x5a, 0xd8, 0x28, 0xf2, 0x67) typedef struct AMFPropertyStorageEx AMFPropertyStorageEx; typedef struct AMFPropertyStorageExVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFPropertyStorageEx* pThis); amf_long (AMF_STD_CALL *Release)(AMFPropertyStorageEx* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFPropertyStorageEx* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFPropertyStorageEx* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFPropertyStorageEx* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFPropertyStorageEx* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFPropertyStorageEx* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFPropertyStorageEx* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFPropertyStorageEx* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFPropertyStorageEx* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFPropertyStorageEx* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFPropertyStorageEx* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFPropertyStorageEx* pThis, AMFPropertyStorageObserver* pObserver); // AMFPropertyStorageEx interface amf_size (AMF_STD_CALL *GetPropertiesInfoCount)(AMFPropertyStorageEx* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyInfoAt)(AMFPropertyStorageEx* pThis, amf_size index, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *GetPropertyInfo)(AMFPropertyStorageEx* pThis, const wchar_t* name, const AMFPropertyInfo** ppInfo); AMF_RESULT (AMF_STD_CALL *ValidateProperty)(AMFPropertyStorageEx* pThis, const wchar_t* name, AMFVariantStruct value, AMFVariantStruct* pOutValidated); } AMFPropertyStorageExVtbl; struct AMFPropertyStorageEx { const AMFPropertyStorageExVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) } //namespace amf #endif #endif //#ifndef AMF_PropertyStorageEx_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Result.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Result_h #define AMF_Result_h #pragma once #include "Platform.h" //---------------------------------------------------------------------------------------------- // result codes //---------------------------------------------------------------------------------------------- typedef enum AMF_RESULT { AMF_OK = 0, AMF_FAIL , // common errors AMF_UNEXPECTED , AMF_ACCESS_DENIED , AMF_INVALID_ARG , AMF_OUT_OF_RANGE , AMF_OUT_OF_MEMORY , AMF_INVALID_POINTER , AMF_NO_INTERFACE , AMF_NOT_IMPLEMENTED , AMF_NOT_SUPPORTED , AMF_NOT_FOUND , AMF_ALREADY_INITIALIZED , AMF_NOT_INITIALIZED , AMF_INVALID_FORMAT ,// invalid data format AMF_WRONG_STATE , AMF_FILE_NOT_OPEN ,// cannot open file // device common codes AMF_NO_DEVICE , // device directx AMF_DIRECTX_FAILED , // device opencl AMF_OPENCL_FAILED , // device opengl AMF_GLX_FAILED ,//failed to use GLX // device XV AMF_XV_FAILED , //failed to use Xv extension // device alsa AMF_ALSA_FAILED ,//failed to use ALSA // component common codes //result codes AMF_EOF , AMF_REPEAT , AMF_INPUT_FULL ,//returned by AMFComponent::SubmitInput if input queue is full AMF_RESOLUTION_CHANGED ,//resolution changed client needs to Drain/Terminate/Init AMF_RESOLUTION_UPDATED ,//resolution changed in adaptive mode. New ROI will be set on output on newly decoded frames //error codes AMF_INVALID_DATA_TYPE ,//invalid data type AMF_INVALID_RESOLUTION ,//invalid resolution (width or height) AMF_CODEC_NOT_SUPPORTED ,//codec not supported AMF_SURFACE_FORMAT_NOT_SUPPORTED ,//surface format not supported AMF_SURFACE_MUST_BE_SHARED ,//surface should be shared (DX11: (MiscFlags & D3D11_RESOURCE_MISC_SHARED) == 0, DX9: No shared handle found) // component video decoder AMF_DECODER_NOT_PRESENT ,//failed to create the decoder AMF_DECODER_SURFACE_ALLOCATION_FAILED ,//failed to create the surface for decoding AMF_DECODER_NO_FREE_SURFACES , // component video encoder AMF_ENCODER_NOT_PRESENT ,//failed to create the encoder // component video processor // component video conveter // component dem AMF_DEM_ERROR , AMF_DEM_PROPERTY_READONLY , AMF_DEM_REMOTE_DISPLAY_CREATE_FAILED , AMF_DEM_START_ENCODING_FAILED , AMF_DEM_QUERY_OUTPUT_FAILED , // component TAN AMF_TAN_CLIPPING_WAS_REQUIRED , // Resulting data was truncated to meet output type's value limits. AMF_TAN_UNSUPPORTED_VERSION , // Not supported version requested, solely for TANCreateContext(). AMF_NEED_MORE_INPUT ,//returned by AMFComponent::SubmitInput did not produce a buffer because more input submissions are required. // device vulkan AMF_VULKAN_FAILED , } AMF_RESULT; #endif //#ifndef AMF_Result_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Surface.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Surface_h #define AMF_Surface_h #pragma once #include "Data.h" #include "Plane.h" #if defined(_MSC_VER) #pragma warning( push ) #pragma warning(disable : 4263) #pragma warning(disable : 4264) #endif #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- typedef enum AMF_SURFACE_FORMAT { AMF_SURFACE_UNKNOWN = 0, AMF_SURFACE_NV12, ///< 1 - planar 4:2:0 Y width x height + packed UV width/2 x height/2 - 8 bit per component AMF_SURFACE_YV12, ///< 2 - planar 4:2:0 Y width x height + V width/2 x height/2 + U width/2 x height/2 - 8 bit per component AMF_SURFACE_BGRA, ///< 3 - packed 4:4:4 - 8 bit per component AMF_SURFACE_ARGB, ///< 4 - packed 4:4:4 - 8 bit per component AMF_SURFACE_RGBA, ///< 5 - packed 4:4:4 - 8 bit per component AMF_SURFACE_GRAY8, ///< 6 - single component - 8 bit AMF_SURFACE_YUV420P, ///< 7 - planar 4:2:0 Y width x height + U width/2 x height/2 + V width/2 x height/2 - 8 bit per component AMF_SURFACE_U8V8, ///< 8 - packed double component - 8 bit per component AMF_SURFACE_YUY2, ///< 9 - packed 4:2:2 Byte 0=8-bit Y'0; Byte 1=8-bit Cb; Byte 2=8-bit Y'1; Byte 3=8-bit Cr AMF_SURFACE_P010, ///< 10 - planar 4:2:0 Y width x height + packed UV width/2 x height/2 - 10 bit per component (16 allocated, upper 10 bits are used) AMF_SURFACE_RGBA_F16, ///< 11 - packed 4:4:4 - 16 bit per component float AMF_SURFACE_UYVY, ///< 12 - packed 4:2:2 the similar to YUY2 but Y and UV swapped: Byte 0=8-bit Cb; Byte 1=8-bit Y'0; Byte 2=8-bit Cr Byte 3=8-bit Y'1; (used the same DX/CL/Vulkan storage as YUY2) AMF_SURFACE_R10G10B10A2, ///< 13 - packed 4:4:4 to 4 bytes, 10 bit per RGB component, 2 bits per A AMF_SURFACE_Y210, ///< 14 - packed 4:2:2 - Word 0=10-bit Y'0; Word 1=10-bit Cb; Word 2=10-bit Y'1; Word 3=10-bit Cr AMF_SURFACE_AYUV, ///< 15 - packed 4:4:4 - 8 bit per component YUVA AMF_SURFACE_Y410, ///< 16 - packed 4:4:4 - 10 bit per YUV component, 2 bits per A, AVYU AMF_SURFACE_Y416, ///< 16 - packed 4:4:4 - 16 bit per component 4 bytes, AVYU AMF_SURFACE_GRAY32, ///< 17 - single component - 32 bit AMF_SURFACE_P012, ///< 18 - planar 4:2:0 Y width x height + packed UV width/2 x height/2 - 12 bit per component (16 allocated, upper 12 bits are used) AMF_SURFACE_P016, ///< 19 - planar 4:2:0 Y width x height + packed UV width/2 x height/2 - 16 bit per component (16 allocated, all bits are used) AMF_SURFACE_FIRST = AMF_SURFACE_NV12, AMF_SURFACE_LAST = AMF_SURFACE_P016 } AMF_SURFACE_FORMAT; //---------------------------------------------------------------------------------------------- // AMF_SURFACE_USAGE translates to D3D11_BIND_FLAG or VkImageUsageFlags // bit mask //---------------------------------------------------------------------------------------------- typedef enum AMF_SURFACE_USAGE_BITS { // D3D11 D3D12 Vulkan AMF_SURFACE_USAGE_DEFAULT = 0x80000000, // will apply default D3D12_RESOURCE_FLAG_NONE VK_IMAGE_USAGE_TRANSFER_SRC_BIT| VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_INPUT_ATTACHMENT_BIT AMF_SURFACE_USAGE_NONE = 0x00000000, // 0, D3D12_RESOURCE_FLAG_NONE, 0 AMF_SURFACE_USAGE_SHADER_RESOURCE = 0x00000001, // D3D11_BIND_SHADER_RESOURCE, D3D12_RESOURCE_FLAG_NONE VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_INPUT_ATTACHMENT_BIT AMF_SURFACE_USAGE_RENDER_TARGET = 0x00000002, // D3D11_BIND_RENDER_TARGET, D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET, VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT AMF_SURFACE_USAGE_UNORDERED_ACCESS = 0x00000004, // D3D11_BIND_UNORDERED_ACCESS, D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS, VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_INPUT_ATTACHMENT_BIT AMF_SURFACE_USAGE_TRANSFER_SRC = 0x00000008, // D3D12_RESOURCE_FLAG_NONE VK_IMAGE_USAGE_TRANSFER_SRC_BIT AMF_SURFACE_USAGE_TRANSFER_DST = 0x00000010, // D3D12_RESOURCE_FLAG_NONE VK_IMAGE_USAGE_TRANSFER_DST_BIT AMF_SURFACE_USAGE_LINEAR = 0x00000020, // AMF_SURFACE_USAGE_NOSYNC = 0x00000040, // no fence (AMFFenceGUID) created no semaphore (AMFVulkanSync::hSemaphore) created AMF_SURFACE_USAGE_DECODER_DST = 0x00000080, // VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR AMF_SURFACE_USAGE_DECODER_DPB = 0x00000100, // VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR } AMF_SURFACE_USAGE_BITS; typedef amf_flags AMF_SURFACE_USAGE; //---------------------------------------------------------------------------------------------- #if defined(_WIN32) AMF_WEAK GUID AMFFormatGUID = { 0x8cd592d0, 0x8063, 0x4af8, {0xa7, 0xd0, 0x32, 0x5b, 0xc5, 0xf7, 0x48, 0xab}}; // UINT(AMF_SURFACE_FORMAT), default - AMF_SURFACE_UNKNOWN; to be set on ID3D11Texture2D objects when used natively (i.e. force UYVY on DXGI_FORMAT_YUY2 texture) #endif //---------------------------------------------------------------------------------------------- // frame type //---------------------------------------------------------------------------------------------- typedef enum AMF_FRAME_TYPE { // flags AMF_FRAME_STEREO_FLAG = 0x10000000, AMF_FRAME_LEFT_FLAG = AMF_FRAME_STEREO_FLAG | 0x20000000, AMF_FRAME_RIGHT_FLAG = AMF_FRAME_STEREO_FLAG | 0x40000000, AMF_FRAME_BOTH_FLAG = AMF_FRAME_LEFT_FLAG | AMF_FRAME_RIGHT_FLAG, AMF_FRAME_INTERLEAVED_FLAG = 0x01000000, AMF_FRAME_FIELD_FLAG = 0x02000000, AMF_FRAME_EVEN_FLAG = 0x04000000, AMF_FRAME_ODD_FLAG = 0x08000000, // values AMF_FRAME_UNKNOWN =-1, AMF_FRAME_PROGRESSIVE = 0, AMF_FRAME_INTERLEAVED_EVEN_FIRST = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_EVEN_FLAG, AMF_FRAME_INTERLEAVED_ODD_FIRST = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_ODD_FLAG, AMF_FRAME_FIELD_SINGLE_EVEN = AMF_FRAME_FIELD_FLAG | AMF_FRAME_EVEN_FLAG, AMF_FRAME_FIELD_SINGLE_ODD = AMF_FRAME_FIELD_FLAG | AMF_FRAME_ODD_FLAG, AMF_FRAME_STEREO_LEFT = AMF_FRAME_LEFT_FLAG, AMF_FRAME_STEREO_RIGHT = AMF_FRAME_RIGHT_FLAG, AMF_FRAME_STEREO_BOTH = AMF_FRAME_BOTH_FLAG, AMF_FRAME_INTERLEAVED_EVEN_FIRST_STEREO_LEFT = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_EVEN_FLAG | AMF_FRAME_LEFT_FLAG, AMF_FRAME_INTERLEAVED_EVEN_FIRST_STEREO_RIGHT = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_EVEN_FLAG | AMF_FRAME_RIGHT_FLAG, AMF_FRAME_INTERLEAVED_EVEN_FIRST_STEREO_BOTH = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_EVEN_FLAG | AMF_FRAME_BOTH_FLAG, AMF_FRAME_INTERLEAVED_ODD_FIRST_STEREO_LEFT = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_ODD_FLAG | AMF_FRAME_LEFT_FLAG, AMF_FRAME_INTERLEAVED_ODD_FIRST_STEREO_RIGHT = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_ODD_FLAG | AMF_FRAME_RIGHT_FLAG, AMF_FRAME_INTERLEAVED_ODD_FIRST_STEREO_BOTH = AMF_FRAME_INTERLEAVED_FLAG | AMF_FRAME_ODD_FLAG | AMF_FRAME_BOTH_FLAG, } AMF_FRAME_TYPE; typedef enum AMF_ROTATION_ENUM { AMF_ROTATION_NONE = 0, AMF_ROTATION_90 = 1, AMF_ROTATION_180 = 2, AMF_ROTATION_270 = 3, } AMF_ROTATION_ENUM; #define AMF_SURFACE_ROTATION L"Rotation" // amf_int64(AMF_ROTATION_ENUM); default = AMF_ROTATION_NONE, can be set on surfaces //---------------------------------------------------------------------------------------------- // AMFSurfaceObserver interface - callback; is called before internal release resources. //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMFSurface; class AMF_NO_VTABLE AMFSurfaceObserver { public: virtual void AMF_STD_CALL OnSurfaceDataRelease(AMFSurface* pSurface) = 0; }; #else // #if defined(__cplusplus) typedef struct AMFSurface AMFSurface; typedef struct AMFSurfaceObserver AMFSurfaceObserver; typedef struct AMFSurfaceObserverVtbl { void (AMF_STD_CALL *OnSurfaceDataRelease)(AMFSurfaceObserver* pThis, AMFSurface* pSurface); } AMFSurfaceObserverVtbl; struct AMFSurfaceObserver { const AMFSurfaceObserverVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFSurface interface //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFSurface : public AMFData { public: AMF_DECLARE_IID(0x3075dbe3, 0x8718, 0x4cfa, 0x86, 0xfb, 0x21, 0x14, 0xc0, 0xa5, 0xa4, 0x51) virtual AMF_SURFACE_FORMAT AMF_STD_CALL GetFormat() = 0; // do not store planes outside. should be used together with Surface virtual amf_size AMF_STD_CALL GetPlanesCount() = 0; virtual AMFPlane* AMF_STD_CALL GetPlaneAt(amf_size index) = 0; virtual AMFPlane* AMF_STD_CALL GetPlane(AMF_PLANE_TYPE type) = 0; virtual AMF_FRAME_TYPE AMF_STD_CALL GetFrameType() = 0; virtual void AMF_STD_CALL SetFrameType(AMF_FRAME_TYPE type) = 0; virtual AMF_RESULT AMF_STD_CALL SetCrop(amf_int32 x,amf_int32 y, amf_int32 width, amf_int32 height) = 0; virtual AMF_RESULT AMF_STD_CALL CopySurfaceRegion(AMFSurface* pDest, amf_int32 dstX, amf_int32 dstY, amf_int32 srcX, amf_int32 srcY, amf_int32 width, amf_int32 height) = 0; // Observer management #ifdef __clang__ #pragma clang diagnostic push #pragma clang diagnostic ignored "-Woverloaded-virtual" #endif virtual void AMF_STD_CALL AddObserver(AMFSurfaceObserver* pObserver) = 0; virtual void AMF_STD_CALL RemoveObserver(AMFSurfaceObserver* pObserver) = 0; #ifdef __clang__ #pragma clang diagnostic pop #endif }; //---------------------------------------------------------------------------------------------- // smart pointer //---------------------------------------------------------------------------------------------- typedef AMFInterfacePtr_T AMFSurfacePtr; //---------------------------------------------------------------------------------------------- #else // #if defined(__cplusplus) AMF_DECLARE_IID(AMFSurface, 0x3075dbe3, 0x8718, 0x4cfa, 0x86, 0xfb, 0x21, 0x14, 0xc0, 0xa5, 0xa4, 0x51) typedef struct AMFSurfaceVtbl { // AMFInterface interface amf_long (AMF_STD_CALL *Acquire)(AMFSurface* pThis); amf_long (AMF_STD_CALL *Release)(AMFSurface* pThis); enum AMF_RESULT (AMF_STD_CALL *QueryInterface)(AMFSurface* pThis, const struct AMFGuid *interfaceID, void** ppInterface); // AMFPropertyStorage interface AMF_RESULT (AMF_STD_CALL *SetProperty)(AMFSurface* pThis, const wchar_t* name, AMFVariantStruct value); AMF_RESULT (AMF_STD_CALL *GetProperty)(AMFSurface* pThis, const wchar_t* name, AMFVariantStruct* pValue); amf_bool (AMF_STD_CALL *HasProperty)(AMFSurface* pThis, const wchar_t* name); amf_size (AMF_STD_CALL *GetPropertyCount)(AMFSurface* pThis); AMF_RESULT (AMF_STD_CALL *GetPropertyAt)(AMFSurface* pThis, amf_size index, wchar_t* name, amf_size nameSize, AMFVariantStruct* pValue); AMF_RESULT (AMF_STD_CALL *Clear)(AMFSurface* pThis); AMF_RESULT (AMF_STD_CALL *AddTo)(AMFSurface* pThis, AMFPropertyStorage* pDest, amf_bool overwrite, amf_bool deep); AMF_RESULT (AMF_STD_CALL *CopyTo)(AMFSurface* pThis, AMFPropertyStorage* pDest, amf_bool deep); void (AMF_STD_CALL *AddObserver)(AMFSurface* pThis, AMFPropertyStorageObserver* pObserver); void (AMF_STD_CALL *RemoveObserver)(AMFSurface* pThis, AMFPropertyStorageObserver* pObserver); // AMFData interface AMF_MEMORY_TYPE (AMF_STD_CALL *GetMemoryType)(AMFSurface* pThis); AMF_RESULT (AMF_STD_CALL *Duplicate)(AMFSurface* pThis, AMF_MEMORY_TYPE type, AMFData** ppData); AMF_RESULT (AMF_STD_CALL *Convert)(AMFSurface* pThis, AMF_MEMORY_TYPE type); // optimal interop if possilble. Copy through host memory if needed AMF_RESULT (AMF_STD_CALL *Interop)(AMFSurface* pThis, AMF_MEMORY_TYPE type); // only optimal interop if possilble. No copy through host memory for GPU objects AMF_DATA_TYPE (AMF_STD_CALL *GetDataType)(AMFSurface* pThis); amf_bool (AMF_STD_CALL *IsReusable)(AMFSurface* pThis); void (AMF_STD_CALL *SetPts)(AMFSurface* pThis, amf_pts pts); amf_pts (AMF_STD_CALL *GetPts)(AMFSurface* pThis); void (AMF_STD_CALL *SetDuration)(AMFSurface* pThis, amf_pts duration); amf_pts (AMF_STD_CALL *GetDuration)(AMFSurface* pThis); // AMFSurface interface AMF_SURFACE_FORMAT (AMF_STD_CALL *GetFormat)(AMFSurface* pThis); // do not store planes outside. should be used together with Surface amf_size (AMF_STD_CALL *GetPlanesCount)(AMFSurface* pThis); AMFPlane* (AMF_STD_CALL *GetPlaneAt)(AMFSurface* pThis, amf_size index); AMFPlane* (AMF_STD_CALL *GetPlane)(AMFSurface* pThis, AMF_PLANE_TYPE type); AMF_FRAME_TYPE (AMF_STD_CALL *GetFrameType)(AMFSurface* pThis); void (AMF_STD_CALL *SetFrameType)(AMFSurface* pThis, AMF_FRAME_TYPE type); AMF_RESULT (AMF_STD_CALL *SetCrop)(AMFSurface* pThis, amf_int32 x,amf_int32 y, amf_int32 width, amf_int32 height); AMF_RESULT (AMF_STD_CALL *CopySurfaceRegion)(AMFSurface* pThis, AMFSurface* pDest, amf_int32 dstX, amf_int32 dstY, amf_int32 srcX, amf_int32 srcY, amf_int32 width, amf_int32 height); // Observer management void (AMF_STD_CALL *AddObserver_Surface)(AMFSurface* pThis, AMFSurfaceObserver* pObserver); void (AMF_STD_CALL *RemoveObserver_Surface)(AMFSurface* pThis, AMFSurfaceObserver* pObserver); } AMFSurfaceVtbl; struct AMFSurface { const AMFSurfaceVtbl *pVtbl; }; #endif // #if defined(__cplusplus) #if defined(__cplusplus) } #endif #if defined(_MSC_VER) #pragma warning( pop ) #endif #endif //#ifndef AMF_Surface_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Trace.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Trace_h #define AMF_Trace_h #pragma once #include "Platform.h" #include "Result.h" #include "Surface.h" #include "AudioBuffer.h" #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // trace levels //---------------------------------------------------------------------------------------------- #define AMF_TRACE_ERROR 0 #define AMF_TRACE_WARNING 1 #define AMF_TRACE_INFO 2 // default in sdk #define AMF_TRACE_DEBUG 3 #define AMF_TRACE_TRACE 4 #define AMF_TRACE_TEST 5 #define AMF_TRACE_NOLOG 100 //---------------------------------------------------------------------------------------------- // available trace writers //---------------------------------------------------------------------------------------------- #define AMF_TRACE_WRITER_CONSOLE L"Console" #define AMF_TRACE_WRITER_DEBUG_OUTPUT L"DebugOutput" #define AMF_TRACE_WRITER_FILE L"File" //---------------------------------------------------------------------------------------------- // AMFTraceWriter interface - callback //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFTraceWriter { public: virtual void AMF_CDECL_CALL Write(const wchar_t* scope, const wchar_t* message) = 0; virtual void AMF_CDECL_CALL Flush() = 0; }; #else // #if defined(__cplusplus) typedef struct AMFTraceWriter AMFTraceWriter; typedef struct AMFTraceWriterVtbl { // AMFTraceWriter interface void (AMF_CDECL_CALL *Write)(AMFTraceWriter* pThis, const wchar_t* scope, const wchar_t* message); void (AMF_CDECL_CALL *Flush)(AMFTraceWriter* pThis); } AMFTraceWriterVtbl; struct AMFTraceWriter { const AMFTraceWriterVtbl *pVtbl; }; #endif // #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMFTrace interface - singleton //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) class AMF_NO_VTABLE AMFTrace { public: virtual void AMF_STD_CALL TraceW(const wchar_t* src_path, amf_int32 line, amf_int32 level, const wchar_t* scope,amf_int32 countArgs, const wchar_t* format, ...) = 0; virtual void AMF_STD_CALL Trace(const wchar_t* src_path, amf_int32 line, amf_int32 level, const wchar_t* scope, const wchar_t* message, va_list* pArglist) = 0; virtual amf_int32 AMF_STD_CALL SetGlobalLevel(amf_int32 level) = 0; virtual amf_int32 AMF_STD_CALL GetGlobalLevel() = 0; virtual amf_bool AMF_STD_CALL EnableWriter(const wchar_t* writerID, bool enable) = 0; virtual amf_bool AMF_STD_CALL WriterEnabled(const wchar_t* writerID) = 0; virtual AMF_RESULT AMF_STD_CALL TraceEnableAsync(amf_bool enable) = 0; virtual AMF_RESULT AMF_STD_CALL TraceFlush() = 0; virtual AMF_RESULT AMF_STD_CALL SetPath(const wchar_t* path) = 0; virtual AMF_RESULT AMF_STD_CALL GetPath(wchar_t* path, amf_size* pSize) = 0; virtual amf_int32 AMF_STD_CALL SetWriterLevel(const wchar_t* writerID, amf_int32 level) = 0; virtual amf_int32 AMF_STD_CALL GetWriterLevel(const wchar_t* writerID) = 0; virtual amf_int32 AMF_STD_CALL SetWriterLevelForScope(const wchar_t* writerID, const wchar_t* scope, amf_int32 level) = 0; virtual amf_int32 AMF_STD_CALL GetWriterLevelForScope(const wchar_t* writerID, const wchar_t* scope) = 0; virtual amf_int32 AMF_STD_CALL GetIndentation() = 0; virtual void AMF_STD_CALL Indent(amf_int32 addIndent) = 0; virtual void AMF_STD_CALL RegisterWriter(const wchar_t* writerID, AMFTraceWriter* pWriter, amf_bool enable) = 0; virtual void AMF_STD_CALL UnregisterWriter(const wchar_t* writerID) = 0; virtual const wchar_t* AMF_STD_CALL GetResultText(AMF_RESULT res) = 0; virtual const wchar_t* AMF_STD_CALL SurfaceGetFormatName(const AMF_SURFACE_FORMAT eSurfaceFormat) = 0; virtual AMF_SURFACE_FORMAT AMF_STD_CALL SurfaceGetFormatByName(const wchar_t* name) = 0; virtual const wchar_t* AMF_STD_CALL GetMemoryTypeName(const AMF_MEMORY_TYPE memoryType) = 0; virtual AMF_MEMORY_TYPE AMF_STD_CALL GetMemoryTypeByName(const wchar_t* name) = 0; virtual const wchar_t* AMF_STD_CALL GetSampleFormatName(const AMF_AUDIO_FORMAT eFormat) = 0; virtual AMF_AUDIO_FORMAT AMF_STD_CALL GetSampleFormatByName(const wchar_t* name) = 0; }; #else // #if defined(__cplusplus) typedef struct AMFTrace AMFTrace; typedef struct AMFTraceVtbl { // AMFTrace interface void (AMF_STD_CALL *TraceW)(AMFTrace* pThis, const wchar_t* src_path, amf_int32 line, amf_int32 level, const wchar_t* scope,amf_int32 countArgs, const wchar_t* format, ...); void (AMF_STD_CALL *Trace)(AMFTrace* pThis, const wchar_t* src_path, amf_int32 line, amf_int32 level, const wchar_t* scope, const wchar_t* message, va_list* pArglist); amf_int32 (AMF_STD_CALL *SetGlobalLevel)(AMFTrace* pThis, amf_int32 level); amf_int32 (AMF_STD_CALL *GetGlobalLevel)(AMFTrace* pThis); amf_bool (AMF_STD_CALL *EnableWriter)(AMFTrace* pThis, const wchar_t* writerID, amf_bool enable); amf_bool (AMF_STD_CALL *WriterEnabled)(AMFTrace* pThis, const wchar_t* writerID); AMF_RESULT (AMF_STD_CALL *TraceEnableAsync)(AMFTrace* pThis, amf_bool enable); AMF_RESULT (AMF_STD_CALL *TraceFlush)(AMFTrace* pThis); AMF_RESULT (AMF_STD_CALL *SetPath)(AMFTrace* pThis, const wchar_t* path); AMF_RESULT (AMF_STD_CALL *GetPath)(AMFTrace* pThis, wchar_t* path, amf_size* pSize); amf_int32 (AMF_STD_CALL *SetWriterLevel)(AMFTrace* pThis, const wchar_t* writerID, amf_int32 level); amf_int32 (AMF_STD_CALL *GetWriterLevel)(AMFTrace* pThis, const wchar_t* writerID); amf_int32 (AMF_STD_CALL *SetWriterLevelForScope)(AMFTrace* pThis, const wchar_t* writerID, const wchar_t* scope, amf_int32 level); amf_int32 (AMF_STD_CALL *GetWriterLevelForScope)(AMFTrace* pThis, const wchar_t* writerID, const wchar_t* scope); amf_int32 (AMF_STD_CALL *GetIndentation)(AMFTrace* pThis); void (AMF_STD_CALL *Indent)(AMFTrace* pThis, amf_int32 addIndent); void (AMF_STD_CALL *RegisterWriter)(AMFTrace* pThis, const wchar_t* writerID, AMFTraceWriter* pWriter, amf_bool enable); void (AMF_STD_CALL *UnregisterWriter)(AMFTrace* pThis, const wchar_t* writerID); const wchar_t* (AMF_STD_CALL *GetResultText)(AMFTrace* pThis, AMF_RESULT res); const wchar_t* (AMF_STD_CALL *SurfaceGetFormatName)(AMFTrace* pThis, const AMF_SURFACE_FORMAT eSurfaceFormat); AMF_SURFACE_FORMAT (AMF_STD_CALL *SurfaceGetFormatByName)(AMFTrace* pThis, const wchar_t* name); const wchar_t* (AMF_STD_CALL *GetMemoryTypeName)(AMFTrace* pThis, const AMF_MEMORY_TYPE memoryType); AMF_MEMORY_TYPE (AMF_STD_CALL *GetMemoryTypeByName)(AMFTrace* pThis, const wchar_t* name); const wchar_t* (AMF_STD_CALL *GetSampleFormatName)(AMFTrace* pThis, const AMF_AUDIO_FORMAT eFormat); AMF_AUDIO_FORMAT (AMF_STD_CALL *GetSampleFormatByName)(AMFTrace* pThis, const wchar_t* name); } AMFTraceVtbl; struct AMFTrace { const AMFTraceVtbl *pVtbl; }; #endif #if defined(__cplusplus) } #endif #endif // AMF_Trace_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Variant.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef AMF_Variant_h #define AMF_Variant_h #pragma once #if defined(_MSC_VER) #pragma warning(disable: 4996) #endif #include "Interface.h" #include #include #include #if defined(__cplusplus) namespace amf { #endif //---------------------------------------------------------------------------------------------- // variant types //---------------------------------------------------------------------------------------------- typedef enum AMF_VARIANT_TYPE { AMF_VARIANT_EMPTY = 0, AMF_VARIANT_BOOL = 1, AMF_VARIANT_INT64 = 2, AMF_VARIANT_DOUBLE = 3, AMF_VARIANT_RECT = 4, AMF_VARIANT_SIZE = 5, AMF_VARIANT_POINT = 6, AMF_VARIANT_RATE = 7, AMF_VARIANT_RATIO = 8, AMF_VARIANT_COLOR = 9, AMF_VARIANT_STRING = 10, // value is char* AMF_VARIANT_WSTRING = 11, // value is wchar_t* AMF_VARIANT_INTERFACE = 12, // value is AMFInterface* AMF_VARIANT_FLOAT = 13, AMF_VARIANT_FLOAT_SIZE = 14, AMF_VARIANT_FLOAT_POINT2D = 15, AMF_VARIANT_FLOAT_POINT3D = 16, AMF_VARIANT_FLOAT_VECTOR4D = 17 } AMF_VARIANT_TYPE; //---------------------------------------------------------------------------------------------- // variant struct //---------------------------------------------------------------------------------------------- typedef struct AMFVariantStruct { AMF_VARIANT_TYPE type; union { amf_bool boolValue; amf_int64 int64Value; amf_double doubleValue; char* stringValue; wchar_t* wstringValue; AMFInterface* pInterface; struct AMFRect rectValue; struct AMFSize sizeValue; struct AMFPoint pointValue; struct AMFRate rateValue; struct AMFRatio ratioValue; struct AMFColor colorValue; amf_float floatValue; struct AMFFloatSize floatSizeValue; struct AMFFloatPoint2D floatPoint2DValue; struct AMFFloatPoint3D floatPoint3DValue; struct AMFFloatVector4D floatVector4DValue; }; } AMFVariantStruct; //---------------------------------------------------------------------------------------------- // variant accessors //---------------------------------------------------------------------------------------------- static AMF_INLINE AMF_VARIANT_TYPE AMF_STD_CALL AMFVariantGetType(const AMFVariantStruct* _variant) { return (_variant)->type; } #if defined(__cplusplus) static AMF_INLINE AMF_VARIANT_TYPE& AMF_STD_CALL AMFVariantGetType(AMFVariantStruct* _variant) { return (_variant)->type; } #endif static AMF_INLINE amf_bool AMF_STD_CALL AMFVariantGetBool(const AMFVariantStruct* _variant) { return (_variant)->boolValue; } static AMF_INLINE amf_int64 AMF_STD_CALL AMFVariantGetInt64(const AMFVariantStruct* _variant) { return (_variant)->int64Value; } static AMF_INLINE amf_double AMF_STD_CALL AMFVariantGetDouble(const AMFVariantStruct* _variant) { return (_variant)->doubleValue; } static AMF_INLINE amf_float AMF_STD_CALL AMFVariantGetFloat(const AMFVariantStruct* _variant) { return (_variant)->floatValue; } static AMF_INLINE const char* AMF_STD_CALL AMFVariantGetString(const AMFVariantStruct* _variant) { return (_variant)->stringValue; } static AMF_INLINE const wchar_t* AMF_STD_CALL AMFVariantGetWString(const AMFVariantStruct* _variant) { return (_variant)->wstringValue; } #if defined(__cplusplus) static AMF_INLINE const AMFInterface* AMF_STD_CALL AMFVariantGetInterface(const AMFVariantStruct* _variant) { return (_variant)->pInterface; } #endif static AMF_INLINE AMFInterface* AMF_STD_CALL AMFVariantGetInterface(AMFVariantStruct* _variant) { return (_variant)->pInterface; } #if defined(__cplusplus) static AMF_INLINE const AMFRect & AMF_STD_CALL AMFVariantGetRect (const AMFVariantStruct* _variant) { return (_variant)->rectValue; } static AMF_INLINE const AMFSize & AMF_STD_CALL AMFVariantGetSize (const AMFVariantStruct* _variant) { return (_variant)->sizeValue; } static AMF_INLINE const AMFPoint& AMF_STD_CALL AMFVariantGetPoint(const AMFVariantStruct* _variant) { return (_variant)->pointValue; } static AMF_INLINE const AMFFloatSize& AMF_STD_CALL AMFVariantGetFloatSize(const AMFVariantStruct* _variant) { return (_variant)->floatSizeValue; } static AMF_INLINE const AMFFloatPoint2D& AMF_STD_CALL AMFVariantGetFloatPoint2D(const AMFVariantStruct* _variant) { return (_variant)->floatPoint2DValue; } static AMF_INLINE const AMFFloatPoint3D& AMF_STD_CALL AMFVariantGetFloatPoint3D(const AMFVariantStruct* _variant) { return (_variant)->floatPoint3DValue; } static AMF_INLINE const AMFFloatVector4D& AMF_STD_CALL AMFVariantGetFloatVector4D(const AMFVariantStruct* _variant) { return (_variant)->floatVector4DValue; } static AMF_INLINE const AMFRate & AMF_STD_CALL AMFVariantGetRate (const AMFVariantStruct* _variant) { return (_variant)->rateValue; } static AMF_INLINE const AMFRatio& AMF_STD_CALL AMFVariantGetRatio(const AMFVariantStruct* _variant) { return (_variant)->ratioValue; } static AMF_INLINE const AMFColor& AMF_STD_CALL AMFVariantGetColor(const AMFVariantStruct* _variant) { return (_variant)->colorValue; } #else // #if defined(__cplusplus) static AMF_INLINE const AMFRect AMF_STD_CALL AMFVariantGetRect (const AMFVariantStruct* _variant) { return (_variant)->rectValue; } static AMF_INLINE const AMFSize AMF_STD_CALL AMFVariantGetSize (const AMFVariantStruct* _variant) { return (_variant)->sizeValue; } static AMF_INLINE const AMFPoint AMF_STD_CALL AMFVariantGetPoint(const AMFVariantStruct* _variant) { return (_variant)->pointValue; } static AMF_INLINE const AMFFloatSize AMF_STD_CALL AMFVariantGetFloatSize(const AMFVariantStruct* _variant) { return (_variant)->floatSizeValue; } static AMF_INLINE const AMFFloatPoint2D AMF_STD_CALL AMFVariantGetFloatPoint2D(const AMFVariantStruct* _variant) { return (_variant)->floatPoint2DValue; } static AMF_INLINE const AMFFloatPoint3D AMF_STD_CALL AMFVariantGetFloatPoint3D(const AMFVariantStruct* _variant) { return (_variant)->floatPoint3DValue; } static AMF_INLINE const AMFFloatVector4D AMF_STD_CALL AMFVariantGetFloatVector4D(const AMFVariantStruct* _variant) { return (_variant)->floatVector4DValue; } static AMF_INLINE const AMFRate AMF_STD_CALL AMFVariantGetRate (const AMFVariantStruct* _variant) { return (_variant)->rateValue; } static AMF_INLINE const AMFRatio AMF_STD_CALL AMFVariantGetRatio(const AMFVariantStruct* _variant) { return (_variant)->ratioValue; } static AMF_INLINE const AMFColor AMF_STD_CALL AMFVariantGetColor(const AMFVariantStruct* _variant) { return (_variant)->colorValue; } #endif // #if defined(__cplusplus) #define AMFVariantEmpty(_variant) 0 #define AMFVariantBool(_variant) (_variant)->boolValue #define AMFVariantInt64(_variant) (_variant)->int64Value #define AMFVariantDouble(_variant) (_variant)->doubleValue #define AMFVariantFloat(_variant) (_variant)->floatValue #define AMFVariantRect(_variant) (_variant)->rectValue #define AMFVariantSize(_variant) (_variant)->sizeValue #define AMFVariantPoint(_variant) (_variant)->pointValue #define AMFVariantFloatSize(_variant) (_variant)->floatSizeValue #define AMFVariantFloatPoint2D(_variant) (_variant)->floatPoint2DValue #define AMFVariantFloatPoint3D(_variant) (_variant)->floatPoint3DValue #define AMFVariantFloatVector4D(_variant) (_variant)->floatVector4DValue #define AMFVariantRate(_variant) (_variant)->rateValue #define AMFVariantRatio(_variant) (_variant)->ratioValue #define AMFVariantColor(_variant) (_variant)->colorValue #define AMFVariantString(_variant) (_variant)->stringValue #define AMFVariantWString(_variant) (_variant)->wstringValue #define AMFVariantInterface(_variant) (_variant)->pInterface //---------------------------------------------------------------------------------------------- // variant hleper functions //---------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantInit(AMFVariantStruct* pVariant); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantClear(AMFVariantStruct* pVariant); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantCompare(const AMFVariantStruct* pFirst, const AMFVariantStruct* pSecond, amf_bool* pEqual); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantCopy(AMFVariantStruct* pDest, const AMFVariantStruct* pSrc); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignBool(AMFVariantStruct* pDest, amf_bool value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignInt64(AMFVariantStruct* pDest, amf_int64 value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignDouble(AMFVariantStruct* pDest, amf_double value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloat(AMFVariantStruct* pDest, amf_float value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignString(AMFVariantStruct* pDest, const char* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignWString(AMFVariantStruct* pDest, const wchar_t* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignInterface(AMFVariantStruct* pDest, AMFInterface* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRect(AMFVariantStruct* pDest, const AMFRect* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignSize(AMFVariantStruct* pDest, const AMFSize* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignPoint(AMFVariantStruct* pDest, const AMFPoint* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatSize(AMFVariantStruct* pDest, const AMFFloatSize* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint2D(AMFVariantStruct* pDest, const AMFFloatPoint2D* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint3D(AMFVariantStruct* pDest, const AMFFloatPoint3D* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatVector4D(AMFVariantStruct* pDest, const AMFFloatVector4D* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRate(AMFVariantStruct* pDest, const AMFRate* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRatio(AMFVariantStruct* pDest, const AMFRatio* pValue); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignColor(AMFVariantStruct* pDest, const AMFColor* pValue); #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRect(AMFVariantStruct* pDest, const AMFRect& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignSize(AMFVariantStruct* pDest, const AMFSize& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignPoint(AMFVariantStruct* pDest, const AMFPoint& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatSize(AMFVariantStruct* pDest, const AMFFloatSize& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint2D(AMFVariantStruct* pDest, const AMFFloatPoint2D& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint3D(AMFVariantStruct* pDest, const AMFFloatPoint3D& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatVector4D(AMFVariantStruct* pDest, const AMFFloatVector4D& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRate(AMFVariantStruct* pDest, const AMFRate& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRatio(AMFVariantStruct* pDest, const AMFRatio& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignColor(AMFVariantStruct* pDest, const AMFColor& value); static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantChangeType(AMFVariantStruct* pDest, const AMFVariantStruct* pSrc, AMF_VARIANT_TYPE newType); #endif static AMF_INLINE char* AMF_CDECL_CALL AMFVariantDuplicateString(const char* pFrom); static AMF_INLINE void AMF_CDECL_CALL AMFVariantFreeString(char* pFrom); static AMF_INLINE wchar_t* AMF_CDECL_CALL AMFVariantDuplicateWString(const wchar_t* pFrom); static AMF_INLINE void AMF_CDECL_CALL AMFVariantFreeWString(wchar_t* pFrom); #if defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMF_INLINE Variant helper class //---------------------------------------------------------------------------------------------- class AMFVariant : public AMFVariantStruct { public: class String; class WString; public: AMFVariant() { AMFVariantInit(this); } explicit AMFVariant(const AMFVariantStruct& other) { AMFVariantInit(this); AMFVariantCopy(this, const_cast(&other)); } explicit AMFVariant(const AMFVariantStruct* pOther); template explicit AMFVariant(const AMFInterfacePtr_T& pValue); AMFVariant(const AMFVariant& other) { AMFVariantInit(this); AMFVariantCopy(this, const_cast(static_cast(&other))); } explicit AMF_INLINE AMFVariant(amf_bool value) { AMFVariantInit(this); AMFVariantAssignBool(this, value); } explicit AMF_INLINE AMFVariant(amf_int64 value) { AMFVariantInit(this); AMFVariantAssignInt64(this, value); } explicit AMF_INLINE AMFVariant(amf_uint64 value) { AMFVariantInit(this); AMFVariantAssignInt64(this, (amf_int64)value); } explicit AMF_INLINE AMFVariant(amf_int32 value) { AMFVariantInit(this); AMFVariantAssignInt64(this, value); } explicit AMF_INLINE AMFVariant(amf_uint32 value) { AMFVariantInit(this); AMFVariantAssignInt64(this, value); } explicit AMF_INLINE AMFVariant(amf_double value) { AMFVariantInit(this); AMFVariantAssignDouble(this, value); } explicit AMF_INLINE AMFVariant(amf_float value) { AMFVariantInit(this); AMFVariantAssignFloat(this, value); } explicit AMF_INLINE AMFVariant(const AMFRect & value) { AMFVariantInit(this); AMFVariantAssignRect(this, &value); } explicit AMF_INLINE AMFVariant(const AMFSize & value) { AMFVariantInit(this); AMFVariantAssignSize(this, &value); } explicit AMF_INLINE AMFVariant(const AMFPoint& value) { AMFVariantInit(this); AMFVariantAssignPoint(this, &value); } explicit AMF_INLINE AMFVariant(const AMFFloatSize& value) { AMFVariantInit(this); AMFVariantAssignFloatSize(this, &value); } explicit AMF_INLINE AMFVariant(const AMFFloatPoint2D& value) { AMFVariantInit(this); AMFVariantAssignFloatPoint2D(this, &value); } explicit AMF_INLINE AMFVariant(const AMFFloatPoint3D& value) { AMFVariantInit(this); AMFVariantAssignFloatPoint3D(this, &value); } explicit AMF_INLINE AMFVariant(const AMFFloatVector4D& value) { AMFVariantInit(this); AMFVariantAssignFloatVector4D(this, &value); } explicit AMF_INLINE AMFVariant(const AMFRate & value) { AMFVariantInit(this); AMFVariantAssignRate(this, &value); } explicit AMF_INLINE AMFVariant(const AMFRatio& value) { AMFVariantInit(this); AMFVariantAssignRatio(this, &value); } explicit AMF_INLINE AMFVariant(const AMFColor& value) { AMFVariantInit(this); AMFVariantAssignColor(this, &value); } explicit AMF_INLINE AMFVariant(const char* pValue) { AMFVariantInit(this); AMFVariantAssignString(this, pValue); } explicit AMF_INLINE AMFVariant(const wchar_t* pValue) { AMFVariantInit(this); AMFVariantAssignWString(this, pValue); } explicit AMF_INLINE AMFVariant(AMFInterface* pValue) { AMFVariantInit(this); AMFVariantAssignInterface(this, pValue); } ~AMFVariant() { AMFVariantClear(this); } AMFVariant& operator=(const AMFVariantStruct& other); AMFVariant& operator=(const AMFVariantStruct* pOther); AMFVariant& operator=(const AMFVariant& other); AMFVariant& operator=(amf_bool value) { AMFVariantAssignBool(this, value); return *this;} AMFVariant& operator=(amf_int64 value) { AMFVariantAssignInt64(this, value); return *this;} AMFVariant& operator=(amf_uint64 value) { AMFVariantAssignInt64(this, (amf_int64)value); return *this;} AMFVariant& operator=(amf_int32 value) { AMFVariantAssignInt64(this, value); return *this;} AMFVariant& operator=(amf_uint32 value) { AMFVariantAssignInt64(this, value); return *this;} AMFVariant& operator=(amf_double value) { AMFVariantAssignDouble(this, value); return *this;} AMFVariant& operator=(amf_float value) { AMFVariantAssignFloat(this, value); return *this; } AMFVariant& operator=(const AMFRect & value) { AMFVariantAssignRect(this, &value); return *this;} AMFVariant& operator=(const AMFSize & value) { AMFVariantAssignSize(this, &value); return *this;} AMFVariant& operator=(const AMFPoint& value) { AMFVariantAssignPoint(this, &value); return *this;} AMFVariant& operator=(const AMFFloatSize& value) { AMFVariantAssignFloatSize(this, &value); return *this; } AMFVariant& operator=(const AMFFloatPoint2D& value) { AMFVariantAssignFloatPoint2D(this, &value); return *this; } AMFVariant& operator=(const AMFFloatPoint3D& value) { AMFVariantAssignFloatPoint3D(this, &value); return *this; } AMFVariant& operator=(const AMFFloatVector4D& value) { AMFVariantAssignFloatVector4D(this, &value); return *this; } AMFVariant& operator=(const AMFRate & value) { AMFVariantAssignRate(this, &value); return *this;} AMFVariant& operator=(const AMFRatio& value) { AMFVariantAssignRatio(this, &value); return *this;} AMFVariant& operator=(const AMFColor& value) { AMFVariantAssignColor(this, &value); return *this;} AMFVariant& operator=(const char* pValue) { AMFVariantAssignString(this, pValue); return *this;} AMFVariant& operator=(const wchar_t* pValue) { AMFVariantAssignWString(this, pValue); return *this;} AMFVariant& operator=(AMFInterface* pValue) { AMFVariantAssignInterface(this, pValue); return *this;} template AMFVariant& operator=(const AMFInterfacePtr_T& value); operator amf_bool() const { return ToBool(); } operator amf_int64() const { return ToInt64(); } operator amf_uint64() const { return ToUInt64(); } operator amf_int32() const { return ToInt32(); } operator amf_uint32() const { return ToUInt32(); } operator amf_double() const { return ToDouble(); } operator amf_float() const { return ToFloat(); } operator AMFRect () const { return ToRect (); } operator AMFSize () const { return ToSize (); } operator AMFPoint() const { return ToPoint(); } operator AMFFloatSize() const { return ToFloatSize(); } operator AMFFloatPoint2D() const { return ToFloatPoint2D(); } operator AMFFloatPoint3D() const { return ToFloatPoint3D(); } operator AMFFloatVector4D() const { return ToFloatVector4D(); } operator AMFRate () const { return ToRate (); } operator AMFRatio() const { return ToRatio(); } operator AMFColor() const { return ToColor(); } operator AMFInterface*() const { return ToInterface(); } AMF_INLINE amf_bool ToBool() const { return Empty() ? false : GetValue(AMFVariantGetBool); } AMF_INLINE amf_int64 ToInt64() const { return Empty() ? 0 : GetValue(AMFVariantGetInt64); } AMF_INLINE amf_uint64 ToUInt64() const { return Empty() ? 0 : GetValue(AMFVariantGetInt64); } AMF_INLINE amf_int32 ToInt32() const { return Empty() ? 0 : GetValue(AMFVariantGetInt64); } AMF_INLINE amf_uint32 ToUInt32() const { return Empty() ? 0 : GetValue(AMFVariantGetInt64); } AMF_INLINE amf_double ToDouble() const { return Empty() ? 0 : GetValue(AMFVariantGetDouble); } AMF_INLINE amf_float ToFloat() const { return Empty() ? 0 : GetValue(AMFVariantGetFloat); } AMF_INLINE AMFRect ToRect () const { return Empty() ? AMFRect() : GetValue(AMFVariantGetRect); } AMF_INLINE AMFSize ToSize () const { return Empty() ? AMFSize() : GetValue(AMFVariantGetSize); } AMF_INLINE AMFPoint ToPoint() const { return Empty() ? AMFPoint() : GetValue(AMFVariantGetPoint); } AMF_INLINE AMFFloatSize ToFloatSize() const { return Empty() ? AMFFloatSize() : GetValue(AMFVariantGetFloatSize); } AMF_INLINE AMFFloatPoint2D ToFloatPoint2D() const { return Empty() ? AMFFloatPoint2D() : GetValue(AMFVariantGetFloatPoint2D); } AMF_INLINE AMFFloatPoint3D ToFloatPoint3D() const { return Empty() ? AMFFloatPoint3D() : GetValue(AMFVariantGetFloatPoint3D); } AMF_INLINE AMFFloatVector4D ToFloatVector4D() const { return Empty() ? AMFFloatVector4D() : GetValue(AMFVariantGetFloatVector4D); } AMF_INLINE AMFRate ToRate () const { return Empty() ? AMFRate() : GetValue(AMFVariantGetRate); } AMF_INLINE AMFRatio ToRatio() const { return Empty() ? AMFRatio() : GetValue(AMFVariantGetRatio); } AMF_INLINE AMFColor ToColor() const { return Empty() ? AMFColor() : GetValue(AMFVariantGetColor); } AMF_INLINE AMFInterface* ToInterface() const { return AMFVariantGetType(this) == AMF_VARIANT_INTERFACE ? this->pInterface : nullptr; } AMF_INLINE String ToString() const; AMF_INLINE WString ToWString() const; bool operator==(const AMFVariantStruct& other) const; bool operator==(const AMFVariantStruct* pOther) const; bool operator!=(const AMFVariantStruct& other) const; bool operator!=(const AMFVariantStruct* pOther) const; void Clear() { AMFVariantClear(this); } void Attach(AMFVariantStruct& variant); AMFVariantStruct Detach(); AMFVariantStruct& GetVariant(); void ChangeType(AMF_VARIANT_TYPE type, const AMFVariant* pSrc = nullptr); bool Empty() const; private: template ReturnType GetValue(Getter getter) const; }; //---------------------------------------------------------------------------------------------- // helper String class //---------------------------------------------------------------------------------------------- class AMFVariant::String { friend class AMFVariant; private: void Free() { if (m_Str != nullptr) { AMFVariantFreeString(m_Str); m_Str = nullptr; } } public: String() :m_Str(nullptr){} String(const char* str) : m_Str(nullptr) { m_Str = AMFVariantDuplicateString(str); } String(const String& p_other) : m_Str(nullptr) { operator=(p_other); } #if (__cplusplus == 201103L) || defined(__GXX_EXPERIMENTAL_CXX0X) || (_MSC_VER >= 1600) #pragma warning (push) #pragma warning (disable : 26439) //This kind of function may not throw. Declare it 'noexcept'. String(String&& p_other) : m_Str(nullptr) { operator=(p_other); } #endif ~String() { Free(); } char& operator[](size_t index) { if (index >= size()) { resize(index); } return m_Str[index]; } String& operator=(const String& p_other) { Free(); m_Str = AMFVariantDuplicateString(p_other.m_Str); return *this; } #if (__cplusplus == 201103L) || defined(__GXX_EXPERIMENTAL_CXX0X) || (_MSC_VER >= 1600) String& operator=(String&& p_other) { Free(); m_Str = p_other.m_Str; p_other.m_Str = nullptr; // Transfer the ownership return *this; } #endif bool operator==(const String& p_other) const { if (m_Str == nullptr && p_other.m_Str == nullptr) { return true; } else if ((m_Str == nullptr && p_other.m_Str != nullptr) || (m_Str != nullptr && p_other.m_Str == nullptr)) { return false; } return strcmp(c_str(), p_other.c_str()) == 0; } const char* c_str() const { return m_Str; } size_t size() const { if (m_Str == nullptr) { return 0; } return (size_t)strlen(m_Str); } AMF_INLINE size_t length() const { return size(); } void resize(size_t sizeAlloc) { if (sizeAlloc == 0) { Free(); return; } char* str = (char*)amf_variant_alloc(sizeof(char)*(sizeAlloc + 1)); if (m_Str != nullptr) { size_t copySize = sizeAlloc; if (copySize > size()) { copySize = size(); } memcpy(str, m_Str, copySize * sizeof(char)); Free(); str[sizeAlloc] = 0; } m_Str = str; } private: char* m_Str; }; //---------------------------------------------------------------------------------------------- // helper WString class //---------------------------------------------------------------------------------------------- class AMFVariant::WString { friend class AMFVariant; private: void Free() { if (m_Str != nullptr) { AMFVariantFreeWString(m_Str); m_Str = nullptr; } } public: WString() :m_Str(nullptr){} WString(const wchar_t* str) : m_Str(nullptr) { m_Str = AMFVariantDuplicateWString(str); } WString(const WString& p_other) : m_Str(nullptr) { operator=(p_other); } #if (__cplusplus == 201103L) || defined(__GXX_EXPERIMENTAL_CXX0X) || (_MSC_VER >= 1600) WString(WString&& p_other) : m_Str(nullptr) { operator=(p_other); } #endif ~WString() { Free(); } WString& operator=(const WString& p_other) { Free(); m_Str = AMFVariantDuplicateWString(p_other.m_Str); return *this; } #if (__cplusplus == 201103L) || defined(__GXX_EXPERIMENTAL_CXX0X) || (_MSC_VER >= 1600) WString& operator=(WString&& p_other) { Free(); m_Str = p_other.m_Str; p_other.m_Str = nullptr; // Transfer the ownership return *this; } #pragma warning (pop) #endif wchar_t& operator[](size_t index) { if (index >= size()) { resize(index); } return m_Str[index]; } bool operator==(const WString& p_other) const { if (m_Str == nullptr && p_other.m_Str == nullptr) { return true; } else if ((m_Str == nullptr && p_other.m_Str != nullptr) || (m_Str != nullptr && p_other.m_Str == nullptr)) { return false; } return wcscmp(c_str(), p_other.c_str()) == 0; } const wchar_t* c_str() const { return m_Str; } size_t size() const { if (m_Str == nullptr) { return 0; } return (size_t)wcslen(m_Str); } AMF_INLINE size_t length() const { return size(); } void resize(size_t sizeAlloc) { if (sizeAlloc == 0) { Free(); return; } wchar_t* str = (wchar_t*)amf_variant_alloc(sizeof(wchar_t)*(sizeAlloc + 1)); if (m_Str != nullptr) { size_t copySize = sizeAlloc; if (copySize > size()) { copySize = size(); } memcpy(str, m_Str, copySize * sizeof(wchar_t)); Free(); str[sizeAlloc] = 0; } m_Str = str; } private: wchar_t* m_Str; }; //------------------------------------------------------------------------------------------------- AMFVariant::String AMFVariant::ToString() const { String temp = GetValue(AMFVariantGetString); return String(temp.c_str()); } //------------------------------------------------------------------------------------------------- AMFVariant::WString AMFVariant::ToWString() const { WString temp = GetValue(AMFVariantGetWString); return WString(temp.c_str()); } #endif // defined(__cplusplus) //---------------------------------------------------------------------------------------------- // AMF_INLINE implementation of helper functions //---------------------------------------------------------------------------------------------- #define AMF_VARIANT_RETURN_IF_INVALID_POINTER(p) \ { \ if (p == NULL) \ { \ return AMF_INVALID_POINTER; \ } \ } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantInit(AMFVariantStruct* pVariant) { AMF_VARIANT_RETURN_IF_INVALID_POINTER(pVariant); pVariant->type = AMF_VARIANT_EMPTY; return AMF_OK; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantClear(AMFVariantStruct* pVariant) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pVariant); switch (AMFVariantGetType(pVariant)) { case AMF_VARIANT_STRING: amf_variant_free(AMFVariantString(pVariant)); pVariant->type = AMF_VARIANT_EMPTY; break; case AMF_VARIANT_WSTRING: amf_variant_free(AMFVariantWString(pVariant)); pVariant->type = AMF_VARIANT_EMPTY; break; case AMF_VARIANT_INTERFACE: if (AMFVariantInterface(pVariant) != NULL) { #if defined(__cplusplus) AMFVariantInterface(pVariant)->Release(); #else AMFVariantInterface(pVariant)->pVtbl->Release(AMFVariantInterface(pVariant)); #endif AMFVariantInterface(pVariant) = NULL; } pVariant->type = AMF_VARIANT_EMPTY; break; default: pVariant->type = AMF_VARIANT_EMPTY; break; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantCompare(const AMFVariantStruct* pFirst, const AMFVariantStruct* pSecond, amf_bool* pEqual) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pFirst); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pSecond); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pEqual); if (pFirst == pSecond) { *pEqual = true; } else if (AMFVariantGetType(pFirst) != AMFVariantGetType(pSecond)) { *pEqual = false; } else { switch (AMFVariantGetType(pFirst)) { case AMF_VARIANT_EMPTY: *pEqual = true; break; case AMF_VARIANT_BOOL: *pEqual = AMFVariantGetBool(pFirst) == AMFVariantBool(pSecond); break; case AMF_VARIANT_INT64: *pEqual = AMFVariantGetInt64(pFirst) == AMFVariantInt64(pSecond); break; case AMF_VARIANT_DOUBLE: *pEqual = AMFVariantGetDouble(pFirst) == AMFVariantDouble(pSecond); break; case AMF_VARIANT_FLOAT: *pEqual = AMFVariantGetFloat(pFirst) == AMFVariantFloat(pSecond); break; case AMF_VARIANT_RECT: #if defined(__cplusplus) *pEqual = AMFVariantGetRect(pFirst) == AMFVariantGetRect(pSecond); #else *pEqual = memcmp(&pFirst->rectValue, &pSecond->rectValue, sizeof(AMFRect)) == 0; #endif break; case AMF_VARIANT_SIZE: #if defined(__cplusplus) *pEqual = AMFVariantGetSize(pFirst) == AMFVariantGetSize(pSecond); #else *pEqual = memcmp(&pFirst->sizeValue, &pSecond->sizeValue, sizeof(AMFSize)) == 0; #endif break; case AMF_VARIANT_POINT: #if defined(__cplusplus) *pEqual = AMFVariantGetPoint(pFirst) == AMFVariantGetPoint(pSecond); #else *pEqual = memcmp(&pFirst->pointValue, &pSecond->pointValue, sizeof(AMFPoint)) == 0; #endif break; case AMF_VARIANT_FLOAT_SIZE: #if defined(__cplusplus) *pEqual = AMFVariantGetFloatSize(pFirst) == AMFVariantGetFloatSize(pSecond); #else *pEqual = memcmp(&pFirst->floatSizeValue, &pSecond->floatSizeValue, sizeof(AMFFloatPoint2D)) == 0; #endif break; case AMF_VARIANT_FLOAT_POINT2D: #if defined(__cplusplus) *pEqual = AMFVariantGetFloatPoint2D(pFirst) == AMFVariantGetFloatPoint2D(pSecond); #else *pEqual = memcmp(&pFirst->floatPoint2DValue, &pSecond->floatPoint2DValue, sizeof(AMFFloatPoint2D)) == 0; #endif break; case AMF_VARIANT_FLOAT_POINT3D: #if defined(__cplusplus) *pEqual = AMFVariantGetFloatPoint3D(pFirst) == AMFVariantGetFloatPoint3D(pSecond); #else *pEqual = memcmp(&pFirst->floatPoint3DValue, &pSecond->floatPoint3DValue, sizeof(AMFFloatPoint3D)) == 0; #endif break; case AMF_VARIANT_FLOAT_VECTOR4D: #if defined(__cplusplus) *pEqual = AMFVariantGetFloatVector4D(pFirst) == AMFVariantGetFloatVector4D(pSecond); #else *pEqual = memcmp(&pFirst->floatVector4DValue, &pSecond->floatVector4DValue, sizeof(AMFFloatPoint3D)) == 0; #endif break; case AMF_VARIANT_RATE: #if defined(__cplusplus) *pEqual = AMFVariantGetRate(pFirst) == AMFVariantGetRate(pSecond); #else *pEqual = memcmp(&pFirst->rateValue, &pSecond->rateValue, sizeof(AMFRate)) == 0; #endif break; case AMF_VARIANT_RATIO: #if defined(__cplusplus) *pEqual = AMFVariantGetRatio(pFirst) == AMFVariantGetRatio(pSecond); #else *pEqual = memcmp(&pFirst->ratioValue, &pSecond->ratioValue, sizeof(AMFRatio)) == 0; #endif break; case AMF_VARIANT_COLOR: #if defined(__cplusplus) *pEqual = AMFVariantGetColor(pFirst) == AMFVariantGetColor(pSecond); #else *pEqual = memcmp(&pFirst->colorValue, &pSecond->colorValue, sizeof(AMFColor)) == 0; #endif break; case AMF_VARIANT_STRING: *pEqual = strcmp(AMFVariantString(pFirst), AMFVariantString(pSecond)) == 0; break; case AMF_VARIANT_WSTRING: *pEqual = wcscmp(AMFVariantWString(pFirst), AMFVariantWString(pSecond)) == 0; break; case AMF_VARIANT_INTERFACE: *pEqual = AMFVariantInterface(pFirst) == AMFVariantInterface(pSecond); break; default: errRet = AMF_INVALID_ARG; break; } } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantCopy(AMFVariantStruct* pDest, const AMFVariantStruct* pSrc) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pSrc); if (pDest != pSrc) { switch (AMFVariantGetType(pSrc)) { case AMF_VARIANT_EMPTY: errRet = AMFVariantInit(pDest); break; case AMF_VARIANT_BOOL: errRet = AMFVariantAssignBool(pDest, AMFVariantBool(pSrc)); break; case AMF_VARIANT_INT64: errRet = AMFVariantAssignInt64(pDest, AMFVariantInt64(pSrc)); break; case AMF_VARIANT_DOUBLE: errRet = AMFVariantAssignDouble(pDest, AMFVariantDouble(pSrc)); break; case AMF_VARIANT_FLOAT: errRet = AMFVariantAssignFloat(pDest, AMFVariantFloat(pSrc)); break; case AMF_VARIANT_RECT: errRet = AMFVariantAssignRect(pDest, &pSrc->rectValue); break; case AMF_VARIANT_SIZE: errRet = AMFVariantAssignSize(pDest, &pSrc->sizeValue); break; case AMF_VARIANT_POINT: errRet = AMFVariantAssignPoint(pDest, &pSrc->pointValue); break; case AMF_VARIANT_FLOAT_SIZE: errRet = AMFVariantAssignFloatSize(pDest, &pSrc->floatSizeValue); break; case AMF_VARIANT_FLOAT_POINT2D: errRet = AMFVariantAssignFloatPoint2D(pDest, &pSrc->floatPoint2DValue); break; case AMF_VARIANT_FLOAT_POINT3D: errRet = AMFVariantAssignFloatPoint3D(pDest, &pSrc->floatPoint3DValue); break; case AMF_VARIANT_FLOAT_VECTOR4D: errRet = AMFVariantAssignFloatVector4D(pDest, &pSrc->floatVector4DValue); break; case AMF_VARIANT_RATE: errRet = AMFVariantAssignRate(pDest, &pSrc->rateValue); break; case AMF_VARIANT_RATIO: errRet = AMFVariantAssignRatio(pDest, &pSrc->ratioValue); break; case AMF_VARIANT_COLOR: errRet = AMFVariantAssignColor(pDest, &pSrc->colorValue); break; case AMF_VARIANT_STRING: errRet = AMFVariantAssignString(pDest, AMFVariantString(pSrc)); break; case AMF_VARIANT_WSTRING: errRet = AMFVariantAssignWString(pDest, AMFVariantWString(pSrc)); break; case AMF_VARIANT_INTERFACE: errRet = AMFVariantAssignInterface(pDest, AMFVariantInterface(pSrc)); break; default: errRet = AMF_INVALID_ARG; break; } } return errRet; } #define AMFVariantTypeEmpty AMF_VARIANT_EMPTY #define AMFVariantTypeBool AMF_VARIANT_BOOL #define AMFVariantTypeInt64 AMF_VARIANT_INT64 #define AMFVariantTypeDouble AMF_VARIANT_DOUBLE #define AMFVariantTypeFloat AMF_VARIANT_FLOAT #define AMFVariantTypeRect AMF_VARIANT_RECT #define AMFVariantTypeSize AMF_VARIANT_SIZE #define AMFVariantTypePoint AMF_VARIANT_POINT #define AMFVariantTypeFloatPoint2D AMF_VARIANT_FLOAT_POINT2D #define AMFVariantTypeFloatPoint3D AMF_VARIANT_FLOAT_POINT3D #define AMFVariantTypeFloatVector4D AMF_VARIANT_FLOAT_VECTOR4D #define AMFVariantTypeRate AMF_VARIANT_RATE #define AMFVariantTypeRatio AMF_VARIANT_RATIO #define AMFVariantTypeColor AMF_VARIANT_COLOR #define AMFVariantTypeString AMF_VARIANT_STRING #define AMFVariantTypeWString AMF_VARIANT_WSTRING #define AMFVariantTypeInterface AMF_VARIANT_INTERFACE #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignString(AMFVariantStruct* pDest, const AMFVariant::String& value) { return AMFVariantAssignString(pDest, value.c_str()); } static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignWString(AMFVariantStruct* pDest, const AMFVariant::WString& value) { return AMFVariantAssignWString(pDest, value.c_str()); } static AMF_INLINE amf_bool AMFConvertEmptyToBool(void*, AMF_RESULT& res) { res = AMF_OK; return false; } static AMF_INLINE amf_int64 AMFConvertEmptyToInt64(void*, AMF_RESULT& res) {res = AMF_OK; return 0; } static AMF_INLINE amf_double AMFConvertEmptyToDouble(void*, AMF_RESULT& res) {res = AMF_OK; return 0; } static AMF_INLINE amf_float AMFConvertEmptyToFloat(void*, AMF_RESULT& res) { res = AMF_OK; return 0; } static AMF_INLINE AMFVariant::String AMFConvertEmptyToString(void*, AMF_RESULT& res) {res = AMF_OK; return ""; } static AMF_INLINE AMFVariant::WString AMFConvertEmptyToWString(void*, AMF_RESULT& res) {res = AMF_OK; return L""; } static AMF_INLINE amf_int64 AMFConvertBoolToInt64(bool value, AMF_RESULT& res){res = AMF_OK; return value ? 1 : 0;} static AMF_INLINE amf_double AMFConvertBoolToDouble(bool value, AMF_RESULT& res){res = AMF_OK; return value ? 1.0 : 0.0;} static AMF_INLINE amf_float AMFConvertBoolToFloat(bool value, AMF_RESULT& res) { res = AMF_OK; return value ? 1.0f : 0.0f; } static AMF_INLINE AMFVariant::String AMFConvertBoolToString(bool value, AMF_RESULT& res){res = AMF_OK; return value ? "true" : "false";} static AMF_INLINE AMFVariant::WString AMFConvertBoolToWString(bool value, AMF_RESULT& res){res = AMF_OK; return value ? L"true" : L"false";} static AMF_INLINE bool AMFConvertInt64ToBool(amf_int64 value, AMF_RESULT& res){res = AMF_OK;return value != 0;} static AMF_INLINE amf_double AMFConvertInt64ToDouble(amf_int64 value, AMF_RESULT& res){res = AMF_OK;return (amf_double)value;} static AMF_INLINE amf_float AMFConvertInt64ToFloat(amf_int64 value, AMF_RESULT& res) { res = AMF_OK; return (amf_float)value; } static AMF_INLINE AMFVariant::String AMFConvertInt64ToString(amf_int64 value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%" AMFPRId64, (long long)value); return buff; } static AMF_INLINE AMFVariant::WString AMFConvertInt64ToWString(amf_int64 value, AMF_RESULT& res) { res = AMF_OK; wchar_t buff[0xFF]; swprintf(buff, 0xFF, L"%" LPRId64, (long long)value); return buff; } static AMF_INLINE bool AMFConvertDoubleToBool(amf_double value, AMF_RESULT& res){res = AMF_OK;return value != 0;} static AMF_INLINE bool AMFConvertFloatToBool(amf_float value, AMF_RESULT& res) { res = AMF_OK; return value != 0; } static AMF_INLINE amf_int64 AMFConvertDoubleToInt64(amf_double value, AMF_RESULT& res){res = AMF_OK;return amf_int64(value);} static AMF_INLINE amf_int64 AMFConvertFloatToInt64(amf_float value, AMF_RESULT& res) { res = AMF_OK; return amf_int64(value); } static AMF_INLINE AMFVariant::String AMFConvertDoubleToString(amf_double value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%lf", value); return buff; } static AMF_INLINE AMFVariant::String AMFConvertFloatToString(amf_float value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%f", value); return buff; } static AMF_INLINE AMFVariant::WString AMFConvertDoubleToWString(amf_double value, AMF_RESULT& res) { res = AMF_OK; wchar_t buff[0xFF]; swprintf(buff, 0xFF, L"%lf", value); return buff; } static AMF_INLINE AMFVariant::WString AMFConvertFloatToWString(amf_float value, AMF_RESULT& res) { res = AMF_OK; wchar_t buff[0xFF]; swprintf(buff, 0xFF, L"%f", value); return buff; } static AMF_INLINE bool AMFConvertStringToBool(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFVariant::String tmp = value; if (( tmp == "true") || ( tmp == "True") || ( tmp == "TRUE") || ( tmp == "1") ) { return true; } else { if (( tmp == "false") || ( tmp == "False") || ( tmp == "FALSE") || ( tmp == "0") ) { return false; } } res = AMF_INVALID_ARG; return false; } static AMF_INLINE amf_int64 AMFConvertStringToInt64(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; long long tmp = 0; int readElements = 0; if (value.size() > 2 && ( value.c_str()[0] == '0') && ( value.c_str()[1] == 'x') ) { readElements = sscanf(value.c_str(), "0x%" AMFPRIx64, &tmp); } else if (value.size() > 0) { readElements = sscanf(value.c_str(), "%" AMFPRId64, &tmp); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return 0; } static AMF_INLINE amf_double AMFConvertStringToDouble(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; amf_double tmp = 0; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%lf", &tmp); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return 0; } static AMF_INLINE amf_float AMFConvertStringToFloat(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; amf_float tmp = 0; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%f", &tmp); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return 0; } static AMF_INLINE AMFVariant::WString AMFConvertStringToWString(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; // return amf_from_utf8_to_unicode(value); AMFVariant::WString result; if (0 == value.size()) { return result; } const char* pUtf8Buff = value.c_str(); #if defined(_WIN32) _configthreadlocale(_ENABLE_PER_THREAD_LOCALE); int UnicodeBuffSize = ::MultiByteToWideChar(CP_UTF8, 0, pUtf8Buff, -1, NULL, 0); if (0 == UnicodeBuffSize) { return result; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); UnicodeBuffSize = ::MultiByteToWideChar(CP_UTF8, 0, pUtf8Buff, -1, (LPWSTR)result.c_str(), UnicodeBuffSize); UnicodeBuffSize--; #elif defined(__ANDROID__) // on android mbstowcs cannot be used to define length char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); mbstate_t mbs; mbrlen(NULL, 0, &mbs); int len = value.size(); const char* pt = pUtf8Buff; int UnicodeBuffSize = 0; while (len > 0) { size_t length = mbrlen (pt, len, &mbs); //MM TODO Android always return 1 if ((length == 0) || (length > len)) { break; } UnicodeBuffSize++; len -= length; pt += length; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); mbrlen (NULL, 0, &mbs); len = value.size(); pt = pUtf8Buff; UnicodeBuffSize = 0; while (len > 0) { size_t length = mbrlen (pt, len, &mbs); if ((length == 0) || (length > len)) { break; } mbrtowc(&((wchar_t*)(result.c_str()))[UnicodeBuffSize], pt, length, &mbs); //MM TODO Android always return 1 char UnicodeBuffSize++; len -= length; pt += length; } setlocale(LC_CTYPE, old_locale); #else char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); size_t UnicodeBuffSize = mbstowcs(NULL, pUtf8Buff, 0); if (0 == UnicodeBuffSize) { return result; } UnicodeBuffSize += 8; // get some extra space result.resize(UnicodeBuffSize); UnicodeBuffSize = mbstowcs((wchar_t*)result.c_str(), pUtf8Buff, UnicodeBuffSize + 1); setlocale(LC_CTYPE, old_locale); #endif result.resize(UnicodeBuffSize); return result; } static AMF_INLINE AMFVariant::String AMFConvertWStringToString(const AMFVariant::WString& value, AMF_RESULT& res) { res = AMF_OK; // return amf_from_unicode_to_utf8(value); AMFVariant::String result; if (0 == value.size()) { return result; } const wchar_t* pwBuff = value.c_str(); #if defined(_WIN32) _configthreadlocale(_ENABLE_PER_THREAD_LOCALE); int Utf8BuffSize = ::WideCharToMultiByte(CP_UTF8, 0, pwBuff, -1, NULL, 0, NULL, NULL); if (0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); Utf8BuffSize = ::WideCharToMultiByte(CP_UTF8, 0, pwBuff, -1, (LPSTR)result.c_str(), Utf8BuffSize, NULL, NULL); Utf8BuffSize--; #elif defined(__ANDROID__) char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); int Utf8BuffSize = value.length(); if (0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); mbstate_t mbs; mbrlen(NULL, 0, &mbs); Utf8BuffSize = 0; for (int i = 0; i < value.length(); i++) { //MM TODO Android - not implemented //int written = wcrtomb(&result[Utf8BuffSize], pwBuff[i], &mbs); ((char*)(result.c_str()))[Utf8BuffSize] = (char)(pwBuff[i]); int written = 1; // temp replacement Utf8BuffSize += written; } setlocale(LC_CTYPE, old_locale); #else char* old_locale = setlocale(LC_CTYPE, "en_US.UTF8"); size_t Utf8BuffSize = wcstombs(NULL, pwBuff, 0); if (0 == Utf8BuffSize) { return result; } Utf8BuffSize += 8; // get some extra space result.resize(Utf8BuffSize); Utf8BuffSize = wcstombs((char*)result.c_str(), pwBuff, Utf8BuffSize + 1); setlocale(LC_CTYPE, old_locale); #endif result.resize(Utf8BuffSize); return result; } static AMF_INLINE bool AMFConvertWStringToBool(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToBool(AMFConvertWStringToString(value, res), res); } static AMF_INLINE amf_int64 AMFConvertWStringToInt64(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToInt64(AMFConvertWStringToString(value, res), res); } static AMF_INLINE amf_double AMFConvertWStringToDouble(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToDouble(AMFConvertWStringToString(value, res), res); } static AMF_INLINE amf_float AMFConvertWStringToFloat(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToFloat(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertRectToString(const AMFRect& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%d,%d,%d,%d", value.left, value.top, value.right, value.bottom); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertSizeToString(const AMFSize& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%d,%d", value.width, value.height); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertPointToString(const AMFPoint& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%d,%d", value.x, value.y); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertFloatSizeToString(const AMFFloatSize& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%f,%f", value.width, value.height); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertFloatPoint2DToString(const AMFFloatPoint2D& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%f,%f", value.x, value.y); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertFloatPoint3DToString(const AMFFloatPoint3D& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%f,%f,%f", value.x, value.y, value.z); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertFloatVector4DToString(const AMFFloatVector4D& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%f,%f,%f,%f", value.x, value.y, value.z, value.w); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertRateToString(const AMFRate& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%u,%u", value.num, value.den); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertRatioToString(const AMFRatio& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%u,%u", value.num, value.den); return buff; } static AMF_INLINE AMFVariant::String AMF_STD_CALL AMFConvertColorToString(const AMFColor& value, AMF_RESULT& res) { res = AMF_OK; char buff[0xFF]; sprintf(buff, "%u,%u,%u,%u", value.r, value.g, value.b, value.a); return buff; } static AMF_INLINE AMFRect AMF_STD_CALL AMFConvertStringToRect(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFRect tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%d,%d,%d,%d", &tmp.left, &tmp.top, &tmp.right, &tmp.bottom); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFSize AMF_STD_CALL AMFConvertStringToSize(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFSize tmp = {}; int readElements = 0; if (value.size() > 0) { if (strchr(value.c_str(), ',') != nullptr) { readElements = sscanf(value.c_str(), "%d,%d", &tmp.width, &tmp.height); } else if (strchr(value.c_str(), 'x') != nullptr) { readElements = sscanf(value.c_str(), "%dx%d", &tmp.width, &tmp.height); } } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFPoint AMF_STD_CALL AMFConvertStringToPoint(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFPoint tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%d,%d", &tmp.x, &tmp.y); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFFloatSize AMF_STD_CALL AMFConvertStringToFloatSize(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFFloatSize tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%f,%f", &tmp.width, &tmp.height); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFFloatPoint2D AMF_STD_CALL AMFConvertStringToFloatPoint2D(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFFloatPoint2D tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%f,%f", &tmp.x, &tmp.y); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFFloatPoint3D AMF_STD_CALL AMFConvertStringToFloatPoint3D(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFFloatPoint3D tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%f,%f,%f", &tmp.x, &tmp.y, &tmp.z); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFFloatVector4D AMF_STD_CALL AMFConvertStringToFloatVector4D(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFFloatVector4D tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%f,%f,%f,%f", &tmp.x, &tmp.y, &tmp.z, &tmp.w); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFRate AMF_STD_CALL AMFConvertStringToRate(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFRate tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%u,%u", &tmp.num, &tmp.den); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFRatio AMF_STD_CALL AMFConvertStringToRatio(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; AMFRatio tmp = {}; int readElements = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%u,%u", &tmp.num, &tmp.den); } if (readElements) { return tmp; } res = AMF_INVALID_ARG; return tmp; } static AMF_INLINE AMFColor AMF_STD_CALL AMFConvertStringToColor(const AMFVariant::String& value, AMF_RESULT& res) { res = AMF_OK; int readElements = 0; amf_uint32 r = 0; amf_uint32 g = 0; amf_uint32 b = 0; amf_uint32 a = 0; if (value.size() > 0) { readElements = sscanf(value.c_str(), "%u,%u,%u,%u", &r, &g, &b, &a); } if (readElements) { return AMFConstructColor((amf_uint8)r, (amf_uint8)g, (amf_uint8)b, (amf_uint8)a); } res = AMF_INVALID_ARG; return AMFConstructColor(0, 0, 0, 255); } /////////////////////// static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertRectToWString(const AMFRect& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertRectToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertSizeToWString(const AMFSize& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertSizeToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertPointToWString(const AMFPoint& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertPointToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertFloatSizeToWString(const AMFFloatSize& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertFloatSizeToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertFloatPoint2DToWString(const AMFFloatPoint2D& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertFloatPoint2DToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertFloatPoint3DToWString(const AMFFloatPoint3D& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertFloatPoint3DToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertFloatVector4DToWString(const AMFFloatVector4D& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertFloatVector4DToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertRateToWString(const AMFRate& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertRateToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertRatioToWString(const AMFRatio& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertRatioToString(value, res), res); } static AMF_INLINE AMFVariant::WString AMF_STD_CALL AMFConvertColorToWString(const AMFColor& value, AMF_RESULT& res) { return AMFConvertStringToWString(AMFConvertColorToString(value, res), res); } static AMF_INLINE AMFRect AMF_STD_CALL AMFConvertWStringToRect(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToRect(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFSize AMF_STD_CALL AMFConvertWStringToSize(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToSize(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFPoint AMF_STD_CALL AMFConvertWStringToPoint(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToPoint(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFFloatSize AMF_STD_CALL AMFConvertWStringToFloatSize(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToFloatSize(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFFloatPoint2D AMF_STD_CALL AMFConvertWStringToFloatPoint2D(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToFloatPoint2D(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFFloatPoint3D AMF_STD_CALL AMFConvertWStringToFloatPoint3D(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToFloatPoint3D(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFFloatVector4D AMF_STD_CALL AMFConvertWStringToFloatVector4D(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToFloatVector4D(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFRate AMF_STD_CALL AMFConvertWStringToRate(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToRate(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFRatio AMF_STD_CALL AMFConvertWStringToRatio(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToRatio(AMFConvertWStringToString(value, res), res); } static AMF_INLINE AMFColor AMF_STD_CALL AMFConvertWStringToColor(const AMFVariant::WString& value, AMF_RESULT& res) { return AMFConvertStringToColor(AMFConvertWStringToString(value, res), res); } //------------------------------------------------------------------------------------------------- #define AMFConvertTool(srcType, dstType)\ if (AMFVariantGetType(pSrc) == AMFVariantType##srcType && newType == AMFVariantType##dstType)\ {\ AMF_RESULT res = AMF_OK;\ AMFVariantAssign##dstType(pDest, AMFConvert##srcType##To##dstType(AMFVariant##srcType(pSrc), res));\ return res;\ }\ //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantChangeType(AMFVariantStruct* pDest, const AMFVariantStruct* pSrc, AMF_VARIANT_TYPE newType) { AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); if (pSrc == nullptr) { pSrc = pDest; } if (AMFVariantGetType(pSrc) == newType) { if (pDest == pSrc) { return AMF_OK; } return AMFVariantCopy(pDest, pSrc); } if (pDest != pSrc) { AMFVariantClear(pDest); } AMFConvertTool(Empty, Bool); AMFConvertTool(Empty, Int64); AMFConvertTool(Empty, Double); AMFConvertTool(Empty, Float); AMFConvertTool(Empty, String); AMFConvertTool(Empty, WString); AMFConvertTool(Bool, Int64); AMFConvertTool(Bool, Double); AMFConvertTool(Bool, Float); AMFConvertTool(Bool, String); AMFConvertTool(Bool, WString); AMFConvertTool(Int64, Bool); AMFConvertTool(Int64, Double); AMFConvertTool(Int64, Float); AMFConvertTool(Int64, String); AMFConvertTool(Int64, WString); AMFConvertTool(Double, Bool); AMFConvertTool(Double, Int64); AMFConvertTool(Double, String); AMFConvertTool(Double, WString); AMFConvertTool(Float, Bool); AMFConvertTool(Float, Int64); AMFConvertTool(Float, String); AMFConvertTool(Float, WString); AMFConvertTool(String, Bool); AMFConvertTool(String, Int64); AMFConvertTool(String, Double); AMFConvertTool(String, Float); AMFConvertTool(String, WString); AMFConvertTool(WString, Bool); AMFConvertTool(WString, Int64); AMFConvertTool(WString, Double); AMFConvertTool(WString, Float); AMFConvertTool(WString, String); AMFConvertTool(String, Rect); AMFConvertTool(String, Size); AMFConvertTool(String, Point); AMFConvertTool(String, Rate); AMFConvertTool(String, Ratio); AMFConvertTool(String, Color); AMFConvertTool(Rect , String); AMFConvertTool(Size , String); AMFConvertTool(Point, String); AMFConvertTool(Rate , String); AMFConvertTool(Ratio, String); AMFConvertTool(Color, String); AMFConvertTool(WString, Rect); AMFConvertTool(WString, Size); AMFConvertTool(WString, Point); AMFConvertTool(WString, Rate); AMFConvertTool(WString, Ratio); AMFConvertTool(WString, Color); AMFConvertTool(Rect , WString); AMFConvertTool(Size , WString); AMFConvertTool(Point, WString); AMFConvertTool(Rate , WString); AMFConvertTool(Ratio, WString); AMFConvertTool(Color, WString); return AMF_INVALID_ARG; } #endif // #if defined(__cplusplus) //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignBool(AMFVariantStruct* pDest, amf_bool value) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_BOOL; AMFVariantBool(pDest) = value; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignInt64(AMFVariantStruct* pDest, amf_int64 value) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_INT64; AMFVariantInt64(pDest) = value; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignDouble(AMFVariantStruct* pDest, amf_double value) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_DOUBLE; AMFVariantDouble(pDest) = value; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloat(AMFVariantStruct* pDest, amf_float value) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_FLOAT; AMFVariantFloat(pDest) = value; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignString(AMFVariantStruct* pDest, const char* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { const size_t size = (strlen(pValue) + 1); pDest->type = AMF_VARIANT_STRING; AMFVariantString(pDest) = (char*)amf_variant_alloc(size * sizeof(char)); if (AMFVariantString(pDest)) { memcpy(AMFVariantString(pDest), pValue, size * sizeof(char)); } else { errRet = AMF_OUT_OF_MEMORY; } } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignWString(AMFVariantStruct* pDest, const wchar_t* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { const size_t size = (wcslen(pValue) + 1); pDest->type = AMF_VARIANT_WSTRING; AMFVariantWString(pDest) = (wchar_t*)amf_variant_alloc(size * sizeof(wchar_t)); if (AMFVariantWString(pDest)) { memcpy(AMFVariantWString(pDest), pValue, size * sizeof(wchar_t)); } else { errRet = AMF_OUT_OF_MEMORY; } } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignInterface(AMFVariantStruct* pDest, AMFInterface* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); //AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue);//can be NULL errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_INTERFACE; AMFVariantInterface(pDest) = pValue; if (AMFVariantInterface(pDest)) { #if defined(__cplusplus) AMFVariantInterface(pDest)->Acquire(); #else AMFVariantInterface(pDest)->pVtbl->Acquire(AMFVariantInterface(pDest)); #endif } } return errRet; } //------------------------------------------------------------------------------------------------- #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRect(AMFVariantStruct* pDest, const AMFRect& value) { return AMFVariantAssignRect(pDest, &value); } #endif //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRect (AMFVariantStruct* pDest, const AMFRect* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_RECT; AMFVariantRect(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignSize (AMFVariantStruct* pDest, const AMFSize& value) { return AMFVariantAssignSize (pDest, &value); } #endif //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignSize (AMFVariantStruct* pDest, const AMFSize* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_SIZE; AMFVariantSize(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignPoint(AMFVariantStruct* pDest, const AMFPoint& value) { return AMFVariantAssignPoint(pDest, &value); } static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatSize(AMFVariantStruct* pDest, const AMFFloatSize& value) { return AMFVariantAssignFloatSize(pDest, &value); } static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint2D(AMFVariantStruct* pDest, const AMFFloatPoint2D& value) { return AMFVariantAssignFloatPoint2D(pDest, &value); } static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint3D(AMFVariantStruct* pDest, const AMFFloatPoint3D& value) { return AMFVariantAssignFloatPoint3D(pDest, &value); } static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatVector4D(AMFVariantStruct* pDest, const AMFFloatVector4D& value) { return AMFVariantAssignFloatVector4D(pDest, &value); } #endif //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignPoint(AMFVariantStruct* pDest, const AMFPoint* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_POINT; AMFVariantPoint(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatSize(AMFVariantStruct* pDest, const AMFFloatSize* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_FLOAT_SIZE; AMFVariantFloatSize(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint2D(AMFVariantStruct* pDest, const AMFFloatPoint2D* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_FLOAT_POINT2D; AMFVariantFloatPoint2D(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatPoint3D(AMFVariantStruct* pDest, const AMFFloatPoint3D* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_FLOAT_POINT3D; AMFVariantFloatPoint3D(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignFloatVector4D(AMFVariantStruct* pDest, const AMFFloatVector4D* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_FLOAT_VECTOR4D; AMFVariantFloatVector4D(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRate (AMFVariantStruct* pDest, const AMFRate& value) { return AMFVariantAssignRate (pDest, &value); } #endif //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRate (AMFVariantStruct* pDest, const AMFRate* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_RATE; AMFVariantRate(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRatio(AMFVariantStruct* pDest, const AMFRatio& value) { return AMFVariantAssignRatio(pDest, &value); } #endif //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignRatio(AMFVariantStruct* pDest, const AMFRatio* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_RATIO; AMFVariantRatio(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- #if defined(__cplusplus) static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignColor(AMFVariantStruct* pDest, const AMFColor& value) { return AMFVariantAssignColor(pDest, &value); } #endif //------------------------------------------------------------------------------------------------- static AMF_INLINE AMF_RESULT AMF_CDECL_CALL AMFVariantAssignColor(AMFVariantStruct* pDest, const AMFColor* pValue) { AMF_RESULT errRet = AMF_OK; AMF_VARIANT_RETURN_IF_INVALID_POINTER(pDest); AMF_VARIANT_RETURN_IF_INVALID_POINTER(pValue); errRet = AMFVariantInit(pDest); if (errRet == AMF_OK) { pDest->type = AMF_VARIANT_COLOR; AMFVariantColor(pDest) = *pValue; } return errRet; } //------------------------------------------------------------------------------------------------- static AMF_INLINE char* AMF_CDECL_CALL AMFVariantDuplicateString(const char* pFrom) { char* ret = 0; if (pFrom) { ret = (char*)amf_variant_alloc(sizeof(char)*(strlen(pFrom) + 1)); if (ret) { strcpy(ret, pFrom); } } return ret; } //------------------------------------------------------------------------------------------------- static AMF_INLINE void AMF_CDECL_CALL AMFVariantFreeString(char* pFrom) { amf_variant_free(pFrom); } //------------------------------------------------------------------------------------------------- static AMF_INLINE wchar_t* AMF_CDECL_CALL AMFVariantDuplicateWString(const wchar_t* pFrom) { wchar_t* ret = 0; if (pFrom) { ret = (wchar_t*)amf_variant_alloc(sizeof(wchar_t)*(wcslen(pFrom) + 1)); if (ret) { wcscpy(ret, pFrom); } } return ret; } //------------------------------------------------------------------------------------------------- static AMF_INLINE void AMF_CDECL_CALL AMFVariantFreeWString(wchar_t* pFrom) { amf_variant_free(pFrom); } //---------------------------------------------------------------------------------------------- // AMF_INLINE implementation of AMFVariant class //---------------------------------------------------------------------------------------------- #if defined(__cplusplus) AMF_INLINE AMFVariant::AMFVariant(const AMFVariantStruct* pOther) { AMFVariantInit(this); if (pOther != nullptr) { AMFVariantCopy(this, const_cast(pOther)); } } //------------------------------------------------------------------------------------------------- template AMFVariant::AMFVariant(const AMFInterfacePtr_T& pValue) { AMFVariantInit(this); AMFVariantAssignInterface(this, pValue); } //------------------------------------------------------------------------------------------------- template ReturnType AMFVariant::GetValue(Getter getter) const { ReturnType str = ReturnType(); if (AMFVariantGetType(this) == variantType) { str = static_cast(getter(this)); } else { AMFVariant varDest; varDest.ChangeType(variantType, this); if (varDest.type != AMF_VARIANT_EMPTY) { str = static_cast(getter(&varDest)); } } return str; } //------------------------------------------------------------------------------------------------- AMF_INLINE AMFVariant& AMFVariant::operator=(const AMFVariantStruct& other) { AMFVariantClear(this); AMFVariantCopy(this, const_cast(&other)); return *this; } //------------------------------------------------------------------------------------------------- AMF_INLINE AMFVariant& AMFVariant::operator=(const AMFVariantStruct* pOther) { if (pOther != nullptr) { AMFVariantClear(this); AMFVariantCopy(this, const_cast(pOther)); } return *this; } //------------------------------------------------------------------------------------------------- AMF_INLINE AMFVariant& AMFVariant::operator=(const AMFVariant& other) { AMFVariantClear(this); AMFVariantCopy(this, const_cast(static_cast(&other))); return *this; } //------------------------------------------------------------------------------------------------- template AMFVariant& AMFVariant::operator=(const AMFInterfacePtr_T& value) { AMFVariantClear(this); AMFVariantAssignInterface(this, value); return *this; } //------------------------------------------------------------------------------------------------- AMF_INLINE bool AMFVariant::operator==(const AMFVariantStruct& other) const { return *this == &other; } //------------------------------------------------------------------------------------------------- AMF_INLINE bool AMFVariant::operator==(const AMFVariantStruct* pOther) const { //TODO: double check amf_bool ret = false; if (pOther == nullptr) { ret = false; } else { AMFVariantCompare(this, pOther, &ret); } return ret; } //------------------------------------------------------------------------------------------------- AMF_INLINE bool AMFVariant::operator!=(const AMFVariantStruct& other) const { return !(*this == &other); } //------------------------------------------------------------------------------------------------- AMF_INLINE bool AMFVariant::operator!=(const AMFVariantStruct* pOther) const { return !(*this == pOther); } //------------------------------------------------------------------------------------------------- AMF_INLINE void AMFVariant::Attach(AMFVariantStruct& pVariant) { Clear(); memcpy(this, &pVariant, sizeof(pVariant)); AMFVariantGetType(&pVariant) = AMF_VARIANT_EMPTY; } //------------------------------------------------------------------------------------------------- AMF_INLINE AMFVariantStruct AMFVariant::Detach() { AMFVariantStruct varResult = *this; AMFVariantGetType(this) = AMF_VARIANT_EMPTY; return varResult; } //------------------------------------------------------------------------------------------------- AMF_INLINE AMFVariantStruct& AMFVariant::GetVariant() { return *static_cast(this); } //------------------------------------------------------------------------------------------------- AMF_INLINE void AMFVariant::ChangeType(AMF_VARIANT_TYPE newType, const AMFVariant* pSrc) { AMFVariantChangeType(this, pSrc, newType); } //------------------------------------------------------------------------------------------------- AMF_INLINE bool AMFVariant::Empty() const { return type == AMF_VARIANT_EMPTY; } //------------------------------------------------------------------------------------------------- #endif // #if defined(__cplusplus) #if defined(__cplusplus) } //namespace amf #endif #endif //#ifndef AMF_Variant_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/Version.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2017 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // /** *************************************************************************************************** * @file Version.h * @brief Version declaration *************************************************************************************************** */ #ifndef AMF_Version_h #define AMF_Version_h #pragma once #include "Platform.h" #define AMF_MAKE_FULL_VERSION(VERSION_MAJOR, VERSION_MINOR, VERSION_RELEASE, VERSION_BUILD_NUM) ( ((amf_uint64)(VERSION_MAJOR) << 48ull) | ((amf_uint64)(VERSION_MINOR) << 32ull) | ((amf_uint64)(VERSION_RELEASE) << 16ull) | (amf_uint64)(VERSION_BUILD_NUM)) #define AMF_GET_MAJOR_VERSION(x) ((x >> 48ull) & 0xFFFF) #define AMF_GET_MINOR_VERSION(x) ((x >> 32ull) & 0xFFFF) #define AMF_GET_SUBMINOR_VERSION(x) ((x >> 16ull) & 0xFFFF) #define AMF_GET_BUILD_VERSION(x) ((x >> 0ull) & 0xFFFF) #define AMF_VERSION_MAJOR 1 #define AMF_VERSION_MINOR 4 #define AMF_VERSION_RELEASE 33 #define AMF_VERSION_BUILD_NUM 0 #define AMF_FULL_VERSION AMF_MAKE_FULL_VERSION(AMF_VERSION_MAJOR, AMF_VERSION_MINOR, AMF_VERSION_RELEASE, AMF_VERSION_BUILD_NUM) #endif //#ifndef AMF_Version_h ================================================ FILE: alvr/server_openvr/cpp/shared/amf/public/include/core/VulkanAMF.h ================================================ // // Notice Regarding Standards. AMD does not provide a license or sublicense to // any Intellectual Property Rights relating to any standards, including but not // limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; // AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 // (collectively, the "Media Technologies"). For clarity, you will pay any // royalties due for such third party technologies, which may include the Media // Technologies that are owed as a result of AMD providing the Software to you. // // MIT license // // Copyright (c) 2018 Advanced Micro Devices, Inc. All rights reserved. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in // all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN // THE SOFTWARE. // #ifndef __VulkanAMF_h__ #define __VulkanAMF_h__ #pragma once #include "Platform.h" #include "vulkan/vulkan.h" #if defined(__cplusplus) namespace amf { #endif typedef struct AMFVulkanDevice { amf_size cbSizeof; // sizeof(AMFVulkanDevice) void* pNext; // reserved for extensions VkInstance hInstance; VkPhysicalDevice hPhysicalDevice; VkDevice hDevice; } AMFVulkanDevice; typedef struct AMFVulkanSync { amf_size cbSizeof; // sizeof(AMFVulkanSync) void* pNext; // reserved for extensions VkSemaphore hSemaphore; // VkSemaphore; can be nullptr amf_bool bSubmitted; // if true - wait for hSemaphore. re-submit hSemaphore if not synced by other ways and set to true VkFence hFence; // To sync on CPU; can be nullptr. Submitted in vkQueueSubmit. If waited for hFence, null it, do not delete or reset. } AMFVulkanSync; typedef struct AMFVulkanBuffer { amf_size cbSizeof; // sizeof(AMFVulkanBuffer) void* pNext; // reserved for extensions VkBuffer hBuffer; VkDeviceMemory hMemory; amf_int64 iSize; amf_int64 iAllocatedSize; // for reuse amf_uint32 eAccessFlags; // VkAccessFlagBits amf_uint32 eUsage; // AMF_BUFFER_USAGE amf_uint32 eAccess; // AMF_MEMORY_CPU_ACCESS AMFVulkanSync Sync; } AMFVulkanBuffer; typedef struct AMFVulkanSurface { amf_size cbSizeof; // sizeof(AMFVulkanSurface) void* pNext; // reserved for extensions // surface properties VkImage hImage; // vulkan native image for which the surface is created VkDeviceMemory hMemory; // memory for hImage, can be nullptr amf_int64 iSize; // memory size amf_uint32 eFormat; // VkFormat amf_int32 iWidth; // image width amf_int32 iHeight; // image height amf_uint32 eCurrentLayout; // VkImageLayout amf_uint32 eUsage; // AMF_SURFACE_USAGE amf_uint32 eAccess; // AMF_MEMORY_CPU_ACCESS AMFVulkanSync Sync; // To sync on GPU } AMFVulkanSurface; typedef struct AMFVulkanSurface1 { amf_size cbSizeof; // sizeof(AMFVulkanSurface) void* pNext; // reserved for extensions // surface properties amf_uint32 eTiling; // VkImageTiling } AMFVulkanSurface1; typedef struct AMFVulkanView { amf_size cbSizeof; // sizeof(AMFVulkanView) void* pNext; // reserved for extensions // surface properties AMFVulkanSurface *pSurface; VkImageView hView; amf_int32 iPlaneWidth; amf_int32 iPlaneHeight; amf_int32 iPlaneWidthPitch; amf_int32 iPlaneHeightPitch; } AMFVulkanView; #define AMF_CONTEXT_VULKAN_COMPUTE_QUEUE L"VulkanComputeQueue" // amf_int64; default=0; Compute queue index in range [0, (VkQueueFamilyProperties.queueCount-1)] of the compute queue family. #if defined(__cplusplus) } // namespace amf #endif #endif // __VulkanAMF_h__ ================================================ FILE: alvr/server_openvr/cpp/shared/backward.cpp ================================================ // Pick your poison. // // On GNU/Linux, you have few choices to get the most out of your stack trace. // // By default you get: // - object filename // - function name // // In order to add: // - source filename // - line and column numbers // - source code snippet (assuming the file is accessible) // Install one of the following libraries then uncomment one of the macro (or // better, add the detection of the lib and the macro definition in your build // system) // - apt-get install libdw-dev ... // - g++/clang++ -ldw ... // #define BACKWARD_HAS_DW 1 // - apt-get install binutils-dev ... // - g++/clang++ -lbfd ... // #define BACKWARD_HAS_BFD 1 // - apt-get install libdwarf-dev ... // - g++/clang++ -ldwarf ... // #define BACKWARD_HAS_DWARF 1 // Regardless of the library you choose to read the debug information, // for potentially more detailed stack traces you can use libunwind // - apt-get install libunwind-dev // - g++/clang++ -lunwind // #define BACKWARD_HAS_LIBUNWIND 1 #include "backward.hpp" namespace backward { backward::SignalHandling sh; } // namespace backward ================================================ FILE: alvr/server_openvr/cpp/shared/backward.hpp ================================================ /* * backward.hpp * Copyright 2013 Google Inc. All Rights Reserved. * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to deal * in the Software without restriction, including without limitation the rights * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell * copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #ifndef H_6B9572DA_A64B_49E6_B234_051480991C89 #define H_6B9572DA_A64B_49E6_B234_051480991C89 #ifndef __cplusplus #error "It's not going to compile without a C++ compiler..." #endif #if defined(BACKWARD_CXX11) #elif defined(BACKWARD_CXX98) #else #if __cplusplus >= 201103L || (defined(_MSC_VER) && _MSC_VER >= 1800) #define BACKWARD_CXX11 #define BACKWARD_ATLEAST_CXX11 #define BACKWARD_ATLEAST_CXX98 #if __cplusplus >= 201703L || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L) #define BACKWARD_ATLEAST_CXX17 #endif #else #define BACKWARD_CXX98 #define BACKWARD_ATLEAST_CXX98 #endif #endif // You can define one of the following (or leave it to the auto-detection): // // #define BACKWARD_SYSTEM_LINUX // - specialization for linux // // #define BACKWARD_SYSTEM_DARWIN // - specialization for Mac OS X 10.5 and later. // // #define BACKWARD_SYSTEM_WINDOWS // - specialization for Windows (Clang 9 and MSVC2017) // // #define BACKWARD_SYSTEM_UNKNOWN // - placebo implementation, does nothing. // #if defined(BACKWARD_SYSTEM_LINUX) #elif defined(BACKWARD_SYSTEM_DARWIN) #elif defined(BACKWARD_SYSTEM_UNKNOWN) #elif defined(BACKWARD_SYSTEM_WINDOWS) #else #if defined(__linux) || defined(__linux__) #define BACKWARD_SYSTEM_LINUX #elif defined(__APPLE__) #define BACKWARD_SYSTEM_DARWIN #elif defined(_WIN32) #define BACKWARD_SYSTEM_WINDOWS #else #define BACKWARD_SYSTEM_UNKNOWN #endif #endif #define NOINLINE __attribute__((noinline)) #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #if defined(BACKWARD_SYSTEM_LINUX) // On linux, backtrace can back-trace or "walk" the stack using the following // libraries: // // #define BACKWARD_HAS_UNWIND 1 // - unwind comes from libgcc, but I saw an equivalent inside clang itself. // - with unwind, the stacktrace is as accurate as it can possibly be, since // this is used by the C++ runtime in gcc/clang for stack unwinding on // exception. // - normally libgcc is already linked to your program by default. // // #define BACKWARD_HAS_LIBUNWIND 1 // - libunwind provides, in some cases, a more accurate stacktrace as it knows // to decode signal handler frames and lets us edit the context registers when // unwinding, allowing stack traces over bad function references. // // #define BACKWARD_HAS_BACKTRACE == 1 // - backtrace seems to be a little bit more portable than libunwind, but on // linux, it uses unwind anyway, but abstract away a tiny information that is // sadly really important in order to get perfectly accurate stack traces. // - backtrace is part of the (e)glib library. // // The default is: // #define BACKWARD_HAS_UNWIND == 1 // // Note that only one of the define should be set to 1 at a time. // #if BACKWARD_HAS_UNWIND == 1 #elif BACKWARD_HAS_LIBUNWIND == 1 #elif BACKWARD_HAS_BACKTRACE == 1 #else #undef BACKWARD_HAS_UNWIND #define BACKWARD_HAS_UNWIND 1 #undef BACKWARD_HAS_LIBUNWIND #define BACKWARD_HAS_LIBUNWIND 0 #undef BACKWARD_HAS_BACKTRACE #define BACKWARD_HAS_BACKTRACE 0 #endif // On linux, backward can extract detailed information about a stack trace // using one of the following libraries: // // #define BACKWARD_HAS_DW 1 // - libdw gives you the most juicy details out of your stack traces: // - object filename // - function name // - source filename // - line and column numbers // - source code snippet (assuming the file is accessible) // - variable names (if not optimized out) // - variable values (not supported by backward-cpp) // - You need to link with the lib "dw": // - apt-get install libdw-dev // - g++/clang++ -ldw ... // // #define BACKWARD_HAS_BFD 1 // - With libbfd, you get a fair amount of details: // - object filename // - function name // - source filename // - line numbers // - source code snippet (assuming the file is accessible) // - You need to link with the lib "bfd": // - apt-get install binutils-dev // - g++/clang++ -lbfd ... // // #define BACKWARD_HAS_DWARF 1 // - libdwarf gives you the most juicy details out of your stack traces: // - object filename // - function name // - source filename // - line and column numbers // - source code snippet (assuming the file is accessible) // - variable names (if not optimized out) // - variable values (not supported by backward-cpp) // - You need to link with the lib "dwarf": // - apt-get install libdwarf-dev // - g++/clang++ -ldwarf ... // // #define BACKWARD_HAS_BACKTRACE_SYMBOL 1 // - backtrace provides minimal details for a stack trace: // - object filename // - function name // - backtrace is part of the (e)glib library. // // The default is: // #define BACKWARD_HAS_BACKTRACE_SYMBOL == 1 // // Note that only one of the define should be set to 1 at a time. // #if BACKWARD_HAS_DW == 1 #elif BACKWARD_HAS_BFD == 1 #elif BACKWARD_HAS_DWARF == 1 #elif BACKWARD_HAS_BACKTRACE_SYMBOL == 1 #else #undef BACKWARD_HAS_DW #define BACKWARD_HAS_DW 0 #undef BACKWARD_HAS_BFD #define BACKWARD_HAS_BFD 0 #undef BACKWARD_HAS_DWARF #define BACKWARD_HAS_DWARF 0 #undef BACKWARD_HAS_BACKTRACE_SYMBOL #define BACKWARD_HAS_BACKTRACE_SYMBOL 1 #endif #include #include #ifdef __ANDROID__ // Old Android API levels define _Unwind_Ptr in both link.h and // unwind.h Rename the one in link.h as we are not going to be using // it #define _Unwind_Ptr _Unwind_Ptr_Custom #include #undef _Unwind_Ptr #else #include #endif #if defined(__ppc__) || defined(__powerpc) || defined(__powerpc__) || \ defined(__POWERPC__) // Linux kernel header required for the struct pt_regs definition // to access the NIP (Next Instruction Pointer) register value #include #endif #include #include #include #include #ifndef _GNU_SOURCE #define _GNU_SOURCE #include #undef _GNU_SOURCE #else #include #endif #if BACKWARD_HAS_BFD == 1 // NOTE: defining PACKAGE{,_VERSION} is required before including // bfd.h on some platforms, see also: // https://sourceware.org/bugzilla/show_bug.cgi?id=14243 #ifndef PACKAGE #define PACKAGE #endif #ifndef PACKAGE_VERSION #define PACKAGE_VERSION #endif #include #endif #if BACKWARD_HAS_DW == 1 #include #include #include #endif #if BACKWARD_HAS_DWARF == 1 #include #include #include #include #include #endif #if (BACKWARD_HAS_BACKTRACE == 1) || (BACKWARD_HAS_BACKTRACE_SYMBOL == 1) // then we shall rely on backtrace #include #endif #endif // defined(BACKWARD_SYSTEM_LINUX) #if defined(BACKWARD_SYSTEM_DARWIN) // On Darwin, backtrace can back-trace or "walk" the stack using the following // libraries: // // #define BACKWARD_HAS_UNWIND 1 // - unwind comes from libgcc, but I saw an equivalent inside clang itself. // - with unwind, the stacktrace is as accurate as it can possibly be, since // this is used by the C++ runtime in gcc/clang for stack unwinding on // exception. // - normally libgcc is already linked to your program by default. // // #define BACKWARD_HAS_LIBUNWIND 1 // - libunwind comes from clang, which implements an API compatible version. // - libunwind provides, in some cases, a more accurate stacktrace as it knows // to decode signal handler frames and lets us edit the context registers when // unwinding, allowing stack traces over bad function references. // // #define BACKWARD_HAS_BACKTRACE == 1 // - backtrace is available by default, though it does not produce as much // information as another library might. // // The default is: // #define BACKWARD_HAS_UNWIND == 1 // // Note that only one of the define should be set to 1 at a time. // #if BACKWARD_HAS_UNWIND == 1 #elif BACKWARD_HAS_BACKTRACE == 1 #elif BACKWARD_HAS_LIBUNWIND == 1 #else #undef BACKWARD_HAS_UNWIND #define BACKWARD_HAS_UNWIND 1 #undef BACKWARD_HAS_BACKTRACE #define BACKWARD_HAS_BACKTRACE 0 #undef BACKWARD_HAS_LIBUNWIND #define BACKWARD_HAS_LIBUNWIND 0 #endif // On Darwin, backward can extract detailed information about a stack trace // using one of the following libraries: // // #define BACKWARD_HAS_BACKTRACE_SYMBOL 1 // - backtrace provides minimal details for a stack trace: // - object filename // - function name // // The default is: // #define BACKWARD_HAS_BACKTRACE_SYMBOL == 1 // #if BACKWARD_HAS_BACKTRACE_SYMBOL == 1 #else #undef BACKWARD_HAS_BACKTRACE_SYMBOL #define BACKWARD_HAS_BACKTRACE_SYMBOL 1 #endif #include #include #include #include #include #include #if (BACKWARD_HAS_BACKTRACE == 1) || (BACKWARD_HAS_BACKTRACE_SYMBOL == 1) #include #endif #endif // defined(BACKWARD_SYSTEM_DARWIN) #if defined(BACKWARD_SYSTEM_WINDOWS) #include #include #include #include #ifdef _WIN64 typedef SSIZE_T ssize_t; #else typedef int ssize_t; #endif #ifndef NOMINMAX #define NOMINMAX #endif #include #include #include #include #ifndef __clang__ #undef NOINLINE #define NOINLINE __declspec(noinline) #endif #ifdef _MSC_VER #pragma comment(lib, "psapi.lib") #pragma comment(lib, "dbghelp.lib") #endif // Comment / packing is from stackoverflow: // https://stackoverflow.com/questions/6205981/windows-c-stack-trace-from-a-running-app/28276227#28276227 // Some versions of imagehlp.dll lack the proper packing directives themselves // so we need to do it. #pragma pack(push, before_imagehlp, 8) #include #pragma pack(pop, before_imagehlp) // TODO maybe these should be undefined somewhere else? #undef BACKWARD_HAS_UNWIND #undef BACKWARD_HAS_BACKTRACE #if BACKWARD_HAS_PDB_SYMBOL == 1 #else #undef BACKWARD_HAS_PDB_SYMBOL #define BACKWARD_HAS_PDB_SYMBOL 1 #endif #endif #if BACKWARD_HAS_UNWIND == 1 #include // while gcc's unwind.h defines something like that: // extern _Unwind_Ptr _Unwind_GetIP (struct _Unwind_Context *); // extern _Unwind_Ptr _Unwind_GetIPInfo (struct _Unwind_Context *, int *); // // clang's unwind.h defines something like this: // uintptr_t _Unwind_GetIP(struct _Unwind_Context* __context); // // Even if the _Unwind_GetIPInfo can be linked to, it is not declared, worse we // cannot just redeclare it because clang's unwind.h doesn't define _Unwind_Ptr // anyway. // // Luckily we can play on the fact that the guard macros have a different name: #ifdef __CLANG_UNWIND_H // In fact, this function still comes from libgcc (on my different linux boxes, // clang links against libgcc). #include extern "C" uintptr_t _Unwind_GetIPInfo(_Unwind_Context *, int *); #endif #endif // BACKWARD_HAS_UNWIND == 1 #if BACKWARD_HAS_LIBUNWIND == 1 #define UNW_LOCAL_ONLY #include #endif // BACKWARD_HAS_LIBUNWIND == 1 #ifdef BACKWARD_ATLEAST_CXX11 #include #include // for std::swap namespace backward { namespace details { template struct hashtable { typedef std::unordered_map type; }; using std::move; } // namespace details } // namespace backward #else // NOT BACKWARD_ATLEAST_CXX11 #define nullptr NULL #define override #include namespace backward { namespace details { template struct hashtable { typedef std::map type; }; template const T &move(const T &v) { return v; } template T &move(T &v) { return v; } } // namespace details } // namespace backward #endif // BACKWARD_ATLEAST_CXX11 namespace backward { namespace details { #if defined(BACKWARD_SYSTEM_WINDOWS) const char kBackwardPathDelimiter[] = ";"; #else const char kBackwardPathDelimiter[] = ":"; #endif } // namespace details } // namespace backward namespace backward { namespace system_tag { struct linux_tag; // seems that I cannot call that "linux" because the name // is already defined... so I am adding _tag everywhere. struct darwin_tag; struct windows_tag; struct unknown_tag; #if defined(BACKWARD_SYSTEM_LINUX) typedef linux_tag current_tag; #elif defined(BACKWARD_SYSTEM_DARWIN) typedef darwin_tag current_tag; #elif defined(BACKWARD_SYSTEM_WINDOWS) typedef windows_tag current_tag; #elif defined(BACKWARD_SYSTEM_UNKNOWN) typedef unknown_tag current_tag; #else #error "May I please get my system defines?" #endif } // namespace system_tag namespace trace_resolver_tag { #if defined(BACKWARD_SYSTEM_LINUX) struct libdw; struct libbfd; struct libdwarf; struct backtrace_symbol; #if BACKWARD_HAS_DW == 1 typedef libdw current; #elif BACKWARD_HAS_BFD == 1 typedef libbfd current; #elif BACKWARD_HAS_DWARF == 1 typedef libdwarf current; #elif BACKWARD_HAS_BACKTRACE_SYMBOL == 1 typedef backtrace_symbol current; #else #error "You shall not pass, until you know what you want." #endif #elif defined(BACKWARD_SYSTEM_DARWIN) struct backtrace_symbol; #if BACKWARD_HAS_BACKTRACE_SYMBOL == 1 typedef backtrace_symbol current; #else #error "You shall not pass, until you know what you want." #endif #elif defined(BACKWARD_SYSTEM_WINDOWS) struct pdb_symbol; #if BACKWARD_HAS_PDB_SYMBOL == 1 typedef pdb_symbol current; #else #error "You shall not pass, until you know what you want." #endif #endif } // namespace trace_resolver_tag namespace details { template struct rm_ptr { typedef T type; }; template struct rm_ptr { typedef T type; }; template struct rm_ptr { typedef const T type; }; template struct deleter { template void operator()(U &ptr) const { (*F)(ptr); } }; template struct default_delete { void operator()(T &ptr) const { delete ptr; } }; template > class handle { struct dummy; T _val; bool _empty; #ifdef BACKWARD_ATLEAST_CXX11 handle(const handle &) = delete; handle &operator=(const handle &) = delete; #endif public: ~handle() { if (!_empty) { Deleter()(_val); } } explicit handle() : _val(), _empty(true) {} explicit handle(T val) : _val(val), _empty(false) { if (!_val) _empty = true; } #ifdef BACKWARD_ATLEAST_CXX11 handle(handle &&from) : _empty(true) { swap(from); } handle &operator=(handle &&from) { swap(from); return *this; } #else explicit handle(const handle &from) : _empty(true) { // some sort of poor man's move semantic. swap(const_cast(from)); } handle &operator=(const handle &from) { // some sort of poor man's move semantic. swap(const_cast(from)); return *this; } #endif void reset(T new_val) { handle tmp(new_val); swap(tmp); } void update(T new_val) { _val = new_val; _empty = !static_cast(new_val); } operator const dummy *() const { if (_empty) { return nullptr; } return reinterpret_cast(_val); } T get() { return _val; } T release() { _empty = true; return _val; } void swap(handle &b) { using std::swap; swap(b._val, _val); // can throw, we are safe here. swap(b._empty, _empty); // should not throw: if you cannot swap two // bools without throwing... It's a lost cause anyway! } T &operator->() { return _val; } const T &operator->() const { return _val; } typedef typename rm_ptr::type &ref_t; typedef const typename rm_ptr::type &const_ref_t; ref_t operator*() { return *_val; } const_ref_t operator*() const { return *_val; } ref_t operator[](size_t idx) { return _val[idx]; } // Watch out, we've got a badass over here T *operator&() { _empty = false; return &_val; } }; // Default demangler implementation (do nothing). template struct demangler_impl { static std::string demangle(const char *funcname) { return funcname; } }; #if defined(BACKWARD_SYSTEM_LINUX) || defined(BACKWARD_SYSTEM_DARWIN) template <> struct demangler_impl { demangler_impl() : _demangle_buffer_length(0) {} std::string demangle(const char *funcname) { using namespace details; char *result = abi::__cxa_demangle(funcname, _demangle_buffer.get(), &_demangle_buffer_length, nullptr); if (result) { _demangle_buffer.update(result); return result; } return funcname; } private: details::handle _demangle_buffer; size_t _demangle_buffer_length; }; #endif // BACKWARD_SYSTEM_LINUX || BACKWARD_SYSTEM_DARWIN struct demangler : public demangler_impl {}; // Split a string on the platform's PATH delimiter. Example: if delimiter // is ":" then: // "" --> [] // ":" --> ["",""] // "::" --> ["","",""] // "/a/b/c" --> ["/a/b/c"] // "/a/b/c:/d/e/f" --> ["/a/b/c","/d/e/f"] // etc. inline std::vector split_source_prefixes(const std::string &s) { std::vector out; size_t last = 0; size_t next = 0; size_t delimiter_size = sizeof(kBackwardPathDelimiter) - 1; while ((next = s.find(kBackwardPathDelimiter, last)) != std::string::npos) { out.push_back(s.substr(last, next - last)); last = next + delimiter_size; } if (last <= s.length()) { out.push_back(s.substr(last)); } return out; } } // namespace details /*************** A TRACE ***************/ struct Trace { void *addr; size_t idx; Trace() : addr(nullptr), idx(0) {} explicit Trace(void *_addr, size_t _idx) : addr(_addr), idx(_idx) {} }; struct ResolvedTrace : public Trace { struct SourceLoc { std::string function; std::string filename; unsigned line; unsigned col; SourceLoc() : line(0), col(0) {} bool operator==(const SourceLoc &b) const { return function == b.function && filename == b.filename && line == b.line && col == b.col; } bool operator!=(const SourceLoc &b) const { return !(*this == b); } }; // In which binary object this trace is located. std::string object_filename; // The function in the object that contain the trace. This is not the same // as source.function which can be an function inlined in object_function. std::string object_function; // The source location of this trace. It is possible for filename to be // empty and for line/col to be invalid (value 0) if this information // couldn't be deduced, for example if there is no debug information in the // binary object. SourceLoc source; // An optionals list of "inliners". All the successive sources location // from where the source location of the trace (the attribute right above) // is inlined. It is especially useful when you compiled with optimization. typedef std::vector source_locs_t; source_locs_t inliners; ResolvedTrace() : Trace() {} ResolvedTrace(const Trace &mini_trace) : Trace(mini_trace) {} }; /*************** STACK TRACE ***************/ // default implemention. template class StackTraceImpl { public: size_t size() const { return 0; } Trace operator[](size_t) const { return Trace(); } size_t load_here(size_t = 0) { return 0; } size_t load_from(void *, size_t = 0, void * = nullptr, void * = nullptr) { return 0; } size_t thread_id() const { return 0; } void skip_n_firsts(size_t) {} void *const *begin() const { return nullptr; } }; class StackTraceImplBase { public: StackTraceImplBase() : _thread_id(0), _skip(0), _context(nullptr), _error_addr(nullptr) {} size_t thread_id() const { return _thread_id; } void skip_n_firsts(size_t n) { _skip = n; } protected: void load_thread_info() { #ifdef BACKWARD_SYSTEM_LINUX #ifndef __ANDROID__ _thread_id = static_cast(syscall(SYS_gettid)); #else _thread_id = static_cast(gettid()); #endif if (_thread_id == static_cast(getpid())) { // If the thread is the main one, let's hide that. // I like to keep little secret sometimes. _thread_id = 0; } #elif defined(BACKWARD_SYSTEM_DARWIN) _thread_id = reinterpret_cast(pthread_self()); if (pthread_main_np() == 1) { // If the thread is the main one, let's hide that. _thread_id = 0; } #endif } void set_context(void *context) { _context = context; } void *context() const { return _context; } void set_error_addr(void *error_addr) { _error_addr = error_addr; } void *error_addr() const { return _error_addr; } size_t skip_n_firsts() const { return _skip; } private: size_t _thread_id; size_t _skip; void *_context; void *_error_addr; }; class StackTraceImplHolder : public StackTraceImplBase { public: size_t size() const { return (_stacktrace.size() >= skip_n_firsts()) ? _stacktrace.size() - skip_n_firsts() : 0; } Trace operator[](size_t idx) const { if (idx >= size()) { return Trace(); } return Trace(_stacktrace[idx + skip_n_firsts()], idx); } void *const *begin() const { if (size()) { return &_stacktrace[skip_n_firsts()]; } return nullptr; } protected: std::vector _stacktrace; }; #if BACKWARD_HAS_UNWIND == 1 namespace details { template class Unwinder { public: size_t operator()(F &f, size_t depth) { _f = &f; _index = -1; _depth = depth; _Unwind_Backtrace(&this->backtrace_trampoline, this); if (_index == -1) { // _Unwind_Backtrace has failed to obtain any backtraces return 0; } else { return static_cast(_index); } } private: F *_f; ssize_t _index; size_t _depth; static _Unwind_Reason_Code backtrace_trampoline(_Unwind_Context *ctx, void *self) { return (static_cast(self))->backtrace(ctx); } _Unwind_Reason_Code backtrace(_Unwind_Context *ctx) { if (_index >= 0 && static_cast(_index) >= _depth) return _URC_END_OF_STACK; int ip_before_instruction = 0; uintptr_t ip = _Unwind_GetIPInfo(ctx, &ip_before_instruction); if (!ip_before_instruction) { // calculating 0-1 for unsigned, looks like a possible bug to sanitizers, // so let's do it explicitly: if (ip == 0) { ip = std::numeric_limits::max(); // set it to 0xffff... (as // from casting 0-1) } else { ip -= 1; // else just normally decrement it (no overflow/underflow will // happen) } } if (_index >= 0) { // ignore first frame. (*_f)(static_cast(_index), reinterpret_cast(ip)); } _index += 1; return _URC_NO_REASON; } }; template size_t unwind(F f, size_t depth) { Unwinder unwinder; return unwinder(f, depth); } } // namespace details template <> class StackTraceImpl : public StackTraceImplHolder { public: NOINLINE size_t load_here(size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { load_thread_info(); set_context(context); set_error_addr(error_addr); if (depth == 0) { return 0; } _stacktrace.resize(depth); size_t trace_cnt = details::unwind(callback(*this), depth); _stacktrace.resize(trace_cnt); skip_n_firsts(0); return size(); } size_t load_from(void *addr, size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { load_here(depth + 8, context, error_addr); for (size_t i = 0; i < _stacktrace.size(); ++i) { if (_stacktrace[i] == addr) { skip_n_firsts(i); break; } } _stacktrace.resize(std::min(_stacktrace.size(), skip_n_firsts() + depth)); return size(); } private: struct callback { StackTraceImpl &self; callback(StackTraceImpl &_self) : self(_self) {} void operator()(size_t idx, void *addr) { self._stacktrace[idx] = addr; } }; }; #elif BACKWARD_HAS_LIBUNWIND == 1 template <> class StackTraceImpl : public StackTraceImplHolder { public: __attribute__((noinline)) size_t load_here(size_t depth = 32, void *_context = nullptr, void *_error_addr = nullptr) { set_context(_context); set_error_addr(_error_addr); load_thread_info(); if (depth == 0) { return 0; } _stacktrace.resize(depth + 1); int result = 0; unw_context_t ctx; size_t index = 0; // Add the tail call. If the Instruction Pointer is the crash address it // means we got a bad function pointer dereference, so we "unwind" the // bad pointer manually by using the return address pointed to by the // Stack Pointer as the Instruction Pointer and letting libunwind do // the rest if (context()) { ucontext_t *uctx = reinterpret_cast(context()); #ifdef REG_RIP // x86_64 if (uctx->uc_mcontext.gregs[REG_RIP] == reinterpret_cast(error_addr())) { uctx->uc_mcontext.gregs[REG_RIP] = *reinterpret_cast(uctx->uc_mcontext.gregs[REG_RSP]); } _stacktrace[index] = reinterpret_cast(uctx->uc_mcontext.gregs[REG_RIP]); ++index; ctx = *reinterpret_cast(uctx); #elif defined(REG_EIP) // x86_32 if (uctx->uc_mcontext.gregs[REG_EIP] == reinterpret_cast(error_addr())) { uctx->uc_mcontext.gregs[REG_EIP] = *reinterpret_cast(uctx->uc_mcontext.gregs[REG_ESP]); } _stacktrace[index] = reinterpret_cast(uctx->uc_mcontext.gregs[REG_EIP]); ++index; ctx = *reinterpret_cast(uctx); #elif defined(__arm__) // libunwind uses its own context type for ARM unwinding. // Copy the registers from the signal handler's context so we can // unwind unw_getcontext(&ctx); ctx.regs[UNW_ARM_R0] = uctx->uc_mcontext.arm_r0; ctx.regs[UNW_ARM_R1] = uctx->uc_mcontext.arm_r1; ctx.regs[UNW_ARM_R2] = uctx->uc_mcontext.arm_r2; ctx.regs[UNW_ARM_R3] = uctx->uc_mcontext.arm_r3; ctx.regs[UNW_ARM_R4] = uctx->uc_mcontext.arm_r4; ctx.regs[UNW_ARM_R5] = uctx->uc_mcontext.arm_r5; ctx.regs[UNW_ARM_R6] = uctx->uc_mcontext.arm_r6; ctx.regs[UNW_ARM_R7] = uctx->uc_mcontext.arm_r7; ctx.regs[UNW_ARM_R8] = uctx->uc_mcontext.arm_r8; ctx.regs[UNW_ARM_R9] = uctx->uc_mcontext.arm_r9; ctx.regs[UNW_ARM_R10] = uctx->uc_mcontext.arm_r10; ctx.regs[UNW_ARM_R11] = uctx->uc_mcontext.arm_fp; ctx.regs[UNW_ARM_R12] = uctx->uc_mcontext.arm_ip; ctx.regs[UNW_ARM_R13] = uctx->uc_mcontext.arm_sp; ctx.regs[UNW_ARM_R14] = uctx->uc_mcontext.arm_lr; ctx.regs[UNW_ARM_R15] = uctx->uc_mcontext.arm_pc; // If we have crashed in the PC use the LR instead, as this was // a bad function dereference if (reinterpret_cast(error_addr()) == uctx->uc_mcontext.arm_pc) { ctx.regs[UNW_ARM_R15] = uctx->uc_mcontext.arm_lr - sizeof(unsigned long); } _stacktrace[index] = reinterpret_cast(ctx.regs[UNW_ARM_R15]); ++index; #elif defined(__APPLE__) && defined(__x86_64__) unw_getcontext(&ctx); // OS X's implementation of libunwind uses its own context object // so we need to convert the passed context to libunwind's format // (information about the data layout taken from unw_getcontext.s // in Apple's libunwind source ctx.data[0] = uctx->uc_mcontext->__ss.__rax; ctx.data[1] = uctx->uc_mcontext->__ss.__rbx; ctx.data[2] = uctx->uc_mcontext->__ss.__rcx; ctx.data[3] = uctx->uc_mcontext->__ss.__rdx; ctx.data[4] = uctx->uc_mcontext->__ss.__rdi; ctx.data[5] = uctx->uc_mcontext->__ss.__rsi; ctx.data[6] = uctx->uc_mcontext->__ss.__rbp; ctx.data[7] = uctx->uc_mcontext->__ss.__rsp; ctx.data[8] = uctx->uc_mcontext->__ss.__r8; ctx.data[9] = uctx->uc_mcontext->__ss.__r9; ctx.data[10] = uctx->uc_mcontext->__ss.__r10; ctx.data[11] = uctx->uc_mcontext->__ss.__r11; ctx.data[12] = uctx->uc_mcontext->__ss.__r12; ctx.data[13] = uctx->uc_mcontext->__ss.__r13; ctx.data[14] = uctx->uc_mcontext->__ss.__r14; ctx.data[15] = uctx->uc_mcontext->__ss.__r15; ctx.data[16] = uctx->uc_mcontext->__ss.__rip; // If the IP is the same as the crash address we have a bad function // dereference The caller's address is pointed to by %rsp, so we // dereference that value and set it to be the next frame's IP. if (uctx->uc_mcontext->__ss.__rip == reinterpret_cast<__uint64_t>(error_addr())) { ctx.data[16] = *reinterpret_cast<__uint64_t *>(uctx->uc_mcontext->__ss.__rsp); } _stacktrace[index] = reinterpret_cast(ctx.data[16]); ++index; #elif defined(__APPLE__) unw_getcontext(&ctx) // TODO: Convert the ucontext_t to libunwind's unw_context_t like // we do in 64 bits if (ctx.uc_mcontext->__ss.__eip == reinterpret_cast(error_addr())) { ctx.uc_mcontext->__ss.__eip = ctx.uc_mcontext->__ss.__esp; } _stacktrace[index] = reinterpret_cast(ctx.uc_mcontext->__ss.__eip); ++index; #endif } unw_cursor_t cursor; if (context()) { #if defined(UNW_INIT_SIGNAL_FRAME) result = unw_init_local2(&cursor, &ctx, UNW_INIT_SIGNAL_FRAME); #else result = unw_init_local(&cursor, &ctx); #endif } else { unw_getcontext(&ctx); ; result = unw_init_local(&cursor, &ctx); } if (result != 0) return 1; unw_word_t ip = 0; while (index <= depth && unw_step(&cursor) > 0) { result = unw_get_reg(&cursor, UNW_REG_IP, &ip); if (result == 0) { _stacktrace[index] = reinterpret_cast(--ip); ++index; } } --index; _stacktrace.resize(index + 1); skip_n_firsts(0); return size(); } size_t load_from(void *addr, size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { load_here(depth + 8, context, error_addr); for (size_t i = 0; i < _stacktrace.size(); ++i) { if (_stacktrace[i] == addr) { skip_n_firsts(i); _stacktrace[i] = (void *)((uintptr_t)_stacktrace[i]); break; } } _stacktrace.resize(std::min(_stacktrace.size(), skip_n_firsts() + depth)); return size(); } }; #elif defined(BACKWARD_HAS_BACKTRACE) template <> class StackTraceImpl : public StackTraceImplHolder { public: NOINLINE size_t load_here(size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { set_context(context); set_error_addr(error_addr); load_thread_info(); if (depth == 0) { return 0; } _stacktrace.resize(depth + 1); size_t trace_cnt = backtrace(&_stacktrace[0], _stacktrace.size()); _stacktrace.resize(trace_cnt); skip_n_firsts(1); return size(); } size_t load_from(void *addr, size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { load_here(depth + 8, context, error_addr); for (size_t i = 0; i < _stacktrace.size(); ++i) { if (_stacktrace[i] == addr) { skip_n_firsts(i); _stacktrace[i] = (void *)((uintptr_t)_stacktrace[i] + 1); break; } } _stacktrace.resize(std::min(_stacktrace.size(), skip_n_firsts() + depth)); return size(); } }; #elif defined(BACKWARD_SYSTEM_WINDOWS) template <> class StackTraceImpl : public StackTraceImplHolder { public: // We have to load the machine type from the image info // So we first initialize the resolver, and it tells us this info void set_machine_type(DWORD machine_type) { machine_type_ = machine_type; } void set_context(CONTEXT *ctx) { ctx_ = ctx; } void set_thread_handle(HANDLE handle) { thd_ = handle; } NOINLINE size_t load_here(size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { set_context(static_cast(context)); set_error_addr(error_addr); CONTEXT localCtx; // used when no context is provided if (depth == 0) { return 0; } if (!ctx_) { ctx_ = &localCtx; RtlCaptureContext(ctx_); } if (!thd_) { thd_ = GetCurrentThread(); } HANDLE process = GetCurrentProcess(); STACKFRAME64 s; memset(&s, 0, sizeof(STACKFRAME64)); // TODO: 32 bit context capture s.AddrStack.Mode = AddrModeFlat; s.AddrFrame.Mode = AddrModeFlat; s.AddrPC.Mode = AddrModeFlat; #ifdef _M_X64 s.AddrPC.Offset = ctx_->Rip; s.AddrStack.Offset = ctx_->Rsp; s.AddrFrame.Offset = ctx_->Rbp; #else s.AddrPC.Offset = ctx_->Eip; s.AddrStack.Offset = ctx_->Esp; s.AddrFrame.Offset = ctx_->Ebp; #endif if (!machine_type_) { #ifdef _M_X64 machine_type_ = IMAGE_FILE_MACHINE_AMD64; #else machine_type_ = IMAGE_FILE_MACHINE_I386; #endif } for (;;) { // NOTE: this only works if PDBs are already loaded! SetLastError(0); if (!StackWalk64(machine_type_, process, thd_, &s, ctx_, NULL, SymFunctionTableAccess64, SymGetModuleBase64, NULL)) break; if (s.AddrReturn.Offset == 0) break; _stacktrace.push_back(reinterpret_cast(s.AddrPC.Offset)); if (size() >= depth) break; } return size(); } size_t load_from(void *addr, size_t depth = 32, void *context = nullptr, void *error_addr = nullptr) { load_here(depth + 8, context, error_addr); for (size_t i = 0; i < _stacktrace.size(); ++i) { if (_stacktrace[i] == addr) { skip_n_firsts(i); break; } } _stacktrace.resize(std::min(_stacktrace.size(), skip_n_firsts() + depth)); return size(); } private: DWORD machine_type_ = 0; HANDLE thd_ = 0; CONTEXT *ctx_ = nullptr; }; #endif class StackTrace : public StackTraceImpl {}; /*************** TRACE RESOLVER ***************/ class TraceResolverImplBase { public: virtual ~TraceResolverImplBase() {} virtual void load_addresses(void *const*addresses, int address_count) { (void)addresses; (void)address_count; } template void load_stacktrace(ST &st) { load_addresses(st.begin(), static_cast(st.size())); } virtual ResolvedTrace resolve(ResolvedTrace t) { return t; } protected: std::string demangle(const char *funcname) { return _demangler.demangle(funcname); } private: details::demangler _demangler; }; template class TraceResolverImpl; #ifdef BACKWARD_SYSTEM_UNKNOWN template <> class TraceResolverImpl : public TraceResolverImplBase {}; #endif #ifdef BACKWARD_SYSTEM_LINUX class TraceResolverLinuxBase : public TraceResolverImplBase { public: TraceResolverLinuxBase() : argv0_(get_argv0()), exec_path_(read_symlink("/proc/self/exe")) {} std::string resolve_exec_path(Dl_info &symbol_info) const { // mutates symbol_info.dli_fname to be filename to open and returns filename // to display if (symbol_info.dli_fname == argv0_) { // dladdr returns argv[0] in dli_fname for symbols contained in // the main executable, which is not a valid path if the // executable was found by a search of the PATH environment // variable; In that case, we actually open /proc/self/exe, which // is always the actual executable (even if it was deleted/replaced!) // but display the path that /proc/self/exe links to. // However, this right away reduces probability of successful symbol // resolution, because libbfd may try to find *.debug files in the // same dir, in case symbols are stripped. As a result, it may try // to find a file /proc/self/.debug, which obviously does // not exist. /proc/self/exe is a last resort. First load attempt // should go for the original executable file path. symbol_info.dli_fname = "/proc/self/exe"; return exec_path_; } else { return symbol_info.dli_fname; } } private: std::string argv0_; std::string exec_path_; static std::string get_argv0() { std::string argv0; std::ifstream ifs("/proc/self/cmdline"); std::getline(ifs, argv0, '\0'); return argv0; } static std::string read_symlink(std::string const &symlink_path) { std::string path; path.resize(100); while (true) { ssize_t len = ::readlink(symlink_path.c_str(), &*path.begin(), path.size()); if (len < 0) { return ""; } if (static_cast(len) == path.size()) { path.resize(path.size() * 2); } else { path.resize(static_cast(len)); break; } } return path; } }; template class TraceResolverLinuxImpl; #if BACKWARD_HAS_BACKTRACE_SYMBOL == 1 template <> class TraceResolverLinuxImpl : public TraceResolverLinuxBase { public: void load_addresses(void *const*addresses, int address_count) override { if (address_count == 0) { return; } _symbols.reset(backtrace_symbols(addresses, address_count)); } ResolvedTrace resolve(ResolvedTrace trace) override { char *filename = _symbols[trace.idx]; char *funcname = filename; while (*funcname && *funcname != '(') { funcname += 1; } trace.object_filename.assign(filename, funcname); // ok even if funcname is the ending // \0 (then we assign entire string) if (*funcname) { // if it's not end of string (e.g. from last frame ip==0) funcname += 1; char *funcname_end = funcname; while (*funcname_end && *funcname_end != ')' && *funcname_end != '+') { funcname_end += 1; } *funcname_end = '\0'; trace.object_function = this->demangle(funcname); trace.source.function = trace.object_function; // we cannot do better. } return trace; } private: details::handle _symbols; }; #endif // BACKWARD_HAS_BACKTRACE_SYMBOL == 1 #if BACKWARD_HAS_BFD == 1 template <> class TraceResolverLinuxImpl : public TraceResolverLinuxBase { public: TraceResolverLinuxImpl() : _bfd_loaded(false) {} ResolvedTrace resolve(ResolvedTrace trace) override { Dl_info symbol_info; // trace.addr is a virtual address in memory pointing to some code. // Let's try to find from which loaded object it comes from. // The loaded object can be yourself btw. if (!dladdr(trace.addr, &symbol_info)) { return trace; // dat broken trace... } // Now we get in symbol_info: // .dli_fname: // pathname of the shared object that contains the address. // .dli_fbase: // where the object is loaded in memory. // .dli_sname: // the name of the nearest symbol to trace.addr, we expect a // function name. // .dli_saddr: // the exact address corresponding to .dli_sname. if (symbol_info.dli_sname) { trace.object_function = demangle(symbol_info.dli_sname); } if (!symbol_info.dli_fname) { return trace; } trace.object_filename = resolve_exec_path(symbol_info); bfd_fileobject *fobj; // Before rushing to resolution need to ensure the executable // file still can be used. For that compare inode numbers of // what is stored by the executable's file path, and in the // dli_fname, which not necessarily equals to the executable. // It can be a shared library, or /proc/self/exe, and in the // latter case has drawbacks. See the exec path resolution for // details. In short - the dli object should be used only as // the last resort. // If inode numbers are equal, it is known dli_fname and the // executable file are the same. This is guaranteed by Linux, // because if the executable file is changed/deleted, it will // be done in a new inode. The old file will be preserved in // /proc/self/exe, and may even have inode 0. The latter can // happen if the inode was actually reused, and the file was // kept only in the main memory. // struct stat obj_stat; struct stat dli_stat; if (stat(trace.object_filename.c_str(), &obj_stat) == 0 && stat(symbol_info.dli_fname, &dli_stat) == 0 && obj_stat.st_ino == dli_stat.st_ino) { // The executable file, and the shared object containing the // address are the same file. Safe to use the original path. // this is preferable. Libbfd will search for stripped debug // symbols in the same directory. fobj = load_object_with_bfd(trace.object_filename); } else{ // The original object file was *deleted*! The only hope is // that the debug symbols are either inside the shared // object file, or are in the same directory, and this is // not /proc/self/exe. fobj = nullptr; } if (fobj == nullptr || !fobj->handle) { fobj = load_object_with_bfd(symbol_info.dli_fname); if (!fobj->handle) { return trace; } } find_sym_result *details_selected; // to be filled. // trace.addr is the next instruction to be executed after returning // from the nested stack frame. In C++ this usually relate to the next // statement right after the function call that leaded to a new stack // frame. This is not usually what you want to see when printing out a // stacktrace... find_sym_result details_call_site = find_symbol_details(fobj, trace.addr, symbol_info.dli_fbase); details_selected = &details_call_site; #if BACKWARD_HAS_UNWIND == 0 // ...this is why we also try to resolve the symbol that is right // before the return address. If we are lucky enough, we will get the // line of the function that was called. But if the code is optimized, // we might get something absolutely not related since the compiler // can reschedule the return address with inline functions and // tail-call optimization (among other things that I don't even know // or cannot even dream about with my tiny limited brain). find_sym_result details_adjusted_call_site = find_symbol_details( fobj, (void *)(uintptr_t(trace.addr) - 1), symbol_info.dli_fbase); // In debug mode, we should always get the right thing(TM). if (details_call_site.found && details_adjusted_call_site.found) { // Ok, we assume that details_adjusted_call_site is a better estimation. details_selected = &details_adjusted_call_site; trace.addr = (void *)(uintptr_t(trace.addr) - 1); } if (details_selected == &details_call_site && details_call_site.found) { // we have to re-resolve the symbol in order to reset some // internal state in BFD... so we can call backtrace_inliners // thereafter... details_call_site = find_symbol_details(fobj, trace.addr, symbol_info.dli_fbase); } #endif // BACKWARD_HAS_UNWIND if (details_selected->found) { if (details_selected->filename) { trace.source.filename = details_selected->filename; } trace.source.line = details_selected->line; if (details_selected->funcname) { // this time we get the name of the function where the code is // located, instead of the function were the address is // located. In short, if the code was inlined, we get the // function corresponding to the code. Else we already got in // trace.function. trace.source.function = demangle(details_selected->funcname); if (!symbol_info.dli_sname) { // for the case dladdr failed to find the symbol name of // the function, we might as well try to put something // here. trace.object_function = trace.source.function; } } // Maybe the source of the trace got inlined inside the function // (trace.source.function). Let's see if we can get all the inlined // calls along the way up to the initial call site. trace.inliners = backtrace_inliners(fobj, *details_selected); #if 0 if (trace.inliners.size() == 0) { // Maybe the trace was not inlined... or maybe it was and we // are lacking the debug information. Let's try to make the // world better and see if we can get the line number of the // function (trace.source.function) now. // // We will get the location of where the function start (to be // exact: the first instruction that really start the // function), not where the name of the function is defined. // This can be quite far away from the name of the function // btw. // // If the source of the function is the same as the source of // the trace, we cannot say if the trace was really inlined or // not. However, if the filename of the source is different // between the function and the trace... we can declare it as // an inliner. This is not 100% accurate, but better than // nothing. if (symbol_info.dli_saddr) { find_sym_result details = find_symbol_details(fobj, symbol_info.dli_saddr, symbol_info.dli_fbase); if (details.found) { ResolvedTrace::SourceLoc diy_inliner; diy_inliner.line = details.line; if (details.filename) { diy_inliner.filename = details.filename; } if (details.funcname) { diy_inliner.function = demangle(details.funcname); } else { diy_inliner.function = trace.source.function; } if (diy_inliner != trace.source) { trace.inliners.push_back(diy_inliner); } } } } #endif } return trace; } private: bool _bfd_loaded; typedef details::handle > bfd_handle_t; typedef details::handle bfd_symtab_t; struct bfd_fileobject { bfd_handle_t handle; bfd_vma base_addr; bfd_symtab_t symtab; bfd_symtab_t dynamic_symtab; }; typedef details::hashtable::type fobj_bfd_map_t; fobj_bfd_map_t _fobj_bfd_map; bfd_fileobject *load_object_with_bfd(const std::string &filename_object) { using namespace details; if (!_bfd_loaded) { using namespace details; bfd_init(); _bfd_loaded = true; } fobj_bfd_map_t::iterator it = _fobj_bfd_map.find(filename_object); if (it != _fobj_bfd_map.end()) { return &it->second; } // this new object is empty for now. bfd_fileobject *r = &_fobj_bfd_map[filename_object]; // we do the work temporary in this one; bfd_handle_t bfd_handle; int fd = open(filename_object.c_str(), O_RDONLY); bfd_handle.reset(bfd_fdopenr(filename_object.c_str(), "default", fd)); if (!bfd_handle) { close(fd); return r; } if (!bfd_check_format(bfd_handle.get(), bfd_object)) { return r; // not an object? You lose. } if ((bfd_get_file_flags(bfd_handle.get()) & HAS_SYMS) == 0) { return r; // that's what happen when you forget to compile in debug. } ssize_t symtab_storage_size = bfd_get_symtab_upper_bound(bfd_handle.get()); ssize_t dyn_symtab_storage_size = bfd_get_dynamic_symtab_upper_bound(bfd_handle.get()); if (symtab_storage_size <= 0 && dyn_symtab_storage_size <= 0) { return r; // weird, is the file is corrupted? } bfd_symtab_t symtab, dynamic_symtab; ssize_t symcount = 0, dyn_symcount = 0; if (symtab_storage_size > 0) { symtab.reset(static_cast( malloc(static_cast(symtab_storage_size)))); symcount = bfd_canonicalize_symtab(bfd_handle.get(), symtab.get()); } if (dyn_symtab_storage_size > 0) { dynamic_symtab.reset(static_cast( malloc(static_cast(dyn_symtab_storage_size)))); dyn_symcount = bfd_canonicalize_dynamic_symtab(bfd_handle.get(), dynamic_symtab.get()); } if (symcount <= 0 && dyn_symcount <= 0) { return r; // damned, that's a stripped file that you got there! } r->handle = move(bfd_handle); r->symtab = move(symtab); r->dynamic_symtab = move(dynamic_symtab); return r; } struct find_sym_result { bool found; const char *filename; const char *funcname; unsigned int line; }; struct find_sym_context { TraceResolverLinuxImpl *self; bfd_fileobject *fobj; void *addr; void *base_addr; find_sym_result result; }; find_sym_result find_symbol_details(bfd_fileobject *fobj, void *addr, void *base_addr) { find_sym_context context; context.self = this; context.fobj = fobj; context.addr = addr; context.base_addr = base_addr; context.result.found = false; bfd_map_over_sections(fobj->handle.get(), &find_in_section_trampoline, static_cast(&context)); return context.result; } static void find_in_section_trampoline(bfd *, asection *section, void *data) { find_sym_context *context = static_cast(data); context->self->find_in_section( reinterpret_cast(context->addr), reinterpret_cast(context->base_addr), context->fobj, section, context->result); } void find_in_section(bfd_vma addr, bfd_vma base_addr, bfd_fileobject *fobj, asection *section, find_sym_result &result) { if (result.found) return; #ifdef bfd_get_section_flags if ((bfd_get_section_flags(fobj->handle.get(), section) & SEC_ALLOC) == 0) #else if ((bfd_section_flags(section) & SEC_ALLOC) == 0) #endif return; // a debug section is never loaded automatically. #ifdef bfd_get_section_vma bfd_vma sec_addr = bfd_get_section_vma(fobj->handle.get(), section); #else bfd_vma sec_addr = bfd_section_vma(section); #endif #ifdef bfd_get_section_size bfd_size_type size = bfd_get_section_size(section); #else bfd_size_type size = bfd_section_size(section); #endif // are we in the boundaries of the section? if (addr < sec_addr || addr >= sec_addr + size) { addr -= base_addr; // oops, a relocated object, lets try again... if (addr < sec_addr || addr >= sec_addr + size) { return; } } #if defined(__clang__) #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wzero-as-null-pointer-constant" #endif if (!result.found && fobj->symtab) { result.found = bfd_find_nearest_line( fobj->handle.get(), section, fobj->symtab.get(), addr - sec_addr, &result.filename, &result.funcname, &result.line); } if (!result.found && fobj->dynamic_symtab) { result.found = bfd_find_nearest_line( fobj->handle.get(), section, fobj->dynamic_symtab.get(), addr - sec_addr, &result.filename, &result.funcname, &result.line); } #if defined(__clang__) #pragma clang diagnostic pop #endif } ResolvedTrace::source_locs_t backtrace_inliners(bfd_fileobject *fobj, find_sym_result previous_result) { // This function can be called ONLY after a SUCCESSFUL call to // find_symbol_details. The state is global to the bfd_handle. ResolvedTrace::source_locs_t results; while (previous_result.found) { find_sym_result result; result.found = bfd_find_inliner_info(fobj->handle.get(), &result.filename, &result.funcname, &result.line); if (result .found) /* and not ( cstrings_eq(previous_result.filename, result.filename) and cstrings_eq(previous_result.funcname, result.funcname) and result.line == previous_result.line )) */ { ResolvedTrace::SourceLoc src_loc; src_loc.line = result.line; if (result.filename) { src_loc.filename = result.filename; } if (result.funcname) { src_loc.function = demangle(result.funcname); } results.push_back(src_loc); } previous_result = result; } return results; } bool cstrings_eq(const char *a, const char *b) { if (!a || !b) { return false; } return strcmp(a, b) == 0; } }; #endif // BACKWARD_HAS_BFD == 1 #if BACKWARD_HAS_DW == 1 template <> class TraceResolverLinuxImpl : public TraceResolverLinuxBase { public: TraceResolverLinuxImpl() : _dwfl_handle_initialized(false) {} ResolvedTrace resolve(ResolvedTrace trace) override { using namespace details; Dwarf_Addr trace_addr = reinterpret_cast(trace.addr); if (!_dwfl_handle_initialized) { // initialize dwfl... _dwfl_cb.reset(new Dwfl_Callbacks); _dwfl_cb->find_elf = &dwfl_linux_proc_find_elf; _dwfl_cb->find_debuginfo = &dwfl_standard_find_debuginfo; _dwfl_cb->debuginfo_path = 0; _dwfl_handle.reset(dwfl_begin(_dwfl_cb.get())); _dwfl_handle_initialized = true; if (!_dwfl_handle) { return trace; } // ...from the current process. dwfl_report_begin(_dwfl_handle.get()); int r = dwfl_linux_proc_report(_dwfl_handle.get(), getpid()); dwfl_report_end(_dwfl_handle.get(), NULL, NULL); if (r < 0) { return trace; } } if (!_dwfl_handle) { return trace; } // find the module (binary object) that contains the trace's address. // This is not using any debug information, but the addresses ranges of // all the currently loaded binary object. Dwfl_Module *mod = dwfl_addrmodule(_dwfl_handle.get(), trace_addr); if (mod) { // now that we found it, lets get the name of it, this will be the // full path to the running binary or one of the loaded library. const char *module_name = dwfl_module_info(mod, 0, 0, 0, 0, 0, 0, 0); if (module_name) { trace.object_filename = module_name; } // We also look after the name of the symbol, equal or before this // address. This is found by walking the symtab. We should get the // symbol corresponding to the function (mangled) containing the // address. If the code corresponding to the address was inlined, // this is the name of the out-most inliner function. const char *sym_name = dwfl_module_addrname(mod, trace_addr); if (sym_name) { trace.object_function = demangle(sym_name); } } // now let's get serious, and find out the source location (file and // line number) of the address. // This function will look in .debug_aranges for the address and map it // to the location of the compilation unit DIE in .debug_info and // return it. Dwarf_Addr mod_bias = 0; Dwarf_Die *cudie = dwfl_module_addrdie(mod, trace_addr, &mod_bias); #if 1 if (!cudie) { // Sadly clang does not generate the section .debug_aranges, thus // dwfl_module_addrdie will fail early. Clang doesn't either set // the lowpc/highpc/range info for every compilation unit. // // So in order to save the world: // for every compilation unit, we will iterate over every single // DIEs. Normally functions should have a lowpc/highpc/range, which // we will use to infer the compilation unit. // note that this is probably badly inefficient. while ((cudie = dwfl_module_nextcu(mod, cudie, &mod_bias))) { Dwarf_Die die_mem; Dwarf_Die *fundie = find_fundie_by_pc(cudie, trace_addr - mod_bias, &die_mem); if (fundie) { break; } } } #endif //#define BACKWARD_I_DO_NOT_RECOMMEND_TO_ENABLE_THIS_HORRIBLE_PIECE_OF_CODE #ifdef BACKWARD_I_DO_NOT_RECOMMEND_TO_ENABLE_THIS_HORRIBLE_PIECE_OF_CODE if (!cudie) { // If it's still not enough, lets dive deeper in the shit, and try // to save the world again: for every compilation unit, we will // load the corresponding .debug_line section, and see if we can // find our address in it. Dwarf_Addr cfi_bias; Dwarf_CFI *cfi_cache = dwfl_module_eh_cfi(mod, &cfi_bias); Dwarf_Addr bias; while ((cudie = dwfl_module_nextcu(mod, cudie, &bias))) { if (dwarf_getsrc_die(cudie, trace_addr - bias)) { // ...but if we get a match, it might be a false positive // because our (address - bias) might as well be valid in a // different compilation unit. So we throw our last card on // the table and lookup for the address into the .eh_frame // section. handle frame; dwarf_cfi_addrframe(cfi_cache, trace_addr - cfi_bias, &frame); if (frame) { break; } } } } #endif if (!cudie) { return trace; // this time we lost the game :/ } // Now that we have a compilation unit DIE, this function will be able // to load the corresponding section in .debug_line (if not already // loaded) and hopefully find the source location mapped to our // address. Dwarf_Line *srcloc = dwarf_getsrc_die(cudie, trace_addr - mod_bias); if (srcloc) { const char *srcfile = dwarf_linesrc(srcloc, 0, 0); if (srcfile) { trace.source.filename = srcfile; } int line = 0, col = 0; dwarf_lineno(srcloc, &line); dwarf_linecol(srcloc, &col); trace.source.line = static_cast(line); trace.source.col = static_cast(col); } deep_first_search_by_pc(cudie, trace_addr - mod_bias, inliners_search_cb(trace)); if (trace.source.function.size() == 0) { // fallback. trace.source.function = trace.object_function; } return trace; } private: typedef details::handle > dwfl_handle_t; details::handle > _dwfl_cb; dwfl_handle_t _dwfl_handle; bool _dwfl_handle_initialized; // defined here because in C++98, template function cannot take locally // defined types... grrr. struct inliners_search_cb { void operator()(Dwarf_Die *die) { switch (dwarf_tag(die)) { const char *name; case DW_TAG_subprogram: if ((name = dwarf_diename(die))) { trace.source.function = name; } break; case DW_TAG_inlined_subroutine: ResolvedTrace::SourceLoc sloc; Dwarf_Attribute attr_mem; if ((name = dwarf_diename(die))) { sloc.function = name; } if ((name = die_call_file(die))) { sloc.filename = name; } Dwarf_Word line = 0, col = 0; dwarf_formudata(dwarf_attr(die, DW_AT_call_line, &attr_mem), &line); dwarf_formudata(dwarf_attr(die, DW_AT_call_column, &attr_mem), &col); sloc.line = static_cast(line); sloc.col = static_cast(col); trace.inliners.push_back(sloc); break; }; } ResolvedTrace &trace; inliners_search_cb(ResolvedTrace &t) : trace(t) {} }; static bool die_has_pc(Dwarf_Die *die, Dwarf_Addr pc) { Dwarf_Addr low, high; // continuous range if (dwarf_hasattr(die, DW_AT_low_pc) && dwarf_hasattr(die, DW_AT_high_pc)) { if (dwarf_lowpc(die, &low) != 0) { return false; } if (dwarf_highpc(die, &high) != 0) { Dwarf_Attribute attr_mem; Dwarf_Attribute *attr = dwarf_attr(die, DW_AT_high_pc, &attr_mem); Dwarf_Word value; if (dwarf_formudata(attr, &value) != 0) { return false; } high = low + value; } return pc >= low && pc < high; } // non-continuous range. Dwarf_Addr base; ptrdiff_t offset = 0; while ((offset = dwarf_ranges(die, offset, &base, &low, &high)) > 0) { if (pc >= low && pc < high) { return true; } } return false; } static Dwarf_Die *find_fundie_by_pc(Dwarf_Die *parent_die, Dwarf_Addr pc, Dwarf_Die *result) { if (dwarf_child(parent_die, result) != 0) { return 0; } Dwarf_Die *die = result; do { switch (dwarf_tag(die)) { case DW_TAG_subprogram: case DW_TAG_inlined_subroutine: if (die_has_pc(die, pc)) { return result; } }; bool declaration = false; Dwarf_Attribute attr_mem; dwarf_formflag(dwarf_attr(die, DW_AT_declaration, &attr_mem), &declaration); if (!declaration) { // let's be curious and look deeper in the tree, // function are not necessarily at the first level, but // might be nested inside a namespace, structure etc. Dwarf_Die die_mem; Dwarf_Die *indie = find_fundie_by_pc(die, pc, &die_mem); if (indie) { *result = die_mem; return result; } } } while (dwarf_siblingof(die, result) == 0); return 0; } template static bool deep_first_search_by_pc(Dwarf_Die *parent_die, Dwarf_Addr pc, CB cb) { Dwarf_Die die_mem; if (dwarf_child(parent_die, &die_mem) != 0) { return false; } bool branch_has_pc = false; Dwarf_Die *die = &die_mem; do { bool declaration = false; Dwarf_Attribute attr_mem; dwarf_formflag(dwarf_attr(die, DW_AT_declaration, &attr_mem), &declaration); if (!declaration) { // let's be curious and look deeper in the tree, function are // not necessarily at the first level, but might be nested // inside a namespace, structure, a function, an inlined // function etc. branch_has_pc = deep_first_search_by_pc(die, pc, cb); } if (!branch_has_pc) { branch_has_pc = die_has_pc(die, pc); } if (branch_has_pc) { cb(die); } } while (dwarf_siblingof(die, &die_mem) == 0); return branch_has_pc; } static const char *die_call_file(Dwarf_Die *die) { Dwarf_Attribute attr_mem; Dwarf_Word file_idx = 0; dwarf_formudata(dwarf_attr(die, DW_AT_call_file, &attr_mem), &file_idx); if (file_idx == 0) { return 0; } Dwarf_Die die_mem; Dwarf_Die *cudie = dwarf_diecu(die, &die_mem, 0, 0); if (!cudie) { return 0; } Dwarf_Files *files = 0; size_t nfiles; dwarf_getsrcfiles(cudie, &files, &nfiles); if (!files) { return 0; } return dwarf_filesrc(files, file_idx, 0, 0); } }; #endif // BACKWARD_HAS_DW == 1 #if BACKWARD_HAS_DWARF == 1 template <> class TraceResolverLinuxImpl : public TraceResolverLinuxBase { public: TraceResolverLinuxImpl() : _dwarf_loaded(false) {} ResolvedTrace resolve(ResolvedTrace trace) override { // trace.addr is a virtual address in memory pointing to some code. // Let's try to find from which loaded object it comes from. // The loaded object can be yourself btw. Dl_info symbol_info; int dladdr_result = 0; #if defined(__GLIBC__) link_map *link_map; // We request the link map so we can get information about offsets dladdr_result = dladdr1(trace.addr, &symbol_info, reinterpret_cast(&link_map), RTLD_DL_LINKMAP); #else // Android doesn't have dladdr1. Don't use the linker map. dladdr_result = dladdr(trace.addr, &symbol_info); #endif if (!dladdr_result) { return trace; // dat broken trace... } // Now we get in symbol_info: // .dli_fname: // pathname of the shared object that contains the address. // .dli_fbase: // where the object is loaded in memory. // .dli_sname: // the name of the nearest symbol to trace.addr, we expect a // function name. // .dli_saddr: // the exact address corresponding to .dli_sname. // // And in link_map: // .l_addr: // difference between the address in the ELF file and the address // in memory // l_name: // absolute pathname where the object was found if (symbol_info.dli_sname) { trace.object_function = demangle(symbol_info.dli_sname); } if (!symbol_info.dli_fname) { return trace; } trace.object_filename = resolve_exec_path(symbol_info); dwarf_fileobject &fobj = load_object_with_dwarf(symbol_info.dli_fname); if (!fobj.dwarf_handle) { return trace; // sad, we couldn't load the object :( } #if defined(__GLIBC__) // Convert the address to a module relative one by looking at // the module's loading address in the link map Dwarf_Addr address = reinterpret_cast(trace.addr) - reinterpret_cast(link_map->l_addr); #else Dwarf_Addr address = reinterpret_cast(trace.addr); #endif if (trace.object_function.empty()) { symbol_cache_t::iterator it = fobj.symbol_cache.lower_bound(address); if (it != fobj.symbol_cache.end()) { if (it->first != address) { if (it != fobj.symbol_cache.begin()) { --it; } } trace.object_function = demangle(it->second.c_str()); } } // Get the Compilation Unit DIE for the address Dwarf_Die die = find_die(fobj, address); if (!die) { return trace; // this time we lost the game :/ } // libdwarf doesn't give us direct access to its objects, it always // allocates a copy for the caller. We keep that copy alive in a cache // and we deallocate it later when it's no longer required. die_cache_entry &die_object = get_die_cache(fobj, die); if (die_object.isEmpty()) return trace; // We have no line section for this DIE die_linemap_t::iterator it = die_object.line_section.lower_bound(address); if (it != die_object.line_section.end()) { if (it->first != address) { if (it == die_object.line_section.begin()) { // If we are on the first item of the line section // but the address does not match it means that // the address is below the range of the DIE. Give up. return trace; } else { --it; } } } else { return trace; // We didn't find the address. } // Get the Dwarf_Line that the address points to and call libdwarf // to get source file, line and column info. Dwarf_Line line = die_object.line_buffer[it->second]; Dwarf_Error error = DW_DLE_NE; char *filename; if (dwarf_linesrc(line, &filename, &error) == DW_DLV_OK) { trace.source.filename = std::string(filename); dwarf_dealloc(fobj.dwarf_handle.get(), filename, DW_DLA_STRING); } Dwarf_Unsigned number = 0; if (dwarf_lineno(line, &number, &error) == DW_DLV_OK) { trace.source.line = number; } else { trace.source.line = 0; } if (dwarf_lineoff_b(line, &number, &error) == DW_DLV_OK) { trace.source.col = number; } else { trace.source.col = 0; } std::vector namespace_stack; deep_first_search_by_pc(fobj, die, address, namespace_stack, inliners_search_cb(trace, fobj, die)); dwarf_dealloc(fobj.dwarf_handle.get(), die, DW_DLA_DIE); return trace; } public: static int close_dwarf(Dwarf_Debug dwarf) { return dwarf_finish(dwarf, NULL); } private: bool _dwarf_loaded; typedef details::handle > dwarf_file_t; typedef details::handle > dwarf_elf_t; typedef details::handle > dwarf_handle_t; typedef std::map die_linemap_t; typedef std::map die_specmap_t; struct die_cache_entry { die_specmap_t spec_section; die_linemap_t line_section; Dwarf_Line *line_buffer; Dwarf_Signed line_count; Dwarf_Line_Context line_context; inline bool isEmpty() { return line_buffer == NULL || line_count == 0 || line_context == NULL || line_section.empty(); } die_cache_entry() : line_buffer(0), line_count(0), line_context(0) {} ~die_cache_entry() { if (line_context) { dwarf_srclines_dealloc_b(line_context); } } }; typedef std::map die_cache_t; typedef std::map symbol_cache_t; struct dwarf_fileobject { dwarf_file_t file_handle; dwarf_elf_t elf_handle; dwarf_handle_t dwarf_handle; symbol_cache_t symbol_cache; // Die cache die_cache_t die_cache; die_cache_entry *current_cu; }; typedef details::hashtable::type fobj_dwarf_map_t; fobj_dwarf_map_t _fobj_dwarf_map; static bool cstrings_eq(const char *a, const char *b) { if (!a || !b) { return false; } return strcmp(a, b) == 0; } dwarf_fileobject &load_object_with_dwarf(const std::string &filename_object) { if (!_dwarf_loaded) { // Set the ELF library operating version // If that fails there's nothing we can do _dwarf_loaded = elf_version(EV_CURRENT) != EV_NONE; } fobj_dwarf_map_t::iterator it = _fobj_dwarf_map.find(filename_object); if (it != _fobj_dwarf_map.end()) { return it->second; } // this new object is empty for now dwarf_fileobject &r = _fobj_dwarf_map[filename_object]; dwarf_file_t file_handle; file_handle.reset(open(filename_object.c_str(), O_RDONLY)); if (file_handle.get() < 0) { return r; } // Try to get an ELF handle. We need to read the ELF sections // because we want to see if there is a .gnu_debuglink section // that points to a split debug file dwarf_elf_t elf_handle; elf_handle.reset(elf_begin(file_handle.get(), ELF_C_READ, NULL)); if (!elf_handle) { return r; } const char *e_ident = elf_getident(elf_handle.get(), 0); if (!e_ident) { return r; } // Get the number of sections // We use the new APIs as elf_getshnum is deprecated size_t shdrnum = 0; if (elf_getshdrnum(elf_handle.get(), &shdrnum) == -1) { return r; } // Get the index to the string section size_t shdrstrndx = 0; if (elf_getshdrstrndx(elf_handle.get(), &shdrstrndx) == -1) { return r; } std::string debuglink; // Iterate through the ELF sections to try to get a gnu_debuglink // note and also to cache the symbol table. // We go the preprocessor way to avoid having to create templated // classes or using gelf (which might throw a compiler error if 64 bit // is not supported #define ELF_GET_DATA(ARCH) \ Elf_Scn *elf_section = 0; \ Elf_Data *elf_data = 0; \ Elf##ARCH##_Shdr *section_header = 0; \ Elf_Scn *symbol_section = 0; \ size_t symbol_count = 0; \ size_t symbol_strings = 0; \ Elf##ARCH##_Sym *symbol = 0; \ const char *section_name = 0; \ \ while ((elf_section = elf_nextscn(elf_handle.get(), elf_section)) != NULL) { \ section_header = elf##ARCH##_getshdr(elf_section); \ if (section_header == NULL) { \ return r; \ } \ \ if ((section_name = elf_strptr(elf_handle.get(), shdrstrndx, \ section_header->sh_name)) == NULL) { \ return r; \ } \ \ if (cstrings_eq(section_name, ".gnu_debuglink")) { \ elf_data = elf_getdata(elf_section, NULL); \ if (elf_data && elf_data->d_size > 0) { \ debuglink = \ std::string(reinterpret_cast(elf_data->d_buf)); \ } \ } \ \ switch (section_header->sh_type) { \ case SHT_SYMTAB: \ symbol_section = elf_section; \ symbol_count = section_header->sh_size / section_header->sh_entsize; \ symbol_strings = section_header->sh_link; \ break; \ \ /* We use .dynsyms as a last resort, we prefer .symtab */ \ case SHT_DYNSYM: \ if (!symbol_section) { \ symbol_section = elf_section; \ symbol_count = section_header->sh_size / section_header->sh_entsize; \ symbol_strings = section_header->sh_link; \ } \ break; \ } \ } \ \ if (symbol_section && symbol_count && symbol_strings) { \ elf_data = elf_getdata(symbol_section, NULL); \ symbol = reinterpret_cast(elf_data->d_buf); \ for (size_t i = 0; i < symbol_count; ++i) { \ int type = ELF##ARCH##_ST_TYPE(symbol->st_info); \ if (type == STT_FUNC && symbol->st_value > 0) { \ r.symbol_cache[symbol->st_value] = std::string( \ elf_strptr(elf_handle.get(), symbol_strings, symbol->st_name)); \ } \ ++symbol; \ } \ } if (e_ident[EI_CLASS] == ELFCLASS32) { ELF_GET_DATA(32) } else if (e_ident[EI_CLASS] == ELFCLASS64) { // libelf might have been built without 64 bit support #if __LIBELF64 ELF_GET_DATA(64) #endif } if (!debuglink.empty()) { // We have a debuglink section! Open an elf instance on that // file instead. If we can't open the file, then return // the elf handle we had already opened. dwarf_file_t debuglink_file; debuglink_file.reset(open(debuglink.c_str(), O_RDONLY)); if (debuglink_file.get() > 0) { dwarf_elf_t debuglink_elf; debuglink_elf.reset(elf_begin(debuglink_file.get(), ELF_C_READ, NULL)); // If we have a valid elf handle, return the new elf handle // and file handle and discard the original ones if (debuglink_elf) { elf_handle = move(debuglink_elf); file_handle = move(debuglink_file); } } } // Ok, we have a valid ELF handle, let's try to get debug symbols Dwarf_Debug dwarf_debug; Dwarf_Error error = DW_DLE_NE; dwarf_handle_t dwarf_handle; int dwarf_result = dwarf_elf_init(elf_handle.get(), DW_DLC_READ, NULL, NULL, &dwarf_debug, &error); // We don't do any special handling for DW_DLV_NO_ENTRY specially. // If we get an error, or the file doesn't have debug information // we just return. if (dwarf_result != DW_DLV_OK) { return r; } dwarf_handle.reset(dwarf_debug); r.file_handle = move(file_handle); r.elf_handle = move(elf_handle); r.dwarf_handle = move(dwarf_handle); return r; } die_cache_entry &get_die_cache(dwarf_fileobject &fobj, Dwarf_Die die) { Dwarf_Error error = DW_DLE_NE; // Get the die offset, we use it as the cache key Dwarf_Off die_offset; if (dwarf_dieoffset(die, &die_offset, &error) != DW_DLV_OK) { die_offset = 0; } die_cache_t::iterator it = fobj.die_cache.find(die_offset); if (it != fobj.die_cache.end()) { fobj.current_cu = &it->second; return it->second; } die_cache_entry &de = fobj.die_cache[die_offset]; fobj.current_cu = &de; Dwarf_Addr line_addr; Dwarf_Small table_count; // The addresses in the line section are not fully sorted (they might // be sorted by block of code belonging to the same file), which makes // it necessary to do so before searching is possible. // // As libdwarf allocates a copy of everything, let's get the contents // of the line section and keep it around. We also create a map of // program counter to line table indices so we can search by address // and get the line buffer index. // // To make things more difficult, the same address can span more than // one line, so we need to keep the index pointing to the first line // by using insert instead of the map's [ operator. // Get the line context for the DIE if (dwarf_srclines_b(die, 0, &table_count, &de.line_context, &error) == DW_DLV_OK) { // Get the source lines for this line context, to be deallocated // later if (dwarf_srclines_from_linecontext(de.line_context, &de.line_buffer, &de.line_count, &error) == DW_DLV_OK) { // Add all the addresses to our map for (int i = 0; i < de.line_count; i++) { if (dwarf_lineaddr(de.line_buffer[i], &line_addr, &error) != DW_DLV_OK) { line_addr = 0; } de.line_section.insert(std::pair(line_addr, i)); } } } // For each CU, cache the function DIEs that contain the // DW_AT_specification attribute. When building with -g3 the function // DIEs are separated in declaration and specification, with the // declaration containing only the name and parameters and the // specification the low/high pc and other compiler attributes. // // We cache those specifications so we don't skip over the declarations, // because they have no pc, and we can do namespace resolution for // DWARF function names. Dwarf_Debug dwarf = fobj.dwarf_handle.get(); Dwarf_Die current_die = 0; if (dwarf_child(die, ¤t_die, &error) == DW_DLV_OK) { for (;;) { Dwarf_Die sibling_die = 0; Dwarf_Half tag_value; dwarf_tag(current_die, &tag_value, &error); if (tag_value == DW_TAG_subprogram || tag_value == DW_TAG_inlined_subroutine) { Dwarf_Bool has_attr = 0; if (dwarf_hasattr(current_die, DW_AT_specification, &has_attr, &error) == DW_DLV_OK) { if (has_attr) { Dwarf_Attribute attr_mem; if (dwarf_attr(current_die, DW_AT_specification, &attr_mem, &error) == DW_DLV_OK) { Dwarf_Off spec_offset = 0; if (dwarf_formref(attr_mem, &spec_offset, &error) == DW_DLV_OK) { Dwarf_Off spec_die_offset; if (dwarf_dieoffset(current_die, &spec_die_offset, &error) == DW_DLV_OK) { de.spec_section[spec_offset] = spec_die_offset; } } } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } } } int result = dwarf_siblingof(dwarf, current_die, &sibling_die, &error); if (result == DW_DLV_ERROR) { break; } else if (result == DW_DLV_NO_ENTRY) { break; } if (current_die != die) { dwarf_dealloc(dwarf, current_die, DW_DLA_DIE); current_die = 0; } current_die = sibling_die; } } return de; } static Dwarf_Die get_referenced_die(Dwarf_Debug dwarf, Dwarf_Die die, Dwarf_Half attr, bool global) { Dwarf_Error error = DW_DLE_NE; Dwarf_Attribute attr_mem; Dwarf_Die found_die = NULL; if (dwarf_attr(die, attr, &attr_mem, &error) == DW_DLV_OK) { Dwarf_Off offset; int result = 0; if (global) { result = dwarf_global_formref(attr_mem, &offset, &error); } else { result = dwarf_formref(attr_mem, &offset, &error); } if (result == DW_DLV_OK) { if (dwarf_offdie(dwarf, offset, &found_die, &error) != DW_DLV_OK) { found_die = NULL; } } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } return found_die; } static std::string get_referenced_die_name(Dwarf_Debug dwarf, Dwarf_Die die, Dwarf_Half attr, bool global) { Dwarf_Error error = DW_DLE_NE; std::string value; Dwarf_Die found_die = get_referenced_die(dwarf, die, attr, global); if (found_die) { char *name; if (dwarf_diename(found_die, &name, &error) == DW_DLV_OK) { if (name) { value = std::string(name); } dwarf_dealloc(dwarf, name, DW_DLA_STRING); } dwarf_dealloc(dwarf, found_die, DW_DLA_DIE); } return value; } // Returns a spec DIE linked to the passed one. The caller should // deallocate the DIE static Dwarf_Die get_spec_die(dwarf_fileobject &fobj, Dwarf_Die die) { Dwarf_Debug dwarf = fobj.dwarf_handle.get(); Dwarf_Error error = DW_DLE_NE; Dwarf_Off die_offset; if (fobj.current_cu && dwarf_die_CU_offset(die, &die_offset, &error) == DW_DLV_OK) { die_specmap_t::iterator it = fobj.current_cu->spec_section.find(die_offset); // If we have a DIE that completes the current one, check if // that one has the pc we are looking for if (it != fobj.current_cu->spec_section.end()) { Dwarf_Die spec_die = 0; if (dwarf_offdie(dwarf, it->second, &spec_die, &error) == DW_DLV_OK) { return spec_die; } } } // Maybe we have an abstract origin DIE with the function information? return get_referenced_die(fobj.dwarf_handle.get(), die, DW_AT_abstract_origin, true); } static bool die_has_pc(dwarf_fileobject &fobj, Dwarf_Die die, Dwarf_Addr pc) { Dwarf_Addr low_pc = 0, high_pc = 0; Dwarf_Half high_pc_form = 0; Dwarf_Form_Class return_class; Dwarf_Error error = DW_DLE_NE; Dwarf_Debug dwarf = fobj.dwarf_handle.get(); bool has_lowpc = false; bool has_highpc = false; bool has_ranges = false; if (dwarf_lowpc(die, &low_pc, &error) == DW_DLV_OK) { // If we have a low_pc check if there is a high pc. // If we don't have a high pc this might mean we have a base // address for the ranges list or just an address. has_lowpc = true; if (dwarf_highpc_b(die, &high_pc, &high_pc_form, &return_class, &error) == DW_DLV_OK) { // We do have a high pc. In DWARF 4+ this is an offset from the // low pc, but in earlier versions it's an absolute address. has_highpc = true; // In DWARF 2/3 this would be a DW_FORM_CLASS_ADDRESS if (return_class == DW_FORM_CLASS_CONSTANT) { high_pc = low_pc + high_pc; } // We have low and high pc, check if our address // is in that range return pc >= low_pc && pc < high_pc; } } else { // Reset the low_pc, in case dwarf_lowpc failing set it to some // undefined value. low_pc = 0; } // Check if DW_AT_ranges is present and search for the PC in the // returned ranges list. We always add the low_pc, as it not set it will // be 0, in case we had a DW_AT_low_pc and DW_AT_ranges pair bool result = false; Dwarf_Attribute attr; if (dwarf_attr(die, DW_AT_ranges, &attr, &error) == DW_DLV_OK) { Dwarf_Off offset; if (dwarf_global_formref(attr, &offset, &error) == DW_DLV_OK) { Dwarf_Ranges *ranges; Dwarf_Signed ranges_count = 0; Dwarf_Unsigned byte_count = 0; if (dwarf_get_ranges_a(dwarf, offset, die, &ranges, &ranges_count, &byte_count, &error) == DW_DLV_OK) { has_ranges = ranges_count != 0; for (int i = 0; i < ranges_count; i++) { if (ranges[i].dwr_addr1 != 0 && pc >= ranges[i].dwr_addr1 + low_pc && pc < ranges[i].dwr_addr2 + low_pc) { result = true; break; } } dwarf_ranges_dealloc(dwarf, ranges, ranges_count); } } } // Last attempt. We might have a single address set as low_pc. if (!result && low_pc != 0 && pc == low_pc) { result = true; } // If we don't have lowpc, highpc and ranges maybe this DIE is a // declaration that relies on a DW_AT_specification DIE that happens // later. Use the specification cache we filled when we loaded this CU. if (!result && (!has_lowpc && !has_highpc && !has_ranges)) { Dwarf_Die spec_die = get_spec_die(fobj, die); if (spec_die) { result = die_has_pc(fobj, spec_die, pc); dwarf_dealloc(dwarf, spec_die, DW_DLA_DIE); } } return result; } static void get_type(Dwarf_Debug dwarf, Dwarf_Die die, std::string &type) { Dwarf_Error error = DW_DLE_NE; Dwarf_Die child = 0; if (dwarf_child(die, &child, &error) == DW_DLV_OK) { get_type(dwarf, child, type); } if (child) { type.insert(0, "::"); dwarf_dealloc(dwarf, child, DW_DLA_DIE); } char *name; if (dwarf_diename(die, &name, &error) == DW_DLV_OK) { type.insert(0, std::string(name)); dwarf_dealloc(dwarf, name, DW_DLA_STRING); } else { type.insert(0, ""); } } static std::string get_type_by_signature(Dwarf_Debug dwarf, Dwarf_Die die) { Dwarf_Error error = DW_DLE_NE; Dwarf_Sig8 signature; Dwarf_Bool has_attr = 0; if (dwarf_hasattr(die, DW_AT_signature, &has_attr, &error) == DW_DLV_OK) { if (has_attr) { Dwarf_Attribute attr_mem; if (dwarf_attr(die, DW_AT_signature, &attr_mem, &error) == DW_DLV_OK) { if (dwarf_formsig8(attr_mem, &signature, &error) != DW_DLV_OK) { return std::string(""); } } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } } Dwarf_Unsigned next_cu_header; Dwarf_Sig8 tu_signature; std::string result; bool found = false; while (dwarf_next_cu_header_d(dwarf, 0, 0, 0, 0, 0, 0, 0, &tu_signature, 0, &next_cu_header, 0, &error) == DW_DLV_OK) { if (strncmp(signature.signature, tu_signature.signature, 8) == 0) { Dwarf_Die type_cu_die = 0; if (dwarf_siblingof_b(dwarf, 0, 0, &type_cu_die, &error) == DW_DLV_OK) { Dwarf_Die child_die = 0; if (dwarf_child(type_cu_die, &child_die, &error) == DW_DLV_OK) { get_type(dwarf, child_die, result); found = !result.empty(); dwarf_dealloc(dwarf, child_die, DW_DLA_DIE); } dwarf_dealloc(dwarf, type_cu_die, DW_DLA_DIE); } } } if (found) { while (dwarf_next_cu_header_d(dwarf, 0, 0, 0, 0, 0, 0, 0, 0, 0, &next_cu_header, 0, &error) == DW_DLV_OK) { // Reset the cu header state. Unfortunately, libdwarf's // next_cu_header API keeps its own iterator per Dwarf_Debug // that can't be reset. We need to keep fetching elements until // the end. } } else { // If we couldn't resolve the type just print out the signature std::ostringstream string_stream; string_stream << "<0x" << std::hex << std::setfill('0'); for (int i = 0; i < 8; ++i) { string_stream << std::setw(2) << std::hex << (int)(unsigned char)(signature.signature[i]); } string_stream << ">"; result = string_stream.str(); } return result; } struct type_context_t { bool is_const; bool is_typedef; bool has_type; bool has_name; std::string text; type_context_t() : is_const(false), is_typedef(false), has_type(false), has_name(false) { } }; // Types are resolved from right to left: we get the variable name first // and then all specifiers (like const or pointer) in a chain of DW_AT_type // DIEs. Call this function recursively until we get a complete type // string. static void set_parameter_string(dwarf_fileobject &fobj, Dwarf_Die die, type_context_t &context) { char *name; Dwarf_Error error = DW_DLE_NE; // typedefs contain also the base type, so we skip it and only // print the typedef name if (!context.is_typedef) { if (dwarf_diename(die, &name, &error) == DW_DLV_OK) { if (!context.text.empty()) { context.text.insert(0, " "); } context.text.insert(0, std::string(name)); dwarf_dealloc(fobj.dwarf_handle.get(), name, DW_DLA_STRING); } } else { context.is_typedef = false; context.has_type = true; if (context.is_const) { context.text.insert(0, "const "); context.is_const = false; } } bool next_type_is_const = false; bool is_keyword = true; Dwarf_Half tag = 0; Dwarf_Bool has_attr = 0; if (dwarf_tag(die, &tag, &error) == DW_DLV_OK) { switch (tag) { case DW_TAG_structure_type: case DW_TAG_union_type: case DW_TAG_class_type: case DW_TAG_enumeration_type: context.has_type = true; if (dwarf_hasattr(die, DW_AT_signature, &has_attr, &error) == DW_DLV_OK) { // If we have a signature it means the type is defined // in .debug_types, so we need to load the DIE pointed // at by the signature and resolve it if (has_attr) { std::string type = get_type_by_signature(fobj.dwarf_handle.get(), die); if (context.is_const) type.insert(0, "const "); if (!context.text.empty()) context.text.insert(0, " "); context.text.insert(0, type); } // Treat enums like typedefs, and skip printing its // base type context.is_typedef = (tag == DW_TAG_enumeration_type); } break; case DW_TAG_const_type: next_type_is_const = true; break; case DW_TAG_pointer_type: context.text.insert(0, "*"); break; case DW_TAG_reference_type: context.text.insert(0, "&"); break; case DW_TAG_restrict_type: context.text.insert(0, "restrict "); break; case DW_TAG_rvalue_reference_type: context.text.insert(0, "&&"); break; case DW_TAG_volatile_type: context.text.insert(0, "volatile "); break; case DW_TAG_typedef: // Propagate the const-ness to the next type // as typedefs are linked to its base type next_type_is_const = context.is_const; context.is_typedef = true; context.has_type = true; break; case DW_TAG_base_type: context.has_type = true; break; case DW_TAG_formal_parameter: context.has_name = true; break; default: is_keyword = false; break; } } if (!is_keyword && context.is_const) { context.text.insert(0, "const "); } context.is_const = next_type_is_const; Dwarf_Die ref = get_referenced_die(fobj.dwarf_handle.get(), die, DW_AT_type, true); if (ref) { set_parameter_string(fobj, ref, context); dwarf_dealloc(fobj.dwarf_handle.get(), ref, DW_DLA_DIE); } if (!context.has_type && context.has_name) { context.text.insert(0, "void "); context.has_type = true; } } // Resolve the function return type and parameters static void set_function_parameters(std::string &function_name, std::vector &ns, dwarf_fileobject &fobj, Dwarf_Die die) { Dwarf_Debug dwarf = fobj.dwarf_handle.get(); Dwarf_Error error = DW_DLE_NE; Dwarf_Die current_die = 0; std::string parameters; bool has_spec = true; // Check if we have a spec DIE. If we do we use it as it contains // more information, like parameter names. Dwarf_Die spec_die = get_spec_die(fobj, die); if (!spec_die) { has_spec = false; spec_die = die; } std::vector::const_iterator it = ns.begin(); std::string ns_name; for (it = ns.begin(); it < ns.end(); ++it) { ns_name.append(*it).append("::"); } if (!ns_name.empty()) { function_name.insert(0, ns_name); } // See if we have a function return type. It can be either on the // current die or in its spec one (usually true for inlined functions) std::string return_type = get_referenced_die_name(dwarf, die, DW_AT_type, true); if (return_type.empty()) { return_type = get_referenced_die_name(dwarf, spec_die, DW_AT_type, true); } if (!return_type.empty()) { return_type.append(" "); function_name.insert(0, return_type); } if (dwarf_child(spec_die, ¤t_die, &error) == DW_DLV_OK) { for (;;) { Dwarf_Die sibling_die = 0; Dwarf_Half tag_value; dwarf_tag(current_die, &tag_value, &error); if (tag_value == DW_TAG_formal_parameter) { // Ignore artificial (ie, compiler generated) parameters bool is_artificial = false; Dwarf_Attribute attr_mem; if (dwarf_attr(current_die, DW_AT_artificial, &attr_mem, &error) == DW_DLV_OK) { Dwarf_Bool flag = 0; if (dwarf_formflag(attr_mem, &flag, &error) == DW_DLV_OK) { is_artificial = flag != 0; } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } if (!is_artificial) { type_context_t context; set_parameter_string(fobj, current_die, context); if (parameters.empty()) { parameters.append("("); } else { parameters.append(", "); } parameters.append(context.text); } } int result = dwarf_siblingof(dwarf, current_die, &sibling_die, &error); if (result == DW_DLV_ERROR) { break; } else if (result == DW_DLV_NO_ENTRY) { break; } if (current_die != die) { dwarf_dealloc(dwarf, current_die, DW_DLA_DIE); current_die = 0; } current_die = sibling_die; } } if (parameters.empty()) parameters = "("; parameters.append(")"); // If we got a spec DIE we need to deallocate it if (has_spec) dwarf_dealloc(dwarf, spec_die, DW_DLA_DIE); function_name.append(parameters); } // defined here because in C++98, template function cannot take locally // defined types... grrr. struct inliners_search_cb { void operator()(Dwarf_Die die, std::vector &ns) { Dwarf_Error error = DW_DLE_NE; Dwarf_Half tag_value; Dwarf_Attribute attr_mem; Dwarf_Debug dwarf = fobj.dwarf_handle.get(); dwarf_tag(die, &tag_value, &error); switch (tag_value) { char *name; case DW_TAG_subprogram: if (!trace.source.function.empty()) break; if (dwarf_diename(die, &name, &error) == DW_DLV_OK) { trace.source.function = std::string(name); dwarf_dealloc(dwarf, name, DW_DLA_STRING); } else { // We don't have a function name in this DIE. // Check if there is a referenced non-defining // declaration. trace.source.function = get_referenced_die_name(dwarf, die, DW_AT_abstract_origin, true); if (trace.source.function.empty()) { trace.source.function = get_referenced_die_name(dwarf, die, DW_AT_specification, true); } } // Append the function parameters, if available set_function_parameters(trace.source.function, ns, fobj, die); // If the object function name is empty, it's possible that // there is no dynamic symbol table (maybe the executable // was stripped or not built with -rdynamic). See if we have // a DWARF linkage name to use instead. We try both // linkage_name and MIPS_linkage_name because the MIPS tag // was the unofficial one until it was adopted in DWARF4. // Old gcc versions generate MIPS_linkage_name if (trace.object_function.empty()) { details::demangler demangler; if (dwarf_attr(die, DW_AT_linkage_name, &attr_mem, &error) != DW_DLV_OK) { if (dwarf_attr(die, DW_AT_MIPS_linkage_name, &attr_mem, &error) != DW_DLV_OK) { break; } } char *linkage; if (dwarf_formstring(attr_mem, &linkage, &error) == DW_DLV_OK) { trace.object_function = demangler.demangle(linkage); dwarf_dealloc(dwarf, linkage, DW_DLA_STRING); } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } break; case DW_TAG_inlined_subroutine: ResolvedTrace::SourceLoc sloc; if (dwarf_diename(die, &name, &error) == DW_DLV_OK) { sloc.function = std::string(name); dwarf_dealloc(dwarf, name, DW_DLA_STRING); } else { // We don't have a name for this inlined DIE, it could // be that there is an abstract origin instead. // Get the DW_AT_abstract_origin value, which is a // reference to the source DIE and try to get its name sloc.function = get_referenced_die_name(dwarf, die, DW_AT_abstract_origin, true); } set_function_parameters(sloc.function, ns, fobj, die); std::string file = die_call_file(dwarf, die, cu_die); if (!file.empty()) sloc.filename = file; Dwarf_Unsigned number = 0; if (dwarf_attr(die, DW_AT_call_line, &attr_mem, &error) == DW_DLV_OK) { if (dwarf_formudata(attr_mem, &number, &error) == DW_DLV_OK) { sloc.line = number; } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } if (dwarf_attr(die, DW_AT_call_column, &attr_mem, &error) == DW_DLV_OK) { if (dwarf_formudata(attr_mem, &number, &error) == DW_DLV_OK) { sloc.col = number; } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } trace.inliners.push_back(sloc); break; }; } ResolvedTrace &trace; dwarf_fileobject &fobj; Dwarf_Die cu_die; inliners_search_cb(ResolvedTrace &t, dwarf_fileobject &f, Dwarf_Die c) : trace(t), fobj(f), cu_die(c) {} }; static Dwarf_Die find_fundie_by_pc(dwarf_fileobject &fobj, Dwarf_Die parent_die, Dwarf_Addr pc, Dwarf_Die result) { Dwarf_Die current_die = 0; Dwarf_Error error = DW_DLE_NE; Dwarf_Debug dwarf = fobj.dwarf_handle.get(); if (dwarf_child(parent_die, ¤t_die, &error) != DW_DLV_OK) { return NULL; } for (;;) { Dwarf_Die sibling_die = 0; Dwarf_Half tag_value; dwarf_tag(current_die, &tag_value, &error); switch (tag_value) { case DW_TAG_subprogram: case DW_TAG_inlined_subroutine: if (die_has_pc(fobj, current_die, pc)) { return current_die; } }; bool declaration = false; Dwarf_Attribute attr_mem; if (dwarf_attr(current_die, DW_AT_declaration, &attr_mem, &error) == DW_DLV_OK) { Dwarf_Bool flag = 0; if (dwarf_formflag(attr_mem, &flag, &error) == DW_DLV_OK) { declaration = flag != 0; } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } if (!declaration) { // let's be curious and look deeper in the tree, functions are // not necessarily at the first level, but might be nested // inside a namespace, structure, a function, an inlined // function etc. Dwarf_Die die_mem = 0; Dwarf_Die indie = find_fundie_by_pc(fobj, current_die, pc, die_mem); if (indie) { result = die_mem; return result; } } int res = dwarf_siblingof(dwarf, current_die, &sibling_die, &error); if (res == DW_DLV_ERROR) { return NULL; } else if (res == DW_DLV_NO_ENTRY) { break; } if (current_die != parent_die) { dwarf_dealloc(dwarf, current_die, DW_DLA_DIE); current_die = 0; } current_die = sibling_die; } return NULL; } template static bool deep_first_search_by_pc(dwarf_fileobject &fobj, Dwarf_Die parent_die, Dwarf_Addr pc, std::vector &ns, CB cb) { Dwarf_Die current_die = 0; Dwarf_Debug dwarf = fobj.dwarf_handle.get(); Dwarf_Error error = DW_DLE_NE; if (dwarf_child(parent_die, ¤t_die, &error) != DW_DLV_OK) { return false; } bool branch_has_pc = false; bool has_namespace = false; for (;;) { Dwarf_Die sibling_die = 0; Dwarf_Half tag; if (dwarf_tag(current_die, &tag, &error) == DW_DLV_OK) { if (tag == DW_TAG_namespace || tag == DW_TAG_class_type) { char *ns_name = NULL; if (dwarf_diename(current_die, &ns_name, &error) == DW_DLV_OK) { if (ns_name) { ns.push_back(std::string(ns_name)); } else { ns.push_back(""); } dwarf_dealloc(dwarf, ns_name, DW_DLA_STRING); } else { ns.push_back(""); } has_namespace = true; } } bool declaration = false; Dwarf_Attribute attr_mem; if (tag != DW_TAG_class_type && dwarf_attr(current_die, DW_AT_declaration, &attr_mem, &error) == DW_DLV_OK) { Dwarf_Bool flag = 0; if (dwarf_formflag(attr_mem, &flag, &error) == DW_DLV_OK) { declaration = flag != 0; } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); } if (!declaration) { // let's be curious and look deeper in the tree, function are // not necessarily at the first level, but might be nested // inside a namespace, structure, a function, an inlined // function etc. branch_has_pc = deep_first_search_by_pc(fobj, current_die, pc, ns, cb); } if (!branch_has_pc) { branch_has_pc = die_has_pc(fobj, current_die, pc); } if (branch_has_pc) { cb(current_die, ns); } int result = dwarf_siblingof(dwarf, current_die, &sibling_die, &error); if (result == DW_DLV_ERROR) { return false; } else if (result == DW_DLV_NO_ENTRY) { break; } if (current_die != parent_die) { dwarf_dealloc(dwarf, current_die, DW_DLA_DIE); current_die = 0; } if (has_namespace) { has_namespace = false; ns.pop_back(); } current_die = sibling_die; } if (has_namespace) { ns.pop_back(); } return branch_has_pc; } static std::string die_call_file(Dwarf_Debug dwarf, Dwarf_Die die, Dwarf_Die cu_die) { Dwarf_Attribute attr_mem; Dwarf_Error error = DW_DLE_NE; Dwarf_Unsigned file_index; std::string file; if (dwarf_attr(die, DW_AT_call_file, &attr_mem, &error) == DW_DLV_OK) { if (dwarf_formudata(attr_mem, &file_index, &error) != DW_DLV_OK) { file_index = 0; } dwarf_dealloc(dwarf, attr_mem, DW_DLA_ATTR); if (file_index == 0) { return file; } char **srcfiles = 0; Dwarf_Signed file_count = 0; if (dwarf_srcfiles(cu_die, &srcfiles, &file_count, &error) == DW_DLV_OK) { if (file_count > 0 && file_index <= static_cast(file_count)) { file = std::string(srcfiles[file_index - 1]); } // Deallocate all strings! for (int i = 0; i < file_count; ++i) { dwarf_dealloc(dwarf, srcfiles[i], DW_DLA_STRING); } dwarf_dealloc(dwarf, srcfiles, DW_DLA_LIST); } } return file; } Dwarf_Die find_die(dwarf_fileobject &fobj, Dwarf_Addr addr) { // Let's get to work! First see if we have a debug_aranges section so // we can speed up the search Dwarf_Debug dwarf = fobj.dwarf_handle.get(); Dwarf_Error error = DW_DLE_NE; Dwarf_Arange *aranges; Dwarf_Signed arange_count; Dwarf_Die returnDie; bool found = false; if (dwarf_get_aranges(dwarf, &aranges, &arange_count, &error) != DW_DLV_OK) { aranges = NULL; } if (aranges) { // We have aranges. Get the one where our address is. Dwarf_Arange arange; if (dwarf_get_arange(aranges, arange_count, addr, &arange, &error) == DW_DLV_OK) { // We found our address. Get the compilation-unit DIE offset // represented by the given address range. Dwarf_Off cu_die_offset; if (dwarf_get_cu_die_offset(arange, &cu_die_offset, &error) == DW_DLV_OK) { // Get the DIE at the offset returned by the aranges search. // We set is_info to 1 to specify that the offset is from // the .debug_info section (and not .debug_types) int dwarf_result = dwarf_offdie_b(dwarf, cu_die_offset, 1, &returnDie, &error); found = dwarf_result == DW_DLV_OK; } dwarf_dealloc(dwarf, arange, DW_DLA_ARANGE); } } if (found) return returnDie; // The caller is responsible for freeing the die // The search for aranges failed. Try to find our address by scanning // all compilation units. Dwarf_Unsigned next_cu_header; Dwarf_Half tag = 0; returnDie = 0; while (!found && dwarf_next_cu_header_d(dwarf, 1, 0, 0, 0, 0, 0, 0, 0, 0, &next_cu_header, 0, &error) == DW_DLV_OK) { if (returnDie) dwarf_dealloc(dwarf, returnDie, DW_DLA_DIE); if (dwarf_siblingof(dwarf, 0, &returnDie, &error) == DW_DLV_OK) { if ((dwarf_tag(returnDie, &tag, &error) == DW_DLV_OK) && tag == DW_TAG_compile_unit) { if (die_has_pc(fobj, returnDie, addr)) { found = true; } } } } if (found) { while (dwarf_next_cu_header_d(dwarf, 1, 0, 0, 0, 0, 0, 0, 0, 0, &next_cu_header, 0, &error) == DW_DLV_OK) { // Reset the cu header state. Libdwarf's next_cu_header API // keeps its own iterator per Dwarf_Debug that can't be reset. // We need to keep fetching elements until the end. } } if (found) return returnDie; // We couldn't find any compilation units with ranges or a high/low pc. // Try again by looking at all DIEs in all compilation units. Dwarf_Die cudie; while (dwarf_next_cu_header_d(dwarf, 1, 0, 0, 0, 0, 0, 0, 0, 0, &next_cu_header, 0, &error) == DW_DLV_OK) { if (dwarf_siblingof(dwarf, 0, &cudie, &error) == DW_DLV_OK) { Dwarf_Die die_mem = 0; Dwarf_Die resultDie = find_fundie_by_pc(fobj, cudie, addr, die_mem); if (resultDie) { found = true; break; } } } if (found) { while (dwarf_next_cu_header_d(dwarf, 1, 0, 0, 0, 0, 0, 0, 0, 0, &next_cu_header, 0, &error) == DW_DLV_OK) { // Reset the cu header state. Libdwarf's next_cu_header API // keeps its own iterator per Dwarf_Debug that can't be reset. // We need to keep fetching elements until the end. } } if (found) return cudie; // We failed. return NULL; } }; #endif // BACKWARD_HAS_DWARF == 1 template <> class TraceResolverImpl : public TraceResolverLinuxImpl {}; #endif // BACKWARD_SYSTEM_LINUX #ifdef BACKWARD_SYSTEM_DARWIN template class TraceResolverDarwinImpl; template <> class TraceResolverDarwinImpl : public TraceResolverImplBase { public: void load_addresses(void *const*addresses, int address_count) override { if (address_count == 0) { return; } _symbols.reset(backtrace_symbols(addresses, address_count)); } ResolvedTrace resolve(ResolvedTrace trace) override { // parse: // + char *filename = _symbols[trace.idx]; // skip " " while (*filename && *filename != ' ') filename++; while (*filename == ' ') filename++; // find start of from end ( may contain a space) char *p = filename + strlen(filename) - 1; // skip to start of " + " while (p > filename && *p != ' ') p--; while (p > filename && *p == ' ') p--; while (p > filename && *p != ' ') p--; while (p > filename && *p == ' ') p--; char *funcname_end = p + 1; // skip to start of "" while (p > filename && *p != ' ') p--; char *funcname = p + 1; // skip to start of " " while (p > filename && *p == ' ') p--; while (p > filename && *p != ' ') p--; while (p > filename && *p == ' ') p--; // skip "", handling the case where it contains a char *filename_end = p + 1; if (p == filename) { // something went wrong, give up filename_end = filename + strlen(filename); funcname = filename_end; } trace.object_filename.assign( filename, filename_end); // ok even if filename_end is the ending \0 // (then we assign entire string) if (*funcname) { // if it's not end of string *funcname_end = '\0'; trace.object_function = this->demangle(funcname); trace.object_function += " "; trace.object_function += (funcname_end + 1); trace.source.function = trace.object_function; // we cannot do better. } return trace; } private: details::handle _symbols; }; template <> class TraceResolverImpl : public TraceResolverDarwinImpl {}; #endif // BACKWARD_SYSTEM_DARWIN #ifdef BACKWARD_SYSTEM_WINDOWS // Load all symbol info // Based on: // https://stackoverflow.com/questions/6205981/windows-c-stack-trace-from-a-running-app/28276227#28276227 struct module_data { std::string image_name; std::string module_name; void *base_address; DWORD load_size; }; class get_mod_info { HANDLE process; static const int buffer_length = 4096; public: get_mod_info(HANDLE h) : process(h) {} module_data operator()(HMODULE module) { module_data ret; char temp[buffer_length]; MODULEINFO mi; GetModuleInformation(process, module, &mi, sizeof(mi)); ret.base_address = mi.lpBaseOfDll; ret.load_size = mi.SizeOfImage; GetModuleFileNameExA(process, module, temp, sizeof(temp)); ret.image_name = temp; GetModuleBaseNameA(process, module, temp, sizeof(temp)); ret.module_name = temp; std::vector img(ret.image_name.begin(), ret.image_name.end()); std::vector mod(ret.module_name.begin(), ret.module_name.end()); SymLoadModule64(process, 0, &img[0], &mod[0], (DWORD64)ret.base_address, ret.load_size); return ret; } }; template <> class TraceResolverImpl : public TraceResolverImplBase { public: TraceResolverImpl() { HANDLE process = GetCurrentProcess(); std::vector modules; DWORD cbNeeded; std::vector module_handles(1); SymInitialize(process, NULL, false); DWORD symOptions = SymGetOptions(); symOptions |= SYMOPT_LOAD_LINES | SYMOPT_UNDNAME; SymSetOptions(symOptions); EnumProcessModules(process, &module_handles[0], static_cast(module_handles.size() * sizeof(HMODULE)), &cbNeeded); module_handles.resize(cbNeeded / sizeof(HMODULE)); EnumProcessModules(process, &module_handles[0], static_cast(module_handles.size() * sizeof(HMODULE)), &cbNeeded); std::transform(module_handles.begin(), module_handles.end(), std::back_inserter(modules), get_mod_info(process)); void *base = modules[0].base_address; IMAGE_NT_HEADERS *h = ImageNtHeader(base); image_type = h->FileHeader.Machine; } static const int max_sym_len = 255; struct symbol_t { SYMBOL_INFO sym; char buffer[max_sym_len]; } sym; DWORD64 displacement; ResolvedTrace resolve(ResolvedTrace t) override { HANDLE process = GetCurrentProcess(); char name[256]; memset(&sym, 0, sizeof(sym)); sym.sym.SizeOfStruct = sizeof(SYMBOL_INFO); sym.sym.MaxNameLen = max_sym_len; if (!SymFromAddr(process, (ULONG64)t.addr, &displacement, &sym.sym)) { // TODO: error handling everywhere char* lpMsgBuf; DWORD dw = GetLastError(); if (FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, dw, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (char*)&lpMsgBuf, 0, NULL)) { std::fprintf(stderr, "%s\n", lpMsgBuf); LocalFree(lpMsgBuf); } // abort(); } UnDecorateSymbolName(sym.sym.Name, (PSTR)name, 256, UNDNAME_COMPLETE); DWORD offset = 0; IMAGEHLP_LINE line; if (SymGetLineFromAddr(process, (ULONG64)t.addr, &offset, &line)) { t.object_filename = line.FileName; t.source.filename = line.FileName; t.source.line = line.LineNumber; t.source.col = offset; } t.source.function = name; t.object_filename = ""; t.object_function = name; return t; } DWORD machine_type() const { return image_type; } private: DWORD image_type; }; #endif class TraceResolver : public TraceResolverImpl {}; /*************** CODE SNIPPET ***************/ class SourceFile { public: typedef std::vector > lines_t; SourceFile() {} SourceFile(const std::string &path) { // 1. If BACKWARD_CXX_SOURCE_PREFIXES is set then assume it contains // a colon-separated list of path prefixes. Try prepending each // to the given path until a valid file is found. const std::vector &prefixes = get_paths_from_env_variable(); for (size_t i = 0; i < prefixes.size(); ++i) { // Double slashes (//) should not be a problem. std::string new_path = prefixes[i] + '/' + path; _file.reset(new std::ifstream(new_path.c_str())); if (is_open()) break; } // 2. If no valid file found then fallback to opening the path as-is. if (!_file || !is_open()) { _file.reset(new std::ifstream(path.c_str())); } } bool is_open() const { return _file->is_open(); } lines_t &get_lines(unsigned line_start, unsigned line_count, lines_t &lines) { using namespace std; // This function make uses of the dumbest algo ever: // 1) seek(0) // 2) read lines one by one and discard until line_start // 3) read line one by one until line_start + line_count // // If you are getting snippets many time from the same file, it is // somewhat a waste of CPU, feel free to benchmark and propose a // better solution ;) _file->clear(); _file->seekg(0); string line; unsigned line_idx; for (line_idx = 1; line_idx < line_start; ++line_idx) { std::getline(*_file, line); if (!*_file) { return lines; } } // think of it like a lambda in C++98 ;) // but look, I will reuse it two times! // What a good boy am I. struct isspace { bool operator()(char c) { return std::isspace(c); } }; bool started = false; for (; line_idx < line_start + line_count; ++line_idx) { getline(*_file, line); if (!*_file) { return lines; } if (!started) { if (std::find_if(line.begin(), line.end(), not_isspace()) == line.end()) continue; started = true; } lines.push_back(make_pair(line_idx, line)); } lines.erase( std::find_if(lines.rbegin(), lines.rend(), not_isempty()).base(), lines.end()); return lines; } lines_t get_lines(unsigned line_start, unsigned line_count) { lines_t lines; return get_lines(line_start, line_count, lines); } // there is no find_if_not in C++98, lets do something crappy to // workaround. struct not_isspace { bool operator()(char c) { return !std::isspace(c); } }; // and define this one here because C++98 is not happy with local defined // struct passed to template functions, fuuuu. struct not_isempty { bool operator()(const lines_t::value_type &p) { return !(std::find_if(p.second.begin(), p.second.end(), not_isspace()) == p.second.end()); } }; void swap(SourceFile &b) { _file.swap(b._file); } #ifdef BACKWARD_ATLEAST_CXX11 SourceFile(SourceFile &&from) : _file(nullptr) { swap(from); } SourceFile &operator=(SourceFile &&from) { swap(from); return *this; } #else explicit SourceFile(const SourceFile &from) { // some sort of poor man's move semantic. swap(const_cast(from)); } SourceFile &operator=(const SourceFile &from) { // some sort of poor man's move semantic. swap(const_cast(from)); return *this; } #endif // Allow adding to paths gotten from BACKWARD_CXX_SOURCE_PREFIXES after loading the // library; this can be useful when the library is loaded when the locations are unknown // Warning: Because this edits the static paths variable, it is *not* intrinsiclly thread safe static void add_paths_to_env_variable_impl(const std::string & to_add) { get_mutable_paths_from_env_variable().push_back(to_add); } private: details::handle > _file; static std::vector get_paths_from_env_variable_impl() { std::vector paths; const char *prefixes_str = std::getenv("BACKWARD_CXX_SOURCE_PREFIXES"); if (prefixes_str && prefixes_str[0]) { paths = details::split_source_prefixes(prefixes_str); } return paths; } static std::vector &get_mutable_paths_from_env_variable() { static volatile std::vector paths = get_paths_from_env_variable_impl(); return const_cast&>(paths); } static const std::vector &get_paths_from_env_variable() { return get_mutable_paths_from_env_variable(); } #ifdef BACKWARD_ATLEAST_CXX11 SourceFile(const SourceFile &) = delete; SourceFile &operator=(const SourceFile &) = delete; #endif }; class SnippetFactory { public: typedef SourceFile::lines_t lines_t; lines_t get_snippet(const std::string &filename, unsigned line_start, unsigned context_size) { SourceFile &src_file = get_src_file(filename); unsigned start = line_start - context_size / 2; return src_file.get_lines(start, context_size); } lines_t get_combined_snippet(const std::string &filename_a, unsigned line_a, const std::string &filename_b, unsigned line_b, unsigned context_size) { SourceFile &src_file_a = get_src_file(filename_a); SourceFile &src_file_b = get_src_file(filename_b); lines_t lines = src_file_a.get_lines(line_a - context_size / 4, context_size / 2); src_file_b.get_lines(line_b - context_size / 4, context_size / 2, lines); return lines; } lines_t get_coalesced_snippet(const std::string &filename, unsigned line_a, unsigned line_b, unsigned context_size) { SourceFile &src_file = get_src_file(filename); using std::max; using std::min; unsigned a = min(line_a, line_b); unsigned b = max(line_a, line_b); if ((b - a) < (context_size / 3)) { return src_file.get_lines((a + b - context_size + 1) / 2, context_size); } lines_t lines = src_file.get_lines(a - context_size / 4, context_size / 2); src_file.get_lines(b - context_size / 4, context_size / 2, lines); return lines; } private: typedef details::hashtable::type src_files_t; src_files_t _src_files; SourceFile &get_src_file(const std::string &filename) { src_files_t::iterator it = _src_files.find(filename); if (it != _src_files.end()) { return it->second; } SourceFile &new_src_file = _src_files[filename]; new_src_file = SourceFile(filename); return new_src_file; } }; /*************** PRINTER ***************/ namespace ColorMode { enum type { automatic, never, always }; } class cfile_streambuf : public std::streambuf { public: cfile_streambuf(FILE *_sink) : sink(_sink) {} int_type underflow() override { return traits_type::eof(); } int_type overflow(int_type ch) override { if (traits_type::not_eof(ch) && fputc(ch, sink) != EOF) { return ch; } return traits_type::eof(); } std::streamsize xsputn(const char_type *s, std::streamsize count) override { return static_cast( fwrite(s, sizeof *s, static_cast(count), sink)); } #ifdef BACKWARD_ATLEAST_CXX11 public: cfile_streambuf(const cfile_streambuf &) = delete; cfile_streambuf &operator=(const cfile_streambuf &) = delete; #else private: cfile_streambuf(const cfile_streambuf &); cfile_streambuf &operator=(const cfile_streambuf &); #endif private: FILE *sink; std::vector buffer; }; #ifdef BACKWARD_SYSTEM_LINUX namespace Color { enum type { yellow = 33, purple = 35, reset = 39 }; } // namespace Color class Colorize { public: Colorize(std::ostream &os) : _os(os), _reset(false), _enabled(false) {} void activate(ColorMode::type mode) { _enabled = mode == ColorMode::always; } void activate(ColorMode::type mode, FILE *fp) { activate(mode, fileno(fp)); } void set_color(Color::type ccode) { if (!_enabled) return; // I assume that the terminal can handle basic colors. Seriously I // don't want to deal with all the termcap shit. _os << "\033[" << static_cast(ccode) << "m"; _reset = (ccode != Color::reset); } ~Colorize() { if (_reset) { set_color(Color::reset); } } private: void activate(ColorMode::type mode, int fd) { activate(mode == ColorMode::automatic && isatty(fd) ? ColorMode::always : mode); } std::ostream &_os; bool _reset; bool _enabled; }; #else // ndef BACKWARD_SYSTEM_LINUX namespace Color { enum type { yellow = 0, purple = 0, reset = 0 }; } // namespace Color class Colorize { public: Colorize(std::ostream &) {} void activate(ColorMode::type) {} void activate(ColorMode::type, FILE *) {} void set_color(Color::type) {} }; #endif // BACKWARD_SYSTEM_LINUX class Printer { public: bool snippet; ColorMode::type color_mode; bool address; bool object; int inliner_context_size; int trace_context_size; bool reverse; Printer() : snippet(true), color_mode(ColorMode::automatic), address(false), object(false), inliner_context_size(5), trace_context_size(7), reverse(true) {} template FILE *print(ST &st, FILE *fp = stderr) { cfile_streambuf obuf(fp); std::ostream os(&obuf); Colorize colorize(os); colorize.activate(color_mode, fp); print_stacktrace(st, os, colorize); return fp; } template std::ostream &print(ST &st, std::ostream &os) { Colorize colorize(os); colorize.activate(color_mode); print_stacktrace(st, os, colorize); return os; } template FILE *print(IT begin, IT end, FILE *fp = stderr, size_t thread_id = 0) { cfile_streambuf obuf(fp); std::ostream os(&obuf); Colorize colorize(os); colorize.activate(color_mode, fp); print_stacktrace(begin, end, os, thread_id, colorize); return fp; } template std::ostream &print(IT begin, IT end, std::ostream &os, size_t thread_id = 0) { Colorize colorize(os); colorize.activate(color_mode); print_stacktrace(begin, end, os, thread_id, colorize); return os; } TraceResolver const &resolver() const { return _resolver; } private: TraceResolver _resolver; SnippetFactory _snippets; template void print_stacktrace(ST &st, std::ostream &os, Colorize &colorize) { print_header(os, st.thread_id()); _resolver.load_stacktrace(st); if ( reverse ) { for (size_t trace_idx = st.size(); trace_idx > 0; --trace_idx) { print_trace(os, _resolver.resolve(st[trace_idx - 1]), colorize); } } else { for (size_t trace_idx = 0; trace_idx < st.size(); ++trace_idx) { print_trace(os, _resolver.resolve(st[trace_idx]), colorize); } } } template void print_stacktrace(IT begin, IT end, std::ostream &os, size_t thread_id, Colorize &colorize) { print_header(os, thread_id); for (; begin != end; ++begin) { print_trace(os, *begin, colorize); } } void print_header(std::ostream &os, size_t thread_id) { os << "Stack trace (most recent call last)"; if (thread_id) { os << " in thread " << thread_id; } os << ":\n"; } void print_trace(std::ostream &os, const ResolvedTrace &trace, Colorize &colorize) { os << "#" << std::left << std::setw(2) << trace.idx << std::right; bool already_indented = true; if (!trace.source.filename.size() || object) { os << " Object \"" << trace.object_filename << "\", at " << trace.addr << ", in " << trace.object_function << "\n"; already_indented = false; } for (size_t inliner_idx = trace.inliners.size(); inliner_idx > 0; --inliner_idx) { if (!already_indented) { os << " "; } const ResolvedTrace::SourceLoc &inliner_loc = trace.inliners[inliner_idx - 1]; print_source_loc(os, " | ", inliner_loc); if (snippet) { print_snippet(os, " | ", inliner_loc, colorize, Color::purple, inliner_context_size); } already_indented = false; } if (trace.source.filename.size()) { if (!already_indented) { os << " "; } print_source_loc(os, " ", trace.source, trace.addr); if (snippet) { print_snippet(os, " ", trace.source, colorize, Color::yellow, trace_context_size); } } } void print_snippet(std::ostream &os, const char *indent, const ResolvedTrace::SourceLoc &source_loc, Colorize &colorize, Color::type color_code, int context_size) { using namespace std; typedef SnippetFactory::lines_t lines_t; lines_t lines = _snippets.get_snippet(source_loc.filename, source_loc.line, static_cast(context_size)); for (lines_t::const_iterator it = lines.begin(); it != lines.end(); ++it) { if (it->first == source_loc.line) { colorize.set_color(color_code); os << indent << ">"; } else { os << indent << " "; } os << std::setw(4) << it->first << ": " << it->second << "\n"; if (it->first == source_loc.line) { colorize.set_color(Color::reset); } } } void print_source_loc(std::ostream &os, const char *indent, const ResolvedTrace::SourceLoc &source_loc, void *addr = nullptr) { os << indent << "Source \"" << source_loc.filename << "\", line " << source_loc.line << ", in " << source_loc.function; if (address && addr != nullptr) { os << " [" << addr << "]"; } os << "\n"; } }; /*************** SIGNALS HANDLING ***************/ #if defined(BACKWARD_SYSTEM_LINUX) || defined(BACKWARD_SYSTEM_DARWIN) class SignalHandling { public: static std::vector make_default_signals() { const int posix_signals[] = { // Signals for which the default action is "Core". SIGABRT, // Abort signal from abort(3) SIGBUS, // Bus error (bad memory access) SIGFPE, // Floating point exception SIGILL, // Illegal Instruction SIGIOT, // IOT trap. A synonym for SIGABRT SIGQUIT, // Quit from keyboard SIGSEGV, // Invalid memory reference SIGSYS, // Bad argument to routine (SVr4) SIGTRAP, // Trace/breakpoint trap SIGXCPU, // CPU time limit exceeded (4.2BSD) SIGXFSZ, // File size limit exceeded (4.2BSD) #if defined(BACKWARD_SYSTEM_DARWIN) SIGEMT, // emulation instruction executed #endif }; return std::vector(posix_signals, posix_signals + sizeof posix_signals / sizeof posix_signals[0]); } SignalHandling(const std::vector &posix_signals = make_default_signals()) : _loaded(false) { bool success = true; const size_t stack_size = 1024 * 1024 * 8; _stack_content.reset(static_cast(malloc(stack_size))); if (_stack_content) { stack_t ss; ss.ss_sp = _stack_content.get(); ss.ss_size = stack_size; ss.ss_flags = 0; if (sigaltstack(&ss, nullptr) < 0) { success = false; } } else { success = false; } for (size_t i = 0; i < posix_signals.size(); ++i) { struct sigaction action; memset(&action, 0, sizeof action); action.sa_flags = static_cast(SA_SIGINFO | SA_ONSTACK | SA_NODEFER | SA_RESETHAND); sigfillset(&action.sa_mask); sigdelset(&action.sa_mask, posix_signals[i]); #if defined(__clang__) #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wdisabled-macro-expansion" #endif action.sa_sigaction = &sig_handler; #if defined(__clang__) #pragma clang diagnostic pop #endif int r = sigaction(posix_signals[i], &action, nullptr); if (r < 0) success = false; } _loaded = success; } bool loaded() const { return _loaded; } static void handleSignal(int, siginfo_t *info, void *_ctx) { ucontext_t *uctx = static_cast(_ctx); StackTrace st; void *error_addr = nullptr; #ifdef REG_RIP // x86_64 error_addr = reinterpret_cast(uctx->uc_mcontext.gregs[REG_RIP]); #elif defined(REG_EIP) // x86_32 error_addr = reinterpret_cast(uctx->uc_mcontext.gregs[REG_EIP]); #elif defined(__arm__) error_addr = reinterpret_cast(uctx->uc_mcontext.arm_pc); #elif defined(__aarch64__) #if defined(__APPLE__) error_addr = reinterpret_cast(uctx->uc_mcontext->__ss.__pc); #else error_addr = reinterpret_cast(uctx->uc_mcontext.pc); #endif #elif defined(__mips__) error_addr = reinterpret_cast( reinterpret_cast(&uctx->uc_mcontext)->sc_pc); #elif defined(__ppc__) || defined(__powerpc) || defined(__powerpc__) || \ defined(__POWERPC__) error_addr = reinterpret_cast(uctx->uc_mcontext.regs->nip); #elif defined(__riscv) error_addr = reinterpret_cast(uctx->uc_mcontext.__gregs[REG_PC]); #elif defined(__s390x__) error_addr = reinterpret_cast(uctx->uc_mcontext.psw.addr); #elif defined(__APPLE__) && defined(__x86_64__) error_addr = reinterpret_cast(uctx->uc_mcontext->__ss.__rip); #elif defined(__APPLE__) error_addr = reinterpret_cast(uctx->uc_mcontext->__ss.__eip); #else #warning ":/ sorry, ain't know no nothing none not of your architecture!" #endif if (error_addr) { st.load_from(error_addr, 32, reinterpret_cast(uctx), info->si_addr); } else { st.load_here(32, reinterpret_cast(uctx), info->si_addr); } Printer printer; printer.address = true; printer.print(st, stderr); #if (defined(_XOPEN_SOURCE) && _XOPEN_SOURCE >= 700) || \ (defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE >= 200809L) psiginfo(info, nullptr); #else (void)info; #endif } private: details::handle _stack_content; bool _loaded; #ifdef __GNUC__ __attribute__((noreturn)) #endif static void sig_handler(int signo, siginfo_t *info, void *_ctx) { handleSignal(signo, info, _ctx); // try to forward the signal. raise(info->si_signo); // terminate the process immediately. puts("watf? exit"); _exit(EXIT_FAILURE); } }; #endif // BACKWARD_SYSTEM_LINUX || BACKWARD_SYSTEM_DARWIN #ifdef BACKWARD_SYSTEM_WINDOWS class SignalHandling { public: SignalHandling(const std::vector & = std::vector()) : reporter_thread_([]() { /* We handle crashes in a utility thread: backward structures and some Windows functions called here need stack space, which we do not have when we encounter a stack overflow. To support reporting stack traces during a stack overflow, we create a utility thread at startup, which waits until a crash happens or the program exits normally. */ { std::unique_lock lk(mtx()); cv().wait(lk, [] { return crashed() != crash_status::running; }); } if (crashed() == crash_status::crashed) { handle_stacktrace(skip_recs()); } { std::unique_lock lk(mtx()); crashed() = crash_status::ending; } cv().notify_one(); }) { SetUnhandledExceptionFilter(crash_handler); signal(SIGABRT, signal_handler); _set_abort_behavior(0, _WRITE_ABORT_MSG | _CALL_REPORTFAULT); std::set_terminate(&terminator); #ifndef BACKWARD_ATLEAST_CXX17 std::set_unexpected(&terminator); #endif _set_purecall_handler(&terminator); _set_invalid_parameter_handler(&invalid_parameter_handler); } bool loaded() const { return true; } ~SignalHandling() { { std::unique_lock lk(mtx()); crashed() = crash_status::normal_exit; } cv().notify_one(); reporter_thread_.join(); } private: static CONTEXT *ctx() { static CONTEXT data; return &data; } enum class crash_status { running, crashed, normal_exit, ending }; static crash_status &crashed() { static crash_status data; return data; } static std::mutex &mtx() { static std::mutex data; return data; } static std::condition_variable &cv() { static std::condition_variable data; return data; } static HANDLE &thread_handle() { static HANDLE handle; return handle; } std::thread reporter_thread_; // TODO: how not to hardcode these? static const constexpr int signal_skip_recs = #ifdef __clang__ // With clang, RtlCaptureContext also captures the stack frame of the // current function Below that, there are 3 internal Windows functions 4 #else // With MSVC cl, RtlCaptureContext misses the stack frame of the current // function The first entries during StackWalk are the 3 internal Windows // functions 3 #endif ; static int &skip_recs() { static int data; return data; } static inline void terminator() { crash_handler(signal_skip_recs); abort(); } static inline void signal_handler(int) { crash_handler(signal_skip_recs); abort(); } static inline void __cdecl invalid_parameter_handler(const wchar_t *, const wchar_t *, const wchar_t *, unsigned int, uintptr_t) { crash_handler(signal_skip_recs); abort(); } NOINLINE static LONG WINAPI crash_handler(EXCEPTION_POINTERS *info) { // The exception info supplies a trace from exactly where the issue was, // no need to skip records crash_handler(0, info->ContextRecord); return EXCEPTION_CONTINUE_SEARCH; } NOINLINE static void crash_handler(int skip, CONTEXT *ct = nullptr) { if (ct == nullptr) { RtlCaptureContext(ctx()); } else { memcpy(ctx(), ct, sizeof(CONTEXT)); } DuplicateHandle(GetCurrentProcess(), GetCurrentThread(), GetCurrentProcess(), &thread_handle(), 0, FALSE, DUPLICATE_SAME_ACCESS); skip_recs() = skip; { std::unique_lock lk(mtx()); crashed() = crash_status::crashed; } cv().notify_one(); { std::unique_lock lk(mtx()); cv().wait(lk, [] { return crashed() != crash_status::crashed; }); } } static void handle_stacktrace(int skip_frames = 0) { // printer creates the TraceResolver, which can supply us a machine type // for stack walking. Without this, StackTrace can only guess using some // macros. // StackTrace also requires that the PDBs are already loaded, which is done // in the constructor of TraceResolver Printer printer; StackTrace st; st.set_machine_type(printer.resolver().machine_type()); st.set_thread_handle(thread_handle()); st.load_here(32 + skip_frames, ctx()); st.skip_n_firsts(skip_frames); printer.address = true; printer.print(st, std::cerr); } }; #endif // BACKWARD_SYSTEM_WINDOWS #ifdef BACKWARD_SYSTEM_UNKNOWN class SignalHandling { public: SignalHandling(const std::vector & = std::vector()) {} bool init() { return false; } bool loaded() { return false; } }; #endif // BACKWARD_SYSTEM_UNKNOWN } // namespace backward #endif /* H_GUARD */ ================================================ FILE: alvr/server_openvr/cpp/shared/threadtools.cpp ================================================ //===================== Copyright (c) Valve Corporation. All Rights Reserved. ====================== #include "threadtools.h" //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- CThread::CThread() : m_pThread( NULL ) {} //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- CThread::~CThread() { Join(); } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CThread::Start() { if ( Init() ) { m_pThread = new std::thread( &CThread::Run, this ); } } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- void CThread::Join() { if ( m_pThread ) { m_pThread->join(); delete m_pThread; m_pThread = NULL; } } #ifdef _WIN32 //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- CThreadEvent::CThreadEvent( bool bManualReset ) { m_hSyncObject = CreateEvent( NULL, bManualReset, FALSE, NULL ); } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- CThreadEvent::~CThreadEvent() { if ( m_hSyncObject ) { CloseHandle( m_hSyncObject ); } } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CThreadEvent::Wait( uint32_t nTimeoutMs ) { return WaitForSingleObject( m_hSyncObject, nTimeoutMs ) == WAIT_OBJECT_0; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CThreadEvent::Set() { return SetEvent( m_hSyncObject ) != 0; } //-------------------------------------------------------------------------------------------------- //-------------------------------------------------------------------------------------------------- bool CThreadEvent::Reset() { return ResetEvent( m_hSyncObject ) != 0; } #endif ================================================ FILE: alvr/server_openvr/cpp/shared/threadtools.h ================================================ //===================== Copyright (c) Valve Corporation. All Rights Reserved. ====================== // // Helper classes for working with threads. // //================================================================================================== #pragma once #include #ifdef _WIN32 #include #endif #define THREAD_PRIORITY_MOST_URGENT 15 class CThread { public: CThread(); virtual ~CThread(); virtual bool Init() { return true; } virtual void Run() = 0; void Start(); void Join(); private: std::thread *m_pThread; }; #ifdef _WIN32 class CThreadEvent { public: CThreadEvent( bool bManualReset = false ); ~CThreadEvent(); bool Wait( uint32_t nTimeoutMs = INFINITE ); bool Set(); bool Reset(); private: HANDLE m_hSyncObject; }; #endif ================================================ FILE: alvr/server_openvr/src/graphics.rs ================================================ static FRAME_RENDER_VS_CSO: &[u8] = include_bytes!("../cpp/platform/win32/FrameRenderVS.cso"); static FRAME_RENDER_PS_CSO: &[u8] = include_bytes!("../cpp/platform/win32/FrameRenderPS.cso"); static QUAD_SHADER_CSO: &[u8] = include_bytes!("../cpp/platform/win32/QuadVertexShader.cso"); static COMPRESS_AXIS_ALIGNED_CSO: &[u8] = include_bytes!("../cpp/platform/win32/CompressAxisAlignedPixelShader.cso"); static COLOR_CORRECTION_CSO: &[u8] = include_bytes!("../cpp/platform/win32/ColorCorrectionPixelShader.cso"); static RGBTOYUV420_CSO: &[u8] = include_bytes!("../cpp/platform/win32/rgbtoyuv420.cso"); static QUAD_SHADER_COMP_SPV: &[u8] = include_bytes!("../cpp/platform/linux/shader/quad.comp.spv"); static COLOR_SHADER_COMP_SPV: &[u8] = include_bytes!("../cpp/platform/linux/shader/color.comp.spv"); static FFR_SHADER_COMP_SPV: &[u8] = include_bytes!("../cpp/platform/linux/shader/ffr.comp.spv"); static RGBTOYUV420_SHADER_COMP_SPV: &[u8] = include_bytes!("../cpp/platform/linux/shader/rgbtoyuv420.comp.spv"); pub fn initialize_shaders() { unsafe { crate::FRAME_RENDER_VS_CSO_PTR = FRAME_RENDER_VS_CSO.as_ptr(); crate::FRAME_RENDER_VS_CSO_LEN = FRAME_RENDER_VS_CSO.len() as _; crate::FRAME_RENDER_PS_CSO_PTR = FRAME_RENDER_PS_CSO.as_ptr(); crate::FRAME_RENDER_PS_CSO_LEN = FRAME_RENDER_PS_CSO.len() as _; crate::QUAD_SHADER_CSO_PTR = QUAD_SHADER_CSO.as_ptr(); crate::QUAD_SHADER_CSO_LEN = QUAD_SHADER_CSO.len() as _; crate::COMPRESS_AXIS_ALIGNED_CSO_PTR = COMPRESS_AXIS_ALIGNED_CSO.as_ptr(); crate::COMPRESS_AXIS_ALIGNED_CSO_LEN = COMPRESS_AXIS_ALIGNED_CSO.len() as _; crate::COLOR_CORRECTION_CSO_PTR = COLOR_CORRECTION_CSO.as_ptr(); crate::COLOR_CORRECTION_CSO_LEN = COLOR_CORRECTION_CSO.len() as _; crate::RGBTOYUV420_CSO_PTR = RGBTOYUV420_CSO.as_ptr(); crate::RGBTOYUV420_CSO_LEN = RGBTOYUV420_CSO.len() as _; crate::QUAD_SHADER_COMP_SPV_PTR = QUAD_SHADER_COMP_SPV.as_ptr(); crate::QUAD_SHADER_COMP_SPV_LEN = QUAD_SHADER_COMP_SPV.len() as _; crate::COLOR_SHADER_COMP_SPV_PTR = COLOR_SHADER_COMP_SPV.as_ptr(); crate::COLOR_SHADER_COMP_SPV_LEN = COLOR_SHADER_COMP_SPV.len() as _; crate::FFR_SHADER_COMP_SPV_PTR = FFR_SHADER_COMP_SPV.as_ptr(); crate::FFR_SHADER_COMP_SPV_LEN = FFR_SHADER_COMP_SPV.len() as _; crate::RGBTOYUV420_SHADER_COMP_SPV_PTR = RGBTOYUV420_SHADER_COMP_SPV.as_ptr(); crate::RGBTOYUV420_SHADER_COMP_SPV_LEN = RGBTOYUV420_SHADER_COMP_SPV.len() as _; } } ================================================ FILE: alvr/server_openvr/src/lib.rs ================================================ mod graphics; mod props; mod tracking; #[allow( non_camel_case_types, non_upper_case_globals, dead_code, non_snake_case, clippy::unseparated_literal_suffix )] mod bindings { include!(concat!(env!("OUT_DIR"), "/bindings.rs")); } use bindings::*; use alvr_common::{ BUTTON_INFO, HAND_LEFT_ID, HAND_RIGHT_ID, HAND_TRACKER_LEFT_ID, HAND_TRACKER_RIGHT_ID, HEAD_ID, Pose, ViewParams, error, parking_lot::{Mutex, RwLock}, settings_schema::Switch, warn, }; use alvr_filesystem as afs; use alvr_packets::{ButtonValue, Haptics}; use alvr_server_core::{HandType, ServerCoreContext, ServerCoreEvent}; use alvr_session::{CodecType, ControllersConfig}; use std::{ collections::VecDeque, ffi::{CString, OsStr, c_char, c_void}, ptr, sync::{Once, mpsc}, thread, time::{Duration, Instant}, }; static SERVER_CORE_CONTEXT: RwLock> = RwLock::new(None); static LOCAL_VIEW_PARAMS: RwLock<[ViewParams; 2]> = RwLock::new([ViewParams::DUMMY; 2]); static HEAD_POSE_QUEUE: Mutex> = Mutex::new(VecDeque::new()); fn event_loop(events_receiver: mpsc::Receiver) { thread::spawn(move || { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.start_connection(); } let mut last_resync = Instant::now(); loop { let event = match events_receiver.recv_timeout(Duration::from_millis(5)) { Ok(event) => event, Err(mpsc::RecvTimeoutError::Timeout) => continue, Err(mpsc::RecvTimeoutError::Disconnected) => break, }; match event { ServerCoreEvent::SetOpenvrProperty { device_id, prop } => { props::set_openvr_prop(None, device_id, prop) } ServerCoreEvent::ClientConnected => unsafe { if InitializeStreaming() { RequestDriverResync(); } else { SERVER_CORE_CONTEXT.write().take(); ShutdownSteamvr(); } }, ServerCoreEvent::ClientDisconnected => unsafe { DeinitializeStreaming() }, ServerCoreEvent::Battery(info) => unsafe { SetBattery(info.device_id, info.gauge_value, info.is_plugged) }, ServerCoreEvent::PlayspaceSync(bounds) => unsafe { SetChaperoneArea(bounds.x, bounds.y) }, ServerCoreEvent::LocalViewParams(params) => unsafe { *LOCAL_VIEW_PARAMS.write() = params; let ffi_params = [ tracking::to_ffi_view_params(params[0]), tracking::to_ffi_view_params(params[1]), ]; SetLocalViewParams(ffi_params.as_ptr()); }, ServerCoreEvent::Tracking { poll_timestamp } => { let headset_config = &alvr_server_core::settings().headset; let controllers_config = headset_config.controllers.clone().into_option(); let track_body = headset_config.body_tracking.enabled(); let tracked = controllers_config.as_ref().is_some_and(|c| c.tracked); let detached_controllers = headset_config .multimodal_tracking .as_option() .is_some_and(|c| c.detached_controllers_steamvr_sink); if let Some(context) = &*SERVER_CORE_CONTEXT.read() { let target_timestamp = poll_timestamp + context.get_motion_to_photon_latency(); let controllers_pose_time_offset = context.get_tracker_pose_time_offset(); // We need to remove the additional offset that SteamVR adds let target_controller_timestamp = target_timestamp.saturating_sub(controllers_pose_time_offset); let ffi_head_motion = if let Some(motion) = context.get_device_motion(*HEAD_ID, poll_timestamp) { let motion = motion.predict(poll_timestamp, target_timestamp); let mut head_pose_queue_lock = HEAD_POSE_QUEUE.lock(); head_pose_queue_lock.push_back((poll_timestamp, motion.pose)); while head_pose_queue_lock.len() > 360 { head_pose_queue_lock.pop_front(); } tracking::to_ffi_motion(*HEAD_ID, motion) } else { FfiDeviceMotion::default() }; let (ffi_left_controller_motion, ffi_right_controller_motion) = if tracked && let Some(config) = &controllers_config { let ffi_left_controller_motion = context .get_device_motion(*HAND_LEFT_ID, poll_timestamp) .map(|motion| { let motion = motion .predict(poll_timestamp, target_controller_timestamp); let motion = tracking::offset_controller_motion( config, *HAND_LEFT_ID, motion, ); tracking::to_ffi_motion(*HAND_LEFT_ID, motion) }); let ffi_right_controller_motion = context .get_device_motion(*HAND_RIGHT_ID, poll_timestamp) .map(|motion| { let motion = motion .predict(poll_timestamp, target_controller_timestamp); let motion = tracking::offset_controller_motion( config, *HAND_RIGHT_ID, motion, ); tracking::to_ffi_motion(*HAND_RIGHT_ID, motion) }); (ffi_left_controller_motion, ffi_right_controller_motion) } else { (None, None) }; let ( ffi_left_hand_skeleton, ffi_right_hand_skeleton, use_separate_hand_trackers, predict_hand_skeleton, ) = if let Some(ControllersConfig { hand_skeleton: Switch::Enabled(hand_skeleton_config), .. }) = controllers_config { let left_hand_skeleton = context .get_hand_skeleton(HandType::Left, poll_timestamp) .map(|s| { tracking::to_openvr_ffi_hand_skeleton( headset_config, *HAND_LEFT_ID, &s, ) }); let right_hand_skeleton = context .get_hand_skeleton(HandType::Right, poll_timestamp) .map(|s| { tracking::to_openvr_ffi_hand_skeleton( headset_config, *HAND_RIGHT_ID, &s, ) }); ( tracked.then_some(left_hand_skeleton).flatten(), tracked.then_some(right_hand_skeleton).flatten(), hand_skeleton_config.steamvr_input_2_0, hand_skeleton_config.predict, ) } else { (None, None, false, false) }; let ffi_left_hand_data = FfiHandData { controllerMotion: if let Some(motion) = &ffi_left_controller_motion { motion } else { ptr::null() }, handSkeleton: if let Some(skeleton) = &ffi_left_hand_skeleton { skeleton } else { ptr::null() }, isHandTracker: use_separate_hand_trackers && ffi_left_controller_motion.is_none() && ffi_left_hand_skeleton.is_some(), predictHandSkeleton: predict_hand_skeleton, }; let ffi_right_hand_data = FfiHandData { controllerMotion: if let Some(motion) = &ffi_right_controller_motion { motion } else { ptr::null() }, handSkeleton: if let Some(skeleton) = &ffi_right_hand_skeleton { skeleton } else { ptr::null() }, isHandTracker: use_separate_hand_trackers && ffi_right_controller_motion.is_none() && ffi_right_hand_skeleton.is_some(), predictHandSkeleton: predict_hand_skeleton, }; let ffi_body_tracker_motions = if track_body || detached_controllers { tracking::BODY_TRACKER_IDS .iter() .filter_map(|id| { Some(tracking::to_ffi_motion( *id, context.get_device_motion(*id, poll_timestamp)?, )) }) .collect::>() } else { vec![] }; // There are two pairs of controllers/hand tracking devices registered in // OpenVR, two lefts and two rights. If enabled with use_separate_hand_trackers, // we select at runtime which device to use (selected for left and right hand // independently. Selection is done by setting deviceIsConnected. unsafe { SetTracking( poll_timestamp.as_nanos() as _, controllers_pose_time_offset.as_secs_f32(), ffi_head_motion, ffi_left_hand_data, ffi_right_hand_data, ffi_body_tracker_motions.as_ptr(), ffi_body_tracker_motions.len() as i32, ) }; } } ServerCoreEvent::Buttons(entries) => { for entry in entries { let value = match entry.value { ButtonValue::Binary(value) => FfiButtonValue { type_: FfiButtonType_BUTTON_TYPE_BINARY, __bindgen_anon_1: FfiButtonValue__bindgen_ty_1 { binary: value.into(), }, }, ButtonValue::Scalar(value) => FfiButtonValue { type_: FfiButtonType_BUTTON_TYPE_SCALAR, __bindgen_anon_1: FfiButtonValue__bindgen_ty_1 { scalar: value }, }, }; unsafe { SetButton(entry.path_id, value) }; } } ServerCoreEvent::RequestIDR => unsafe { RequestIDR() }, ServerCoreEvent::CaptureFrame => unsafe { CaptureFrame() }, ServerCoreEvent::GameRenderLatencyFeedback(game_latency) => { if cfg!(target_os = "linux") && game_latency.as_secs_f32() > 0.25 { let now = Instant::now(); if now.saturating_duration_since(last_resync).as_secs_f32() > 0.1 { last_resync = now; warn!("Desync detected. Attempting recovery."); unsafe { RequestDriverResync(); } } } } ServerCoreEvent::ShutdownPending => { SERVER_CORE_CONTEXT.write().take(); unsafe { ShutdownSteamvr() }; } ServerCoreEvent::RestartPending => { if let Some(context) = SERVER_CORE_CONTEXT.write().take() { context.restart(); } unsafe { ShutdownSteamvr() }; } ServerCoreEvent::ProximityState(headset_is_worn) => unsafe { SetProximityState(headset_is_worn) }, } } unsafe { ShutdownOpenvrClient() }; }); } extern "C" fn driver_ready_idle(set_default_chap: bool) { thread::spawn(move || { unsafe { InitOpenvrClient() }; if set_default_chap { // call this when inside a new thread. Calling this on the parent thread will crash SteamVR unsafe { SetChaperoneArea(2.0, 2.0); } } }); } /// # Safety /// `instance_ptr` is a valid pointer to a `TrackedDevice` instance pub unsafe extern "C" fn register_buttons(instance_ptr: *mut c_void, device_id: u64) { let mapped_device_id = if device_id == *HAND_TRACKER_LEFT_ID { *HAND_LEFT_ID } else if device_id == *HAND_TRACKER_RIGHT_ID { *HAND_RIGHT_ID } else { device_id }; for button_id in alvr_server_core::registered_button_set() { if let Some(info) = BUTTON_INFO.get(&button_id) { if info.device_id == mapped_device_id { unsafe { RegisterButton(instance_ptr, button_id) }; } } else { error!("Cannot register unrecognized button ID {button_id}"); } } } extern "C" fn send_haptics(device_id: u64, duration_s: f32, frequency: f32, amplitude: f32) { if let Ok(duration) = Duration::try_from_secs_f32(duration_s) && let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.send_haptics(Haptics { device_id, duration, frequency, amplitude, }); } } extern "C" fn set_video_config_nals(buffer_ptr: *const u8, len: i32, codec: i32) { let codec = if codec == 0 { CodecType::H264 } else if codec == 1 { CodecType::Hevc } else { CodecType::AV1 }; let mut config_buffer = vec![0; len as usize]; unsafe { ptr::copy_nonoverlapping(buffer_ptr, config_buffer.as_mut_ptr(), len as usize) }; if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.set_video_config_nals(config_buffer, codec); } } extern "C" fn send_video(timestamp_ns: u64, buffer_ptr: *mut u8, len: i32, is_idr: bool) { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { let timestamp = Duration::from_nanos(timestamp_ns); let buffer = unsafe { std::slice::from_raw_parts(buffer_ptr, len as usize) }; let Some(head_pose) = HEAD_POSE_QUEUE .lock() .iter() .find_map(|(ts, pose)| (*ts == timestamp).then_some(*pose)) else { // We can't submit the frame without its pose return; }; let local_views_params = LOCAL_VIEW_PARAMS.read(); let global_view_params = [ ViewParams { pose: head_pose * local_views_params[0].pose, fov: local_views_params[0].fov, }, ViewParams { pose: head_pose * local_views_params[1].pose, fov: local_views_params[1].fov, }, ]; context.send_video_nal(timestamp, global_view_params, is_idr, buffer.to_vec()); } } extern "C" fn get_dynamic_encoder_params() -> FfiDynamicEncoderParams { if let Some(context) = &*SERVER_CORE_CONTEXT.read() && let Some(params) = context.get_dynamic_encoder_params() { FfiDynamicEncoderParams { updated: 1, bitrate_bps: params.bitrate_bps as u64, framerate: params.framerate, } } else { FfiDynamicEncoderParams::default() } } extern "C" fn report_composed(timestamp_ns: u64, offset_ns: u64) { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.report_composed( Duration::from_nanos(timestamp_ns), Duration::from_nanos(offset_ns), ); } } extern "C" fn report_present(timestamp_ns: u64, offset_ns: u64) { if let Some(context) = &*SERVER_CORE_CONTEXT.read() { context.report_present( Duration::from_nanos(timestamp_ns), Duration::from_nanos(offset_ns), ); } } extern "C" fn wait_for_vsync() { // Default 120Hz-ish wait if StatisticsManager isn't up. // We use 120Hz-ish so that SteamVR doesn't accidentally get // any weird ideas about our display Hz with its frame pacing. static PRE_HEADSET_STATS_WAIT_INTERVAL: Duration = Duration::from_millis(8); // NB: don't sleep while locking SERVER_DATA_MANAGER or SERVER_CORE_CONTEXT let sleep_duration = SERVER_CORE_CONTEXT .read() .as_ref() .and_then(|ctx| ctx.duration_until_next_vsync()); if let Some(duration) = sleep_duration { if alvr_server_core::settings() .video .enforce_server_frame_pacing { thread::sleep(duration); } else { thread::yield_now(); } } else { // StatsManager isn't up because the headset hasn't connected, // safety fallback to prevent deadlocking. thread::sleep(PRE_HEADSET_STATS_WAIT_INTERVAL); } } pub extern "C" fn shutdown_driver() { SERVER_CORE_CONTEXT.write().take(); } /// This is the SteamVR/OpenVR entry point /// # Safety #[unsafe(no_mangle)] pub unsafe extern "C" fn HmdDriverFactory( interface_name: *const c_char, return_code: *mut i32, ) -> *mut c_void { let Ok(driver_dir) = alvr_server_io::get_driver_dir_from_registered() else { return ptr::null_mut(); }; let Some(filesystem_layout) = alvr_filesystem::filesystem_layout_from_openvr_driver_root_dir(&driver_dir) else { return ptr::null_mut(); }; let dashboard_process_paths = sysinfo::System::new_all() .processes_by_name(OsStr::new(&afs::dashboard_fname())) .filter_map(|proc| Some(proc.exe()?.to_owned())) .collect::>(); // Check that there is no active dashboard instance not part of this driver installation // Note: if the iterator is empty, `all()` returns true if !dashboard_process_paths .iter() .all(|path| *path == filesystem_layout.dashboard_exe()) { return ptr::null_mut(); } static ONCE: Once = Once::new(); ONCE.call_once(move || { alvr_server_core::initialize_environment(filesystem_layout.clone()); let log_to_disk = alvr_server_core::settings().extra.logging.log_to_disk; alvr_server_core::init_logging( log_to_disk.then(|| filesystem_layout.session_log()), Some(filesystem_layout.crash_log()), ); unsafe { g_sessionPath = CString::new(filesystem_layout.session().to_string_lossy().to_string()) .unwrap() .into_raw(); g_driverRootDir = CString::new( filesystem_layout .openvr_driver_root_dir .to_string_lossy() .to_string(), ) .unwrap() .into_raw(); }; graphics::initialize_shaders(); unsafe { LogError = Some(alvr_server_core::alvr_error); LogWarn = Some(alvr_server_core::alvr_warn); LogInfo = Some(alvr_server_core::alvr_info); LogDebug = Some(alvr_server_core::alvr_dbg_server_impl); LogEncoder = Some(alvr_server_core::alvr_dbg_encoder); LogPeriodically = Some(alvr_server_core::alvr_log_periodically); PathStringToHash = Some(alvr_server_core::alvr_path_to_id); GetSerialNumber = Some(props::get_serial_number); SetOpenvrProps = Some(props::set_device_openvr_props); RegisterButtons = Some(register_buttons); DriverReadyIdle = Some(driver_ready_idle); HapticsSend = Some(send_haptics); SetVideoConfigNals = Some(set_video_config_nals); VideoSend = Some(send_video); GetDynamicEncoderParams = Some(get_dynamic_encoder_params); ReportComposed = Some(report_composed); ReportPresent = Some(report_present); WaitForVSync = Some(wait_for_vsync); ShutdownRuntime = Some(shutdown_driver); // When there is already a ALVR dashboard running, initialize the HMD device early to // avoid buggy SteamVR behavior // NB: we already bail out before if the dashboards don't belong to this streamer let early_hmd_initialization = !dashboard_process_paths.is_empty(); CppInit(early_hmd_initialization); } let (context, events_receiver) = ServerCoreContext::new(); *SERVER_CORE_CONTEXT.write() = Some(context); event_loop(events_receiver); }); unsafe { CppOpenvrEntryPoint(interface_name, return_code) } } ================================================ FILE: alvr/server_openvr/src/props.rs ================================================ // Note: many properties are missing or are stubs. // todo: fill out more properties for headset and controllers // todo: add more emulation modes use crate::{ FfiOpenvrProperty, FfiOpenvrPropertyType_Bool, FfiOpenvrPropertyType_Double, FfiOpenvrPropertyType_Float, FfiOpenvrPropertyType_Int32, FfiOpenvrPropertyType_String, FfiOpenvrPropertyType_Uint64, FfiOpenvrPropertyType_Vector3, FfiOpenvrPropertyValue, }; use alvr_common::{ BODY_CHEST_ID, BODY_HIPS_ID, BODY_LEFT_ELBOW_ID, BODY_LEFT_FOOT_ID, BODY_LEFT_KNEE_ID, BODY_RIGHT_ELBOW_ID, BODY_RIGHT_FOOT_ID, BODY_RIGHT_KNEE_ID, DEVICE_ID_TO_PATH, HAND_LEFT_ID, HAND_RIGHT_ID, HAND_TRACKER_LEFT_ID, HAND_TRACKER_RIGHT_ID, HEAD_ID, debug, settings_schema::Switch, warn, }; use alvr_session::{ ControllersEmulationMode, HeadsetEmulationMode, OpenvrPropKey, OpenvrPropType, OpenvrProperty, }; use std::{ ffi::{CString, c_char, c_void}, ptr, }; pub fn set_openvr_prop(instance_ptr: Option<*mut c_void>, device_id: u64, prop: OpenvrProperty) { let key = prop.key as u32; let ty = alvr_session::openvr_prop_key_to_type(prop.key); let value = prop.value; let device_name = DEVICE_ID_TO_PATH.get(&device_id).unwrap_or(&"Unknown"); let (type_, maybe_value) = match ty { OpenvrPropType::Bool => ( FfiOpenvrPropertyType_Bool, value .parse::() .ok() .map(|bool_| FfiOpenvrPropertyValue { bool_: bool_.into(), }), ), OpenvrPropType::Float => ( FfiOpenvrPropertyType_Float, value .parse::() .ok() .map(|float_| FfiOpenvrPropertyValue { float_ }), ), OpenvrPropType::Int32 => ( FfiOpenvrPropertyType_Int32, value .parse::() .ok() .map(|int32| FfiOpenvrPropertyValue { int32 }), ), OpenvrPropType::Uint64 => ( FfiOpenvrPropertyType_Uint64, value .parse::() .ok() .map(|uint64| FfiOpenvrPropertyValue { uint64 }), ), OpenvrPropType::Vector3 => ( FfiOpenvrPropertyType_Vector3, serde_json::from_str::<[f32; 3]>(&value) .ok() .map(|vector3| FfiOpenvrPropertyValue { vector3 }), ), OpenvrPropType::Double => ( FfiOpenvrPropertyType_Double, value .parse::() .ok() .map(|double_| FfiOpenvrPropertyValue { double_ }), ), OpenvrPropType::String => { let c_string = CString::new(value.clone()).unwrap(); let mut string = [0; 256]; unsafe { ptr::copy_nonoverlapping( c_string.as_ptr(), string.as_mut_ptr(), c_string.as_bytes_with_nul().len(), ); } ( FfiOpenvrPropertyType_String, Some(FfiOpenvrPropertyValue { string }), ) } }; let Some(ffi_value) = maybe_value else { warn!( "Failed to parse {device_name} value for OpenVR property: {:?}={value}", prop.key ); return; }; debug!("Setting {device_name} OpenVR prop: {:?}={value}", prop.key); let ffi_prop = FfiOpenvrProperty { key, type_, value: ffi_value, }; if let Some(instance_ptr) = instance_ptr { unsafe { crate::SetOpenvrProperty(instance_ptr, ffi_prop) } } else { unsafe { crate::SetOpenvrPropByDeviceID(device_id, ffi_prop) } } } fn serial_number(device_id: u64) -> String { let settings = alvr_server_core::settings(); if device_id == *HEAD_ID { match &settings.headset.emulation_mode { HeadsetEmulationMode::RiftS => "1WMGH000XX0000".into(), HeadsetEmulationMode::Quest1 => "1PASH0X0X00000".into(), HeadsetEmulationMode::Quest2 => "1WMHH000X00000".into(), HeadsetEmulationMode::QuestPro => "230YC0XXXX00XX".into(), HeadsetEmulationMode::Pico4 => "VRLINKHMDPICO4".into(), HeadsetEmulationMode::Vive => "HTCVive-001".into(), HeadsetEmulationMode::Custom { serial_number, .. } => serial_number.clone(), } } else if device_id == *HAND_LEFT_ID || device_id == *HAND_RIGHT_ID { if let Switch::Enabled(controllers) = &settings.headset.controllers { let serial_number = match &controllers.emulation_mode { ControllersEmulationMode::Quest1Touch => "1PALCXXXX00000_Controller", // 1PALCLC left, 1PALCRC right ControllersEmulationMode::Quest2Touch => "1WMHH000X00000_Controller", ControllersEmulationMode::Quest3Plus => "2G0YXX0X0000XX_Controller", // 2G0YY Left 2G0YZ Right ControllersEmulationMode::QuestPro => "230YXXXXXXXXXX_Controller", // 230YT left, 230YV right ControllersEmulationMode::RiftSTouch | ControllersEmulationMode::Pico4 | ControllersEmulationMode::PSVR2Sense | ControllersEmulationMode::ValveIndex | ControllersEmulationMode::ViveWand | ControllersEmulationMode::ViveTracker => "ALVR Remote Controller", ControllersEmulationMode::Custom { serial_number, .. } => serial_number, }; if device_id == *HAND_LEFT_ID { format!("{serial_number}_Left") } else { format!("{serial_number}_Right") } } else { "Unknown".into() } } else if device_id == *HAND_TRACKER_LEFT_ID { "ALVR_Left_Hand_Full_Skeletal".into() } else if device_id == *HAND_TRACKER_RIGHT_ID { "ALVR_Right_Hand_Full_Skeletal".into() } else if device_id == *BODY_CHEST_ID { "ALVR Tracker (chest)".into() } else if device_id == *BODY_HIPS_ID { "ALVR Tracker (waist)".into() } else if device_id == *BODY_LEFT_ELBOW_ID { "ALVR Tracker (left elbow)".into() } else if device_id == *BODY_RIGHT_ELBOW_ID { "ALVR Tracker (right elbow)".into() } else if device_id == *BODY_LEFT_KNEE_ID { "ALVR Tracker (left knee)".into() } else if device_id == *BODY_RIGHT_KNEE_ID { "ALVR Tracker (right knee)".into() } else if device_id == *BODY_LEFT_FOOT_ID { "ALVR Tracker (left foot)".into() } else if device_id == *BODY_RIGHT_FOOT_ID { "ALVR Tracker (right foot)".into() } else { "Unknown".into() } } #[unsafe(no_mangle)] pub extern "C" fn get_serial_number(device_id: u64, out_str: *mut c_char) -> u64 { let string = serial_number(device_id); let cstring = CString::new(string).unwrap(); let len = cstring.to_bytes_with_nul().len(); if !out_str.is_null() { unsafe { ptr::copy_nonoverlapping(cstring.as_ptr(), out_str, len) }; } len as u64 } #[unsafe(no_mangle)] pub extern "C" fn set_device_openvr_props(instance_ptr: *mut c_void, device_id: u64) { #[expect(clippy::enum_glob_use)] use OpenvrPropKey::*; let settings = alvr_server_core::settings(); let set_prop = |key, value: &str| { set_openvr_prop( Some(instance_ptr), device_id, OpenvrProperty { key, value: value.into(), }, ); }; let set_icons = |base_path: &str| { set_prop( NamedIconPathDeviceOffString, format!("{base_path}_off.png").as_str(), ); set_prop( NamedIconPathDeviceSearchingString, format!("{base_path}_searching.gif").as_str(), ); set_prop( NamedIconPathDeviceSearchingAlertString, format!("{base_path}_searching_alert.gif").as_str(), ); set_prop( NamedIconPathDeviceReadyString, format!("{base_path}_ready.png").as_str(), ); set_prop( NamedIconPathDeviceReadyAlertString, format!("{base_path}_ready_alert.png").as_str(), ); set_prop( NamedIconPathDeviceAlertLowString, format!("{base_path}_ready_low.png").as_str(), ); set_prop( NamedIconPathDeviceStandbyString, format!("{base_path}_standby.png").as_str(), ); set_prop( NamedIconPathDeviceStandbyAlertString, format!("{base_path}_standby_alert.gif").as_str(), ); }; let device_serial = &serial_number(device_id); let headset_serial = &serial_number(*HEAD_ID); if device_id == *HEAD_ID { // Closure for all the common Quest headset properties let set_oculus_common_headset_props = || { set_prop( RegisteredDeviceTypeString, format!("oculus/{headset_serial}").as_str(), ); set_icons("{oculus}/icons/quest_headset"); }; // Per-device props match &settings.headset.emulation_mode { HeadsetEmulationMode::RiftS => { set_prop(TrackingSystemNameString, "oculus"); set_prop(ModelNumberString, "Oculus Rift S"); set_prop(ManufacturerNameString, "Oculus"); set_prop(RenderModelNameString, "generic_hmd"); set_prop(DriverVersionString, "1.42.0"); set_icons("{oculus}/icons/rifts_headset"); } HeadsetEmulationMode::Quest1 => { set_prop(TrackingSystemNameString, "oculus"); set_prop(ModelNumberString, "Oculus Quest"); set_prop(ManufacturerNameString, "Oculus"); set_prop(RenderModelNameString, "generic_hmd"); set_prop(DriverVersionString, "1.111.0"); set_oculus_common_headset_props(); } HeadsetEmulationMode::Quest2 => { set_prop(TrackingSystemNameString, "oculus"); set_prop(ModelNumberString, "Miramar"); set_prop(ManufacturerNameString, "Oculus"); set_prop(RenderModelNameString, "generic_hmd"); set_prop(DriverVersionString, "1.55.0"); set_oculus_common_headset_props(); } HeadsetEmulationMode::QuestPro => { set_prop(TrackingSystemNameString, "oculus"); set_prop(ModelNumberString, "Meta Quest Pro"); set_prop(ManufacturerNameString, "Oculus"); set_prop(RenderModelNameString, "generic_hmd"); set_prop(DriverVersionString, "1.55.0"); set_oculus_common_headset_props(); } HeadsetEmulationMode::Pico4 => { set_prop(TrackingSystemNameString, "vrlink"); set_prop(ModelNumberString, "PICO 4"); set_prop(ManufacturerNameString, "ByteDance"); set_prop(RenderModelNameString, "generic_hmd"); set_prop(RegisteredDeviceTypeString, "pico"); set_prop(DriverVersionString, ""); set_icons("{vrlink}/icons/headset_pico4"); } HeadsetEmulationMode::Vive => { set_prop(TrackingSystemNameString, "Vive Tracker"); set_prop(ModelNumberString, "ALVR driver server"); set_prop(ManufacturerNameString, "HTC"); set_prop(RenderModelNameString, "generic_hmd"); set_prop(RegisteredDeviceTypeString, "vive"); set_prop(DriverVersionString, ""); set_icons("{htc}/icons/vive_headset"); } HeadsetEmulationMode::Custom { .. } => (), } set_prop(UserIpdMetersFloat, "0.063"); set_prop(UserHeadToEyeDepthMetersFloat, "0.0"); set_prop(SecondsFromVsyncToPhotonsFloat, "0.0"); // return a constant that's not 0 (invalid) or 1 (reserved for Oculus) set_prop(CurrentUniverseIdUint64, "2"); if cfg!(windows) { // avoid "not fullscreen" warnings from vrmonitor set_prop(IsOnDesktopBool, "false"); // We let SteamVR handle VSyncs. We just wait in PostPresent(). set_prop(DriverDirectModeSendsVsyncEventsBool, "false"); } set_prop(DeviceProvidesBatteryStatusBool, "true"); set_prop(ContainsProximitySensorBool, "true"); for prop in &settings.headset.extra_openvr_props { set_prop(prop.key, &prop.value); } } else if device_id == *HAND_LEFT_ID || device_id == *HAND_RIGHT_ID || device_id == *HAND_TRACKER_LEFT_ID || device_id == *HAND_TRACKER_RIGHT_ID { let left_hand = device_id == *HAND_LEFT_ID || device_id == *HAND_TRACKER_LEFT_ID; let right_hand = device_id == *HAND_RIGHT_ID || device_id == *HAND_TRACKER_RIGHT_ID; let full_skeletal_hand = device_id == *HAND_TRACKER_LEFT_ID || device_id == *HAND_TRACKER_RIGHT_ID; if let Switch::Enabled(config) = &settings.headset.controllers { // Closure for all the common Oculus/Meta controller properties let set_oculus_common_props = || { set_prop(TrackingSystemNameString, "oculus"); set_prop(ControllerTypeString, "oculus_touch"); set_prop(InputProfilePathString, "{oculus}/input/touch_profile.json"); if left_hand { set_prop( RegisteredDeviceTypeString, format!("oculus/{headset_serial}_Controller_Left").as_str(), ); set_icons("{oculus}/icons/rifts_left_controller"); } else if right_hand { set_prop( RegisteredDeviceTypeString, format!("oculus/{headset_serial}_Controller_Right").as_str(), ); set_icons("{oculus}/icons/rifts_right_controller"); } }; // Controller-specific properties, not shared match config.emulation_mode { ControllersEmulationMode::RiftSTouch => { set_prop(ManufacturerNameString, "Oculus"); if left_hand { set_prop(ModelNumberString, "Oculus Rift S (Left Controller)"); set_prop(RenderModelNameString, "oculus_rifts_controller_left"); } else if right_hand { set_prop(ModelNumberString, "Oculus Rift S (Right Controller)"); set_prop(RenderModelNameString, "oculus_rifts_controller_right"); } set_prop(ControllerTypeString, "oculus_touch"); set_prop(InputProfilePathString, "{oculus}/input/touch_profile.json"); set_oculus_common_props(); } ControllersEmulationMode::Quest1Touch => { set_prop(ManufacturerNameString, "Oculus"); if left_hand { set_prop(ModelNumberString, "Oculus Quest (Left Controller)"); set_prop(RenderModelNameString, "oculus_quest_controller_left"); } else if right_hand { set_prop(ModelNumberString, "Oculus Quest (Right Controller)"); set_prop(RenderModelNameString, "oculus_quest_controller_right"); } set_prop(ControllerTypeString, "oculus_touch"); set_prop(InputProfilePathString, "{oculus}/input/touch_profile.json"); set_oculus_common_props(); } ControllersEmulationMode::Quest2Touch => { set_prop(ManufacturerNameString, "Oculus"); if left_hand { set_prop(ModelNumberString, "Miramar (Left Controller)"); set_prop(RenderModelNameString, "oculus_quest2_controller_left"); } else if right_hand { set_prop(ModelNumberString, "Miramar (Right Controller)"); set_prop(RenderModelNameString, "oculus_quest2_controller_right"); } set_prop(ControllerTypeString, "oculus_touch"); set_prop(InputProfilePathString, "{oculus}/input/touch_profile.json"); set_oculus_common_props(); } ControllersEmulationMode::Quest3Plus => { set_prop(ManufacturerNameString, "Meta"); if left_hand { set_prop(ModelNumberString, "Meta Quest 3 (Left Controller)"); set_prop(RenderModelNameString, "oculus_quest_plus_controller_left"); } else if right_hand { set_prop(ModelNumberString, "Meta Quest 3 (Right Controller)"); set_prop(RenderModelNameString, "oculus_quest_plus_controller_right"); } set_prop(ControllerTypeString, "oculus_touch"); set_prop(InputProfilePathString, "{oculus}/input/touch_profile.json"); set_oculus_common_props(); } ControllersEmulationMode::QuestPro => { set_prop(ManufacturerNameString, "Meta"); if left_hand { set_prop(ModelNumberString, "Meta Quest Pro (Left Controller)"); set_prop(RenderModelNameString, "oculus_quest_pro_controller_left"); } else if right_hand { set_prop(ModelNumberString, "Meta Quest Pro (Right Controller)"); set_prop(RenderModelNameString, "oculus_quest_pro_controller_right"); } set_oculus_common_props(); } ControllersEmulationMode::Pico4 => { set_prop(TrackingSystemNameString, "vrlink"); set_prop(ManufacturerNameString, "ByteDance"); if left_hand { set_prop(ModelNumberString, "PICO 4 (Left Controller)"); set_prop( RenderModelNameString, "{vrlink}/rendermodels/pico_4_controller_left", ); set_icons("{vrlink}/icons/left_pico4"); } else if right_hand { set_prop(ModelNumberString, "PICO 4 (Right Controller)"); set_prop( RenderModelNameString, "{vrlink}/rendermodels/pico_4_controller_right", ); set_icons("{vrlink}/icons/right_pico4"); } set_prop(ControllerTypeString, "pico_controller"); set_prop( InputProfilePathString, "{vrlink}/input/pico_controller_profile.json", ); } ControllersEmulationMode::PSVR2Sense => { set_prop(TrackingSystemNameString, "playstation_vr2"); if left_hand { set_prop(ModelNumberString, "PlayStation VR2 Sense Left"); set_prop(SerialNumberString, "playstation_vr2_sense_controller_left"); set_prop( RenderModelNameString, "{alvr_server}/rendermodels/playstation_vr2_sense_left", ); set_icons("{alvr_server}/icons/left_controller_status"); } else if right_hand { set_prop(ModelNumberString, "PlayStation VR2 Sense Right"); set_prop(SerialNumberString, "playstation_vr2_sense_controller_right"); set_prop( RenderModelNameString, "{alvr_server}/rendermodels/playstation_vr2_sense_right", ); set_icons("{alvr_server}/icons/right_controller_status"); } set_prop(TrackingFirmwareVersionString, "0303"); set_prop(HardwareRevisionString, "MP"); set_prop(ControllerTypeString, "playstation_vr2_sense"); set_prop( InputProfilePathString, "{alvr_server}/input/playstation_vr2_sense_controller_profile.json", ); } ControllersEmulationMode::ValveIndex => { set_prop(TrackingSystemNameString, "indexcontroller"); set_prop(ManufacturerNameString, "Valve"); if left_hand { set_prop(ModelNumberString, "Knuckles (Left Controller)"); set_prop( RenderModelNameString, "{indexcontroller}valve_controller_knu_1_0_left", ); set_prop( RegisteredDeviceTypeString, "valve/index_controllerLHR-E217CD00_Left", ); set_icons("{indexcontroller}/icons/left_controller_status"); } else if right_hand { set_prop(ModelNumberString, "Knuckles (Right Controller)"); set_prop( RenderModelNameString, "{indexcontroller}valve_controller_knu_1_0_right", ); set_prop( RegisteredDeviceTypeString, "valve/index_controllerLHR-E217CD00_Right", ); set_icons("{indexcontroller}/icons/right_controller_status"); } set_prop(ControllerTypeString, "knuckles"); set_prop( InputProfilePathString, "{indexcontroller}/input/index_controller_profile.json", ); } ControllersEmulationMode::ViveWand => { set_prop(TrackingSystemNameString, "htc"); set_prop(ManufacturerNameString, "HTC"); set_prop(RenderModelNameString, "vr_controller_vive_1_5"); if left_hand { set_prop( ModelNumberString, "ALVR Remote Controller (Left Controller)", ); set_prop(RegisteredDeviceTypeString, "htc/vive_controller_Left"); } else if right_hand { set_prop( ModelNumberString, "ALVR Remote Controller (Right Controller)", ); set_prop(RegisteredDeviceTypeString, "htc/vive_controller_Right"); } set_prop(ControllerTypeString, "vive_controller"); set_prop(InputProfilePathString, "{oculus}/input/touch_profile.json"); set_icons("{htc}/icons/controller"); } ControllersEmulationMode::ViveTracker => { set_prop(TrackingSystemNameString, "lighthouse"); set_prop(RenderModelNameString, "{htc}vr_tracker_vive_1_0"); if left_hand { set_prop(ModelNumberString, "Vive Tracker Pro MV (Left Controller)"); set_prop(RegisteredDeviceTypeString, "ALVR/tracker/left_foot"); set_prop(ControllerTypeString, "vive_tracker_left_foot"); } else if right_hand { set_prop(ModelNumberString, "Vive Tracker Pro MV (Right Controller)"); set_prop(RegisteredDeviceTypeString, "ALVR/tracker/right_foot"); set_prop(ControllerTypeString, "vive_tracker_right_foot"); } set_prop( InputProfilePathString, "{htc}/input/vive_tracker_profile.json", ); set_icons("{htc}/icons/tracker"); // All of these property values were dumped from real a vive tracker via // https://github.com/SDraw/openvr_dumper and were copied from // https://github.com/SDraw/driver_kinectV2 set_prop(ResourceRootString, "htc"); set_prop(WillDriftInYawBool, "false"); set_prop( TrackingFirmwareVersionString, "1541800000 RUNNER-WATCHMAN$runner-watchman@runner-watchman 2018-01-01 FPGA 512(2.56/0/0) BL 0 VRC 1541800000 Radio 1518800000", ); set_prop( HardwareRevisionString, "product 128 rev 2.5.6 lot 2000/0/0 0", ); set_prop(ConnectedWirelessDongleString, "D0000BE000"); set_prop(DeviceIsWirelessBool, "true"); set_prop(DeviceIsChargingBool, "false"); set_prop(ControllerHandSelectionPriorityInt32, "-1"); // vr::HmdMatrix34_t l_transform = { // {{-1.f, 0.f, 0.f, 0.f}, {0.f, 0.f, -1.f, 0.f}, {0.f, -1.f, 0.f, 0.f}}}; // vr_properties->SetProperty(this->prop_container, // vr::Prop_StatusDisplayTransform_Matrix34, // &l_transform, // sizeof(vr::HmdMatrix34_t), // vr::k_unHmdMatrix34PropertyTag); set_prop(FirmwareUpdateAvailableBool, "false"); set_prop(FirmwareManualUpdateBool, "false"); set_prop( FirmwareManualUpdateURLString, "https://developer.valvesoftware.com/wiki/SteamVR/HowTo_Update_Firmware", ); set_prop(HardwareRevisionUint64, "2214720000"); set_prop(FirmwareVersionUint64, "1541800000"); set_prop(FPGAVersionUint64, "512"); set_prop(VRCVersionUint64, "1514800000"); set_prop(RadioVersionUint64, "1518800000"); set_prop(DongleVersionUint64, "8933539758"); set_prop(DeviceCanPowerOffBool, "true"); // vr_properties->SetStringProperty(this->prop_container, // vr::Prop_Firmware_ProgrammingTargetString, // GetSerialNumber().c_str()); set_prop(FirmwareForceUpdateRequiredBool, device_serial); set_prop(FirmwareRemindUpdateBool, "false"); set_prop(HasDisplayComponentBool, "false"); set_prop(HasCameraComponentBool, "false"); set_prop(HasDriverDirectModeComponentBool, "false"); set_prop(HasVirtualDisplayComponentBool, "false"); } ControllersEmulationMode::Custom { .. } => {} } set_prop(SerialNumberString, device_serial); set_prop(AttachedDeviceIdString, device_serial); if full_skeletal_hand { set_prop(TrackingSystemNameString, "vrlink"); set_prop(ManufacturerNameString, "VRLink"); set_prop(RenderModelNameString, "{vrlink}/rendermodels/shuttlecock"); set_prop(ControllerTypeString, "svl_hand_interaction_augmented"); set_prop( InputProfilePathString, "{vrlink}/input/svl_hand_interaction_augmented_input_profile.json", ); if left_hand { set_prop(ModelNumberString, "VRLink Hand Tracker (Left Hand)"); set_prop( RegisteredDeviceTypeString, "vrlink/VRLINKQ_HandTracker_Left", ); set_prop(SerialNumberString, "VRLINKQ_Hand_Left"); set_prop(AttachedDeviceIdString, "VRLINKQ_Hand_Left"); set_icons("{vrlink}/icons/left_handtracking"); } else if right_hand { set_prop(ModelNumberString, "VRLink Hand Tracker (Right Hand)"); set_prop( RegisteredDeviceTypeString, "vrlink/VRLINKQ_HandTracker_Right", ); set_prop(SerialNumberString, "VRLINKQ_Hand_Right"); set_prop(AttachedDeviceIdString, "VRLINKQ_Hand_Right"); set_icons("{vrlink}/icons/right_handtracking"); } } set_prop( SupportedButtonsUint64, 0xFFFFFFFFFFFFFFFF_u64.to_string().as_str(), ); // OpenXR does not support controller battery set_prop(DeviceProvidesBatteryStatusBool, "false"); // k_eControllerAxis_Joystick = 2 set_prop(Axis0TypeInt32, "2"); if matches!(config.emulation_mode, ControllersEmulationMode::ViveTracker) { // TrackedControllerRole_Invalid set_prop(ControllerRoleHintInt32, "0"); } else if left_hand { // TrackedControllerRole_LeftHand set_prop(ControllerRoleHintInt32, "1"); } else if right_hand { // TrackedControllerRole_RightHand set_prop(ControllerRoleHintInt32, "2"); } for prop in &config.extra_openvr_props { set_prop(prop.key, &prop.value); } } } } ================================================ FILE: alvr/server_openvr/src/tracking.rs ================================================ use crate::{FfiDeviceMotion, FfiFov, FfiHandSkeleton, FfiPose, FfiQuat, FfiViewParams}; use alvr_common::{ BODY_CHEST_ID, BODY_HIPS_ID, BODY_LEFT_ELBOW_ID, BODY_LEFT_FOOT_ID, BODY_LEFT_KNEE_ID, BODY_RIGHT_ELBOW_ID, BODY_RIGHT_FOOT_ID, BODY_RIGHT_KNEE_ID, DeviceMotion, Fov, HAND_LEFT_ID, HAND_RIGHT_ID, Pose, ViewParams, glam::{EulerRot, Quat, Vec3}, settings_schema::Switch, }; use alvr_session::{ControllersConfig, HeadsetConfig}; use std::{ f32::consts::{FRAC_PI_2, PI}, sync::LazyLock, }; const DEG_TO_RAD: f32 = PI / 180.0; pub static BODY_TRACKER_IDS: LazyLock<[u64; 8]> = LazyLock::new(|| { [ // Upper body *BODY_CHEST_ID, *BODY_HIPS_ID, *BODY_LEFT_ELBOW_ID, *BODY_RIGHT_ELBOW_ID, // Legs *BODY_LEFT_KNEE_ID, *BODY_LEFT_FOOT_ID, *BODY_RIGHT_KNEE_ID, *BODY_RIGHT_FOOT_ID, ] }); fn to_ffi_fov(fov: Fov) -> FfiFov { FfiFov { left: fov.left, right: fov.right, up: fov.up, down: fov.down, } } fn to_ffi_quat(quat: Quat) -> FfiQuat { FfiQuat { x: quat.x, y: quat.y, z: quat.z, w: quat.w, } } fn to_ffi_pose(pose: Pose) -> FfiPose { FfiPose { orientation: to_ffi_quat(pose.orientation), position: pose.position.to_array(), } } pub fn to_ffi_motion(device_id: u64, motion: DeviceMotion) -> FfiDeviceMotion { FfiDeviceMotion { deviceID: device_id, pose: to_ffi_pose(motion.pose), linearVelocity: motion.linear_velocity.to_array(), angularVelocity: motion.angular_velocity.to_array(), } } pub fn to_ffi_view_params(params: ViewParams) -> FfiViewParams { FfiViewParams { pose: to_ffi_pose(params.pose), fov: to_ffi_fov(params.fov), } } fn get_hand_skeleton_offsets(config: &HeadsetConfig) -> (Pose, Pose) { let left_offset; let right_offset; if let Switch::Enabled(controllers) = &config.controllers { let t = controllers.left_hand_tracking_position_offset; let r = controllers.left_hand_tracking_rotation_offset; left_offset = Pose { orientation: Quat::from_euler( EulerRot::XYZ, r[0] * DEG_TO_RAD, r[1] * DEG_TO_RAD, r[2] * DEG_TO_RAD, ), position: Vec3::new(t[0], t[1], t[2]), }; right_offset = Pose { orientation: Quat::from_euler( EulerRot::XYZ, r[0] * DEG_TO_RAD, -r[1] * DEG_TO_RAD, -r[2] * DEG_TO_RAD, ), position: Vec3::new(-t[0], t[1], t[2]), }; } else { left_offset = Pose::IDENTITY; right_offset = Pose::IDENTITY; } (left_offset, right_offset) } fn to_ffi_skeleton(skeleton: &[Pose; 31]) -> FfiHandSkeleton { FfiHandSkeleton { jointRotations: skeleton .iter() .map(|j| to_ffi_quat(j.orientation)) .collect::>() .try_into() .unwrap(), jointPositions: skeleton .iter() .map(|j| j.position.to_array()) .collect::>() .try_into() .unwrap(), } } pub fn to_openvr_ffi_hand_skeleton( config: &HeadsetConfig, device_id: u64, hand_skeleton: &[Pose; 26], ) -> FfiHandSkeleton { let (left_hand_skeleton_offset, right_hand_skeleton_offset) = get_hand_skeleton_offsets(config); let id = device_id; let pose_offset = if id == *HAND_LEFT_ID { left_hand_skeleton_offset } else { right_hand_skeleton_offset }; // global joints let gj = hand_skeleton; // Correct the orientation for auxiliary bones. pub fn aux_orientation(id: u64, pose: Pose) -> Pose { let o = pose.orientation; let p = pose.position; // Convert to SteamVR basis orientations let (orientation, position) = if id == *HAND_LEFT_ID { ( Quat::from_xyzw(o.x, o.y, o.z, o.w) * Quat::from_euler(EulerRot::YXZ, -FRAC_PI_2, FRAC_PI_2, 0.0), Vec3::new(p.x, p.y, p.z), ) } else { ( Quat::from_xyzw(o.x, o.y, o.z, o.w) * Quat::from_euler(EulerRot::YXZ, FRAC_PI_2, -FRAC_PI_2, 0.0), Vec3::new(p.x, p.y, p.z), ) }; Pose { orientation, position, } } // Convert from global to local joint pose. The orientation frame of reference is also // converted from OpenXR to SteamVR (hand-specific!) pub fn local_pose(id: u64, parent: Pose, current: Pose) -> Pose { let o = parent.orientation.conjugate() * current.orientation; let p = parent.orientation.conjugate() * (current.position - parent.position); // Convert to SteamVR frame of reference let (orientation, position) = if id == *HAND_LEFT_ID { ( Quat::from_xyzw(-o.z, -o.y, -o.x, o.w), Vec3::new(-p.z, -p.y, -p.x), ) } else { ( Quat::from_xyzw(o.z, o.y, -o.x, o.w), Vec3::new(p.z, p.y, -p.x), ) }; Pose { orientation, position, } } // Adjust hand position based on the emulated controller for joints // parented to the root. let root_parented_pose = |pose: Pose| -> Pose { let sign = if id == *HAND_LEFT_ID { -1.0 } else { 1.0 }; let orientation = pose_offset.orientation.conjugate() * gj[0].orientation.conjugate() * pose.orientation * Quat::from_euler(EulerRot::XZY, PI, sign * FRAC_PI_2, 0.0); let position = -pose_offset.position + pose_offset.orientation.conjugate() * gj[0].orientation.conjugate() * (pose.position - gj[0].position); Pose { orientation, position, } }; let fixed_g_wrist = Pose { orientation: gj[1].orientation * Quat::from_euler(EulerRot::YXZ, -FRAC_PI_2, FRAC_PI_2, 0.0), position: gj[1].position, }; let skeleton = [ // Palm. NB: this is ignored by SteamVR Pose { orientation: gj[0].orientation * pose_offset.orientation, position: gj[0].position + gj[0].orientation * pose_offset.orientation * pose_offset.position, }, // Wrist root_parented_pose(gj[1]), // Thumb local_pose(id, fixed_g_wrist, gj[2]), local_pose(id, gj[2], gj[3]), local_pose(id, gj[3], gj[4]), local_pose(id, gj[4], gj[5]), // Index local_pose(id, fixed_g_wrist, gj[6]), local_pose(id, gj[6], gj[7]), local_pose(id, gj[7], gj[8]), local_pose(id, gj[8], gj[9]), local_pose(id, gj[9], gj[10]), // Middle local_pose(id, fixed_g_wrist, gj[11]), local_pose(id, gj[11], gj[12]), local_pose(id, gj[12], gj[13]), local_pose(id, gj[13], gj[14]), local_pose(id, gj[14], gj[15]), // Ring local_pose(id, fixed_g_wrist, gj[16]), local_pose(id, gj[16], gj[17]), local_pose(id, gj[17], gj[18]), local_pose(id, gj[18], gj[19]), local_pose(id, gj[19], gj[20]), // Little local_pose(id, fixed_g_wrist, gj[21]), local_pose(id, gj[21], gj[22]), local_pose(id, gj[22], gj[23]), local_pose(id, gj[23], gj[24]), local_pose(id, gj[24], gj[25]), // Aux bones aux_orientation(id, root_parented_pose(gj[4])), aux_orientation(id, root_parented_pose(gj[9])), aux_orientation(id, root_parented_pose(gj[14])), aux_orientation(id, root_parented_pose(gj[19])), aux_orientation(id, root_parented_pose(gj[24])), ]; to_ffi_skeleton(&skeleton) } // Apply controller offsets workarounds for SteamVR pub fn offset_controller_motion( config: &ControllersConfig, device_id: u64, motion: DeviceMotion, ) -> DeviceMotion { let t = config.left_controller_position_offset; let r = config.left_controller_rotation_offset; let pose_offset = if device_id == *HAND_LEFT_ID { Pose { orientation: Quat::from_euler( EulerRot::XYZ, r[0] * DEG_TO_RAD, r[1] * DEG_TO_RAD, r[2] * DEG_TO_RAD, ), position: Vec3::new(t[0], t[1], t[2]), } } else if device_id == *HAND_RIGHT_ID { Pose { orientation: Quat::from_euler( EulerRot::XYZ, r[0] * DEG_TO_RAD, -r[1] * DEG_TO_RAD, -r[2] * DEG_TO_RAD, ), position: Vec3::new(-t[0], t[1], t[2]), } } else { panic!("device_id is not associated to a controller"); }; DeviceMotion { pose: motion.pose * pose_offset, linear_velocity: motion.linear_velocity + motion .angular_velocity .cross(motion.pose.orientation * pose_offset.position), angular_velocity: motion.pose.orientation.conjugate() * motion.angular_velocity, } } ================================================ FILE: alvr/session/Cargo.toml ================================================ [package] name = "alvr_session" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_system_info.workspace = true bytemuck = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] } serde_json = "1" settings-schema = { git = "https://github.com/alvr-org/settings-schema-rs", rev = "676185f" } [build-dependencies] alvr_filesystem.workspace = true regex = "1" ================================================ FILE: alvr/session/build.rs ================================================ use regex::Regex; use std::{env, fmt::Write, fs, path::PathBuf}; fn main() { let openvr_driver_header_string = fs::read_to_string(alvr_filesystem::workspace_dir().join("openvr/headers/openvr_driver.h")) .expect("Missing openvr header files, did you clone the submodule?\n"); let property_finder = Regex::new( r"\tProp_([A-Za-z\d_]+)_(Bool|Int32|Uint64|Float|String|Vector3)[\t ]+= ([0-9]+)", ) .unwrap(); struct PropInfo { name: String, ty: String, code: String, } let prop_info = property_finder .captures_iter(&openvr_driver_header_string) .map(|cap| { let code = cap[3].into(); let name = format!("{}{}", cap[1].replace('_', ""), &cap[2]); PropInfo { name, ty: cap[2].into(), code, } }) .collect::>(); let mut mappings_fn_string: String = String::from( r" #[repr(u32)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy, Debug)] pub enum OpenvrPropKey {", ); for info in &prop_info { write!( mappings_fn_string, r" {} = {},", &info.name, &info.code ) .unwrap(); } mappings_fn_string.push_str( r" } #[expect(clippy::match_same_arms)] pub fn openvr_prop_key_to_type(key: OpenvrPropKey) -> OpenvrPropType { match key {", ); for info in &prop_info { write!( mappings_fn_string, r" OpenvrPropKey::{} => OpenvrPropType::{},", &info.name, info.ty ) .unwrap(); } mappings_fn_string.push_str( r" } } ", ); fs::write( PathBuf::from(env::var("OUT_DIR").unwrap()).join("openvr_property_keys.rs"), mappings_fn_string, ) .unwrap(); } ================================================ FILE: alvr/session/src/lib.rs ================================================ mod settings; pub use settings::*; pub use settings_schema; use alvr_common::{ ALVR_VERSION, ConnectionState, ToAny, anyhow::{Result, bail}, semver::Version, }; use serde::{Deserialize, Serialize}; use serde_json as json; use settings_schema::{NumberType, SchemaNode}; use std::{ collections::{HashMap, HashSet}, net::IpAddr, }; // SessionSettings is similar to Settings but it contains every branch, even unused ones. This is // the settings representation that the UI uses. pub type SessionSettings = settings::SettingsDefault; // This structure is used to store the minimum configuration data that ALVR driver needs to // initialize OpenVR before having the chance to communicate with a client. When a client is // connected, a new OpenvrConfig instance is generated, then the connection is accepted only if that // instance is equivalent to the one stored in the session, otherwise SteamVR is restarted. // Other components (like the encoder, audio recorder) don't need this treatment and are initialized // dynamically. // todo: properties that can be set after the OpenVR initialization should be removed and set with // UpdateForStream. #[expect(clippy::pub_underscore_fields)] #[derive(Serialize, Deserialize, PartialEq, Default, Clone, Debug)] pub struct OpenvrConfig { pub eye_resolution_width: u32, pub eye_resolution_height: u32, pub target_eye_resolution_width: u32, pub target_eye_resolution_height: u32, pub tracking_ref_only: bool, pub enable_vive_tracker_proxy: bool, pub minimum_idr_interval_ms: u64, pub adapter_index: u32, pub codec: u8, pub h264_profile: u32, pub refresh_rate: u32, pub use_10bit_encoder: bool, pub encoding_gamma: f32, pub enable_hdr: bool, pub force_hdr_srgb_correction: bool, pub clamp_hdr_extended_range: bool, pub enable_amf_pre_analysis: bool, pub enable_vbaq: bool, pub enable_amf_hmqb: bool, pub use_amf_preproc: bool, pub amf_preproc_sigma: u32, pub amf_preproc_tor: u32, pub encoder_quality_preset: u32, pub rate_control_mode: u32, pub filler_data: bool, pub entropy_coding: u32, pub force_sw_encoding: bool, pub sw_thread_count: u32, pub controller_is_tracker: bool, pub controllers_enabled: bool, pub body_tracking_vive_enabled: bool, pub body_tracking_has_legs: bool, pub enable_foveated_encoding: bool, pub foveation_center_size_x: f32, pub foveation_center_size_y: f32, pub foveation_center_shift_x: f32, pub foveation_center_shift_y: f32, pub foveation_edge_ratio_x: f32, pub foveation_edge_ratio_y: f32, pub enable_color_correction: bool, pub brightness: f32, pub contrast: f32, pub saturation: f32, pub gamma: f32, pub sharpening: f32, pub linux_async_compute: bool, pub linux_async_reprojection: bool, pub nvenc_quality_preset: u32, pub nvenc_tuning_preset: u32, pub nvenc_multi_pass: u32, pub nvenc_adaptive_quantization_mode: u32, pub nvenc_low_delay_key_frame_scale: i64, pub nvenc_refresh_rate: i64, pub enable_intra_refresh: bool, pub intra_refresh_period: i64, pub intra_refresh_count: i64, pub max_num_ref_frames: i64, pub gop_length: i64, pub p_frame_strategy: i64, pub nvenc_rate_control_mode: i64, pub rc_buffer_size: i64, pub rc_initial_delay: i64, pub rc_max_bitrate: i64, pub rc_average_bitrate: i64, pub nvenc_enable_weighted_prediction: bool, pub capture_frame_dir: String, pub amd_bitrate_corruption_fix: bool, pub use_separate_hand_trackers: bool, // these settings are not used on the C++ side, but we need them to correctly trigger a SteamVR // restart pub _controller_profile: i32, pub _server_impl_debug: bool, pub _client_impl_debug: bool, pub _server_core_debug: bool, pub _client_core_debug: bool, pub _connection_debug: bool, pub _sockets_debug: bool, pub _server_gfx_debug: bool, pub _client_gfx_debug: bool, pub _encoder_debug: bool, pub _decoder_debug: bool, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct ClientConnectionConfig { pub display_name: String, pub current_ip: Option, pub manual_ips: HashSet, pub trusted: bool, pub connection_state: ConnectionState, } #[derive(Serialize, Deserialize, Clone, Debug)] pub struct SessionConfig { pub server_version: Version, pub openvr_config: OpenvrConfig, // The hashmap key is the hostname pub client_connections: HashMap, pub session_settings: SessionSettings, } impl Default for SessionConfig { fn default() -> Self { Self { server_version: ALVR_VERSION.clone(), openvr_config: OpenvrConfig { // avoid realistic resolutions, as on first start, on Linux, it // could trigger direct mode on an existing monitor eye_resolution_width: 800, eye_resolution_height: 900, target_eye_resolution_width: 800, target_eye_resolution_height: 900, adapter_index: 0, refresh_rate: 60, controllers_enabled: false, body_tracking_vive_enabled: false, enable_foveated_encoding: false, enable_color_correction: false, linux_async_reprojection: false, capture_frame_dir: "/tmp".into(), ..<_>::default() }, client_connections: HashMap::new(), session_settings: settings::session_settings_default(), } } } impl SessionConfig { // If json_value is not a valid representation of SessionConfig (because of version upgrade), // use some fuzzy logic to extrapolate as much information as possible. // Since SessionConfig cannot have a schema (because SessionSettings would need to also have a // schema, but it is generated out of our control), we only do basic name checking on fields and // deserialization will fail if the type of values does not match. Because of this, // `session_settings` must be handled separately to do a better job of retrieving data using the // settings schema. pub fn merge_from_json(&mut self, json_value: &json::Value) -> Result<()> { const SESSION_SETTINGS_STR: &str = "session_settings"; if let Ok(session_desc) = json::from_value(json_value.clone()) { *self = session_desc; return Ok(()); } // Note: unwrap is safe because current session is expected to serialize correctly let old_session_json = json::to_value(self.clone()).unwrap(); let old_session_fields = old_session_json.as_object().unwrap(); let maybe_session_settings_json = json_value .get(SESSION_SETTINGS_STR) .map(|new_session_settings_json| { extrapolate_session_settings_from_session_settings( &old_session_fields[SESSION_SETTINGS_STR], new_session_settings_json, &Settings::schema(settings::session_settings_default()), ) }); let new_fields = old_session_fields .iter() .map(|(name, json_field_value)| { let new_json_field_value = if name == SESSION_SETTINGS_STR { json::to_value(settings::session_settings_default()).unwrap() } else { json_value.get(name).unwrap_or(json_field_value).clone() }; (name.clone(), new_json_field_value) }) .collect(); // Failure to extrapolate other session_desc fields is not notified. let mut session_desc_mut = json::from_value::(json::Value::Object(new_fields)).unwrap_or_default(); match maybe_session_settings_json .to_any() .and_then(|s| serde_json::from_value::(s).map_err(|e| e.into())) { Ok(session_settings) => { session_desc_mut.session_settings = session_settings; *self = session_desc_mut; Ok(()) } Err(e) => { *self = session_desc_mut; bail!("Error while deserializing extrapolated session settings: {e}") } } } pub fn to_settings(&self) -> Settings { let session_settings_json = json::to_value(&self.session_settings).unwrap(); let schema = Settings::schema(settings::session_settings_default()); json::from_value::(json_session_settings_to_settings( &session_settings_json, &schema, )) .map_err(|e| dbg!(e)) .unwrap() } } // Current data extrapolation strategy: match both field name and value type exactly. // Integer bounds are not validated, if they do not match the schema, deserialization will fail and // all data is lost. // Future strategies: check if value respects schema constraints, fuzzy field name matching, accept // integer to float and float to integer, tree traversal. fn extrapolate_session_settings_from_session_settings( old_session_settings: &json::Value, new_session_settings: &json::Value, schema: &SchemaNode, ) -> json::Value { match schema { SchemaNode::Section { entries, gui_collapsible, } => json::Value::Object({ let mut entries: json::Map = entries .iter() .map(|named_entry| { let value_json = extrapolate_session_settings_from_session_settings( &old_session_settings[&named_entry.name], &new_session_settings[&named_entry.name], &named_entry.content, ); (named_entry.name.clone(), value_json) }) .collect(); if *gui_collapsible { let collapsed_json = if new_session_settings["gui_collapsed"].is_boolean() { new_session_settings["gui_collapsed"].clone() } else { old_session_settings["gui_collapsed"].clone() }; entries.insert("gui_collapsed".into(), collapsed_json); } entries }), SchemaNode::Choice { variants, .. } => { let variant_json = json::from_value(new_session_settings["variant"].clone()) .ok() .filter(|variant_str| { variants .iter() .any(|named_entry| *variant_str == named_entry.name) }) .map_or_else( || old_session_settings["variant"].clone(), json::Value::String, ); let mut fields: json::Map<_, _> = variants .iter() .filter_map(|named_entry| { named_entry.content.as_ref().map(|data_schema| { let value_json = extrapolate_session_settings_from_session_settings( &old_session_settings[&named_entry.name], &new_session_settings[&named_entry.name], data_schema, ); (named_entry.name.clone(), value_json) }) }) .collect(); fields.insert("variant".into(), variant_json); json::Value::Object(fields) } SchemaNode::Optional { content, .. } => { let set_value = new_session_settings["set"] .as_bool() .unwrap_or_else(|| old_session_settings["set"].as_bool().unwrap()); let content_json = extrapolate_session_settings_from_session_settings( &old_session_settings["content"], &new_session_settings["content"], content, ); json::json!({ "set": set_value, "content": content_json }) } SchemaNode::Switch { content, .. } => { let enabled = new_session_settings["enabled"] .as_bool() .unwrap_or_else(|| old_session_settings["enabled"].as_bool().unwrap()); let content_json = extrapolate_session_settings_from_session_settings( &old_session_settings["content"], &new_session_settings["content"], content, ); json::json!({ "enabled": enabled, "content": content_json }) } SchemaNode::Boolean { .. } => { if new_session_settings.is_boolean() { new_session_settings.clone() } else { old_session_settings.clone() } } SchemaNode::Number { ty, .. } => { if let Some(value) = new_session_settings.as_f64() { match ty { NumberType::UnsignedInteger => json::Value::from(value.abs() as u64), NumberType::SignedInteger => json::Value::from(value as i64), NumberType::Float => new_session_settings.clone(), } } else { old_session_settings.clone() } } SchemaNode::Text { .. } => { if new_session_settings.is_string() { new_session_settings.clone() } else { old_session_settings.clone() } } SchemaNode::Array(array_schema) => { let gui_collapsed = new_session_settings["gui_collapsed"] .as_bool() .unwrap_or_else(|| old_session_settings["gui_collapsed"].as_bool().unwrap()); let array_vec = array_schema .iter() .enumerate() .map(|(idx, schema)| { extrapolate_session_settings_from_session_settings( &old_session_settings["content"][idx], &new_session_settings["content"][idx], schema, ) }) .collect::>(); json::json!({ "gui_collapsed": gui_collapsed, "content": array_vec }) } SchemaNode::Vector { default_element, .. } => { let gui_collapsed = new_session_settings["gui_collapsed"] .as_bool() .unwrap_or_else(|| old_session_settings["gui_collapsed"].as_bool().unwrap()); let element_json = extrapolate_session_settings_from_session_settings( &old_session_settings["element"], &new_session_settings["element"], default_element, ); let content_json = json::from_value::>(new_session_settings["content"].clone()) .ok() .map(|vec| { vec.iter() .enumerate() .map(|(idx, new_element)| { extrapolate_session_settings_from_session_settings( &old_session_settings["content"] .get(idx) .cloned() .unwrap_or_else(|| element_json.clone()), new_element, default_element, ) }) .collect() }) .map_or_else( || old_session_settings["content"].clone(), json::Value::Array, ); json::json!({ "gui_collapsed": gui_collapsed, "element": element_json, "content": content_json }) } SchemaNode::Dictionary { default_value, .. } => { let gui_collapsed = new_session_settings["gui_collapsed"] .as_bool() .unwrap_or_else(|| old_session_settings["gui_collapsed"].as_bool().unwrap()); let key_str = new_session_settings["key"] .as_str() .unwrap_or_else(|| old_session_settings["key"].as_str().unwrap()); let value_json = extrapolate_session_settings_from_session_settings( &old_session_settings["value"], &new_session_settings["value"], default_value, ); let content_json = json::from_value::>( new_session_settings["content"].clone(), ) .ok() .map(|map| { map.iter() .map(|(key, new_value)| { let value = extrapolate_session_settings_from_session_settings( &old_session_settings["content"] .get(key) .cloned() .unwrap_or_else(|| value_json.clone()), new_value, default_value, ); (key, value) }) .map(|(key, value)| { json::Value::Array(vec![json::Value::String(key.clone()), value]) }) .collect() }) .map_or_else( || old_session_settings["content"].clone(), json::Value::Array, ); json::json!({ "gui_collapsed": gui_collapsed, "key": key_str, "value": value_json, "content": content_json }) } _ => unreachable!(), } } // session_settings does not get validated here, it must be already valid fn json_session_settings_to_settings( session_settings: &json::Value, schema: &SchemaNode, ) -> json::Value { match schema { SchemaNode::Section { entries, .. } => json::Value::Object( entries .iter() .map(|named_entry| { ( named_entry.name.clone(), json_session_settings_to_settings( &session_settings[&named_entry.name], &named_entry.content, ), ) }) .collect(), ), SchemaNode::Choice { variants, .. } => { let variant = session_settings["variant"].as_str().unwrap(); let maybe_content = variants .iter() .find(|named_entry| named_entry.name == variant) .and_then(|named_entry| named_entry.content.as_ref()) .map(|data_schema| { json_session_settings_to_settings(&session_settings[variant], data_schema) }); if let Some(content) = maybe_content { json::json!({ variant: content }) } else { json::Value::String(variant.to_owned()) } } SchemaNode::Optional { content, .. } => { if session_settings["set"].as_bool().unwrap() { json_session_settings_to_settings(&session_settings["content"], content) } else { json::Value::Null } } SchemaNode::Switch { content, .. } => { if session_settings["enabled"].as_bool().unwrap() { let content = json_session_settings_to_settings(&session_settings["content"], content); json::json!({ "Enabled": content }) } else { json::Value::String("Disabled".into()) } } SchemaNode::Boolean { .. } | SchemaNode::Number { .. } | SchemaNode::Text { .. } => { session_settings.clone() } SchemaNode::Array(array_schema) => json::Value::Array( array_schema .iter() .enumerate() .map(|(idx, element_schema)| { json_session_settings_to_settings( &session_settings["content"][idx], element_schema, ) }) .collect(), ), SchemaNode::Vector { default_element, .. } => json::to_value( session_settings["content"] .as_array() .unwrap() .iter() .map(|element_json| { json_session_settings_to_settings(element_json, default_element) }) .collect::>(), ) .unwrap(), SchemaNode::Dictionary { default_value, .. } => json::to_value( json::from_value::>(session_settings["content"].clone()) .unwrap() .into_iter() .map(|(key, value_json)| { ( key, json_session_settings_to_settings(&value_json, default_value), ) }) .collect::>(), ) .unwrap(), _ => unreachable!(), } } #[cfg(test)] mod tests { use super::*; #[test] fn test_manual_session_to_settings() { let default = session_settings_default(); let settings_schema = json::to_value(&default).unwrap(); let schema = Settings::schema(default); let _settings = json::from_value::(json_session_settings_to_settings( &settings_schema, &schema, )) .err(); } #[test] fn test_session_to_settings() { let _settings = SessionConfig::default().to_settings(); } #[test] fn test_session_extrapolation_trivial() { SessionConfig::default() .merge_from_json(&json::to_value(SessionConfig::default()).unwrap()) .unwrap(); } #[test] fn test_session_extrapolation_diff() { let input_json_string = r#"{ "session_settings": { "fjdshfks": false, "video": { "preferred_fps": 60.0 }, "headset": { "gui_collapsed": false, "controllers": { "enabled": false } } } }"#; let mut session = SessionConfig::default(); session .merge_from_json(&json::from_str(input_json_string).unwrap()) .unwrap(); let settings = session.to_settings(); assert_eq!(settings.video.preferred_fps, 60.0); assert!(settings.headset.controllers.as_option().is_none()); } } ================================================ FILE: alvr/session/src/settings.rs ================================================ use alvr_common::{ ALVR_VERSION, DebugGroupsConfig, DebugGroupsConfigDefault, LogSeverity, LogSeverityDefault, LogSeverityDefaultVariant, }; use alvr_system_info::{ClientFlavor, ClientFlavorDefault, ClientFlavorDefaultVariant}; use bytemuck::{Pod, Zeroable}; use serde::{Deserialize, Serialize}; use settings_schema::{ ArrayDefault, DictionaryDefault, OptionalDefault, SettingsSchema, Switch, SwitchDefault, VectorDefault, }; include!(concat!(env!("OUT_DIR"), "/openvr_property_keys.rs")); pub enum OpenvrPropType { Bool, Float, Int32, Uint64, Vector3, Double, String, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Debug)] pub struct OpenvrProperty { pub key: OpenvrPropKey, pub value: String, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(gui = "button_group")] pub enum FrameSize { Scale(#[schema(gui(slider(min = 0.25, max = 2.0, step = 0.01)))] f32), Absolute { #[schema(gui(slider(min = 32, max = 8192, step = 32)))] width: u32, #[schema(gui(slider(min = 32, max = 8192, step = 32)))] height: Option, }, } #[repr(u32)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum EncoderQualityPreset { Quality = 0, Balanced = 1, Speed = 2, } #[repr(u32)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum EncoderQualityPresetNvidia { P1 = 1, P2 = 2, P3 = 3, P4 = 4, P5 = 5, P6 = 6, P7 = 7, } #[repr(u32)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum NvencTuningPreset { HighQuality = 1, LowLatency = 2, UltraLowLatency = 3, Lossless = 4, } #[repr(u32)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum NvencMultiPass { Disabled = 0, #[schema(strings(display_name = "1/4 resolution"))] QuarterResolution = 1, FullResolution = 2, } #[repr(u32)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum NvencAdaptiveQuantizationMode { Disabled = 0, Spatial = 1, Temporal = 2, } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(gui = "button_group")] pub enum RateControlMode { #[schema(strings(display_name = "CBR"))] Cbr = 0, #[schema(strings(display_name = "VBR"))] Vbr = 1, } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(gui = "button_group")] pub enum EntropyCoding { #[schema(strings(display_name = "CAVLC"))] Cavlc = 1, #[schema(strings(display_name = "CABAC"))] Cabac = 0, } /// Except for preset, the value of these fields is not applied if == -1 (flag) #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct NvencConfig { #[schema(strings( help = "P1 is the fastest preset and P7 is the preset that produces better quality. P6 and P7 are too slow to be usable." ))] #[schema(flag = "steamvr-restart")] pub quality_preset: EncoderQualityPresetNvidia, #[schema(flag = "steamvr-restart")] pub tuning_preset: NvencTuningPreset, #[schema(strings( help = "Reduce compression artifacts at the cost of small performance penalty" ))] #[schema(flag = "steamvr-restart")] pub multi_pass: NvencMultiPass, #[schema(strings( help = r#"Spatial: Helps reduce color banding, but high-complexity scenes might look worse. Temporal: Helps improve overall encoding quality, very small trade-off in speed."# ))] #[schema(flag = "steamvr-restart")] pub adaptive_quantization_mode: NvencAdaptiveQuantizationMode, #[schema(flag = "steamvr-restart")] pub low_delay_key_frame_scale: i64, #[schema(flag = "steamvr-restart")] pub refresh_rate: i64, #[schema(flag = "steamvr-restart")] pub enable_intra_refresh: bool, #[schema(flag = "steamvr-restart")] pub intra_refresh_period: i64, #[schema(flag = "steamvr-restart")] pub intra_refresh_count: i64, #[schema(flag = "steamvr-restart")] pub max_num_ref_frames: i64, #[schema(flag = "steamvr-restart")] pub gop_length: i64, #[schema(flag = "steamvr-restart")] pub p_frame_strategy: i64, #[schema(flag = "steamvr-restart")] pub rate_control_mode: i64, #[schema(flag = "steamvr-restart")] pub rc_buffer_size: i64, #[schema(flag = "steamvr-restart")] pub rc_initial_delay: i64, #[schema(flag = "steamvr-restart")] pub rc_max_bitrate: i64, #[schema(flag = "steamvr-restart")] pub rc_average_bitrate: i64, #[schema(flag = "steamvr-restart")] pub enable_weighted_prediction: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct AmfConfig { #[schema( strings( display_name = "Enable High-Motion Quality Boost", help = r#"Enables high motion quality boost mode. Allows the encoder to perform pre-analysis the motion of the video and use the information for better encoding"# ), flag = "steamvr-restart" )] pub enable_hmqb: bool, #[schema(flag = "steamvr-restart")] pub use_preproc: bool, #[schema(gui(slider(min = 0, max = 10)))] #[schema(flag = "steamvr-restart")] pub preproc_sigma: u32, #[schema(gui(slider(min = 0, max = 10)))] #[schema(flag = "steamvr-restart")] pub preproc_tor: u32, #[schema( strings( display_name = "Enable Pre-analysis", help = r#"Enables pre-analysis during encoding. This will likely result in reduced performance, but may increase quality. Does not work with the "Reduce color banding" option, requires enabling "Use preproc""# ), flag = "steamvr-restart" )] pub enable_pre_analysis: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct SoftwareEncodingConfig { #[schema(strings( display_name = "Force software encoding", help = "Forces the encoder to use CPU instead of GPU" ))] #[schema(flag = "steamvr-restart")] pub force_software_encoding: bool, #[schema(strings(display_name = "Encoder thread count"))] #[schema(flag = "steamvr-restart")] pub thread_count: u32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct HDRConfig { #[schema(strings( display_name = "Enable HDR", help = "If the client has no preference, enables compositing VR layers to an RGBA float16 framebuffer, and doing sRGB/YUV conversions in shader code." ))] #[schema(flag = "steamvr-restart")] pub enable: Option, #[schema(strings( display_name = "Force HDR sRGB Correction", help = "Forces sRGB correction on all composited SteamVR layers. Useful if an HDR-injected game is too dark." ))] #[schema(flag = "steamvr-restart")] pub force_hdr_srgb_correction: bool, #[schema(strings( display_name = "Clamp HDR extended range", help = "Clamps HDR extended range to 0.0~1.0, useful if you only want HDR to reduce banding." ))] #[schema(flag = "steamvr-restart")] pub clamp_hdr_extended_range: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct EncoderConfig { #[schema(flag = "steamvr-restart")] #[schema(strings( display_name = "Quality preset", help = "Controls overall quality preset of the encoder. Works only on Windows AMD AMF, Linux VAAPI (AMD/Intel)." ))] pub quality_preset: EncoderQualityPreset, #[schema( strings( display_name = "Enable VBAQ/CAQ", help = "Enables Variance Based Adaptive Quantization on h264 and HEVC, and Content Adaptive Quantization on AV1" ), flag = "steamvr-restart" )] pub enable_vbaq: bool, #[cfg_attr(not(target_os = "windows"), schema(flag = "hidden"))] #[schema(strings(help = r#"CBR: Constant BitRate mode. This is recommended. VBR: Variable BitRate mode. Not commended because it may throw off the adaptive bitrate algorithm. This is only supported on Windows and only with AMD/Nvidia GPUs"#))] #[schema(flag = "steamvr-restart")] pub rate_control_mode: RateControlMode, #[schema(strings( display_name = "h264: Profile", help = "Whenever possible, attempts to use this profile. May increase compatibility with varying mobile devices. Only has an effect for h264. Doesn't affect NVENC on Windows." ))] #[schema(flag = "steamvr-restart")] pub h264_profile: H264Profile, #[schema(strings(help = r#"CAVLC algorithm is recommended. CABAC produces better compression but it's significantly slower and may lead to runaway latency"#))] #[schema(flag = "steamvr-restart")] pub entropy_coding: EntropyCoding, #[schema(strings( help = r#"In CBR mode, this makes sure the bitrate does not fall below the assigned value. This is mostly useful for debugging."# ))] #[schema(flag = "steamvr-restart")] pub filler_data: bool, #[schema(strings( display_name = "10-bit encoding", help = "Sets the encoder to use 10 bits per channel instead of 8, if the client has no preference. Does not work on Linux with Nvidia" ))] #[schema(flag = "steamvr-restart")] pub use_10bit: Option, #[schema(strings( display_name = "Encoding Gamma", help = "To prioritize darker pixels at the expense of potentially additional banding in midtones, set to 2.2. To allow the encoder to decide priority on its own, set to 1.0." ))] #[schema(flag = "steamvr-restart")] pub encoding_gamma: Option, #[schema(strings(display_name = "HDR"))] #[schema(flag = "steamvr-restart")] pub hdr: HDRConfig, #[schema(strings(display_name = "NVENC"))] #[schema(flag = "steamvr-restart")] pub nvenc: NvencConfig, #[cfg_attr(not(target_os = "windows"), schema(flag = "hidden"))] #[schema(strings(display_name = "AMF"))] #[schema(flag = "steamvr-restart")] pub amf: AmfConfig, #[schema(strings(display_name = "Software (CPU) encoding"))] pub software: SoftwareEncodingConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Debug, PartialEq)] pub enum MediacodecPropType { Float, Int32, Int64, String, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Debug, PartialEq)] pub struct MediacodecProperty { #[schema(strings(display_name = "Type"))] pub ty: MediacodecPropType, pub value: String, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub struct EncoderLatencyLimiter { #[schema(strings( help = "Allowed percentage of frame interval to allocate for video encoding" ))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.3, max = 1.0, step = 0.01)))] pub max_saturation_multiplier: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct DecoderLatencyLimiter { #[schema(strings( display_name = "Maximum decoder latency", help = "When the decoder latency goes above this threshold, the bitrate will be reduced" ))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1, max = 50)), suffix = "ms")] pub max_decoder_latency_ms: u64, #[schema(strings( display_name = "latency overstep", help = "Number of consecutive frames above the threshold to trigger a bitrate reduction" ))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1, max = 100)), suffix = " frames")] pub latency_overstep_frames: usize, #[schema(strings( help = "Controls how much the bitrate is reduced when the decoder latency goes above the threshold" ))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.5, max = 1.0)))] pub latency_overstep_multiplier: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(gui = "button_group")] pub enum BitrateMode { #[schema(strings(display_name = "Constant"))] ConstantMbps(#[schema(gui(slider(min = 5, max = 1000, logarithmic)), suffix = "Mbps")] u64), #[schema(collapsible)] Adaptive { #[schema(strings( help = "Percentage of network bandwidth to allocate for video transmission" ))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.5, max = 5.0, step = 0.01)))] saturation_multiplier: f32, #[schema(strings(display_name = "Maximum bitrate"))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1, max = 1000, logarithmic)), suffix = "Mbps")] max_throughput_mbps: Switch, #[schema(strings(display_name = "Minimum bitrate"))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1, max = 100, logarithmic)), suffix = "Mbps")] min_throughput_mbps: Switch, #[schema(strings(display_name = "Maximum network latency"))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1, max = 50)), suffix = "ms")] max_network_latency_ms: Switch, #[schema(flag = "real-time")] encoder_latency_limiter: Switch, #[schema(strings( help = "Currently there is a bug where the decoder latency keeps rising when above a certain bitrate" ))] #[schema(flag = "real-time")] decoder_latency_limiter: Switch, }, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub struct BitrateAdaptiveFramerateConfig { #[schema(strings( display_name = "FPS reset threshold multiplier", help = "If the framerate changes more than this factor, trigger a parameters update", ))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1.0, max = 3.0, step = 0.1)))] pub framerate_reset_threshold_multiplier: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct BitrateConfig { #[schema(flag = "real-time")] pub mode: BitrateMode, #[schema(strings( help = "Ensure that the specified bitrate value is respected regardless of the framerate" ))] #[schema(flag = "real-time")] pub adapt_to_framerate: Switch, #[schema(strings(help = "Controls the smoothness during calculations"))] pub history_size: usize, #[schema(strings( help = "When this is enabled, an IDR frame is requested after the bitrate is changed. This has an effect only on AMD GPUs." ))] #[schema(flag = "steamvr-restart")] pub image_corruption_fix: bool, } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Copy, Clone, PartialEq)] pub enum ClientsideFoveationLevel { Low = 1, Medium = 2, High = 3, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum ClientsideFoveationMode { Static { level: ClientsideFoveationLevel }, Dynamic { max_level: ClientsideFoveationLevel }, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub struct ClientsideFoveationConfig { pub mode: ClientsideFoveationMode, #[schema(strings(display_name = "Foveation offset"))] #[schema(gui(slider(min = -45.0, max = 45.0, step = 0.1)), suffix = "°")] pub vertical_offset_deg: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(collapsible)] pub struct FoveatedEncodingConfig { #[schema(strings(help = "Force enable on smartphone clients"))] pub force_enable: bool, #[schema(strings(display_name = "Center region width"))] #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub center_size_x: f32, #[schema(strings(display_name = "Center region height"))] #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub center_size_y: f32, #[schema(strings(display_name = "Center shift X"))] #[schema(gui(slider(min = -1.0, max = 1.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub center_shift_x: f32, #[schema(strings(display_name = "Center shift Y"))] #[schema(gui(slider(min = -1.0, max = 1.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub center_shift_y: f32, #[schema(strings(display_name = "Horizontal edge ratio"))] #[schema(gui(slider(min = 1.0, max = 10.0, step = 1.0)))] #[schema(flag = "steamvr-restart")] pub edge_ratio_x: f32, #[schema(strings(display_name = "Vertical edge ratio"))] #[schema(gui(slider(min = 1.0, max = 10.0, step = 1.0)))] #[schema(flag = "steamvr-restart")] pub edge_ratio_y: f32, } #[repr(C)] #[derive(SettingsSchema, Clone, Copy, Serialize, Deserialize, Pod, Zeroable)] pub struct ColorCorrectionConfig { #[schema(gui(slider(min = -1.0, max = 1.0, step = 0.001)))] #[schema(flag = "steamvr-restart")] pub brightness: f32, #[schema(gui(slider(min = -1.0, max = 1.0, step = 0.001)))] #[schema(flag = "steamvr-restart")] pub contrast: f32, #[schema(gui(slider(min = -1.0, max = 1.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub saturation: f32, #[schema(gui(slider(min = 0.0, max = 5.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub gamma: f32, #[schema(gui(slider(min = -1.0, max = 5.0, step = 0.01)))] #[schema(flag = "steamvr-restart")] pub sharpening: f32, } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Debug, Copy, Clone, PartialEq, Eq, Default)] #[schema(gui = "button_group")] pub enum CodecType { #[default] #[schema(strings(display_name = "h264"))] H264 = 0, #[schema(strings(display_name = "HEVC"))] Hevc = 1, #[schema(strings(display_name = "AV1"))] AV1 = 2, } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Debug, Copy, Clone, PartialEq, Eq)] #[schema(gui = "button_group")] pub enum H264Profile { #[schema(strings(display_name = "High"))] High = 0, #[schema(strings(display_name = "Main"))] Main = 1, #[schema(strings(display_name = "Baseline"))] Baseline = 2, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Debug)] pub struct RgbChromaKeyConfig { #[schema(flag = "real-time")] #[schema(gui(slider(min = 0, max = 255)))] pub red: u8, #[schema(flag = "real-time")] #[schema(gui(slider(min = 0, max = 255)))] pub green: u8, #[schema(flag = "real-time")] #[schema(gui(slider(min = 0, max = 255)))] pub blue: u8, #[schema(strings(help = "The threshold is applied per-channel"))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 1, max = 255)))] pub distance_threshold: u8, #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.01, max = 1.0, step = 0.01)))] pub feathering: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Debug)] pub struct HsvChromaKeyConfig { #[schema(strings(display_name = "Hue start max"), suffix = "°")] #[schema(flag = "real-time")] #[schema(gui(slider(min = -179.0, max = 539.0, step = 1.0)))] pub hue_start_max_deg: f32, #[schema(strings(display_name = "Hue start min"), suffix = "°")] #[schema(flag = "real-time")] #[schema(gui(slider(min = -179.0, max = 539.0, step = 1.0)))] pub hue_start_min_deg: f32, #[schema(strings(display_name = "Hue end min"), suffix = "°")] #[schema(flag = "real-time")] #[schema(gui(slider(min = -179.0, max = 539.0, step = 1.0)))] pub hue_end_min_deg: f32, #[schema(strings(display_name = "Hue end max"), suffix = "°")] #[schema(flag = "real-time")] #[schema(gui(slider(min = -179.0, max = 539.0, step = 1.0)))] pub hue_end_max_deg: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub saturation_start_max: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub saturation_start_min: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub saturation_end_min: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5,step = 0.01)))] pub saturation_end_max: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub value_start_max: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub value_start_min: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub value_end_min: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -0.5, max = 1.5, step = 0.01)))] pub value_end_max: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Debug)] #[schema(gui = "button_group")] pub enum PassthroughMode { Blend { #[schema(strings( help = "Enabling this will adapt transparency based on the brightness of each pixel. This is a similar effect to AR glasses." ))] #[schema(flag = "real-time")] premultiplied_alpha: bool, #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] threshold: f32, }, #[schema(strings(display_name = "RGB Chroma Key"))] RgbChromaKey(#[schema(flag = "real-time")] RgbChromaKeyConfig), #[schema(strings(display_name = "HSV Chroma Key"))] HsvChromaKey(#[schema(flag = "real-time")] HsvChromaKeyConfig), } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Debug, Copy, Clone, PartialEq, Eq)] #[schema(gui = "button_group")] pub enum ClientsidePostProcessingSuperSamplingMode { Disabled = 0, Normal = 1 << 0, Quality = 1 << 1, } #[repr(u8)] #[derive(SettingsSchema, Serialize, Deserialize, Debug, Copy, Clone, PartialEq, Eq)] #[schema(gui = "button_group")] pub enum ClientsidePostProcessingSharpeningMode { Disabled = 0, Normal = 1 << 2, Quality = 1 << 3, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Debug)] pub struct ClientsidePostProcessingConfig { #[schema(strings( help = "Reduce flicker for high contrast edges.\nUseful when the input resolution is high compared to the headset display" ))] pub super_sampling: ClientsidePostProcessingSuperSamplingMode, #[schema(strings( help = "Improve clarity of high contrast edges and counteract blur.\nUseful when the input resolution is low compared to the headset display" ))] pub sharpening: ClientsidePostProcessingSharpeningMode, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Debug)] pub struct UpscalingConfig { #[schema(strings( help = "Improves visual quality by using the edge direction to upscale at a slight performance loss" ))] pub edge_direction: bool, #[schema(gui(slider(min = 1.0, max = 16.0, step = 1.0)))] pub edge_threshold: f32, #[schema(gui(slider(min = 1.0, max = 2.0, step = 0.01)))] pub edge_sharpness: f32, #[schema(gui(slider(min = 1.0, max = 3.0, step = 0.01)))] #[schema(strings( help = "Dimensional resolution multiplier, high values will cause performance issues with weaker headset hardware or higher resolutions" ))] pub upscale_factor: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct VideoConfig { #[schema(flag = "real-time")] pub passthrough: Switch, pub bitrate: BitrateConfig, #[schema(strings( help = "HEVC may provide better visual fidelity at the cost of increased encoder latency" ))] #[schema(flag = "steamvr-restart")] pub preferred_codec: CodecType, #[schema(strings( notice = r"Disabling foveated encoding may result in significantly higher encode/decode times and stuttering, or even crashing. If you want to reduce the amount of pixelation on the edges, increase the center region width and height" ))] #[schema(flag = "steamvr-restart")] pub foveated_encoding: Switch, #[schema(flag = "steamvr-restart")] pub color_correction: Switch, #[schema( strings( display_name = "Maximum buffering", help = "Increasing this value will help reduce stutter but it will increase latency" ), gui(slider(min = 1.0, max = 10.0, step = 0.1, logarithmic)), suffix = " frames" )] pub max_buffering_frames: f32, #[schema(gui(slider(min = 0.50, max = 0.99, step = 0.01)))] pub buffering_history_weight: f32, #[cfg_attr(not(target_os = "windows"), schema(flag = "hidden"))] #[schema(strings( help = r"This works only on Windows. It shouldn't be disabled except in certain circumstances when you know the VR game will not meet the target framerate." ))] #[schema(flag = "real-time")] pub enforce_server_frame_pacing: bool, #[schema(flag = "steamvr-restart")] pub encoder_config: EncoderConfig, #[schema(strings( help = "Attempts to use a software decoder on the device. Slow, but may work around broken codecs." ))] pub force_software_decoder: bool, pub mediacodec_extra_options: Vec<(String, MediacodecProperty)>, #[schema(strings( help = "Resolution used for encoding and decoding. Relative to a single eye view." ))] #[schema(flag = "steamvr-restart")] pub transcoding_view_resolution: FrameSize, #[schema(strings( help = "This is the resolution that SteamVR will use as default for the game rendering. Relative to a single eye view." ))] #[schema(flag = "steamvr-restart")] pub emulated_headset_view_resolution: FrameSize, #[schema(strings(display_name = "Preferred FPS"))] #[schema(gui(slider(min = 60.0, max = 120.0)), suffix = "Hz")] #[schema(flag = "steamvr-restart")] pub preferred_fps: f32, #[cfg_attr(not(target_os = "windows"), schema(flag = "hidden"))] #[schema(strings( help = "You probably don't want to change this. Allows for changing adapter for ALVR compositor." ))] #[schema(flag = "steamvr-restart")] pub adapter_index: u32, #[schema(strings(display_name = "Client-side foveation"))] pub clientside_foveation: Switch, #[schema(strings( display_name = "Client-side post-processing", help = "Hardware optimized algorithms, available on Quest and Pico headsets" ))] #[schema(flag = "real-time")] pub clientside_post_processing: Switch, #[schema(strings(help = "Snapdragon Game Super Resolution client-side upscaling"))] pub upscaling: Switch, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(gui = "button_group")] pub enum CustomAudioDeviceConfig { #[schema(strings(display_name = "By name (substring)"))] NameSubstring(String), #[schema(strings(display_name = "By index"))] Index(usize), } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct AudioBufferingConfig { #[schema(strings(display_name = "Average buffering"))] #[schema(gui(slider(min = 0, max = 200)), suffix = "ms")] pub average_buffering_ms: u64, #[schema(strings(display_name = "Batch size"))] #[schema(gui(slider(min = 1, max = 20)), suffix = "ms")] pub batch_ms: u64, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct GameAudioConfig { #[cfg_attr(target_os = "linux", schema(flag = "hidden"))] pub device: Option, #[cfg_attr(target_os = "linux", schema(flag = "hidden"))] #[schema(strings(display_name = "Mute desktop audio when streaming"))] pub mute_when_streaming: bool, pub buffering: AudioBufferingConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub enum MicrophoneDevicesConfig { Automatic, #[schema(strings(display_name = "Virtual Audio Cable"))] VAC, #[schema(strings(display_name = "VB Cable"))] VBCable, #[schema(strings(display_name = "VoiceMeeter"))] VoiceMeeter, #[schema(strings(display_name = "VoiceMeeter Aux"))] VoiceMeeterAux, #[schema(strings(display_name = "VoiceMeeter VAIO3"))] VoiceMeeterVaio3, Custom { #[schema(strings(help = "This device is used by ALVR to output microphone audio"))] sink: CustomAudioDeviceConfig, #[schema(strings(help = "This device is set in SteamVR as the default microphone"))] source: CustomAudioDeviceConfig, }, } // Note: sample rate is a free parameter for microphone, because both server and client supports // resampling. In contrary, for game audio, the server does not support resampling. #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct MicrophoneConfig { #[cfg_attr(target_os = "linux", schema(flag = "hidden"))] pub devices: MicrophoneDevicesConfig, pub buffering: AudioBufferingConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct AudioConfig { #[schema(strings(display_name = "Headset speaker"))] pub game_audio: Switch, #[cfg_attr( windows, schema(strings( display_name = "Headset microphone", notice = r"To be able to use the microphone on Windows, you need to install Virtual Audio Cable" )) )] #[cfg_attr(not(windows), schema(strings(display_name = "Headset microphone")))] pub microphone: Switch, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub enum HeadsetEmulationMode { #[schema(strings(display_name = "Rift S"))] RiftS, #[schema(strings(display_name = "Quest 1"))] Quest1, #[schema(strings(display_name = "Quest 2"))] Quest2, #[schema(strings(display_name = "Quest Pro"))] QuestPro, #[schema(strings(display_name = "Pico 4"))] Pico4, Vive, Custom { serial_number: String, }, } #[derive(SettingsSchema, Serialize, Deserialize, PartialEq, Clone)] pub enum PerformanceLevel { #[schema(strings(display_name = "Power Saving"))] PowerSavings, #[schema(strings(display_name = "Sustained Low"))] SustainedLow, #[schema(strings(display_name = "Sustained High"))] SustainedHigh, #[schema(flag = "hidden")] Boost, } #[derive(SettingsSchema, Serialize, Deserialize, PartialEq, Clone)] pub struct PerformanceLevelConfig { #[schema(flag = "real-time")] #[schema(strings( display_name = "CPU", help = "When disabling this, the client needs to be restarted for the change to be applied." ))] pub cpu: Switch, #[schema(flag = "real-time")] #[schema(strings( display_name = "GPU", help = "When disabling this, the client needs to be restarted for the change to be applied." ))] pub gpu: Switch, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub enum FaceTrackingSourcesConfig { PreferEyeTrackingOnly, PreferFullFaceTracking, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub enum FaceTrackingSinkConfig { #[schema(strings(display_name = "VRChat Eye OSC"))] VrchatEyeOsc { port: u16 }, #[schema(strings(display_name = "VRCFaceTracking"))] VrcFaceTracking, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct FaceTrackingConfig { pub sources: FaceTrackingSourcesConfig, pub sink: FaceTrackingSinkConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Default)] pub struct BodyTrackingMetaConfig { pub prefer_full_body: bool, #[schema(strings(help = "Prefer active upper body tracking, Quest 3 only"))] pub prefer_high_fidelity: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] #[schema(gui = "button_group")] pub enum BodyTrackingBDConfig { #[schema(strings(display_name = "Body Tracking"))] BodyTracking { #[schema(strings( help = "Improves accuracy of the tracking at the cost of higher latency." ))] high_accuracy: bool, #[schema(strings( help = "If trackers have not been calibrated before, the calibration process will start after you connect to the streamer." ))] prompt_calibration_on_start: bool, }, #[schema(strings(display_name = "Object Tracking"))] ObjectTracking, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq)] pub struct BodyTrackingSourcesConfig { pub meta: BodyTrackingMetaConfig, pub bd: BodyTrackingBDConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub enum BodyTrackingSinkConfig { #[schema(strings(display_name = "Fake Vive Trackers"))] FakeViveTracker, #[schema(strings(display_name = "VRChat Body OSC"))] VrchatBodyOsc { port: u16 }, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct BodyTrackingConfig { pub sources: BodyTrackingSourcesConfig, pub sink: BodyTrackingSinkConfig, #[schema(strings(help = "Turn this off to temporarily pause tracking."))] #[schema(flag = "real-time")] pub tracked: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct VMCConfig { pub host: String, pub port: u16, #[schema(strings(help = "Turn this off to temporarily pause sending data."))] #[schema(flag = "real-time")] pub publish: bool, #[schema(flag = "real-time")] pub orientation_correction: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, PartialEq, Eq)] pub enum ControllersEmulationMode { #[schema(strings(display_name = "Rift S Touch"))] RiftSTouch, #[schema(strings(display_name = "Quest 1 Touch"))] Quest1Touch, #[schema(strings(display_name = "Quest 2 Touch"))] Quest2Touch, #[schema(strings(display_name = "Quest 3 Touch Plus"))] Quest3Plus, #[schema(strings(display_name = "Quest Pro"))] QuestPro, #[schema(strings(display_name = "Pico 4"))] Pico4, #[schema(strings(display_name = "PSVR2 Sense Controller"))] PSVR2Sense, #[schema(strings(display_name = "Valve Index"))] ValveIndex, #[schema(strings(display_name = "Vive Wand"))] ViveWand, #[schema(strings(display_name = "Vive Tracker"))] ViveTracker, Custom { serial_number: String, button_set: Vec, }, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub struct HysteresisThreshold { #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub value: f32, #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub deviation: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub struct BinaryToScalarStates { #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub off: f32, #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub on: f32, } // Remaps 0..1 to custom range #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub struct Range { #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub min: f32, #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub max: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub enum ButtonMappingType { Passthrough, HysteresisThreshold(HysteresisThreshold), BinaryToScalar(BinaryToScalarStates), Remap(Range), } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct ButtonBindingTarget { pub destination: String, pub mapping_type: ButtonMappingType, pub binary_conditions: Vec, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct AutomaticButtonMappingConfig { pub click_threshold: HysteresisThreshold, pub touch_threshold: HysteresisThreshold, pub force_threshold: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct HandTrackingInteractionConfig { #[schema(flag = "real-time")] pub only_touch: bool, #[schema(flag = "real-time")] #[schema(strings( help = "How close the tips of your fingers need to be to register a pinch click." ))] #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)), suffix = "cm")] pub pinch_touch_distance: f32, #[schema(flag = "real-time")] #[schema(strings( help = "How close together the tips of your fingers need to be to start registering a pinch trigger pull." ))] #[schema(gui(slider(min = 0.0, max = 2.5, step = 0.025)), suffix = "cm")] pub pinch_trigger_distance: f32, #[schema(flag = "real-time")] #[schema(strings( help = "How close to your palm the tips of your fingers need to be to register a curl click." ))] #[schema(gui(slider(min = 0.0, max = 5.0)), suffix = "cm")] pub curl_touch_distance: f32, #[schema(flag = "real-time")] #[schema(strings( help = "How close to your palm the tips of your fingers need to be to start registering a trigger pull." ))] #[schema(gui(slider(min = 0.0, max = 10.0)), suffix = "cm")] pub curl_trigger_distance: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.0, max = 100.0)), suffix = "%")] pub joystick_deadzone: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -5.0, max = 5.0)), suffix = "cm")] pub joystick_offset_horizontal: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = -5.0, max = 5.0)), suffix = "cm")] pub joystick_offset_vertical: f32, #[schema(flag = "real-time")] #[schema(strings( help = "The radius of motion of the joystick. The joystick can be controlled if the thumb is within 2x this range." ))] #[schema(gui(slider(min = 0.0, max = 5.0)), suffix = "cm")] pub joystick_range: f32, #[schema(flag = "real-time")] #[schema(strings( help = "How long the gesture must be continuously held before it is activated." ))] #[schema(gui(slider(min = 0, max = 1000)), suffix = "ms")] pub activation_delay: u32, #[schema(flag = "real-time")] #[schema(strings( help = "How long the gesture must be continuously released before it is deactivated." ))] #[schema(gui(slider(min = 0, max = 1000)), suffix = "ms")] pub deactivation_delay: u32, #[schema(flag = "real-time")] #[schema(strings( help = "How long the after the gesture has been deactivated before it can be activated again." ))] #[schema(gui(slider(min = 0, max = 1000)), suffix = "ms")] pub repeat_delay: u32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct HapticsConfig { #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.0, max = 5.0, step = 0.1)))] pub intensity_multiplier: f32, #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)))] pub amplitude_curve: f32, #[schema(strings(display_name = "Minimum duration"))] #[schema(flag = "real-time")] #[schema(gui(slider(min = 0.0, max = 0.1, step = 0.001)), suffix = "s")] pub min_duration_s: f32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct HandSkeletonConfig { #[schema(flag = "steamvr-restart")] #[schema(strings( display_name = "SteamVR input 2.0", help = r"Enabling this will use separate tracker objects with the full skeletal tracking level when hand tracking is detected. This is required for VRChat hand tracking." ))] pub steamvr_input_2_0: bool, #[schema(flag = "real-time")] #[schema(strings( help = r"Predict hand skeleton to make it less floaty. It may make hands too jittery." ))] pub predict: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] #[schema(collapsible)] pub struct ControllersConfig { #[schema(strings(help = "Turning this off will make the controllers appear powered off."))] #[schema(flag = "real-time")] pub tracked: bool, #[schema(flag = "steamvr-restart")] #[schema(strings( help = "Enabling this passes skeletal hand data (finger tracking) to SteamVR." ))] pub hand_skeleton: Switch, #[schema(flag = "real-time")] #[schema(strings( help = "Enabling this allows using hand gestures to emulate controller inputs." ))] pub hand_tracking_interaction: Switch, #[schema(strings( display_name = "Prediction", help = r"Higher values make the controllers track smoother. Technically, this is the time (counted in frames) between pose submitted to SteamVR and the corresponding virtual vsync happens. Currently this cannot be reliably estimated automatically. The correct value should be 2 but 3 is default for smoother tracking at the cost of slight lag." ))] #[schema(gui(slider(min = 1.0, max = 10.0, logarithmic)), suffix = "frames")] pub steamvr_pipeline_frames: f32, #[schema(flag = "real-time")] pub haptics: Switch, #[schema(flag = "steamvr-restart")] pub emulation_mode: ControllersEmulationMode, #[schema(flag = "steamvr-restart")] #[schema(strings(display_name = "Extra OpenVR properties"))] pub extra_openvr_props: Vec, #[schema(flag = "real-time")] // note: logarithmic scale seems to be glitchy for this control #[schema(gui(slider(min = 0.0, max = 1.0, step = 0.01)), suffix = "m/s")] pub linear_velocity_cutoff: f32, #[schema(flag = "real-time")] // note: logarithmic scale seems to be glitchy for this control #[schema(gui(slider(min = 0.0, max = 100.0, step = 1.0)), suffix = "°/s")] pub angular_velocity_cutoff: f32, #[schema(flag = "real-time")] #[schema(strings(help = "Right controller offset is mirrored horizontally"))] // note: logarithmic scale seems to be glitchy for this control #[schema(gui(slider(min = -0.5, max = 0.5, step = 0.001)), suffix = "m")] pub left_controller_position_offset: [f32; 3], #[schema(flag = "real-time")] #[schema(strings(help = "Right controller offset is mirrored horizontally"))] #[schema(gui(slider(min = -180.0, max = 180.0, step = 1.0)), suffix = "°")] pub left_controller_rotation_offset: [f32; 3], #[schema(flag = "real-time")] #[schema(strings(help = "Right controller offset is mirrored horizontally"))] // note: logarithmic scale seems to be glitchy for this control #[schema(gui(slider(min = -0.5, max = 0.5, step = 0.001)), suffix = "m")] pub left_hand_tracking_position_offset: [f32; 3], #[schema(flag = "real-time")] #[schema(strings(help = "Right controller offset is mirrored horizontally"))] #[schema(gui(slider(min = -180.0, max = 180.0, step = 1.0)), suffix = "°")] pub left_hand_tracking_rotation_offset: [f32; 3], #[schema(strings(help = "List of OpenXR-syle paths"))] pub button_mappings: Option)>>, pub button_mapping_config: AutomaticButtonMappingConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub enum PositionRecenteringMode { Disabled, LocalFloor, Local { #[schema(gui(slider(min = 0.0, max = 3.0)), suffix = "m")] view_height: f32, }, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub enum RotationRecenteringMode { Disabled, Yaw, Tilted, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub struct MultimodalTracking { pub enabled: bool, #[schema(flag = "steamvr-restart")] #[schema(strings( display_name = "Map non-held controllers to SteamVR trackers", help = "Non-held controllers are mapped to left and right feet. This will be configurable in the future." ))] pub detached_controllers_steamvr_sink: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct HeadsetConfig { #[schema(strings( help = r#"Disabled: the playspace origin is determined by the room-scale guardian setup. Local floor: the origin is on the floor and resets when long pressing the oculus button. Local: the origin resets when long pressing the oculus button, and is calculated as an offset from the current head position."# ))] #[schema(flag = "real-time")] pub position_recentering_mode: PositionRecenteringMode, #[schema(strings( help = r#"Disabled: the playspace orientation is determined by the room-scale guardian setup. Yaw: the forward direction is reset when long pressing the oculus button. Tilted: the world gets tilted when long pressing the oculus button. This is useful for using VR while laying down."# ))] #[schema(flag = "real-time")] pub rotation_recentering_mode: RotationRecenteringMode, #[schema(flag = "steamvr-restart")] pub controllers: Switch, #[schema(flag = "steamvr-restart")] pub emulation_mode: HeadsetEmulationMode, #[schema(strings( help = r#"Power Savings might increase latency or reduce framerate consistency but decreases temperatures and improves battery life. Sustained Low provides consistent framerates but might increase latency if necessary. Sustained High provides consistent framerates but increases temperature. This is mainly for Quest headsets, mileage may vary on other devices."# ))] pub performance_level: PerformanceLevelConfig, #[schema(flag = "steamvr-restart")] #[schema(strings(display_name = "Extra OpenVR properties"))] pub extra_openvr_props: Vec, #[schema(flag = "steamvr-restart")] pub tracking_ref_only: bool, #[schema(flag = "steamvr-restart")] pub enable_vive_tracker_proxy: bool, pub face_tracking: Switch, #[schema(flag = "steamvr-restart")] #[schema(strings( help = r"Track hand skeleton while holding controllers. This will reduce hand tracking frequency to 30Hz. Because of runtime limitations, this option is ignored when body tracking is active." ))] pub multimodal_tracking: Switch, #[schema(flag = "steamvr-restart")] pub body_tracking: Switch, #[schema(flag = "steamvr-restart")] #[schema(strings(display_name = "VMC"))] pub vmc: Switch, #[schema(strings( help = "Maximum prediction for head and controllers. Used to avoid too much jitter during loading." ))] #[schema(gui(slider(min = 0, max = 200, step = 5)), suffix = "ms")] pub max_prediction_ms: u64, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] #[schema(gui = "button_group")] pub enum SocketProtocol { #[schema(strings(display_name = "UDP"))] Udp, #[schema(strings(display_name = "TCP"))] Tcp, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct DiscoveryConfig { #[cfg_attr(target_os = "linux", schema(flag = "hidden"))] #[schema(strings( help = "Allow untrusted clients to connect without confirmation. This is not recommended for security reasons." ))] pub auto_trust_clients: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy, Default)] pub enum SocketBufferSize { #[default] Default, Maximum, Custom(#[schema(suffix = "B")] u32), } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy, Default)] pub struct SocketBufferConfig { #[schema(strings(display_name = "Send size"))] pub send_size_bytes: SocketBufferSize, #[schema(strings(display_name = "Receive size"))] pub recv_size_bytes: SocketBufferSize, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct WiredClientAutoLaunchConfig { #[schema(strings( help = "Delay in seconds to wait after booting the headset before trying to launch the client." ))] pub boot_delay: u32, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct ConnectionConfig { #[schema(strings( help = r#"UDP: Faster, but less stable than TCP. Try this if your network is well optimized and free of interference. TCP: Slower than UDP, but more stable. Pick this if you experience video or audio stutters with UDP."# ))] pub stream_protocol: SocketProtocol, pub client_discovery: Switch, #[schema(strings( help = r#"Which release type of client should ALVR look for when establishing a wired connection."# ))] pub wired_client_type: ClientFlavor, #[schema(strings( help = r#"Wether ALVR should try to automatically launch the client when establishing a wired connection."# ))] pub wired_client_autolaunch: Switch, #[cfg_attr( windows, schema(strings( help = "If on_connect.bat exists alongside session.json, it will be run on headset connect. Env var ACTION will be set to `connect`." )) )] #[cfg_attr( not(windows), schema(strings( help = "If on_connect.sh exists alongside session.json, it will be run on headset connect. Env var ACTION will be set to `connect`." )) )] pub enable_on_connect_script: bool, #[cfg_attr( windows, schema(strings( help = "If on_disconnect.bat exists alongside session.json, it will be run on headset disconnect. Env var ACTION will be set to `disconnect`." )) )] #[cfg_attr( not(windows), schema(strings( help = "If on_disconnect.sh exists alongside session.json, it will be run on headset disconnect. Env var ACTION will be set to `disconnect`." )) )] #[schema(flag = "real-time")] pub enable_on_disconnect_script: bool, #[schema(strings( display_name = "Allow untrusted HTTP", help = "Allow cross-origin browser requests to control ALVR settings remotely." ))] pub allow_untrusted_http: bool, #[schema(strings( help = r#"If the client, server or the network discarded one packet, discard packets until a IDR packet is found."# ))] pub avoid_video_glitching: bool, #[schema(gui(slider(min = 1024, max = 65507, logarithmic)), suffix = "B")] pub packet_size: i32, pub stream_port: u16, pub web_server_port: u16, #[schema(strings(display_name = "Local OSC port"))] pub osc_local_port: u16, pub server_buffer_config: SocketBufferConfig, pub client_buffer_config: SocketBufferConfig, #[schema(strings( help = r#"The server discards video packets if it can't push them to the network. This could happen on TCP. A IDR frame is requested in this case."# ))] pub max_queued_server_video_frames: usize, #[schema(suffix = " frames")] pub statistics_history_size: usize, #[schema(strings(display_name = "Minimum IDR interval"))] #[schema(flag = "steamvr-restart")] #[schema(gui(slider(min = 5, max = 1000, step = 5)), suffix = "ms")] pub minimum_idr_interval_ms: u64, #[schema(strings(display_name = "DSCP (packet prio hints)"))] pub dscp: Option, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] #[repr(u8)] #[schema(gui = "button_group")] pub enum DropProbability { Low = 0x01, Medium = 0x10, High = 0x11, } #[derive(SettingsSchema, Serialize, Deserialize, Clone, Copy)] pub enum DscpTos { BestEffort, ClassSelector(#[schema(gui(slider(min = 1, max = 7)))] u8), AssuredForwarding { #[schema(gui(slider(min = 1, max = 4)))] class: u8, drop_probability: DropProbability, }, ExpeditedForwarding, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct RawEventsConfig { #[schema(flag = "real-time")] pub hide_spammy_events: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct LoggingConfig { #[schema(strings(help = "Notification tips teach you how to use ALVR"))] pub show_notification_tip: bool, #[schema(strings(help = "This applies only to certain error or warning messages."))] #[schema(flag = "steamvr-restart")] pub prefer_backtrace: bool, #[schema(flag = "real-time")] pub notification_level: LogSeverity, pub client_log_report_level: Switch, #[schema(flag = "real-time")] pub show_raw_events: Switch, #[schema(strings(help = "Write logs into the session_log.txt file."))] pub log_to_disk: bool, #[schema(flag = "real-time")] pub log_tracking: bool, #[schema(flag = "real-time")] pub log_button_presses: bool, #[schema(flag = "real-time")] pub log_haptics: bool, #[cfg_attr(not(debug_assertions), schema(flag = "hidden"))] #[schema(strings(help = "These settings enable extra spammy logs for debugging purposes."))] pub debug_groups: DebugGroupsConfig, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct SteamvrLauncher { #[schema(strings( display_name = "Open and close SteamVR automatically", help = "Launches SteamVR automatically when the ALVR dashboard is opened, and closes it when the dashboard is closed." ))] pub open_close_steamvr_with_dashboard: bool, #[cfg_attr( windows, schema(strings(help = "Directly start the VR server, bypassing Steam. \ Will run start_server.bat if it exists alongside session.json, and try to automatically find SteamVR otherwise.")) )] #[cfg_attr( not(windows), schema(strings(help = "Directly start the VR server, bypassing Steam. \ Will run start_server.sh if it exists alongside session.json, and try to automatically find SteamVR otherwise.")) )] pub direct_launch: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct RollingVideoFilesConfig { #[schema(strings(display_name = "Duration"))] #[schema(suffix = "s")] pub duration_s: u64, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct CaptureConfig { #[schema(strings(display_name = "Start video recording at client connection"))] pub startup_video_recording: bool, pub rolling_video_files: Switch, #[schema(flag = "steamvr-restart")] pub capture_frame_dir: String, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct Patches { #[schema(strings( help = "Async Compute is currently broken in SteamVR, keep disabled. ONLY FOR TESTING." ))] #[schema(flag = "steamvr-restart")] pub linux_async_compute: bool, #[schema(strings( help = "Async reprojection only works if you can always hit at least half of your refresh rate.", ))] #[schema(flag = "steamvr-restart")] pub linux_async_reprojection: bool, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct NewVersionPopupConfig { pub hide_while_version: String, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct ExtraConfig { #[schema(strings(display_name = "SteamVR Launcher"))] pub steamvr_launcher: SteamvrLauncher, pub capture: CaptureConfig, pub logging: LoggingConfig, #[cfg_attr(not(target_os = "linux"), schema(flag = "hidden"))] pub patches: Patches, #[schema( strings(help = "Linear and angular velocity multiplier for debug purposes. It does not update in real time.") )] pub velocities_multiplier: f32, pub open_setup_wizard: bool, pub new_version_popup: Switch, } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub struct Settings { pub video: VideoConfig, pub audio: AudioConfig, pub headset: HeadsetConfig, pub connection: ConnectionConfig, pub extra: ExtraConfig, } pub fn session_settings_default() -> SettingsDefault { let view_resolution = FrameSizeDefault { variant: FrameSizeDefaultVariant::Absolute, Scale: 1.0, Absolute: FrameSizeAbsoluteDefault { width: 2144, height: OptionalDefault { set: false, content: 1072, }, }, }; let default_custom_audio_device = CustomAudioDeviceConfigDefault { NameSubstring: "".into(), Index: 0, variant: CustomAudioDeviceConfigDefaultVariant::NameSubstring, }; let default_custom_openvr_props = VectorDefault { gui_collapsed: true, element: OpenvrPropertyDefault { key: OpenvrPropKeyDefault { variant: OpenvrPropKeyDefaultVariant::TrackingSystemNameString, }, value: "".into(), }, content: vec![], }; let socket_buffer = SocketBufferSizeDefault { Custom: 100000, variant: SocketBufferSizeDefaultVariant::Maximum, }; let socket_buffer_config = SocketBufferConfigDefault { send_size_bytes: socket_buffer.clone(), recv_size_bytes: socket_buffer, }; SettingsDefault { video: VideoConfigDefault { passthrough: SwitchDefault { enabled: false, content: PassthroughModeDefault { variant: PassthroughModeDefaultVariant::Blend, Blend: PassthroughModeBlendDefault { premultiplied_alpha: true, threshold: 0.5, }, RgbChromaKey: RgbChromaKeyConfigDefault { red: 0, green: 255, blue: 0, distance_threshold: 85, feathering: 0.05, }, HsvChromaKey: HsvChromaKeyConfigDefault { hue_start_max_deg: 70.0, hue_start_min_deg: 80.0, hue_end_min_deg: 160.0, hue_end_max_deg: 170.0, saturation_start_max: 0.2, saturation_start_min: 0.3, saturation_end_min: 1.0, saturation_end_max: 1.1, value_start_max: 0.0, value_start_min: 0.1, value_end_min: 1.0, value_end_max: 1.1, }, }, }, clientside_post_processing: SwitchDefault { enabled: false, content: ClientsidePostProcessingConfigDefault { super_sampling: ClientsidePostProcessingSuperSamplingModeDefault { variant: ClientsidePostProcessingSuperSamplingModeDefaultVariant::Quality, }, sharpening: ClientsidePostProcessingSharpeningModeDefault { variant: ClientsidePostProcessingSharpeningModeDefaultVariant::Quality, }, }, }, upscaling: SwitchDefault { enabled: false, content: UpscalingConfigDefault { edge_direction: true, edge_threshold: 4.0, edge_sharpness: 2.0, upscale_factor: 1.5, }, }, adapter_index: 0, transcoding_view_resolution: view_resolution.clone(), emulated_headset_view_resolution: view_resolution, preferred_fps: 72., max_buffering_frames: 2.0, buffering_history_weight: 0.90, enforce_server_frame_pacing: true, bitrate: BitrateConfigDefault { gui_collapsed: false, mode: BitrateModeDefault { ConstantMbps: 30, Adaptive: BitrateModeAdaptiveDefault { gui_collapsed: true, saturation_multiplier: 0.95, max_throughput_mbps: SwitchDefault { enabled: false, content: 100, }, min_throughput_mbps: SwitchDefault { enabled: false, content: 5, }, max_network_latency_ms: SwitchDefault { enabled: false, content: 8, }, encoder_latency_limiter: SwitchDefault { enabled: true, content: EncoderLatencyLimiterDefault { max_saturation_multiplier: 0.9, }, }, decoder_latency_limiter: SwitchDefault { enabled: true, content: DecoderLatencyLimiterDefault { gui_collapsed: false, max_decoder_latency_ms: 30, latency_overstep_frames: 90, latency_overstep_multiplier: 0.99, }, }, }, variant: BitrateModeDefaultVariant::ConstantMbps, }, adapt_to_framerate: SwitchDefault { enabled: false, content: BitrateAdaptiveFramerateConfigDefault { framerate_reset_threshold_multiplier: 2.0, }, }, history_size: 256, image_corruption_fix: false, }, preferred_codec: CodecTypeDefault { variant: CodecTypeDefaultVariant::H264, }, encoder_config: EncoderConfigDefault { gui_collapsed: true, rate_control_mode: RateControlModeDefault { variant: RateControlModeDefaultVariant::Cbr, }, filler_data: false, h264_profile: H264ProfileDefault { variant: H264ProfileDefaultVariant::High, }, entropy_coding: EntropyCodingDefault { variant: EntropyCodingDefaultVariant::Cavlc, }, use_10bit: OptionalDefault { set: false, content: false, }, encoding_gamma: OptionalDefault { set: false, content: 1.0, }, hdr: HDRConfigDefault { gui_collapsed: true, enable: OptionalDefault { set: false, content: false, }, force_hdr_srgb_correction: false, clamp_hdr_extended_range: false, }, nvenc: NvencConfigDefault { gui_collapsed: true, quality_preset: EncoderQualityPresetNvidiaDefault { variant: EncoderQualityPresetNvidiaDefaultVariant::P1, }, tuning_preset: NvencTuningPresetDefault { variant: NvencTuningPresetDefaultVariant::LowLatency, }, multi_pass: NvencMultiPassDefault { variant: NvencMultiPassDefaultVariant::QuarterResolution, }, adaptive_quantization_mode: NvencAdaptiveQuantizationModeDefault { variant: NvencAdaptiveQuantizationModeDefaultVariant::Spatial, }, low_delay_key_frame_scale: -1, refresh_rate: -1, enable_intra_refresh: false, intra_refresh_period: -1, intra_refresh_count: -1, max_num_ref_frames: -1, gop_length: -1, p_frame_strategy: -1, rate_control_mode: -1, rc_buffer_size: -1, rc_initial_delay: -1, rc_max_bitrate: -1, rc_average_bitrate: -1, enable_weighted_prediction: false, }, quality_preset: EncoderQualityPresetDefault { variant: EncoderQualityPresetDefaultVariant::Speed, }, enable_vbaq: false, amf: AmfConfigDefault { gui_collapsed: true, enable_pre_analysis: false, enable_hmqb: false, use_preproc: false, preproc_sigma: 4, preproc_tor: 7, }, software: SoftwareEncodingConfigDefault { gui_collapsed: true, force_software_encoding: false, thread_count: 0, }, }, mediacodec_extra_options: { fn int32_default(int32: i32) -> MediacodecPropertyDefault { MediacodecPropertyDefault { ty: MediacodecPropTypeDefault { variant: MediacodecPropTypeDefaultVariant::Int32, }, value: int32.to_string(), } } DictionaryDefault { gui_collapsed: true, key: "".into(), value: int32_default(0), content: vec![ ("operating-rate".into(), int32_default(i32::MAX)), ("priority".into(), int32_default(0)), // low-latency: only applicable on API level 30. Quest 1 and 2 might not be // cabable, since they are on level 29. // ("low-latency".into(), int32_default(1)), // Android smartphones crashes enabling this feature (https://github.com/PhoneVR-Developers/alvr-cardboard/issues/5) ( "vendor.qti-ext-dec-low-latency.enable".into(), int32_default(1), ), ], } }, foveated_encoding: SwitchDefault { enabled: true, content: FoveatedEncodingConfigDefault { gui_collapsed: true, force_enable: false, center_size_x: 0.45, center_size_y: 0.4, center_shift_x: 0.4, center_shift_y: 0.1, edge_ratio_x: 4., edge_ratio_y: 5., }, }, clientside_foveation: SwitchDefault { enabled: false, content: ClientsideFoveationConfigDefault { mode: ClientsideFoveationModeDefault { Static: ClientsideFoveationModeStaticDefault { level: ClientsideFoveationLevelDefault { variant: ClientsideFoveationLevelDefaultVariant::High, }, }, Dynamic: ClientsideFoveationModeDynamicDefault { max_level: ClientsideFoveationLevelDefault { variant: ClientsideFoveationLevelDefaultVariant::High, }, }, variant: ClientsideFoveationModeDefaultVariant::Dynamic, }, vertical_offset_deg: 0.0, }, }, force_software_decoder: false, color_correction: SwitchDefault { enabled: false, content: ColorCorrectionConfigDefault { brightness: 0., contrast: 0., saturation: 0.5, gamma: 1., sharpening: 0.5, }, }, }, audio: AudioConfigDefault { game_audio: SwitchDefault { enabled: true, content: GameAudioConfigDefault { gui_collapsed: true, device: OptionalDefault { set: false, content: default_custom_audio_device.clone(), }, mute_when_streaming: true, buffering: AudioBufferingConfigDefault { gui_collapsed: true, average_buffering_ms: 50, batch_ms: 10, }, }, }, microphone: SwitchDefault { enabled: cfg!(target_os = "linux"), content: MicrophoneConfigDefault { gui_collapsed: true, devices: MicrophoneDevicesConfigDefault { Custom: MicrophoneDevicesConfigCustomDefault { source: default_custom_audio_device.clone(), sink: default_custom_audio_device, }, variant: MicrophoneDevicesConfigDefaultVariant::Automatic, }, buffering: AudioBufferingConfigDefault { gui_collapsed: true, average_buffering_ms: 50, batch_ms: 10, }, }, }, }, headset: HeadsetConfigDefault { emulation_mode: HeadsetEmulationModeDefault { Custom: HeadsetEmulationModeCustomDefault { serial_number: "Unknown".into(), }, variant: HeadsetEmulationModeDefaultVariant::Quest2, }, performance_level: PerformanceLevelConfigDefault { cpu: SwitchDefault { enabled: false, content: PerformanceLevelDefault { variant: PerformanceLevelDefaultVariant::PowerSavings, }, }, gpu: SwitchDefault { enabled: false, content: PerformanceLevelDefault { variant: PerformanceLevelDefaultVariant::PowerSavings, }, }, }, extra_openvr_props: default_custom_openvr_props.clone(), tracking_ref_only: false, enable_vive_tracker_proxy: false, face_tracking: SwitchDefault { enabled: false, content: FaceTrackingConfigDefault { gui_collapsed: true, sources: FaceTrackingSourcesConfigDefault { variant: FaceTrackingSourcesConfigDefaultVariant::PreferFullFaceTracking, }, sink: FaceTrackingSinkConfigDefault { VrchatEyeOsc: FaceTrackingSinkConfigVrchatEyeOscDefault { port: 9000 }, variant: FaceTrackingSinkConfigDefaultVariant::VrchatEyeOsc, }, }, }, multimodal_tracking: SwitchDefault { enabled: false, content: MultimodalTrackingDefault { enabled: true, detached_controllers_steamvr_sink: false, }, }, body_tracking: SwitchDefault { enabled: false, content: BodyTrackingConfigDefault { gui_collapsed: true, sources: BodyTrackingSourcesConfigDefault { meta: BodyTrackingMetaConfigDefault { prefer_full_body: true, prefer_high_fidelity: true, }, bd: BodyTrackingBDConfigDefault { BodyTracking: BodyTrackingBDConfigBodyTrackingDefault { high_accuracy: true, prompt_calibration_on_start: true, }, variant: BodyTrackingBDConfigDefaultVariant::BodyTracking, }, }, sink: BodyTrackingSinkConfigDefault { VrchatBodyOsc: BodyTrackingSinkConfigVrchatBodyOscDefault { port: 9000 }, variant: BodyTrackingSinkConfigDefaultVariant::FakeViveTracker, }, tracked: true, }, }, vmc: SwitchDefault { enabled: false, content: VMCConfigDefault { gui_collapsed: true, host: "127.0.0.1".into(), port: 39539, publish: true, orientation_correction: true, }, }, controllers: SwitchDefault { enabled: true, content: ControllersConfigDefault { gui_collapsed: false, tracked: true, hand_skeleton: SwitchDefault { enabled: true, content: HandSkeletonConfigDefault { steamvr_input_2_0: true, predict: false, }, }, emulation_mode: ControllersEmulationModeDefault { Custom: ControllersEmulationModeCustomDefault { serial_number: "ALVR Controller".into(), button_set: VectorDefault { gui_collapsed: false, element: "/user/hand/left/input/a/click".into(), content: vec![], }, }, variant: ControllersEmulationModeDefaultVariant::Quest2Touch, }, extra_openvr_props: default_custom_openvr_props, button_mappings: OptionalDefault { set: false, content: DictionaryDefault { gui_collapsed: false, key: "/user/hand/left/input/a/click".into(), value: VectorDefault { gui_collapsed: false, element: ButtonBindingTargetDefault { destination: "/user/hand/left/input/a/click".into(), mapping_type: ButtonMappingTypeDefault { HysteresisThreshold: HysteresisThresholdDefault { value: 0.5, deviation: 0.05, }, BinaryToScalar: BinaryToScalarStatesDefault { off: 0.0, on: 1.0, }, Remap: RangeDefault { min: 0.0, max: 1.0 }, variant: ButtonMappingTypeDefaultVariant::Passthrough, }, binary_conditions: VectorDefault { gui_collapsed: true, element: "/user/hand/left/input/trigger/touch".into(), content: vec![], }, }, content: vec![], }, content: vec![], }, }, button_mapping_config: AutomaticButtonMappingConfigDefault { gui_collapsed: true, click_threshold: HysteresisThresholdDefault { value: 0.5, deviation: 0.05, }, touch_threshold: HysteresisThresholdDefault { value: 0.1, deviation: 0.05, }, force_threshold: 0.8, }, hand_tracking_interaction: SwitchDefault { enabled: false, content: HandTrackingInteractionConfigDefault { only_touch: false, pinch_touch_distance: 0.0, pinch_trigger_distance: 0.25, curl_touch_distance: 2.0, curl_trigger_distance: 2.5, joystick_deadzone: 40.0, joystick_offset_horizontal: 0.0, joystick_offset_vertical: 0.0, joystick_range: 1.0, repeat_delay: 100, activation_delay: 50, deactivation_delay: 100, }, }, steamvr_pipeline_frames: 2.1, linear_velocity_cutoff: 0.05, angular_velocity_cutoff: 10.0, left_controller_position_offset: ArrayDefault { gui_collapsed: true, content: [0.0, 0.0, -0.11], }, left_controller_rotation_offset: ArrayDefault { gui_collapsed: true, content: [0.0; 3], }, left_hand_tracking_position_offset: ArrayDefault { gui_collapsed: true, content: [0.04, -0.02, -0.13], }, left_hand_tracking_rotation_offset: ArrayDefault { gui_collapsed: true, content: [0.0, -45.0, -90.0], }, haptics: SwitchDefault { enabled: true, content: HapticsConfigDefault { gui_collapsed: true, intensity_multiplier: 1.0, amplitude_curve: 1.0, min_duration_s: 0.01, }, }, }, }, position_recentering_mode: PositionRecenteringModeDefault { Local: PositionRecenteringModeLocalDefault { view_height: 1.5 }, variant: PositionRecenteringModeDefaultVariant::LocalFloor, }, rotation_recentering_mode: RotationRecenteringModeDefault { variant: RotationRecenteringModeDefaultVariant::Yaw, }, max_prediction_ms: 100, }, connection: ConnectionConfigDefault { stream_protocol: SocketProtocolDefault { variant: SocketProtocolDefaultVariant::Udp, }, client_discovery: SwitchDefault { enabled: true, content: DiscoveryConfigDefault { auto_trust_clients: cfg!(debug_assertions), }, }, wired_client_type: ClientFlavorDefault { Custom: "alvr.client".to_owned(), variant: if alvr_common::is_stable() { ClientFlavorDefaultVariant::Store } else { ClientFlavorDefaultVariant::Github }, }, wired_client_autolaunch: SwitchDefault { enabled: true, content: WiredClientAutoLaunchConfigDefault { boot_delay: 0 }, }, web_server_port: 8082, stream_port: 9944, osc_local_port: 9942, dscp: OptionalDefault { set: false, content: DscpTosDefault { ClassSelector: 7, AssuredForwarding: DscpTosAssuredForwardingDefault { class: 4, drop_probability: DropProbabilityDefault { variant: DropProbabilityDefaultVariant::Low, }, }, variant: DscpTosDefaultVariant::ExpeditedForwarding, }, }, server_buffer_config: socket_buffer_config.clone(), client_buffer_config: socket_buffer_config, max_queued_server_video_frames: 1024, avoid_video_glitching: false, minimum_idr_interval_ms: 100, enable_on_connect_script: false, enable_on_disconnect_script: false, allow_untrusted_http: false, packet_size: 1400, statistics_history_size: 256, }, extra: ExtraConfigDefault { logging: LoggingConfigDefault { client_log_report_level: SwitchDefault { enabled: true, content: LogSeverityDefault { variant: LogSeverityDefaultVariant::Error, }, }, log_to_disk: cfg!(debug_assertions), log_button_presses: false, log_tracking: false, log_haptics: false, notification_level: LogSeverityDefault { variant: if cfg!(debug_assertions) { LogSeverityDefaultVariant::Info } else { LogSeverityDefaultVariant::Warning }, }, show_raw_events: SwitchDefault { enabled: false, content: RawEventsConfigDefault { hide_spammy_events: false, }, }, prefer_backtrace: false, show_notification_tip: true, debug_groups: DebugGroupsConfigDefault { server_impl: false, client_impl: false, server_core: false, client_core: false, connection: false, sockets: false, server_gfx: false, client_gfx: false, encoder: false, decoder: false, }, }, steamvr_launcher: SteamvrLauncherDefault { open_close_steamvr_with_dashboard: false, direct_launch: false, }, capture: CaptureConfigDefault { startup_video_recording: false, rolling_video_files: SwitchDefault { enabled: false, content: RollingVideoFilesConfigDefault { duration_s: 5 }, }, capture_frame_dir: if !cfg!(target_os = "linux") { "/tmp".into() } else { "".into() }, }, patches: PatchesDefault { linux_async_compute: false, linux_async_reprojection: false, }, velocities_multiplier: 1.0, open_setup_wizard: alvr_common::is_stable() || alvr_common::is_nightly(), new_version_popup: SwitchDefault { enabled: alvr_common::is_stable(), content: NewVersionPopupConfigDefault { hide_while_version: ALVR_VERSION.to_string(), }, }, }, } } ================================================ FILE: alvr/sockets/Cargo.toml ================================================ [package] name = "alvr_sockets" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [features] trace-performance = ["profiling/profile-with-tracy"] [dependencies] alvr_common.workspace = true alvr_session.workspace = true bincode = { version = "2", features = ["serde"] } profiling = { version = "1", optional = true } serde = "1" serde_json = "1" socket2 = "0.6" ================================================ FILE: alvr/sockets/src/control_socket.rs ================================================ use crate::{CONTROL_PORT, LOCAL_IP}; use alvr_common::{ConResult, HandleTryAgain, ToCon, anyhow::Result, con_bail}; use alvr_session::{DscpTos, SocketBufferConfig}; use bincode::config; use serde::{Serialize, de::DeserializeOwned}; use std::{ io::{Read, Write}, marker::PhantomData, mem, net::{IpAddr, SocketAddr, TcpListener, TcpStream}, time::{Duration, Instant}, }; // This corresponds to the length of the payload const FRAMED_PREFIX_LENGTH: usize = mem::size_of::(); pub fn bind( timeout: Duration, port: u16, dscp: Option, buffer_config: SocketBufferConfig, ) -> Result { let socket = TcpListener::bind((LOCAL_IP, port))?.into(); crate::set_socket_buffers(&socket, buffer_config).ok(); crate::set_dscp(&socket, dscp); socket.set_read_timeout(Some(timeout))?; Ok(socket.into()) } pub fn accept_from_server( listener: &TcpListener, server_ip: Option, timeout: Duration, ) -> ConResult<(TcpStream, TcpStream)> { // Uses timeout set during bind() let (socket, server_address) = listener.accept().handle_try_again()?; if let Some(ip) = server_ip && server_address.ip() != ip { con_bail!( "Connected to wrong client: Expected: {ip}, Found {}", server_address.ip() ); } socket.set_read_timeout(Some(timeout)).to_con()?; socket.set_nodelay(true).to_con()?; Ok((socket.try_clone().to_con()?, socket)) } pub fn connect_to_client( timeout: Duration, client_ips: &[IpAddr], port: u16, buffer_config: SocketBufferConfig, ) -> ConResult<(TcpStream, TcpStream)> { let split_timeout = timeout / client_ips.len() as u32; let mut res = alvr_common::try_again(); for ip in client_ips { res = TcpStream::connect_timeout(&SocketAddr::new(*ip, port), split_timeout) .handle_try_again(); if res.is_ok() { break; } } let socket = res?.into(); crate::set_socket_buffers(&socket, buffer_config).ok(); socket.set_read_timeout(Some(timeout)).to_con()?; let socket = TcpStream::from(socket); socket.set_nodelay(true).to_con()?; Ok((socket.try_clone().to_con()?, socket)) } fn framed_send( socket: &mut TcpStream, buffer: &mut Vec, packet: &S, ) -> Result<()> { buffer.resize(FRAMED_PREFIX_LENGTH, 0); let encoded_size = bincode::serde::encode_into_std_write(packet, buffer, config::standard())?; buffer[0..FRAMED_PREFIX_LENGTH].copy_from_slice(&(encoded_size as u32).to_le_bytes()); socket.write_all(buffer)?; Ok(()) } fn framed_recv( socket: &mut TcpStream, buffer: &mut Vec, recv_cursor: &mut Option, timeout: Duration, ) -> ConResult { let deadline = Instant::now() + timeout; let recv_cursor_ref = if let Some(cursor) = recv_cursor { cursor } else { let mut payload_size_bytes = [0; FRAMED_PREFIX_LENGTH]; loop { let count = socket.peek(&mut payload_size_bytes).handle_try_again()?; if count == FRAMED_PREFIX_LENGTH { break; } else if Instant::now() > deadline { return alvr_common::try_again(); } } let size = FRAMED_PREFIX_LENGTH + u32::from_le_bytes(payload_size_bytes) as usize; buffer.resize(size, 0); recv_cursor.insert(0) }; loop { *recv_cursor_ref += socket .read(&mut buffer[*recv_cursor_ref..]) .handle_try_again()?; if *recv_cursor_ref == buffer.len() { break; } else if Instant::now() > deadline { return alvr_common::try_again(); } } let (packet, _) = bincode::serde::decode_from_slice(&buffer[FRAMED_PREFIX_LENGTH..], config::standard()) .to_con()?; *recv_cursor = None; Ok(packet) } pub struct ControlSocketSender { inner: TcpStream, buffer: Vec, _phantom: PhantomData, } impl ControlSocketSender { pub fn send(&mut self, packet: &S) -> Result<()> { framed_send(&mut self.inner, &mut self.buffer, packet) } } pub struct ControlSocketReceiver { inner: TcpStream, buffer: Vec, recv_cursor: Option, _phantom: PhantomData, } impl ControlSocketReceiver { pub fn recv(&mut self, timeout: Duration) -> ConResult { framed_recv( &mut self.inner, &mut self.buffer, &mut self.recv_cursor, timeout, ) } } pub fn get_server_listener(timeout: Duration) -> Result { let listener = bind(timeout, CONTROL_PORT, None, SocketBufferConfig::default())?; Ok(listener) } // Proto-control-socket that can send and receive any packet. After the split, only the packets of // the specified types can be exchanged pub struct ProtoControlSocket { inner: TcpStream, } pub enum PeerType<'a> { AnyClient(Vec), Server(&'a TcpListener), } impl ProtoControlSocket { pub fn connect_to(timeout: Duration, peer: PeerType<'_>) -> ConResult<(Self, IpAddr)> { let socket = match peer { PeerType::AnyClient(ips) => { connect_to_client(timeout, &ips, CONTROL_PORT, SocketBufferConfig::default())?.0 } PeerType::Server(listener) => accept_from_server(listener, None, timeout)?.0, }; let peer_ip = socket.peer_addr().to_con()?.ip(); Ok((Self { inner: socket }, peer_ip)) } pub fn send(&mut self, packet: &S) -> Result<()> { framed_send(&mut self.inner, &mut vec![], packet) } pub fn recv(&mut self, timeout: Duration) -> ConResult { framed_recv(&mut self.inner, &mut vec![], &mut None, timeout) } pub fn split( self, timeout: Duration, ) -> Result<(ControlSocketSender, ControlSocketReceiver)> { self.inner.set_read_timeout(Some(timeout))?; Ok(( ControlSocketSender { inner: self.inner.try_clone()?, buffer: vec![0; FRAMED_PREFIX_LENGTH], _phantom: PhantomData, }, ControlSocketReceiver { inner: self.inner, buffer: vec![0; FRAMED_PREFIX_LENGTH], recv_cursor: None, _phantom: PhantomData, }, )) } } ================================================ FILE: alvr/sockets/src/lib.rs ================================================ mod control_socket; mod stream_socket; use alvr_common::{anyhow::Result, info}; use alvr_session::{DscpTos, SocketBufferConfig, SocketBufferSize}; use socket2::Socket; use std::{ net::{IpAddr, Ipv4Addr}, time::Duration, }; pub use control_socket::*; pub use stream_socket::*; pub const LOCAL_IP: IpAddr = IpAddr::V4(Ipv4Addr::UNSPECIFIED); pub const CONTROL_PORT: u16 = 9943; pub const HANDSHAKE_PACKET_SIZE_BYTES: usize = 56; // this may change in future protocols pub const KEEPALIVE_INTERVAL: Duration = Duration::from_millis(500); pub const KEEPALIVE_TIMEOUT: Duration = Duration::from_secs(2); pub const MDNS_SERVICE_TYPE: &str = "_alvr._tcp.local."; pub const MDNS_PROTOCOL_KEY: &str = "protocol"; pub const MDNS_DEVICE_ID_KEY: &str = "device_id"; pub const WIRED_CLIENT_HOSTNAME: &str = "client.wired"; fn set_socket_buffers(socket: &socket2::Socket, buffer_config: SocketBufferConfig) -> Result<()> { info!( "Initial socket buffer size: send: {}B, recv: {}B", socket.send_buffer_size()?, socket.recv_buffer_size()? ); { let maybe_size = match buffer_config.send_size_bytes { SocketBufferSize::Default => None, SocketBufferSize::Maximum => Some(u32::MAX), SocketBufferSize::Custom(size) => Some(size), }; if let Some(size) = maybe_size { if let Err(e) = socket.set_send_buffer_size(size as usize) { info!("Error setting socket send buffer: {e}"); } else { info!( "Set socket send buffer succeeded: {}", socket.send_buffer_size()? ); } } } { let maybe_size = match buffer_config.recv_size_bytes { SocketBufferSize::Default => None, SocketBufferSize::Maximum => Some(u32::MAX), SocketBufferSize::Custom(size) => Some(size), }; if let Some(size) = maybe_size { if let Err(e) = socket.set_recv_buffer_size(size as usize) { info!("Error setting socket recv buffer: {e}"); } else { info!( "Set socket recv buffer succeeded: {}", socket.recv_buffer_size()? ); } } } Ok(()) } fn set_dscp(socket: &Socket, dscp: Option) { // https://en.wikipedia.org/wiki/Differentiated_services if let Some(dscp) = dscp { let tos = match dscp { DscpTos::BestEffort => 0, DscpTos::ClassSelector(precedence) => precedence << 3, DscpTos::AssuredForwarding { class, drop_probability, } => (class << 3) | drop_probability as u8, DscpTos::ExpeditedForwarding => 0b101110, }; socket.set_tos_v4((tos << 2) as u32).ok(); } } ================================================ FILE: alvr/sockets/src/stream_socket/mod.rs ================================================ // Note: for StreamSocket, the client uses a server socket, the server uses a client socket. // This is because of certificate management. The server needs to trust a client and its certificate // // StreamSender and StreamReceiver endpoints allow for convenient conversion of the header to/from // bytes while still handling the additional byte buffer with zero copies and extra allocations. // Performance analysis: // We want to minimize the transmission time for various sizes of packets. // The current code locks the write socket *per shard* and not *per packet*. This leds to the best // performance outcome given that the possible packets can be either very small (one shard) or very // large (hundreds/thousands of shards, for video). if we don't allow interleaving shards, a very // small packet will need to wait a long time before getting received if there was an ongoing // transmission of a big packet before. If we allow interleaving shards, small packets can be // transmitted quicker, with only minimal latency increase for the ongoing transmission of the big // packet. // Note: We can't clone the underlying socket for each StreamSender and the mutex around the socket // cannot be removed. This is because we need to make sure at least shards are written whole. mod tcp; mod udp; use alvr_common::{ AnyhowToCon, ConResult, HandleTryAgain, ToCon, anyhow::Result, parking_lot::Mutex, }; use alvr_session::{DscpTos, SocketBufferConfig, SocketProtocol}; use bincode::config; use serde::{Serialize, de::DeserializeOwned}; use std::{ cmp::Ordering, collections::HashMap, marker::PhantomData, mem, net::{IpAddr, TcpListener, UdpSocket}, ops::{Deref, DerefMut}, sync::{Arc, mpsc}, time::Duration, }; trait MultiplexedSocketWriter { // Note: consts are not trait-safe, we require a method fn payload_offset(&self) -> usize; fn send(&mut self, stream_id: u16, packet_index: u32, buffer: &mut Vec) -> Result<()>; } struct ReconstructedPacket { index: u32, buffer: Vec, } struct StreamRecvQueues { used_buffer_sender: mpsc::Sender>, used_buffer_receiver: mpsc::Receiver>, packet_queue: mpsc::Sender, } trait MultiplexedSocketReader { fn payload_offset(&self) -> usize; fn recv(&mut self, stream_queues: &HashMap) -> ConResult; } /// Memory buffer that contains a hidden prefix #[derive(Default)] pub struct Buffer { inner: Vec, raw_payload_offset: usize, // this corresponds to prefix + header _phantom: PhantomData, } impl Deref for Buffer { type Target = [u8]; fn deref(&self) -> &[u8] { &self.inner[self.raw_payload_offset..] } } impl DerefMut for Buffer { fn deref_mut(&mut self) -> &mut Self::Target { &mut self.inner[self.raw_payload_offset..] } } #[derive(Clone)] pub struct StreamSender { inner: Arc>>, stream_id: u16, payload_offset: usize, next_packet_index: u32, used_buffers: Vec>, _phantom: PhantomData, } impl StreamSender { /// Shard and send a buffer with zero copies and zero allocations. /// The prefix of each shard is written over the previously sent shard to avoid reallocations. pub fn send(&mut self, mut buffer: Buffer) -> Result<()> { self.inner .lock() .send(self.stream_id, self.next_packet_index, &mut buffer.inner)?; self.used_buffers.push(buffer.inner); self.next_packet_index = self.next_packet_index.wrapping_add(1); Ok(()) } } impl StreamSender { pub fn get_buffer(&mut self, header: &H, raw_payload_len: usize) -> Result> { let mut buffer = self.used_buffers.pop().unwrap_or_default(); buffer.resize(self.payload_offset, 0); let encoded_payload_size = bincode::serde::encode_into_std_write(header, &mut buffer, config::standard())?; let raw_payload_offset = self.payload_offset + encoded_payload_size; buffer.resize(raw_payload_offset + raw_payload_len, 0); Ok(Buffer { inner: buffer, raw_payload_offset, _phantom: PhantomData, }) } pub fn send_header_with_payload(&mut self, header: &H, raw_payload: &[u8]) -> Result<()> { let mut buffer = self.get_buffer(header, raw_payload.len())?; buffer.copy_from_slice(raw_payload); self.send(buffer) } pub fn send_header(&mut self, header: &H) -> Result<()> { self.send_header_with_payload(header, &[]) } } pub struct ReceiverData { buffer: Vec, payload_offset: usize, used_buffer_queue: mpsc::Sender>, had_packet_loss: bool, _phantom: PhantomData, } impl ReceiverData { pub fn had_packet_loss(&self) -> bool { self.had_packet_loss } } impl ReceiverData { pub fn get(&self) -> Result<(H, &[u8])> { let payload = &self.buffer[self.payload_offset..]; let (header, decoded_size) = bincode::serde::decode_from_slice(payload, config::standard())?; Ok((header, &payload[decoded_size..])) } pub fn get_header(&self) -> Result { Ok(self.get()?.0) } } impl Drop for ReceiverData { fn drop(&mut self) { self.used_buffer_queue .send(mem::take(&mut self.buffer)) .ok(); } } pub struct StreamReceiver { payload_offset: usize, packet_receiver: mpsc::Receiver, used_buffer_queue: mpsc::Sender>, last_packet_index: Option, _phantom: PhantomData, } // Wrapping comparison for packet indices. // Problem: packet indices have a finite bit-width and we have to wrap around upon reaching the // maximum value. This function provides proper ordering capability when wrapping around. The // maximum index distance that two packets can have is u32::MAX / 2 (which is plenty for any // reasonable circumstance). fn wrapping_cmp(lhs: u32, rhs: u32) -> Ordering { const LOWER_BOUND: u32 = u32::MAX / 4; const UPPER_BOUND: u32 = u32::MAX - LOWER_BOUND; if lhs < LOWER_BOUND && rhs > UPPER_BOUND { // lhs wrapped around, so its "unwrapped" value would be `lhs as u64 + u32::MAX` Ordering::Greater } else if lhs > UPPER_BOUND && rhs < LOWER_BOUND { // rhs wrapped around Ordering::Less } else { lhs.cmp(&rhs) } } impl StreamReceiver { pub fn recv(&mut self, timeout: Duration) -> ConResult> { let packet = self .packet_receiver .recv_timeout(timeout) .handle_try_again()?; let mut had_packet_loss = false; if let Some(last_idx) = self.last_packet_index { // Use wrapping arithmetics match wrapping_cmp(packet.index, last_idx.wrapping_add(1)) { Ordering::Equal => (), Ordering::Greater => { // Skipped some indices had_packet_loss = true } Ordering::Less => { // Old packet, discard self.used_buffer_queue.send(packet.buffer).to_con()?; return alvr_common::try_again(); } } } self.last_packet_index = Some(packet.index); Ok(ReceiverData { buffer: packet.buffer, payload_offset: self.payload_offset, used_buffer_queue: self.used_buffer_queue.clone(), had_packet_loss, _phantom: PhantomData, }) } } pub enum StreamSocketBuilder { Tcp(TcpListener), Udp(UdpSocket), } impl StreamSocketBuilder { pub fn listen_for_server( timeout: Duration, port: u16, stream_socket_config: SocketProtocol, stream_tos_config: Option, buffer_config: SocketBufferConfig, ) -> Result { Ok(match stream_socket_config { SocketProtocol::Udp => { StreamSocketBuilder::Udp(udp::bind(port, stream_tos_config, buffer_config)?) } SocketProtocol::Tcp => StreamSocketBuilder::Tcp(tcp::bind( timeout, port, stream_tos_config, buffer_config, )?), }) } pub fn accept_from_server( self, server_ip: IpAddr, port: u16, max_packet_size: usize, timeout: Duration, ) -> ConResult { let (send_socket, receive_socket) = match self { StreamSocketBuilder::Udp(socket) => { udp::connect(&socket, server_ip, port, timeout).to_con()?; udp::split_multiplexed(socket, max_packet_size).to_con()? } StreamSocketBuilder::Tcp(listener) => { let socket = tcp::accept_from_server(&listener, Some(server_ip), timeout)?; tcp::split_multiplexed(socket, timeout).to_con()? } }; Ok(StreamSocket { send_socket: Arc::new(Mutex::new(send_socket)), receive_socket, queues: HashMap::new(), }) } #[allow(clippy::too_many_arguments)] pub fn connect_to_client( timeout: Duration, client_ip: IpAddr, port: u16, protocol: SocketProtocol, dscp: Option, buffer_config: SocketBufferConfig, max_packet_size: usize, ) -> ConResult { let (send_socket, receive_socket) = match protocol { SocketProtocol::Udp => { let socket = udp::bind(port, dscp, buffer_config).to_con()?; udp::connect(&socket, client_ip, port, timeout).to_con()?; udp::split_multiplexed(socket, max_packet_size).to_con()? } SocketProtocol::Tcp => { let socket = tcp::connect_to_client(timeout, &[client_ip], port, buffer_config)?; tcp::split_multiplexed(socket, timeout).to_con()? } }; Ok(StreamSocket { send_socket: Arc::new(Mutex::new(send_socket)), receive_socket, queues: HashMap::new(), }) } } pub struct StreamSocket { send_socket: Arc>>, receive_socket: Box, queues: HashMap, } impl StreamSocket { pub fn request_stream(&self, stream_id: u16) -> StreamSender { StreamSender { inner: Arc::clone(&self.send_socket), stream_id, payload_offset: self.send_socket.lock().payload_offset(), next_packet_index: 0, used_buffers: vec![], _phantom: PhantomData, } } // max_concurrent_buffers: number of buffers allocated by this call which will be reused to // receive packets for this stream ID. If packets are not read fast enough, the shards received // for this particular stream will be discarded pub fn subscribe_to_stream( &mut self, stream_id: u16, max_concurrent_buffers: usize, ) -> StreamReceiver { let (packet_sender, packet_receiver) = mpsc::channel(); let (used_buffer_sender, used_buffer_receiver) = mpsc::channel(); for _ in 0..max_concurrent_buffers { used_buffer_sender.send(vec![]).ok(); } self.queues.insert( stream_id, StreamRecvQueues { used_buffer_sender: used_buffer_sender.clone(), used_buffer_receiver, packet_queue: packet_sender, }, ); StreamReceiver { payload_offset: self.receive_socket.payload_offset(), packet_receiver, used_buffer_queue: used_buffer_sender, last_packet_index: None, _phantom: PhantomData, } } pub fn recv(&mut self) -> ConResult { self.receive_socket.recv(&self.queues) } } ================================================ FILE: alvr/sockets/src/stream_socket/tcp.rs ================================================ use super::{ MultiplexedSocketReader, MultiplexedSocketWriter, ReconstructedPacket, StreamRecvQueues, }; use crate::LOCAL_IP; use alvr_common::{ConResult, HandleTryAgain, ToCon, anyhow::Result, con_bail}; use alvr_session::{DscpTos, SocketBufferConfig}; use socket2::Socket; use std::{ collections::HashMap, io::Write, mem::{self, MaybeUninit}, net::{IpAddr, SocketAddr, TcpListener, TcpStream}, time::Duration, }; pub const PACKET_PREFIX_SIZE: usize = mem::size_of::() // stream ID + mem::size_of::() // packet index + mem::size_of::(); // payload size pub fn bind( timeout: Duration, port: u16, dscp: Option, buffer_config: SocketBufferConfig, ) -> Result { let socket = TcpListener::bind((LOCAL_IP, port))?.into(); crate::set_socket_buffers(&socket, buffer_config).ok(); crate::set_dscp(&socket, dscp); socket.set_read_timeout(Some(timeout))?; Ok(socket.into()) } pub fn accept_from_server( listener: &TcpListener, server_ip: Option, timeout: Duration, ) -> ConResult { // Uses timeout set during bind() let (socket, server_address) = listener.accept().handle_try_again()?; if let Some(ip) = server_ip && server_address.ip() != ip { con_bail!( "Connected to wrong client: Expected: {ip}, Found {}", server_address.ip() ); } socket.set_read_timeout(Some(timeout)).to_con()?; socket.set_nodelay(true).to_con()?; Ok(socket) } pub fn connect_to_client( timeout: Duration, client_ips: &[IpAddr], port: u16, buffer_config: SocketBufferConfig, ) -> ConResult { let split_timeout = timeout / client_ips.len() as u32; let mut res = alvr_common::try_again(); for ip in client_ips { res = TcpStream::connect_timeout(&SocketAddr::new(*ip, port), split_timeout) .handle_try_again(); if res.is_ok() { break; } } let socket = res?.into(); crate::set_socket_buffers(&socket, buffer_config).ok(); socket.set_read_timeout(Some(timeout)).to_con()?; let socket = TcpStream::from(socket); socket.set_nodelay(true).to_con()?; Ok(socket) } pub struct MultiplexedTcpWriter { inner: TcpStream, } impl MultiplexedSocketWriter for MultiplexedTcpWriter { fn payload_offset(&self) -> usize { PACKET_PREFIX_SIZE } // `buffer` contains the payload offset by `payload_offset()` fn send(&mut self, stream_id: u16, packet_index: u32, buffer: &mut Vec) -> Result<()> { let payload_size = buffer.len() - PACKET_PREFIX_SIZE; buffer[0..2].copy_from_slice(&stream_id.to_le_bytes()); buffer[2..6].copy_from_slice(&packet_index.to_le_bytes()); buffer[6..10].copy_from_slice(&(payload_size as u32).to_le_bytes()); self.inner.write_all(buffer)?; Ok(()) } } struct InProgressPacket { stream_id: u16, packet_index: u32, buffer: Vec, buffer_size: usize, cursor: usize, } pub struct MultiplexedTcpReader { inner: Socket, in_progress_packet: Option, used_buffers_poll_timeout: Duration, } impl MultiplexedSocketReader for MultiplexedTcpReader { fn payload_offset(&self) -> usize { PACKET_PREFIX_SIZE } fn recv(&mut self, stream_queues: &HashMap) -> ConResult { let in_progress_packet = if let Some(packet) = &mut self.in_progress_packet { packet } else { let mut prefix_bytes = [0u8; PACKET_PREFIX_SIZE]; loop { let count = self .inner .peek(unsafe { &mut *(&mut prefix_bytes as *mut [u8] as *mut [MaybeUninit]) }) .handle_try_again()?; if count == PACKET_PREFIX_SIZE { break; } } let stream_id = u16::from_le_bytes(prefix_bytes[0..2].try_into().unwrap()); let packet_index = u32::from_le_bytes(prefix_bytes[2..6].try_into().unwrap()); let payload_size = u32::from_le_bytes(prefix_bytes[6..10].try_into().unwrap()) as usize; let mut buffer = match stream_queues.get(&stream_id) { Some(queue) => queue .used_buffer_receiver .recv_timeout(self.used_buffers_poll_timeout) .handle_try_again()?, None => { // This is a packet with an invalid stream id, but we must read it anyway. We // can't obtain a used buffer so we create a new one vec![] } }; buffer.clear(); buffer.reserve(PACKET_PREFIX_SIZE + payload_size); self.in_progress_packet.insert(InProgressPacket { stream_id, packet_index, buffer, buffer_size: PACKET_PREFIX_SIZE + payload_size, cursor: 0, }) }; while in_progress_packet.cursor < in_progress_packet.buffer_size { let sub_buffer = &mut in_progress_packet.buffer.spare_capacity_mut() [in_progress_packet.cursor..in_progress_packet.buffer_size]; in_progress_packet.cursor += self.inner.recv(sub_buffer).handle_try_again()?; } // All writing was done to uninit capacity, here we set the final buffer length unsafe { in_progress_packet .buffer .set_len(in_progress_packet.buffer_size) }; if let Some(queues) = stream_queues.get(&in_progress_packet.stream_id) { // Safety: here self.in_progress_packet is always Some queues .packet_queue .send(ReconstructedPacket { index: in_progress_packet.packet_index, buffer: self.in_progress_packet.take().unwrap().buffer, }) .to_con()?; } else { // The packet had an invalid stream ID. Discard the buffer self.in_progress_packet.take(); } Ok(()) } } pub fn split_multiplexed( socket: TcpStream, used_buffers_poll_timeout: Duration, ) -> Result<( Box, Box, )> { let writer = MultiplexedTcpWriter { inner: socket.try_clone()?, }; let reader = MultiplexedTcpReader { inner: socket.into(), in_progress_packet: None, used_buffers_poll_timeout, }; Ok((Box::new(writer), Box::new(reader))) } ================================================ FILE: alvr/sockets/src/stream_socket/udp.rs ================================================ use super::{ MultiplexedSocketReader, MultiplexedSocketWriter, ReconstructedPacket, StreamRecvQueues, }; use crate::LOCAL_IP; use alvr_common::{ConResult, HandleTryAgain, ToCon, anyhow::Result}; use alvr_session::{DscpTos, SocketBufferConfig}; use socket2::{MaybeUninitSlice, Socket}; use std::ffi::c_int; use std::{ cmp::Ordering, collections::{HashMap, HashSet}, mem::{self, MaybeUninit}, net::{IpAddr, UdpSocket}, ptr, time::Duration, }; pub const SHARD_PREFIX_SIZE: usize = mem::size_of::() // stream ID + mem::size_of::() // packet index + mem::size_of::() // shards count + mem::size_of::(); // shards index fn socket_peek(socket: &mut Socket, buffer: &mut [u8]) -> ConResult { #[cfg(windows)] const FLAGS: c_int = 0x02 | 0x8000; // MSG_PEEK | MSG_PARTIAL #[cfg(not(windows))] const FLAGS: c_int = 0x02 | 0x20; // MSG_PEEK | MSG_TRUNC let buffer = MaybeUninitSlice::new(unsafe { &mut *(ptr::from_mut(buffer) as *mut [MaybeUninit]) }); // NB: Using the non vectored call doesn't seem to work Ok(socket .recv_vectored_with_flags(&mut [buffer], FLAGS) .handle_try_again()? .0) } // Create tokio socket, convert to socket2, apply settings, convert back to tokio. This is done to // let tokio set all the internal parameters it needs from the start. pub fn bind( port: u16, dscp: Option, buffer_config: SocketBufferConfig, ) -> Result { let socket = UdpSocket::bind((LOCAL_IP, port))?.into(); crate::set_socket_buffers(&socket, buffer_config).ok(); crate::set_dscp(&socket, dscp); Ok(socket.into()) } pub fn connect(socket: &UdpSocket, peer_ip: IpAddr, port: u16, timeout: Duration) -> Result<()> { socket.connect((peer_ip, port))?; socket.set_read_timeout(Some(timeout))?; Ok(()) } pub struct MultiplexedUdpWriter { inner: UdpSocket, max_packet_size: usize, } impl MultiplexedSocketWriter for MultiplexedUdpWriter { fn payload_offset(&self) -> usize { SHARD_PREFIX_SIZE } fn send(&mut self, stream_id: u16, packet_index: u32, buffer: &mut Vec) -> Result<()> { let max_shard_size = self.max_packet_size - SHARD_PREFIX_SIZE; let payload_size = buffer.len() - SHARD_PREFIX_SIZE; // rounding up: let shards_count = payload_size.div_ceil(max_shard_size); for shard_idx in 0..shards_count { // this overlaps with the previous shard, this is intended behavior and allows to // reduce allocations let shard_start_position = shard_idx * max_shard_size; let shard_size = usize::min(max_shard_size, payload_size - shard_start_position); let shard_view = &mut buffer[shard_start_position..][..SHARD_PREFIX_SIZE + shard_size]; shard_view[0..2].copy_from_slice(&stream_id.to_le_bytes()); shard_view[2..6].copy_from_slice(&packet_index.to_le_bytes()); shard_view[6..10].copy_from_slice(&(shards_count as u32).to_le_bytes()); shard_view[10..14].copy_from_slice(&(shard_idx as u32).to_le_bytes()); self.inner.send(shard_view)?; } Ok(()) } } // We need to store the size seaparately because we use use the buffer as unallocated memory and // the capacity cannot be set precisely. // Note: Why do we need to keep space for the prefix in the final buffer? // UDP works with discrete packets, reading must be done in a single call. There is no way to // preemptively discard the bytes of the prefix. struct InProgressPacket { buffer: Vec, // contains the prefix buffer_size: usize, // size of the packet counting prefix shards_count: usize, received_shard_indices: HashSet, } pub struct MultiplexedUdpReader { inner: Socket, max_packet_size: usize, in_progress_packets: HashMap>, } impl MultiplexedSocketReader for MultiplexedUdpReader { fn payload_offset(&self) -> usize { SHARD_PREFIX_SIZE } fn recv(&mut self, stream_queues: &HashMap) -> ConResult { let max_shard_data_size = self.max_packet_size - SHARD_PREFIX_SIZE; let discard_and_try_again = move |socket: &Socket| { // Reading with any sized buffer (even 0) will consume the whole datagram socket.recv(&mut []).ok(); alvr_common::try_again() }; let mut prefix_bytes = [0; SHARD_PREFIX_SIZE]; let peek_size = socket_peek(&mut self.inner, &mut prefix_bytes)?; if peek_size < SHARD_PREFIX_SIZE { return discard_and_try_again(&self.inner); } // The values obtained from the prefix (stream ID, packet index, shards count, shard index) // could be corrupted somehow. This method has safety checks against corrupted values and // the relative packet would be discarded. let stream_id = u16::from_le_bytes(prefix_bytes[0..2].try_into().unwrap()); let packet_index = u32::from_le_bytes(prefix_bytes[2..6].try_into().unwrap()); let maybe_shards_count = u32::from_le_bytes(prefix_bytes[6..10].try_into().unwrap()) as usize; let shard_index = u32::from_le_bytes(prefix_bytes[10..14].try_into().unwrap()) as usize; if maybe_shards_count == 0 { return discard_and_try_again(&self.inner); } let Some(queues) = stream_queues.get(&stream_id) else { return discard_and_try_again(&self.inner); }; let in_progress_packets = self.in_progress_packets.entry(stream_id).or_default(); let in_progress_packet = if let Some(packet) = in_progress_packets.get_mut(&packet_index) { packet } else if let Some(mut buffer) = queues.used_buffer_receiver.try_recv().ok().or_else(|| { // By default, try to dequeue a used buffer. In case none were found, recycle one of the // in progress packets, chances are these buffers are "dead" because one of their shards // has been dropped by the network. let idx = *in_progress_packets.iter().next()?.0; Some(in_progress_packets.remove(&idx).unwrap().buffer) }) { // The first shard prefix will dictate the actual number of shards of the packet. The // reserved capacity is an upper bound: we don't know yet the exact size, the last // shard could be smaller than max_shard_data_size buffer.clear(); buffer.reserve(SHARD_PREFIX_SIZE + max_shard_data_size * maybe_shards_count); in_progress_packets .entry(packet_index) .or_insert(InProgressPacket { buffer, buffer_size: 0, shards_count: maybe_shards_count, // todo: find a way to skipping this allocation received_shard_indices: HashSet::with_capacity(maybe_shards_count), }) } else { // This branch may be hit in case the thread related to the stream hangs for some reason return discard_and_try_again(&self.inner); }; if shard_index >= in_progress_packet.shards_count || in_progress_packet .received_shard_indices .contains(&shard_index) { return discard_and_try_again(&self.inner); } // Note: there is no prefix offset, since we want to write the prefix too. let packet_start_index = shard_index * max_shard_data_size; // Note: this is a MaybeUninit slice let sub_buffer = &mut in_progress_packet.buffer.spare_capacity_mut()[packet_start_index..]; // Safety: bound checks lead from the previous code let overwritten_data_backup: [_; SHARD_PREFIX_SIZE] = sub_buffer[..SHARD_PREFIX_SIZE].try_into().unwrap(); // This call should never fail because the peek call succeded before. // Note: in unexpected circumstances, here .to_con() is used not to emit TryAgain, which // would mess with the state of the code. The connection would need to be closed instead. // NB: the received_size contains the prefix let received_size = self.inner.recv(sub_buffer).to_con()?; // Restore backed up bytes sub_buffer[..SHARD_PREFIX_SIZE].copy_from_slice(&overwritten_data_backup); in_progress_packet.buffer_size = usize::max( in_progress_packet.buffer_size, packet_start_index + received_size, ); in_progress_packet .received_shard_indices .insert(shard_index); // Check if packet is complete (and not dummy) and send if in_progress_packet.received_shard_indices.len() == in_progress_packet.shards_count { if let Some(mut packet) = in_progress_packets.remove(&packet_index) { // All writing was done to uninit capacity, here we set the final buffer length unsafe { packet.buffer.set_len(packet.buffer_size) }; queues .packet_queue .send(ReconstructedPacket { index: packet_index, buffer: packet.buffer, }) .ok(); } // Discard older in-progress packets while let Some((idx, _)) = in_progress_packets .iter() .find(|(idx, _)| super::wrapping_cmp(**idx, packet_index) == Ordering::Less) { let idx = *idx; // fix borrow rule let packet = in_progress_packets.remove(&idx).unwrap(); // Recycle buffer queues.used_buffer_sender.send(packet.buffer).ok(); } } Ok(()) } } pub fn split_multiplexed( socket: UdpSocket, max_packet_size: usize, ) -> Result<( Box, Box, )> { let writer = MultiplexedUdpWriter { inner: socket.try_clone()?, max_packet_size, }; let reader = MultiplexedUdpReader { inner: socket.into(), max_packet_size, in_progress_packets: HashMap::new(), }; Ok((Box::new(writer), Box::new(reader))) } ================================================ FILE: alvr/system_info/Cargo.toml ================================================ [package] name = "alvr_system_info" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true jni = "0.21" local-ip-address = "0.6" serde = { version = "1", features = ["derive"] } settings-schema = { git = "https://github.com/alvr-org/settings-schema-rs", rev = "676185f" } [target.'cfg(target_os = "android")'.dependencies] ndk = { version = "0.9", features = ["api-level-28", "media"] } ndk-context = "0.1" ndk-sys = "0.6" ================================================ FILE: alvr/system_info/src/android.rs ================================================ use alvr_common::warn; use jni::{JNIEnv, JavaVM, objects::JObject, sys::jobject}; use std::net::{IpAddr, Ipv4Addr}; pub const MICROPHONE_PERMISSION: &str = "android.permission.RECORD_AUDIO"; pub fn vm() -> JavaVM { unsafe { JavaVM::from_raw(ndk_context::android_context().vm().cast()).unwrap() } } pub fn context() -> jobject { ndk_context::android_context().context().cast() } fn get_api_level() -> i32 { let vm = vm(); let mut env = vm.attach_current_thread().unwrap(); env.get_static_field("android/os/Build$VERSION", "SDK_INT", "I") .unwrap() .i() .unwrap() } pub fn try_get_permission(permission: &str) { let vm = vm(); let mut env = vm.attach_current_thread().unwrap(); let mic_perm_jstring = env.new_string(permission).unwrap(); let permission_status = env .call_method( unsafe { JObject::from_raw(context()) }, "checkSelfPermission", "(Ljava/lang/String;)I", &[(&mic_perm_jstring).into()], ) .unwrap() .i() .unwrap(); if permission_status != 0 { let string_class = env.find_class("java/lang/String").unwrap(); let perm_array = env .new_object_array(1, string_class, mic_perm_jstring) .unwrap(); env.call_method( unsafe { JObject::from_raw(context()) }, "requestPermissions", "([Ljava/lang/String;I)V", &[(&perm_array).into(), 0.into()], ) .unwrap(); // todo: handle case where permission is rejected } } pub fn build_string(ty: &str) -> String { let vm = vm(); let mut env = vm.attach_current_thread().unwrap(); let jname = env .get_static_field("android/os/Build", ty, "Ljava/lang/String;") .unwrap() .l() .unwrap(); let name_raw = env.get_string((&jname).into()).unwrap(); name_raw.to_string_lossy().as_ref().to_owned() } pub fn device_name() -> String { build_string("DEVICE") } pub fn model_name() -> String { build_string("MODEL") } pub fn manufacturer_name() -> String { build_string("MANUFACTURER") } pub fn product_name() -> String { build_string("PRODUCT") } fn get_system_service<'a>(env: &mut JNIEnv<'a>, service_name: &str) -> JObject<'a> { let service_str = env.new_string(service_name).unwrap(); env.call_method( unsafe { JObject::from_raw(context()) }, "getSystemService", "(Ljava/lang/String;)Ljava/lang/Object;", &[(&service_str).into()], ) .unwrap() .l() .unwrap() } // Note: tried and failed to use libc pub fn local_ip() -> IpAddr { let vm = vm(); let mut env = vm.attach_current_thread().unwrap(); let wifi_manager = get_system_service(&mut env, "wifi"); let wifi_info = env .call_method( wifi_manager, "getConnectionInfo", "()Landroid/net/wifi/WifiInfo;", &[], ) .unwrap() .l() .unwrap(); let ip_i32 = env .call_method(wifi_info, "getIpAddress", "()I", &[]) .unwrap() .i() .unwrap(); let ip_arr = ip_i32.to_le_bytes(); IpAddr::V4(Ipv4Addr::new(ip_arr[0], ip_arr[1], ip_arr[2], ip_arr[3])) } // This is needed to avoid wifi scans that disrupt streaming. // Code inspired from https://github.com/Meumeu/WiVRn/blob/master/client/application.cpp pub fn set_wifi_lock(enabled: bool) { let vm = vm(); let mut env = vm.attach_current_thread().unwrap(); let wifi_manager = get_system_service(&mut env, "wifi"); fn set_lock<'a>(env: &mut JNIEnv<'a>, lock: &JObject, enabled: bool) { env.call_method(lock, "setReferenceCounted", "(Z)V", &[false.into()]) .unwrap(); env.call_method( &lock, if enabled { "acquire" } else { "release" }, "()V", &[], ) .unwrap(); let lock_is_aquired = env .call_method(lock, "isHeld", "()Z", &[]) .unwrap() .z() .unwrap(); if lock_is_aquired != enabled { warn!("Failed to set wifi lock: expected {enabled}, got {lock_is_aquired}"); } } let wifi_lock_jstring = env.new_string("alvr_wifi_lock").unwrap(); let wifi_lock = env .call_method( &wifi_manager, "createWifiLock", "(ILjava/lang/String;)Landroid/net/wifi/WifiManager$WifiLock;", &[ if get_api_level() >= 29 { // Recommended for virtual reality since it disables WIFI scans 4 // WIFI_MODE_FULL_LOW_LATENCY } else { 3 // WIFI_MODE_FULL_HIGH_PERF } .into(), (&wifi_lock_jstring).into(), ], ) .unwrap() .l() .unwrap(); set_lock(&mut env, &wifi_lock, enabled); let multicast_lock_jstring = env.new_string("alvr_multicast_lock").unwrap(); let multicast_lock = env .call_method( wifi_manager, "createMulticastLock", "(Ljava/lang/String;)Landroid/net/wifi/WifiManager$MulticastLock;", &[(&multicast_lock_jstring).into()], ) .unwrap() .l() .unwrap(); set_lock(&mut env, &multicast_lock, enabled); } pub fn get_battery_status() -> (f32, bool) { let vm = vm(); let mut env = vm.attach_current_thread().unwrap(); let intent_action_jstring = env .new_string("android.intent.action.BATTERY_CHANGED") .unwrap(); let intent_filter = env .new_object( "android/content/IntentFilter", "(Ljava/lang/String;)V", &[(&intent_action_jstring).into()], ) .unwrap(); let battery_intent = env .call_method( unsafe { JObject::from_raw(context()) }, "registerReceiver", "(Landroid/content/BroadcastReceiver;Landroid/content/IntentFilter;)Landroid/content/Intent;", &[(&JObject::null()).into(), (&intent_filter).into()], ) .unwrap() .l() .unwrap(); let level_jstring = env.new_string("level").unwrap(); let level = env .call_method( &battery_intent, "getIntExtra", "(Ljava/lang/String;I)I", &[(&level_jstring).into(), (-1).into()], ) .unwrap() .i() .unwrap(); let scale_jstring = env.new_string("scale").unwrap(); let scale = env .call_method( &battery_intent, "getIntExtra", "(Ljava/lang/String;I)I", &[(&scale_jstring).into(), (-1).into()], ) .unwrap() .i() .unwrap(); let plugged_jstring = env.new_string("plugged").unwrap(); let plugged = env .call_method( &battery_intent, "getIntExtra", "(Ljava/lang/String;I)I", &[(&plugged_jstring).into(), (-1).into()], ) .unwrap() .i() .unwrap(); (level as f32 / scale as f32, plugged > 0) } ================================================ FILE: alvr/system_info/src/lib.rs ================================================ #[cfg(target_os = "android")] pub mod android; #[cfg(target_os = "android")] pub use android::*; use alvr_common::settings_schema::SettingsSchema; use serde::{Deserialize, Serialize}; use std::fmt::{Display, Formatter}; pub const PACKAGE_NAME_STORE: &str = "alvr.client"; pub const PACKAGE_NAME_GITHUB_DEV: &str = "alvr.client.dev"; pub const PACKAGE_NAME_GITHUB_STABLE: &str = "alvr.client.stable"; // Platform of the device. It is used to match the VR runtime and enable features conditionally. #[derive(PartialEq, Eq, Clone, Copy)] pub enum Platform { Quest1, Quest2, Quest3, Quest3S, QuestPro, QuestUnknown, PicoNeo3, Pico4, Pico4Pro, Pico4Enterprise, Pico4Ultra, PicoG3, PicoUnknown, Focus3, FocusVision, XRElite, ViveUnknown, Yvr, PlayForDreamMR, Lynx, SamsungGalaxyXR, AndroidUnknown, VisionOSHeadset, WindowsPc, LinuxPc, Macos, Unknown, } impl Platform { pub const fn is_quest(&self) -> bool { matches!( self, Self::Quest1 | Self::Quest2 | Self::Quest3 | Self::Quest3S | Self::QuestPro | Self::QuestUnknown ) } pub const fn is_pico(&self) -> bool { matches!( self, Self::PicoG3 | Self::PicoNeo3 | Self::Pico4 | Self::Pico4Pro | Self::Pico4Enterprise | Self::Pico4Ultra | Self::PicoUnknown ) } pub const fn is_vive(&self) -> bool { matches!( self, Self::Focus3 | Self::FocusVision | Self::XRElite | Self::ViveUnknown ) } pub const fn is_yvr(&self) -> bool { matches!(self, Self::Yvr | Self::PlayForDreamMR) } } impl Display for Platform { fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result { let name = match self { Self::Quest1 => "Quest 1", Self::Quest2 => "Quest 2", Self::Quest3 => "Quest 3", Self::Quest3S => "Quest 3S", Self::QuestPro => "Quest Pro", Self::QuestUnknown => "Quest (unknown)", Self::PicoNeo3 => "Pico Neo 3", Self::Pico4 => "Pico 4", Self::Pico4Pro => "Pico 4 Pro", Self::Pico4Enterprise => "Pico 4 Enterprise", Self::Pico4Ultra => "Pico 4 Ultra", Self::PicoG3 => "Pico G3", Self::PicoUnknown => "Pico (unknown)", Self::Focus3 => "VIVE Focus 3", Self::FocusVision => "VIVE Focus Vision", Self::XRElite => "VIVE XR Elite", Self::ViveUnknown => "HTC VIVE (unknown)", Self::Yvr => "YVR", Self::PlayForDreamMR => "Play For Dream MR", Self::Lynx => "Lynx Headset", Self::SamsungGalaxyXR => "Samsung Galaxy XR", Self::AndroidUnknown => "Android (unknown)", Self::VisionOSHeadset => "visionOS Headset", Self::WindowsPc => "Windows PC", Self::LinuxPc => "Linux PC", Self::Macos => "macOS", Self::Unknown => "Unknown", }; write!(f, "{name}") } } #[cfg_attr(not(target_os = "android"), expect(unused_variables))] pub fn platform(runtime_name: Option, runtime_version: Option) -> Platform { #[cfg(target_os = "android")] { let manufacturer = android::manufacturer_name(); let model = android::model_name(); let device = android::device_name(); let product = android::product_name(); // TODO: Better Android XR heuristic // (Maybe check runtime json for /system/lib64/libopenxr.google.so?) alvr_common::info!( "manufacturer: {manufacturer}, model: {model}, device: {device}, product: {product}, \ runtime_name: {runtime_name:?}, runtime_version: {runtime_version:?}", ); match ( manufacturer.as_str(), model.as_str(), device.as_str(), product.as_str(), ) { ("Oculus", _, "monterey", _) => Platform::Quest1, ("Oculus", _, "hollywood", _) => Platform::Quest2, ("Oculus", _, "eureka", _) => Platform::Quest3, ("Oculus", _, "panther", _) => Platform::Quest3S, ("Oculus", _, "seacliff", _) => Platform::QuestPro, ("Oculus", _, _, _) => Platform::QuestUnknown, ("Pico", "Pico Neo 3" | "Pico Neo3 Link", _, _) => Platform::PicoNeo3, ("Pico", _, _, "PICO 4 Pro") => Platform::Pico4Pro, ("Pico", _, _, "PICO 4 Enterprise") => Platform::Pico4Enterprise, ("Pico", _, _, "PICO 4") => Platform::Pico4, ("Pico", _, _, "PICO 4 Ultra") => Platform::Pico4Ultra, ("Pico", _, _, "PICO G3") => Platform::PicoG3, ("Pico", _, _, _) => Platform::PicoUnknown, ("HTC", "VIVE Focus 3", _, _) => Platform::Focus3, ("HTC", "VIVE Focus Vision", _, _) => Platform::FocusVision, ("HTC", "VIVE XR Series", _, _) => Platform::XRElite, ("HTC", _, _, _) => Platform::ViveUnknown, ("YVR", _, _, _) => Platform::Yvr, ("Play For Dream", _, _, _) => Platform::PlayForDreamMR, ("Lynx Mixed Reality", _, _, _) => Platform::Lynx, ("samsung", _, "xrvst2", _) => Platform::SamsungGalaxyXR, _ => Platform::AndroidUnknown, } } #[cfg(not(target_os = "android"))] { match std::env::consts::OS { "visionos" => Platform::VisionOSHeadset, "windows" => Platform::WindowsPc, "linux" => Platform::LinuxPc, "macos" => Platform::Macos, _ => Platform::Unknown, } } } #[cfg(not(target_os = "android"))] pub fn local_ip() -> std::net::IpAddr { use std::net::{IpAddr, Ipv4Addr}; local_ip_address::local_ip().unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED)) } #[derive(SettingsSchema, Serialize, Deserialize, Clone)] pub enum ClientFlavor { Store, Github, Custom(String), } ================================================ FILE: alvr/vrcompositor_wrapper/Cargo.toml ================================================ [package] name = "alvr_vrcompositor_wrapper" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_common.workspace = true alvr_filesystem.workspace = true [build-dependencies] xshell = "0.2" [target.'cfg(target_os = "linux")'.dependencies] exec = "0.3.1" ================================================ FILE: alvr/vrcompositor_wrapper/build.rs ================================================ #[cfg(target_os = "linux")] fn main() { use std::{env, path::PathBuf}; use xshell::{Shell, cmd}; let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); let target_dir = out_dir.join("../../.."); let sh = Shell::new().unwrap(); let command = format!( "g++ -shared -fPIC $(pkg-config --cflags libdrm) drm-lease-shim.cpp -o {}/alvr_drm_lease_shim.so", target_dir.display() ); cmd!(sh, "bash -c {command}").run().unwrap(); } #[cfg(not(target_os = "linux"))] fn main() {} ================================================ FILE: alvr/vrcompositor_wrapper/drm-lease-shim.cpp ================================================ #include #include #include #include #include #include #include #include #include #define PICOJSON_USE_INT64 #include "../server_openvr/cpp/alvr_server/include/picojson.h" #define LOAD_FN(f) \ if (!real_##f) { \ real_##f = reinterpret_cast(dlsym(RTLD_NEXT, #f)); \ if (!real_##f) { \ ERR("Failed to load %s", #f); \ abort(); \ } \ } \ #define LOG(f, ...) printf(f "\n" __VA_OPT__(,) __VA_ARGS__) #define ERR(f, ...) fprintf(stderr, f "\n" __VA_OPT__(,) __VA_ARGS__) template static constexpr bool compare_ptr(X x, Y y) { return reinterpret_cast(x) == reinterpret_cast(y); } struct wl_registry_listener { void (*global)(void *data, struct wl_registry *wl_registry, uint32_t name, const char *interface, uint32_t version); void (*global_remove)(void *data, struct wl_registry *wl_registry, uint32_t name); }; struct wp_drm_lease_device_v1_listener { void (*drm_fd)(void *data, struct wp_drm_lease_device_v1 *wp_drm_lease_device_v1, int32_t fd); void (*connector)(void *data, struct wp_drm_lease_device_v1 *wp_drm_lease_device_v1, struct wp_drm_lease_connector_v1 *id); void (*done)(void *data, struct wp_drm_lease_device_v1 *wp_drm_lease_device_v1); void (*released)(void *data, struct wp_drm_lease_device_v1 *wp_drm_lease_device_v1); }; struct wp_drm_lease_connector_v1_listener { void (*name)(void *data, struct wp_drm_lease_connector_v1 *wp_drm_lease_connector_v1, const char *name); void (*description)(void *data, struct wp_drm_lease_connector_v1 *wp_drm_lease_connector_v1, const char *description); void (*connector_id)(void *data, struct wp_drm_lease_connector_v1 *wp_drm_lease_connector_v1, uint32_t connector_id); void (*done)(void *data, struct wp_drm_lease_connector_v1 *wp_drm_lease_connector_v1); void (*withdrawn)(void *data, struct wp_drm_lease_connector_v1 *wp_drm_lease_connector_v1); }; struct wp_drm_lease_v1_listener { void (*lease_fd)(void *data, struct wp_drm_lease_v1 *wp_drm_lease_v1, int32_t leased_fd); void (*finished)(void *data, struct wp_drm_lease_v1 *wp_drm_lease_v1); }; static struct wp_drm_lease_device_v1 {} fake_device_id; static struct wp_drm_lease_connector_v1 {} fake_connector_id; static struct wp_drm_lease_request_v1 {} fake_lease_request_id; static struct wp_drm_lease_v1 {} fake_lease_id; static int drm_fd = -1; static int drm_connector_id = -1; static void open_drm_fd() { static drmModeResPtr (*real_drmModeGetResources)(int fd) = nullptr; LOAD_FN(drmModeGetResources); for(auto cardCandidate : std::filesystem::directory_iterator("/dev/dri")) { if(cardCandidate.path().filename().string().rfind("card", 0) == 0) { LOG("cardCandidateFound: file=%s", cardCandidate.path().c_str()); drm_fd = open(cardCandidate.path().c_str(), O_RDONLY); auto res = real_drmModeGetResources(drm_fd); if (res && res->count_connectors) { drm_connector_id = res->connectors[0]; break; } } } LOG("DRM: fd=%d, connector_id=%d", drm_fd, drm_connector_id); } static int (*real_wl_proxy_add_listener)(struct wl_proxy *proxy, void (**implementation)(void), void *data); static int hooked_wl_proxy_add_listener(struct wl_proxy *proxy, void (**implementation)(void), void *data) { // wp_drm_lease_connector_v1 if (compare_ptr(proxy, &fake_connector_id)) { LOG("LISTENER wp_drm_lease_connector_v1"); auto listener = reinterpret_cast(implementation); listener->name(data, &fake_connector_id, "ALVR_name"); listener->description(data, &fake_connector_id, "ALVR_description"); listener->connector_id(data, &fake_connector_id, drm_connector_id); listener->done(data, &fake_connector_id); LOG("LISTENER done"); return 0; } // wp_drm_lease_v1 if (compare_ptr(proxy, &fake_lease_id)) { LOG("LISTENER wp_drm_lease_v1"); auto listener = reinterpret_cast(implementation); listener->lease_fd(data, &fake_lease_id, drm_fd); LOG("LISTENER done"); return 0; } // wp_drm_lease_device_v1 if (compare_ptr(proxy, &fake_device_id)) { LOG("LISTENER wp_drm_lease_device_v1"); auto listener = reinterpret_cast(implementation); open_drm_fd(); listener->drm_fd(data, &fake_device_id, drm_fd); if (drm_connector_id != -1) { listener->connector(data, &fake_device_id, &fake_connector_id); } listener->done(data, &fake_device_id); LOG("LISTENER done"); return 0; } const char *name = *(*reinterpret_cast(proxy)); if (strcmp(name, "wl_registry") == 0) { LOG("LISTENER wl_registry"); auto listener = reinterpret_cast(implementation); listener->global(data, reinterpret_cast(proxy), 0, "wp_drm_lease_device_v1", 1); LOG("LISTENER done"); return 0; } return real_wl_proxy_add_listener(proxy, implementation, data); } static struct wl_proxy *(*real_wl_proxy_marshal_flags)(struct wl_proxy *proxy, uint32_t opcode, const struct wl_interface *interface, uint32_t version, uint32_t flags, ...); static struct wl_proxy *hooked_wl_proxy_marshal_flags(struct wl_proxy *proxy, uint32_t opcode, const struct wl_interface *interface, uint32_t version, uint32_t flags, ...) { // wp_drm_lease_connector_v1 if (compare_ptr(proxy, &fake_connector_id)) { if (opcode == 0) { LOG("CALL wp_drm_lease_connector_v1_destroy"); } else { ERR("Unknown wp_drm_lease_connector_v1 opcode=%u", opcode); } return nullptr; } // wp_drm_lease_request_v1 if (compare_ptr(proxy, &fake_lease_request_id)) { if (opcode == 0) { LOG("CALL wp_drm_lease_request_v1_request_connector"); } else if (opcode == 1) { LOG("CALL wp_drm_lease_request_v1_submit"); return reinterpret_cast(&fake_lease_id); } else { ERR("Unknown wp_drm_lease_request_v1 opcode=%u", opcode); } return nullptr; } // wp_drm_lease_device_v1 if (compare_ptr(proxy, &fake_device_id)) { if (opcode == 0) { LOG("CALL wp_drm_lease_device_v1_create_lease_request"); return reinterpret_cast(&fake_lease_request_id); } else if (opcode == 1) { LOG("CALL wp_drm_lease_device_v1_release"); } else { ERR("Unknown wp_drm_lease_device_v1 opcode=%u", opcode); } return nullptr; } const char *name = **reinterpret_cast(proxy); const char *iname = *reinterpret_cast(const_cast(interface)); if (strcmp(name, "wl_registry") == 0 && strcmp(iname, "wp_drm_lease_device_v1") == 0 && opcode == 0) { LOG("CALL wl_registry_bind - wp_drm_lease_device_v1"); return reinterpret_cast(&fake_device_id); } __builtin_return(__builtin_apply(reinterpret_cast(real_wl_proxy_marshal_flags), __builtin_apply_args(), 1024)); } extern "C" void *SDL_LoadFunction(void *handle, const char *name) { static void *(*real_SDL_LoadFunction)(void *handle, const char *name) = nullptr; LOAD_FN(SDL_LoadFunction); #define HOOK(f) \ if (strcmp(name, #f) == 0) { \ LOG("HOOK %s", #f); \ real_##f = reinterpret_cast(real_SDL_LoadFunction(handle, #f)); \ return reinterpret_cast(hooked_##f); \ } \ HOOK(wl_proxy_add_listener); HOOK(wl_proxy_marshal_flags); #undef HOOK return real_SDL_LoadFunction(handle, name); } extern "C" drmModeConnectorPtr drmModeGetConnector(int fd, uint32_t connectorId) { LOG("CALL drmModeGetConnector(%d, %u)", fd, connectorId); static drmModeConnectorPtr (*real_drmModeGetConnector)(int fd, uint32_t connectorId) = nullptr; LOAD_FN(drmModeGetConnector); auto con = real_drmModeGetConnector(fd, connectorId); if (con) { auto sessionFile = std::ifstream(getenv("ALVR_SESSION_JSON")); auto json = std::string(std::istreambuf_iterator(sessionFile), std::istreambuf_iterator()); picojson::value v; picojson::parse(v, json); auto config = v.get("openvr_config"); con->count_modes = 1; con->modes = (drmModeModeInfo*)calloc(1, sizeof(drmModeModeInfo)); con->modes->hdisplay = config.get("eye_resolution_width").get() * 2; con->modes->vdisplay = config.get("eye_resolution_height").get(); } return con; } __attribute__((constructor)) static void lib_init() { LOG("ALVR: drm-lease shim loaded"); unsetenv("LD_PRELOAD"); } ================================================ FILE: alvr/vrcompositor_wrapper/src/main.rs ================================================ #[cfg(target_os = "linux")] fn main() { let argv0 = std::env::args().next().unwrap(); // location of the ALVR vulkan layer manifest let layer_path = match std::fs::read_link(&argv0) { Ok(path) => path .parent() .unwrap() .join("../../share/vulkan/explicit_layer.d"), Err(err) => panic!("Failed to read vrcompositor symlink: {err}"), }; unsafe { std::env::set_var("VK_LAYER_PATH", layer_path); // Vulkan < 1.3.234 std::env::set_var("VK_INSTANCE_LAYERS", "VK_LAYER_ALVR_capture"); std::env::set_var("DISABLE_VK_LAYER_VALVE_steam_fossilize_1", "1"); std::env::set_var("DISABLE_MANGOHUD", "1"); std::env::set_var("DISABLE_VKBASALT", "1"); std::env::set_var("DISABLE_OBS_VKCAPTURE", "1"); // Vulkan >= 1.3.234 std::env::set_var( "VK_LOADER_LAYERS_ENABLE", "VK_LAYER_ALVR_capture,VK_LAYER_MESA_device_select", ); std::env::set_var("VK_LOADER_LAYERS_DISABLE", "*"); } if std::env::var("WAYLAND_DISPLAY").is_ok() { let drm_lease_shim_path = match std::fs::read_link(&argv0) { Ok(path) => path.parent().unwrap().join("alvr_drm_lease_shim.so"), Err(err) => panic!("Failed to read vrcompositor symlink: {err}"), }; unsafe { std::env::set_var("LD_PRELOAD", drm_lease_shim_path); std::env::set_var( "ALVR_SESSION_JSON", alvr_filesystem::filesystem_layout_invalid() .session() .to_string_lossy() .to_string(), ); } } let err = exec::execvp(argv0 + ".real", std::env::args()); println!("Failed to run vrcompositor {err}"); } #[cfg(not(target_os = "linux"))] fn main() {} ================================================ FILE: alvr/vulkan_layer/.gitignore ================================================ CMakeLists.txt.user CMakeCache.txt CMakeFiles CMakeScripts Testing Makefile cmake_install.cmake install_manifest.txt compile_commands.json CTestTestfile.cmake _deps build ================================================ FILE: alvr/vulkan_layer/Cargo.toml ================================================ [package] name = "alvr_vulkan_layer" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [lib] crate-type = ["cdylib"] [dependencies] alvr_common.workspace = true alvr_filesystem.workspace = true [build-dependencies] bindgen = "0.72" cc = { version = "1", features = ["parallel"] } pkg-config = "0.3" walkdir = "2" ================================================ FILE: alvr/vulkan_layer/LICENSE ================================================ Copyright (c) 2019 Arm Limited. Copyright (c) 2021-2024 alvr-org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: alvr/vulkan_layer/README.md ================================================ # ALVR capture vulkan layer ## Introduction The ALVR capture vulkan layer is intended to overcome a limitation of SteamVR runtime on Linux: it does'nt allow software based output devices. The layer is based on [vulkan wsi layer](https://gitlab.freedesktop.org/mesa/vulkan-wsi-layer), which is meant to implement window system integration as layers. The ALVR layer adds a display to the vkGetPhysicalDeviceDisplayPropertiesKHR call, and implements all functions related to that device. It then allows images of the swapchain to be shared to an other process (the alvr server process), and communicates present calls. There are unfortunately a few hacks that make it heavily dependent to SteamVR: requested extentions manipulation to enable the required ones, searching through the stack to find the headset position, and not fully implementing the advertised features. ================================================ FILE: alvr/vulkan_layer/build.rs ================================================ #[cfg(target_os = "linux")] fn main() { use std::{env, path::PathBuf}; let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); let cpp_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap()); let server_cpp_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap()).join("../server_openvr/cpp"); let vulkan = pkg_config::Config::new().probe("vulkan").unwrap(); let libunwind = pkg_config::Config::new().probe("libunwind").unwrap(); let cpp_paths = walkdir::WalkDir::new(".") .into_iter() .filter_map(|maybe_entry| maybe_entry.ok()) .map(|entry| entry.into_path()) .collect::>(); let source_files_paths = cpp_paths.iter().filter(|path| { path.extension() .filter(|ext| ext.to_string_lossy() == "cpp") .is_some() }); let mut build = cc::Build::new(); build .cpp(true) .files(source_files_paths) .flag("-std=c++17") .flag_if_supported("-Wno-unused-parameter") .define("VK_USE_PLATFORM_XLIB_XRANDR_EXT", None) .include(cpp_dir) .include(server_cpp_dir) .includes(vulkan.include_paths) .includes(libunwind.include_paths); build.compile("VkLayer_ALVR"); bindgen::builder() .clang_arg("-xc++") .header("layer/layer.h") .derive_default(true) .generate() .expect("layer bindings") .write_to_file(out_dir.join("layer_bindings.rs")) .expect("layer_bindings.rs"); for lib in libunwind.libs { println!("cargo:rustc-link-lib={lib}"); } // fail build if there are undefined symbols in final library println!("cargo:rustc-cdylib-link-arg=-Wl,--no-undefined"); for path in cpp_paths { println!("cargo:rerun-if-changed={}", path.to_string_lossy()); } } #[cfg(not(target_os = "linux"))] fn main() {} ================================================ FILE: alvr/vulkan_layer/layer/alvr_x86_64.json ================================================ { "file_format_version" : "1.1.2", "layer" : { "name": "VK_LAYER_ALVR_capture", "type": "GLOBAL", "library_path": "../../../lib64/libalvr_vulkan_layer.so", "api_version": "1.0.68", "implementation_version": "1", "description": "ALVR display intercept layer", "instance_extensions": [ { "name" : "VK_EXT_headless_surface", "spec_version" : "1" }, { "name" : "VK_KHR_surface", "spec_version" : "1" }, { "name" : "VK_EXT_acquire_xlib_display", "spec_version" : "1" }, { "name" : "VK_KHR_display", "spec_version" : "1" } ], "device_extensions": [ { "name" : "VK_KHR_swapchain", "spec_version" : "1" } ], "functions": { "vkNegotiateLoaderLayerInterfaceVersion" : "ALVR_Negotiate" }, "disable_environment": { "DISABLE_ALVR_DISPLAY": "1" } } } ================================================ FILE: alvr/vulkan_layer/layer/device_api.cpp ================================================ #include "device_api.hpp" #include "private_data.hpp" #include "wsi/display.hpp" #include "settings.h" #include static const char *alvr_display_name = "ALVR display"; const struct { } alvr_display; const VkDisplayKHR alvr_display_handle = (VkDisplayKHR_T *)&alvr_display; const struct { } alvr_display_mode; const VkDisplayModeKHR alvr_display_mode_handle = (VkDisplayModeKHR_T *)&alvr_display_mode; extern "C" { VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetPhysicalDeviceDisplayPropertiesKHR( VkPhysicalDevice device, uint32_t *pPropertyCount, VkDisplayPropertiesKHR *pProperties) { if (!pProperties) { *pPropertyCount = 1; return VK_SUCCESS; } if (*pPropertyCount < 1) { return VK_INCOMPLETE; } pProperties[0].display = alvr_display_handle; pProperties[0].displayName = alvr_display_name; pProperties[0].physicalDimensions = VkExtent2D{20, 20}; pProperties[0].physicalResolution = VkExtent2D{Settings::Instance().m_renderWidth, Settings::Instance().m_renderHeight}; pProperties[0].supportedTransforms = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR; pProperties[0].planeReorderPossible = VK_FALSE; pProperties[0].persistentContent = VK_TRUE; return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetDisplayModePropertiesKHR( VkPhysicalDevice device, VkDisplayKHR display, uint32_t *pPropertyCount, VkDisplayModePropertiesKHR *pProperties) { if (display != alvr_display_handle) { *pPropertyCount = 0; return VK_ERROR_OUT_OF_HOST_MEMORY; } if (!pProperties) { *pPropertyCount = 1; return VK_SUCCESS; } if (*pPropertyCount < 1) { return VK_INCOMPLETE; } pProperties[0].displayMode = alvr_display_mode_handle; pProperties[0].parameters.visibleRegion = VkExtent2D{Settings::Instance().m_renderWidth, Settings::Instance().m_renderHeight}; pProperties[0].parameters.refreshRate = Settings::Instance().m_refreshRate * 1000; return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetPhysicalDeviceDisplayPlanePropertiesKHR( VkPhysicalDevice device, uint32_t *pPropertyCount, VkDisplayPlanePropertiesKHR *pProperties) { if (!pProperties) { *pPropertyCount = 1; return VK_SUCCESS; } if (*pPropertyCount < 1) { return VK_INCOMPLETE; } pProperties[0].currentDisplay = alvr_display_handle; pProperties[0].currentStackIndex = 0; return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkAcquireXlibDisplayEXT(VkPhysicalDevice device, Display *dpy, VkDisplayKHR display) { if (display != alvr_display_handle) { return VK_ERROR_OUT_OF_HOST_MEMORY; } return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetDrmDisplayEXT(VkPhysicalDevice physicalDevice, int32_t drmFd, uint32_t connectorId, VkDisplayKHR *display) { *display = alvr_display_handle; return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkAcquireDrmDisplayEXT(VkPhysicalDevice physicalDevice, int32_t drmFd, VkDisplayKHR display) { if (display != alvr_display_handle) { return VK_ERROR_INITIALIZATION_FAILED; } return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetDisplayPlaneSupportedDisplaysKHR( VkPhysicalDevice physicalDevice, uint32_t planeIndex, uint32_t *pDisplayCount, VkDisplayKHR *pDisplays) { if (planeIndex != 0) { return VK_ERROR_OUT_OF_HOST_MEMORY; } if (!pDisplays) { *pDisplayCount = 1; return VK_SUCCESS; } pDisplays[0] = alvr_display_handle; return VK_SUCCESS; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkCreateDisplayPlaneSurfaceKHR( VkInstance vkinstance, const VkDisplaySurfaceCreateInfoKHR * /*pCreateInfo*/, const VkAllocationCallbacks *pAllocator, VkSurfaceKHR *pSurface) { auto &instance = layer::instance_private_data::get(vkinstance); VkHeadlessSurfaceCreateInfoEXT createInfo = {}; createInfo.sType = VK_STRUCTURE_TYPE_HEADLESS_SURFACE_CREATE_INFO_EXT; auto res = instance.disp.CreateHeadlessSurfaceEXT(vkinstance, &createInfo, pAllocator, pSurface); if (*pSurface == NULL) std::abort(); instance.add_surface(*pSurface); return res; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkReleaseDisplayEXT(VkPhysicalDevice physicalDevice, VkDisplayKHR display) { return VK_SUCCESS; } VKAPI_ATTR void VKAPI_CALL wsi_layer_vkDestroySurfaceKHR(VkInstance vkinstance, VkSurfaceKHR surface, const VkAllocationCallbacks *pAllocator) { auto &instance = layer::instance_private_data::get(vkinstance); if (instance.should_layer_handle_surface(surface)) { return; } return instance.disp.DestroySurfaceKHR(vkinstance, surface, pAllocator); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkRegisterDisplayEventEXT( VkDevice device, VkDisplayKHR display, const VkDisplayEventInfoEXT *pDisplayEventInfo, const VkAllocationCallbacks *pAllocator, VkFence *pFence) { if (display != alvr_display_handle) { return VK_ERROR_OUT_OF_HOST_MEMORY; } auto &instance = layer::device_private_data::get(device); *pFence = instance.display->get_vsync_fence(); return VK_SUCCESS; } VKAPI_ATTR void VKAPI_CALL wsi_layer_vkDestroyFence(VkDevice device, VkFence fence, const VkAllocationCallbacks *pAllocator) { auto &instance = layer::device_private_data::get(device); auto alvr_fence = instance.display->peek_vsync_fence(); if (fence == alvr_fence) { return; } instance.disp.DestroyFence(device, fence, pAllocator); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkWaitForFences(VkDevice device, uint32_t fenceCount, const VkFence *pFences, VkBool32 waitAll, uint64_t timeout) { auto &instance = layer::device_private_data::get(device); auto alvr_fence = instance.display->peek_vsync_fence(); for (uint32_t i = 0; i < fenceCount; ++i) { if (pFences[i] == alvr_fence) { assert(fenceCount == 1); // only our fence return instance.display->wait_for_vsync(timeout) ? VK_SUCCESS : VK_TIMEOUT; } } return instance.disp.WaitForFences(device, fenceCount, pFences, waitAll, timeout); } VKAPI_ATTR VkResult wsi_layer_vkGetFenceStatus(VkDevice device, VkFence fence) { auto &instance = layer::device_private_data::get(device); auto alvr_fence = instance.display->peek_vsync_fence(); if (fence == alvr_fence) { return instance.display->is_signaled() ? VK_SUCCESS : VK_NOT_READY; } return instance.disp.GetFenceStatus(device, fence); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkCreateDisplayModeKHR( VkPhysicalDevice physicalDevice, VkDisplayKHR display, const VkDisplayModeCreateInfoKHR* pCreateInfo, const VkAllocationCallbacks* pAllocator, VkDisplayModeKHR* pMode) { return VK_ERROR_INITIALIZATION_FAILED; } } ================================================ FILE: alvr/vulkan_layer/layer/device_api.hpp ================================================ #pragma once #include extern "C" { VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetPhysicalDeviceDisplayPropertiesKHR( VkPhysicalDevice device, uint32_t *pPropertyCount, VkDisplayPropertiesKHR *pProperties); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetDisplayModePropertiesKHR( VkPhysicalDevice device, VkDisplayKHR display, uint32_t *pPropertyCount, VkDisplayModePropertiesKHR *pProperties); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetPhysicalDeviceDisplayPlanePropertiesKHR( VkPhysicalDevice device, uint32_t *pPropertyCount, VkDisplayPlanePropertiesKHR *pProperties); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkAcquireXlibDisplayEXT(VkPhysicalDevice device, Display *dpy, VkDisplayKHR display); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetDrmDisplayEXT(VkPhysicalDevice physicalDevice, int32_t drmFd, uint32_t connectorId, VkDisplayKHR *display); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkAcquireDrmDisplayEXT(VkPhysicalDevice physicalDevice, int32_t drmFd, VkDisplayKHR display); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkGetDisplayPlaneSupportedDisplaysKHR( VkPhysicalDevice physicalDevice, uint32_t planeIndex, uint32_t *pDisplayCount, VkDisplayKHR *pDisplays); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkCreateDisplayPlaneSurfaceKHR( VkInstance instance, const VkDisplaySurfaceCreateInfoKHR *pCreateInfo, const VkAllocationCallbacks *pAllocator, VkSurfaceKHR *pSurface); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkReleaseDisplayEXT(VkPhysicalDevice physicalDevice, VkDisplayKHR display); VKAPI_ATTR void VKAPI_CALL wsi_layer_vkDestroySurfaceKHR(VkInstance instance, VkSurfaceKHR surface, const VkAllocationCallbacks *pAllocator); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkRegisterDisplayEventEXT( VkDevice device, VkDisplayKHR display, const VkDisplayEventInfoEXT *pDisplayEventInfo, const VkAllocationCallbacks *pAllocator, VkFence *pFence); VKAPI_ATTR void VKAPI_CALL wsi_layer_vkDestroyFence(VkDevice device, VkFence fence, const VkAllocationCallbacks *pAllocator); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkWaitForFences(VkDevice device, uint32_t fenceCount, const VkFence *pFences, VkBool32 waitAll, uint64_t timeout); VKAPI_ATTR VkResult wsi_layer_vkGetFenceStatus(VkDevice device, VkFence fence); VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkCreateDisplayModeKHR( VkPhysicalDevice physicalDevice, VkDisplayKHR display, const VkDisplayModeCreateInfoKHR* pCreateInfo, const VkAllocationCallbacks* pAllocator, VkDisplayModeKHR* pMode); } ================================================ FILE: alvr/vulkan_layer/layer/layer.cpp ================================================ /* * Copyright (c) 2016-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include #include #include #include #include #include #include "settings.h" #include "device_api.hpp" #include "private_data.hpp" #include "surface_api.hpp" #include "swapchain_api.hpp" #include "util/custom_allocator.hpp" #include "util/extension_list.hpp" #include "wsi/wsi_factory.hpp" #include "layer.h" #define VK_LAYER_API_VERSION VK_MAKE_VERSION(1, 0, VK_HEADER_VERSION) namespace layer { static const VkLayerProperties global_layer = { "VK_LAYER_ALVR_capture", VK_LAYER_API_VERSION, 1, "ALVR capture layer", }; static const VkExtensionProperties device_extension[] = { {VK_KHR_SWAPCHAIN_EXTENSION_NAME, VK_KHR_SWAPCHAIN_SPEC_VERSION}}; static const VkExtensionProperties instance_extension[] = { {VK_KHR_SURFACE_EXTENSION_NAME, VK_KHR_SURFACE_SPEC_VERSION}}; VKAPI_ATTR VkResult extension_properties(const uint32_t count, const VkExtensionProperties *layer_ext, uint32_t *pCount, VkExtensionProperties *pProp) { uint32_t size; if (pProp == NULL || layer_ext == NULL) { *pCount = count; return VK_SUCCESS; } size = *pCount < count ? *pCount : count; memcpy(pProp, layer_ext, size * sizeof(VkExtensionProperties)); *pCount = size; if (size < count) { return VK_INCOMPLETE; } return VK_SUCCESS; } VKAPI_ATTR VkResult layer_properties(const uint32_t count, const VkLayerProperties *layer_prop, uint32_t *pCount, VkLayerProperties *pProp) { uint32_t size; if (pProp == NULL || layer_prop == NULL) { *pCount = count; return VK_SUCCESS; } size = *pCount < count ? *pCount : count; memcpy(pProp, layer_prop, size * sizeof(VkLayerProperties)); *pCount = size; if (size < count) { return VK_INCOMPLETE; } return VK_SUCCESS; } VKAPI_ATTR VkLayerInstanceCreateInfo *get_chain_info(const VkInstanceCreateInfo *pCreateInfo, VkLayerFunction func) { VkLayerInstanceCreateInfo *chain_info = (VkLayerInstanceCreateInfo *)pCreateInfo->pNext; while (chain_info && !(chain_info->sType == VK_STRUCTURE_TYPE_LOADER_INSTANCE_CREATE_INFO && chain_info->function == func)) { chain_info = (VkLayerInstanceCreateInfo *)chain_info->pNext; } return chain_info; } VKAPI_ATTR VkLayerDeviceCreateInfo *get_chain_info(const VkDeviceCreateInfo *pCreateInfo, VkLayerFunction func) { VkLayerDeviceCreateInfo *chain_info = (VkLayerDeviceCreateInfo *)pCreateInfo->pNext; while (chain_info && !(chain_info->sType == VK_STRUCTURE_TYPE_LOADER_DEVICE_CREATE_INFO && chain_info->function == func)) { chain_info = (VkLayerDeviceCreateInfo *)chain_info->pNext; } return chain_info; } /* This is where the layer is initialised and the instance dispatch table is constructed. */ VKAPI_ATTR VkResult create_instance(const VkInstanceCreateInfo *pCreateInfo, const VkAllocationCallbacks *pAllocator, VkInstance *pInstance) { // Make sure settings are loaded before we access them Settings::Instance().Load(); VkLayerInstanceCreateInfo *layerCreateInfo = get_chain_info(pCreateInfo, VK_LAYER_LINK_INFO); PFN_vkSetInstanceLoaderData loader_callback = get_chain_info(pCreateInfo, VK_LOADER_DATA_CALLBACK)->u.pfnSetInstanceLoaderData; if (nullptr == layerCreateInfo || nullptr == layerCreateInfo->u.pLayerInfo) { return VK_ERROR_INITIALIZATION_FAILED; } PFN_vkGetInstanceProcAddr fpGetInstanceProcAddr = layerCreateInfo->u.pLayerInfo->pfnNextGetInstanceProcAddr; PFN_vkCreateInstance fpCreateInstance = (PFN_vkCreateInstance)fpGetInstanceProcAddr(nullptr, "vkCreateInstance"); if (nullptr == fpCreateInstance) { return VK_ERROR_INITIALIZATION_FAILED; } /* Advance the link info for the next element on the chain. */ layerCreateInfo->u.pLayerInfo = layerCreateInfo->u.pLayerInfo->pNext; /* The layer needs some Vulkan 1.2 functionality in order to operate correctly. * We thus change the application info to require this API version, if necessary. * This may have consequences for ICDs whose behaviour depends on apiVersion. */ const uint32_t minimum_required_vulkan_version = VK_API_VERSION_1_2; VkApplicationInfo modified_app_info{}; if (nullptr != pCreateInfo->pApplicationInfo) { modified_app_info = *pCreateInfo->pApplicationInfo; if (modified_app_info.apiVersion < minimum_required_vulkan_version) { modified_app_info.apiVersion = minimum_required_vulkan_version; } } else { modified_app_info.sType = VK_STRUCTURE_TYPE_APPLICATION_INFO; modified_app_info.apiVersion = minimum_required_vulkan_version; } // Hijack one extension name // the headless extension can't be added as a new parameter, because the loader performs a copy before // calling the createInstance functions. The loader must know we activated this function because it // will enable bits in the wsi part, so we switch to vulkan 1.1, and replace one of the extentions // that has been promoted, with a const_cast. for (uint32_t i = 0 ; i < pCreateInfo->enabledExtensionCount ; ++i) { if (strcmp("VK_KHR_external_memory_capabilities", pCreateInfo->ppEnabledExtensionNames[i]) == 0) { const char** ext = const_cast(pCreateInfo->ppEnabledExtensionNames + i); *ext = VK_EXT_HEADLESS_SURFACE_EXTENSION_NAME; } } auto createInfo = *pCreateInfo; createInfo.pApplicationInfo = &modified_app_info; /* Now call create instance on the chain further down the list. * Note that we do not remove the extensions that the layer supports from * modified_info.ppEnabledExtensionNames. Layers have to abide the rule that vkCreateInstance * must not generate an error for unrecognized extension names. Also, the loader filters the * extension list to ensure that ICDs do not see extensions that they do not support. */ VkResult result; result = fpCreateInstance(&createInfo, pAllocator, pInstance); if (result != VK_SUCCESS) { return result; } instance_dispatch_table table; result = table.populate(*pInstance, fpGetInstanceProcAddr); if (result != VK_SUCCESS) { return result; } /* Find all the platforms that the layer can handle based on * pCreateInfo->ppEnabledExtensionNames. */ auto layer_platforms_to_enable = wsi::find_enabled_layer_platforms(pCreateInfo); std::unique_ptr inst_data{ new instance_private_data{table, loader_callback, layer_platforms_to_enable}}; instance_private_data::set(*pInstance, std::move(inst_data)); return VK_SUCCESS; } VKAPI_ATTR VkResult create_device(VkPhysicalDevice physicalDevice, const VkDeviceCreateInfo *pCreateInfo, const VkAllocationCallbacks *pAllocator, VkDevice *pDevice) { VkLayerDeviceCreateInfo *layerCreateInfo = get_chain_info(pCreateInfo, VK_LAYER_LINK_INFO); PFN_vkSetDeviceLoaderData loader_callback = get_chain_info(pCreateInfo, VK_LOADER_DATA_CALLBACK)->u.pfnSetDeviceLoaderData; if (nullptr == layerCreateInfo || nullptr == layerCreateInfo->u.pLayerInfo) { return VK_ERROR_INITIALIZATION_FAILED; } /* Retrieve the vkGetDeviceProcAddr and the vkCreateDevice function pointers for the next layer * in the chain. */ PFN_vkGetInstanceProcAddr fpGetInstanceProcAddr = layerCreateInfo->u.pLayerInfo->pfnNextGetInstanceProcAddr; PFN_vkGetDeviceProcAddr fpGetDeviceProcAddr = layerCreateInfo->u.pLayerInfo->pfnNextGetDeviceProcAddr; PFN_vkCreateDevice fpCreateDevice = (PFN_vkCreateDevice)fpGetInstanceProcAddr(VK_NULL_HANDLE, "vkCreateDevice"); if (nullptr == fpCreateDevice) { return VK_ERROR_INITIALIZATION_FAILED; } /* Advance the link info for the next element on the chain. */ layerCreateInfo->u.pLayerInfo = layerCreateInfo->u.pLayerInfo->pNext; /* Copy the extension to a util::extension_list. */ util::allocator allocator{pAllocator, VK_SYSTEM_ALLOCATION_SCOPE_COMMAND}; util::extension_list enabled_extensions{allocator}; VkResult result; result = enabled_extensions.add(pCreateInfo->ppEnabledExtensionNames, pCreateInfo->enabledExtensionCount); if (result != VK_SUCCESS) { return result; } /* Add the extensions required by the platforms that are being enabled in the layer. */ auto &inst_data = instance_private_data::get(physicalDevice); const util::wsi_platform_set &enabled_platforms = inst_data.get_enabled_platforms(); result = wsi::add_extensions_required_by_layer(physicalDevice, enabled_platforms, enabled_extensions); if (result != VK_SUCCESS) { return result; } util::vector modified_enabled_extensions{allocator}; if (!enabled_extensions.get_extension_strings(modified_enabled_extensions)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } /* Now call create device on the chain further down the list. */ VkDeviceCreateInfo modified_info = *pCreateInfo; modified_info.ppEnabledExtensionNames = modified_enabled_extensions.data(); modified_info.enabledExtensionCount = modified_enabled_extensions.size(); // Enable timeline semaphores VkPhysicalDeviceFeatures2 features = {}; features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2; VkPhysicalDeviceVulkan12Features features12 = {}; features12.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES; VkPhysicalDeviceFeatures2 *features_ptr = nullptr; VkPhysicalDeviceVulkan12Features *features12_ptr = nullptr; VkDeviceCreateInfo *next = &modified_info; while (next->pNext) { if (next->sType == VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2) { features_ptr = (VkPhysicalDeviceFeatures2*)next; } else if (next->sType == VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES) { features12_ptr = (VkPhysicalDeviceVulkan12Features*)next; } next = (VkDeviceCreateInfo*)next->pNext; } if (!features_ptr) { features_ptr = &features; next->pNext = features_ptr; next = (VkDeviceCreateInfo*)features_ptr; } if (!features12_ptr) { features12_ptr = &features12; next->pNext = features12_ptr; next = (VkDeviceCreateInfo*)features12_ptr; } features12_ptr->timelineSemaphore = true; if (modified_info.pEnabledFeatures) { features_ptr->features = *modified_info.pEnabledFeatures; modified_info.pEnabledFeatures = nullptr; } result = fpCreateDevice(physicalDevice, &modified_info, pAllocator, pDevice); if (result != VK_SUCCESS) { return result; } device_dispatch_table table; result = table.populate(*pDevice, fpGetDeviceProcAddr); if (result != VK_SUCCESS) { return result; } std::unique_ptr device{ new device_private_data{inst_data, physicalDevice, *pDevice, table, loader_callback}}; device->display = std::make_unique(); device_private_data::set(*pDevice, std::move(device)); return VK_SUCCESS; } /* Clean up the dispatch table for this instance. */ VKAPI_ATTR void VKAPI_CALL wsi_layer_vkDestroyInstance(VkInstance instance, const VkAllocationCallbacks *pAllocator) { assert(instance); layer::instance_private_data::get(instance).disp.DestroyInstance(instance, pAllocator); layer::instance_private_data::destroy(instance); } VKAPI_ATTR void VKAPI_CALL wsi_layer_vkDestroyDevice(VkDevice device, const VkAllocationCallbacks *pAllocator) { layer::device_private_data::destroy(device); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkCreateInstance(const VkInstanceCreateInfo *pCreateInfo, const VkAllocationCallbacks *pAllocator, VkInstance *pInstance) { return layer::create_instance(pCreateInfo, pAllocator, pInstance); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkCreateDevice(VkPhysicalDevice physicalDevice, const VkDeviceCreateInfo *pCreateInfo, const VkAllocationCallbacks *pAllocator, VkDevice *pDevice) { return layer::create_device(physicalDevice, pCreateInfo, pAllocator, pDevice); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkEnumerateDeviceExtensionProperties( VkPhysicalDevice physicalDevice, const char *pLayerName, uint32_t *pCount, VkExtensionProperties *pProperties) { if (pLayerName && !strcmp(pLayerName, layer::global_layer.layerName)) return layer::extension_properties(1, layer::device_extension, pCount, pProperties); assert(physicalDevice); return layer::instance_private_data::get(physicalDevice) .disp.EnumerateDeviceExtensionProperties(physicalDevice, pLayerName, pCount, pProperties); } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkEnumerateInstanceExtensionProperties( const char *pLayerName, uint32_t *pCount, VkExtensionProperties *pProperties) { if (pLayerName && !strcmp(pLayerName, layer::global_layer.layerName)) return layer::extension_properties(1, layer::instance_extension, pCount, pProperties); return VK_ERROR_LAYER_NOT_PRESENT; } VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_vkEnumerateInstanceLayerProperties(uint32_t *pCount, VkLayerProperties *pProperties) { return layer::layer_properties(1, &layer::global_layer, pCount, pProperties); } #define GET_PROC_ADDR(func) \ if (!strcmp(funcName, #func)) \ return (PFN_vkVoidFunction)&wsi_layer_##func; PFN_vkVoidFunction VKAPI_CALL wsi_layer_vkGetDeviceProcAddr(VkDevice device, const char *funcName) { GET_PROC_ADDR(vkCreateSwapchainKHR); GET_PROC_ADDR(vkDestroySwapchainKHR); GET_PROC_ADDR(vkGetSwapchainImagesKHR); GET_PROC_ADDR(vkAcquireNextImageKHR); GET_PROC_ADDR(vkQueuePresentKHR); GET_PROC_ADDR(vkGetSwapchainCounterEXT); GET_PROC_ADDR(vkRegisterDisplayEventEXT); GET_PROC_ADDR(vkDestroyFence); GET_PROC_ADDR(vkWaitForFences); GET_PROC_ADDR(vkGetFenceStatus); return layer::device_private_data::get(device).disp.GetDeviceProcAddr(device, funcName); } VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL wsi_layer_vkGetInstanceProcAddr(VkInstance instance, const char *funcName) { GET_PROC_ADDR(vkGetDeviceProcAddr); GET_PROC_ADDR(vkGetInstanceProcAddr); GET_PROC_ADDR(vkCreateInstance); GET_PROC_ADDR(vkDestroyInstance); GET_PROC_ADDR(vkCreateDevice); GET_PROC_ADDR(vkDestroyDevice); GET_PROC_ADDR(vkGetPhysicalDeviceSurfaceSupportKHR); GET_PROC_ADDR(vkGetPhysicalDeviceSurfaceCapabilitiesKHR); GET_PROC_ADDR(vkGetPhysicalDeviceSurfaceFormatsKHR); GET_PROC_ADDR(vkGetPhysicalDeviceSurfacePresentModesKHR); GET_PROC_ADDR(vkEnumerateDeviceExtensionProperties); GET_PROC_ADDR(vkEnumerateInstanceExtensionProperties); GET_PROC_ADDR(vkEnumerateInstanceLayerProperties); GET_PROC_ADDR(vkGetPhysicalDeviceDisplayPropertiesKHR); GET_PROC_ADDR(vkGetDisplayModePropertiesKHR); GET_PROC_ADDR(vkGetPhysicalDeviceDisplayPlanePropertiesKHR); GET_PROC_ADDR(vkAcquireXlibDisplayEXT); GET_PROC_ADDR(vkGetDrmDisplayEXT); GET_PROC_ADDR(vkAcquireDrmDisplayEXT); GET_PROC_ADDR(vkGetDisplayPlaneSupportedDisplaysKHR); GET_PROC_ADDR(vkCreateDisplayPlaneSurfaceKHR); GET_PROC_ADDR(vkCreateDisplayModeKHR); GET_PROC_ADDR(vkReleaseDisplayEXT); return layer::instance_private_data::get(instance).disp.GetInstanceProcAddr(instance, funcName); } } /* namespace layer */ const char *g_sessionPath; VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_Negotiate(VkNegotiateLayerInterface *nli) { if (nli->loaderLayerInterfaceVersion < 2) return VK_ERROR_INITIALIZATION_FAILED; nli->loaderLayerInterfaceVersion = 2; nli->pfnGetInstanceProcAddr = layer::wsi_layer_vkGetInstanceProcAddr; nli->pfnGetDeviceProcAddr = layer::wsi_layer_vkGetDeviceProcAddr; return VK_SUCCESS; } ================================================ FILE: alvr/vulkan_layer/layer/layer.h ================================================ #pragma once #include extern "C" const char *g_sessionPath; extern "C" VKAPI_ATTR VkResult VKAPI_CALL wsi_layer_Negotiate(VkNegotiateLayerInterface *nli); ================================================ FILE: alvr/vulkan_layer/layer/private_data.cpp ================================================ /* * Copyright (c) 2018-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include "private_data.hpp" #include "wsi/wsi_factory.hpp" #include namespace layer { static std::mutex g_data_lock; static std::unordered_map> g_instance_data; static std::unordered_map> g_device_data; template static PFN_vkVoidFunction get_proc_helper(object_type obj, get_proc_type get_proc, const char *proc_name, bool required, bool &ok) { PFN_vkVoidFunction ret = get_proc(obj, proc_name); if (nullptr == ret && required) { ok = false; } return ret; } VkResult instance_dispatch_table::populate(VkInstance instance, PFN_vkGetInstanceProcAddr get_proc) { bool ok = true; #define REQUIRED(x) \ x = reinterpret_cast(get_proc_helper(instance, get_proc, "vk" #x, true, ok)); #define OPTIONAL(x) \ x = reinterpret_cast(get_proc_helper(instance, get_proc, "vk" #x, false, ok)); INSTANCE_ENTRYPOINTS_LIST(REQUIRED, OPTIONAL); #undef REQUIRED #undef OPTIONAL return ok ? VK_SUCCESS : VK_ERROR_INITIALIZATION_FAILED; } VkResult device_dispatch_table::populate(VkDevice device, PFN_vkGetDeviceProcAddr get_proc) { bool ok = true; #define REQUIRED(x) \ x = reinterpret_cast(get_proc_helper(device, get_proc, "vk" #x, true, ok)); #define OPTIONAL(x) \ x = reinterpret_cast(get_proc_helper(device, get_proc, "vk" #x, false, ok)); DEVICE_ENTRYPOINTS_LIST(REQUIRED, OPTIONAL); #undef REQUIRED #undef OPTIONAL return ok ? VK_SUCCESS : VK_ERROR_INITIALIZATION_FAILED; } instance_private_data::instance_private_data(const instance_dispatch_table &table, PFN_vkSetInstanceLoaderData set_loader_data, util::wsi_platform_set enabled_layer_platforms) : disp(table), SetInstanceLoaderData(set_loader_data), enabled_layer_platforms(enabled_layer_platforms) {} template static inline void *get_key(dispatchable_type dispatchable_object) { return *reinterpret_cast(dispatchable_object); } void instance_private_data::set(VkInstance inst, std::unique_ptr inst_data) { scoped_mutex lock(g_data_lock); g_instance_data[get_key(inst)] = std::move(inst_data); } template static instance_private_data &get_instance_private_data(dispatchable_type dispatchable_object) { scoped_mutex lock(g_data_lock); return *g_instance_data[get_key(dispatchable_object)]; } instance_private_data &instance_private_data::get(VkInstance instance) { return get_instance_private_data(instance); } instance_private_data &instance_private_data::get(VkPhysicalDevice phys_dev) { return get_instance_private_data(phys_dev); } static VkIcdWsiPlatform get_platform_of_surface(VkSurfaceKHR surface) { VkIcdSurfaceBase *surface_base = reinterpret_cast(surface); return surface_base->platform; } bool instance_private_data::does_layer_support_surface(VkSurfaceKHR surface) { return enabled_layer_platforms.contains(get_platform_of_surface(surface)); } bool instance_private_data::do_icds_support_surface(VkPhysicalDevice, VkSurfaceKHR) { /* For now assume ICDs do not support VK_KHR_surface. This means that the layer will handle all * the surfaces it can handle (even if the ICDs can handle the surface) and only call down for * surfaces it cannot handle. In the future we may allow system integrators to configure which * ICDs have precedence handling which platforms. */ return false; } bool instance_private_data::should_layer_handle_surface(VkSurfaceKHR surface) { return surfaces.find(surface) != surfaces.end(); } void instance_private_data::destroy(VkInstance inst) { scoped_mutex lock(g_data_lock); g_instance_data.erase(get_key(inst)); } void instance_private_data::add_surface(VkSurfaceKHR surface) { scoped_mutex lock(g_data_lock); surfaces.insert(surface); } device_private_data::device_private_data(instance_private_data &inst_data, VkPhysicalDevice phys_dev, VkDevice dev, const device_dispatch_table &table, PFN_vkSetDeviceLoaderData set_loader_data) : disp{table}, instance_data{inst_data}, SetDeviceLoaderData{set_loader_data}, physical_device{phys_dev}, device{dev} {} void device_private_data::set(VkDevice dev, std::unique_ptr dev_data) { scoped_mutex lock(g_data_lock); g_device_data[get_key(dev)] = std::move(dev_data); } template static device_private_data &get_device_private_data(dispatchable_type dispatchable_object) { scoped_mutex lock(g_data_lock); return *g_device_data[get_key(dispatchable_object)]; } device_private_data &device_private_data::get(VkDevice device) { return get_device_private_data(device); } device_private_data &device_private_data::get(VkQueue queue) { return get_device_private_data(queue); } void device_private_data::add_layer_swapchain(VkSwapchainKHR swapchain) { scoped_mutex lock(swapchains_lock); swapchains.insert(swapchain); } bool device_private_data::layer_owns_all_swapchains(const VkSwapchainKHR *swapchain, uint32_t swapchain_count) const { scoped_mutex lock(swapchains_lock); for (uint32_t i = 0; i < swapchain_count; i++) { if (swapchains.find(swapchain[i]) == swapchains.end()) { return false; } } return true; } bool device_private_data::should_layer_create_swapchain(VkSurfaceKHR vk_surface) { return instance_data.should_layer_handle_surface(vk_surface); } bool device_private_data::can_icds_create_swapchain(VkSurfaceKHR vk_surface) { return disp.CreateSwapchainKHR != nullptr; } void device_private_data::destroy(VkDevice dev) { scoped_mutex lock(g_data_lock); g_device_data.erase(get_key(dev)); } } /* namespace layer */ ================================================ FILE: alvr/vulkan_layer/layer/private_data.hpp ================================================ /* * Copyright (c) 2018-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #pragma once #include "util/platform_set.hpp" #include "wsi/display.hpp" #include #include #include #include #include #include #include using scoped_mutex = std::lock_guard; namespace layer { /* List of device entrypoints in the layer's instance dispatch table. * Note that the Vulkan loader implements some of these entrypoints so the fact that these are * non-null doesn't guarantee than we can safely call them. We still mark the entrypoints with * REQUIRED() and OPTIONAL(). The layer fails if vkGetInstanceProcAddr returns null for entrypoints * that are REQUIRED(). */ #define INSTANCE_ENTRYPOINTS_LIST(REQUIRED, OPTIONAL) \ REQUIRED(GetInstanceProcAddr) \ REQUIRED(DestroyInstance) \ REQUIRED(GetPhysicalDeviceProperties) \ REQUIRED(GetPhysicalDeviceProperties2) \ REQUIRED(GetPhysicalDeviceMemoryProperties) \ REQUIRED(GetPhysicalDeviceImageFormatProperties) \ REQUIRED(EnumerateDeviceExtensionProperties) \ OPTIONAL(GetPhysicalDeviceSurfaceCapabilitiesKHR) \ OPTIONAL(GetPhysicalDeviceSurfaceFormatsKHR) \ OPTIONAL(GetPhysicalDeviceSurfacePresentModesKHR) \ OPTIONAL(GetPhysicalDeviceSurfaceSupportKHR) \ OPTIONAL(GetPhysicalDeviceDisplayPropertiesKHR) \ OPTIONAL(GetDisplayModePropertiesKHR) \ OPTIONAL(GetPhysicalDeviceDisplayPlanePropertiesKHR) \ OPTIONAL(AcquireXlibDisplayEXT) \ OPTIONAL(GetDisplayPlaneSupportedDisplaysKHR) \ OPTIONAL(CreateDisplayPlaneSurfaceKHR) \ OPTIONAL(ReleaseDisplayEXT) \ OPTIONAL(DestroySurfaceKHR) \ OPTIONAL(CreateHeadlessSurfaceEXT) \ OPTIONAL(GetPhysicalDeviceQueueFamilyProperties) \ OPTIONAL(CreateDisplayModeKHR) \ struct instance_dispatch_table { VkResult populate(VkInstance instance, PFN_vkGetInstanceProcAddr get_proc); #define DISPATCH_TABLE_ENTRY(x) PFN_vk##x x{}; INSTANCE_ENTRYPOINTS_LIST(DISPATCH_TABLE_ENTRY, DISPATCH_TABLE_ENTRY) #undef DISPATCH_TABLE_ENTRY }; /* List of device entrypoints in the layer's device dispatch table. * The layer fails initializing a device instance when entrypoints marked with REQUIRED() are * retrieved as null. The layer will instead tolerate retrieving a null for entrypoints marked as * OPTIONAL(). Code in the layer needs to check these entrypoints are non-null before calling them. * * Note that we cannot rely on checking whether the physical device supports a particular extension * as the Vulkan loader currently aggregates all extensions advertised by all implicit layers (in * their JSON manifests) and adds them automatically to the output of * vkEnumeratePhysicalDeviceProperties. */ #define DEVICE_ENTRYPOINTS_LIST(REQUIRED, OPTIONAL) \ REQUIRED(GetDeviceProcAddr) \ REQUIRED(GetDeviceQueue) \ REQUIRED(QueueSubmit) \ REQUIRED(QueueWaitIdle) \ REQUIRED(CreateCommandPool) \ REQUIRED(DestroyCommandPool) \ REQUIRED(AllocateCommandBuffers) \ REQUIRED(FreeCommandBuffers) \ REQUIRED(ResetCommandBuffer) \ REQUIRED(BeginCommandBuffer) \ REQUIRED(EndCommandBuffer) \ REQUIRED(CreateImage) \ REQUIRED(DestroyImage) \ REQUIRED(GetImageMemoryRequirements) \ REQUIRED(BindImageMemory) \ REQUIRED(AllocateMemory) \ REQUIRED(FreeMemory) \ REQUIRED(CreateFence) \ REQUIRED(DestroyFence) \ REQUIRED(ResetFences) \ REQUIRED(WaitForFences) \ OPTIONAL(CreateSwapchainKHR) \ OPTIONAL(DestroySwapchainKHR) \ OPTIONAL(GetSwapchainImagesKHR) \ OPTIONAL(AcquireNextImageKHR) \ OPTIONAL(QueuePresentKHR) \ OPTIONAL(GetSwapchainCounterEXT) \ OPTIONAL(RegisterDisplayEventEXT) \ OPTIONAL(GetFenceStatus) \ OPTIONAL(GetMemoryFdKHR) \ OPTIONAL(CreateSemaphore) \ OPTIONAL(GetSemaphoreFdKHR) struct device_dispatch_table { VkResult populate(VkDevice dev, PFN_vkGetDeviceProcAddr get_proc); #define DISPATCH_TABLE_ENTRY(x) PFN_vk##x x{}; DEVICE_ENTRYPOINTS_LIST(DISPATCH_TABLE_ENTRY, DISPATCH_TABLE_ENTRY) #undef DISPATCH_TABLE_ENTRY }; /** * @brief Layer "mirror object" for VkInstance. */ class instance_private_data { public: instance_private_data() = delete; instance_private_data(const instance_private_data &) = delete; instance_private_data &operator=(const instance_private_data &) = delete; instance_private_data(const instance_dispatch_table &table, PFN_vkSetInstanceLoaderData set_loader_data, util::wsi_platform_set enabled_layer_platforms); static void set(VkInstance inst, std::unique_ptr inst_data); /** * @brief Get the mirror object that the layer associates to a given Vulkan instance. */ static instance_private_data &get(VkInstance instance); /** * @brief Get the layer instance object associated to the VkInstance owning the specified * VkPhysicalDevice. */ static instance_private_data &get(VkPhysicalDevice phys_dev); /** * @brief Get the set of enabled platforms that are also supported by the layer. */ const util::wsi_platform_set &get_enabled_platforms() { return enabled_layer_platforms; } /** * @brief Check whether a surface command should be handled by the WSI layer. * * @param phys_dev Physical device involved in the Vulkan command. * @param surface The surface involved in the Vulkan command. * * @retval @c true if the layer should handle commands for the specified surface, which may mean * returning an error if the layer does not support @p surface 's platform. * * @retval @c false if the layer should call down to the layers and ICDs below to handle the * surface commands. */ bool should_layer_handle_surface(VkSurfaceKHR surface); /** * @brief Check whether the given surface is supported for presentation via the layer. * * @param surface A VK_KHR_surface surface. * * @return Whether the WSI layer supports this surface. */ bool does_layer_support_surface(VkSurfaceKHR surface); void add_surface(VkSurfaceKHR); static void destroy(VkInstance inst); const instance_dispatch_table disp; private: /** * @brief Check whether the given surface is already supported for presentation without the * layer. */ bool do_icds_support_surface(VkPhysicalDevice phys_dev, VkSurfaceKHR surface); const PFN_vkSetInstanceLoaderData SetInstanceLoaderData; const util::wsi_platform_set enabled_layer_platforms; std::unordered_set surfaces; }; class device_private_data { public: device_private_data() = delete; device_private_data(const device_private_data &) = delete; device_private_data &operator=(const device_private_data &) = delete; device_private_data(instance_private_data &inst_data, VkPhysicalDevice phys_dev, VkDevice dev, const device_dispatch_table &table, PFN_vkSetDeviceLoaderData set_loader_data); static void set(VkDevice dev, std::unique_ptr dev_data); /** * @brief Get the mirror object that the layer associates to a given Vulkan device. */ static device_private_data &get(VkDevice device); /** * @brief Get the layer device object associated to the VkDevice owning the specified VkQueue. */ static device_private_data &get(VkQueue queue); void add_layer_swapchain(VkSwapchainKHR swapchain); /** * @brief Return whether all the provided swapchains are owned by us (the WSI Layer). */ bool layer_owns_all_swapchains(const VkSwapchainKHR *swapchain, uint32_t swapchain_count) const; /** * @brief Check whether the given swapchain is owned by us (the WSI Layer). */ bool layer_owns_swapchain(VkSwapchainKHR swapchain) const { return layer_owns_all_swapchains(&swapchain, 1); } /** * @brief Check whether the layer can create a swapchain for the given surface. */ bool should_layer_create_swapchain(VkSurfaceKHR vk_surface); /** * @brief Check whether the ICDs or layers below support VK_KHR_swapchain. */ bool can_icds_create_swapchain(VkSurfaceKHR vk_surface); static void destroy(VkDevice dev); const device_dispatch_table disp; instance_private_data &instance_data; const PFN_vkSetDeviceLoaderData SetDeviceLoaderData; const VkPhysicalDevice physical_device; const VkDevice device; std::unique_ptr display; private: std::unordered_set swapchains; mutable std::mutex swapchains_lock; }; } /* namespace layer */ ================================================ FILE: alvr/vulkan_layer/layer/settings.cpp ================================================ #include "settings.h" #define PICOJSON_USE_INT64 #include "alvr_server/include/picojson.h" #include #include #include #include #include #include "layer.h" #include "util/logger.h" using namespace std; extern uint64_t g_DriverTestMode; Settings Settings::m_Instance; Settings::Settings() : m_loaded(false) { } Settings::~Settings() { } void Settings::Load() { try { auto sessionFile = std::ifstream(g_sessionPath); auto json = std::string( std::istreambuf_iterator(sessionFile), std::istreambuf_iterator()); picojson::value v; std::string err = picojson::parse(v, json); if (!err.empty()) { Error("Error on parsing session config (%s): %hs\n", g_sessionPath, err.c_str()); return; } auto config = v.get("openvr_config"); m_renderWidth = config.get("eye_resolution_width").get() * 2; m_renderHeight = config.get("eye_resolution_height").get(); m_refreshRate = (int)config.get("refresh_rate").get(); Debug("Config JSON: %hs\n", json.c_str()); Info("Render Target: %d %d\n", m_renderWidth, m_renderHeight); Info("Refresh Rate: %d\n", m_refreshRate); m_loaded = true; } catch (std::exception &e) { Error("Exception on parsing session config (%s): %hs\n", g_sessionPath, e.what()); } } ================================================ FILE: alvr/vulkan_layer/layer/settings.h ================================================ #pragma once #include #include class Settings { static Settings m_Instance; bool m_loaded; Settings(); virtual ~Settings(); public: void Load(); static Settings &Instance() { return m_Instance; } bool IsLoaded() { return m_loaded; } int m_refreshRate; uint32_t m_renderWidth; uint32_t m_renderHeight; }; ================================================ FILE: alvr/vulkan_layer/layer/surface_api.cpp ================================================ /* * Copyright (c) 2016-2017, 2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include "surface_api.hpp" #include "private_data.hpp" #include #include extern "C" { /** * @brief Implements vkGetPhysicalDeviceSurfaceCapabilitiesKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfaceCapabilitiesKHR( VkPhysicalDevice physicalDevice, VkSurfaceKHR surface, VkSurfaceCapabilitiesKHR *pSurfaceCapabilities) { auto &instance = layer::instance_private_data::get(physicalDevice); if (instance.should_layer_handle_surface(surface)) { wsi::surface_properties *props = wsi::get_surface_properties(surface); assert(props != nullptr); return props->get_surface_capabilities(physicalDevice, surface, pSurfaceCapabilities); } /* If the layer cannot handle this surface, then necessarily the surface must have been created * by the ICDs (or a layer below us.) So it is safe to assume that the ICDs (or layers below us) * support VK_KHR_surface and therefore it is safe to can call down. This holds for other * entrypoints below. */ return instance.disp.GetPhysicalDeviceSurfaceCapabilitiesKHR(physicalDevice, surface, pSurfaceCapabilities); } /** * @brief Implements vkGetPhysicalDeviceSurfaceFormatsKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfaceFormatsKHR( VkPhysicalDevice physicalDevice, VkSurfaceKHR surface, uint32_t *pSurfaceFormatCount, VkSurfaceFormatKHR *pSurfaceFormats) { auto &instance = layer::instance_private_data::get(physicalDevice); if (instance.should_layer_handle_surface(surface)) { wsi::surface_properties *props = wsi::get_surface_properties(surface); assert(props != nullptr); return props->get_surface_formats(physicalDevice, surface, pSurfaceFormatCount, pSurfaceFormats); } return instance.disp.GetPhysicalDeviceSurfaceFormatsKHR(physicalDevice, surface, pSurfaceFormatCount, pSurfaceFormats); } /** * @brief Implements vkGetPhysicalDeviceSurfacePresentModesKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfacePresentModesKHR( VkPhysicalDevice physicalDevice, VkSurfaceKHR surface, uint32_t *pPresentModeCount, VkPresentModeKHR *pPresentModes) { auto &instance = layer::instance_private_data::get(physicalDevice); if (instance.should_layer_handle_surface(surface)) { wsi::surface_properties *props = wsi::get_surface_properties(surface); assert(props != nullptr); return props->get_surface_present_modes(physicalDevice, surface, pPresentModeCount, pPresentModes); } return instance.disp.GetPhysicalDeviceSurfacePresentModesKHR(physicalDevice, surface, pPresentModeCount, pPresentModes); } /** * @brief Implements vkGetPhysicalDeviceSurfaceSupportKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfaceSupportKHR(VkPhysicalDevice physicalDevice, uint32_t queueFamilyIndex, VkSurfaceKHR surface, VkBool32 *pSupported) { auto &instance = layer::instance_private_data::get(physicalDevice); if (instance.should_layer_handle_surface(surface)) { *pSupported = VK_TRUE; return VK_SUCCESS; } return instance.disp.GetPhysicalDeviceSurfaceSupportKHR(physicalDevice, queueFamilyIndex, surface, pSupported); } } /* extern "C" */ ================================================ FILE: alvr/vulkan_layer/layer/surface_api.hpp ================================================ /* * Copyright (c) 2018-2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file surface_api.hpp * * @brief Contains the Vulkan entrypoints for the VkSurfaceKHR. */ #pragma once #include extern "C" { /** * @brief Implements vkGetPhysicalDeviceSurfaceCapabilitiesKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfaceCapabilitiesKHR( VkPhysicalDevice physicalDevice, VkSurfaceKHR surface, VkSurfaceCapabilitiesKHR *pSurfaceCapabilities); /** * @brief Implements vkGetPhysicalDeviceSurfaceFormatsKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfaceFormatsKHR( VkPhysicalDevice physicalDevice, VkSurfaceKHR surface, uint32_t *pSurfaceFormatCount, VkSurfaceFormatKHR *pSurfaceFormats); /** * @brief Implements vkGetPhysicalDeviceSurfacePresentModesKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfacePresentModesKHR( VkPhysicalDevice physicalDevice, VkSurfaceKHR surface, uint32_t *pPresentModeCount, VkPresentModeKHR *pPresentModes); /** * @brief Implements vkGetPhysicalDeviceSurfaceSupportKHR Vulkan entrypoint. */ VKAPI_ATTR VkResult wsi_layer_vkGetPhysicalDeviceSurfaceSupportKHR(VkPhysicalDevice physicalDevice, uint32_t queueFamilyIndex, VkSurfaceKHR surface, VkBool32 *pSupported); } ================================================ FILE: alvr/vulkan_layer/layer/swapchain_api.cpp ================================================ /* * Copyright (c) 2017, 2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file swapchain_api.cpp * * @brief Contains the Vulkan entrypoints for the swapchain. */ #include #include #include #include #include "private_data.hpp" #include "swapchain_api.hpp" extern "C" { VKAPI_ATTR VkResult wsi_layer_vkCreateSwapchainKHR( VkDevice device, const VkSwapchainCreateInfoKHR *pSwapchainCreateInfo, const VkAllocationCallbacks *pAllocator, VkSwapchainKHR *pSwapchain) { assert(pSwapchain != nullptr); layer::device_private_data &device_data = layer::device_private_data::get(device); VkSurfaceKHR surface = pSwapchainCreateInfo->surface; if (!device_data.should_layer_create_swapchain(surface)) { if (!device_data.can_icds_create_swapchain(surface)) { return VK_ERROR_INITIALIZATION_FAILED; } return device_data.disp.CreateSwapchainKHR(device_data.device, pSwapchainCreateInfo, pAllocator, pSwapchain); } wsi::swapchain_base *sc = wsi::allocate_surface_swapchain(surface, device_data, pAllocator); if (sc == nullptr) { return VK_ERROR_OUT_OF_HOST_MEMORY; } VkResult result = sc->init(device, pSwapchainCreateInfo); if (result != VK_SUCCESS) { /* Error occured during initialization, need to free allocated memory. */ wsi::destroy_surface_swapchain(sc, pAllocator); return result; } *pSwapchain = reinterpret_cast(sc); device_data.add_layer_swapchain(*pSwapchain); return result; } VKAPI_ATTR void wsi_layer_vkDestroySwapchainKHR(VkDevice device, VkSwapchainKHR swapc, const VkAllocationCallbacks *pAllocator) { layer::device_private_data &device_data = layer::device_private_data::get(device); if (!device_data.layer_owns_swapchain(swapc)) { return device_data.disp.DestroySwapchainKHR(device_data.device, swapc, pAllocator); } assert(swapc != VK_NULL_HANDLE); wsi::swapchain_base *sc = reinterpret_cast(swapc); wsi::destroy_surface_swapchain(sc, pAllocator); } VKAPI_ATTR VkResult wsi_layer_vkGetSwapchainImagesKHR(VkDevice device, VkSwapchainKHR swapc, uint32_t *pSwapchainImageCount, VkImage *pSwapchainImages) { layer::device_private_data &device_data = layer::device_private_data::get(device); if (!device_data.layer_owns_swapchain(swapc)) { return device_data.disp.GetSwapchainImagesKHR(device_data.device, swapc, pSwapchainImageCount, pSwapchainImages); } assert(pSwapchainImageCount != nullptr); assert(swapc != VK_NULL_HANDLE); wsi::swapchain_base *sc = reinterpret_cast(swapc); return sc->get_swapchain_images(pSwapchainImageCount, pSwapchainImages); } VKAPI_ATTR VkResult wsi_layer_vkAcquireNextImageKHR(VkDevice device, VkSwapchainKHR swapc, uint64_t timeout, VkSemaphore semaphore, VkFence fence, uint32_t *pImageIndex) { layer::device_private_data &device_data = layer::device_private_data::get(device); if (!device_data.layer_owns_swapchain(swapc)) { return device_data.disp.AcquireNextImageKHR(device_data.device, swapc, timeout, semaphore, fence, pImageIndex); } assert(swapc != VK_NULL_HANDLE); assert(semaphore != VK_NULL_HANDLE || fence != VK_NULL_HANDLE); assert(pImageIndex != nullptr); wsi::swapchain_base *sc = reinterpret_cast(swapc); return sc->acquire_next_image(timeout, semaphore, fence, pImageIndex); } VKAPI_ATTR VkResult wsi_layer_vkQueuePresentKHR(VkQueue queue, const VkPresentInfoKHR *pPresentInfo) { assert(queue != VK_NULL_HANDLE); assert(pPresentInfo != nullptr); layer::device_private_data &device_data = layer::device_private_data::get(queue); if (!device_data.layer_owns_all_swapchains(pPresentInfo->pSwapchains, pPresentInfo->swapchainCount)) { return device_data.disp.QueuePresentKHR(queue, pPresentInfo); } VkResult ret = VK_SUCCESS; for (uint32_t i = 0; i < pPresentInfo->swapchainCount; ++i) { VkSwapchainKHR swapc = pPresentInfo->pSwapchains[i]; wsi::swapchain_base *sc = reinterpret_cast(swapc); assert(sc != nullptr); VkResult res = sc->queue_present(queue, pPresentInfo, pPresentInfo->pImageIndices[i]); if (pPresentInfo->pResults != nullptr) { pPresentInfo->pResults[i] = res; } if (res != VK_SUCCESS && ret == VK_SUCCESS) { ret = res; } } return ret; } VKAPI_ATTR VkResult wsi_layer_vkGetSwapchainCounterEXT(VkDevice device, VkSwapchainKHR swapchain, VkSurfaceCounterFlagBitsEXT counter, uint64_t *pCounterValue) { layer::device_private_data &device_data = layer::device_private_data::get(device); if (!device_data.layer_owns_swapchain(swapchain)) { return device_data.disp.GetSwapchainCounterEXT(device, swapchain, counter, pCounterValue); } if (VK_SURFACE_COUNTER_VBLANK_EXT == counter) { *pCounterValue = device_data.display->m_vsync_count; } return VK_SUCCESS; } } /* extern "C" */ ================================================ FILE: alvr/vulkan_layer/layer/swapchain_api.hpp ================================================ /* * Copyright (c) 2018-2019 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file swapchain_api.hpp * * @brief Contains the Vulkan entrypoints for the swapchain. */ #pragma once #include extern "C" { VKAPI_ATTR VkResult wsi_layer_vkCreateSwapchainKHR( VkDevice device, const VkSwapchainCreateInfoKHR *pSwapchainCreateInfo, const VkAllocationCallbacks *pAllocator, VkSwapchainKHR *pSwapchain); VKAPI_ATTR void wsi_layer_vkDestroySwapchainKHR(VkDevice device, VkSwapchainKHR swapc, const VkAllocationCallbacks *pAllocator); VKAPI_ATTR VkResult wsi_layer_vkGetSwapchainImagesKHR(VkDevice device, VkSwapchainKHR swapc, uint32_t *pSwapchainImageCount, VkImage *pSwapchainImages); VKAPI_ATTR VkResult wsi_layer_vkAcquireNextImageKHR(VkDevice device, VkSwapchainKHR swapc, uint64_t timeout, VkSemaphore semaphore, VkFence fence, uint32_t *pImageIndex); VKAPI_ATTR VkResult wsi_layer_vkQueuePresentKHR(VkQueue queue, const VkPresentInfoKHR *pPresentInfo); VKAPI_ATTR VkResult wsi_layer_vkGetSwapchainCounterEXT(VkDevice device, VkSwapchainKHR swapchain, VkSurfaceCounterFlagBitsEXT counter, uint64_t *pCounterValue); } ================================================ FILE: alvr/vulkan_layer/src/lib.rs ================================================ #![cfg(target_os = "linux")] #![allow( dead_code, non_camel_case_types, non_snake_case, non_upper_case_globals, unsafe_op_in_unsafe_fn, unused_imports, clippy::missing_safety_doc, clippy::ptr_offset_with_cast, clippy::too_many_arguments, clippy::useless_transmute, clippy::pedantic, clippy::nursery )] use std::ffi::CString; mod bindings { include!(concat!(env!("OUT_DIR"), "/layer_bindings.rs")); } use bindings::*; #[unsafe(no_mangle)] pub unsafe extern "C" fn ALVR_Negotiate(nli: *mut VkNegotiateLayerInterface) -> VkResult { unsafe { g_sessionPath = CString::new( alvr_filesystem::filesystem_layout_invalid() .session() .to_string_lossy() .to_string(), ) .unwrap() .into_raw(); bindings::wsi_layer_Negotiate(nli) } } ================================================ FILE: alvr/vulkan_layer/util/custom_allocator.cpp ================================================ /* * Copyright (c) 2020-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include "custom_allocator.hpp" extern "C" { static void *default_allocation(void *, size_t size, size_t, VkSystemAllocationScope) { return malloc(size); } static void *default_reallocation(void *, void *pOriginal, size_t size, size_t, VkSystemAllocationScope) { return realloc(pOriginal, size); } static void default_free(void *, void *pMemory) { free(pMemory); } } namespace util { const allocator &allocator::get_generic() { static allocator generic{nullptr, VK_SYSTEM_ALLOCATION_SCOPE_COMMAND}; return generic; } allocator::allocator(const allocator &other, VkSystemAllocationScope new_scope) : allocator{other.get_original_callbacks(), new_scope} {} /* If callbacks is already populated by vulkan then use those specified as default. */ allocator::allocator(const VkAllocationCallbacks *callbacks, VkSystemAllocationScope scope) { m_scope = scope; if (callbacks != nullptr) { m_callbacks = *callbacks; } else { m_callbacks = {}; m_callbacks.pfnAllocation = default_allocation; m_callbacks.pfnReallocation = default_reallocation; m_callbacks.pfnFree = default_free; } } const VkAllocationCallbacks *allocator::get_original_callbacks() const { return m_callbacks.pfnAllocation == default_allocation ? nullptr : &m_callbacks; } } /* namespace util */ ================================================ FILE: alvr/vulkan_layer/util/custom_allocator.hpp ================================================ /* * Copyright (c) 2020-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include #include #include #include #include #pragma once namespace util { /** * @brief Minimalistic wrapper of VkAllocationCallbacks. */ class allocator { public: /** * @brief Get an allocator that can be used if VkAllocationCallbacks are not provided. */ static const allocator &get_generic(); /** * @brief Construct a new wrapper for the given VK callbacks and scope. * @param callbacks Pointer to allocation callbacks. If this is @c nullptr, then default * allocation callbacks are used. These can be accessed through #m_callbacks. * @param scope The scope to use for this allocator. */ allocator(const VkAllocationCallbacks *callbacks, VkSystemAllocationScope scope); /** * @brief Copy the given allocator, but change the allocation scope. */ allocator(const allocator &other, VkSystemAllocationScope new_scope); /** * @brief Get a pointer to the allocation callbacks provided while constructing this object. * @return a copy of the #VkAllocationCallback argument provided in the allocator constructor * or @c nullptr if this argument was provided as @c nullptr. * @note The #m_callbacks member is always populated with callable pointers for pfnAllocation, * pfnReallocation and pfnFree. */ const VkAllocationCallbacks *get_original_callbacks() const; /** * @brief Helper method to allocate and construct objects with a custom allocator. * @param num_objects Number of objects to create. * @return Pointer to the newly created objects or @c nullptr if allocation failed. */ template T *create(size_t num_objects, arg_types &&...args) const noexcept; /** * @brief Helper method to destroy and deallocate objects constructed with allocator::create(). * @param num_objects Number of objects to destroy. */ template void destroy(size_t num_objects, T *obj) const noexcept; VkAllocationCallbacks m_callbacks; VkSystemAllocationScope m_scope; }; /** * @brief Implementation of an allocator that can be used with STL containers. */ template class custom_allocator { public: using value_type = T; using pointer = T *; custom_allocator(const allocator &alloc) : m_alloc(alloc) {} template custom_allocator(const custom_allocator &other) : m_alloc(other.get_data()) {} const allocator &get_data() const { return m_alloc; } pointer allocate(size_t n) const { size_t size = n * sizeof(T); auto &cb = m_alloc.m_callbacks; void *ret = cb.pfnAllocation(cb.pUserData, size, alignof(T), m_alloc.m_scope); if (ret == nullptr) throw std::bad_alloc(); return reinterpret_cast(ret); } pointer allocate(size_t n, void *ptr) const { size_t size = n * sizeof(T); auto &cb = m_alloc.m_callbacks; void *ret = cb.pfnReallocation(cb.pUserData, ptr, size, alignof(T), m_alloc.m_scope); if (ret == nullptr) throw std::bad_alloc(); return reinterpret_cast(ret); } void deallocate(void *ptr, size_t) const noexcept { m_alloc.m_callbacks.pfnFree(m_alloc.m_callbacks.pUserData, ptr); } private: const allocator m_alloc; }; template bool operator==(const custom_allocator &, const custom_allocator &) { return true; } template bool operator!=(const custom_allocator &, const custom_allocator &) { return false; } template T *allocator::create(size_t num_objects, arg_types &&...args) const noexcept { if (num_objects < 1) { return nullptr; } custom_allocator allocator(*this); T *ptr; try { ptr = allocator.allocate(num_objects); } catch (...) { return nullptr; } size_t objects_constructed = 0; try { while (objects_constructed < num_objects) { T *next_object = &ptr[objects_constructed]; new (next_object) T(std::forward(args)...); objects_constructed++; } } catch (...) { /* We catch all exceptions thrown while constructing the object, not just * std::bad_alloc. */ while (objects_constructed > 0) { objects_constructed--; ptr[objects_constructed].~T(); } allocator.deallocate(ptr, num_objects); return nullptr; } return ptr; } template void allocator::destroy(size_t num_objects, T *objects) const noexcept { assert((objects == nullptr) == (num_objects == 0)); if (num_objects == 0) { return; } custom_allocator allocator(*this); for (size_t i = 0; i < num_objects; i++) { objects[i].~T(); } allocator.deallocate(objects, num_objects); } template void destroy_custom(T *obj) { T::destroy(obj); } /** * @brief Vector using a Vulkan custom allocator to allocate its elements. * @note The vector must be passed a custom_allocator during construction and it takes a copy * of it, meaning that the user is free to destroy the custom_allocator after constructing the * vector. */ template class vector : public std::vector> { public: using base = std::vector>; using base::base; /* Delete all methods that can cause allocation failure, i.e. can throw std::bad_alloc. * * Rationale: we want to force users to use our corresponding try_... method instead: * this makes the API slightly more annoying to use, but hopefully safer as it encourages * users to check for allocation failures, which is important for Vulkan. * * Note: deleting each of these methods (below) deletes all its overloads from the base class, * to be precise: the deleted method covers the methods (all overloads) in the base class. * Note: clear() is already noexcept since C++11. */ void insert() = delete; void emplace() = delete; void emplace_back() = delete; void push_back() = delete; void resize() = delete; void reserve() = delete; /* Note pop_back(), erase(), clear() do not throw std::bad_alloc exceptions. */ /* @brief Like std::vector::push_back, but non throwing. * @return @c false iff the operation could not be performed due to an allocation failure. */ template bool try_push_back(arg_types &&...args) noexcept { try { base::push_back(std::forward(args)...); return true; } catch (const std::bad_alloc &e) { return false; } } /* @brief push back multiple elements at once * @return @c false iff the operation could not be performed due to an allocation failure. */ bool try_push_back_many(const T *begin, const T *end) noexcept { for (const T *it = begin; it != end; ++it) { if (!try_push_back(*it)) { return false; } } return true; } /* @brief Like std::vector::resize, but non throwing. * @return @c false iff the operation could not be performed due to an allocation failure. */ template bool try_resize(arg_types &&...args) noexcept { try { base::resize(std::forward(args)...); return true; } catch (const std::bad_alloc &e) { return false; } } }; } /* namespace util */ ================================================ FILE: alvr/vulkan_layer/util/extension_list.cpp ================================================ /* * Copyright (c) 2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include "extension_list.hpp" #include #include #include namespace util { extension_list::extension_list(const util::allocator &allocator) : m_alloc{allocator}, m_ext_props(allocator) {} VkResult extension_list::add(const struct VkEnumerateInstanceExtensionPropertiesChain *chain) { uint32_t count; VkResult m_error = chain->CallDown(nullptr, &count, nullptr); if (m_error == VK_SUCCESS) { if (!m_ext_props.try_resize(count)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } m_error = chain->CallDown(nullptr, &count, m_ext_props.data()); } return m_error; } VkResult extension_list::add(VkPhysicalDevice dev) { layer::instance_private_data &inst_data = layer::instance_private_data::get(dev); uint32_t count; VkResult m_error = inst_data.disp.EnumerateDeviceExtensionProperties(dev, nullptr, &count, nullptr); if (m_error == VK_SUCCESS) { if (!m_ext_props.try_resize(count)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } m_error = inst_data.disp.EnumerateDeviceExtensionProperties(dev, nullptr, &count, m_ext_props.data()); } return m_error; } VkResult extension_list::add( PFN_vkEnumerateInstanceExtensionProperties fpEnumerateInstanceExtensionProperties) { uint32_t count = 0; VkResult m_error = fpEnumerateInstanceExtensionProperties(nullptr, &count, nullptr); if (m_error == VK_SUCCESS) { if (!m_ext_props.try_resize(count)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } m_error = fpEnumerateInstanceExtensionProperties(nullptr, &count, m_ext_props.data()); } return m_error; } VkResult extension_list::add(const char *const *extensions, uint32_t count) { for (uint32_t i = 0; i < count; i++) { VkExtensionProperties props = {}; strncpy(props.extensionName, extensions[i], sizeof(props.extensionName) - 1); props.extensionName[sizeof(props.extensionName) - 1] = '\0'; if (!m_ext_props.try_push_back(props)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } } return VK_SUCCESS; } VkResult extension_list::add(const VkExtensionProperties *props, uint32_t count) { if (!m_ext_props.try_push_back_many(props, props + count)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } return VK_SUCCESS; } VkResult extension_list::add(const char *ext) { if (!contains(ext)) { VkExtensionProperties props = {}; strncpy(props.extensionName, ext, sizeof(props.extensionName) - 1); props.extensionName[sizeof(props.extensionName) - 1] = '\0'; if (!m_ext_props.try_push_back(props)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } } return VK_SUCCESS; } VkResult extension_list::add(VkExtensionProperties ext_prop) { if (!contains(ext_prop.extensionName)) { if (!m_ext_props.try_push_back(ext_prop)) { return VK_ERROR_OUT_OF_HOST_MEMORY; } } return VK_SUCCESS; } VkResult extension_list::add(const char **ext_list, uint32_t count) { for (uint32_t i = 0; i < count; i++) { if (add(ext_list[i]) != VK_SUCCESS) { return VK_ERROR_OUT_OF_HOST_MEMORY; } } return VK_SUCCESS; } VkResult extension_list::add(const extension_list &ext_list) { util::vector ext_vect = ext_list.get_extension_props(); for (auto &ext : ext_vect) { if (add(ext) != VK_SUCCESS) { return VK_ERROR_OUT_OF_HOST_MEMORY; } } return VK_SUCCESS; } bool extension_list::get_extension_strings(util::vector &out) const { size_t old_size = out.size(); size_t new_size = old_size + m_ext_props.size(); if (!out.try_resize(new_size)) { return false; } for (size_t i = old_size; i < new_size; i++) { out[i] = m_ext_props[i - old_size].extensionName; } return true; } bool extension_list::contains(const extension_list &req) const { for (const auto &req_ext : req.m_ext_props) { if (!contains(req_ext.extensionName)) { return false; } } return true; } bool extension_list::contains(const char *extension_name) const { for (const auto &p : m_ext_props) { if (strcmp(p.extensionName, extension_name) == 0) { return true; } } return false; } void extension_list::remove(const char *ext) { m_ext_props.erase(std::remove_if(m_ext_props.begin(), m_ext_props.end(), [&ext](VkExtensionProperties ext_prop) { return (strcmp(ext_prop.extensionName, ext) == 0); })); } } // namespace util ================================================ FILE: alvr/vulkan_layer/util/extension_list.hpp ================================================ /* * Copyright (c) 2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #pragma once #include "util/custom_allocator.hpp" #include #include #include #include namespace util { class extension_list { public: extension_list(const util::allocator &allocator); extension_list(const extension_list &rhs) = delete; const extension_list &operator=(const extension_list &rhs) = delete; /** * @brief Obtain a vector of #VkExtensionProperties equivalent to this extension_list object. */ const util::vector &get_extension_props() const { return m_ext_props; } /** * @brief Get the allocator used to manage the memory of this object. */ const util::allocator get_allocator() const { return m_alloc; } /** * @brief Append pointers to extension strings to the given vector. * * @warning Pointers in the vector are referring to string allocated in this extension_list and * will become invalid if the extension_list is modified (e.g. by adding/removing elements.) * * @param[out] out A vector of C strings to which all extension are appended. * * @return A boolean indicating whether the operation was successful. If this is @c false, then * @p out is unmodified. */ bool get_extension_strings(util::vector &out) const; bool contains(const extension_list &req) const; bool contains(const char *ext) const; void remove(const char *ext); VkResult add(const char *ext); VkResult add(VkExtensionProperties ext_prop); VkResult add(const char **ext_list, uint32_t count); VkResult add(const extension_list &ext_list); VkResult add(const struct VkEnumerateInstanceExtensionPropertiesChain *chain); VkResult add(PFN_vkEnumerateInstanceExtensionProperties fpEnumerateInstanceExtensionProperties); VkResult add(VkPhysicalDevice dev); VkResult add(const char *const *extensions, uint32_t count); VkResult add(const VkExtensionProperties *props, uint32_t count); private: util::allocator m_alloc; util::vector m_ext_props; }; } // namespace util ================================================ FILE: alvr/vulkan_layer/util/logger.cpp ================================================ #include #include #include void _log(const char *format, va_list args, bool err) { vfprintf(err ? stderr : stdout, format, args); } void Error(const char *format, ...) { va_list args; va_start(args, format); _log(format, args, true); va_end(args); } void Warn(const char *format, ...) { va_list args; va_start(args, format); _log(format, args, true); va_end(args); } void Info(const char *format, ...) { va_list args; va_start(args, format); _log(format, args, false); va_end(args); } void Debug(const char *format, ...) { if (getenv("ALVR_LOG_DEBUG") == NULL) return; va_list args; va_start(args, format); _log(format, args, true); va_end(args); } ================================================ FILE: alvr/vulkan_layer/util/logger.h ================================================ #pragma once void Error(const char *format, ...); void Warn(const char *format, ...); void Info(const char *format, ...); void Debug(const char *format, ...); ================================================ FILE: alvr/vulkan_layer/util/platform_set.hpp ================================================ /* * Copyright (c) 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #pragma once #include #include #include namespace util { /** * @brief Set of WSI platforms. * @note This could be implemented via std::unordered_set, but would require handling allocation * callbacks and would therefore be less convenient to use. Instead, we can store all info in the * bits of uint64_t. */ class wsi_platform_set { public: void add(VkIcdWsiPlatform p) { m_platforms |= (static_cast(1) << to_int(p)); } bool contains(VkIcdWsiPlatform p) const { return (m_platforms & (static_cast(1) << to_int(p))) != 0; } private: /** * @brief Convert a VkIcdWsiPlatform to an integer between 0-63. */ static int to_int(VkIcdWsiPlatform p) { assert(static_cast(p) >= 0 && static_cast(p) < 64); return static_cast(p); } uint64_t m_platforms = 0; }; } /* namespace util */ ================================================ FILE: alvr/vulkan_layer/util/pose.cpp ================================================ #include "pose.hpp" #include #include #define UNW_LOCAL_ONLY #include namespace { inline HmdMatrix34_t transposeMul33(const HmdMatrix34_t& a) { HmdMatrix34_t result; for (unsigned i = 0; i < 3; i++) { for (unsigned k = 0; k < 3; k++) { result.m[i][k] = a.m[k][i]; } } result.m[0][3] = a.m[0][3]; result.m[1][3] = a.m[1][3]; result.m[2][3] = a.m[2][3]; return result; } inline HmdMatrix34_t matMul33(const HmdMatrix34_t& a, const HmdMatrix34_t& b) { HmdMatrix34_t result; for (unsigned i = 0; i < 3; i++) { for (unsigned j = 0; j < 3; j++) { result.m[i][j] = 0.0f; for (unsigned k = 0; k < 3; k++) { result.m[i][j] += a.m[i][k] * b.m[k][j]; } } } return result; } bool check_pose(const TrackedDevicePose_t & p) { if (p.bPoseIsValid != 1 or p.bDeviceIsConnected != 1) return false; if (p.eTrackingResult != 200) return false; auto m = matMul33(p.mDeviceToAbsoluteTracking, transposeMul33(p.mDeviceToAbsoluteTracking)); for (int i = 0 ; i < 3; ++i ) { for (int j = 0 ; j < 3 ; ++j) { if (std::abs(m.m[i][j] - (i == j)) > 0.1) return false; } } return true; } } // For a smooth experience, the correct pose for a frame must be known. // Of course this is not part of vulkan parameters, so we must inspect // the stack. // First we look for the correct function (CRenderThread::UpdateAsync), // then we scan all the local variables, and check for a suitable one. // Such a variable is a TrackedDevicePose_t, with both booleans to true, // which we compare to 1 to avoid false positives, a tracking result of // 200, and a rotation matrix (A*transpose(A)) close to identity. const TrackedDevicePose_t & find_pose_in_call_stack() { static TrackedDevicePose_t * res; if (res != nullptr) return *res; static TrackedDevicePose_t notfound; unw_context_t ctx; unw_getcontext(&ctx); unw_cursor_t cursor; unw_init_local(&cursor, &ctx); while (unw_step(&cursor) > 0) { char name[1024]; unw_word_t off; unw_get_proc_name(&cursor, name, sizeof(name), &off); if ((strcmp("_ZN13CRenderThread11UpdateAsyncEv", name) == 0) || (strcmp("_ZN13CRenderThread6UpdateEv", name) == 0)) { unw_word_t sp, sp_end; unw_get_reg(&cursor, UNW_REG_SP, &sp); unw_step(&cursor); unw_get_reg(&cursor, UNW_REG_SP, &sp_end); for (uintptr_t addr = sp ; addr < sp_end; addr += 4) { TrackedDevicePose_t * p = (TrackedDevicePose_t *) addr; if (check_pose(*p)) { res = p; return *p; } } return notfound; } } return notfound; } ================================================ FILE: alvr/vulkan_layer/util/pose.hpp ================================================ #pragma once struct HmdMatrix34_t { float m[3][4]; }; struct HmdVector3_t { float v[3]; }; struct TrackedDevicePose_t { HmdMatrix34_t mDeviceToAbsoluteTracking; HmdVector3_t vVelocity; // velocity in tracker space in m/s HmdVector3_t vAngularVelocity; // angular velocity in radians/s (?) int eTrackingResult; char bPoseIsValid; // This indicates that there is a device connected for this spot in the pose array. // It could go from true to false if the user unplugs the device. char bDeviceIsConnected; }; const TrackedDevicePose_t & find_pose_in_call_stack(); ================================================ FILE: alvr/vulkan_layer/util/timed_semaphore.cpp ================================================ /* * Copyright (c) 2017, 2019 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include #include #include "timed_semaphore.hpp" namespace util { VkResult timed_semaphore::init(unsigned count) { int res; m_count = count; pthread_condattr_t attr; res = pthread_condattr_init(&attr); /* the only failure that can occur is ENOMEM */ assert(res == 0 || res == ENOMEM); if (res != 0) { return VK_ERROR_OUT_OF_HOST_MEMORY; } res = pthread_condattr_setclock(&attr, CLOCK_MONOTONIC); /* only programming error can cause _setclock to fail */ assert(res == 0); res = pthread_cond_init(&m_cond, &attr); /* the only failure that can occur that is not programming error is ENOMEM */ assert(res == 0 || res == ENOMEM); if (res != 0) { pthread_condattr_destroy(&attr); return VK_ERROR_OUT_OF_HOST_MEMORY; } res = pthread_condattr_destroy(&attr); /* only programming error can cause _destroy to fail */ assert(res == 0); res = pthread_mutex_init(&m_mutex, NULL); /* only programming errors can result in failure */ assert(res == 0); initialized = true; return VK_SUCCESS; } timed_semaphore::~timed_semaphore() { int res; (void)res; /* unused when NDEBUG */ if (initialized) { res = pthread_cond_destroy(&m_cond); assert(res == 0); /* only programming error (EBUSY, EINVAL) */ res = pthread_mutex_destroy(&m_mutex); assert(res == 0); /* only programming error (EBUSY, EINVAL) */ } } VkResult timed_semaphore::wait(uint64_t timeout) { VkResult retval = VK_SUCCESS; int res; assert(initialized); res = pthread_mutex_lock(&m_mutex); assert(res == 0); /* only fails with programming error (EINVAL) */ if (m_count == 0) { switch (timeout) { case 0: retval = VK_NOT_READY; break; case UINT64_MAX: res = pthread_cond_wait(&m_cond, &m_mutex); assert(res == 0); /* only fails with programming error (EINVAL) */ break; default: struct timespec diff = {/* narrowing casts */ static_cast(timeout / (1000 * 1000 * 1000)), static_cast(timeout % (1000 * 1000 * 1000))}; struct timespec now; res = clock_gettime(CLOCK_MONOTONIC, &now); assert(res == 0); /* only fails with programming error (EINVAL, EFAULT, EPERM) */ /* add diff to now, handling overflow */ struct timespec end = {now.tv_sec + diff.tv_sec, now.tv_nsec + diff.tv_nsec}; if (end.tv_nsec >= 1000 * 1000 * 1000) { end.tv_nsec -= 1000 * 1000 * 1000; end.tv_sec++; } res = pthread_cond_timedwait(&m_cond, &m_mutex, &end); /* only fails with programming error, other than timeout */ assert(res == 0 || res == ETIMEDOUT); if (res != 0) { retval = VK_TIMEOUT; } } } if (retval == VK_SUCCESS) { assert(m_count > 0); m_count--; } res = pthread_mutex_unlock(&m_mutex); assert(res == 0); /* only fails with programming error (EPERM) */ return retval; } void timed_semaphore::post() { int res; (void)res; /* unused when NDEBUG */ assert(initialized); res = pthread_mutex_lock(&m_mutex); assert(res == 0); /* only fails with programming error (EINVAL) */ m_count++; res = pthread_cond_signal(&m_cond); assert(res == 0); /* only fails with programming error (EINVAL) */ res = pthread_mutex_unlock(&m_mutex); assert(res == 0); /* only fails with programming error (EPERM) */ } } /* namespace util */ ================================================ FILE: alvr/vulkan_layer/util/timed_semaphore.hpp ================================================ /* * Copyright (c) 2017, 2019 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file timed_semaphore.hpp * * @brief Contains the class definition for a semaphore with a relative timedwait * * sem_timedwait takes an absolute time, based on CLOCK_REALTIME. Simply * taking the current time and adding on a relative timeout is not correct, * as the system time may change, resulting in an incorrect timeout period * (potentially by a significant amount). * * We therefore have to re-engineer semaphores using condition variables. * * This code does not use the C++ standard library to avoid exceptions. */ #pragma once extern "C" { #include } #include namespace util { /** * brief semaphore with a safe relative timed wait * * sem_timedwait takes an absolute time, based on CLOCK_REALTIME. Simply * taking the current time and adding on a relative timeout is not correct, * as the system time may change, resulting in an incorrect timeout period * (potentially by a significant amount). * * We therefore have to re-engineer semaphores using condition variables. * * This code does not use the C++ standard library to avoid exceptions. */ class timed_semaphore { public: /* copying not implemented */ timed_semaphore &operator=(const timed_semaphore &) = delete; timed_semaphore(const timed_semaphore &) = delete; ~timed_semaphore(); timed_semaphore() : initialized(false){}; /** * @brief initializes the semaphore * * @param count initial value of the semaphore * @retval VK_ERROR_OUT_OF_HOST_MEMORY out of memory condition from pthread calls * @retval VK_SUCCESS on success */ VkResult init(unsigned count); /** * @brief decrement semaphore, waiting (with timeout) if the value is 0 * * @param timeout time to wait (ns). 0 doesn't block, UINT64_MAX waits indefinately. * @retval VK_TIMEOUT timeout was non-zero and reached the timeout * @retval VK_NOT_READY timeout was zero and count is 0 * @retval VK_SUCCESS on success */ VkResult wait(uint64_t timeout); /** * @brief increment semaphore, potentially unblocking a waiting thread */ void post(); private: /** * @brief true if the semaphore has been initialized * * Determines if the destructor should cleanup the mutex and cond. */ bool initialized; /** * @brief semaphore value */ unsigned m_count; pthread_mutex_t m_mutex; pthread_cond_t m_cond; }; } /* namespace util */ ================================================ FILE: alvr/vulkan_layer/wsi/display.cpp ================================================ #include "display.hpp" #include"layer/settings.h" #include wsi::display::display() { } VkFence wsi::display::get_vsync_fence() { if (not std::atomic_exchange(&m_thread_running, true)) { vsync_fence = reinterpret_cast(this); m_vsync_thread = std::thread([this]() { auto refresh = Settings::Instance().m_refreshRate; auto next_frame = std::chrono::steady_clock::now(); auto frame_time = std::chrono::duration_cast(std::chrono::duration(1. / refresh)); while (not m_exiting) { std::this_thread::sleep_until(next_frame); m_signaled = true; m_cond.notify_all(); m_vsync_count += 1; next_frame += frame_time; } }); } m_signaled = false; return vsync_fence; } wsi::display::~display() { std::unique_lock lock(m_mutex); m_exiting = true; if (m_vsync_thread.joinable()) m_vsync_thread.join(); } bool wsi::display::wait_for_vsync(uint64_t timeoutNs) { if (!m_signaled) { std::unique_lock lock(m_mutex); return m_cond.wait_for(lock, std::chrono::nanoseconds(timeoutNs)) == std::cv_status::no_timeout; } return true; } ================================================ FILE: alvr/vulkan_layer/wsi/display.hpp ================================================ #pragma once #include #include #include #include namespace wsi { class display { public: display(); ~display(); VkFence get_vsync_fence(); VkFence peek_vsync_fence() { return vsync_fence;}; bool is_signaled() const { return m_signaled; } bool wait_for_vsync(uint64_t timeoutNs); std::atomic m_vsync_count{0}; private: std::atomic_bool m_thread_running{false}; std::atomic_bool m_exiting{false}; std::thread m_vsync_thread; VkFence vsync_fence = VK_NULL_HANDLE; std::mutex m_mutex; std::condition_variable m_cond; std::atomic_bool m_signaled = false; }; } // namespace wsi ================================================ FILE: alvr/vulkan_layer/wsi/headless/surface_properties.cpp ================================================ /* * Copyright (c) 2017-2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #include #include #include #include #include #include #include #include #include #include "layer/settings.h" #include "surface_properties.hpp" #define UNUSED(x) ((void)(x)) namespace wsi { namespace headless { surface_properties &surface_properties::get_instance() { static surface_properties instance; return instance; } VkResult surface_properties::get_surface_capabilities(VkPhysicalDevice physical_device, VkSurfaceKHR surface, VkSurfaceCapabilitiesKHR *surface_capabilities) { UNUSED(surface); /* Image count limits */ surface_capabilities->minImageCount = 1; /* There is no maximum theoretically speaking */ surface_capabilities->maxImageCount = UINT32_MAX; /* Surface extents */ surface_capabilities->currentExtent = surface_capabilities->maxImageExtent = surface_capabilities->minImageExtent = {Settings::Instance().m_renderWidth, Settings::Instance().m_renderHeight}; /* Ask the device for max */ VkPhysicalDeviceProperties dev_props; layer::instance_private_data::get(physical_device) .disp.GetPhysicalDeviceProperties(physical_device, &dev_props); surface_capabilities->maxImageArrayLayers = 1; /* Surface transforms */ surface_capabilities->supportedTransforms = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR; surface_capabilities->currentTransform = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR; /* Composite alpha */ surface_capabilities->supportedCompositeAlpha = static_cast( VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR | VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR | VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR | VK_COMPOSITE_ALPHA_POST_MULTIPLIED_BIT_KHR); /* Image usage flags */ surface_capabilities->supportedUsageFlags = VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT | VK_IMAGE_USAGE_INPUT_ATTACHMENT_BIT; return VK_SUCCESS; } VkResult surface_properties::get_surface_formats(VkPhysicalDevice physical_device, VkSurfaceKHR surface, uint32_t *surface_format_count, VkSurfaceFormatKHR *surface_formats) { UNUSED(surface); VkResult res = VK_SUCCESS; /* Construct a list of all formats supported by the driver - for color attachment */ VkFormat formats[] = { VK_FORMAT_R8_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R8G8_UNORM, VK_FORMAT_R16G16_UNORM, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_R8G8B8A8_UNORM}; uint32_t format_count = 0; for (size_t id = 0; id < std::size(formats); id++) { VkImageFormatProperties image_format_props; res = layer::instance_private_data::get(physical_device) .disp.GetPhysicalDeviceImageFormatProperties( physical_device, formats[id], VK_IMAGE_TYPE_2D, VK_IMAGE_TILING_OPTIMAL, VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT, VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT, &image_format_props); if (res != VK_ERROR_FORMAT_NOT_SUPPORTED) { formats[format_count] = formats[id]; format_count++; } } assert(format_count > 0); assert(surface_format_count != nullptr); res = VK_SUCCESS; if (nullptr == surface_formats) { *surface_format_count = format_count; } else { if (format_count > *surface_format_count) { res = VK_INCOMPLETE; } *surface_format_count = std::min(*surface_format_count, format_count); for (uint32_t i = 0; i < *surface_format_count; ++i) { surface_formats[i].format = formats[i]; surface_formats[i].colorSpace = VK_COLORSPACE_SRGB_NONLINEAR_KHR; } } return res; } VkResult surface_properties::get_surface_present_modes(VkPhysicalDevice physical_device, VkSurfaceKHR surface, uint32_t *present_mode_count, VkPresentModeKHR *present_modes) { UNUSED(physical_device); UNUSED(surface); VkResult res = VK_SUCCESS; static const std::array modes = {VK_PRESENT_MODE_FIFO_KHR, VK_PRESENT_MODE_FIFO_RELAXED_KHR}; assert(present_mode_count != nullptr); if (nullptr == present_modes) { *present_mode_count = modes.size(); } else { if (modes.size() > *present_mode_count) { res = VK_INCOMPLETE; } *present_mode_count = std::min(*present_mode_count, static_cast(modes.size())); for (uint32_t i = 0; i < *present_mode_count; ++i) { present_modes[i] = modes[i]; } } return res; } } /* namespace headless */ } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/headless/surface_properties.hpp ================================================ /* * Copyright (c) 2017-2019 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ #pragma once #include #include #include namespace wsi { namespace headless { class surface_properties : public wsi::surface_properties { public: VkResult get_surface_capabilities(VkPhysicalDevice physical_device, VkSurfaceKHR surface, VkSurfaceCapabilitiesKHR *pSurfaceCapabilities) override; VkResult get_surface_formats(VkPhysicalDevice physical_device, VkSurfaceKHR surface, uint32_t *surfaceFormatCount, VkSurfaceFormatKHR *surfaceFormats) override; VkResult get_surface_present_modes(VkPhysicalDevice physical_device, VkSurfaceKHR surface, uint32_t *pPresentModeCount, VkPresentModeKHR *pPresentModes) override; static surface_properties &get_instance(); }; } /* namespace headless */ } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/headless/swapchain.cpp ================================================ /* * Copyright (c) 2017-2020 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file swapchain.cpp * * @brief Contains the implementation for a headless swapchain. */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include "util/logger.h" #include "platform/linux/protocol.h" #include "swapchain.hpp" #include "wsi/display.hpp" namespace wsi { namespace headless { struct image_data { /* Device memory backing the image. */ VkDeviceMemory memory; }; swapchain::swapchain(layer::device_private_data &dev_data, const VkAllocationCallbacks *pAllocator) : wsi::swapchain_base(dev_data, pAllocator), m_display(*dev_data.display) {} swapchain::~swapchain() { /* Call the base's teardown */ close(m_socket); teardown(); } VkResult swapchain::create_image(const VkImageCreateInfo &image_create, wsi::swapchain_image &image) { VkResult res = VK_SUCCESS; m_create_info = image_create; m_create_info.usage |= VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT; res = m_device_data.disp.CreateImage(m_device, &m_create_info, nullptr, &image.image); if (res != VK_SUCCESS) { return res; } m_create_info.pNext = nullptr; m_create_info.pQueueFamilyIndices = nullptr; VkMemoryRequirements memory_requirements; m_device_data.disp.GetImageMemoryRequirements(m_device, image.image, &memory_requirements); /* Find a memory type */ size_t mem_type_idx = 0; VkMemoryPropertyFlags memFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT; VkPhysicalDeviceMemoryProperties prop; m_device_data.instance_data.disp.GetPhysicalDeviceMemoryProperties(m_device_data.physical_device, &prop); for (; mem_type_idx < prop.memoryTypeCount; ++mem_type_idx) { if ((prop.memoryTypes[mem_type_idx].propertyFlags & memFlags) == memFlags && memory_requirements.memoryTypeBits & (1 << mem_type_idx)) { break; } } assert(mem_type_idx < prop.memoryTypeCount); VkExportMemoryAllocateInfo export_info = {}; export_info.sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO; export_info.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; VkMemoryDedicatedAllocateInfo ded_info = {}; ded_info.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO; ded_info.image = image.image; ded_info.pNext = &export_info; VkMemoryAllocateInfo mem_info = {}; mem_info.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; mem_info.allocationSize = memory_requirements.size; mem_info.memoryTypeIndex = mem_type_idx; mem_info.pNext = &ded_info; m_mem_index = mem_type_idx; image_data *data = nullptr; /* Create image_data */ data = m_allocator.create(1); if (data == nullptr) { m_device_data.disp.DestroyImage(m_device, image.image, get_allocation_callbacks()); return VK_ERROR_OUT_OF_HOST_MEMORY; } image.data = reinterpret_cast(data); image.status = wsi::swapchain_image::FREE; res = m_device_data.disp.AllocateMemory(m_device, &mem_info, nullptr, &data->memory); assert(VK_SUCCESS == res); if (res != VK_SUCCESS) { destroy_image(image); return res; } res = m_device_data.disp.BindImageMemory(m_device, image.image, data->memory, 0); assert(VK_SUCCESS == res); if (res != VK_SUCCESS) { destroy_image(image); return res; } /* Initialize presentation fence. */ VkFenceCreateInfo fence_info = {VK_STRUCTURE_TYPE_FENCE_CREATE_INFO, nullptr, 0}; res = m_device_data.disp.CreateFence(m_device, &fence_info, nullptr, &image.present_fence); if (res != VK_SUCCESS) { destroy_image(image); return res; } // Export into a FD to send later VkMemoryGetFdInfoKHR fd_info = {}; fd_info.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR; fd_info.pNext = NULL; fd_info.memory = data->memory; fd_info.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; int fd; res = m_device_data.disp.GetMemoryFdKHR(m_device, &fd_info, &fd); if (res != VK_SUCCESS) { Error("GetMemoryFdKHR failed\n"); destroy_image(image); return res; } m_fds.push_back(fd); Debug("GetMemoryFdKHR returned fd=%d\n", fd); VkExportSemaphoreCreateInfo exp_info = {}; exp_info.sType = VK_STRUCTURE_TYPE_EXPORT_SEMAPHORE_CREATE_INFO; exp_info.handleTypes = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT; VkSemaphoreTypeCreateInfo tim_info = {}; tim_info.sType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO; tim_info.pNext = &exp_info; tim_info.semaphoreType = VK_SEMAPHORE_TYPE_TIMELINE; VkSemaphoreCreateInfo sem_info = {}; sem_info.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO; sem_info.pNext = &tim_info; res = m_device_data.disp.CreateSemaphore(m_device, &sem_info, nullptr, &image.semaphore); if (res != VK_SUCCESS) { Error("CreateSemaphore failed\n"); destroy_image(image); return res; } VkSemaphoreGetFdInfoKHR sem_fd_info = {}; sem_fd_info.sType = VK_STRUCTURE_TYPE_SEMAPHORE_GET_FD_INFO_KHR; sem_fd_info.semaphore = image.semaphore; sem_fd_info.handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT; res = m_device_data.disp.GetSemaphoreFdKHR(m_device, &sem_fd_info, &fd); if (res != VK_SUCCESS) { Error("GetSemaphoreFdKHR failed\n"); destroy_image(image); return res; } m_fds.push_back(fd); Debug("GetSemaphoreFdKHR returned fd=%d\n", fd); return res; } int swapchain::send_fds() { // This function does the arcane magic for sending // file descriptors over unix domain sockets // Stolen from https://gist.github.com/kokjo/75cec0f466fc34fa2922 // // There will always be 6 fds (for the 3 images and sempahores created in the swapchain) so we can avoid // dynamic length. Initially, I tried to send the length in the normal data field (msg.msg_iov / // data) but for some reason it was emptied on arrival, no matter what I did. // struct msghdr msg; struct iovec iov[1]; struct cmsghdr *cmsg = NULL; assert(m_fds.size() == 6); int fds[6]; char ctrl_buf[CMSG_SPACE(sizeof(fds))]; char data[1]; std::copy(m_fds.begin(), m_fds.end(), fds); memset(&msg, 0, sizeof(struct msghdr)); memset(ctrl_buf, 0, CMSG_SPACE(sizeof(fds))); iov[0].iov_base = data; iov[0].iov_len = sizeof(data); msg.msg_name = NULL; msg.msg_namelen = 0; msg.msg_iov = iov; msg.msg_iovlen = 1; msg.msg_controllen = CMSG_SPACE(sizeof(fds)); msg.msg_control = ctrl_buf; cmsg = CMSG_FIRSTHDR(&msg); cmsg->cmsg_level = SOL_SOCKET; cmsg->cmsg_type = SCM_RIGHTS; cmsg->cmsg_len = CMSG_LEN(sizeof(fds)); memcpy(CMSG_DATA(cmsg), fds, sizeof(fds)); int ret = sendmsg(m_socket, &msg, 0); for (auto fd: m_fds) close(fd); return ret; } bool swapchain::try_connect() { Debug("swapchain::try_connect\n"); m_socketPath = getenv("XDG_RUNTIME_DIR"); m_socketPath += "/alvr-ipc"; int ret; if (m_socket == -1) { m_socket = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); if (m_socket == -1) { perror("socket"); exit(1); } } struct sockaddr_un name; memset(&name, 0, sizeof(name)); name.sun_family = AF_UNIX; strncpy(name.sun_path, m_socketPath.c_str(), sizeof(name.sun_path) - 1); ret = connect(m_socket, (const struct sockaddr *)&name, sizeof(name)); if (ret == -1) { return false; // we will try again next frame } VkPhysicalDeviceVulkan11Properties props11 = {}; props11.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_1_PROPERTIES; VkPhysicalDeviceProperties2 props = {}; props.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2; props.pNext = &props11; m_device_data.instance_data.disp.GetPhysicalDeviceProperties2(m_device_data.physical_device, &props); init_packet init{.num_images = uint32_t(m_swapchain_images.size()), .device_uuid = {}, .image_create_info = m_create_info, .mem_index = m_mem_index, .source_pid = getpid()}; memcpy(init.device_uuid.data(), props11.deviceUUID, VK_UUID_SIZE); ret = write(m_socket, &init, sizeof(init)); if (ret == -1) { perror("write"); exit(1); } ret = send_fds(); if (ret == -1) { perror("sendmsg"); exit(1); } Debug("swapchain sent fds\n"); return true; } void swapchain::submit_image(uint32_t pending_index) { const auto & pose = m_swapchain_images[pending_index].pose.mDeviceToAbsoluteTracking.m; if (!m_connected) { m_connected = try_connect(); } if (m_connected) { int ret; present_packet packet; packet.image = pending_index; packet.frame = m_display.m_vsync_count; packet.semaphore_value = m_swapchain_images[pending_index].semaphore_value; memcpy(&packet.pose, pose, sizeof(packet.pose)); ret = write(m_socket, &packet, sizeof(packet)); if (ret == -1) { //FIXME: try to reconnect? } } } void swapchain::present_image(uint32_t pending_index) { if (in_flight_index != UINT32_MAX) unpresent_image(in_flight_index); in_flight_index = pending_index; } void swapchain::destroy_image(wsi::swapchain_image &image) { if (image.status != wsi::swapchain_image::INVALID) { if (image.present_fence != VK_NULL_HANDLE) { m_device_data.disp.DestroyFence(m_device, image.present_fence, nullptr); image.present_fence = VK_NULL_HANDLE; } if (image.image != VK_NULL_HANDLE) { m_device_data.disp.DestroyImage(m_device, image.image, get_allocation_callbacks()); image.image = VK_NULL_HANDLE; } } if (image.data != nullptr) { auto *data = reinterpret_cast(image.data); if (data->memory != VK_NULL_HANDLE) { m_device_data.disp.FreeMemory(m_device, data->memory, nullptr); data->memory = VK_NULL_HANDLE; } m_allocator.destroy(1, data); image.data = nullptr; } image.status = wsi::swapchain_image::INVALID; } } /* namespace headless */ } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/headless/swapchain.hpp ================================================ /* * Copyright (c) 2017-2019 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file swapchain.hpp * * @brief Contains the class definition for a headless swapchain. */ #pragma once #include #include #include #include #include "platform/linux/protocol.h" namespace wsi { namespace headless { /** * @brief Headless swapchain class. * * This class is mostly empty, because all the swapchain stuff is handled by the swapchain class, * which we inherit. This class only provides a way to create an image and page-flip ops. */ class swapchain : public wsi::swapchain_base { public: explicit swapchain(layer::device_private_data &dev_data, const VkAllocationCallbacks *pAllocator); ~swapchain(); protected: /** * @brief Platform specific init */ VkResult init_platform(VkDevice device, const VkSwapchainCreateInfoKHR *pSwapchainCreateInfo) { return VK_SUCCESS; }; /** * @brief Creates a new swapchain image. * * @param image_create_info Data to be used to create the image. * * @param image Handle to the image. * * @return If image creation is successful returns VK_SUCCESS, otherwise * will return VK_ERROR_OUT_OF_DEVICE_MEMORY or VK_ERROR_INITIALIZATION_FAILED * depending on the error that occured. */ VkResult create_image(const VkImageCreateInfo &image_create_info, wsi::swapchain_image &image); void submit_image(uint32_t pendingIndex); /** * @brief Method to perform a present - just calls unpresent_image on headless * * @param pendingIndex Index of the pending image to be presented. * */ void present_image(uint32_t pendingIndex); /** * @brief Method to release a swapchain image * * @param image Handle to the image about to be released. */ void destroy_image(wsi::swapchain_image &image); private: bool try_connect(); int send_fds(); int m_socket = -1; std::string m_socketPath; bool m_connected = false; std::vector m_fds; VkImageCreateInfo m_create_info; size_t m_mem_index; display &m_display; uint32_t in_flight_index = UINT32_MAX; }; } /* namespace headless */ } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/surface_properties.hpp ================================================ /* * Copyright (c) 2017-2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file surface_properties.hpp * * @brief Vulkan WSI surface query interfaces. */ #pragma once #include #include namespace wsi { /** * @brief The base surface property query interface. */ class surface_properties { public: /** * @brief Implementation of vkGetPhysicalDeviceSurfaceCapabilitiesKHR for the specific VkSurface * type. */ virtual VkResult get_surface_capabilities(VkPhysicalDevice physical_device, VkSurfaceKHR surface, VkSurfaceCapabilitiesKHR *surface_capabilities) = 0; /** * @brief Implementation of vkGetPhysicalDeviceSurfaceFormatsKHR for the specific VkSurface * type. */ virtual VkResult get_surface_formats(VkPhysicalDevice physical_device, VkSurfaceKHR surface, uint32_t *surface_format_count, VkSurfaceFormatKHR *surface_formats) = 0; /** * @brief Implementation of vkGetPhysicalDeviceSurfacePresentModesKHR for the specific VkSurface * type. */ virtual VkResult get_surface_present_modes(VkPhysicalDevice physical_device, VkSurfaceKHR surface, uint32_t *present_mode_count, VkPresentModeKHR *present_modes) = 0; /** * @brief Return the device extensions that this surface_properties implementation needs. */ virtual const util::extension_list &get_required_device_extensions() { static const util::extension_list empty{util::allocator::get_generic()}; return empty; } }; } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/swapchain_base.cpp ================================================ /* * Copyright (c) 2017-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file swapchain_base.cpp * * @brief Contains the implementation for the swapchain. * * This file contains much of the swapchain implementation, * that is not specific to how images are created or presented. */ #include #include #include #include #include #include #include #include "display.hpp" #include "swapchain_base.hpp" #if VULKAN_WSI_DEBUG > 0 #define WSI_PRINT_ERROR(...) fprintf(stderr, ##__VA_ARGS__) #else #define WSI_PRINT_ERROR(...) (void)0 #endif namespace wsi { void swapchain_base::page_flip_thread() { auto &sc_images = m_swapchain_images; VkResult vk_res = VK_SUCCESS; uint64_t timeout = UINT64_MAX; constexpr uint64_t SEMAPHORE_TIMEOUT = 250000000; /* 250 ms. */ /* No mutex is needed for the accesses to m_page_flip_thread_run variable as after the variable * is initialized it is only ever changed to false. The while loop will make the thread read the * value repeatedly, and the combination of semaphores and thread joins will force any changes * to the variable to be visible to this thread. */ while (m_page_flip_thread_run) { /* Waiting for the page_flip_semaphore which will be signalled once there is an * image to display.*/ if ((vk_res = m_page_flip_semaphore.wait(SEMAPHORE_TIMEOUT)) == VK_TIMEOUT) { /* Image is not ready yet. */ continue; } assert(vk_res == VK_SUCCESS); /* We want to present the oldest queued for present image from our present queue, * which we can find at the sc->pending_buffer_pool.head index. */ uint32_t pending_index = m_pending_buffer_pool.ring[m_pending_buffer_pool.head]; m_pending_buffer_pool.head = (m_pending_buffer_pool.head + 1) % m_pending_buffer_pool.size; submit_image(pending_index); /* We wait for the fence of the oldest pending image to be signalled. */ vk_res = m_device_data.disp.WaitForFences( m_device, 1, &sc_images[pending_index].present_fence, VK_TRUE, timeout); if (vk_res != VK_SUCCESS) { m_is_valid = false; m_free_image_semaphore.post(); continue; } /* If the descendant has started presenting the queue_present operation has marked the image * as FREE so we simply release it and continue. */ if (sc_images[pending_index].status == swapchain_image::FREE) { destroy_image(sc_images[pending_index]); m_free_image_semaphore.post(); continue; } /* First present of the swapchain. If it has an ancestor, wait until all the pending buffers * from the ancestor have finished page flipping before we set mode. */ if (m_first_present) { if (m_ancestor != VK_NULL_HANDLE) { auto *ancestor = reinterpret_cast(m_ancestor); ancestor->wait_for_pending_buffers(); } sem_post(&m_start_present_semaphore); present_image(pending_index); m_first_present = false; } /* The swapchain has already started presenting. */ else { present_image(pending_index); } } } void swapchain_base::unpresent_image(uint32_t presented_index) { m_swapchain_images[presented_index].status = swapchain_image::FREE; if (m_descendant != VK_NULL_HANDLE) { destroy_image(m_swapchain_images[presented_index]); } m_free_image_semaphore.post(); } swapchain_base::swapchain_base(layer::device_private_data &dev_data, const VkAllocationCallbacks *callbacks) : m_device_data(dev_data), m_page_flip_thread_run(true), m_thread_sem_defined(false), m_first_present(true), m_pending_buffer_pool{nullptr, 0, 0, 0}, m_allocator(callbacks, VK_SYSTEM_ALLOCATION_SCOPE_OBJECT), m_swapchain_images(m_allocator), m_surface(VK_NULL_HANDLE), m_present_mode(VK_PRESENT_MODE_IMMEDIATE_KHR), m_descendant(VK_NULL_HANDLE), m_ancestor(VK_NULL_HANDLE), m_device(VK_NULL_HANDLE), m_queue(VK_NULL_HANDLE) {} VkResult swapchain_base::init(VkDevice device, const VkSwapchainCreateInfoKHR *swapchain_create_info) { assert(device != VK_NULL_HANDLE); assert(swapchain_create_info != nullptr); assert(swapchain_create_info->surface != VK_NULL_HANDLE); int res; VkResult result; m_device = device; m_surface = swapchain_create_info->surface; /* Check presentMode has a compatible value with swapchain - everything else should be taken * care at image creation.*/ static const std::array present_modes = {VK_PRESENT_MODE_FIFO_KHR, VK_PRESENT_MODE_FIFO_RELAXED_KHR}; bool present_mode_found = false; for (uint32_t i = 0; i < present_modes.size() && !present_mode_found; i++) { if (swapchain_create_info->presentMode == present_modes[i]) { present_mode_found = true; } } if (!present_mode_found) { return VK_ERROR_INITIALIZATION_FAILED; } /* Init image to invalid values. */ if (!m_swapchain_images.try_resize(swapchain_create_info->minImageCount)) return VK_ERROR_OUT_OF_HOST_MEMORY; /* Initialize ring buffer. */ m_pending_buffer_pool.ring = m_allocator.create(m_swapchain_images.size(), 0); if (m_pending_buffer_pool.ring == nullptr) { return VK_ERROR_OUT_OF_HOST_MEMORY; } m_pending_buffer_pool.head = 0; m_pending_buffer_pool.tail = 0; m_pending_buffer_pool.size = m_swapchain_images.size(); /* We have allocated images, we can call the platform init function if something needs to be * done. */ result = init_platform(device, swapchain_create_info); if (result != VK_SUCCESS) { return result; } VkExternalMemoryImageCreateInfo ext_info = {}; ext_info.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO; ext_info.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT; VkImageCreateInfo image_create_info = {}; image_create_info.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO; image_create_info.pNext = &ext_info; image_create_info.imageType = VK_IMAGE_TYPE_2D; image_create_info.format = swapchain_create_info->imageFormat; image_create_info.extent = {swapchain_create_info->imageExtent.width, swapchain_create_info->imageExtent.height, 1}; image_create_info.mipLevels = 1; image_create_info.arrayLayers = swapchain_create_info->imageArrayLayers; image_create_info.samples = VK_SAMPLE_COUNT_1_BIT; image_create_info.tiling = VK_IMAGE_TILING_OPTIMAL; image_create_info.usage = swapchain_create_info->imageUsage; image_create_info.flags = VK_IMAGE_CREATE_ALIAS_BIT; image_create_info.sharingMode = swapchain_create_info->imageSharingMode; image_create_info.queueFamilyIndexCount = swapchain_create_info->queueFamilyIndexCount; image_create_info.pQueueFamilyIndices = swapchain_create_info->pQueueFamilyIndices; image_create_info.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; result = m_free_image_semaphore.init(m_swapchain_images.size()); if (result != VK_SUCCESS) { assert(result == VK_ERROR_OUT_OF_HOST_MEMORY); return result; } m_device_data.disp.GetDeviceQueue(m_device, 0, 0, &m_queue); result = m_device_data.SetDeviceLoaderData(m_device, m_queue); if (VK_SUCCESS != result) { return result; } for (auto &img : m_swapchain_images) { result = create_image(image_create_info, img); if (result != VK_SUCCESS) { return result; } } /* Setup semaphore for signaling pageflip thread */ result = m_page_flip_semaphore.init(0); if (result != VK_SUCCESS) { return result; } res = sem_init(&m_start_present_semaphore, 0, 0); /* Only programming error can cause this to fail. */ assert(res == 0); if (res != 0) { return VK_ERROR_OUT_OF_HOST_MEMORY; } m_thread_sem_defined = true; /* Launch page flipping thread */ m_page_flip_thread = std::thread(&swapchain_base::page_flip_thread, this); /* Release the swapchain images of the old swapchain in order * to free up memory for new swapchain. This is necessary especially * on platform with limited display memory size. * * NB: This must be done last in initialization, when the rest of * the swapchain is valid. */ if (swapchain_create_info->oldSwapchain != VK_NULL_HANDLE) { /* Set ancestor. */ m_ancestor = swapchain_create_info->oldSwapchain; auto *ancestor = reinterpret_cast(m_ancestor); ancestor->deprecate(reinterpret_cast(this)); } m_is_valid = true; return VK_SUCCESS; } void swapchain_base::teardown() { /* This method will block until all resources associated with this swapchain * are released. Images in the ACQUIRED or FREE state can be freed * immediately. For images in the PRESENTED state, we will block until the * presentation engine is finished with them. */ int res; bool descendent_started_presenting = false; if (m_descendant != VK_NULL_HANDLE) { auto *desc = reinterpret_cast(m_descendant); for (auto &img : desc->m_swapchain_images) { if (img.status == swapchain_image::PRESENTED || img.status == swapchain_image::PENDING) { /* Here we wait for the start_present_semaphore, once this semaphore is up, * the descendant has finished waiting, we don't want to delete vkImages and * vkFences and semaphores before the waiting is done. */ sem_wait(&desc->m_start_present_semaphore); descendent_started_presenting = true; break; } } } /* If descendant started presenting, there is no pending buffer in the swapchain. */ if (m_is_valid && descendent_started_presenting == false) { wait_for_pending_buffers(); } if (m_queue != VK_NULL_HANDLE) { /* Make sure the vkFences are done signaling. */ m_device_data.disp.QueueWaitIdle(m_queue); } /* We are safe to destroy everything. */ if (m_thread_sem_defined) { /* Tell flip thread to end. */ m_page_flip_thread_run = false; if (m_page_flip_thread.joinable()) { m_page_flip_thread.join(); } else { WSI_PRINT_ERROR("m_page_flip_thread is not joinable"); } res = sem_destroy(&m_start_present_semaphore); if (res != 0) { WSI_PRINT_ERROR("sem_destroy failed for start_present_semaphore with %d\n", errno); } } if (m_descendant != VK_NULL_HANDLE) { auto *sc = reinterpret_cast(m_descendant); sc->clear_ancestor(); } if (m_ancestor != VK_NULL_HANDLE) { auto *sc = reinterpret_cast(m_ancestor); sc->clear_descendant(); } /* Release the images array. */ for (auto &img : m_swapchain_images) { /* Call implementation specific release */ destroy_image(img); } m_allocator.destroy(m_swapchain_images.size(), m_pending_buffer_pool.ring); } VkResult swapchain_base::acquire_next_image(uint64_t timeout, VkSemaphore semaphore, VkFence fence, uint32_t *image_index) { VkResult retval = wait_for_free_buffer(timeout); if (retval != VK_SUCCESS) { return retval; } if (!m_is_valid) { return VK_ERROR_OUT_OF_HOST_MEMORY; } uint32_t i = m_last_acquired_image + 1; for (uint32_t j = 0; j < m_swapchain_images.size(); ++j) { i = (i + 1) % m_pending_buffer_pool.size; if (m_swapchain_images[i].status == swapchain_image::FREE) { m_swapchain_images[i].status = swapchain_image::ACQUIRED; *image_index = i; m_last_acquired_image = i; break; } } assert(i < m_swapchain_images.size()); if (VK_NULL_HANDLE != semaphore || VK_NULL_HANDLE != fence) { VkSubmitInfo submit = {}; submit.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; if (VK_NULL_HANDLE != semaphore) { submit.signalSemaphoreCount = 1; submit.pSignalSemaphores = &semaphore; } submit.commandBufferCount = 0; submit.pCommandBuffers = nullptr; retval = m_device_data.disp.QueueSubmit(m_queue, 1, &submit, fence); assert(retval == VK_SUCCESS); } return retval; } VkResult swapchain_base::get_swapchain_images(uint32_t *swapchain_image_count, VkImage *swapchain_images) { if (swapchain_images == nullptr) { /* Return the number of swapchain images. */ *swapchain_image_count = m_swapchain_images.size(); return VK_SUCCESS; } else { assert(m_swapchain_images.size() > 0); assert(*swapchain_image_count > 0); /* Populate array, write actual number of images returned. */ uint32_t current_image = 0; do { swapchain_images[current_image] = m_swapchain_images[current_image].image; current_image++; if (current_image == m_swapchain_images.size()) { *swapchain_image_count = current_image; return VK_SUCCESS; } } while (current_image < *swapchain_image_count); /* If swapchain_image_count is smaller than the number of presentable images * in the swapchain, VK_INCOMPLETE must be returned instead of VK_SUCCESS. */ *swapchain_image_count = current_image; return VK_INCOMPLETE; } } VkResult swapchain_base::queue_present(VkQueue queue, const VkPresentInfoKHR *present_info, const uint32_t image_index) { VkResult result; bool descendent_started_presenting = false; const auto & pose = find_pose_in_call_stack(); if (m_descendant != VK_NULL_HANDLE) { auto *desc = reinterpret_cast(m_descendant); for (auto &img : desc->m_swapchain_images) { if (img.status == swapchain_image::PRESENTED || img.status == swapchain_image::PENDING) { descendent_started_presenting = true; break; } } } /* When the semaphore that comes in is signalled, we know that all work is done. So, we do not * want to block any future Vulkan queue work on it. So, we pass in BOTTOM_OF_PIPE bit as the * wait flag. */ VkPipelineStageFlags pipeline_stage_flags = VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT; uint64_t signal_value = ++m_swapchain_images[image_index].semaphore_value; VkTimelineSemaphoreSubmitInfo timeline_info = {}; timeline_info.sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO; timeline_info.signalSemaphoreValueCount = 1; timeline_info.pSignalSemaphoreValues = &signal_value; VkSubmitInfo submit_info = {VK_STRUCTURE_TYPE_SUBMIT_INFO, &timeline_info, present_info->waitSemaphoreCount, present_info->pWaitSemaphores, &pipeline_stage_flags, 0, NULL, 1, &m_swapchain_images[image_index].semaphore}; assert(m_swapchain_images[image_index].status == swapchain_image::ACQUIRED); result = m_device_data.disp.ResetFences(m_device, 1, &m_swapchain_images[image_index].present_fence); if (result != VK_SUCCESS) { return result; } result = m_device_data.disp.QueueSubmit(queue, 1, &submit_info, m_swapchain_images[image_index].present_fence); if (result != VK_SUCCESS) { return result; } /* If the descendant has started presenting, we should release the image * however we do not want to block inside the main thread so we mark it * as free and let the page flip thread take care of it. */ if (descendent_started_presenting) { m_swapchain_images[image_index].status = swapchain_image::FREE; m_pending_buffer_pool.ring[m_pending_buffer_pool.tail] = image_index; m_pending_buffer_pool.tail = (m_pending_buffer_pool.tail + 1) % m_pending_buffer_pool.size; m_page_flip_semaphore.post(); return VK_ERROR_OUT_OF_DATE_KHR; } m_swapchain_images[image_index].status = swapchain_image::PENDING; m_swapchain_images[image_index].pose = pose; m_pending_buffer_pool.ring[m_pending_buffer_pool.tail] = image_index; m_pending_buffer_pool.tail = (m_pending_buffer_pool.tail + 1) % m_pending_buffer_pool.size; m_page_flip_semaphore.post(); return VK_SUCCESS; } void swapchain_base::deprecate(VkSwapchainKHR descendant) { for (auto &img : m_swapchain_images) { if (img.status == swapchain_image::FREE) { destroy_image(img); } } /* Set its descendant. */ m_descendant = descendant; } void swapchain_base::wait_for_pending_buffers() { int num_acquired_images = 0; int wait; for (auto &img : m_swapchain_images) { if (img.status == swapchain_image::ACQUIRED) { ++num_acquired_images; } } /* Once all the pending buffers are flipped, the swapchain should have images * in ACQUIRED (application fails to queue them back for presentation), FREE * and one and only one in PRESENTED. */ wait = m_swapchain_images.size() - num_acquired_images - 1; while (wait > 0) { /* Take down one free image semaphore. */ wait_for_free_buffer(UINT64_MAX); --wait; } } void swapchain_base::clear_ancestor() { m_ancestor = VK_NULL_HANDLE; } void swapchain_base::clear_descendant() { m_descendant = VK_NULL_HANDLE; } VkResult swapchain_base::wait_for_free_buffer(uint64_t timeout) { VkResult retval; /* first see if a buffer is already marked as free */ retval = m_free_image_semaphore.wait(0); if (retval == VK_NOT_READY) { /* if not, we still have work to do even if timeout==0 - * the swapchain implementation may be able to get a buffer without * waiting */ retval = get_free_buffer(&timeout); if (retval == VK_SUCCESS) { /* the sub-implementation has done it's thing, so re-check the * semaphore */ retval = m_free_image_semaphore.wait(timeout); } } return retval; } #undef WSI_PRINT_ERROR } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/swapchain_base.hpp ================================================ /* * Copyright (c) 2017-2020 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file swapchain_base.hpp * * @brief Contains the class definition for a base swapchain. */ #pragma once #include #include #include #include #include #include #include #include namespace wsi { struct swapchain_image { enum status { INVALID, ACQUIRED, PENDING, PRESENTED, FREE, }; /* Implementation specific data */ void *data{nullptr}; VkImage image{VK_NULL_HANDLE}; status status{swapchain_image::INVALID}; VkFence present_fence{VK_NULL_HANDLE}; VkSemaphore semaphore{VK_NULL_HANDLE}; uint64_t semaphore_value = 0; TrackedDevicePose_t pose; }; /** * @brief Base swapchain class * * - the swapchain implementations inherit from this class. * - the VkSwapchain will hold a pointer to this class. * - much of the swapchain implementation is done by this class, as the only things needed * in the implementation are how to create a presentable image and how to present an image. */ class swapchain_base { public: swapchain_base(layer::device_private_data &dev_data, const VkAllocationCallbacks *allocator); virtual ~swapchain_base() { /* nop */ } /** * @brief Create swapchain. * * Perform all swapchain initialization, create presentable images etc. */ VkResult init(VkDevice device, const VkSwapchainCreateInfoKHR *swapchain_create_info); /** * @brief Acquires a free image. * * Current implementation blocks until a free image is available. * * @param timeout Unused since we block until a free image is available. * * @param semaphore A semaphore signaled once an image is acquired. * * @param fence A fence signaled once an image is acquired. * * @param pImageIndex The index of the acquired image. * * @return VK_SUCCESS on completion. */ VkResult acquire_next_image(uint64_t timeout, VkSemaphore semaphore, VkFence fence, uint32_t *image_index); /** * @brief Gets the number of swapchain images or a number of at most * m_num_swapchain_images images. * * @param pSwapchainImageCount Used to return number of images in * the swapchain if second parameter is nullptr or represents the * number of images to be returned in second parameter. * * @param pSwapchainImage Array of VkImage handles. * * @return If number of requested images is less than the number of available * images in the swapchain returns VK_INCOMPLETE otherwise VK_SUCCESS. */ VkResult get_swapchain_images(uint32_t *swapchain_image_count, VkImage *swapchain_image); /** * @brief Submits a present request for the supplied image. * * @param queue The queue to which the submission will be made to. * * @param pPresentInfo Information about the swapchain and image to be presented. * * @param imageIndex The index of the image to be presented. * * @return If queue submission fails returns error of vkQueueSubmit, if the * swapchain has a descendant who started presenting returns VK_ERROR_OUT_OF_DATE_KHR, * otherwise returns VK_SUCCESS. */ VkResult queue_present(VkQueue queue, const VkPresentInfoKHR *present_info, const uint32_t image_index); protected: layer::device_private_data &m_device_data; /** * @brief Handle to the page flip thread. */ std::thread m_page_flip_thread; /** * @brief Whether the page flip thread has to continue running or terminate. */ bool m_page_flip_thread_run; /** * @brief In case we encounter threading or drm errors we need a way to * notify the user of the failure. When this flag is false, acquire_next_image * will return an error code. */ bool m_is_valid; struct ring_buffer { /* Ring buffer to hold the image indexes. */ uint32_t *ring; /* Head of the ring. */ uint32_t head; /* End of the ring. */ uint32_t tail; /* Size of the ring. */ uint32_t size; }; /** * @brief A semaphore to be signalled once a page flip event occurs. */ util::timed_semaphore m_page_flip_semaphore; /** * @brief A semaphore to be signalled once the swapchain has one frame on screen. */ sem_t m_start_present_semaphore; /** * @brief Defines if the pthread_t and sem_t members of the class are defined. * * As they are opaque types theer's no known invalid value that we ca initialize to, * and therefore determine if we need to cleanup. */ bool m_thread_sem_defined; /** * @brief A flag to track if it is the first present for the chain. */ bool m_first_present; /** * @brief In order to present the images in a FIFO order we implement * a ring buffer to hold the images queued for presentation. Since the * two pointers (head and tail) are used by different * threads and we do not allow the application to acquire more images * than we have we eliminate race conditions. */ ring_buffer m_pending_buffer_pool; /** * @brief User provided memory allocation callbacks. */ const util::allocator m_allocator; /** * @brief Vector of images in the swapchain. */ util::vector m_swapchain_images; /** * @brief Handle to the surface object this swapchain will present images to. */ VkSurfaceKHR m_surface; /** * @brief present mode to use for this swapchain */ VkPresentModeKHR m_present_mode; /** * @brief Descendant of this swapchain. * Used to check whether or not a descendant of this swapchain has started * presenting images to the surface already. If it has, any calls to queuePresent * for this swapchain will return VK_ERROR_OUT_OF_DATE_KHR. */ VkSwapchainKHR m_descendant; /** * @brief Ancestor of this swapchain. * Used to check whether the ancestor swapchain has completed all of its * pending page flips (this is required before this swapchain presents for the * first time. */ VkSwapchainKHR m_ancestor; /** * @brief Handle to the logical device the swapchain is created for. */ VkDevice m_device; /** * @brief Handle to the queue used for signalling submissions */ VkQueue m_queue; /** * @brief Return the VkAllocationCallbacks passed in this object constructor. */ const VkAllocationCallbacks *get_allocation_callbacks() { return m_allocator.get_original_callbacks(); } /** * @brief Method to wait on all pending buffers to be displayed. */ void wait_for_pending_buffers(); /** * @brief Remove cached ancestor. */ void clear_ancestor(); /** * @brief Remove cached descendant. */ void clear_descendant(); /** * @brief Deprecate this swapchain. * * If an application replaces an old swapchain with a new one, the older swapchain * needs to be deprecated. This method releases all the FREE images and sets the * descendant of the swapchain. We do not need to care about images in other states * at this point since they will be released by the page flip thread. * * @param descendant Handle to the descendant swapchain. */ void deprecate(VkSwapchainKHR descendant); /** * @brief Platform specific initialization */ virtual VkResult init_platform(VkDevice device, const VkSwapchainCreateInfoKHR *swapchain_create_info) = 0; /** * @brief Base swapchain teardown. * * Even though the inheritance gives us a nice way to defer display specific allocation * and presentation outside of the base class, it however robs the children classes - which * also happen to do some of their state setting - the oppurtunity to do the last clean up * call, as the base class' destructor is called at the end. This method provides a way to do * it. The destructor is a virtual function and much of the swapchain teardown happens in this * method which gets called from the child's destructor. */ void teardown(); /** * @brief Creates a new swapchain image. * * @param image_create_info Data to be used to create the image. * * @param image Handle to the image. * * @return If image creation is successful returns VK_SUCCESS, otherwise * will return VK_ERROR_OUT_OF_DEVICE_MEMORY or VK_ERROR_INITIALIZATION_FAILED * depending on the error that occured. */ virtual VkResult create_image(const VkImageCreateInfo &image_create_info, swapchain_image &image) = 0; virtual void submit_image(uint32_t pending_index) = 0; /** * @brief Method to present and image * * @param pending_index Index of the pending image to be presented. * */ virtual void present_image(uint32_t pending_index) = 0; /** * @brief Transition a presented image to free. * * Called by swapchain implementation when a new image has been presented. * * @param presented_index Index of the image to be marked as free. */ void unpresent_image(uint32_t presented_index); /** * @brief Method to release a swapchain image * * @param image Handle to the image about to be released. */ virtual void destroy_image(swapchain_image &image){}; /** * @brief Hook for any actions to free up a buffer for acquire * * @param[in,out] timeout time to wait, in nanoseconds. 0 doesn't block, * UINT64_MAX waits indefinately. The timeout should * be updated if a sleep is required - this can * be set to 0 if the semaphore is now not expected * block. */ virtual VkResult get_free_buffer(uint64_t *timeout) { return VK_SUCCESS; } private: /** * @brief Wait for a buffer to become free. */ VkResult wait_for_free_buffer(uint64_t timeout); /** * @brief A semaphore to be signalled once a free image becomes available. * * Uses a custom semaphore implementation that uses a condition variable. * it is slower, but has a safe timedwait implementation. * * This is kept private as waiting should be done via wait_for_free_buffer(). */ util::timed_semaphore m_free_image_semaphore; /** * @brief Per swapchain thread function that handles page flipping. * * This thread should be running for the lifetime of the swapchain. * The thread simply calls the implementation's present_image() method. * There are 3 main cases we cover here: * * 1. On the first present of the swapchain if the swapchain has * an ancestor we must wait for it to finish presenting. * 2. The normal use case where we do page flipping, in this * case change the currently PRESENTED image with the oldest * PENDING image. * 3. If the enqueued image is marked as FREE it means the * descendant of the swapchain has started presenting so we * should release the image and continue. * * The function always waits on the page_flip_semaphore of the * swapchain. Once it passes that we must wait for the fence of the * oldest pending image to be signalled, this means that the gpu has * finished rendering to it and we can present it. From there on the * logic splits into the above 3 cases and if an image has been * presented then the old one is marked as FREE and the free_image * semaphore of the swapchain will be posted. **/ void page_flip_thread(); uint32_t m_last_acquired_image = 0; std::vector m_fences; }; } /* namespace wsi */ ================================================ FILE: alvr/vulkan_layer/wsi/wsi_factory.cpp ================================================ /* * Copyright (c) 2019-2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file * @brief Implements factory methods for obtaining the specific surface and swapchain * implementations. */ #include "wsi_factory.hpp" #include "headless/surface_properties.hpp" #include "headless/swapchain.hpp" #include #include #include #include #include #include namespace wsi { static struct wsi_extension { VkExtensionProperties extension; VkIcdWsiPlatform platform; } const supported_wsi_extensions[] = { {{VK_KHR_DISPLAY_EXTENSION_NAME, VK_KHR_DISPLAY_SPEC_VERSION}, VK_ICD_WSI_PLATFORM_HEADLESS}}; static surface_properties *get_surface_properties(VkIcdWsiPlatform platform) { switch (platform) { case VK_ICD_WSI_PLATFORM_HEADLESS: return &headless::surface_properties::get_instance(); default: return nullptr; } } surface_properties *get_surface_properties(VkSurfaceKHR) { return get_surface_properties(VK_ICD_WSI_PLATFORM_HEADLESS); } template static swapchain_base *allocate_swapchain(layer::device_private_data &dev_data, const VkAllocationCallbacks *pAllocator) { if (!pAllocator) { return new swapchain_type(dev_data, pAllocator); } void *memory = pAllocator->pfnAllocation(pAllocator->pUserData, sizeof(swapchain_type), alignof(swapchain_type), VK_SYSTEM_ALLOCATION_SCOPE_INSTANCE); return new (memory) swapchain_type(dev_data, pAllocator); } swapchain_base *allocate_surface_swapchain(VkSurfaceKHR, layer::device_private_data &dev_data, const VkAllocationCallbacks *pAllocator) { return allocate_swapchain(dev_data, pAllocator); } util::wsi_platform_set find_enabled_layer_platforms(const VkInstanceCreateInfo *pCreateInfo) { util::wsi_platform_set ret; for (const auto &ext_provided_by_layer : supported_wsi_extensions) { for (uint32_t i = 0; i < pCreateInfo->enabledExtensionCount; i++) { const char *ext_requested_by_user = pCreateInfo->ppEnabledExtensionNames[i]; if (strcmp(ext_requested_by_user, ext_provided_by_layer.extension.extensionName) == 0) { ret.add(ext_provided_by_layer.platform); } } } return ret; } VkResult add_extensions_required_by_layer(VkPhysicalDevice phys_dev, const util::wsi_platform_set enabled_platforms, util::extension_list &extensions_to_enable) { util::allocator allocator{extensions_to_enable.get_allocator(), VK_SYSTEM_ALLOCATION_SCOPE_COMMAND}; util::extension_list device_extensions{allocator}; VkResult res = device_extensions.add(phys_dev); if (res != VK_SUCCESS) { return res; } for (const auto &wsi_ext : supported_wsi_extensions) { /* Skip iterating over platforms not enabled in the instance. */ if (!enabled_platforms.contains(wsi_ext.platform)) { continue; } surface_properties *props = get_surface_properties(wsi_ext.platform); const auto &extensions_required_by_layer = props->get_required_device_extensions(); bool supported = device_extensions.contains(extensions_required_by_layer); if (!supported) { /* Can we accept failure? The layer unconditionally advertises support for this platform * and the loader uses this information to enable its own support of the * vkCreate*SurfaceKHR entrypoints. The rest of the Vulkan stack may not support this * extension so we cannot blindly fall back to it. For now treat this as an error. */ return VK_ERROR_INITIALIZATION_FAILED; } res = extensions_to_enable.add(extensions_required_by_layer); if (res != VK_SUCCESS) { return res; } } return VK_SUCCESS; } void destroy_surface_swapchain(swapchain_base *swapchain, const VkAllocationCallbacks *pAllocator) { assert(swapchain); if (!pAllocator) { delete swapchain; } else { swapchain->~swapchain_base(); pAllocator->pfnFree(pAllocator->pUserData, reinterpret_cast(swapchain)); } } } // namespace wsi ================================================ FILE: alvr/vulkan_layer/wsi/wsi_factory.hpp ================================================ /* * Copyright (c) 2019, 2021 Arm Limited. * * SPDX-License-Identifier: MIT * * Permission is hereby granted, free of charge, to any person obtaining a copy * of this software and associated documentation files (the "Software"), to * deal in the Software without restriction, including without limitation the * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or * sell copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in all * copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ /** * @file * @brief Contains the factory methods for obtaining the specific surface and swapchain * implementations. */ #pragma once #include "surface_properties.hpp" #include "swapchain_base.hpp" #include "util/platform_set.hpp" #include namespace wsi { /** * @brief Obtains the surface properties for the specific surface type. * * @param surface The surface for which to get the properties. * * @return nullptr if surface type is unsupported. */ surface_properties *get_surface_properties(VkSurfaceKHR surface); /** * @brief Allocates a surface specific swapchain. * * @param surface The surface for which a swapchain is allocated. * @param dev_data The device specific data. * @param pAllocator The allocator from which to allocate any memory. * * @return nullptr on failure. */ swapchain_base *allocate_surface_swapchain(VkSurfaceKHR surface, layer::device_private_data &dev_data, const VkAllocationCallbacks *pAllocator); /** * @brief Destroys a swapchain and frees memory. Used with @ref allocate_surface_swapchain. * * @param swapchain Pointer to the swapchain to destroy. * @param pAllocator The allocator to use for freeing memory. */ void destroy_surface_swapchain(swapchain_base *swapchain, const VkAllocationCallbacks *pAllocator); /** * @brief Return which platforms the layer can handle for an instance constructed in the specified * way. * * @details This function looks at the extensions specified in @p pCreateInfo and based on this * returns a list of platforms that the layer can support. For example, if the @c * pCreateInfo.ppEnabledExtensionNames contains the string "VK_EXT_headless_surface" then the * returned platform set will contain @c VK_ICD_WSI_PLATFORM_HEADLESS. * * @param pCreateInfo Structure used when creating the instance in vkCreateInstance(). * * @return A list of WS platforms supported by the layer. */ util::wsi_platform_set find_enabled_layer_platforms(const VkInstanceCreateInfo *pCreateInfo); /** * @brief Add extra extensions that the layer requires to support the specified list of enabled * platforms. * * @details Check whether @p phys_dev has support for the extensions required by the layer in order * to support the platforms it implements. The extensions that the layer requires to operate are * added to @p extensions_to_enable. * * @param[in] phys_dev The physical device to check. * @param[in] enabled_platforms All the platforms that the layer must enable for @p phys_dev. * @param[in,out] extensions_to_enable All the extensions required by the layer are added to this * list. * * @retval @c VK_SUCCESS if the operation was successful. */ VkResult add_extensions_required_by_layer(VkPhysicalDevice phys_dev, const util::wsi_platform_set enabled_platforms, util::extension_list &extensions_to_enable); } // namespace wsi ================================================ FILE: alvr/xtask/Cargo.toml ================================================ [package] name = "alvr_xtask" version.workspace = true edition.workspace = true rust-version.workspace = true authors.workspace = true license.workspace = true [dependencies] alvr_filesystem.workspace = true serde = { version = "1.0", features = ["derive"] } serde_json = "1" pico-args = "0.5" xshell = "0.2" walkdir = "2" [target.'cfg(target_os = "linux")'.dependencies] pkg-config = "0.3" ================================================ FILE: alvr/xtask/LICENSE ================================================ Copyright (c) 2020-2024 alvr-org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: alvr/xtask/README.md ================================================ # alvr_xtask Custom tailored build utilities. Inspired by [cargo-xtask](https://github.com/matklad/cargo-xtask). ================================================ FILE: alvr/xtask/build.rs ================================================ fn main() {} ================================================ FILE: alvr/xtask/firewall/alvr-firewalld.xml ================================================ Stream VR games from your PC to your headset via Wi-Fi ALVR is an open source remote VR display which allows playing SteamVR games on a standalone headset such as Gear VR or Oculus Go/Quest. ================================================ FILE: alvr/xtask/firewall/alvr_fw_config.sh ================================================ #!/usr/bin/env bash # Basic script to add / remove firewall configuration for ALVR # Usage: ./alvr_fw_config.sh add|remove # Exit codes: # 1 - Invalid command # 2 - Invalid action # 3 - Failed to copy UFW configuration # 99 - Firewall not found # 126 - pkexec failed - Request dismissed firewalld_cfg() { # Iterate around each active zone for zone in $(firewall-cmd --get-active-zones | grep -P --only-matching '^[\w/-]+'); do if [ "${1}" == 'add' ]; then # If running or permanent alvr service is missing, add it if ! firewall-cmd --zone="${zone}" --list-services | grep 'alvr' >/dev/null 2>&1; then firewall-cmd --zone="${zone}" --add-service='alvr' fi if ! firewall-cmd --zone="${zone}" --list-services --permanent | grep 'alvr' >/dev/null 2>&1; then firewall-cmd --zone="${zone}" --add-service='alvr' --permanent fi elif [ "${1}" == 'remove' ]; then # If running or persistent alvr service exists, remove it if firewall-cmd --zone="${zone}" --list-services | grep 'alvr' >/dev/null 2>&1; then firewall-cmd --zone="${zone}" --remove-service='alvr' fi if firewall-cmd --zone="${zone}" --list-services --permanent | grep 'alvr' >/dev/null 2>&1; then firewall-cmd --zone="${zone}" --remove-service='alvr' --permanent fi else exit 2 fi done } ufw_cfg() { # Try and install the application file if ! ufw app info 'alvr'; then # Pull application file from local build first if the script lives inside it if [ -f "$(dirname "$(realpath "${0}")")/ufw-alvr" ]; then cp "$(dirname "$(realpath "${0}")")/ufw-alvr" '/etc/ufw/applications.d/' elif [ -f '/usr/share/alvr/ufw-alvr' ]; then cp '/usr/share/alvr/ufw-alvr' '/etc/ufw/applications.d/' else exit 3 fi fi if [ "${1}" == 'add' ] && ! ufw status | grep 'alvr' >/dev/null 2>&1; then ufw allow 'alvr' elif [ "${1}" == 'remove' ] && ufw status | grep 'alvr' >/dev/null 2>&1; then ufw delete allow 'alvr' else exit 2 fi } iptables_cfg() { first_port_match_count=$(iptables -S | grep -c '9943') second_port_match_count=$(iptables -S | grep -c '9944') if [ "${1}" == 'add' ]; then if [ "$first_port_match_count" == "0" ] || [ "$second_port_match_count" == "0" ]; then if [ ! -d '/etc/iptables' ]; then mkdir '/etc/iptables' fi iptables -I OUTPUT -p tcp --sport 9943 -j ACCEPT iptables -I INPUT -p tcp --dport 9943 -j ACCEPT iptables -I OUTPUT -p udp --sport 9943 -j ACCEPT iptables -I INPUT -p udp --dport 9943 -j ACCEPT iptables -I OUTPUT -p tcp --sport 9944 -j ACCEPT iptables -I INPUT -p tcp --dport 9944 -j ACCEPT iptables -I OUTPUT -p udp --sport 9944 -j ACCEPT iptables -I INPUT -p udp --dport 9944 -j ACCEPT iptables-save >/etc/iptables/rules.v4 fi elif [ "${1}" == 'remove' ]; then if [ "$first_port_match_count" == "4" ] || [ "$second_port_match_count" == "4" ]; then iptables -D OUTPUT -p tcp --sport 9943 -j ACCEPT iptables -D INPUT -p tcp --dport 9943 -j ACCEPT iptables -D OUTPUT -p udp --sport 9943 -j ACCEPT iptables -D INPUT -p udp --dport 9943 -j ACCEPT iptables -D OUTPUT -p tcp --sport 9944 -j ACCEPT iptables -D INPUT -p tcp --dport 9944 -j ACCEPT iptables -D OUTPUT -p udp --sport 9944 -j ACCEPT iptables -D INPUT -p udp --dport 9944 -j ACCEPT iptables-save >/etc/iptables/rules.v4 fi else exit 2 fi } main() { # If we're not root use pkexec for GUI prompt if [ "${USER}" == 'root' ]; then # Check if firewall-cmd exists and firewalld is running if which firewall-cmd >/dev/null 2>&1 && firewall-cmd --state >/dev/null 2>&1; then firewalld_cfg "${1,,}" # Check if ufw exists and is running elif which ufw >/dev/null 2>&1 && ! ufw status | grep 'Status: inactive' >/dev/null 2>&1; then ufw_cfg "${1,,}" elif which iptables >/dev/null 2>&1; then iptables_cfg "${1,,}" else exit 99 fi else pkexec "$(realpath "${0}")" "${@}" fi } main "${@}" ================================================ FILE: alvr/xtask/firewall/ufw-alvr ================================================ [alvr] title=ALVR description=Stream VR games from your PC to your headset via Wi-Fi ports=9943:9944/tcp|9943:9944/udp ================================================ FILE: alvr/xtask/flatpak/README.md ================================================ Content of this file was moved to the appropiate wiki page: https://github.com/alvr-org/ALVR/wiki/Installing-ALVR-and-using-SteamVR-on-Linux-through-Flatpak ================================================ FILE: alvr/xtask/flatpak/build_and_install.sh ================================================ #!/bin/sh -e flatpak run org.flatpak.Builder --user --install --force-clean build-dir com.valvesoftware.Steam.Utility.alvr.json ================================================ FILE: alvr/xtask/flatpak/com.valvesoftware.Steam.Utility.alvr.desktop ================================================ [Desktop Entry] Version=1.0 Type=Application Name=ALVR Launcher GenericName=Game Comment=ALVR is an open source remote VR display which allows playing SteamVR games on a standalone headset such as Gear VR or Oculus Go/Quest. Exec=/usr/bin/flatpak run --command=/app/utils/alvr/bin/alvr_launcher com.valvesoftware.Steam Icon=application-alvr-launcher Categories=Game; StartupNotify=true StartupWMClass=ALVR ================================================ FILE: alvr/xtask/flatpak/com.valvesoftware.Steam.Utility.alvr.json ================================================ { "id": "com.valvesoftware.Steam.Utility.alvr", "branch": "stable", "sdk": "org.freedesktop.Sdk//24.08", "sdk-extensions": [ "org.freedesktop.Sdk.Extension.llvm19", "org.freedesktop.Sdk.Extension.rust-stable" ], "runtime": "com.valvesoftware.Steam", "runtime-version": "stable", "appstream-compose": false, "separate-locales": false, "build-extension": true, "build-options": { "append-path": "/usr/lib/sdk/llvm19/bin:/usr/lib/sdk/rust-stable/bin", "build-args": [ "--share=network", "--filesystem=xdg-data" ], "strip": true, "env": { "RUST_BACKTRACE": "full" } }, "finish-args": [ "--share=ipc", "--share=network", "--device=all", "--filesystem=home", "--talk-name=org.freedesktop.Notifications" ], "modules": [ { "name": "alvr", "buildsystem": "simple", "build-commands": [ "mkdir -p /app/utils/alvr/bin", "cd alvr/launcher", "cargo xtask prepare-deps --platform linux --no-nvidia", "cargo xtask build-launcher --release", "mv '/run/build/alvr/build/alvr_launcher_linux/ALVR Launcher' /app/utils/alvr/bin/alvr_launcher", " # get read-only errors from following lines - because this is an extension - so leave as comments until someone figures this out", " # install -D dashboard.ico /app/share/icons/alvr_launcher.ico", " # install -D com.valvesoftware.Steam.Utility.alvr.desktop /app/share/applications/com.valvesoftware.Steam.Utility.alvr.desktop" ], "sources": [ { "type": "dir", "path": "../../../" } ] } ] } ================================================ FILE: alvr/xtask/flatpak/run_with_adb_keys.sh ================================================ #!/bin/sh -e export ADB_VENDOR_KEYS=~/.android/adbkey.pub flatpak override --user --filesystem=~/.android com.valvesoftware.Steam.Utility.alvr flatpak run --env=ADB_VENDOR_KEYS=$ADB_VENDOR_KEYS --env=QT_QPA_PLATFORM=xcb --command=alvr_launcher com.valvesoftware.Steam ================================================ FILE: alvr/xtask/flatpak/setup_xdg_shortcut.sh ================================================ #!/bin/sh -e # Flatpaks usually export their own shortcut - but ALVR is an extension to steamvr, and we can't put our shortcut in steams folder # so for now we need to install the shortcut manually # Shortcut can be installed for user or systemwide - but system folder needs sudo # Note that newly installed shortcuts will not appear available until the desktop environment reloads it's settings # This can be done by restarting computer, or by logging off and on, or similar command like "plasmashell --replace" # systemwide shortcut # sudo cp com.valvesoftware.Steam.Utility.alvr.desktop /var/lib/flatpak/exports/share/applications/ # users local folder cp com.valvesoftware.Steam.Utility.alvr.desktop $HOME/.local/share/flatpak/exports/share/applications/ # copy icon as well xdg-icon-resource install --size 256 alvr_icon.png application-alvr-launcher ================================================ FILE: alvr/xtask/licenses_template.hbs ================================================

Third Party Licenses

This page lists the licenses of the projects used in ALVR.

Overview of licenses:

    {{#each overview}}
  • {{name}} ({{count}})
    • {{/each}}

All license text:

================================================ FILE: alvr/xtask/patches/0001-Add-AV_VAAPI_DRIVER_QUIRK_HEVC_ENCODER_ALIGN_64_16-f.patch ================================================ From 3d142fff33196e284b9c9df7d33c2becf609ea60 Mon Sep 17 00:00:00 2001 From: nyanmisaka Date: Thu, 15 Jun 2023 23:01:58 +0800 Subject: [PATCH] Add AV_VAAPI_DRIVER_QUIRK_HEVC_ENCODER_ALIGN_64_16 for AMD VA-API Signed-off-by: nyanmisaka --- libavcodec/vaapi_encode_h265.c | 10 ++++++++-- libavutil/hwcontext_vaapi.c | 5 +++++ libavutil/hwcontext_vaapi.h | 6 ++++++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index aa7e532f9a..4ea0cf54be 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1236,8 +1236,14 @@ static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) "min CB size %dx%d.\n", priv->ctu_size, priv->ctu_size, priv->min_cb_size, priv->min_cb_size); - ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); - ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); + if (priv->common.hwctx->driver_quirks & + AV_VAAPI_DRIVER_QUIRK_HEVC_ENCODER_ALIGN_64_16) { + ctx->surface_width = FFALIGN(avctx->width, 64); + ctx->surface_height = FFALIGN(avctx->height, 16); + } else { + ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); + ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); + } ctx->slice_block_width = ctx->slice_block_height = priv->ctu_size; diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 6c3a227ddd..337a69344e 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -380,6 +380,11 @@ static const struct { "Splitted-Desktop Systems VDPAU backend for VA-API", AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES, }, + { + "AMD Radeon", + "AMD Radeon", + AV_VAAPI_DRIVER_QUIRK_HEVC_ENCODER_ALIGN_64_16, + } }; static int vaapi_device_init(AVHWDeviceContext *hwdev) diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h index 0b2e071cb3..e4ee2de9a4 100644 --- a/libavutil/hwcontext_vaapi.h +++ b/libavutil/hwcontext_vaapi.h @@ -58,6 +58,12 @@ enum { * and the results of the vaQuerySurfaceAttributes() call will be faked. */ AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3), + + /** + * The driver requires to align the surface with 64x16 for the HEVC encoder, + * and it does not use the min_cb_size like other hardware implementations. + */ + AV_VAAPI_DRIVER_QUIRK_HEVC_ENCODER_ALIGN_64_16 = (1 << 4), }; /** -- 2.34.1 ================================================ FILE: alvr/xtask/patches/0001-av1-encode-backport.patch ================================================ From a105b11a9d1d8be33cd9ba29da41314c1abf7c82 Mon Sep 17 00:00:00 2001 From: Andreas Rheinhardt Date: Fri, 1 Jul 2022 11:44:36 +0200 Subject: [PATCH] avcodec/cbs: Add specialization for ff_cbs_(read|write)_unsigned() These functions allow not only to read and write unsigned values, but also to check ranges and to emit trace output which can be beautified when processing arrays (indices like "[i]" are replaced by their actual numbers). Yet lots of callers actually only need something simpler: Their range is only implicitly restricted by the amount of bits used and they are not part of arrays, hence don't need this beautification. This commit adds specializations for these callers; this is very beneficial size-wise (it reduced the size of .text by 23312 bytes here), as a call is now cheaper. Signed-off-by: Andreas Rheinhardt --- libavcodec/cbs.c | 34 ++++++++++++++++++++++++++++++---- libavcodec/cbs_av1.c | 28 +++++++++++++++++++--------- libavcodec/cbs_h2645.c | 15 +++++++++++++-- libavcodec/cbs_internal.h | 11 ++++++++++- libavcodec/cbs_mpeg2.c | 15 +++++++++++++-- libavcodec/cbs_vp9.c | 14 ++++++++++++-- 6 files changed, 97 insertions(+), 20 deletions(-) diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 13a01bef5133..3ec8285e21ea 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -546,10 +546,13 @@ void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position, position, name, pad, bits, value); } -int ff_cbs_read_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, - int width, const char *name, - const int *subscripts, uint32_t *write_to, - uint32_t range_min, uint32_t range_max) +static av_always_inline int cbs_read_unsigned(CodedBitstreamContext *ctx, + GetBitContext *gbc, + int width, const char *name, + const int *subscripts, + uint32_t *write_to, + uint32_t range_min, + uint32_t range_max) { uint32_t value; int position; @@ -589,6 +592,22 @@ int ff_cbs_read_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, return 0; } +int ff_cbs_read_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, + int width, const char *name, + const int *subscripts, uint32_t *write_to, + uint32_t range_min, uint32_t range_max) +{ + return cbs_read_unsigned(ctx, gbc, width, name, subscripts, + write_to, range_min, range_max); +} + +int ff_cbs_read_simple_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, + int width, const char *name, uint32_t *write_to) +{ + return cbs_read_unsigned(ctx, gbc, width, name, NULL, + write_to, 0, UINT32_MAX); +} + int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, int width, const char *name, const int *subscripts, uint32_t value, @@ -625,6 +644,13 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, return 0; } +int ff_cbs_write_simple_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, + int width, const char *name, uint32_t value) +{ + return ff_cbs_write_unsigned(ctx, pbc, width, name, NULL, + value, 0, MAX_UINT_BITS(width)); +} + int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc, int width, const char *name, const int *subscripts, int32_t *write_to, diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 8788fee09905..452e022b3680 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -412,9 +412,8 @@ static int cbs_av1_read_subexp(CodedBitstreamContext *ctx, GetBitContext *gbc, } if (len < max_len) { - err = ff_cbs_read_unsigned(ctx, gbc, range_bits, - "subexp_bits", NULL, &value, - 0, MAX_UINT_BITS(range_bits)); + err = ff_cbs_read_simple_unsigned(ctx, gbc, range_bits, + "subexp_bits", &value); if (err < 0) return err; @@ -476,10 +475,9 @@ static int cbs_av1_write_subexp(CodedBitstreamContext *ctx, PutBitContext *pbc, return err; if (len < max_len) { - err = ff_cbs_write_unsigned(ctx, pbc, range_bits, - "subexp_bits", NULL, - value - range_offset, - 0, MAX_UINT_BITS(range_bits)); + err = ff_cbs_write_simple_unsigned(ctx, pbc, range_bits, + "subexp_bits", + value - range_offset); if (err < 0) return err; @@ -546,8 +544,6 @@ static size_t cbs_av1_get_payload_bytes_left(GetBitContext *gbc) #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) -#define fb(width, name) \ - xf(width, name, current->name, 0, MAX_UINT_BITS(width), 0, ) #define fc(width, name, range_min, range_max) \ xf(width, name, current->name, range_min, range_max, 0, ) #define flag(name) fb(1, name) @@ -573,6 +569,13 @@ static size_t cbs_av1_get_payload_bytes_left(GetBitContext *gbc) #define READWRITE read #define RWContext GetBitContext +#define fb(width, name) do { \ + uint32_t value; \ + CHECK(ff_cbs_read_simple_unsigned(ctx, rw, width, \ + #name, &value)); \ + current->name = value; \ + } while (0) + #define xf(width, name, var, range_min, range_max, subs, ...) do { \ uint32_t value; \ CHECK(ff_cbs_read_unsigned(ctx, rw, width, #name, \ @@ -645,6 +648,7 @@ static size_t cbs_av1_get_payload_bytes_left(GetBitContext *gbc) #undef READ #undef READWRITE #undef RWContext +#undef fb #undef xf #undef xsu #undef uvlc @@ -661,6 +665,11 @@ static size_t cbs_av1_get_payload_bytes_left(GetBitContext *gbc) #define READWRITE write #define RWContext PutBitContext +#define fb(width, name) do { \ + CHECK(ff_cbs_write_simple_unsigned(ctx, rw, width, #name, \ + current->name)); \ + } while (0) + #define xf(width, name, var, range_min, range_max, subs, ...) do { \ CHECK(ff_cbs_write_unsigned(ctx, rw, width, #name, \ SUBSCRIPTS(subs, __VA_ARGS__), \ @@ -723,6 +732,7 @@ static size_t cbs_av1_get_payload_bytes_left(GetBitContext *gbc) #undef WRITE #undef READWRITE #undef RWContext +#undef fb #undef xf #undef xsu #undef uvlc diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 21c8bc76d50b..318c997d9436 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -264,8 +264,6 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo #define u(width, name, range_min, range_max) \ xu(width, name, current->name, range_min, range_max, 0, ) -#define ub(width, name) \ - xu(width, name, current->name, 0, MAX_UINT_BITS(width), 0, ) #define flag(name) ub(1, name) #define ue(name, range_min, range_max) \ xue(name, current->name, range_min, range_max, 0, ) @@ -301,6 +299,12 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo #define READWRITE read #define RWContext GetBitContext +#define ub(width, name) do { \ + uint32_t value; \ + CHECK(ff_cbs_read_simple_unsigned(ctx, rw, width, #name, \ + &value)); \ + current->name = value; \ + } while (0) #define xu(width, name, var, range_min, range_max, subs, ...) do { \ uint32_t value; \ CHECK(ff_cbs_read_unsigned(ctx, rw, width, #name, \ @@ -379,6 +383,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef READ #undef READWRITE #undef RWContext +#undef ub #undef xu #undef xi #undef xue @@ -394,6 +399,11 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #define READWRITE write #define RWContext PutBitContext +#define ub(width, name) do { \ + uint32_t value = current->name; \ + CHECK(ff_cbs_write_simple_unsigned(ctx, rw, width, #name, \ + value)); \ + } while (0) #define xu(width, name, var, range_min, range_max, subs, ...) do { \ uint32_t value = var; \ CHECK(ff_cbs_write_unsigned(ctx, rw, width, #name, \ @@ -461,6 +471,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef WRITE #undef READWRITE #undef RWContext +#undef ub #undef xu #undef xi #undef xue diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index b752d6468452..da84697a29f8 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -163,18 +163,27 @@ void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position, // Helper functions for read/write of common bitstream elements, including -// generation of trace output. +// generation of trace output. The simple functions are equivalent to +// their non-simple counterparts except that their range is unrestricted +// (i.e. only limited by the amount of bits used) and they lack +// the ability to use subscripts. int ff_cbs_read_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, int width, const char *name, const int *subscripts, uint32_t *write_to, uint32_t range_min, uint32_t range_max); +int ff_cbs_read_simple_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, + int width, const char *name, uint32_t *write_to); + int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, int width, const char *name, const int *subscripts, uint32_t value, uint32_t range_min, uint32_t range_max); +int ff_cbs_write_simple_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, + int width, const char *name, uint32_t value); + int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc, int width, const char *name, const int *subscripts, int32_t *write_to, diff --git a/libavcodec/cbs_mpeg2.c b/libavcodec/cbs_mpeg2.c index 04b0c7f87ddb..37fc28a4e62b 100644 --- a/libavcodec/cbs_mpeg2.c +++ b/libavcodec/cbs_mpeg2.c @@ -40,8 +40,6 @@ #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) -#define ui(width, name) \ - xui(width, name, current->name, 0, MAX_UINT_BITS(width), 0, ) #define uir(width, name) \ xui(width, name, current->name, 1, MAX_UINT_BITS(width), 0, ) #define uis(width, name, subs, ...) \ @@ -65,6 +63,12 @@ #define READWRITE read #define RWContext GetBitContext +#define ui(width, name) do { \ + uint32_t value; \ + CHECK(ff_cbs_read_simple_unsigned(ctx, rw, width, #name, \ + &value)); \ + current->name = value; \ + } while (0) #define xuia(width, string, var, range_min, range_max, subs, ...) do { \ uint32_t value; \ CHECK(ff_cbs_read_unsigned(ctx, rw, width, string, \ @@ -95,6 +99,7 @@ #undef READ #undef READWRITE #undef RWContext +#undef ui #undef xuia #undef xsi #undef nextbits @@ -105,6 +110,11 @@ #define READWRITE write #define RWContext PutBitContext +#define ui(width, name) do { \ + CHECK(ff_cbs_write_simple_unsigned(ctx, rw, width, #name, \ + current->name)); \ + } while (0) + #define xuia(width, string, var, range_min, range_max, subs, ...) do { \ CHECK(ff_cbs_write_unsigned(ctx, rw, width, string, \ SUBSCRIPTS(subs, __VA_ARGS__), \ @@ -134,6 +144,7 @@ #undef WRITE #undef READWRITE #undef RWContext +#undef ui #undef xuia #undef xsi #undef nextbits diff --git a/libavcodec/cbs_vp9.c b/libavcodec/cbs_vp9.c index 184fdcade6f8..b0d5bd8763fd 100644 --- a/libavcodec/cbs_vp9.c +++ b/libavcodec/cbs_vp9.c @@ -251,8 +251,6 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, PutBitContext *pbc, #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) -#define f(width, name) \ - xf(width, name, current->name, 0, ) #define s(width, name) \ xs(width, name, current->name, 0, ) #define fs(width, name, subs, ...) \ @@ -264,6 +262,12 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, PutBitContext *pbc, #define READWRITE read #define RWContext GetBitContext +#define f(width, name) do { \ + uint32_t value; \ + CHECK(ff_cbs_read_simple_unsigned(ctx, rw, width, #name, \ + &value)); \ + current->name = value; \ + } while (0) #define xf(width, name, var, subs, ...) do { \ uint32_t value; \ CHECK(ff_cbs_read_unsigned(ctx, rw, width, #name, \ @@ -329,6 +333,7 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, PutBitContext *pbc, #undef READ #undef READWRITE #undef RWContext +#undef f #undef xf #undef xs #undef increment @@ -344,6 +349,10 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, PutBitContext *pbc, #define READWRITE write #define RWContext PutBitContext +#define f(width, name) do { \ + CHECK(ff_cbs_write_simple_unsigned(ctx, rw, width, #name, \ + current->name)); \ + } while (0) #define xf(width, name, var, subs, ...) do { \ CHECK(ff_cbs_write_unsigned(ctx, rw, width, #name, \ SUBSCRIPTS(subs, __VA_ARGS__), \ @@ -396,6 +405,7 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, PutBitContext *pbc, #undef WRITE #undef READWRITE #undef RWContext +#undef f #undef xf #undef xs #undef increment From 95b5c8172968e942704cb4eaceddf8730d9e501c Mon Sep 17 00:00:00 2001 From: James Almer Date: Thu, 21 Sep 2023 22:02:52 -0300 Subject: [PATCH] avcodec/extract_extradata: use size_t as parameter type in val_in_array() It only gets passed the return value of FF_ARRAY_ELEMS(), which is a size_t. Signed-off-by: James Almer --- libavcodec/extract_extradata_bsf.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/libavcodec/extract_extradata_bsf.c b/libavcodec/extract_extradata_bsf.c index efc843736b5a..baa629295fba 100644 --- a/libavcodec/extract_extradata_bsf.c +++ b/libavcodec/extract_extradata_bsf.c @@ -49,10 +49,9 @@ typedef struct ExtractExtradataContext { int remove; } ExtractExtradataContext; -static int val_in_array(const int *arr, int len, int val) +static int val_in_array(const int *arr, size_t len, int val) { - int i; - for (i = 0; i < len; i++) + for (size_t i = 0; i < len; i++) if (arr[i] == val) return 1; return 0; @@ -177,7 +176,7 @@ static int extract_extradata_h2645(AVBSFContext *ctx, AVPacket *pkt, int extradata_size = 0, filtered_size = 0; const int *extradata_nal_types; - int nb_extradata_nal_types; + size_t nb_extradata_nal_types; int i, has_sps = 0, has_vps = 0, ret = 0; if (ctx->par_in->codec_id == AV_CODEC_ID_VVC) { From abe16daea1b72323e3544cb6ec12bec010b6ba54 Mon Sep 17 00:00:00 2001 From: Mark Thompson Date: Mon, 11 Sep 2023 15:52:26 +0800 Subject: [PATCH] cbs: Make tracing more general Turn tracing into callbacks for each syntax element, with default callbacks to match current trace_headers behaviour for debug. Move the construction of bit strings into the trace callback, which simplifies all of the read and write functions. Signed-off-by: Fei Wang Reviewed-by: Neal Gompa --- libavcodec/cbs.c | 121 +++++++++---------- libavcodec/cbs.h | 88 +++++++++++++- libavcodec/cbs_av1.c | 206 +++++++++------------------------ libavcodec/cbs_bsf.c | 5 + libavcodec/cbs_h2645.c | 134 +++++++++------------ libavcodec/cbs_internal.h | 85 +++++++++++++- libavcodec/cbs_vp9.c | 108 +++++------------ libavcodec/trace_headers_bsf.c | 2 + 8 files changed, 373 insertions(+), 376 deletions(-) diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 3ec8285e21ea..daf7f66427f4 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -117,8 +117,9 @@ av_cold int ff_cbs_init(CodedBitstreamContext **ctx_ptr, ctx->decompose_unit_types = NULL; - ctx->trace_enable = 0; - ctx->trace_level = AV_LOG_TRACE; + ctx->trace_enable = 0; + ctx->trace_level = AV_LOG_TRACE; + ctx->trace_context = ctx; *ctx_ptr = ctx; return 0; @@ -496,19 +497,27 @@ void ff_cbs_trace_header(CodedBitstreamContext *ctx, av_log(ctx->log_ctx, ctx->trace_level, "%s\n", name); } -void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position, - const char *str, const int *subscripts, - const char *bits, int64_t value) +void ff_cbs_trace_read_log(void *trace_context, + GetBitContext *gbc, int length, + const char *str, const int *subscripts, + int64_t value) { + CodedBitstreamContext *ctx = trace_context; char name[256]; + char bits[256]; size_t name_len, bits_len; int pad, subs, i, j, k, n; - - if (!ctx->trace_enable) - return; + int position; av_assert0(value >= INT_MIN && value <= UINT32_MAX); + position = get_bits_count(gbc); + + av_assert0(length < 256); + for (i = 0; i < length; i++) + bits[i] = get_bits1(gbc) ? '1' : '0'; + bits[length] = 0; + subs = subscripts ? subscripts[0] : 0; n = 0; for (i = j = 0; str[i];) { @@ -535,7 +544,7 @@ void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position, av_assert0(n == subs); name_len = strlen(name); - bits_len = strlen(bits); + bits_len = length; if (name_len + bits_len > 60) pad = bits_len + 2; @@ -546,6 +555,36 @@ void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position, position, name, pad, bits, value); } +void ff_cbs_trace_write_log(void *trace_context, + PutBitContext *pbc, int length, + const char *str, const int *subscripts, + int64_t value) +{ + CodedBitstreamContext *ctx = trace_context; + + // Ensure that the syntax element is written to the output buffer, + // make a GetBitContext pointed at the start position, then call the + // read log function which can read the bits back to log them. + + GetBitContext gbc; + int position; + + if (length > 0) { + PutBitContext flush; + flush = *pbc; + flush_put_bits(&flush); + } + + position = put_bits_count(pbc); + av_assert0(position >= length); + + init_get_bits(&gbc, pbc->buf, position); + + skip_bits_long(&gbc, position - length); + + ff_cbs_trace_read_log(ctx, &gbc, length, str, subscripts, value); +} + static av_always_inline int cbs_read_unsigned(CodedBitstreamContext *ctx, GetBitContext *gbc, int width, const char *name, @@ -555,7 +594,8 @@ static av_always_inline int cbs_read_unsigned(CodedBitstreamContext *ctx, uint32_t range_max) { uint32_t value; - int position; + + CBS_TRACE_READ_START(); av_assert0(width > 0 && width <= 32); @@ -565,21 +605,9 @@ static av_always_inline int cbs_read_unsigned(CodedBitstreamContext *ctx, return AVERROR_INVALIDDATA; } - if (ctx->trace_enable) - position = get_bits_count(gbc); - value = get_bits_long(gbc, width); - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < width; i++) - bits[i] = value >> (width - i - 1) & 1 ? '1' : '0'; - bits[i] = 0; - - ff_cbs_trace_syntax_element(ctx, position, name, subscripts, - bits, value); - } + CBS_TRACE_READ_END(); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -613,6 +641,8 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, const int *subscripts, uint32_t value, uint32_t range_min, uint32_t range_max) { + CBS_TRACE_WRITE_START(); + av_assert0(width > 0 && width <= 32); if (value < range_min || value > range_max) { @@ -625,22 +655,13 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, if (put_bits_left(pbc) < width) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < width; i++) - bits[i] = value >> (width - i - 1) & 1 ? '1' : '0'; - bits[i] = 0; - - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, subscripts, bits, value); - } - if (width < 32) put_bits(pbc, width, value); else put_bits32(pbc, value); + CBS_TRACE_WRITE_END(); + return 0; } @@ -657,7 +678,8 @@ int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc, int32_t range_min, int32_t range_max) { int32_t value; - int position; + + CBS_TRACE_READ_START(); av_assert0(width > 0 && width <= 32); @@ -667,21 +689,9 @@ int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc, return AVERROR_INVALIDDATA; } - if (ctx->trace_enable) - position = get_bits_count(gbc); - value = get_sbits_long(gbc, width); - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < width; i++) - bits[i] = value & (1U << (width - i - 1)) ? '1' : '0'; - bits[i] = 0; - - ff_cbs_trace_syntax_element(ctx, position, name, subscripts, - bits, value); - } + CBS_TRACE_READ_END(); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -699,6 +709,8 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, const int *subscripts, int32_t value, int32_t range_min, int32_t range_max) { + CBS_TRACE_WRITE_START(); + av_assert0(width > 0 && width <= 32); if (value < range_min || value > range_max) { @@ -711,22 +723,13 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, if (put_bits_left(pbc) < width) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < width; i++) - bits[i] = value & (1U << (width - i - 1)) ? '1' : '0'; - bits[i] = 0; - - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, subscripts, bits, value); - } - if (width < 32) put_sbits(pbc, width, value); else put_bits32(pbc, value); + CBS_TRACE_WRITE_END(); + return 0; } diff --git a/libavcodec/cbs.h b/libavcodec/cbs.h index b4131db5fe27..ffb2797761bb 100644 --- a/libavcodec/cbs.h +++ b/libavcodec/cbs.h @@ -168,6 +168,51 @@ typedef struct CodedBitstreamFragment { CodedBitstreamUnit *units; } CodedBitstreamFragment; + +struct CodedBitstreamContext; +struct GetBitContext; +struct PutBitContext; + +/** + * Callback type for read tracing. + * + * @param ctx User-set trace context. + * @param gbc A GetBitContext set at the start of the syntax + * element. This is a copy, the callee does not + * need to preserve it. + * @param length Length in bits of the syntax element. + * @param name String name of the syntax elements. + * @param subscripts If the syntax element is an array, a pointer to + * an array of subscripts into the array. + * @param value Parsed value of the syntax element. + */ +typedef void (*CBSTraceReadCallback)(void *trace_context, + struct GetBitContext *gbc, + int start_position, + const char *name, + const int *subscripts, + int64_t value); + +/** + * Callback type for write tracing. + * + * @param ctx User-set trace context. + * @param pbc A PutBitContext set at the end of the syntax + * element. The user must not modify this, but may + * inspect it to determine state. + * @param length Length in bits of the syntax element. + * @param name String name of the syntax elements. + * @param subscripts If the syntax element is an array, a pointer to + * an array of subscripts into the array. + * @param value Written value of the syntax element. + */ +typedef void (*CBSTraceWriteCallback)(void *trace_context, + struct PutBitContext *pbc, + int start_position, + const char *name, + const int *subscripts, + int64_t value); + /** * Context structure for coded bitstream operations. */ @@ -211,11 +256,29 @@ typedef struct CodedBitstreamContext { */ int trace_enable; /** - * Log level to use for trace output. + * Log level to use for default trace output. * * From AV_LOG_*; defaults to AV_LOG_TRACE. */ int trace_level; + /** + * User context pointer to pass to trace callbacks. + */ + void *trace_context; + /** + * Callback for read tracing. + * + * If tracing is enabled then this is called once for each syntax + * element parsed. + */ + CBSTraceReadCallback trace_read_callback; + /** + * Callback for write tracing. + * + * If tracing is enabled then this is called once for each syntax + * element written. + */ + CBSTraceWriteCallback trace_write_callback; /** * Write buffer. Used as intermediate buffer when writing units. @@ -450,4 +513,27 @@ void ff_cbs_discard_units(CodedBitstreamContext *ctx, enum AVDiscard skip, int flags); + +/** + * Helper function for read tracing which formats the syntax element + * and logs the result. + * + * Trace context should be set to the CodedBitstreamContext. + */ +void ff_cbs_trace_read_log(void *trace_context, + struct GetBitContext *gbc, int length, + const char *str, const int *subscripts, + int64_t value); + +/** + * Helper function for write tracing which formats the syntax element + * and logs the result. + * + * Trace context should be set to the CodedBitstreamContext. + */ +void ff_cbs_trace_write_log(void *trace_context, + struct PutBitContext *pbc, int length, + const char *str, const int *subscripts, + int64_t value); + #endif /* AVCODEC_CBS_H */ diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 7fb5bd8b42ea..6c478603f173 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -31,10 +31,8 @@ static int cbs_av1_read_uvlc(CodedBitstreamContext *ctx, GetBitContext *gbc, uint32_t range_min, uint32_t range_max) { uint32_t zeroes, bits_value, value; - int position; - if (ctx->trace_enable) - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); zeroes = 0; while (1) { @@ -50,6 +48,9 @@ static int cbs_av1_read_uvlc(CodedBitstreamContext *ctx, GetBitContext *gbc, } if (zeroes >= 32) { + // Note that the spec allows an arbitrarily large number of + // zero bits followed by a one bit in this case, but the + // libaom implementation does not support it. value = MAX_UINT_BITS(32); } else { if (get_bits_left(gbc) < zeroes) { @@ -62,36 +63,7 @@ static int cbs_av1_read_uvlc(CodedBitstreamContext *ctx, GetBitContext *gbc, value = bits_value + (UINT32_C(1) << zeroes) - 1; } - if (ctx->trace_enable) { - char bits[65]; - int i, j, k; - - if (zeroes >= 32) { - while (zeroes > 32) { - k = FFMIN(zeroes - 32, 32); - for (i = 0; i < k; i++) - bits[i] = '0'; - bits[i] = 0; - ff_cbs_trace_syntax_element(ctx, position, name, - NULL, bits, 0); - zeroes -= k; - position += k; - } - } - - for (i = 0; i < zeroes; i++) - bits[i] = '0'; - bits[i++] = '1'; - - if (zeroes < 32) { - for (j = 0; j < zeroes; j++) - bits[i++] = (bits_value >> (zeroes - j - 1) & 1) ? '1' : '0'; - } - - bits[i] = 0; - ff_cbs_trace_syntax_element(ctx, position, name, - NULL, bits, value); - } + CBS_TRACE_READ_END_NO_SUBSCRIPTS(); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -109,7 +81,9 @@ static int cbs_av1_write_uvlc(CodedBitstreamContext *ctx, PutBitContext *pbc, uint32_t range_min, uint32_t range_max) { uint32_t v; - int position, zeroes; + int zeroes; + + CBS_TRACE_WRITE_START(); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -118,28 +92,17 @@ static int cbs_av1_write_uvlc(CodedBitstreamContext *ctx, PutBitContext *pbc, return AVERROR_INVALIDDATA; } - if (ctx->trace_enable) - position = put_bits_count(pbc); - zeroes = av_log2(value + 1); v = value - (1U << zeroes) + 1; + + if (put_bits_left(pbc) < 2 * zeroes + 1) + return AVERROR(ENOSPC); + put_bits(pbc, zeroes, 0); put_bits(pbc, 1, 1); put_bits(pbc, zeroes, v); - if (ctx->trace_enable) { - char bits[65]; - int i, j; - i = 0; - for (j = 0; j < zeroes; j++) - bits[i++] = '0'; - bits[i++] = '1'; - for (j = 0; j < zeroes; j++) - bits[i++] = (v >> (zeroes - j - 1) & 1) ? '1' : '0'; - bits[i++] = 0; - ff_cbs_trace_syntax_element(ctx, position, name, NULL, - bits, value); - } + CBS_TRACE_WRITE_END_NO_SUBSCRIPTS(); return 0; } @@ -148,20 +111,19 @@ static int cbs_av1_read_leb128(CodedBitstreamContext *ctx, GetBitContext *gbc, const char *name, uint64_t *write_to) { uint64_t value; - int position, err, i; + uint32_t byte; + int i; - if (ctx->trace_enable) - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); value = 0; for (i = 0; i < 8; i++) { - int subscript[2] = { 1, i }; - uint32_t byte; - err = ff_cbs_read_unsigned(ctx, gbc, 8, "leb128_byte[i]", subscript, - &byte, 0x00, 0xff); - if (err < 0) - return err; - + if (get_bits_left(gbc) < 8) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid leb128 at " + "%s: bitstream ended.\n", name); + return AVERROR_INVALIDDATA; + } + byte = get_bits(gbc, 8); value |= (uint64_t)(byte & 0x7f) << (i * 7); if (!(byte & 0x80)) break; @@ -170,8 +132,7 @@ static int cbs_av1_read_leb128(CodedBitstreamContext *ctx, GetBitContext *gbc, if (value > UINT32_MAX) return AVERROR_INVALIDDATA; - if (ctx->trace_enable) - ff_cbs_trace_syntax_element(ctx, position, name, NULL, "", value); + CBS_TRACE_READ_END_NO_SUBSCRIPTS(); *write_to = value; return 0; @@ -180,29 +141,25 @@ static int cbs_av1_read_leb128(CodedBitstreamContext *ctx, GetBitContext *gbc, static int cbs_av1_write_leb128(CodedBitstreamContext *ctx, PutBitContext *pbc, const char *name, uint64_t value) { - int position, err, len, i; + int len, i; uint8_t byte; - len = (av_log2(value) + 7) / 7; + CBS_TRACE_WRITE_START(); - if (ctx->trace_enable) - position = put_bits_count(pbc); + len = (av_log2(value) + 7) / 7; for (i = 0; i < len; i++) { - int subscript[2] = { 1, i }; + if (put_bits_left(pbc) < 8) + return AVERROR(ENOSPC); byte = value >> (7 * i) & 0x7f; if (i < len - 1) byte |= 0x80; - err = ff_cbs_write_unsigned(ctx, pbc, 8, "leb128_byte[i]", subscript, - byte, 0x00, 0xff); - if (err < 0) - return err; + put_bits(pbc, 8, byte); } - if (ctx->trace_enable) - ff_cbs_trace_syntax_element(ctx, position, name, NULL, "", value); + CBS_TRACE_WRITE_END_NO_SUBSCRIPTS(); return 0; } @@ -212,12 +169,11 @@ static int cbs_av1_read_ns(CodedBitstreamContext *ctx, GetBitContext *gbc, const int *subscripts, uint32_t *write_to) { uint32_t m, v, extra_bit, value; - int position, w; + int w; - av_assert0(n > 0); + CBS_TRACE_READ_START(); - if (ctx->trace_enable) - position = get_bits_count(gbc); + av_assert0(n > 0); w = av_log2(n) + 1; m = (1 << w) - n; @@ -240,18 +196,7 @@ static int cbs_av1_read_ns(CodedBitstreamContext *ctx, GetBitContext *gbc, value = (v << 1) - m + extra_bit; } - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < w - 1; i++) - bits[i] = (v >> i & 1) ? '1' : '0'; - if (v >= m) - bits[i++] = extra_bit ? '1' : '0'; - bits[i] = 0; - - ff_cbs_trace_syntax_element(ctx, position, - name, subscripts, bits, value); - } + CBS_TRACE_READ_END(); *write_to = value; return 0; @@ -262,7 +207,8 @@ static int cbs_av1_write_ns(CodedBitstreamContext *ctx, PutBitContext *pbc, const int *subscripts, uint32_t value) { uint32_t w, m, v, extra_bit; - int position; + + CBS_TRACE_WRITE_START(); if (value > n) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -271,9 +217,6 @@ static int cbs_av1_write_ns(CodedBitstreamContext *ctx, PutBitContext *pbc, return AVERROR_INVALIDDATA; } - if (ctx->trace_enable) - position = put_bits_count(pbc); - w = av_log2(n) + 1; m = (1 << w) - n; @@ -290,18 +233,7 @@ static int cbs_av1_write_ns(CodedBitstreamContext *ctx, PutBitContext *pbc, put_bits(pbc, 1, extra_bit); } - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < w - 1; i++) - bits[i] = (v >> i & 1) ? '1' : '0'; - if (value >= m) - bits[i++] = extra_bit ? '1' : '0'; - bits[i] = 0; - - ff_cbs_trace_syntax_element(ctx, position, - name, subscripts, bits, value); - } + CBS_TRACE_WRITE_END(); return 0; } @@ -311,33 +243,24 @@ static int cbs_av1_read_increment(CodedBitstreamContext *ctx, GetBitContext *gbc const char *name, uint32_t *write_to) { uint32_t value; - int position, i; - char bits[33]; - av_assert0(range_min <= range_max && range_max - range_min < sizeof(bits) - 1); - if (ctx->trace_enable) - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); - for (i = 0, value = range_min; value < range_max;) { + av_assert0(range_min <= range_max && range_max - range_min < 32); + + for (value = range_min; value < range_max;) { if (get_bits_left(gbc) < 1) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid increment value at " "%s: bitstream ended.\n", name); return AVERROR_INVALIDDATA; } - if (get_bits1(gbc)) { - bits[i++] = '1'; + if (get_bits1(gbc)) ++value; - } else { - bits[i++] = '0'; + else break; - } } - if (ctx->trace_enable) { - bits[i] = 0; - ff_cbs_trace_syntax_element(ctx, position, - name, NULL, bits, value); - } + CBS_TRACE_READ_END_NO_SUBSCRIPTS(); *write_to = value; return 0; @@ -349,6 +272,8 @@ static int cbs_av1_write_increment(CodedBitstreamContext *ctx, PutBitContext *pb { int len; + CBS_TRACE_WRITE_START(); + av_assert0(range_min <= range_max && range_max - range_min < 32); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -364,23 +289,11 @@ static int cbs_av1_write_increment(CodedBitstreamContext *ctx, PutBitContext *pb if (put_bits_left(pbc) < len) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < len; i++) { - if (range_min + i == value) - bits[i] = '0'; - else - bits[i] = '1'; - } - bits[i] = 0; - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, NULL, bits, value); - } - if (len > 0) put_bits(pbc, len, (1 << len) - 1 - (value != range_max)); + CBS_TRACE_WRITE_END_NO_SUBSCRIPTS(); + return 0; } @@ -388,12 +301,10 @@ static int cbs_av1_read_subexp(CodedBitstreamContext *ctx, GetBitContext *gbc, uint32_t range_max, const char *name, const int *subscripts, uint32_t *write_to) { - uint32_t value; - int position, err; - uint32_t max_len, len, range_offset, range_bits; + uint32_t value, max_len, len, range_offset, range_bits; + int err; - if (ctx->trace_enable) - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); av_assert0(range_max > 0); max_len = av_log2(range_max - 1) - 3; @@ -425,9 +336,7 @@ static int cbs_av1_read_subexp(CodedBitstreamContext *ctx, GetBitContext *gbc, } value += range_offset; - if (ctx->trace_enable) - ff_cbs_trace_syntax_element(ctx, position, - name, subscripts, "", value); + CBS_TRACE_READ_END_VALUE_ONLY(); *write_to = value; return err; @@ -437,9 +346,11 @@ static int cbs_av1_write_subexp(CodedBitstreamContext *ctx, PutBitContext *pbc, uint32_t range_max, const char *name, const int *subscripts, uint32_t value) { - int position, err; + int err; uint32_t max_len, len, range_offset, range_bits; + CBS_TRACE_WRITE_START(); + if (value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " "%"PRIu32", but must be in [0,%"PRIu32"].\n", @@ -447,9 +358,6 @@ static int cbs_av1_write_subexp(CodedBitstreamContext *ctx, PutBitContext *pbc, return AVERROR_INVALIDDATA; } - if (ctx->trace_enable) - position = put_bits_count(pbc); - av_assert0(range_max > 0); max_len = av_log2(range_max - 1) - 3; @@ -489,9 +397,7 @@ static int cbs_av1_write_subexp(CodedBitstreamContext *ctx, PutBitContext *pbc, return err; } - if (ctx->trace_enable) - ff_cbs_trace_syntax_element(ctx, position, - name, subscripts, "", value); + CBS_TRACE_WRITE_END_VALUE_ONLY(); return err; } diff --git a/libavcodec/cbs_bsf.c b/libavcodec/cbs_bsf.c index 069f6e99186a..b25285483bc7 100644 --- a/libavcodec/cbs_bsf.c +++ b/libavcodec/cbs_bsf.c @@ -123,6 +123,11 @@ int ff_cbs_bsf_generic_init(AVBSFContext *bsf, const CBSBSFType *type) if (err < 0) return err; + ctx->output->trace_enable = 1; + ctx->output->trace_level = AV_LOG_TRACE; + ctx->output->trace_context = ctx->output; + ctx->output->trace_write_callback = ff_cbs_trace_write_log; + if (bsf->par_in->extradata) { err = ff_cbs_read_extradata(ctx->input, frag, bsf->par_in); if (err < 0) { diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 318c997d9436..0a1c8ea42660 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -36,41 +36,38 @@ static int cbs_read_ue_golomb(CodedBitstreamContext *ctx, GetBitContext *gbc, uint32_t *write_to, uint32_t range_min, uint32_t range_max) { - uint32_t value; - int position, i, j; - unsigned int k; - char bits[65]; + uint32_t leading_bits, value; + int max_length, leading_zeroes; - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); - for (i = 0; i < 32; i++) { - if (get_bits_left(gbc) < i + 1) { + max_length = FFMIN(get_bits_left(gbc), 32); + + leading_bits = show_bits_long(gbc, max_length); + if (leading_bits == 0) { + if (max_length >= 32) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid ue-golomb code at " + "%s: more than 31 zeroes.\n", name); + return AVERROR_INVALIDDATA; + } else { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid ue-golomb code at " "%s: bitstream ended.\n", name); return AVERROR_INVALIDDATA; } - k = get_bits1(gbc); - bits[i] = k ? '1' : '0'; - if (k) - break; } - if (i >= 32) { + + leading_zeroes = max_length - 1 - av_log2(leading_bits); + skip_bits_long(gbc, leading_zeroes); + + if (get_bits_left(gbc) < leading_zeroes + 1) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid ue-golomb code at " - "%s: more than 31 zeroes.\n", name); + "%s: bitstream ended.\n", name); return AVERROR_INVALIDDATA; } - value = 1; - for (j = 0; j < i; j++) { - k = get_bits1(gbc); - bits[i + j + 1] = k ? '1' : '0'; - value = value << 1 | k; - } - bits[i + j + 1] = 0; - --value; - if (ctx->trace_enable) - ff_cbs_trace_syntax_element(ctx, position, name, subscripts, - bits, value); + value = get_bits_long(gbc, leading_zeroes + 1) - 1; + + CBS_TRACE_READ_END(); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -88,45 +85,44 @@ static int cbs_read_se_golomb(CodedBitstreamContext *ctx, GetBitContext *gbc, int32_t *write_to, int32_t range_min, int32_t range_max) { + uint32_t leading_bits, unsigned_value; + int max_length, leading_zeroes; int32_t value; - int position, i, j; - unsigned int k; - uint32_t v; - char bits[65]; - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); - for (i = 0; i < 32; i++) { - if (get_bits_left(gbc) < i + 1) { + max_length = FFMIN(get_bits_left(gbc), 32); + + leading_bits = show_bits_long(gbc, max_length); + if (leading_bits == 0) { + if (max_length >= 32) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid se-golomb code at " + "%s: more than 31 zeroes.\n", name); + return AVERROR_INVALIDDATA; + } else { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid se-golomb code at " "%s: bitstream ended.\n", name); return AVERROR_INVALIDDATA; } - k = get_bits1(gbc); - bits[i] = k ? '1' : '0'; - if (k) - break; } - if (i >= 32) { + + leading_zeroes = max_length - 1 - av_log2(leading_bits); + skip_bits_long(gbc, leading_zeroes); + + if (get_bits_left(gbc) < leading_zeroes + 1) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid se-golomb code at " - "%s: more than 31 zeroes.\n", name); + "%s: bitstream ended.\n", name); return AVERROR_INVALIDDATA; } - v = 1; - for (j = 0; j < i; j++) { - k = get_bits1(gbc); - bits[i + j + 1] = k ? '1' : '0'; - v = v << 1 | k; - } - bits[i + j + 1] = 0; - if (v & 1) - value = -(int32_t)(v / 2); + + unsigned_value = get_bits_long(gbc, leading_zeroes + 1); + + if (unsigned_value & 1) + value = -(int32_t)(unsigned_value / 2); else - value = v / 2; + value = unsigned_value / 2; - if (ctx->trace_enable) - ff_cbs_trace_syntax_element(ctx, position, name, subscripts, - bits, value); + CBS_TRACE_READ_END(); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -146,6 +142,8 @@ static int cbs_write_ue_golomb(CodedBitstreamContext *ctx, PutBitContext *pbc, { int len; + CBS_TRACE_WRITE_START(); + if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " "%"PRIu32", but must be in [%"PRIu32",%"PRIu32"].\n", @@ -158,27 +156,14 @@ static int cbs_write_ue_golomb(CodedBitstreamContext *ctx, PutBitContext *pbc, if (put_bits_left(pbc) < 2 * len + 1) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[65]; - int i; - - for (i = 0; i < len; i++) - bits[i] = '0'; - bits[len] = '1'; - for (i = 0; i < len; i++) - bits[len + i + 1] = (value + 1) >> (len - i - 1) & 1 ? '1' : '0'; - bits[len + len + 1] = 0; - - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, subscripts, bits, value); - } - put_bits(pbc, len, 0); if (len + 1 < 32) put_bits(pbc, len + 1, value + 1); else put_bits32(pbc, value + 1); + CBS_TRACE_WRITE_END(); + return 0; } @@ -190,6 +175,8 @@ static int cbs_write_se_golomb(CodedBitstreamContext *ctx, PutBitContext *pbc, int len; uint32_t uvalue; + CBS_TRACE_WRITE_START(); + if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " "%"PRId32", but must be in [%"PRId32",%"PRId32"].\n", @@ -209,27 +196,14 @@ static int cbs_write_se_golomb(CodedBitstreamContext *ctx, PutBitContext *pbc, if (put_bits_left(pbc) < 2 * len + 1) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[65]; - int i; - - for (i = 0; i < len; i++) - bits[i] = '0'; - bits[len] = '1'; - for (i = 0; i < len; i++) - bits[len + i + 1] = (uvalue + 1) >> (len - i - 1) & 1 ? '1' : '0'; - bits[len + len + 1] = 0; - - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, subscripts, bits, value); - } - put_bits(pbc, len, 0); if (len + 1 < 32) put_bits(pbc, len + 1, uvalue + 1); else put_bits32(pbc, uvalue + 1); + CBS_TRACE_WRITE_END(); + return 0; } diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index da84697a29f8..285deb40d516 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -157,10 +157,6 @@ typedef struct CodedBitstreamType { void ff_cbs_trace_header(CodedBitstreamContext *ctx, const char *name); -void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position, - const char *name, const int *subscripts, - const char *bitstring, int64_t value); - // Helper functions for read/write of common bitstream elements, including // generation of trace output. The simple functions are equivalent to @@ -206,6 +202,87 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, // range_min in the above functions. #define MIN_INT_BITS(length) (-(INT64_C(1) << ((length) - 1))) + +// Start of a syntax element during read tracing. +#define CBS_TRACE_READ_START() \ + GetBitContext trace_start; \ + do { \ + if (ctx->trace_enable) \ + trace_start = *gbc; \ + } while (0) + +// End of a syntax element for tracing, make callback. +#define CBS_TRACE_READ_END() \ + do { \ + if (ctx->trace_enable) { \ + int start_position = get_bits_count(&trace_start); \ + int end_position = get_bits_count(gbc); \ + av_assert0(start_position <= end_position); \ + ctx->trace_read_callback(ctx->trace_context, &trace_start, \ + end_position - start_position, \ + name, subscripts, value); \ + } \ + } while (0) + +// End of a syntax element with no subscript entries. +#define CBS_TRACE_READ_END_NO_SUBSCRIPTS() \ + do { \ + const int *subscripts = NULL; \ + CBS_TRACE_READ_END(); \ + } while (0) + +// End of a syntax element which is made up of subelements which +// are aleady traced, so we are only showing the value. +#define CBS_TRACE_READ_END_VALUE_ONLY() \ + do { \ + if (ctx->trace_enable) { \ + ctx->trace_read_callback(ctx->trace_context, &trace_start, 0, \ + name, subscripts, value); \ + } \ + } while (0) + +// Start of a syntax element during write tracing. +#define CBS_TRACE_WRITE_START() \ + int start_position; \ + do { \ + if (ctx->trace_enable) \ + start_position = put_bits_count(pbc);; \ + } while (0) + +// End of a syntax element for tracing, make callback. +#define CBS_TRACE_WRITE_END() \ + do { \ + if (ctx->trace_enable) { \ + int end_position = put_bits_count(pbc); \ + av_assert0(start_position <= end_position); \ + ctx->trace_write_callback(ctx->trace_context, pbc, \ + end_position - start_position, \ + name, subscripts, value); \ + } \ + } while (0) + +// End of a syntax element with no subscript entries. +#define CBS_TRACE_WRITE_END_NO_SUBSCRIPTS() \ + do { \ + const int *subscripts = NULL; \ + CBS_TRACE_WRITE_END(); \ + } while (0) + +// End of a syntax element which is made up of subelements which are +// aleady traced, so we are only showing the value. This forges a +// PutBitContext to point to the position of the start of the syntax +// element, but the other state doesn't matter because length is zero. +#define CBS_TRACE_WRITE_END_VALUE_ONLY() \ + do { \ + if (ctx->trace_enable) { \ + PutBitContext tmp; \ + init_put_bits(&tmp, pbc->buf, start_position); \ + skip_put_bits(&tmp, start_position); \ + ctx->trace_write_callback(ctx->trace_context, &tmp, 0, \ + name, subscripts, value); \ + } \ + } while (0) + #define TYPE_LIST(...) { __VA_ARGS__ } #define CBS_UNIT_TYPE_POD(type_, structure) { \ .nb_unit_types = 1, \ diff --git a/libavcodec/cbs_vp9.c b/libavcodec/cbs_vp9.c index b0d5bd8763fd..816d06da04d4 100644 --- a/libavcodec/cbs_vp9.c +++ b/libavcodec/cbs_vp9.c @@ -28,11 +28,10 @@ static int cbs_vp9_read_s(CodedBitstreamContext *ctx, GetBitContext *gbc, const int *subscripts, int32_t *write_to) { uint32_t magnitude; - int position, sign; + int sign; int32_t value; - if (ctx->trace_enable) - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); if (get_bits_left(gbc) < width + 1) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid signed value at " @@ -44,17 +43,7 @@ static int cbs_vp9_read_s(CodedBitstreamContext *ctx, GetBitContext *gbc, sign = get_bits1(gbc); value = sign ? -(int32_t)magnitude : magnitude; - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < width; i++) - bits[i] = magnitude >> (width - i - 1) & 1 ? '1' : '0'; - bits[i] = sign ? '1' : '0'; - bits[i + 1] = 0; - - ff_cbs_trace_syntax_element(ctx, position, name, subscripts, - bits, value); - } + CBS_TRACE_READ_END(); *write_to = value; return 0; @@ -67,27 +56,19 @@ static int cbs_vp9_write_s(CodedBitstreamContext *ctx, PutBitContext *pbc, uint32_t magnitude; int sign; + CBS_TRACE_WRITE_START(); + if (put_bits_left(pbc) < width + 1) return AVERROR(ENOSPC); sign = value < 0; magnitude = sign ? -value : value; - if (ctx->trace_enable) { - char bits[33]; - int i; - for (i = 0; i < width; i++) - bits[i] = magnitude >> (width - i - 1) & 1 ? '1' : '0'; - bits[i] = sign ? '1' : '0'; - bits[i + 1] = 0; - - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, subscripts, bits, value); - } - put_bits(pbc, width, magnitude); put_bits(pbc, 1, sign); + CBS_TRACE_WRITE_END(); + return 0; } @@ -96,32 +77,24 @@ static int cbs_vp9_read_increment(CodedBitstreamContext *ctx, GetBitContext *gbc const char *name, uint32_t *write_to) { uint32_t value; - int position, i; - char bits[8]; - av_assert0(range_min <= range_max && range_max - range_min < sizeof(bits) - 1); - if (ctx->trace_enable) - position = get_bits_count(gbc); + CBS_TRACE_READ_START(); - for (i = 0, value = range_min; value < range_max;) { + av_assert0(range_min <= range_max && range_max - range_min < 32); + + for (value = range_min; value < range_max;) { if (get_bits_left(gbc) < 1) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid increment value at " "%s: bitstream ended.\n", name); return AVERROR_INVALIDDATA; } - if (get_bits1(gbc)) { - bits[i++] = '1'; + if (get_bits1(gbc)) ++value; - } else { - bits[i++] = '0'; + else break; - } } - if (ctx->trace_enable) { - bits[i] = 0; - ff_cbs_trace_syntax_element(ctx, position, name, NULL, bits, value); - } + CBS_TRACE_READ_END_NO_SUBSCRIPTS(); *write_to = value; return 0; @@ -133,6 +106,8 @@ static int cbs_vp9_write_increment(CodedBitstreamContext *ctx, PutBitContext *pb { int len; + CBS_TRACE_WRITE_START(); + av_assert0(range_min <= range_max && range_max - range_min < 8); if (value < range_min || value > range_max) { av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: " @@ -148,23 +123,11 @@ static int cbs_vp9_write_increment(CodedBitstreamContext *ctx, PutBitContext *pb if (put_bits_left(pbc) < len) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[8]; - int i; - for (i = 0; i < len; i++) { - if (range_min + i == value) - bits[i] = '0'; - else - bits[i] = '1'; - } - bits[i] = 0; - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, NULL, bits, value); - } - if (len > 0) put_bits(pbc, len, (1 << len) - 1 - (value != range_max)); + CBS_TRACE_WRITE_END_NO_SUBSCRIPTS(); + return 0; } @@ -173,12 +136,11 @@ static int cbs_vp9_read_le(CodedBitstreamContext *ctx, GetBitContext *gbc, const int *subscripts, uint32_t *write_to) { uint32_t value; - int position, b; + int b; - av_assert0(width % 8 == 0); + CBS_TRACE_READ_START(); - if (ctx->trace_enable) - position = get_bits_count(gbc); + av_assert0(width % 8 == 0); if (get_bits_left(gbc) < width) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid le value at " @@ -190,17 +152,7 @@ static int cbs_vp9_read_le(CodedBitstreamContext *ctx, GetBitContext *gbc, for (b = 0; b < width; b += 8) value |= get_bits(gbc, 8) << b; - if (ctx->trace_enable) { - char bits[33]; - int i; - for (b = 0; b < width; b += 8) - for (i = 0; i < 8; i++) - bits[b + i] = value >> (b + i) & 1 ? '1' : '0'; - bits[b] = 0; - - ff_cbs_trace_syntax_element(ctx, position, name, subscripts, - bits, value); - } + CBS_TRACE_READ_END(); *write_to = value; return 0; @@ -212,26 +164,18 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, PutBitContext *pbc, { int b; + CBS_TRACE_WRITE_START(); + av_assert0(width % 8 == 0); if (put_bits_left(pbc) < width) return AVERROR(ENOSPC); - if (ctx->trace_enable) { - char bits[33]; - int i; - for (b = 0; b < width; b += 8) - for (i = 0; i < 8; i++) - bits[b + i] = value >> (b + i) & 1 ? '1' : '0'; - bits[b] = 0; - - ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc), - name, subscripts, bits, value); - } - for (b = 0; b < width; b += 8) put_bits(pbc, 8, value >> b & 0xff); + CBS_TRACE_WRITE_END(); + return 0; } diff --git a/libavcodec/trace_headers_bsf.c b/libavcodec/trace_headers_bsf.c index 028b0a1e599a..8781f5f10012 100644 --- a/libavcodec/trace_headers_bsf.c +++ b/libavcodec/trace_headers_bsf.c @@ -44,6 +44,8 @@ static int trace_headers_init(AVBSFContext *bsf) ctx->cbc->trace_enable = 1; ctx->cbc->trace_level = AV_LOG_INFO; + ctx->cbc->trace_context = ctx->cbc; + ctx->cbc->trace_read_callback = ff_cbs_trace_read_log; if (bsf->par_in->extradata) { CodedBitstreamFragment *frag = &ctx->fragment; From 695477a1c7d3dd5ad249442b0770ad0acd99dd87 Mon Sep 17 00:00:00 2001 From: Fei Wang Date: Mon, 11 Sep 2023 15:52:27 +0800 Subject: [PATCH] avcodec/cbs_av1: Allow specifying obu size byte length Signed-off-by: Fei Wang Reviewed-by: Neal Gompa --- libavcodec/cbs_av1.c | 30 +++++++++++++++++++++--------- libavcodec/cbs_av1.h | 1 + 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 6c478603f173..4e687ace79d5 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -138,15 +138,19 @@ static int cbs_av1_read_leb128(CodedBitstreamContext *ctx, GetBitContext *gbc, return 0; } +/** Minimum byte length will be used to indicate the len128 of value if byte_len is 0. */ static int cbs_av1_write_leb128(CodedBitstreamContext *ctx, PutBitContext *pbc, - const char *name, uint64_t value) + const char *name, uint64_t value, uint8_t byte_len) { int len, i; uint8_t byte; CBS_TRACE_WRITE_START(); - len = (av_log2(value) + 7) / 7; + if (byte_len) + av_assert0(byte_len >= (av_log2(value) + 7) / 7); + + len = byte_len ? byte_len : (av_log2(value) + 7) / 7; for (i = 0; i < len; i++) { if (put_bits_left(pbc) < 8) @@ -618,7 +622,7 @@ static size_t cbs_av1_get_payload_bytes_left(GetBitContext *gbc) } while (0) #define leb128(name) do { \ - CHECK(cbs_av1_write_leb128(ctx, rw, #name, current->name)); \ + CHECK(cbs_av1_write_leb128(ctx, rw, #name, current->name, 0)); \ } while (0) #define infer(name, value) do { \ @@ -1002,9 +1006,14 @@ static int cbs_av1_write_obu(CodedBitstreamContext *ctx, if (obu->header.obu_has_size_field) { pbc_tmp = *pbc; - // Add space for the size field to fill later. - put_bits32(pbc, 0); - put_bits32(pbc, 0); + if (obu->obu_size_byte_len) { + for (int i = 0; i < obu->obu_size_byte_len; i++) + put_bits(pbc, 8, 0); + } else { + // Add space for the size field to fill later. + put_bits32(pbc, 0); + put_bits32(pbc, 0); + } } td = NULL; @@ -1124,7 +1133,7 @@ static int cbs_av1_write_obu(CodedBitstreamContext *ctx, end_pos /= 8; *pbc = pbc_tmp; - err = cbs_av1_write_leb128(ctx, pbc, "obu_size", obu->obu_size); + err = cbs_av1_write_leb128(ctx, pbc, "obu_size", obu->obu_size, obu->obu_size_byte_len); if (err < 0) goto error; @@ -1141,8 +1150,11 @@ static int cbs_av1_write_obu(CodedBitstreamContext *ctx, } if (obu->obu_size > 0) { - memmove(pbc->buf + data_pos, - pbc->buf + start_pos, header_size); + if (!obu->obu_size_byte_len) { + obu->obu_size_byte_len = start_pos - data_pos; + memmove(pbc->buf + data_pos, + pbc->buf + start_pos, header_size); + } skip_put_bytes(pbc, header_size); if (td) { diff --git a/libavcodec/cbs_av1.h b/libavcodec/cbs_av1.h index 64dfdce9c4eb..a9e2d2284fff 100644 --- a/libavcodec/cbs_av1.h +++ b/libavcodec/cbs_av1.h @@ -401,6 +401,7 @@ typedef struct AV1RawOBU { AV1RawOBUHeader header; size_t obu_size; + uint8_t obu_size_byte_len; union { AV1RawSequenceHeader sequence_header; From 6c3a5d625f917631d1c8bab31ed65ebe26d14f47 Mon Sep 17 00:00:00 2001 From: Fei Wang Date: Mon, 11 Sep 2023 15:52:25 +0800 Subject: [PATCH] avcodec/cbs_av1: Add tx mode enum values Signed-off-by: Fei Wang Reviewed-by: Neal Gompa --- libavcodec/av1.h | 7 +++++++ libavcodec/cbs_av1_syntax_template.c | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/libavcodec/av1.h b/libavcodec/av1.h index 384f7cddc7eb..8704bc41c122 100644 --- a/libavcodec/av1.h +++ b/libavcodec/av1.h @@ -175,6 +175,13 @@ enum { AV1_RESTORE_SWITCHABLE = 3, }; +// TX mode (section 6.8.21) +enum { + AV1_ONLY_4X4 = 0, + AV1_TX_MODE_LARGEST = 1, + AV1_TX_MODE_SELECT = 2, +}; + // Sequence Headers are actually unbounded because one can use // an arbitrary number of leading zeroes when encoding via uvlc. // The following estimate is based around using the lowest number diff --git a/libavcodec/cbs_av1_syntax_template.c b/libavcodec/cbs_av1_syntax_template.c index 6f09c4e4104e..3be1f2d30f96 100644 --- a/libavcodec/cbs_av1_syntax_template.c +++ b/libavcodec/cbs_av1_syntax_template.c @@ -1028,9 +1028,9 @@ static int FUNC(read_tx_mode)(CodedBitstreamContext *ctx, RWContext *rw, int err; if (priv->coded_lossless) - infer(tx_mode, 0); + infer(tx_mode, AV1_ONLY_4X4); else - increment(tx_mode, 1, 2); + increment(tx_mode, AV1_TX_MODE_LARGEST, AV1_TX_MODE_SELECT); return 0; } From 4a4400709ccfc026c19028f4b2d798eed9322f64 Mon Sep 17 00:00:00 2001 From: Fei Wang Date: Mon, 11 Sep 2023 15:52:28 +0800 Subject: [PATCH] lavc/vaapi_encode: Init pic at the beginning of API Signed-off-by: Fei Wang Reviewed-by: Neal Gompa --- libavcodec/vaapi_encode.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 0316fe5c1855..5ae63c9f2557 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1205,7 +1205,7 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) { VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic; + VAAPIEncodePicture *pic = NULL; AVFrame *frame = ctx->frame; int err; @@ -1228,8 +1228,6 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) } if (ctx->has_sync_buffer_func) { - pic = NULL; - if (av_fifo_can_write(ctx->encode_fifo)) { err = vaapi_encode_pick_next(avctx, &pic); if (!err) { @@ -1255,7 +1253,6 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) av_fifo_read(ctx->encode_fifo, &pic, 1); ctx->encode_order = pic->encode_order + 1; } else { - pic = NULL; err = vaapi_encode_pick_next(avctx, &pic); if (err < 0) return err; From 11b81838ae64095fcc130f4747a6adc8676a4998 Mon Sep 17 00:00:00 2001 From: Fei Wang Date: Mon, 11 Sep 2023 15:52:29 +0800 Subject: [PATCH] lavc/vaapi_encode: Extract set output pkt property function Signed-off-by: Fei Wang Reviewed-by: Neal Gompa --- libavcodec/vaapi_encode.c | 65 +++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 27 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 5ae63c9f2557..46762342eb40 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -650,6 +650,41 @@ static int vaapi_encode_issue(AVCodecContext *avctx, return err; } +static int vaapi_encode_set_output_property(AVCodecContext *avctx, + VAAPIEncodePicture *pic, + AVPacket *pkt) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + + if (pic->type == PICTURE_TYPE_IDR) + pkt->flags |= AV_PKT_FLAG_KEY; + + pkt->pts = pic->pts; + pkt->duration = pic->duration; + + // for no-delay encoders this is handled in generic codec + if (avctx->codec->capabilities & AV_CODEC_CAP_DELAY && + avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { + pkt->opaque = pic->opaque; + pkt->opaque_ref = pic->opaque_ref; + pic->opaque_ref = NULL; + } + + if (ctx->output_delay == 0) { + pkt->dts = pkt->pts; + } else if (pic->encode_order < ctx->decode_delay) { + if (ctx->ts_ring[pic->encode_order] < INT64_MIN + ctx->dts_pts_diff) + pkt->dts = INT64_MIN; + else + pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; + } else { + pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % + (3 * ctx->output_delay + ctx->async_depth)]; + } + + return 0; +} + static int vaapi_encode_output(AVCodecContext *avctx, VAAPIEncodePicture *pic, AVPacket *pkt) { @@ -691,12 +726,6 @@ static int vaapi_encode_output(AVCodecContext *avctx, ptr += buf->size; } - if (pic->type == PICTURE_TYPE_IDR) - pkt->flags |= AV_PKT_FLAG_KEY; - - pkt->pts = pic->pts; - pkt->duration = pic->duration; - vas = vaUnmapBuffer(ctx->hwctx->display, pic->output_buffer); if (vas != VA_STATUS_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to unmap output buffers: " @@ -705,14 +734,6 @@ static int vaapi_encode_output(AVCodecContext *avctx, goto fail; } - // for no-delay encoders this is handled in generic codec - if (avctx->codec->capabilities & AV_CODEC_CAP_DELAY && - avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { - pkt->opaque = pic->opaque; - pkt->opaque_ref = pic->opaque_ref; - pic->opaque_ref = NULL; - } - av_buffer_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; @@ -1273,19 +1294,9 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) return err; } - if (ctx->output_delay == 0) { - pkt->dts = pkt->pts; - } else if (pic->encode_order < ctx->decode_delay) { - if (ctx->ts_ring[pic->encode_order] < INT64_MIN + ctx->dts_pts_diff) - pkt->dts = INT64_MIN; - else - pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; - } else { - pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % - (3 * ctx->output_delay + ctx->async_depth)]; - } - av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64" dts %"PRId64".\n", - pkt->pts, pkt->dts); + vaapi_encode_set_output_property(avctx, pic, pkt); + av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " + "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); ctx->output_order = pic->encode_order; vaapi_encode_clear_old(avctx); From 254c5a8134a177244fc0995c3b2998079a755848 Mon Sep 17 00:00:00 2001 From: Fei Wang Date: Mon, 11 Sep 2023 15:52:30 +0800 Subject: [PATCH] lavc/vaapi_encode: Separate reference frame into previous/future list To support more reference frames from different directions. Signed-off-by: Fei Wang Reviewed-by: Neal Gompa --- libavcodec/vaapi_encode.c | 112 +++++++++++++++++++++++++------- libavcodec/vaapi_encode.h | 15 +++-- libavcodec/vaapi_encode_h264.c | 94 +++++++++++++-------------- libavcodec/vaapi_encode_h265.c | 76 +++++++++++++--------- libavcodec/vaapi_encode_mpeg2.c | 6 +- libavcodec/vaapi_encode_vp8.c | 6 +- libavcodec/vaapi_encode_vp9.c | 26 ++++---- 7 files changed, 208 insertions(+), 127 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 46762342eb40..79036673e7a9 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -276,21 +276,34 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " "as type %s.\n", pic->display_order, pic->encode_order, picture_type_name[pic->type]); - if (pic->nb_refs == 0) { + if (pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0) { av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); } else { - av_log(avctx, AV_LOG_DEBUG, "Refers to:"); - for (i = 0; i < pic->nb_refs; i++) { + av_log(avctx, AV_LOG_DEBUG, "L0 refers to"); + for (i = 0; i < pic->nb_refs[0]; i++) { av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, - pic->refs[i]->display_order, pic->refs[i]->encode_order); + pic->refs[0][i]->display_order, pic->refs[0][i]->encode_order); } av_log(avctx, AV_LOG_DEBUG, ".\n"); + + if (pic->nb_refs[1]) { + av_log(avctx, AV_LOG_DEBUG, "L1 refers to"); + for (i = 0; i < pic->nb_refs[1]; i++) { + av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, + pic->refs[1][i]->display_order, pic->refs[1][i]->encode_order); + } + av_log(avctx, AV_LOG_DEBUG, ".\n"); + } } av_assert0(!pic->encode_issued); - for (i = 0; i < pic->nb_refs; i++) { - av_assert0(pic->refs[i]); - av_assert0(pic->refs[i]->encode_issued); + for (i = 0; i < pic->nb_refs[0]; i++) { + av_assert0(pic->refs[0][i]); + av_assert0(pic->refs[0][i]->encode_issued); + } + for (i = 0; i < pic->nb_refs[1]; i++) { + av_assert0(pic->refs[1][i]); + av_assert0(pic->refs[1][i]->encode_issued); } av_log(avctx, AV_LOG_DEBUG, "Input surface is %#x.\n", pic->input_surface); @@ -832,8 +845,12 @@ static void vaapi_encode_add_ref(AVCodecContext *avctx, if (is_ref) { av_assert0(pic != target); - av_assert0(pic->nb_refs < MAX_PICTURE_REFERENCES); - pic->refs[pic->nb_refs++] = target; + av_assert0(pic->nb_refs[0] < MAX_PICTURE_REFERENCES && + pic->nb_refs[1] < MAX_PICTURE_REFERENCES); + if (target->display_order < pic->display_order) + pic->refs[0][pic->nb_refs[0]++] = target; + else + pic->refs[1][pic->nb_refs[1]++] = target; ++refs; } @@ -862,10 +879,16 @@ static void vaapi_encode_remove_refs(AVCodecContext *avctx, if (pic->ref_removed[level]) return; - for (i = 0; i < pic->nb_refs; i++) { - av_assert0(pic->refs[i]); - --pic->refs[i]->ref_count[level]; - av_assert0(pic->refs[i]->ref_count[level] >= 0); + for (i = 0; i < pic->nb_refs[0]; i++) { + av_assert0(pic->refs[0][i]); + --pic->refs[0][i]->ref_count[level]; + av_assert0(pic->refs[0][i]->ref_count[level] >= 0); + } + + for (i = 0; i < pic->nb_refs[1]; i++) { + av_assert0(pic->refs[1][i]); + --pic->refs[1][i]->ref_count[level]; + av_assert0(pic->refs[1][i]->ref_count[level] >= 0); } for (i = 0; i < pic->nb_dpb_pics; i++) { @@ -910,7 +933,7 @@ static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); - for (ref = end->refs[1]; ref; ref = ref->refs[1]) + for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); } *last = prev; @@ -933,7 +956,7 @@ static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); - for (ref = end->refs[1]; ref; ref = ref->refs[1]) + for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); if (i > 1) @@ -947,11 +970,44 @@ static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, } } +static void vaapi_encode_add_next_prev(AVCodecContext *avctx, + VAAPIEncodePicture *pic) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + int i; + + if (!pic) + return; + + if (pic->type == PICTURE_TYPE_IDR) { + for (i = 0; i < ctx->nb_next_prev; i++) { + --ctx->next_prev[i]->ref_count[0]; + ctx->next_prev[i] = NULL; + } + ctx->next_prev[0] = pic; + ++pic->ref_count[0]; + ctx->nb_next_prev = 1; + + return; + } + + if (ctx->nb_next_prev < MAX_PICTURE_REFERENCES) { + ctx->next_prev[ctx->nb_next_prev++] = pic; + ++pic->ref_count[0]; + } else { + --ctx->next_prev[0]->ref_count[0]; + for (i = 0; i < MAX_PICTURE_REFERENCES - 1; i++) + ctx->next_prev[i] = ctx->next_prev[i + 1]; + ctx->next_prev[i] = pic; + ++pic->ref_count[0]; + } +} + static int vaapi_encode_pick_next(AVCodecContext *avctx, VAAPIEncodePicture **pic_out) { VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic = NULL, *next, *start; + VAAPIEncodePicture *pic = NULL, *prev = NULL, *next, *start; int i, b_counter, closed_gop_end; // If there are any B-frames already queued, the next one to encode @@ -962,11 +1018,18 @@ static int vaapi_encode_pick_next(AVCodecContext *avctx, continue; if (pic->type != PICTURE_TYPE_B) continue; - for (i = 0; i < pic->nb_refs; i++) { - if (!pic->refs[i]->encode_issued) + for (i = 0; i < pic->nb_refs[0]; i++) { + if (!pic->refs[0][i]->encode_issued) break; } - if (i == pic->nb_refs) + if (i != pic->nb_refs[0]) + continue; + + for (i = 0; i < pic->nb_refs[1]; i++) { + if (!pic->refs[1][i]->encode_issued) + break; + } + if (i == pic->nb_refs[1]) break; } @@ -1068,18 +1131,17 @@ static int vaapi_encode_pick_next(AVCodecContext *avctx, vaapi_encode_add_ref(avctx, pic, start, pic->type == PICTURE_TYPE_P, b_counter > 0, 0); - vaapi_encode_add_ref(avctx, pic, ctx->next_prev, 0, 0, 1); + vaapi_encode_add_ref(avctx, pic, ctx->next_prev[ctx->nb_next_prev - 1], 0, 0, 1); } - if (ctx->next_prev) - --ctx->next_prev->ref_count[0]; if (b_counter > 0) { vaapi_encode_set_b_pictures(avctx, start, pic, pic, 1, - &ctx->next_prev); + &prev); } else { - ctx->next_prev = pic; + prev = pic; } - ++ctx->next_prev->ref_count[0]; + vaapi_encode_add_next_prev(avctx, prev); + return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index bd25cd5c953a..977bc2d94634 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -49,6 +49,7 @@ enum { // A.4.1: table A.6 allows at most 20 tile columns for any level. MAX_TILE_COLS = 20, MAX_ASYNC_DEPTH = 64, + MAX_REFERENCE_LIST_NUM = 2, }; extern const AVCodecHWConfigInternal *const ff_vaapi_encode_hw_configs[]; @@ -116,10 +117,11 @@ typedef struct VAAPIEncodePicture { // but not if it isn't. int nb_dpb_pics; struct VAAPIEncodePicture *dpb[MAX_DPB_SIZE]; - // The reference pictures used in decoding this picture. If they are - // used by later pictures they will also appear in the DPB. - int nb_refs; - struct VAAPIEncodePicture *refs[MAX_PICTURE_REFERENCES]; + // The reference pictures used in decoding this picture. If they are + // used by later pictures they will also appear in the DPB. ref[0][] for + // previous reference frames. ref[1][] for future reference frames. + int nb_refs[MAX_REFERENCE_LIST_NUM]; + struct VAAPIEncodePicture *refs[MAX_REFERENCE_LIST_NUM][MAX_PICTURE_REFERENCES]; // The previous reference picture in encode order. Must be in at least // one of the reference list and DPB list. struct VAAPIEncodePicture *prev; @@ -290,8 +292,9 @@ typedef struct VAAPIEncodeContext { // Current encoding window, in display (input) order. VAAPIEncodePicture *pic_start, *pic_end; // The next picture to use as the previous reference picture in - // encoding order. - VAAPIEncodePicture *next_prev; + // encoding order. Order from small to large in encoding order. + VAAPIEncodePicture *next_prev[MAX_PICTURE_REFERENCES]; + int nb_next_prev; // Next input order index (display order). int64_t input_order; diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index 09e130011390..57b5ea2babf7 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -628,7 +628,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, VAAPIEncodePicture *prev = pic->prev; VAAPIEncodeH264Picture *hprev = prev ? prev->priv_data : NULL; VAEncPictureParameterBufferH264 *vpic = pic->codec_picture_params; - int i; + int i, j = 0; if (pic->type == PICTURE_TYPE_IDR) { av_assert0(pic->display_order == pic->encode_order); @@ -729,24 +729,26 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, .TopFieldOrderCnt = hpic->pic_order_cnt, .BottomFieldOrderCnt = hpic->pic_order_cnt, }; - - for (i = 0; i < pic->nb_refs; i++) { - VAAPIEncodePicture *ref = pic->refs[i]; - VAAPIEncodeH264Picture *href; - - av_assert0(ref && ref->encode_order < pic->encode_order); - href = ref->priv_data; - - vpic->ReferenceFrames[i] = (VAPictureH264) { - .picture_id = ref->recon_surface, - .frame_idx = href->frame_num, - .flags = VA_PICTURE_H264_SHORT_TERM_REFERENCE, - .TopFieldOrderCnt = href->pic_order_cnt, - .BottomFieldOrderCnt = href->pic_order_cnt, - }; + for (int k = 0; k < MAX_REFERENCE_LIST_NUM; k++) { + for (i = 0; i < pic->nb_refs[k]; i++) { + VAAPIEncodePicture *ref = pic->refs[k][i]; + VAAPIEncodeH264Picture *href; + + av_assert0(ref && ref->encode_order < pic->encode_order); + href = ref->priv_data; + + vpic->ReferenceFrames[j++] = (VAPictureH264) { + .picture_id = ref->recon_surface, + .frame_idx = href->frame_num, + .flags = VA_PICTURE_H264_SHORT_TERM_REFERENCE, + .TopFieldOrderCnt = href->pic_order_cnt, + .BottomFieldOrderCnt = href->pic_order_cnt, + }; + } } - for (; i < FF_ARRAY_ELEMS(vpic->ReferenceFrames); i++) { - vpic->ReferenceFrames[i] = (VAPictureH264) { + + for (; j < FF_ARRAY_ELEMS(vpic->ReferenceFrames); j++) { + vpic->ReferenceFrames[j] = (VAPictureH264) { .picture_id = VA_INVALID_ID, .flags = VA_PICTURE_H264_INVALID, }; @@ -948,17 +950,17 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, if (pic->type == PICTURE_TYPE_P) { int need_rplm = 0; - for (i = 0; i < pic->nb_refs; i++) { - av_assert0(pic->refs[i]); - if (pic->refs[i] != def_l0[i]) + for (i = 0; i < pic->nb_refs[0]; i++) { + av_assert0(pic->refs[0][i]); + if (pic->refs[0][i] != def_l0[i]) need_rplm = 1; } sh->ref_pic_list_modification_flag_l0 = need_rplm; if (need_rplm) { int pic_num = hpic->frame_num; - for (i = 0; i < pic->nb_refs; i++) { - href = pic->refs[i]->priv_data; + for (i = 0; i < pic->nb_refs[0]; i++) { + href = pic->refs[0][i]->priv_data; av_assert0(href->frame_num != pic_num); if (href->frame_num < pic_num) { sh->rplm_l0[i].modification_of_pic_nums_idc = 0; @@ -977,28 +979,29 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, } else { int need_rplm_l0 = 0, need_rplm_l1 = 0; int n0 = 0, n1 = 0; - for (i = 0; i < pic->nb_refs; i++) { - av_assert0(pic->refs[i]); - href = pic->refs[i]->priv_data; - av_assert0(href->pic_order_cnt != hpic->pic_order_cnt); - if (href->pic_order_cnt < hpic->pic_order_cnt) { - if (pic->refs[i] != def_l0[n0]) - need_rplm_l0 = 1; - ++n0; - } else { - if (pic->refs[i] != def_l1[n1]) - need_rplm_l1 = 1; - ++n1; - } + for (i = 0; i < pic->nb_refs[0]; i++) { + av_assert0(pic->refs[0][i]); + href = pic->refs[0][i]->priv_data; + av_assert0(href->pic_order_cnt < hpic->pic_order_cnt); + if (pic->refs[0][i] != def_l0[n0]) + need_rplm_l0 = 1; + ++n0; + } + + for (i = 0; i < pic->nb_refs[1]; i++) { + av_assert0(pic->refs[1][i]); + href = pic->refs[1][i]->priv_data; + av_assert0(href->pic_order_cnt > hpic->pic_order_cnt); + if (pic->refs[1][i] != def_l1[n1]) + need_rplm_l1 = 1; + ++n1; } sh->ref_pic_list_modification_flag_l0 = need_rplm_l0; if (need_rplm_l0) { int pic_num = hpic->frame_num; - for (i = j = 0; i < pic->nb_refs; i++) { - href = pic->refs[i]->priv_data; - if (href->pic_order_cnt > hpic->pic_order_cnt) - continue; + for (i = j = 0; i < pic->nb_refs[0]; i++) { + href = pic->refs[0][i]->priv_data; av_assert0(href->frame_num != pic_num); if (href->frame_num < pic_num) { sh->rplm_l0[j].modification_of_pic_nums_idc = 0; @@ -1019,10 +1022,8 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, sh->ref_pic_list_modification_flag_l1 = need_rplm_l1; if (need_rplm_l1) { int pic_num = hpic->frame_num; - for (i = j = 0; i < pic->nb_refs; i++) { - href = pic->refs[i]->priv_data; - if (href->pic_order_cnt < hpic->pic_order_cnt) - continue; + for (i = j = 0; i < pic->nb_refs[1]; i++) { + href = pic->refs[1][i]->priv_data; av_assert0(href->frame_num != pic_num); if (href->frame_num < pic_num) { sh->rplm_l1[j].modification_of_pic_nums_idc = 0; @@ -1062,14 +1063,13 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, vslice->RefPicList1[i].flags = VA_PICTURE_H264_INVALID; } - av_assert0(pic->nb_refs <= 2); - if (pic->nb_refs >= 1) { + if (pic->nb_refs[0]) { // Backward reference for P- or B-frame. av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); vslice->RefPicList0[0] = vpic->ReferenceFrames[0]; } - if (pic->nb_refs >= 2) { + if (pic->nb_refs[1]) { // Forward reference for B-frame. av_assert0(pic->type == PICTURE_TYPE_B); vslice->RefPicList1[0] = vpic->ReferenceFrames[1]; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index efa59aecf58c..239ef2359a08 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -764,7 +764,7 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, VAAPIEncodePicture *prev = pic->prev; VAAPIEncodeH265Picture *hprev = prev ? prev->priv_data : NULL; VAEncPictureParameterBufferHEVC *vpic = pic->codec_picture_params; - int i; + int i, j = 0; if (pic->type == PICTURE_TYPE_IDR) { av_assert0(pic->display_order == pic->encode_order); @@ -789,8 +789,8 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, hpic->pic_type = 1; } else { VAAPIEncodePicture *irap_ref; - av_assert0(pic->refs[0] && pic->refs[1]); - for (irap_ref = pic; irap_ref; irap_ref = irap_ref->refs[1]) { + av_assert0(pic->refs[0][0] && pic->refs[1][0]); + for (irap_ref = pic; irap_ref; irap_ref = irap_ref->refs[1][0]) { if (irap_ref->type == PICTURE_TYPE_I) break; } @@ -915,24 +915,27 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, .flags = 0, }; - for (i = 0; i < pic->nb_refs; i++) { - VAAPIEncodePicture *ref = pic->refs[i]; - VAAPIEncodeH265Picture *href; - - av_assert0(ref && ref->encode_order < pic->encode_order); - href = ref->priv_data; - - vpic->reference_frames[i] = (VAPictureHEVC) { - .picture_id = ref->recon_surface, - .pic_order_cnt = href->pic_order_cnt, - .flags = (ref->display_order < pic->display_order ? - VA_PICTURE_HEVC_RPS_ST_CURR_BEFORE : 0) | - (ref->display_order > pic->display_order ? - VA_PICTURE_HEVC_RPS_ST_CURR_AFTER : 0), - }; + for (int k = 0; k < MAX_REFERENCE_LIST_NUM; k++) { + for (i = 0; i < pic->nb_refs[k]; i++) { + VAAPIEncodePicture *ref = pic->refs[k][i]; + VAAPIEncodeH265Picture *href; + + av_assert0(ref && ref->encode_order < pic->encode_order); + href = ref->priv_data; + + vpic->reference_frames[j++] = (VAPictureHEVC) { + .picture_id = ref->recon_surface, + .pic_order_cnt = href->pic_order_cnt, + .flags = (ref->display_order < pic->display_order ? + VA_PICTURE_HEVC_RPS_ST_CURR_BEFORE : 0) | + (ref->display_order > pic->display_order ? + VA_PICTURE_HEVC_RPS_ST_CURR_AFTER : 0), + }; + } } - for (; i < FF_ARRAY_ELEMS(vpic->reference_frames); i++) { - vpic->reference_frames[i] = (VAPictureHEVC) { + + for (; j < FF_ARRAY_ELEMS(vpic->reference_frames); j++) { + vpic->reference_frames[j] = (VAPictureHEVC) { .picture_id = VA_INVALID_ID, .flags = VA_PICTURE_HEVC_INVALID, }; @@ -1016,21 +1019,33 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, memset(rps, 0, sizeof(*rps)); rps_pics = 0; - for (i = 0; i < pic->nb_refs; i++) { - strp = pic->refs[i]->priv_data; - rps_poc[rps_pics] = strp->pic_order_cnt; - rps_used[rps_pics] = 1; - ++rps_pics; + for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { + for (j = 0; j < pic->nb_refs[i]; j++) { + strp = pic->refs[i][j]->priv_data; + rps_poc[rps_pics] = strp->pic_order_cnt; + rps_used[rps_pics] = 1; + ++rps_pics; + } } + for (i = 0; i < pic->nb_dpb_pics; i++) { if (pic->dpb[i] == pic) continue; - for (j = 0; j < pic->nb_refs; j++) { - if (pic->dpb[i] == pic->refs[j]) + + for (j = 0; j < pic->nb_refs[0]; j++) { + if (pic->dpb[i] == pic->refs[0][j]) + break; + } + if (j < pic->nb_refs[0]) + continue; + + for (j = 0; j < pic->nb_refs[1]; j++) { + if (pic->dpb[i] == pic->refs[1][j]) break; } - if (j < pic->nb_refs) + if (j < pic->nb_refs[1]) continue; + strp = pic->dpb[i]->priv_data; rps_poc[rps_pics] = strp->pic_order_cnt; rps_used[rps_pics] = 0; @@ -1155,8 +1170,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, vslice->ref_pic_list1[i].flags = VA_PICTURE_HEVC_INVALID; } - av_assert0(pic->nb_refs <= 2); - if (pic->nb_refs >= 1) { + if (pic->nb_refs[0]) { // Backward reference for P- or B-frame. av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); @@ -1165,7 +1179,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, // Reference for GPB B-frame, L0 == L1 vslice->ref_pic_list1[0] = vpic->reference_frames[0]; } - if (pic->nb_refs >= 2) { + if (pic->nb_refs[1]) { // Forward reference for B-frame. av_assert0(pic->type == PICTURE_TYPE_B); vslice->ref_pic_list1[0] = vpic->reference_frames[1]; diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index 2edb0c35864f..d1904bf4f5a3 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -458,12 +458,12 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, break; case PICTURE_TYPE_P: vpic->picture_type = VAEncPictureTypePredictive; - vpic->forward_reference_picture = pic->refs[0]->recon_surface; + vpic->forward_reference_picture = pic->refs[0][0]->recon_surface; break; case PICTURE_TYPE_B: vpic->picture_type = VAEncPictureTypeBidirectional; - vpic->forward_reference_picture = pic->refs[0]->recon_surface; - vpic->backward_reference_picture = pic->refs[1]->recon_surface; + vpic->forward_reference_picture = pic->refs[0][0]->recon_surface; + vpic->backward_reference_picture = pic->refs[1][0]->recon_surface; break; default: av_assert0(0 && "invalid picture type"); diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index ea8abb2418ab..8a557b967e60 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -86,7 +86,7 @@ static int vaapi_encode_vp8_init_picture_params(AVCodecContext *avctx, switch (pic->type) { case PICTURE_TYPE_IDR: case PICTURE_TYPE_I: - av_assert0(pic->nb_refs == 0); + av_assert0(pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0); vpic->ref_flags.bits.force_kf = 1; vpic->ref_last_frame = vpic->ref_gf_frame = @@ -94,14 +94,14 @@ static int vaapi_encode_vp8_init_picture_params(AVCodecContext *avctx, VA_INVALID_SURFACE; break; case PICTURE_TYPE_P: - av_assert0(pic->nb_refs == 1); + av_assert0(!pic->nb_refs[1]); vpic->ref_flags.bits.no_ref_last = 0; vpic->ref_flags.bits.no_ref_gf = 1; vpic->ref_flags.bits.no_ref_arf = 1; vpic->ref_last_frame = vpic->ref_gf_frame = vpic->ref_arf_frame = - pic->refs[0]->recon_surface; + pic->refs[0][0]->recon_surface; break; default: av_assert0(0 && "invalid picture type"); diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 87429881f1cf..c2a8dec71b9d 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -96,15 +96,15 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, switch (pic->type) { case PICTURE_TYPE_IDR: - av_assert0(pic->nb_refs == 0); + av_assert0(pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0); vpic->ref_flags.bits.force_kf = 1; vpic->refresh_frame_flags = 0xff; hpic->slot = 0; break; case PICTURE_TYPE_P: - av_assert0(pic->nb_refs == 1); + av_assert0(!pic->nb_refs[1]); { - VAAPIEncodeVP9Picture *href = pic->refs[0]->priv_data; + VAAPIEncodeVP9Picture *href = pic->refs[0][0]->priv_data; av_assert0(href->slot == 0 || href->slot == 1); if (ctx->max_b_depth > 0) { @@ -120,10 +120,10 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, } break; case PICTURE_TYPE_B: - av_assert0(pic->nb_refs == 2); + av_assert0(pic->nb_refs[0] && pic->nb_refs[1]); { - VAAPIEncodeVP9Picture *href0 = pic->refs[0]->priv_data, - *href1 = pic->refs[1]->priv_data; + VAAPIEncodeVP9Picture *href0 = pic->refs[0][0]->priv_data, + *href1 = pic->refs[1][0]->priv_data; av_assert0(href0->slot < pic->b_depth + 1 && href1->slot < pic->b_depth + 1); @@ -157,12 +157,14 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, for (i = 0; i < FF_ARRAY_ELEMS(vpic->reference_frames); i++) vpic->reference_frames[i] = VA_INVALID_SURFACE; - for (i = 0; i < pic->nb_refs; i++) { - VAAPIEncodePicture *ref_pic = pic->refs[i]; - int slot; - slot = ((VAAPIEncodeVP9Picture*)ref_pic->priv_data)->slot; - av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); - vpic->reference_frames[slot] = ref_pic->recon_surface; + for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { + for (int j = 0; j < pic->nb_refs[i]; j++) { + VAAPIEncodePicture *ref_pic = pic->refs[i][j]; + int slot; + slot = ((VAAPIEncodeVP9Picture*)ref_pic->priv_data)->slot; + av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); + vpic->reference_frames[slot] = ref_pic->recon_surface; + } } vpic->pic_flags.bits.frame_type = (pic->type != PICTURE_TYPE_IDR); From 3be81e3b449febfb04596f3f187aae27f18d02f7 Mon Sep 17 00:00:00 2001 From: Fei Wang Date: Mon, 11 Sep 2023 15:52:31 +0800 Subject: [PATCH] lavc/vaapi_encode: Add VAAPI AV1 encoder Signed-off-by: Fei Wang Acked-by: Neal Gompa --- configure | 3 + doc/encoders.texi | 14 + libavcodec/Makefile | 2 + libavcodec/allcodecs.c | 1 + libavcodec/av1_levels.c | 92 ++++ libavcodec/av1_levels.h | 58 +++ libavcodec/vaapi_encode.c | 198 +++++-- libavcodec/vaapi_encode.h | 24 + libavcodec/vaapi_encode_av1.c | 949 ++++++++++++++++++++++++++++++++++ 11 files changed, 1311 insertions(+), 33 deletions(-) create mode 100644 libavcodec/av1_levels.c create mode 100644 libavcodec/av1_levels.h create mode 100644 libavcodec/vaapi_encode_av1.c diff --git a/configure b/configure index e40dcce09e4c..1ee840961721 100755 --- a/configure +++ b/configure @@ -3323,6 +3323,8 @@ av1_qsv_decoder_select="qsvdec" av1_qsv_encoder_select="qsvenc" av1_qsv_encoder_deps="libvpl" av1_amf_encoder_deps="amf" +av1_vaapi_encoder_deps="VAEncPictureParameterBufferAV1" +av1_vaapi_encoder_select="cbs_av1 vaapi_encode" # parsers aac_parser_select="adts_header mpeg4audio" @@ -7108,6 +7110,7 @@ if enabled vaapi; then check_type "va/va.h va/va_enc_jpeg.h" "VAEncPictureParameterBufferJPEG" check_type "va/va.h va/va_enc_vp8.h" "VAEncPictureParameterBufferVP8" check_type "va/va.h va/va_enc_vp9.h" "VAEncPictureParameterBufferVP9" + check_type "va/va.h va/va_enc_av1.h" "VAEncPictureParameterBufferAV1" fi if enabled_all opencl libdrm ; then diff --git a/doc/encoders.texi b/doc/encoders.texi index 6f8f5e127e87..d7d9584a0cae 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3995,6 +3995,20 @@ Average variable bitrate. Each encoder also has its own specific options: @table @option +@item av1_vaapi +@option{profile} sets the value of @emph{seq_profile}. +@option{tier} sets the value of @emph{seq_tier}. +@option{level} sets the value of @emph{seq_level_idx}. + +@table @option +@item tiles +Set the number of tiles to encode the input video with, as columns x rows. +(default is auto, which means use minimal tile column/row number). +@item tile_groups +Set tile groups number. All the tiles will be distributed as evenly as possible to +each tile group. (default is 1). +@end table + @item h264_vaapi @option{profile} sets the value of @emph{profile_idc} and the @emph{constraint_set*_flag}s. @option{level} sets the value of @emph{level_idc}. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index bf3b0a93f975..cae2e773a158 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -258,6 +258,7 @@ OBJS-$(CONFIG_AV1_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_AV1_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_AV1_NVENC_ENCODER) += nvenc_av1.o nvenc.o OBJS-$(CONFIG_AV1_QSV_ENCODER) += qsvenc_av1.o +OBJS-$(CONFIG_AV1_VAAPI_ENCODER) += vaapi_encode_av1.o av1_levels.o OBJS-$(CONFIG_AVRN_DECODER) += avrndec.o OBJS-$(CONFIG_AVRP_DECODER) += r210dec.o OBJS-$(CONFIG_AVRP_ENCODER) += r210enc.o @@ -1322,6 +1323,7 @@ TESTPROGS = avcodec \ jpeg2000dwt \ mathops \ +TESTPROGS-$(CONFIG_AV1_VAAPI_ENCODER) += av1_levels TESTPROGS-$(CONFIG_CABAC) += cabac TESTPROGS-$(CONFIG_DCT) += avfft TESTPROGS-$(CONFIG_FFT) += fft fft-fixed32 diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 6e95ca563691..5136a566f1b5 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -845,6 +845,7 @@ extern const FFCodec ff_av1_nvenc_encoder; extern const FFCodec ff_av1_qsv_decoder; extern const FFCodec ff_av1_qsv_encoder; extern const FFCodec ff_av1_amf_encoder; +extern const FFCodec ff_av1_vaapi_encoder; extern const FFCodec ff_libopenh264_encoder; extern const FFCodec ff_libopenh264_decoder; extern const FFCodec ff_h264_amf_encoder; diff --git a/libavcodec/av1_levels.c b/libavcodec/av1_levels.c new file mode 100644 index 000000000000..19b6ee173625 --- /dev/null +++ b/libavcodec/av1_levels.c @@ -0,0 +1,92 @@ +/* + * Copyright (c) 2023 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "libavutil/macros.h" +#include "av1_levels.h" + +/** ignore entries which named in spec but no details. Like level 2.2 and 7.0. */ +static const AV1LevelDescriptor av1_levels[] = { + // Name MaxVSize MainMbps MaxTiles + // | level_idx | MaxDisplayRate | HighMbps | MaxTileCols + // | | MaxPicSize | | MaxDecodeRate | | MainCR | | + // | | | MaxHSize | | | MaxHeaderRate | | | HighCR| | + // | | | | | | | | | | | | | | + { "2.0", 0, 147456, 2048, 1152, 4423680, 5529600, 150, 1.5, 0, 2, 0, 8, 4 }, + { "2.1", 1, 278784, 2816, 1584, 8363520, 10454400, 150, 3.0, 0, 2, 0, 8, 4 }, + { "3.0", 4, 665856, 4352, 2448, 19975680, 24969600, 150, 6.0, 0, 2, 0, 16, 6 }, + { "3.1", 5, 1065024, 5504, 3096, 31950720, 39938400, 150, 10.0, 0, 2, 0, 16, 6 }, + { "4.0", 8, 2359296, 6144, 3456, 70778880, 77856768, 300, 12.0, 30.0, 4, 4, 32, 8 }, + { "4.1", 9, 2359296, 6144, 3456, 141557760, 155713536, 300, 20.0, 50.0, 4, 4, 32, 8 }, + { "5.0", 12, 8912896, 8192, 4352, 267386880, 273715200, 300, 30.0, 100.0, 6, 4, 64, 8 }, + { "5.1", 13, 8912896, 8192, 4352, 534773760, 547430400, 300, 40.0, 160.0, 8, 4, 64, 8 }, + { "5.2", 14, 8912896, 8192, 4352, 1069547520, 1094860800, 300, 60.0, 240.0, 8, 4, 64, 8 }, + { "5.3", 15, 8912896, 8192, 4352, 1069547520, 1176502272, 300, 60.0, 240.0, 8, 4, 64, 8 }, + { "6.0", 16, 35651584, 16384, 8704, 1069547520, 1176502272, 300, 60.0, 240.0, 8, 4, 128, 16 }, + { "6.1", 17, 35651584, 16384, 8704, 2139095040, 2189721600, 300, 100.0, 480.0, 8, 4, 128, 16 }, + { "6.2", 18, 35651584, 16384, 8704, 4278190080, 4379443200, 300, 160.0, 800.0, 8, 4, 128, 16 }, + { "6.3", 19, 35651584, 16384, 8704, 4278190080, 4706009088, 300, 160.0, 800.0, 8, 4, 128, 16 }, +}; + +const AV1LevelDescriptor *ff_av1_guess_level(int64_t bitrate, + int tier, + int width, + int height, + int tiles, + int tile_cols, + float fps) +{ + int pic_size; + uint64_t display_rate; + float max_br; + + pic_size = width * height; + display_rate = (uint64_t)pic_size * fps; + + for (int i = 0; i < FF_ARRAY_ELEMS(av1_levels); i++) { + const AV1LevelDescriptor *level = &av1_levels[i]; + // Limitation: decode rate, header rate, compress rate, etc. are not considered. + if (pic_size > level->max_pic_size) + continue; + if (width > level->max_h_size) + continue; + if (height > level->max_v_size) + continue; + if (display_rate > level->max_display_rate) + continue; + + if (tier) + max_br = level->high_mbps; + else + max_br = level->main_mbps; + if (!max_br) + continue; + if (bitrate > (int64_t)(1000000.0 * max_br)) + continue; + + if (tiles > level->max_tiles) + continue; + if (tile_cols > level->max_tile_cols) + continue; + return level; + } + + return NULL; +} diff --git a/libavcodec/av1_levels.h b/libavcodec/av1_levels.h new file mode 100644 index 000000000000..164cb876ba10 --- /dev/null +++ b/libavcodec/av1_levels.h @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2023 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_AV1_LEVELS_H +#define AVCODEC_AV1_LEVELS_H + +#include + +typedef struct AV1LevelDescriptor { + char name[4]; + uint8_t level_idx; + + uint32_t max_pic_size; + uint32_t max_h_size; + uint32_t max_v_size; + uint64_t max_display_rate; + uint64_t max_decode_rate; + + uint32_t max_header_rate; + float main_mbps; + float high_mbps; + uint32_t main_cr; + uint32_t high_cr; + uint32_t max_tiles; + uint32_t max_tile_cols; +} AV1LevelDescriptor; + +/** + * Guess the level of a stream from some parameters. + * + * Unknown parameters may be zero, in which case they will be ignored. + */ +const AV1LevelDescriptor *ff_av1_guess_level(int64_t bitrate, + int tier, + int width, + int height, + int tile_rows, + int tile_cols, + float fps); + +#endif /* AVCODEC_AV1_LEVELS_H */ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 79036673e7a9..e3820956d160 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -683,6 +683,11 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, pic->opaque_ref = NULL; } + if (ctx->codec->flags & FLAG_TIMESTAMP_NO_DELAY) { + pkt->dts = pkt->pts; + return 0; + } + if (ctx->output_delay == 0) { pkt->dts = pkt->pts; } else if (pic->encode_order < ctx->decode_delay) { @@ -698,65 +703,160 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, return 0; } -static int vaapi_encode_output(AVCodecContext *avctx, - VAAPIEncodePicture *pic, AVPacket *pkt) +static int vaapi_encode_get_coded_buffer_size(AVCodecContext *avctx, VABufferID buf_id) { VAAPIEncodeContext *ctx = avctx->priv_data; VACodedBufferSegment *buf_list, *buf; + int size = 0; VAStatus vas; - int total_size = 0; - uint8_t *ptr; int err; - err = vaapi_encode_wait(avctx, pic); - if (err < 0) - return err; - - buf_list = NULL; - vas = vaMapBuffer(ctx->hwctx->display, pic->output_buffer, + vas = vaMapBuffer(ctx->hwctx->display, buf_id, (void**)&buf_list); if (vas != VA_STATUS_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to map output buffers: " "%d (%s).\n", vas, vaErrorStr(vas)); err = AVERROR(EIO); - goto fail; + return err; } for (buf = buf_list; buf; buf = buf->next) - total_size += buf->size; + size += buf->size; - err = ff_get_encode_buffer(avctx, pkt, total_size, 0); - ptr = pkt->data; + vas = vaUnmapBuffer(ctx->hwctx->display, buf_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to unmap output buffers: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + return err; + } - if (err < 0) - goto fail_mapped; + return size; +} + +static int vaapi_encode_get_coded_buffer_data(AVCodecContext *avctx, + VABufferID buf_id, uint8_t **dst) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VACodedBufferSegment *buf_list, *buf; + VAStatus vas; + int err; + + vas = vaMapBuffer(ctx->hwctx->display, buf_id, + (void**)&buf_list); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to map output buffers: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + return err; + } for (buf = buf_list; buf; buf = buf->next) { av_log(avctx, AV_LOG_DEBUG, "Output buffer: %u bytes " "(status %08x).\n", buf->size, buf->status); - memcpy(ptr, buf->buf, buf->size); - ptr += buf->size; + memcpy(*dst, buf->buf, buf->size); + *dst += buf->size; } - vas = vaUnmapBuffer(ctx->hwctx->display, pic->output_buffer); + vas = vaUnmapBuffer(ctx->hwctx->display, buf_id); if (vas != VA_STATUS_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to unmap output buffers: " "%d (%s).\n", vas, vaErrorStr(vas)); err = AVERROR(EIO); - goto fail; + return err; + } + + return 0; +} + +static int vaapi_encode_get_coded_data(AVCodecContext *avctx, + VAAPIEncodePicture *pic, AVPacket *pkt) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VABufferID output_buffer_prev; + int total_size = 0; + uint8_t *ptr; + int ret; + + if (ctx->coded_buffer_ref) { + output_buffer_prev = (VABufferID)(uintptr_t)ctx->coded_buffer_ref->data; + ret = vaapi_encode_get_coded_buffer_size(avctx, output_buffer_prev); + if (ret < 0) + goto end; + total_size += ret; } + ret = vaapi_encode_get_coded_buffer_size(avctx, pic->output_buffer); + if (ret < 0) + goto end; + total_size += ret; + + ret = ff_get_encode_buffer(avctx, pkt, total_size, 0); + if (ret < 0) + goto end; + ptr = pkt->data; + + if (ctx->coded_buffer_ref) { + ret = vaapi_encode_get_coded_buffer_data(avctx, output_buffer_prev, &ptr); + if (ret < 0) + goto end; + } + + ret = vaapi_encode_get_coded_buffer_data(avctx, pic->output_buffer, &ptr); + if (ret < 0) + goto end; + +end: + if (ctx->coded_buffer_ref) { + av_buffer_unref(&ctx->coded_buffer_ref); + } av_buffer_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; + return ret; +} + +static int vaapi_encode_output(AVCodecContext *avctx, + VAAPIEncodePicture *pic, AVPacket *pkt) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + AVPacket *pkt_ptr = pkt; + int err; + + err = vaapi_encode_wait(avctx, pic); + if (err < 0) + return err; + + if (pic->non_independent_frame) { + av_assert0(!ctx->coded_buffer_ref); + ctx->coded_buffer_ref = av_buffer_ref(pic->output_buffer_ref); + + if (pic->tail_size) { + if (ctx->tail_pkt->size) { + err = AVERROR(AVERROR_BUG); + goto end; + } + + err = ff_get_encode_buffer(avctx, ctx->tail_pkt, pic->tail_size, 0); + if (err < 0) + goto end; + + memcpy(ctx->tail_pkt->data, pic->tail_data, pic->tail_size); + pkt_ptr = ctx->tail_pkt; + } + } else { + err = vaapi_encode_get_coded_data(avctx, pic, pkt); + if (err < 0) + goto end; + } + av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", pic->display_order, pic->encode_order); - return 0; -fail_mapped: - vaUnmapBuffer(ctx->hwctx->display, pic->output_buffer); -fail: + vaapi_encode_set_output_property(avctx, pic, pkt_ptr); + +end: av_buffer_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; return err; @@ -1128,9 +1228,19 @@ static int vaapi_encode_pick_next(AVCodecContext *avctx, vaapi_encode_add_ref(avctx, pic, pic, 0, 1, 0); if (pic->type != PICTURE_TYPE_IDR) { - vaapi_encode_add_ref(avctx, pic, start, - pic->type == PICTURE_TYPE_P, - b_counter > 0, 0); + // TODO: apply both previous and forward multi reference for all vaapi encoders. + // And L0/L1 reference frame number can be set dynamically through query + // VAConfigAttribEncMaxRefFrames attribute. + if (avctx->codec_id == AV_CODEC_ID_AV1) { + for (i = 0; i < ctx->nb_next_prev; i++) + vaapi_encode_add_ref(avctx, pic, ctx->next_prev[i], + pic->type == PICTURE_TYPE_P, + b_counter > 0, 0); + } else + vaapi_encode_add_ref(avctx, pic, start, + pic->type == PICTURE_TYPE_P, + b_counter > 0, 0); + vaapi_encode_add_ref(avctx, pic, ctx->next_prev[ctx->nb_next_prev - 1], 0, 0, 1); } @@ -1292,6 +1402,19 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) AVFrame *frame = ctx->frame; int err; +start: + /** if no B frame before repeat P frame, sent repeat P frame out. */ + if (ctx->tail_pkt->size) { + for (VAAPIEncodePicture *tmp = ctx->pic_start; tmp; tmp = tmp->next) { + if (tmp->type == PICTURE_TYPE_B && tmp->pts < ctx->tail_pkt->pts) + break; + else if (!tmp->next) { + av_packet_move_ref(pkt, ctx->tail_pkt); + goto end; + } + } + } + err = ff_encode_get_frame(avctx, frame); if (err < 0 && err != AVERROR_EOF) return err; @@ -1356,17 +1479,21 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) return err; } - vaapi_encode_set_output_property(avctx, pic, pkt); - av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " - "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); - ctx->output_order = pic->encode_order; vaapi_encode_clear_old(avctx); + /** loop to get an available pkt in encoder flushing. */ + if (ctx->end_of_stream && !pkt->size) + goto start; + +end: + if (pkt->size) + av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " + "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); + return 0; } - static av_cold void vaapi_encode_add_global_param(AVCodecContext *avctx, int type, void *buffer, size_t size) { @@ -2667,6 +2794,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) ctx->device = (AVHWDeviceContext*)ctx->device_ref->data; ctx->hwctx = ctx->device->hwctx; + ctx->tail_pkt = av_packet_alloc(); + if (!ctx->tail_pkt) { + err = AVERROR(ENOMEM); + goto fail; + } + err = vaapi_encode_profile_entrypoint(avctx); if (err < 0) goto fail; @@ -2859,6 +2992,7 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) } av_frame_free(&ctx->frame); + av_packet_free(&ctx->tail_pkt); av_freep(&ctx->codec_sequence_params); av_freep(&ctx->codec_picture_params); diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 977bc2d94634..d0d6cc2adf46 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -133,6 +133,17 @@ typedef struct VAAPIEncodePicture { int nb_slices; VAAPIEncodeSlice *slices; + + /** + * indicate if current frame is an independent frame that the coded data + * can be pushed to downstream directly. Coded of non-independent frame + * data will be concatenated into next independent frame. + */ + int non_independent_frame; + /** Tail data of current pic, used only for repeat header of AV1. */ + char tail_data[MAX_PARAM_BUFFER_SIZE]; + /** Byte length of tail_data. */ + size_t tail_size; } VAAPIEncodePicture; typedef struct VAAPIEncodeProfile { @@ -367,6 +378,16 @@ typedef struct VAAPIEncodeContext { AVFifo *encode_fifo; // Max number of frame buffered in encoder. int async_depth; + + /** Head data for current output pkt, used only for AV1. */ + //void *header_data; + //size_t header_data_size; + + /** Buffered coded data of a pic if it is an non-independent frame. */ + AVBufferRef *coded_buffer_ref; + + /** Tail data of a pic, now only used for av1 repeat frame header. */ + AVPacket *tail_pkt; } VAAPIEncodeContext; enum { @@ -383,6 +404,9 @@ enum { // Codec supports non-IDR key pictures (that is, key pictures do // not necessarily empty the DPB). FLAG_NON_IDR_KEY_PICTURES = 1 << 5, + // Codec output packet without timestamp delay, which means the + // output packet has same PTS and DTS. + FLAG_TIMESTAMP_NO_DELAY = 1 << 6, }; typedef struct VAAPIEncodeType { diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c new file mode 100644 index 000000000000..3ff1c47b532d --- /dev/null +++ b/libavcodec/vaapi_encode_av1.c @@ -0,0 +1,949 @@ +/* + * Copyright (c) 2023 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" + +#include "cbs_av1.h" +#include "put_bits.h" +#include "codec_internal.h" +#include "av1_levels.h" +#include "vaapi_encode.h" + +#define AV1_MAX_QUANT 255 + +typedef struct VAAPIEncodeAV1Picture { + int64_t last_idr_frame; + int slot; +} VAAPIEncodeAV1Picture; + +typedef struct VAAPIEncodeAV1Context { + VAAPIEncodeContext common; + AV1RawOBU sh; /**< sequence header.*/ + AV1RawOBU fh; /**< frame header.*/ + CodedBitstreamContext *cbc; + CodedBitstreamFragment current_obu; + VAConfigAttribValEncAV1 attr; + VAConfigAttribValEncAV1Ext1 attr_ext1; + VAConfigAttribValEncAV1Ext2 attr_ext2; + + char sh_data[MAX_PARAM_BUFFER_SIZE]; /**< coded sequence header data. */ + size_t sh_data_len; /**< bit length of sh_data. */ + char fh_data[MAX_PARAM_BUFFER_SIZE]; /**< coded frame header data. */ + size_t fh_data_len; /**< bit length of fh_data. */ + + uint8_t uniform_tile; + uint8_t use_128x128_superblock; + int sb_cols; + int sb_rows; + int tile_cols_log2; + int tile_rows_log2; + int max_tile_width_sb; + int max_tile_height_sb; + uint8_t width_in_sbs_minus_1[AV1_MAX_TILE_COLS]; + uint8_t height_in_sbs_minus_1[AV1_MAX_TILE_ROWS]; + + int min_log2_tile_cols; + int max_log2_tile_cols; + int min_log2_tile_rows; + int max_log2_tile_rows; + + int q_idx_idr; + int q_idx_p; + int q_idx_b; + + /** bit positions in current frame header */ + int qindex_offset; + int loopfilter_offset; + int cdef_start_offset; + int cdef_param_size; + + /** user options */ + int profile; + int level; + int tier; + int tile_cols, tile_rows; + int tile_groups; +} VAAPIEncodeAV1Context; + +static void vaapi_encode_av1_trace_write_log(void *ctx, + PutBitContext *pbc, int length, + const char *str, const int *subscripts, + int64_t value) +{ + VAAPIEncodeAV1Context *priv = ctx; + int position; + + position = put_bits_count(pbc); + av_assert0(position >= length); + + if (!strcmp(str, "base_q_idx")) + priv->qindex_offset = position - length; + else if (!strcmp(str, "loop_filter_level[0]")) + priv->loopfilter_offset = position - length; + else if (!strcmp(str, "cdef_damping_minus_3")) + priv->cdef_start_offset = position - length; + else if (!strcmp(str, "cdef_uv_sec_strength[i]")) + priv->cdef_param_size = position - priv->cdef_start_offset; +} + +static av_cold int vaapi_encode_av1_get_encoder_caps(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; + + // Surfaces must be aligned to superblock boundaries. + ctx->surface_width = FFALIGN(avctx->width, priv->use_128x128_superblock ? 128 : 64); + ctx->surface_height = FFALIGN(avctx->height, priv->use_128x128_superblock ? 128 : 64); + + return 0; +} + +static av_cold int vaapi_encode_av1_configure(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; + int ret; + + ret = ff_cbs_init(&priv->cbc, AV_CODEC_ID_AV1, avctx); + if (ret < 0) + return ret; + priv->cbc->trace_enable = 1; + priv->cbc->trace_level = AV_LOG_DEBUG; + priv->cbc->trace_context = ctx; + priv->cbc->trace_write_callback = vaapi_encode_av1_trace_write_log; + + if (ctx->rc_mode->quality) { + priv->q_idx_p = av_clip(ctx->rc_quality, 0, AV1_MAX_QUANT); + if (fabs(avctx->i_quant_factor) > 0.0) + priv->q_idx_idr = + av_clip((fabs(avctx->i_quant_factor) * priv->q_idx_p + + avctx->i_quant_offset) + 0.5, + 0, AV1_MAX_QUANT); + else + priv->q_idx_idr = priv->q_idx_p; + + if (fabs(avctx->b_quant_factor) > 0.0) + priv->q_idx_b = + av_clip((fabs(avctx->b_quant_factor) * priv->q_idx_p + + avctx->b_quant_offset) + 0.5, + 0, AV1_MAX_QUANT); + else + priv->q_idx_b = priv->q_idx_p; + } else { + /** Arbitrary value */ + priv->q_idx_idr = priv->q_idx_p = priv->q_idx_b = 128; + } + + return 0; +} + +static int vaapi_encode_av1_add_obu(AVCodecContext *avctx, + CodedBitstreamFragment *au, + uint8_t type, + void *obu_unit) +{ + int ret; + + ret = ff_cbs_insert_unit_content(au, -1, + type, obu_unit, NULL); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to add OBU unit: " + "type = %d.\n", type); + return ret; + } + + return 0; +} + +static int vaapi_encode_av1_write_obu(AVCodecContext *avctx, + char *data, size_t *data_len, + CodedBitstreamFragment *bs) +{ + VAAPIEncodeAV1Context *priv = avctx->priv_data; + int ret; + + ret = ff_cbs_write_fragment_data(priv->cbc, bs); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write packed header.\n"); + return ret; + } + + if ((size_t)8 * MAX_PARAM_BUFFER_SIZE < 8 * bs->data_size - bs->data_bit_padding) { + av_log(avctx, AV_LOG_ERROR, "Access unit too large: " + "%zu < %zu.\n", (size_t)8 * MAX_PARAM_BUFFER_SIZE, + 8 * bs->data_size - bs->data_bit_padding); + return AVERROR(ENOSPC); + } + + memcpy(data, bs->data, bs->data_size); + *data_len = 8 * bs->data_size - bs->data_bit_padding; + + return 0; +} + +static int tile_log2(int blkSize, int target) { + int k; + for (k = 0; (blkSize << k) < target; k++); + return k; +} + +static int vaapi_encode_av1_set_tile(AVCodecContext *avctx) +{ + VAAPIEncodeAV1Context *priv = avctx->priv_data; + int mi_cols, mi_rows, sb_shift, sb_size; + int max_tile_area_sb, max_tile_area_sb_varied; + int tile_width_sb, tile_height_sb, widest_tile_sb; + int tile_cols, tile_rows; + int min_log2_tiles; + int i; + + if (priv->tile_cols > AV1_MAX_TILE_COLS || + priv->tile_rows > AV1_MAX_TILE_ROWS) { + av_log(avctx, AV_LOG_ERROR, "Invalid tile number %dx%d, should less than %dx%d.\n", + priv->tile_cols, priv->tile_rows, AV1_MAX_TILE_COLS, AV1_MAX_TILE_ROWS); + return AVERROR(EINVAL); + } + + mi_cols = 2 * ((avctx->width + 7) >> 3); + mi_rows = 2 * ((avctx->height + 7) >> 3); + priv->sb_cols = priv->use_128x128_superblock ? + ((mi_cols + 31) >> 5) : ((mi_cols + 15) >> 4); + priv->sb_rows = priv->use_128x128_superblock ? + ((mi_rows + 31) >> 5) : ((mi_rows + 15) >> 4); + sb_shift = priv->use_128x128_superblock ? 5 : 4; + sb_size = sb_shift + 2; + priv->max_tile_width_sb = AV1_MAX_TILE_WIDTH >> sb_size; + max_tile_area_sb = AV1_MAX_TILE_AREA >> (2 * sb_size); + + priv->min_log2_tile_cols = tile_log2(priv->max_tile_width_sb, priv->sb_cols); + priv->max_log2_tile_cols = tile_log2(1, FFMIN(priv->sb_cols, AV1_MAX_TILE_COLS)); + priv->max_log2_tile_rows = tile_log2(1, FFMIN(priv->sb_rows, AV1_MAX_TILE_ROWS)); + min_log2_tiles = FFMAX(priv->min_log2_tile_cols, + tile_log2(max_tile_area_sb, priv->sb_rows * priv->sb_cols)); + + tile_cols = av_clip(priv->tile_cols, (priv->sb_cols + priv->max_tile_width_sb - 1) / priv->max_tile_width_sb, priv->sb_cols); + + if (!priv->tile_cols) + priv->tile_cols = tile_cols; + else if (priv->tile_cols != tile_cols){ + av_log(avctx, AV_LOG_ERROR, "Invalid tile cols %d, should be in range of %d~%d\n", + priv->tile_cols, + (priv->sb_cols + priv->max_tile_width_sb - 1) / priv->max_tile_width_sb, + priv->sb_cols); + return AVERROR(EINVAL); + } + + priv->tile_cols_log2 = tile_log2(1, priv->tile_cols); + tile_width_sb = (priv->sb_cols + (1 << priv->tile_cols_log2) - 1) >> + priv->tile_cols_log2; + + if (priv->tile_rows > priv->sb_rows) { + av_log(avctx, AV_LOG_ERROR, "Invalid tile rows %d, should be less than %d.\n", + priv->tile_rows, priv->sb_rows); + return AVERROR(EINVAL); + } + + /** Try user setting tile rows number first. */ + tile_rows = priv->tile_rows ? priv->tile_rows : 1; + for (; tile_rows <= priv->sb_rows && tile_rows <= AV1_MAX_TILE_ROWS; tile_rows++) { + /** try uniformed tile. */ + priv->tile_rows_log2 = tile_log2(1, tile_rows); + if ((priv->sb_cols + tile_width_sb - 1) / tile_width_sb == priv->tile_cols) { + for (i = 0; i < priv->tile_cols - 1; i++) + priv->width_in_sbs_minus_1[i] = tile_width_sb - 1; + priv->width_in_sbs_minus_1[i] = priv->sb_cols - (priv->tile_cols - 1) * tile_width_sb - 1; + + tile_height_sb = (priv->sb_rows + (1 << priv->tile_rows_log2) - 1) >> + priv->tile_rows_log2; + + if ((priv->sb_rows + tile_height_sb - 1) / tile_height_sb == tile_rows && + tile_height_sb <= max_tile_area_sb / tile_width_sb) { + for (i = 0; i < tile_rows - 1; i++) + priv->height_in_sbs_minus_1[i] = tile_height_sb - 1; + priv->height_in_sbs_minus_1[i] = priv->sb_rows - (tile_rows - 1) * tile_height_sb - 1; + + priv->uniform_tile = 1; + priv->min_log2_tile_rows = FFMAX(min_log2_tiles - priv->tile_cols_log2, 0); + + break; + } + } + + /** try non-uniformed tile. */ + widest_tile_sb = 0; + for (i = 0; i < priv->tile_cols; i++) { + priv->width_in_sbs_minus_1[i] = (i + 1) * priv->sb_cols / priv->tile_cols - i * priv->sb_cols / priv->tile_cols - 1; + widest_tile_sb = FFMAX(widest_tile_sb, priv->width_in_sbs_minus_1[i] + 1); + } + + if (min_log2_tiles) + max_tile_area_sb_varied = (priv->sb_rows * priv->sb_cols) >> (min_log2_tiles + 1); + else + max_tile_area_sb_varied = priv->sb_rows * priv->sb_cols; + priv->max_tile_height_sb = FFMAX(1, max_tile_area_sb_varied / widest_tile_sb); + + if (tile_rows == av_clip(tile_rows, (priv->sb_rows + priv->max_tile_height_sb - 1) / priv->max_tile_height_sb, priv->sb_rows)) { + for (i = 0; i < tile_rows; i++) + priv->height_in_sbs_minus_1[i] = (i + 1) * priv->sb_rows / tile_rows - i * priv->sb_rows / tile_rows - 1; + + break; + } + + /** Return invalid parameter if explicit tile rows is set. */ + if (priv->tile_rows) { + av_log(avctx, AV_LOG_ERROR, "Invalid tile rows %d.\n", priv->tile_rows); + return AVERROR(EINVAL); + } + } + + priv->tile_rows = tile_rows; + av_log(avctx, AV_LOG_DEBUG, "Setting tile cols/rows to %d/%d.\n", + priv->tile_cols, priv->tile_rows); + + /** check if tile cols/rows is supported by driver. */ + if (priv->attr_ext2.bits.max_tile_num_minus1) { + if ((priv->tile_cols * priv->tile_rows - 1) > priv->attr_ext2.bits.max_tile_num_minus1) { + av_log(avctx, AV_LOG_ERROR, "Unsupported tile num %d * %d = %d by driver, " + "should be at most %d.\n", priv->tile_cols, priv->tile_rows, + priv->tile_cols * priv->tile_rows, + priv->attr_ext2.bits.max_tile_num_minus1 + 1); + return AVERROR(EINVAL); + } + } + + /** check if tile group numbers is valid. */ + if (priv->tile_groups > priv->tile_cols * priv->tile_rows) { + av_log(avctx, AV_LOG_WARNING, "Invalid tile groups number %d, " + "correct to %d.\n", priv->tile_groups, priv->tile_cols * priv->tile_rows); + priv->tile_groups = priv->tile_cols * priv->tile_rows; + } + + return 0; +} + +static int vaapi_encode_av1_write_sequence_header(AVCodecContext *avctx, + char *data, size_t *data_len) +{ + VAAPIEncodeAV1Context *priv = avctx->priv_data; + + memcpy(data, &priv->sh_data, MAX_PARAM_BUFFER_SIZE * sizeof(char)); + *data_len = priv->sh_data_len; + + return 0; +} + +static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; + AV1RawOBU *sh_obu = &priv->sh; + AV1RawSequenceHeader *sh = &sh_obu->obu.sequence_header; + VAEncSequenceParameterBufferAV1 *vseq = ctx->codec_sequence_params; + CodedBitstreamFragment *obu = &priv->current_obu; + const AVPixFmtDescriptor *desc; + int ret; + + memset(sh_obu, 0, sizeof(*sh_obu)); + sh_obu->header.obu_type = AV1_OBU_SEQUENCE_HEADER; + + desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + av_assert0(desc); + + sh->seq_profile = avctx->profile; + if (!sh->seq_force_screen_content_tools) + sh->seq_force_integer_mv = AV1_SELECT_INTEGER_MV; + sh->frame_width_bits_minus_1 = av_log2(avctx->width); + sh->frame_height_bits_minus_1 = av_log2(avctx->height); + sh->max_frame_width_minus_1 = avctx->width - 1; + sh->max_frame_height_minus_1 = avctx->height - 1; + sh->seq_tier[0] = priv->tier; + /** enable order hint and reserve maximum 8 bits for it by default. */ + sh->enable_order_hint = 1; + sh->order_hint_bits_minus_1 = 7; + + sh->color_config = (AV1RawColorConfig) { + .high_bitdepth = desc->comp[0].depth == 8 ? 0 : 1, + .color_primaries = avctx->color_primaries, + .transfer_characteristics = avctx->color_trc, + .matrix_coefficients = avctx->colorspace, + .color_description_present_flag = (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED || + avctx->color_trc != AVCOL_TRC_UNSPECIFIED || + avctx->colorspace != AVCOL_SPC_UNSPECIFIED), + .color_range = avctx->color_range == AVCOL_RANGE_JPEG, + .subsampling_x = desc->log2_chroma_w, + .subsampling_y = desc->log2_chroma_h, + }; + + switch (avctx->chroma_sample_location) { + case AVCHROMA_LOC_LEFT: + sh->color_config.chroma_sample_position = AV1_CSP_VERTICAL; + break; + case AVCHROMA_LOC_TOPLEFT: + sh->color_config.chroma_sample_position = AV1_CSP_COLOCATED; + break; + default: + sh->color_config.chroma_sample_position = AV1_CSP_UNKNOWN; + break; + } + + if (avctx->level != FF_PROFILE_UNKNOWN) { + sh->seq_level_idx[0] = avctx->level; + } else { + const AV1LevelDescriptor *level; + float framerate; + + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + framerate = avctx->framerate.num / avctx->framerate.den; + else + framerate = 0; + + level = ff_av1_guess_level(avctx->bit_rate, priv->tier, + ctx->surface_width, ctx->surface_height, + priv->tile_rows * priv->tile_cols, + priv->tile_cols, framerate); + if (level) { + av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); + sh->seq_level_idx[0] = level->level_idx; + } else { + av_log(avctx, AV_LOG_VERBOSE, "Stream will not conform to " + "any normal level, using maximum parameters level by default.\n"); + sh->seq_level_idx[0] = 31; + sh->seq_tier[0] = 1; + } + } + vseq->seq_profile = sh->seq_profile; + vseq->seq_level_idx = sh->seq_level_idx[0]; + vseq->seq_tier = sh->seq_tier[0]; + vseq->order_hint_bits_minus_1 = sh->order_hint_bits_minus_1; + vseq->intra_period = ctx->gop_size; + vseq->ip_period = ctx->b_per_p + 1; + + vseq->seq_fields.bits.enable_order_hint = sh->enable_order_hint; + + if (!(ctx->va_rc_mode & VA_RC_CQP)) { + vseq->bits_per_second = ctx->va_bit_rate; + vseq->seq_fields.bits.enable_cdef = sh->enable_cdef = 1; + } + + ret = vaapi_encode_av1_add_obu(avctx, obu, AV1_OBU_SEQUENCE_HEADER, &priv->sh); + if (ret < 0) + goto end; + + ret = vaapi_encode_av1_write_obu(avctx, priv->sh_data, &priv->sh_data_len, obu); + if (ret < 0) + goto end; + +end: + ff_cbs_fragment_reset(obu); + return ret; +} + +static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, + VAAPIEncodePicture *pic) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; + VAAPIEncodeAV1Picture *hpic = pic->priv_data; + AV1RawOBU *fh_obu = &priv->fh; + AV1RawFrameHeader *fh = &fh_obu->obu.frame.header; + VAEncPictureParameterBufferAV1 *vpic = pic->codec_picture_params; + CodedBitstreamFragment *obu = &priv->current_obu; + VAAPIEncodePicture *ref; + VAAPIEncodeAV1Picture *href; + int slot, i; + int ret; + static const int8_t default_loop_filter_ref_deltas[AV1_TOTAL_REFS_PER_FRAME] = + { 1, 0, 0, 0, -1, 0, -1, -1 }; + + memset(fh_obu, 0, sizeof(*fh_obu)); + pic->nb_slices = priv->tile_groups; + pic->non_independent_frame = pic->encode_order < pic->display_order; + fh_obu->header.obu_type = AV1_OBU_FRAME_HEADER; + fh_obu->header.obu_has_size_field = 1; + + switch (pic->type) { + case PICTURE_TYPE_IDR: + av_assert0(pic->nb_refs[0] == 0 || pic->nb_refs[1]); + fh->frame_type = AV1_FRAME_KEY; + fh->refresh_frame_flags = 0xFF; + fh->base_q_idx = priv->q_idx_idr; + hpic->slot = 0; + hpic->last_idr_frame = pic->display_order; + break; + case PICTURE_TYPE_P: + av_assert0(pic->nb_refs[0]); + fh->frame_type = AV1_FRAME_INTER; + fh->base_q_idx = priv->q_idx_p; + ref = pic->refs[0][pic->nb_refs[0] - 1]; + href = ref->priv_data; + hpic->slot = !href->slot; + hpic->last_idr_frame = href->last_idr_frame; + fh->refresh_frame_flags = 1 << hpic->slot; + + /** set the nearest frame in L0 as all reference frame. */ + for (i = 0; i < AV1_REFS_PER_FRAME; i++) { + fh->ref_frame_idx[i] = href->slot; + } + fh->primary_ref_frame = href->slot; + fh->ref_order_hint[href->slot] = ref->display_order - href->last_idr_frame; + vpic->ref_frame_ctrl_l0.fields.search_idx0 = AV1_REF_FRAME_LAST; + + /** set the 2nd nearest frame in L0 as Golden frame. */ + if (pic->nb_refs[0] > 1) { + ref = pic->refs[0][pic->nb_refs[0] - 2]; + href = ref->priv_data; + fh->ref_frame_idx[3] = href->slot; + fh->ref_order_hint[href->slot] = ref->display_order - href->last_idr_frame; + vpic->ref_frame_ctrl_l0.fields.search_idx1 = AV1_REF_FRAME_GOLDEN; + } + break; + case PICTURE_TYPE_B: + av_assert0(pic->nb_refs[0] && pic->nb_refs[1]); + fh->frame_type = AV1_FRAME_INTER; + fh->base_q_idx = priv->q_idx_b; + fh->refresh_frame_flags = 0x0; + fh->reference_select = 1; + + /** B frame will not be referenced, disable its recon frame. */ + vpic->picture_flags.bits.disable_frame_recon = 1; + + /** Use LAST_FRAME and BWDREF_FRAME for reference. */ + vpic->ref_frame_ctrl_l0.fields.search_idx0 = AV1_REF_FRAME_LAST; + vpic->ref_frame_ctrl_l1.fields.search_idx0 = AV1_REF_FRAME_BWDREF; + + ref = pic->refs[0][pic->nb_refs[0] - 1]; + href = ref->priv_data; + hpic->last_idr_frame = href->last_idr_frame; + fh->primary_ref_frame = href->slot; + fh->ref_order_hint[href->slot] = ref->display_order - href->last_idr_frame; + for (i = 0; i < AV1_REF_FRAME_GOLDEN; i++) { + fh->ref_frame_idx[i] = href->slot; + } + + ref = pic->refs[1][pic->nb_refs[1] - 1]; + href = ref->priv_data; + fh->ref_order_hint[href->slot] = ref->display_order - href->last_idr_frame; + for (i = AV1_REF_FRAME_GOLDEN; i < AV1_REFS_PER_FRAME; i++) { + fh->ref_frame_idx[i] = href->slot; + } + break; + default: + av_assert0(0 && "invalid picture type"); + } + + fh->show_frame = pic->display_order <= pic->encode_order; + fh->showable_frame = fh->frame_type != AV1_FRAME_KEY; + fh->frame_width_minus_1 = avctx->width - 1; + fh->frame_height_minus_1 = avctx->height - 1; + fh->render_width_minus_1 = fh->frame_width_minus_1; + fh->render_height_minus_1 = fh->frame_height_minus_1; + fh->order_hint = pic->display_order - hpic->last_idr_frame; + fh->tile_cols = priv->tile_cols; + fh->tile_rows = priv->tile_rows; + fh->tile_cols_log2 = priv->tile_cols_log2; + fh->tile_rows_log2 = priv->tile_rows_log2; + fh->uniform_tile_spacing_flag = priv->uniform_tile; + fh->tile_size_bytes_minus1 = priv->attr_ext2.bits.tile_size_bytes_minus1; + + /** ignore ONLY_4x4 mode for codedlossless is not fully implemented. */ + if (priv->attr_ext2.bits.tx_mode_support & 0x04) + fh->tx_mode = AV1_TX_MODE_SELECT; + else if (priv->attr_ext2.bits.tx_mode_support & 0x02) + fh->tx_mode = AV1_TX_MODE_LARGEST; + else { + av_log(avctx, AV_LOG_ERROR, "No available tx mode found.\n"); + return AVERROR(EINVAL); + } + + for (i = 0; i < fh->tile_cols; i++) + fh->width_in_sbs_minus_1[i] = vpic->width_in_sbs_minus_1[i] = priv->width_in_sbs_minus_1[i]; + + for (i = 0; i < fh->tile_rows; i++) + fh->height_in_sbs_minus_1[i] = vpic->height_in_sbs_minus_1[i] = priv->height_in_sbs_minus_1[i]; + + memcpy(fh->loop_filter_ref_deltas, default_loop_filter_ref_deltas, + AV1_TOTAL_REFS_PER_FRAME * sizeof(int8_t)); + + if (fh->frame_type == AV1_FRAME_KEY && fh->show_frame) { + fh->error_resilient_mode = 1; + } + + if (fh->frame_type == AV1_FRAME_KEY || fh->error_resilient_mode) + fh->primary_ref_frame = AV1_PRIMARY_REF_NONE; + + vpic->base_qindex = fh->base_q_idx; + vpic->frame_width_minus_1 = fh->frame_width_minus_1; + vpic->frame_height_minus_1 = fh->frame_height_minus_1; + vpic->primary_ref_frame = fh->primary_ref_frame; + vpic->reconstructed_frame = pic->recon_surface; + vpic->coded_buf = pic->output_buffer; + vpic->tile_cols = fh->tile_cols; + vpic->tile_rows = fh->tile_rows; + vpic->order_hint = fh->order_hint; +#if VA_CHECK_VERSION(1, 15, 0) + vpic->refresh_frame_flags = fh->refresh_frame_flags; +#endif + + vpic->picture_flags.bits.enable_frame_obu = 0; + vpic->picture_flags.bits.frame_type = fh->frame_type; + vpic->picture_flags.bits.reduced_tx_set = fh->reduced_tx_set; + vpic->picture_flags.bits.error_resilient_mode = fh->error_resilient_mode; + + /** let driver decide to use single or compound reference prediction mode. */ + vpic->mode_control_flags.bits.reference_mode = fh->reference_select ? 2 : 0; + vpic->mode_control_flags.bits.tx_mode = fh->tx_mode; + + vpic->tile_group_obu_hdr_info.bits.obu_has_size_field = 1; + + /** set reference. */ + for (i = 0; i < AV1_REFS_PER_FRAME; i++) + vpic->ref_frame_idx[i] = fh->ref_frame_idx[i]; + + for (i = 0; i < FF_ARRAY_ELEMS(vpic->reference_frames); i++) + vpic->reference_frames[i] = VA_INVALID_SURFACE; + + for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { + for (int j = 0; j < pic->nb_refs[i]; j++) { + VAAPIEncodePicture *ref_pic = pic->refs[i][j]; + + slot = ((VAAPIEncodeAV1Picture*)ref_pic->priv_data)->slot; + av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); + + vpic->reference_frames[slot] = ref_pic->recon_surface; + } + } + + fh_obu->obu_size_byte_len = priv->attr_ext2.bits.obu_size_bytes_minus1 + 1; + ret = vaapi_encode_av1_add_obu(avctx, obu, AV1_OBU_FRAME_HEADER, &priv->fh); + if (ret < 0) + goto end; + + ret = vaapi_encode_av1_write_obu(avctx, priv->fh_data, &priv->fh_data_len, obu); + if (ret < 0) + goto end; + + if (!(ctx->va_rc_mode & VA_RC_CQP)) { + vpic->min_base_qindex = av_clip(avctx->qmin, 1, AV1_MAX_QUANT); + vpic->max_base_qindex = av_clip(avctx->qmax, 1, AV1_MAX_QUANT); + + vpic->bit_offset_qindex = priv->qindex_offset; + vpic->bit_offset_loopfilter_params = priv->loopfilter_offset; + vpic->bit_offset_cdef_params = priv->cdef_start_offset; + vpic->size_in_bits_cdef_params = priv->cdef_param_size; + vpic->size_in_bits_frame_hdr_obu = priv->fh_data_len; + vpic->byte_offset_frame_hdr_obu_size = (((pic->type == PICTURE_TYPE_IDR) ? + priv->sh_data_len / 8 : 0) + + (fh_obu->header.obu_extension_flag ? + 2 : 1)); + } + +end: + ff_cbs_fragment_reset(obu); + return ret; +} + +static int vaapi_encode_av1_init_slice_params(AVCodecContext *avctx, + VAAPIEncodePicture *pic, + VAAPIEncodeSlice *slice) +{ + VAAPIEncodeAV1Context *priv = avctx->priv_data; + VAEncTileGroupBufferAV1 *vslice = slice->codec_slice_params; + CodedBitstreamAV1Context *cbctx = priv->cbc->priv_data; + int div; + + /** Set tile group info. */ + div = priv->tile_cols * priv->tile_rows / priv->tile_groups; + vslice->tg_start = slice->index * div; + if (slice->index == (priv->tile_groups - 1)) { + vslice->tg_end = priv->tile_cols * priv->tile_rows - 1; + cbctx->seen_frame_header = 0; + } else { + vslice->tg_end = (slice->index + 1) * div - 1; + } + + return 0; +} + +static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, + VAAPIEncodePicture *pic, + char *data, size_t *data_len) +{ + VAAPIEncodeAV1Context *priv = avctx->priv_data; + CodedBitstreamFragment *obu = &priv->current_obu; + CodedBitstreamAV1Context *cbctx = priv->cbc->priv_data; + AV1RawOBU *fh_obu = &priv->fh; + AV1RawFrameHeader *rep_fh = &fh_obu->obu.frame_header; + VAAPIEncodeAV1Picture *href; + int ret = 0; + + pic->tail_size = 0; + /** Pack repeat frame header. */ + if (pic->display_order > pic->encode_order) { + memset(fh_obu, 0, sizeof(*fh_obu)); + href = pic->refs[0][pic->nb_refs[0] - 1]->priv_data; + fh_obu->header.obu_type = AV1_OBU_FRAME_HEADER; + fh_obu->header.obu_has_size_field = 1; + + rep_fh->show_existing_frame = 1; + rep_fh->frame_to_show_map_idx = href->slot == 0; + rep_fh->frame_type = AV1_FRAME_INTER; + rep_fh->frame_width_minus_1 = avctx->width - 1; + rep_fh->frame_height_minus_1 = avctx->height - 1; + rep_fh->render_width_minus_1 = rep_fh->frame_width_minus_1; + rep_fh->render_height_minus_1 = rep_fh->frame_height_minus_1; + + cbctx->seen_frame_header = 0; + + ret = vaapi_encode_av1_add_obu(avctx, obu, AV1_OBU_FRAME_HEADER, &priv->fh); + if (ret < 0) + goto end; + + ret = vaapi_encode_av1_write_obu(avctx, pic->tail_data, &pic->tail_size, obu); + if (ret < 0) + goto end; + + pic->tail_size /= 8; + } + + memcpy(data, &priv->fh_data, MAX_PARAM_BUFFER_SIZE * sizeof(char)); + *data_len = priv->fh_data_len; + +end: + ff_cbs_fragment_reset(obu); + return ret; +} + +static const VAAPIEncodeProfile vaapi_encode_av1_profiles[] = { + { FF_PROFILE_AV1_MAIN, 8, 3, 1, 1, VAProfileAV1Profile0 }, + { FF_PROFILE_AV1_MAIN, 10, 3, 1, 1, VAProfileAV1Profile0 }, + { FF_PROFILE_UNKNOWN } +}; + +static const VAAPIEncodeType vaapi_encode_type_av1 = { + .profiles = vaapi_encode_av1_profiles, + .flags = FLAG_B_PICTURES | FLAG_TIMESTAMP_NO_DELAY, + .default_quality = 25, + + .get_encoder_caps = &vaapi_encode_av1_get_encoder_caps, + .configure = &vaapi_encode_av1_configure, + + .sequence_header_type = VAEncPackedHeaderSequence, + .sequence_params_size = sizeof(VAEncSequenceParameterBufferAV1), + .init_sequence_params = &vaapi_encode_av1_init_sequence_params, + .write_sequence_header = &vaapi_encode_av1_write_sequence_header, + + .picture_priv_data_size = sizeof(VAAPIEncodeAV1Picture), + .picture_header_type = VAEncPackedHeaderPicture, + .picture_params_size = sizeof(VAEncPictureParameterBufferAV1), + .init_picture_params = &vaapi_encode_av1_init_picture_params, + .write_picture_header = &vaapi_encode_av1_write_picture_header, + + .slice_params_size = sizeof(VAEncTileGroupBufferAV1), + .init_slice_params = &vaapi_encode_av1_init_slice_params, +}; + +static av_cold int vaapi_encode_av1_init(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; + VAConfigAttrib attr; + VAStatus vas; + int ret; + + ctx->codec = &vaapi_encode_type_av1; + + ctx->desired_packed_headers = + VA_ENC_PACKED_HEADER_SEQUENCE | + VA_ENC_PACKED_HEADER_PICTURE; + + if (avctx->profile == FF_PROFILE_UNKNOWN) + avctx->profile = priv->profile; + if (avctx->level == FF_PROFILE_UNKNOWN) + avctx->level = priv->level; + + if (avctx->level != FF_PROFILE_UNKNOWN && avctx->level & ~0x1f) { + av_log(avctx, AV_LOG_ERROR, "Invalid level %d\n", avctx->level); + return AVERROR(EINVAL); + } + + ret = ff_vaapi_encode_init(avctx); + if (ret < 0) + return ret; + + attr.type = VAConfigAttribEncAV1; + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } else if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + priv->attr.value = 0; + av_log(avctx, AV_LOG_WARNING, "Attribute type:%d is not " + "supported.\n", attr.type); + } else { + priv->attr.value = attr.value; + } + + attr.type = VAConfigAttribEncAV1Ext1; + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } else if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + priv->attr_ext1.value = 0; + av_log(avctx, AV_LOG_WARNING, "Attribute type:%d is not " + "supported.\n", attr.type); + } else { + priv->attr_ext1.value = attr.value; + } + + /** This attr provides essential indicators, return error if not support. */ + attr.type = VAConfigAttribEncAV1Ext2; + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS || attr.value == VA_ATTRIB_NOT_SUPPORTED) { + av_log(avctx, AV_LOG_ERROR, "Failed to query " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } else { + priv->attr_ext2.value = attr.value; + } + + ret = vaapi_encode_av1_set_tile(avctx); + if (ret < 0) + return ret; + + return 0; +} + +static av_cold int vaapi_encode_av1_close(AVCodecContext *avctx) +{ + VAAPIEncodeAV1Context *priv = avctx->priv_data; + + ff_cbs_fragment_free(&priv->current_obu); + ff_cbs_close(&priv->cbc); + + return ff_vaapi_encode_close(avctx); +} + +#define OFFSET(x) offsetof(VAAPIEncodeAV1Context, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) + +static const AVOption vaapi_encode_av1_options[] = { + VAAPI_ENCODE_COMMON_OPTIONS, + VAAPI_ENCODE_RC_OPTIONS, + { "profile", "Set profile (seq_profile)", + OFFSET(profile), AV_OPT_TYPE_INT, + { .i64 = FF_PROFILE_UNKNOWN }, FF_PROFILE_UNKNOWN, 0xff, FLAGS, "profile" }, + +#define PROFILE(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, "profile" + { PROFILE("main", FF_PROFILE_AV1_MAIN) }, + { PROFILE("high", FF_PROFILE_AV1_HIGH) }, + { PROFILE("professional", FF_PROFILE_AV1_PROFESSIONAL) }, +#undef PROFILE + + { "tier", "Set tier (seq_tier)", + OFFSET(tier), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS, "tier" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = 0 }, 0, 0, FLAGS, "tier" }, + { "high", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = 1 }, 0, 0, FLAGS, "tier" }, + { "level", "Set level (seq_level_idx)", + OFFSET(level), AV_OPT_TYPE_INT, + { .i64 = FF_PROFILE_UNKNOWN }, FF_PROFILE_UNKNOWN, 0x1f, FLAGS, "level" }, + +#define LEVEL(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, "level" + { LEVEL("2.0", 0) }, + { LEVEL("2.1", 1) }, + { LEVEL("3.0", 4) }, + { LEVEL("3.1", 5) }, + { LEVEL("4.0", 8) }, + { LEVEL("4.1", 9) }, + { LEVEL("5.0", 12) }, + { LEVEL("5.1", 13) }, + { LEVEL("5.2", 14) }, + { LEVEL("5.3", 15) }, + { LEVEL("6.0", 16) }, + { LEVEL("6.1", 17) }, + { LEVEL("6.2", 18) }, + { LEVEL("6.3", 19) }, +#undef LEVEL + + { "tiles", "Tile columns x rows (Use minimal tile column/row number automatically by default)", + OFFSET(tile_cols), AV_OPT_TYPE_IMAGE_SIZE, { .str = NULL }, 0, 0, FLAGS }, + { "tile_groups", "Number of tile groups for encoding", + OFFSET(tile_groups), AV_OPT_TYPE_INT, { .i64 = 1 }, 1, AV1_MAX_TILE_ROWS * AV1_MAX_TILE_COLS, FLAGS }, + + { NULL }, +}; + +static const FFCodecDefault vaapi_encode_av1_defaults[] = { + { "b", "0" }, + { "bf", "2" }, + { "g", "120" }, + { "qmin", "1" }, + { "qmax", "255" }, + { NULL }, +}; + +static const AVClass vaapi_encode_av1_class = { + .class_name = "av1_vaapi", + .item_name = av_default_item_name, + .option = vaapi_encode_av1_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_av1_vaapi_encoder = { + .p.name = "av1_vaapi", + CODEC_LONG_NAME("AV1 (VAAPI)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_AV1, + .priv_data_size = sizeof(VAAPIEncodeAV1Context), + .init = &vaapi_encode_av1_init, + FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + .close = &vaapi_encode_av1_close, + .p.priv_class = &vaapi_encode_av1_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | + AV_CODEC_CAP_DR1 | AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | + FF_CODEC_CAP_INIT_CLEANUP, + .defaults = vaapi_encode_av1_defaults, + .p.pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_VAAPI, + AV_PIX_FMT_NONE, + }, + .hw_configs = ff_vaapi_encode_hw_configs, + .p.wrapper_name = "vaapi", +}; ================================================ FILE: alvr/xtask/patches/0001-clip-constants-used-with-shift-instr.patch ================================================ From effadce6c756247ea8bae32dc13bb3e6f464f0eb Mon Sep 17 00:00:00 2001 From: =?utf8?q?R=C3=A9mi=20Denis-Courmont?= Date: Sun, 16 Jul 2023 18:18:02 +0300 Subject: [PATCH] avcodec/x86/mathops: clip constants used with shift instructions within inline assembly Fixes assembling with binutil as >= 2.41 Signed-off-by: James Almer --- libavcodec/x86/mathops.h | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/libavcodec/x86/mathops.h b/libavcodec/x86/mathops.h index 6298f5ed19..ca7e2dffc1 100644 --- a/libavcodec/x86/mathops.h +++ b/libavcodec/x86/mathops.h @@ -35,12 +35,20 @@ static av_always_inline av_const int MULL(int a, int b, unsigned shift) { int rt, dummy; + if (__builtin_constant_p(shift)) __asm__ ( "imull %3 \n\t" "shrdl %4, %%edx, %%eax \n\t" :"=a"(rt), "=d"(dummy) - :"a"(a), "rm"(b), "ci"((uint8_t)shift) + :"a"(a), "rm"(b), "i"(shift & 0x1F) ); + else + __asm__ ( + "imull %3 \n\t" + "shrdl %4, %%edx, %%eax \n\t" + :"=a"(rt), "=d"(dummy) + :"a"(a), "rm"(b), "c"((uint8_t)shift) + ); return rt; } @@ -113,19 +121,31 @@ __asm__ volatile(\ // avoid +32 for shift optimization (gcc should do that ...) #define NEG_SSR32 NEG_SSR32 static inline int32_t NEG_SSR32( int32_t a, int8_t s){ + if (__builtin_constant_p(s)) __asm__ ("sarl %1, %0\n\t" : "+r" (a) - : "ic" ((uint8_t)(-s)) + : "i" (-s & 0x1F) ); + else + __asm__ ("sarl %1, %0\n\t" + : "+r" (a) + : "c" ((uint8_t)(-s)) + ); return a; } #define NEG_USR32 NEG_USR32 static inline uint32_t NEG_USR32(uint32_t a, int8_t s){ + if (__builtin_constant_p(s)) __asm__ ("shrl %1, %0\n\t" : "+r" (a) - : "ic" ((uint8_t)(-s)) + : "i" (-s & 0x1F) ); + else + __asm__ ("shrl %1, %0\n\t" + : "+r" (a) + : "c" ((uint8_t)(-s)) + ); return a; } -- 2.30.2 ================================================ FILE: alvr/xtask/patches/0001-guid-conftest.patch ================================================ From 03823ac0c6a38bd6ba972539e3203a592579792f Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Thu, 1 Jun 2023 23:24:43 +0200 Subject: [PATCH] configure: use non-deprecated nvenc GUID for conftest --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure b/configure index 495493aa0e8a..ae56540f4eb0 100755 --- a/configure +++ b/configure @@ -7079,7 +7079,7 @@ enabled nvenc && test_cc -I$source_path < NV_ENCODE_API_FUNCTION_LIST flist; -void f(void) { struct { const GUID guid; } s[] = { { NV_ENC_PRESET_HQ_GUID } }; } +void f(void) { struct { const GUID guid; } s[] = { { NV_ENC_CODEC_H264_GUID } }; } int main(void) { return 0; } EOF ================================================ FILE: alvr/xtask/patches/0001-lavu-hwcontext_vulkan-Fix-importing-RGBx-frames-to-C.patch ================================================ From f867c4c56ee75d633db2300c0822bfa0020a056e Mon Sep 17 00:00:00 2001 From: David Rosca Date: Tue, 28 Nov 2023 14:04:20 +0100 Subject: [PATCH] lavu/hwcontext_vulkan: Fix importing RGBx frames to CUDA RGBx formats needs NumChannels = 4, but the old code would set it to 1. --- libavutil/hwcontext_vulkan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 204b57c011..e3bd6ace9b 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2859,7 +2859,7 @@ static int vulkan_export_to_cuda(AVHWFramesContext *hwfc, .arrayDesc = { .Depth = 0, .Format = cufmt, - .NumChannels = 1 + ((planes == 2) && i), + .NumChannels = desc->comp[i].step, .Flags = 0, }, .numLevels = 1, -- 2.43.0 ================================================ FILE: alvr/xtask/patches/0001-update-rc-modes.patch ================================================ From 1ebb0e43f9a15a12cd94db44e4bc5424f8a5b0c9 Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Thu, 1 Jun 2023 23:46:46 +0200 Subject: [PATCH] avcodec/nvenc: stop using deprecated rc modes with SDK 12.1 --- libavcodec/nvenc.c | 11 +++++++++++ libavcodec/nvenc.h | 5 +++++ libavcodec/nvenc_h264.c | 12 ++++++++++++ libavcodec/nvenc_hevc.c | 12 ++++++++++++ 4 files changed, 40 insertions(+) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index 15dd819a58f7..3c68ed39300f 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -43,9 +43,14 @@ #define CHECK_CU(x) FF_CUDA_CHECK_DL(avctx, dl_fn->cuda_dl, x) #define NVENC_CAP 0x30 + +#ifndef NVENC_NO_DEPRECATED_RC #define IS_CBR(rc) (rc == NV_ENC_PARAMS_RC_CBR || \ rc == NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ || \ rc == NV_ENC_PARAMS_RC_CBR_HQ) +#else +#define IS_CBR(rc) (rc == NV_ENC_PARAMS_RC_CBR) +#endif const enum AVPixelFormat ff_nvenc_pix_fmts[] = { AV_PIX_FMT_YUV420P, @@ -904,6 +909,7 @@ static void nvenc_override_rate_control(AVCodecContext *avctx) case NV_ENC_PARAMS_RC_CONSTQP: set_constqp(avctx); return; +#ifndef NVENC_NO_DEPRECATED_RC case NV_ENC_PARAMS_RC_VBR_MINQP: if (avctx->qmin < 0) { av_log(avctx, AV_LOG_WARNING, @@ -914,12 +920,15 @@ static void nvenc_override_rate_control(AVCodecContext *avctx) } /* fall through */ case NV_ENC_PARAMS_RC_VBR_HQ: +#endif case NV_ENC_PARAMS_RC_VBR: set_vbr(avctx); break; case NV_ENC_PARAMS_RC_CBR: +#ifndef NVENC_NO_DEPRECATED_RC case NV_ENC_PARAMS_RC_CBR_HQ: case NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ: +#endif break; } @@ -1193,12 +1202,14 @@ static av_cold int nvenc_setup_h264_config(AVCodecContext *avctx) h264->outputPictureTimingSEI = 1; +#ifndef NVENC_NO_DEPRECATED_RC if (cc->rcParams.rateControlMode == NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ || cc->rcParams.rateControlMode == NV_ENC_PARAMS_RC_CBR_HQ || cc->rcParams.rateControlMode == NV_ENC_PARAMS_RC_VBR_HQ) { h264->adaptiveTransformMode = NV_ENC_H264_ADAPTIVE_TRANSFORM_ENABLE; h264->fmoMode = NV_ENC_H264_FMO_DISABLE; } +#endif if (ctx->flags & NVENC_LOSSLESS) { h264->qpPrimeYZeroTransformBypassFlag = 1; diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h index 6cedd5bc2756..3a4b456a41dd 100644 --- a/libavcodec/nvenc.h +++ b/libavcodec/nvenc.h @@ -78,6 +78,11 @@ typedef void ID3D11Device; #define NVENC_HAVE_SINGLE_SLICE_INTRA_REFRESH #endif +// SDK 12.1 compile time feature checks +#if NVENCAPI_CHECK_VERSION(12, 1) +#define NVENC_NO_DEPRECATED_RC +#endif + typedef struct NvencSurface { NV_ENC_INPUT_PTR input_surface; diff --git a/libavcodec/nvenc_h264.c b/libavcodec/nvenc_h264.c index 5dc2961c3bcf..698615855bf5 100644 --- a/libavcodec/nvenc_h264.c +++ b/libavcodec/nvenc_h264.c @@ -100,6 +100,7 @@ static const AVOption options[] = { { "constqp", "Constant QP mode", 0, AV_OPT_TYPE_CONST, { .i64 = NV_ENC_PARAMS_RC_CONSTQP }, 0, 0, VE, "rc" }, { "vbr", "Variable bitrate mode", 0, AV_OPT_TYPE_CONST, { .i64 = NV_ENC_PARAMS_RC_VBR }, 0, 0, VE, "rc" }, { "cbr", "Constant bitrate mode", 0, AV_OPT_TYPE_CONST, { .i64 = NV_ENC_PARAMS_RC_CBR }, 0, 0, VE, "rc" }, +#ifndef NVENC_NO_DEPRECATED_RC { "vbr_minqp", "Variable bitrate mode with MinQP (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR_MINQP) }, 0, 0, VE, "rc" }, { "ll_2pass_quality", "Multi-pass optimized for image quality (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_2_PASS_QUALITY) }, 0, 0, VE, "rc" }, @@ -109,6 +110,17 @@ static const AVOption options[] = { { "cbr_ld_hq", "Constant bitrate low delay high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ) }, 0, 0, VE, "rc" }, { "cbr_hq", "Constant bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR_HQ) }, 0, 0, VE, "rc" }, { "vbr_hq", "Variable bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR_HQ) }, 0, 0, VE, "rc" }, +#else + { "vbr_minqp", "Variable bitrate mode with MinQP (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, + { "ll_2pass_quality", "Multi-pass optimized for image quality (deprecated)", + 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, + { "ll_2pass_size", "Multi-pass optimized for constant frame size (deprecated)", + 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR) }, 0, 0, VE, "rc" }, + { "vbr_2pass", "Multi-pass variable bitrate mode (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, + { "cbr_ld_hq", "Constant bitrate low delay high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR) }, 0, 0, VE, "rc" }, + { "cbr_hq", "Constant bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR) }, 0, 0, VE, "rc" }, + { "vbr_hq", "Variable bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, +#endif { "rc-lookahead", "Number of frames to look ahead for rate-control", OFFSET(rc_lookahead), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, VE }, { "surfaces", "Number of concurrent surfaces", OFFSET(nb_surfaces), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, MAX_REGISTERED_FRAMES, VE }, diff --git a/libavcodec/nvenc_hevc.c b/libavcodec/nvenc_hevc.c index 1362a927c8e4..d99077f17055 100644 --- a/libavcodec/nvenc_hevc.c +++ b/libavcodec/nvenc_hevc.c @@ -89,6 +89,7 @@ static const AVOption options[] = { { "constqp", "Constant QP mode", 0, AV_OPT_TYPE_CONST, { .i64 = NV_ENC_PARAMS_RC_CONSTQP }, 0, 0, VE, "rc" }, { "vbr", "Variable bitrate mode", 0, AV_OPT_TYPE_CONST, { .i64 = NV_ENC_PARAMS_RC_VBR }, 0, 0, VE, "rc" }, { "cbr", "Constant bitrate mode", 0, AV_OPT_TYPE_CONST, { .i64 = NV_ENC_PARAMS_RC_CBR }, 0, 0, VE, "rc" }, +#ifndef NVENC_NO_DEPRECATED_RC { "vbr_minqp", "Variable bitrate mode with MinQP (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR_MINQP) }, 0, 0, VE, "rc" }, { "ll_2pass_quality", "Multi-pass optimized for image quality (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_2_PASS_QUALITY) }, 0, 0, VE, "rc" }, @@ -98,6 +99,17 @@ static const AVOption options[] = { { "cbr_ld_hq", "Constant bitrate low delay high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ) }, 0, 0, VE, "rc" }, { "cbr_hq", "Constant bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR_HQ) }, 0, 0, VE, "rc" }, { "vbr_hq", "Variable bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR_HQ) }, 0, 0, VE, "rc" }, +#else + { "vbr_minqp", "Variable bitrate mode with MinQP (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, + { "ll_2pass_quality", "Multi-pass optimized for image quality (deprecated)", + 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, + { "ll_2pass_size", "Multi-pass optimized for constant frame size (deprecated)", + 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR) }, 0, 0, VE, "rc" }, + { "vbr_2pass", "Multi-pass variable bitrate mode (deprecated)", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, + { "cbr_ld_hq", "Constant bitrate low delay high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR) }, 0, 0, VE, "rc" }, + { "cbr_hq", "Constant bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_CBR) }, 0, 0, VE, "rc" }, + { "vbr_hq", "Variable bitrate high quality mode", 0, AV_OPT_TYPE_CONST, { .i64 = RCD(NV_ENC_PARAMS_RC_VBR) }, 0, 0, VE, "rc" }, +#endif { "rc-lookahead", "Number of frames to look ahead for rate-control", OFFSET(rc_lookahead), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, VE }, { "surfaces", "Number of concurrent surfaces", OFFSET(nb_surfaces), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, MAX_REGISTERED_FRAMES, VE }, ================================================ FILE: alvr/xtask/patches/0001-vaapi_encode-Add-filler_data-option.patch ================================================ From 920a052dbf4ffd92b25df68acbb2b36f5b51128d Mon Sep 17 00:00:00 2001 From: David Rosca Date: Sat, 5 Aug 2023 11:09:35 +0200 Subject: [PATCH] vaapi_encode: Add filler_data option --- libavcodec/vaapi_encode.c | 1 + libavcodec/vaapi_encode.h | 10 +++++++++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index bfca315a7a..895e12a6a8 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1861,6 +1861,7 @@ rc_mode_found: .quality_factor = rc_quality, #endif }; + ctx->rc_params.rc_flags.bits.disable_bit_stuffing = !ctx->filler_data; vaapi_encode_add_global_param(avctx, VAEncMiscParameterTypeRateControl, &ctx->rc_params, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index a1e639f56b..682b943475 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -198,6 +198,9 @@ typedef struct VAAPIEncodeContext { // Max Frame Size int max_frame_size; + // Filler Data + int filler_data; + // Explicitly set RC mode (otherwise attempt to pick from // available modes). int explicit_rc_mode; @@ -490,7 +493,12 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); { "max_frame_size", \ "Maximum frame size (in bytes)",\ OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ - { .i64 = 0 }, 0, INT_MAX, FLAGS } + { .i64 = 0 }, 0, INT_MAX, FLAGS }, \ + { "filler_data", \ + "Enable filler data", \ + OFFSET(common.filler_data), AV_OPT_TYPE_BOOL, \ + { .i64 = 1 }, 0, 1, FLAGS } + #define VAAPI_ENCODE_RC_MODE(name, desc) \ { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \ -- 2.41.0 ================================================ FILE: alvr/xtask/patches/0001-vaapi_encode-Allow-to-dynamically-change-bitrate-and.patch ================================================ From 34c0d23c4e16775811f229cb190e72ba7d1e45ae Mon Sep 17 00:00:00 2001 From: David Rosca Date: Fri, 17 Feb 2023 12:40:10 +0100 Subject: [PATCH] vaapi_encode: Allow to dynamically change bitrate and framerate --- libavcodec/vaapi_encode.c | 147 ++++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 1 + 2 files changed, 148 insertions(+) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 9a58661b51..4add6458fd 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -262,6 +262,130 @@ static int vaapi_encode_make_tile_slice(AVCodecContext *avctx, return 0; } +static av_cold int vaapi_encode_update_rate_control(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + int64_t rc_bits_per_second; + int rc_target_percentage; + int rc_window_size; + int64_t hrd_buffer_size; + int64_t hrd_initial_buffer_fullness; + int fr_num, fr_den; + unsigned int frame_rate; + + if (!ctx->rc_mode || !ctx->rc_mode->bitrate) { + return 0; + } + + if (ctx->rc_mode->mode == RC_MODE_AVBR) { + // For maximum confusion AVBR is hacked into the existing API + // by overloading some of the fields with completely different + // meanings. + + // Target percentage does not apply in AVBR mode. + rc_bits_per_second = avctx->bit_rate; + + // Accuracy tolerance range for meeting the specified target + // bitrate. It's very unclear how this is actually intended + // to work - since we do want to get the specified bitrate, + // set the accuracy to 100% for now. + rc_target_percentage = 100; + + // Convergence period in frames. The GOP size reflects the + // user's intended block size for cutting, so reusing that + // as the convergence period seems a reasonable default. + rc_window_size = avctx->gop_size > 0 ? avctx->gop_size : 60; + + } else if (ctx->rc_mode->maxrate) { + if (avctx->rc_max_rate > 0) { + if (avctx->rc_max_rate < avctx->bit_rate) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitrate settings: " + "bitrate (%"PRId64") must not be greater than " + "maxrate (%"PRId64").\n", avctx->bit_rate, + avctx->rc_max_rate); + return 0; + } + rc_bits_per_second = avctx->rc_max_rate; + rc_target_percentage = (avctx->bit_rate * 100) / + avctx->rc_max_rate; + } else { + // We only have a target bitrate, but this mode requires + // that a maximum rate be supplied as well. Since the + // user does not want this to be a constraint, arbitrarily + // pick a maximum rate of double the target rate. + rc_bits_per_second = 2 * avctx->bit_rate; + rc_target_percentage = 50; + } + } else { + if (avctx->rc_max_rate > avctx->bit_rate) { + av_log(avctx, AV_LOG_WARNING, "Max bitrate is ignored " + "in %s RC mode.\n", ctx->rc_mode->name); + } + rc_bits_per_second = avctx->bit_rate; + rc_target_percentage = 100; + } + + if (ctx->rc_mode->hrd) { + if (avctx->rc_buffer_size) + hrd_buffer_size = avctx->rc_buffer_size; + else if (avctx->rc_max_rate > 0) + hrd_buffer_size = avctx->rc_max_rate; + else + hrd_buffer_size = avctx->bit_rate; + if (avctx->rc_initial_buffer_occupancy) { + if (avctx->rc_initial_buffer_occupancy > hrd_buffer_size) { + av_log(avctx, AV_LOG_ERROR, "Invalid RC buffer settings: " + "must have initial buffer size (%d) <= " + "buffer size (%"PRId64").\n", + avctx->rc_initial_buffer_occupancy, hrd_buffer_size); + return 0; + } + hrd_initial_buffer_fullness = avctx->rc_initial_buffer_occupancy; + } else { + hrd_initial_buffer_fullness = hrd_buffer_size * 3 / 4; + } + + rc_window_size = (hrd_buffer_size * 1000) / rc_bits_per_second; + } else { + if (avctx->rc_buffer_size || avctx->rc_initial_buffer_occupancy) { + av_log(avctx, AV_LOG_WARNING, "Buffering settings are ignored " + "in %s RC mode.\n", ctx->rc_mode->name); + } + + hrd_buffer_size = 0; + hrd_initial_buffer_fullness = 0; + + if (ctx->rc_mode->mode != RC_MODE_AVBR) { + // Already set (with completely different meaning) for AVBR. + rc_window_size = 1000; + } + } + + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + av_reduce(&fr_num, &fr_den, + avctx->framerate.num, avctx->framerate.den, 65535); + else + av_reduce(&fr_num, &fr_den, + avctx->time_base.den, avctx->time_base.num, 65535); + + frame_rate = (unsigned int)fr_den << 16 | fr_num; + + if (ctx->va_bit_rate == rc_bits_per_second && ctx->va_frame_rate == frame_rate) { + return 0; + } + + ctx->va_bit_rate = rc_bits_per_second; + ctx->rc_params.bits_per_second = rc_bits_per_second; + ctx->rc_params.target_percentage = rc_target_percentage; + ctx->rc_params.window_size = rc_window_size; + ctx->hrd_params.initial_buffer_fullness = hrd_initial_buffer_fullness; + ctx->hrd_params.buffer_size = hrd_buffer_size; + ctx->va_frame_rate = frame_rate; + ctx->fr_params.framerate = frame_rate; + + return 1; +} + static int vaapi_encode_issue(AVCodecContext *avctx, VAAPIEncodePicture *pic) { @@ -340,6 +464,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } if (pic->type == PICTURE_TYPE_IDR) { + vaapi_encode_update_rate_control(avctx); for (i = 0; i < ctx->nb_global_params; i++) { err = vaapi_encode_make_misc_param_buffer(avctx, pic, ctx->global_params_type[i], @@ -348,6 +473,27 @@ static int vaapi_encode_issue(AVCodecContext *avctx, if (err < 0) goto fail; } + } else if (vaapi_encode_update_rate_control(avctx)) { + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeRateControl, + &ctx->rc_params, + sizeof(ctx->rc_params)); + if (err < 0) + goto fail; + + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeHRD, + &ctx->hrd_params, + sizeof(ctx->hrd_params)); + if (err < 0) + goto fail; + + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeFrameRate, + &ctx->fr_params, + sizeof(ctx->fr_params)); + if (err < 0) + goto fail; } if (ctx->codec->init_picture_params) { @@ -1875,6 +2021,7 @@ rc_mode_found: ctx->fr_params = (VAEncMiscParameterFrameRate) { .framerate = (unsigned int)fr_den << 16 | fr_num, }; + ctx->va_frame_rate = ctx->fr_params.framerate; #if VA_CHECK_VERSION(0, 40, 0) vaapi_encode_add_global_param(avctx, VAEncMiscParameterTypeFrameRate, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 359f954fff..e7b6482c39 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -235,6 +235,7 @@ typedef struct VAAPIEncodeContext { unsigned int va_rc_mode; // Bitrate for codec-specific encoder parameters. unsigned int va_bit_rate; + unsigned int va_frame_rate; // Packed headers which will actually be sent. unsigned int va_packed_headers; -- 2.39.2 ================================================ FILE: alvr/xtask/patches/0001-vaapi_encode-Enable-global-header.patch ================================================ From 03e6b5617c94bf6073a17a0a7054e70345b512ae Mon Sep 17 00:00:00 2001 From: David Rosca Date: Sat, 1 Jul 2023 10:35:16 +0200 Subject: [PATCH] vaapi_encode: Force enable global header --- libavcodec/vaapi_encode.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index bfca315a7a..ecce9bd721 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -2719,8 +2719,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) } } - if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && - ctx->codec->write_sequence_header && + if (ctx->codec->write_sequence_header && avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { char data[MAX_PARAM_BUFFER_SIZE]; size_t bit_len = 8 * sizeof(data); -- 2.41.0 ================================================ FILE: alvr/xtask/patches/0001-vaapi_encode_h265-Set-vui_parameters_present_flag.patch ================================================ From d96227806cab0de91c4db6b808b842c5b1477ebd Mon Sep 17 00:00:00 2001 From: David Rosca Date: Fri, 21 Jul 2023 21:55:56 +0200 Subject: [PATCH] vaapi_encode_h265: Set vui_parameters_present_flag --- libavcodec/vaapi_encode_h265.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index aa7e532f9a..91212dfb58 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -690,7 +690,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) sps->log2_min_pcm_luma_coding_block_size_minus3 + sps->log2_diff_max_min_pcm_luma_coding_block_size, - .vui_parameters_present_flag = 0, + .vui_parameters_present_flag = 1, }; *vpic = (VAEncPictureParameterBufferHEVC) { -- 2.41.0 ================================================ FILE: alvr/xtask/resources/alvr.desktop ================================================ [Desktop Entry] Version=1.0 Type=Application Name=ALVR GenericName=Game Comment=ALVR is an open source remote VR display which allows playing SteamVR games on a standalone headset such as Gear VR or Oculus Go/Quest. Exec=alvr_dashboard Icon=alvr Categories=Game; StartupNotify=true StartupWMClass=alvr.dashboard ================================================ FILE: alvr/xtask/resources/driver.vrdrivermanifest ================================================ { "alwaysActivate": true, "name" : "alvr_server", "redirectsDisplay": true, "hmd_presence" : [ "*.*" ] } ================================================ FILE: alvr/xtask/src/build.rs ================================================ use alvr_filesystem::{self as afs, Layout}; use std::{ env, fmt::{self, Display, Formatter}, fs, path::PathBuf, vec, }; use xshell::{Shell, cmd}; #[derive(Clone, Copy)] pub enum Profile { Debug, Release, Distribution, } impl Display for Profile { fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result { let string = match self { Profile::Distribution => "distribution", Profile::Release => "release", Profile::Debug => "debug", }; write!(f, "{string}") } } pub fn build_server_lib(profile: Profile, root: Option, reproducible: bool) { let sh = Shell::new().unwrap(); let mut flags = vec![]; match profile { Profile::Distribution => { flags.push("--profile"); flags.push("distribution"); } Profile::Release => flags.push("--release"), Profile::Debug => (), } if reproducible { flags.push("--locked"); } let flags_ref = &flags; let artifacts_dir = afs::target_dir().join(profile.to_string()); let build_dir = afs::build_dir().join("alvr_server_core"); sh.create_dir(&build_dir).unwrap(); if let Some(root) = root { sh.set_var("ALVR_ROOT_DIR", root); } let _push_guard = sh.push_dir(afs::crate_dir("server_core")); cmd!(sh, "cargo build {flags_ref...}").run().unwrap(); sh.copy_file( artifacts_dir.join(afs::dynlib_fname("alvr_server_core")), &build_dir, ) .unwrap(); if cfg!(windows) { sh.copy_file(artifacts_dir.join("alvr_server_core.pdb"), &build_dir) .unwrap(); } let out = build_dir.join("alvr_server_core.h"); cmd!(sh, "cbindgen --output {out}").run().unwrap(); } pub fn build_streamer( profile: Profile, gpl: bool, root: Option, reproducible: bool, profiling: bool, keep_config: bool, ) { let sh = Shell::new().unwrap(); let build_layout = Layout::new(&afs::streamer_build_dir()); let mut common_flags = vec![]; match profile { Profile::Distribution => { common_flags.push("--profile"); common_flags.push("distribution"); } Profile::Release => common_flags.push("--release"), Profile::Debug => (), } if reproducible { common_flags.push("--locked"); } let artifacts_dir = if cfg!(all(windows, target_arch = "aarch64")) { // Fix for cross compilation const TARGET: &str = "x86_64-pc-windows-msvc"; common_flags.push("--target"); common_flags.push(TARGET); afs::target_dir().join(TARGET).join(profile.to_string()) } else { afs::target_dir().join(profile.to_string()) }; let common_flags_ref = &common_flags; let maybe_config = if keep_config { fs::read_to_string(build_layout.session()).ok() } else { None }; sh.remove_path(afs::streamer_build_dir()).ok(); sh.create_dir(build_layout.openvr_driver_lib_dir()).unwrap(); sh.create_dir(&build_layout.executables_dir).unwrap(); if let Some(config) = maybe_config { fs::write(build_layout.session(), config).ok(); } if let Some(root) = root { sh.set_var("ALVR_ROOT_DIR", root); } // build server { let gpl_flag = if gpl { vec!["--features", "gpl"] } else { vec![] }; let profiling_flag = if profiling { vec!["--features", "alvr_server_core/trace-performance"] } else { vec![] }; let _push_guard = sh.push_dir(afs::crate_dir("server_openvr")); cmd!( sh, "cargo build {common_flags_ref...} {gpl_flag...} {profiling_flag...}" ) .run() .unwrap(); sh.copy_file( artifacts_dir.join(afs::dynlib_fname("alvr_server_openvr")), build_layout.openvr_driver_lib(), ) .unwrap(); if cfg!(windows) { sh.copy_file( artifacts_dir.join("alvr_server_openvr.pdb"), build_layout .openvr_driver_lib_dir() .join("alvr_server_openvr.pdb"), ) .unwrap(); } } // Build dashboard { let _push_guard = sh.push_dir(afs::crate_dir("dashboard")); cmd!(sh, "cargo build {common_flags_ref...}").run().unwrap(); sh.copy_file( artifacts_dir.join(afs::exec_fname("alvr_dashboard")), build_layout.dashboard_exe(), ) .unwrap(); } // copy dependencies if cfg!(windows) { sh.copy_file( afs::workspace_dir().join("openvr/bin/win64/openvr_api.dll"), build_layout.openvr_driver_lib_dir(), ) .unwrap(); // copy ffmpeg binaries if gpl { let bin_dir = &build_layout.openvr_driver_lib_dir(); sh.create_dir(bin_dir).unwrap(); for lib_path in sh .read_dir(afs::deps_dir().join("windows/ffmpeg/bin")) .unwrap() .into_iter() .filter(|path| path.file_name().unwrap().to_string_lossy().contains(".dll")) { sh.copy_file(lib_path.clone(), bin_dir).unwrap(); } } // copy libvpl.dll sh.copy_file( afs::deps_dir().join("windows/libvpl/alvr_build/bin/libvpl.dll"), build_layout.openvr_driver_lib_dir(), ) .unwrap(); } else if cfg!(target_os = "linux") { // build compositor wrapper let _push_guard = sh.push_dir(afs::crate_dir("vrcompositor_wrapper")); cmd!(sh, "cargo build {common_flags_ref...}").run().unwrap(); sh.create_dir(&build_layout.vrcompositor_wrapper_dir) .unwrap(); sh.copy_file( artifacts_dir.join("alvr_vrcompositor_wrapper"), build_layout.vrcompositor_wrapper(), ) .unwrap(); sh.copy_file( artifacts_dir.join("alvr_drm_lease_shim.so"), build_layout.drm_lease_shim(), ) .unwrap(); // build vulkan layer let _push_guard = sh.push_dir(afs::crate_dir("vulkan_layer")); cmd!(sh, "cargo build {common_flags_ref...}").run().unwrap(); sh.create_dir(&build_layout.libraries_dir).unwrap(); sh.copy_file( artifacts_dir.join(afs::dynlib_fname("alvr_vulkan_layer")), build_layout.vulkan_layer(), ) .unwrap(); // copy vulkan layer manifest sh.create_dir(&build_layout.vulkan_layer_manifest_dir) .unwrap(); sh.copy_file( afs::crate_dir("vulkan_layer").join("layer/alvr_x86_64.json"), build_layout.vulkan_layer_manifest(), ) .unwrap(); sh.copy_file( afs::workspace_dir().join("openvr/bin/linux64/libopenvr_api.so"), build_layout.openvr_driver_lib_dir(), ) .unwrap(); let firewall_script = afs::crate_dir("xtask").join("firewall/alvr_fw_config.sh"); let firewalld = afs::crate_dir("xtask").join("firewall/alvr-firewalld.xml"); let ufw = afs::crate_dir("xtask").join("firewall/ufw-alvr"); // copy linux specific firewalls sh.copy_file(firewall_script, build_layout.firewall_script()) .unwrap(); sh.copy_file(firewalld, build_layout.firewalld_config()) .unwrap(); sh.copy_file(ufw, build_layout.ufw_config()).unwrap(); } // copy static resources { // copy driver manifest sh.copy_file( afs::crate_dir("xtask").join("resources/driver.vrdrivermanifest"), build_layout.openvr_driver_manifest(), ) .unwrap(); } } pub fn build_launcher(profile: Profile, reproducible: bool) { let sh = Shell::new().unwrap(); let mut common_flags = vec![]; match profile { Profile::Distribution => { common_flags.push("--profile"); common_flags.push("distribution"); } Profile::Release => common_flags.push("--release"), Profile::Debug => (), } if reproducible { common_flags.push("--locked"); } let common_flags_ref = &common_flags; sh.create_dir(afs::launcher_build_dir()).unwrap(); cmd!(sh, "cargo build -p alvr_launcher {common_flags_ref...}") .run() .unwrap(); sh.copy_file( afs::target_dir() .join(profile.to_string()) .join(afs::exec_fname("alvr_launcher")), afs::launcher_build_exe_path(), ) .unwrap(); } fn build_android_lib_impl(dir_name: &str, profile: Profile, link_stdcpp: bool, all_targets: bool) { let sh = Shell::new().unwrap(); let mut ndk_flags = vec!["--no-strip", "-p", "28", "-t", "arm64-v8a"]; if all_targets { ndk_flags.extend(["-t", "armeabi-v7a", "-t", "x86_64", "-t", "x86"]); } let mut rust_flags = vec![]; match profile { Profile::Distribution => { rust_flags.push("--profile"); rust_flags.push("distribution") } Profile::Release => rust_flags.push("--release"), Profile::Debug => (), } if !link_stdcpp { rust_flags.push("--no-default-features"); } let rust_flags_ref = &rust_flags; let build_dir = afs::build_dir().join(format!("alvr_{dir_name}")); sh.create_dir(&build_dir).unwrap(); let _push_guard = sh.push_dir(afs::crate_dir(dir_name)); cmd!( sh, "cargo ndk {ndk_flags...} -o {build_dir} build {rust_flags_ref...}" ) .run() .unwrap(); let out = build_dir.join(format!("alvr_{dir_name}.h")); cmd!(sh, "cbindgen --output {out}").run().unwrap(); } pub fn build_android_client_core_lib(profile: Profile, link_stdcpp: bool, all_targets: bool) { build_android_lib_impl("client_core", profile, link_stdcpp, all_targets) } pub fn build_android_client_openxr_lib(profile: Profile, link_stdcpp: bool) { build_android_lib_impl("client_openxr", profile, link_stdcpp, false) } pub fn build_android_client(profile: Profile) { let sh = Shell::new().unwrap(); let mut flags = vec![]; match profile { Profile::Distribution => { flags.push("--profile"); flags.push("distribution") } Profile::Release => flags.push("--release"), Profile::Debug => (), } let flags_ref = &flags; const ARTIFACT_NAME: &str = "alvr_client_android"; let target_dir = afs::target_dir(); let build_dir = afs::build_dir().join(ARTIFACT_NAME); sh.create_dir(&build_dir).unwrap(); // Create debug keystore (signing will be overwritten by CI) if env::var(format!( "CARGO_APK_{}_KEYSTORE", profile.to_string().to_uppercase() )) .is_err() && matches!(profile, Profile::Release | Profile::Distribution) { let keystore_path = build_dir.join("debug.keystore"); if !keystore_path.exists() { let keytool = PathBuf::from(env::var("JAVA_HOME").expect("Env var JAVA_HOME not set")) .join("bin") .join(afs::exec_fname("keytool")); let pass = "alvrclient"; let other = vec![ "-genkey", "-v", "-alias", "androiddebugkey", "-dname", "CN=Android Debug,O=Android,C=US", "-keyalg", "RSA", "-keysize", "2048", "-validity", "10000", ]; cmd!( sh, "{keytool} -keystore {keystore_path} -storepass {pass} -keypass {pass} {other...}" ) .run() .unwrap(); } } let _push_guard = sh.push_dir(afs::crate_dir("client_openxr")); cmd!( sh, "cargo apk build --target-dir={target_dir} {flags_ref...}" ) .run() .unwrap(); sh.copy_file( afs::target_dir() .join(profile.to_string()) .join("apk/alvr_client_openxr.apk"), build_dir.join(format!("{ARTIFACT_NAME}.apk")), ) .unwrap(); } ================================================ FILE: alvr/xtask/src/ci.rs ================================================ use serde::Deserialize; use serde_json::{self as json, Deserializer, Value}; use xshell::{Shell, cmd}; #[derive(Debug, Deserialize)] #[serde(rename_all = "snake_case")] enum Level { Error, Warning, Note, Help, FailureNote, } #[derive(Debug, Deserialize)] struct Span { file_name: String, line_start: u64, line_end: u64, column_start: u64, column_end: u64, is_primary: bool, } // https://doc.rust-lang.org/rustc/json.html #[derive(Debug, Deserialize)] struct CompilerMessage { #[serde(rename = "$message_type")] message_type: String, level: Level, spans: Vec, rendered: String, } pub fn clippy_ci() { let sh = Shell::new().unwrap(); let out = cmd!(sh, "cargo clippy --message-format=json --color=always") .ignore_status() .output() .unwrap(); std::print!("{}", String::from_utf8_lossy(&out.stderr)); let stream = Deserializer::from_slice(&out.stdout).into_iter::(); // https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages let diagnostic_messages = stream .filter_map(|msg| { let msg = msg.unwrap(); if msg.get("reason")? == "compiler-message" { let msg: CompilerMessage = json::from_value(msg.get("message")?.clone()).ok()?; (msg.message_type == "diagnostic").then_some(msg) } else { None } }) .collect::>(); for msg in &diagnostic_messages { let level = match msg.level { Level::Error => Some("error"), Level::Warning => Some("warning"), Level::Note | Level::Help => Some("notice"), _ => None, }; if let Some(level) = level { let span = msg .spans .iter() .find(|&sp| sp.is_primary) .or(msg.spans.first()) .unwrap(); // may break when xtask gets cross-compiled, but that should not happen esp in ci let file_name = if cfg!(windows) { &span.file_name.replace('\\', "/") } else { &span.file_name }; // https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions println!( "::{level} file={},line={},endLine={},col={},endColumn={}::{}", file_name, span.line_start, span.line_end, span.column_start, span.column_end, msg.rendered .replace('%', "%25") .replace('\r', "%0D") .replace('\n', "%0A"), ); } } if !out.status.success() { panic!("ci clippy didn't exit with 0 code, propagating failure"); } if !diagnostic_messages.is_empty() { panic!("ci clippy produced warnings"); } } ================================================ FILE: alvr/xtask/src/command.rs ================================================ use std::path::Path; use xshell::{Shell, cmd}; pub fn zip(sh: &Shell, source: &Path) -> Result<(), xshell::Error> { let _push_guard = sh.push_dir(source); cmd!(sh, "zip -r9X {source} .").run() } pub fn unzip(sh: &Shell, source: &Path, destination: &Path) -> Result<(), xshell::Error> { cmd!(sh, "unzip {source} -d {destination}").run() } pub fn untar(sh: &Shell, source: &Path, destination: &Path) -> Result<(), xshell::Error> { cmd!(sh, "tar -xvf {source} -C {destination}").run() } pub fn targz(sh: &Shell, source: &Path) -> Result<(), xshell::Error> { let parent_dir = source.parent().unwrap(); let file_name = source.file_name().unwrap(); cmd!(sh, "tar -czvf {source}.tar.gz -C {parent_dir} {file_name}").run() } pub fn download(sh: &Shell, url: &str, destination: &Path) -> Result<(), xshell::Error> { cmd!(sh, "curl -L -o {destination} --url {url}").run() } pub fn download_and_extract_zip(url: &str, destination: &Path) -> Result<(), xshell::Error> { let sh = Shell::new().unwrap(); let temp_dir_guard = sh.create_temp_dir()?; let zip_file = temp_dir_guard.path().join("temp_download.zip"); download(&sh, url, &zip_file)?; unzip(&sh, &zip_file, destination) } pub fn download_and_extract_tar(url: &str, destination: &Path) -> Result<(), xshell::Error> { let sh = Shell::new().unwrap(); let temp_dir_guard = sh.create_temp_dir()?; let tar_file = temp_dir_guard.path().join("temp_download.tar"); download(&sh, url, &tar_file)?; untar(&sh, &tar_file, destination) } pub fn date_utc_yyyymmdd(sh: &Shell) -> Result { if cfg!(windows) { cmd!( sh, "powershell (Get-Date).ToUniversalTime().ToString(\"yyyy.MM.dd\")" ) .read() } else { cmd!(sh, "date -u +%Y.%m.%d").read() } } ================================================ FILE: alvr/xtask/src/dependencies.rs ================================================ use crate::{BuildPlatform, command}; use alvr_filesystem as afs; use std::{fs, path::Path}; use xshell::{Shell, cmd}; pub enum OpenXRLoadersSelection { OnlyGeneric, OnlyPico, All, } pub fn choco_install(sh: &Shell, packages: &[&str]) -> Result<(), xshell::Error> { cmd!( sh, "powershell Start-Process choco -ArgumentList \"install {packages...} -y\" -Verb runAs -Wait" ) .run() } pub fn prepare_x264_windows(deps_path: &Path) { let sh = Shell::new().unwrap(); const VERSION: &str = "0.164"; const REVISION: usize = 3086; let destination = deps_path.join("x264"); command::download_and_extract_zip( &format!( "{}/{VERSION}.r{REVISION}/libx264_{VERSION}.r{REVISION}_msvc16.zip", "https://github.com/ShiftMediaProject/x264/releases/download", ), &destination, ) .unwrap(); fs::write( afs::deps_dir().join("x264.pc"), format!( r#" prefix={} exec_prefix=${{prefix}}/bin/x64 libdir=${{prefix}}/lib/x64 includedir=${{prefix}}/include Name: x264 Description: x264 library Version: {VERSION} Libs: -L${{libdir}} -lx264 Cflags: -I${{includedir}} "#, destination.to_string_lossy().replace('\\', "/") ), ) .unwrap(); cmd!(sh, "setx PKG_CONFIG_PATH {deps_path}").run().unwrap(); } pub fn prepare_ffmpeg_windows(deps_path: &Path) { command::download_and_extract_zip( &format!( "https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/{}", "ffmpeg-n7.1-latest-win64-gpl-shared-7.1.zip" ), deps_path, ) .unwrap(); fs::rename( deps_path.join("ffmpeg-n7.1-latest-win64-gpl-shared-7.1"), deps_path.join("ffmpeg"), ) .unwrap(); } fn prepare_libvpl_windows(deps_path: &Path) { let sh = Shell::new().unwrap(); const VERSION: &str = "2.15.0"; command::download_and_extract_zip( &format!("https://github.com/intel/libvpl/archive/refs/tags/v{VERSION}.zip"), deps_path, ) .unwrap(); let final_path = deps_path.join("libvpl"); fs::rename(deps_path.join(format!("libvpl-{VERSION}")), &final_path).unwrap(); let install_prefix = final_path.join("alvr_build"); let _push_guard = sh.push_dir(final_path); cmd!( sh, "cmake -B build -DUSE_MSVC_STATIC_RUNTIME=ON -DCMAKE_INSTALL_PREFIX={install_prefix}" ) .run() .unwrap(); cmd!(sh, "cmake --build build --config Release") .run() .unwrap(); cmd!(sh, "cmake --install build --config Release") .run() .unwrap(); } pub fn prepare_windows_deps(skip_admin_priv: bool) { let sh = Shell::new().unwrap(); let deps_path = afs::deps_dir().join("windows"); sh.remove_path(&deps_path).ok(); sh.create_dir(&deps_path).unwrap(); if !skip_admin_priv { choco_install( &sh, &[ "zip", "unzip", "llvm", "vulkan-sdk", "pkgconfiglite", "cmake", ], ) .unwrap(); } prepare_x264_windows(&deps_path); prepare_ffmpeg_windows(&deps_path); prepare_libvpl_windows(&deps_path); } pub fn prepare_linux_deps(enable_nvenc: bool) { let sh = Shell::new().unwrap(); let deps_path = afs::deps_dir().join("linux"); sh.remove_path(&deps_path).ok(); sh.create_dir(&deps_path).unwrap(); build_x264_linux(&deps_path); build_ffmpeg_linux(enable_nvenc, &deps_path); } pub fn build_x264_linux(deps_path: &Path) { let sh = Shell::new().unwrap(); // x264 0.164 command::download_and_extract_tar( "https://code.videolan.org/videolan/x264/-/archive/c196240409e4d7c01b47448d93b1f9683aaa7cf7/x264-c196240409e4d7c01b47448d93b1f9683aaa7cf7.tar.bz2", deps_path, ) .unwrap(); let final_path = deps_path.join("x264"); fs::rename( deps_path.join("x264-c196240409e4d7c01b47448d93b1f9683aaa7cf7"), &final_path, ) .unwrap(); let flags = ["--enable-static", "--disable-cli", "--enable-pic"]; let install_prefix = format!("--prefix={}", final_path.join("alvr_build").display()); let _push_guard = sh.push_dir(final_path); cmd!(sh, "./configure {install_prefix} {flags...}") .run() .unwrap(); let nproc = cmd!(sh, "nproc").read().unwrap(); cmd!(sh, "make -j{nproc}").run().unwrap(); cmd!(sh, "make install").run().unwrap(); } pub fn build_ffmpeg_linux(enable_nvenc: bool, deps_path: &Path) { let sh = Shell::new().unwrap(); command::download_and_extract_zip( "https://codeload.github.com/FFmpeg/FFmpeg/zip/n6.0", deps_path, ) .unwrap(); let final_path = deps_path.join("ffmpeg"); fs::rename(deps_path.join("FFmpeg-n6.0"), &final_path).unwrap(); let flags = [ "--enable-gpl", "--enable-version3", "--enable-static", "--disable-programs", "--disable-doc", "--disable-avdevice", "--disable-avformat", "--disable-swresample", "--disable-swscale", "--disable-postproc", "--disable-network", "--disable-everything", "--enable-encoder=h264_vaapi", "--enable-encoder=hevc_vaapi", "--enable-encoder=av1_vaapi", "--enable-hwaccel=h264_vaapi", "--enable-hwaccel=hevc_vaapi", "--enable-hwaccel=av1_vaapi", "--enable-filter=scale_vaapi", "--enable-vulkan", "--enable-libdrm", "--enable-pic", "--enable-rpath", "--fatal-warnings", ]; let install_prefix = format!("--prefix={}", final_path.join("alvr_build").display()); // The reason for 4x$ in LDSOFLAGS var refer to https://stackoverflow.com/a/71429999 // all varients of --extra-ldsoflags='-Wl,-rpath,$ORIGIN' do not work! don't waste your time trying! // let config_vars = r#"-Wl,-rpath,'$$$$ORIGIN'"#; let _push_guard = sh.push_dir(final_path); let _env_vars = sh.push_env("LDSOFLAGS", config_vars); // Patches ffmpeg for workarounds and patches that have yet to be unstreamed let ffmpeg_command = "for p in ../../../alvr/xtask/patches/*; do patch -p1 < $p; done"; cmd!(sh, "bash -c {ffmpeg_command}").run().unwrap(); if enable_nvenc { #[cfg(target_os = "linux")] { let codec_header_version = "12.1.14.0"; let temp_download_dir = deps_path.join("dl_temp"); command::download_and_extract_zip( &format!("https://github.com/FFmpeg/nv-codec-headers/archive/refs/tags/n{codec_header_version}.zip"), &temp_download_dir ) .unwrap(); let header_dir = deps_path.join("nv-codec-headers"); let header_build_dir = header_dir.join("build"); fs::rename( temp_download_dir.join(format!("nv-codec-headers-n{codec_header_version}")), &header_dir, ) .unwrap(); fs::remove_dir_all(temp_download_dir).unwrap(); { let make_header_cmd = format!("make install PREFIX='{}'", header_build_dir.display()); let _header_push_guard = sh.push_dir(&header_dir); cmd!(sh, "bash -c {make_header_cmd}").run().unwrap(); } let nvenc_flags = &[ "--enable-encoder=h264_nvenc", "--enable-encoder=hevc_nvenc", "--enable-encoder=av1_nvenc", ]; let env_vars = format!( "PKG_CONFIG_PATH='{}'", header_build_dir.join("lib/pkgconfig").display() ); let flags_combined = flags.join(" "); let nvenc_flags_combined = nvenc_flags.join(" "); let command = format!( "{env_vars} ./configure {install_prefix} {flags_combined} {nvenc_flags_combined}" ); cmd!(sh, "bash -c {command}").run().unwrap(); } } else { cmd!(sh, "./configure {install_prefix} {flags...}") .run() .unwrap(); } let nproc = cmd!(sh, "nproc").read().unwrap(); cmd!(sh, "make -j{nproc}").run().unwrap(); cmd!(sh, "make install").run().unwrap(); } pub fn prepare_macos_deps() {} pub fn prepare_server_deps( platform: Option, skip_admin_priv: bool, enable_nvenc: bool, ) { match platform { Some(BuildPlatform::Windows) => prepare_windows_deps(skip_admin_priv), Some(BuildPlatform::Linux) => prepare_linux_deps(enable_nvenc), Some(BuildPlatform::Macos) => prepare_macos_deps(), Some(BuildPlatform::Android) => panic!("Android is not supported"), None => { if cfg!(windows) { prepare_windows_deps(skip_admin_priv); } else if cfg!(target_os = "linux") { prepare_linux_deps(enable_nvenc); } else if cfg!(target_os = "macos") { prepare_macos_deps(); } else { panic!("Unsupported platform"); } } } } fn get_android_openxr_loaders(selection: OpenXRLoadersSelection) { fn get_openxr_loader(name: &str, url: &str, source_dir: &str) { let sh = Shell::new().unwrap(); let temp_dir = afs::build_dir().join("temp_download"); sh.remove_path(&temp_dir).ok(); sh.create_dir(&temp_dir).unwrap(); let destination_dir = afs::deps_dir().join("android_openxr/arm64-v8a"); fs::create_dir_all(&destination_dir).unwrap(); command::download_and_extract_zip(url, &temp_dir).unwrap(); fs::copy( temp_dir.join(source_dir).join("libopenxr_loader.so"), destination_dir.join(format!("libopenxr_loader{name}.so")), ) .unwrap(); fs::remove_dir_all(&temp_dir).ok(); } const OPENXR_VERSION: &str = "1.1.36"; get_openxr_loader( "", &format!( "https://github.com/KhronosGroup/OpenXR-SDK-Source/releases/download/\ release-{OPENXR_VERSION}/openxr_loader_for_android-{OPENXR_VERSION}.aar", ), "prefab/modules/openxr_loader/libs/android.arm64-v8a", ); if matches!(selection, OpenXRLoadersSelection::OnlyGeneric) { return; } get_openxr_loader( "_pico_old", "https://sdk.picovr.com/developer-platform/sdk/PICO_OpenXR_SDK_220.zip", "libs/android.arm64-v8a", ); if matches!(selection, OpenXRLoadersSelection::OnlyPico) { return; } get_openxr_loader( "_quest1", "https://securecdn.oculus.com/binaries/download/?id=7577210995650755", // Version 64 "OpenXR/Libs/Android/arm64-v8a/Release", ); get_openxr_loader( "_yvr", "https://developer.yvrdream.com/yvrdoc/sdk/openxr/yvr_openxr_mobile_sdk_2.0.0.zip", "yvr_openxr_mobile_sdk_2.0.0/OpenXR/Libs/Android/arm64-v8a", ); get_openxr_loader( "_lynx", "https://portal.lynx-r.com/downloads/download/16", // version 1.0.0 "jni/arm64-v8a", ); } pub fn build_android_deps( skip_admin_priv: bool, all_targets: bool, openxr_loaders_selection: OpenXRLoadersSelection, ) { let sh = Shell::new().unwrap(); if cfg!(windows) && !skip_admin_priv { choco_install(&sh, &["unzip", "llvm"]).unwrap(); } cmd!(sh, "rustup target add aarch64-linux-android") .run() .unwrap(); if all_targets { cmd!(sh, "rustup target add armv7-linux-androideabi") .run() .unwrap(); cmd!(sh, "rustup target add x86_64-linux-android") .run() .unwrap(); cmd!(sh, "rustup target add i686-linux-android") .run() .unwrap(); } cmd!(sh, "cargo install cbindgen").run().unwrap(); cmd!(sh, "cargo install cargo-ndk --version 3.5.4") .run() .unwrap(); cmd!( sh, "cargo install --git https://github.com/zarik5/cargo-apk cargo-apk" ) .run() .unwrap(); get_android_openxr_loaders(openxr_loaders_selection); } ================================================ FILE: alvr/xtask/src/format.rs ================================================ use alvr_filesystem as afs; use std::fs; use std::mem; use std::path::PathBuf; use walkdir::WalkDir; use xshell::{Shell, cmd}; fn files_to_format_paths() -> Vec { let cpp_dir = afs::crate_dir("server_openvr").join("cpp"); WalkDir::new(cpp_dir) .into_iter() .filter_entry(|entry| { let included = entry.path().is_dir() || entry .path() .extension() .is_some_and(|ext| matches!(ext.to_str().unwrap(), "c" | "cpp" | "h" | "hpp")); let excluded = matches!( entry.file_name().to_str().unwrap(), "shared" | "include" | "NvCodecUtils.h" | "NvEncoder.cpp" | "NvEncoder.h" | "NvEncoderD3D11.cpp" | "NvEncoderD3D11.h" | "nvEncodeAPI.h" ); included && !excluded }) .filter_map(|entry| { let entry = entry.ok()?; entry.file_type().is_file().then(|| entry.path().to_owned()) }) .collect() } pub fn format() { let sh = Shell::new().unwrap(); let dir = sh.push_dir(afs::workspace_dir()); cmd!(sh, "cargo fmt --all").run().unwrap(); for path in files_to_format_paths() { cmd!(sh, "clang-format -i {path}").run().unwrap(); } mem::drop(dir); } pub fn check_format() { let sh = Shell::new().unwrap(); let dir = sh.push_dir(afs::workspace_dir()); cmd!(sh, "cargo fmt --all -- --check") .run() .expect("cargo fmt check failed"); for path in files_to_format_paths() { let content = fs::read_to_string(&path).unwrap(); let mut output = cmd!(sh, "clang-format {path}").read().unwrap(); if !content.ends_with('\n') { panic!("file {} missing final newline", path.display()); } output.push('\n'); if content != output { panic!("clang-format check failed for {}", path.display()); } } mem::drop(dir); } ================================================ FILE: alvr/xtask/src/main.rs ================================================ mod build; mod ci; mod command; mod dependencies; mod format; mod packaging; mod version; use crate::build::Profile; use afs::Layout; use alvr_filesystem as afs; use dependencies::OpenXRLoadersSelection; use packaging::ReleaseFlavor; use pico_args::Arguments; use std::{fs, process, time::Instant}; use xshell::{Shell, cmd}; const HELP_STR: &str = r#" cargo xtask Developement actions for ALVR. USAGE: cargo xtask [FLAG] [ARGS] SUBCOMMANDS: prepare-deps Download and compile streamer and client external dependencies build-streamer Build streamer, then copy binaries to build folder build-launcher Build launcher, then copy binaries to build folder build-server-lib Build a C-ABI ALVR server library and header build-client Build client, then copy binaries to build folder build-client-lib Build a C-ABI ALVR client library and header build-client-xr-lib Build a C-ABI ALVR OpenXR entry point client library and header run-streamer Build streamer and then open the dashboard run-launcher Build launcher and then open it format Autoformat all code check-format Check if code is correctly formatted package-streamer Build streamer with distribution profile, make archive package-launcher Build launcher with distribution profile, make archive package-client Build client with distribution profile package-client-lib Build client library then zip it clean Removes all build artifacts and dependencies bump Bump streamer and client package versions clippy Show warnings for selected clippy lints kill-oculus Kill all Oculus processes FLAGS: --help Print this text --keep-config Preserve the configuration file between rebuilds (session.json) --no-nvidia Disables nVidia support on Linux. For prepare-deps subcommand --release Optimized build with less debug checks. For build subcommands --profiling Enable Profiling --gpl Bundle GPL libraries (FFmpeg). Only for Windows --nightly Append nightly tag to versions. For bump subcommand --no-rebuild Do not rebuild the streamer with run-streamer --ci Do some CI related tweaks. Depends on the other flags and subcommand --no-stdcpp Disable linking to libc++_shared with build-client-lib --all-targets For prepare-deps and build-client-lib subcommand, will build for all android supported ABI targets --meta-store For package-client subcommand, build for Meta Store --pico-store For package-client subcommand, build for Pico Store ARGS: --platform Can be one of: windows, linux, macos, android. Can be omitted --version Specify version to set with the bump-versions subcommand --root Installation root. By default no root is set and paths are calculated using relative paths, which requires conforming to FHS on Linux "#; enum BuildPlatform { Windows, Linux, Macos, Android, } pub fn print_help_and_exit(message: &str) -> ! { eprintln!("\n{message}"); eprintln!("{HELP_STR}"); process::exit(1); } pub fn run_streamer() { let sh = Shell::new().unwrap(); let dashboard_exe = Layout::new(&afs::streamer_build_dir()).dashboard_exe(); cmd!(sh, "{dashboard_exe}").run().unwrap(); } pub fn run_launcher() { let sh = Shell::new().unwrap(); let launcher_exe = afs::launcher_build_exe_path(); cmd!(sh, "{launcher_exe}").run().unwrap(); } pub fn clean() { fs::remove_dir_all(afs::build_dir()).ok(); fs::remove_dir_all(afs::deps_dir()).ok(); if afs::target_dir() == afs::workspace_dir().join("target") { // Detete target folder only if in the local wokspace! fs::remove_dir_all(afs::target_dir()).ok(); } } fn clippy() { // lints updated for Rust 1.59 let restriction_lints = [ "allow_attributes_without_reason", "clone_on_ref_ptr", "create_dir", "decimal_literal_representation", "expect_used", "float_cmp_const", "fn_to_numeric_cast_any", "get_unwrap", "if_then_some_else_none", "let_underscore_must_use", "lossy_float_literal", "mem_forget", "multiple_inherent_impl", "rest_pat_in_fully_bound_structs", // "self_named_module_files", "str_to_string", // "string_slice", "string_to_string", "try_err", "unnecessary_self_imports", "unneeded_field_pattern", "unseparated_literal_suffix", "verbose_file_reads", "wildcard_enum_match_arm", ]; let pedantic_lints = [ "borrow_as_ptr", "enum_glob_use", "explicit_deref_methods", "explicit_into_iter_loop", "explicit_iter_loop", "filter_map_next", "flat_map_option", "float_cmp", // todo: add more lints ]; let flags = restriction_lints .into_iter() .chain(pedantic_lints) .flat_map(|name| ["-W".to_owned(), format!("clippy::{name}")]); let sh = Shell::new().unwrap(); cmd!(sh, "cargo clippy -- {flags...}").run().unwrap(); } // Avoid Oculus link popups when debugging the client pub fn kill_oculus_processes() { let sh = Shell::new().unwrap(); cmd!( sh, "powershell Start-Process taskkill -ArgumentList \"/F /IM OVR* /T\" -Verb runas" ) .run() .unwrap(); } fn main() { let begin_time = Instant::now(); let mut args = Arguments::from_env(); if args.contains(["-h", "--help"]) { println!("{HELP_STR}"); } else if let Ok(Some(subcommand)) = args.subcommand() { let no_nvidia = args.contains("--no-nvidia"); let is_release = args.contains("--release"); let profile = if is_release { Profile::Release } else { Profile::Debug }; let profiling = args.contains("--profiling"); let gpl = args.contains("--gpl"); let is_nightly = args.contains("--nightly"); let no_rebuild = args.contains("--no-rebuild"); let for_ci = args.contains("--ci"); let keep_config = args.contains("--keep-config"); let link_stdcpp = !args.contains("--no-stdcpp"); let all_targets = args.contains("--all-targets"); let platform: Option = args.opt_value_from_str("--platform").unwrap(); let platform = platform.as_deref().map(|platform| match platform { "windows" => BuildPlatform::Windows, "linux" => BuildPlatform::Linux, "macos" => BuildPlatform::Macos, "android" => BuildPlatform::Android, _ => print_help_and_exit("Unrecognized platform"), }); let version: Option = args.opt_value_from_str("--version").unwrap(); let root: Option = args.opt_value_from_str("--root").unwrap(); let package_flavor = if args.contains("--meta-store") { ReleaseFlavor::MetaStore } else if args.contains("--pico-store") { ReleaseFlavor::PicoStore } else { ReleaseFlavor::GitHub }; if args.finish().is_empty() { match subcommand.as_str() { "prepare-deps" => { if let Some(platform) = platform { if matches!(platform, BuildPlatform::Android) { dependencies::build_android_deps( for_ci, all_targets, OpenXRLoadersSelection::All, ); } else { dependencies::prepare_server_deps(Some(platform), for_ci, !no_nvidia); } } else { dependencies::prepare_server_deps(platform, for_ci, !no_nvidia); dependencies::build_android_deps( for_ci, all_targets, OpenXRLoadersSelection::All, ); } } "build-streamer" => { build::build_streamer(profile, gpl, None, false, profiling, keep_config) } "build-launcher" => build::build_launcher(profile, false), "build-server-lib" => build::build_server_lib(profile, None, false), "build-client" => build::build_android_client(profile), "build-client-lib" => { build::build_android_client_core_lib(profile, link_stdcpp, all_targets) } "build-client-xr-lib" => { build::build_android_client_openxr_lib(profile, link_stdcpp) } "run-streamer" => { if !no_rebuild { build::build_streamer(profile, gpl, None, false, profiling, keep_config); } run_streamer(); } "run-launcher" => { if !no_rebuild { build::build_launcher(profile, false); } run_launcher(); } "package-streamer" => { packaging::package_streamer(platform, for_ci, !no_nvidia, gpl, root) } "package-launcher" => packaging::package_launcher(), "package-client" => packaging::package_client_openxr(package_flavor, for_ci), "package-client-lib" => packaging::package_client_lib(link_stdcpp, all_targets), "format" => format::format(), "check-format" => format::check_format(), "clean" => clean(), "bump" => version::bump_version(version, is_nightly), "clippy" => { if for_ci { ci::clippy_ci() } else { clippy() } } "check-msrv" => version::check_msrv(), "check-licenses" => { packaging::generate_licenses(); } "kill-oculus" => kill_oculus_processes(), _ => print_help_and_exit("Unrecognized subcommand."), } } else { print_help_and_exit("Wrong arguments."); } } else { print_help_and_exit("Missing subcommand."); } let elapsed_time = begin_time.elapsed(); println!( "\nDone [{}m {}s]\n", elapsed_time.as_secs() / 60, elapsed_time.as_secs() % 60 ); } ================================================ FILE: alvr/xtask/src/packaging.rs ================================================ use crate::{ BuildPlatform, build::{self, Profile}, command, dependencies::{self, OpenXRLoadersSelection}, }; use alvr_filesystem as afs; use std::{fs, path::Path}; use xshell::{Shell, cmd}; pub enum ReleaseFlavor { GitHub, MetaStore, PicoStore, } pub fn generate_licenses() -> String { let sh = Shell::new().unwrap(); cmd!(sh, "cargo install cargo-about --version 0.8.4 --locked") .run() .unwrap(); let licenses_template = afs::crate_dir("xtask").join("licenses_template.hbs"); cmd!(sh, "cargo about generate {licenses_template}") .read() .unwrap() } pub fn include_licenses(root_path: &Path, gpl: bool) { let sh = Shell::new().unwrap(); // Add licenses let licenses_dir = root_path.join("licenses"); sh.create_dir(&licenses_dir).unwrap(); sh.copy_file( afs::workspace_dir().join("LICENSE"), licenses_dir.join("ALVR.txt"), ) .unwrap(); sh.copy_file( afs::crate_dir("server_openvr").join("LICENSE-Valve"), licenses_dir.join("Valve.txt"), ) .unwrap(); if gpl { sh.copy_file( afs::deps_dir().join("windows/ffmpeg/LICENSE.txt"), licenses_dir.join("FFmpeg.txt"), ) .ok(); } if cfg!(windows) { sh.copy_file( afs::deps_dir().join("windows/libvpl/alvr_build/share/vpl/licensing/license.txt"), licenses_dir.join("libvpl.txt"), ) .unwrap(); } let licenses_content = generate_licenses(); sh.write_file(licenses_dir.join("dependencies.html"), licenses_content) .unwrap(); } pub fn package_streamer( platform: Option, skip_admin_priv: bool, enable_nvenc: bool, gpl: bool, root: Option, ) { let sh = Shell::new().unwrap(); dependencies::prepare_server_deps(platform, skip_admin_priv, enable_nvenc); build::build_streamer(Profile::Distribution, gpl, root, true, false, false); include_licenses(&afs::streamer_build_dir(), gpl); if cfg!(windows) { command::zip(&sh, &afs::streamer_build_dir()).unwrap(); } else { command::targz(&sh, &afs::streamer_build_dir()).unwrap(); } } pub fn package_launcher() { let sh = Shell::new().unwrap(); sh.remove_path(afs::launcher_build_dir()).ok(); build::build_launcher(Profile::Distribution, true); include_licenses(&afs::launcher_build_dir(), false); if cfg!(windows) { command::zip(&sh, &afs::launcher_build_dir()).unwrap(); // todo: installer } else { command::targz(&sh, &afs::launcher_build_dir()).unwrap(); } } pub fn replace_client_openxr_manifest(from_pattern: &str, to: &str) { let manifest_path = afs::crate_dir("client_openxr").join("Cargo.toml"); let manifest_string = fs::read_to_string(&manifest_path) .unwrap() .replace(from_pattern, to); fs::write(manifest_path, manifest_string).unwrap(); } pub fn package_client_openxr(flavor: ReleaseFlavor, skip_admin_priv: bool) { fs::remove_dir_all(afs::deps_dir().join("android_openxr")).ok(); let openxr_selection = match flavor { ReleaseFlavor::GitHub => OpenXRLoadersSelection::All, ReleaseFlavor::MetaStore => OpenXRLoadersSelection::OnlyGeneric, ReleaseFlavor::PicoStore => OpenXRLoadersSelection::OnlyPico, }; dependencies::build_android_deps(skip_admin_priv, false, openxr_selection); if !matches!(flavor, ReleaseFlavor::GitHub) { replace_client_openxr_manifest( r#"package = "alvr.client.stable""#, r#"package = "alvr.client""#, ); } if matches!(flavor, ReleaseFlavor::MetaStore) { replace_client_openxr_manifest(r#"value = "all""#, r#"value = "quest2|questpro|quest3""#); } build::build_android_client(Profile::Distribution); } pub fn package_client_lib(link_stdcpp: bool, all_targets: bool) { let sh = Shell::new().unwrap(); build::build_android_client_core_lib(Profile::Distribution, link_stdcpp, all_targets); command::zip(&sh, &afs::build_dir().join("alvr_client_core")).unwrap(); } ================================================ FILE: alvr/xtask/src/version.rs ================================================ use crate::command; use alvr_filesystem as afs; use std::fs; use xshell::{Shell, cmd}; pub fn split_string(source: &str, start_pattern: &str, end: char) -> (String, String, String) { let start_idx = source.find(start_pattern).unwrap() + start_pattern.len(); let end_idx = start_idx + source[start_idx..].find(end).unwrap(); ( source[..start_idx].to_owned(), source[start_idx..end_idx].to_owned(), source[end_idx..].to_owned(), ) } pub fn version() -> String { let manifest_path = afs::workspace_dir().join("Cargo.toml"); println!("cargo:rerun-if-changed={}", manifest_path.to_string_lossy()); let manifest = fs::read_to_string(manifest_path).unwrap(); let (_, version, _) = split_string(&manifest, "version = \"", '\"'); version } fn bump_cargo_version(new_version: &str) { let manifest_path = afs::workspace_dir().join("Cargo.toml"); let manifest = fs::read_to_string(&manifest_path).unwrap(); let (file_start, _, file_end) = split_string(&manifest, "version = \"", '\"'); let manifest = format!("{file_start}{new_version}{file_end}"); fs::write(manifest_path, manifest).unwrap(); } pub fn bump_version(maybe_version: Option, is_nightly: bool) { let sh = Shell::new().unwrap(); let mut version = maybe_version.unwrap_or_else(version); if is_nightly { version = format!( "{version}+nightly.{}", command::date_utc_yyyymmdd(&sh).unwrap() ); } bump_cargo_version(&version); println!("Git tag:\nv{version}"); } pub fn check_msrv() { let sh = Shell::new().unwrap(); cmd!( sh, "cargo install cargo-msrv --git https://github.com/foresterre/cargo-msrv --rev 14097beaa5fa770aabd66170572cb04f1dac87c2" ) .run() .unwrap(); let paths = [ "alvr/server_openvr", "alvr/dashboard", "alvr/launcher", "alvr/client_openxr", ]; for path in paths { cmd!(sh, "cargo msrv verify --path {path}").run().unwrap() } } ================================================ FILE: wiki/ALVR-Checklist.md ================================================ ## Hardware Requirements * [ ] Intel Core i5-4590/AMD FX 8350 equivalent or better * [ ] At least 4GB of Ram * [ ] NVIDIA GeForce GTX 970, AMD Radeon R9 290, Intel ARC equivalent or better * [ ] A 5Ghz router/access point or my PC can create its own 5Ghz hotspot ## Network settings * [ ] My PC has a wired connection to the router/access point * [ ] The access point is placed in sight of my designated playspace without any obstructions * [ ] I'm using the 5ghz antenna of the router/access point * [ ] No one else is using the router/access point * [ ] I'm the only user of the 5Ghz channel of the router/access point. No one else is using the same channel in the vicinity * [ ] The 5Ghz and 2.4Ghz parts of the access point have different SSIDs to prevent switching to 2.4ghz ## Software settings * [ ] I have the latest Windows 10 updates * [ ] I have a recent version of SteamVR ## Troubleshooting * [ ] The firewall settings where successfully applied with the setup of ALVR * [ ] I did not change the network settings since the installation of ALVR (Private/Public/Work) * [ ] I did not move the installation folder of ALVR since the setup * [ ] The path to the folder of ALVR does not contain any non latin characters or accents (ツ Л Ö ...) ================================================ FILE: wiki/ALVR-wired-setup-(ALVR-over-USB).md ================================================ ## ALVR native wired mode support As of v20.12 ALVR supports wired connections directly through the dashboard. Just enable the "Wired Connection" toggle on the Devices screen, plug in your headset and accept the "Allow USB debugging?" popup displayed by the headset. Note that your headset will need to have Developer Mode and USB Debugging enabled to use this feature. For Quest headsets see [here](https://developers.meta.com/horizon/documentation/native/android/mobile-device-setup/) for instructions. The last step about installing ADB should be skipped, as ALVR downloads a copy of ADB on it's own and uses that. If you have successfully followed all those steps and it still isn't connecting, ensure that the setting "Connection -> Wired Client Type" matches where you installed the client from (for the launcher also use the "Github" option). ## The DEPRECATED (and clunky) way: The following sections list the old and deprecated way to get a wired connection and is only kept as reference. This has exactly the same requirements as the native wired mode, but requires additional software and is more complex to setup, so native mode should be preferred. ## ALVR Streamer (PC) Configuration * **Switch the connection streaming protocol to TCP** in Settings > Connection. * If your headset is detected, click "Trust." Click "Edit", "Add new" and change the IP address to `127.0.0.1`. * If your headset is not detected, click "Add device manually" and use the IP address `127.0.0.1`. Use the hostname displayed on your headset screen. ## Letting your PC communicate with your HMD The Quest, Pico HMDs are Android devices, therefore, we can use [Android Device Bridge](https://developer.android.com/studio/command-line/adb) commands to tell the HMDs to look for data over USB, as well as Wi-Fi, using port forwarding. You can accomplish this with some pre-made applications/scripts (just below), or run the commands manually with [SideQuest](https://sidequestvr.com/setup-howto) If you haven't already, connect a USB cable from your PC to your headset. USB 2.0 will work fine but 3.0 and higher is best. **Make sure to enable dev account and authorize the computer in your headset if you're on quest or enable USB Debug on Pico in settings.** ### Option 1 - Dedicated ADB Applications The following programs serve to wrap and simplify the process of doing manual ADB commands, the first two will also automatically reconnect the headset if the USB connection is interrupted. * [**ADBForwarder (Recommended)**](https://github.com/alvr-org/ADBForwarder) * Easy to use * Downloads ADB for you * Cross-platform (Windows & Linux) * [**Python Script**](https://gist.github.com/Bad-At-Usernames/684784f42cbb69e22688a21173ec263d) * Lightweight and simple * Requires [Python 3](https://www.python.org/downloads/) and [PyWin32](https://pypi.org/project/pywin32/) * Requires [ADB Platform Tools](https://developer.android.com/studio/releases/platform-tools) to be in the same directory as `main.py` * Just extract `platform-tools` to your desktop and place `main.py` in that folder, should work when you run the script * [**Batch Script**](https://gist.github.com/AtlasTheProto/1f03c3aeac70c4af5b4f2fcd9b9273c0) * Requires [ADB Platform Tools](https://developer.android.com/studio/releases/platform-tools), edit the path in line 2 to point to the directory where you extracted `platform-tools` * Needs to be run every time you (re)connect your headset ### Option 2 - [SideQuest](https://sidequestvr.com/setup-howto) * Ensure SideQuest is running, and the headset has authorized the USB connection to the PC * Open the 'Run ADB Commands' menu in SideQuest (top-right, box with an arrow inside it) * Click 'Custom Command' and run these adb commands: * `adb forward tcp:9943 tcp:9943` * `adb forward tcp:9944 tcp:9944` * These commands will need to be run every time you (re)connect your headset. * Keep SideQuest opened until you want to close the connection. *** Once you are finished, the headset should now establish a connection over USB. ================================================ FILE: wiki/Building-From-Source.md ================================================ ALVR can be built on Windows and Linux. The following instructions are for both OSes. # Common Prerequisites Preferred IDE (optional): Visual Studio Code with rust-analyzer extension You need to install [rustup](https://www.rust-lang.org/tools/install). On Windows you also need [Chocolatey](https://chocolatey.org/install). To clone the repository use `git clone --recurse-submodules https://github.com/alvr-org/ALVR.git`. If you previously cloned the repo without submodules, simply run `git submodule update --init --checkout --recursive` in it. # Streamer Building First you need to gather some additional resources in preparation for the build. If you are on Linux, install these additional packages: * **Arch** Note: At time of writing Arch gcc is too new to be compatible with nvcc. This means there is no neat way to compile an nvidia compatible build. Recommended workarounds are to build in some kind of containerised environment. This has been done successfully with both nixos and flatpak - but are not documented yet. ```bash sudo pacman -S clang curl nasm pkgconf yasm vulkan-headers libva-mesa-driver unzip ffmpeg libpipewire ``` * The [`alvr-git`](https://aur.archlinux.org/packages/alvr-git) [AUR package](https://wiki.archlinux.org/title/Arch_User_Repository) may also be used to do this automatically. * **Gentoo** * `media-video/ffmpeg >= 4.4 [encode libdrm vulkan vaapi]` * `sys-libs/libunwind` * `dev-lang/rust >= 1.72` * `media-video/pipewire [jacksdk]` * **Debian 12 / Ubuntu 20.04 / Pop!\_OS 20.04** ```bash sudo apt install pulseaudio-utils build-essential pkg-config libclang-dev libssl-dev libasound2-dev libjack-dev libgtk-3-dev libvulkan-dev libunwind-dev gcc yasm nasm curl libx264-dev libx265-dev libxcb-render0-dev libxcb-shape0-dev libxcb-xfixes0-dev libspeechd-dev libxkbcommon-dev libdrm-dev libva-dev libvulkan-dev vulkan-headers libpipewire-0.3-dev libspa-0.3-dev git ``` * Note: Libpipewire/libspa must be at least 0.3.49 version - make sure to use upstream pipewire * **Fedora** ```bash sudo dnf groupinstall 'Development Tools' | For c++ and build tools sudo dnf install nasm yasm libdrm-devel vulkan-headers pipewire-jack-audio-connection-kit-devel atk-devel gdk-pixbuf2-devel cairo-devel rust-gdk0.15-devel x264-devel vulkan-devel libunwind-devel clang openssl-devel alsa-lib-devel libva-devel pipewire-devel git ``` If you are using Nvidia, see [Fedora cuda installation](https://github.com/alvr-org/ALVR/wiki/Building-From-Source#fedora-cuda-installation) Move to the root directory of the project, then run this command (paying attention to the bullet points below): ```bash cargo xtask prepare-deps --platform [your platform] [--gpl] [--no-nvidia] ``` * Replace `[your platform]` with your computer OS, either `windows` or `linux` * **Windows only:** Use the `--gpl` flag if you want to download, build and bundle FFmpeg inside the ALVR streamer. Keep in mind that this is only needed for software encoding. As the name suggests, if you use this flag you can only redistribute the final package as GPLv2.0 licensed; because of the x264 encoder. * **Linux only:** Use the `--no-nvidia` flag if you have a AMD gpu. Next up is the proper build of the streamer. Run the following: ```bash cargo xtask build-streamer --release [--gpl] ``` **Windows only:** Again, the `--gpl` flag is needed only if you want to bundle FFmpeg. You can find the resulting package in `build/alvr_streamer_[your platform]` If you want to edit and rebuild the code, you can skip the `prepare-deps` command and run only the `build-streamer` command. ## Fedora CUDA installation If you are here for CUDA installation on Fedora you're at the right place! Else continue down to [Android App Building](https://github.com/alvr-org/ALVR/wiki/Building-From-Source#android-app-building) ### 1. Install Nvidia drivers and Fedora CUDA driver ```bash sudo dnf update -y ``` (Reboot if you have a new kernel) ```bash sudo dnf install akmod-nvidia sudo dnf install xorg-x11-drv-nvidia-cuda ``` Wait until ```modinfo -F version nvidia``` doesn't report ```"ERROR: Module nvidia not found"``` anymore ### 2. Install Nvidia's CUDA In the previous step, we installed Fedora's CUDA that doesn't work with ALVR, installing Nvidia's CUDA works and creates directories instead ```bash sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/fedora37/x86_64/cuda-fedora37.repo ``` Change the Fedora version if you are on a different version. You should check if your version is supported by inspecting the repo ```bash sudo dnf clean all sudo dnf module disable nvidia-driver sudo dnf -y install cuda export PATH=/usr/local/cuda-12.3/bin${PATH:+:${PATH}} ``` If your cuda version is different, change it to the version that is installed. You can check installed versions by doing ```ls /usr/local/ | grep "cuda"``` in your terminal #### Note about Nvidia's CUDA * Disabling the nvidia-driver doesn't disable Nvidia drivers but prevents nvidia dkms from installing over the akmod driver ### 3. Install gcc11 install with homebrew Becuase cuda cannot be ran without a gcc version lower than or equal to gcc12, you will need to install a gcc version on homebrew. The fedora gcc11 package got removed so this is the only way sadly To install homebrew, run this command: ```bash /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" ``` Then install gcc11 ```bash brew install gcc@11 ``` #### Notes on installing gcc11 with homebrew * If brew is not found in your path, run the following separately to add brew to your path: ```bash test -d ~/.linuxbrew && eval "$(~/.linuxbrew/bin/brew shellenv)" test -d /home/linuxbrew/.linuxbrew && eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> ~/.bashrc ``` ### 4. Modify dependencies.rs to use correct cuda path and gcc version Because CURA installs as a symlink by default, we need to change the dependencies.rs to use the directory From the ALVR directory edit the ./alvr/xtask/src/dependencies.rs, and change two lines: * Line 159, change ```cuda``` -> ```cuda-12.3``` (or whatever version you have) * Line 179, replace that line with ```--nvccflags=\"-ccbin /home/linuxbrew/.linuxbrew/bin/g++-11 -gencode arch=compute_52,code=sm_52 -O2\"``` (Change homebrew path if needed, default is used) You should be good to go! Refer to [Streamer Building](https://github.com/alvr-org/ALVR/wiki/Building-From-Source#streamer-building) for the commands to build ALVR # Android App Building ## 1. Installing necessary packages For the app you need install: * [Android Studio](https://developer.android.com/studio) or the [sdkmanager](https://developer.android.com/studio/command-line/sdkmanager) * Android SDK Platform-Tools 29 (Android 10) * Latest Android NDK (currently v25.1.8937393) On Linux, the specific package names for the android tools can differ from distro to distro, see up on the wiki for more information: * Gentoo: * * Arch: * * Debian: * * Ubuntu: * * Pop!\_OS: * N/A The three mentioned developer applications can be installed from upstream; although the packages and setup responsible for the required tools can differ between distros, being: * **Arch** * Packages can vary, read up on the Arch Wiki's [Android](https://wiki.archlinux.org/title/Android) page. * **Gentoo** * `dev-util/android-studio` * `dev-util/android-sdk-update-manager` * `dev-util/android-ndk >= 25.1` For Debian, it requires to have the `non-free` repository to be enabled: * **Debian 12 / Ubuntu 22.10 / Pop!\_OS 22.10** ```bash sudo apt install android-sdk-platform-tools-common sdkmanager google-android-ndk-r26b-installer ``` ## 2. Setting environment variables For Windows, set the environment variables: * `JAVA_HOME`: * Example: `C:\Program Files\Android\Android Studio\jre` * `ANDROID_HOME`: * Example: `%LOCALAPPDATA%\Android\Sdk` * `ANDROID_NDK_HOME`: * Example: `%LOCALAPPDATA%\Android\Sdk\ndk\25.1.8937393` For Linux, the correct directories for the environment variables can greatly differ depending on the type of install. See the wiki page of your distro for more information: * Gentoo: * * Ubuntu: * Distro wikis that weren't listed above does not mention of environment variables, although generally they would be as: * `JAVA_HOME`: * `/usr/lib/jvm/default-java/bin` * `ANDROID_HOME`: * Arch: `~/Android/Sdk` * Gentoo: `~/Android` * Debian / Ubuntu / Pop!\_OS: `~/AndroidSDK` * `ANDROID_NDK_HOME`: * Arch: `/opt/android-sdk/ndk` * Linux: `/usr/lib/android-sdk/ndk` ## 3. Building First you need to gather some additional resources in preparation for the build. Move to the root directory of the project, then run this command: ```bash cargo xtask prepare-deps --platform android ``` Before building the app, Android has to have us to agree to the licenses otherwise building the app will halt and fail. To accept the agreements, follow the instructions for your corresponding OS: * Windows: ```shell cd "%ANDROID_SDK_ROOT%\tools\bin" sdkmanager.bat --licenses ``` * Linux: ```bash cd ~/AndroidSDK sdkmanager --licenses ``` Next up is the proper build of the app. Run the following: ```bash cargo xtask build-client --release ``` The built APK will be in `build/alvr_client_quest`. You can then use adb or SideQuest to install it to your headset. To build and run: ```bash cd alvr/client_openxr cargo apk run ``` You need the headset to be connected via USB and with the screen on to successfully launch the debugger and logcat. # Troubleshooting (Linux) On some distributions, Steam Native runs ALVR a little better. To get Steam Native on Ubuntu run it with: ```bash env STEAM_RUNTIME=0 steam ``` On Arch Linux, you can also get all the required libraries by downloading the `steam-native-runtime` package from the multilib repository ```bash sudo pacman -S steam-native-runtime ``` Dependencies might be missing then, so run: ```bash cd ~/.steam/root/ubuntu12_32 file * | grep ELF | cut -d: -f1 | LD_LIBRARY_PATH=. xargs ldd | grep 'not found' | sort | uniq ``` Some dependencies have to be fixed manually for example instead of forcing a downgrade to libffi version 6 (which could downgrade a bunch of the system) you can do a symlink instead (requires testing): ```bash cd /lib/i386-linux-gnu ln -s libffi.so.7 libffi.so.6 ``` and ```bash cd /lib/x86_64-linux-gnu ln -s libffi.so.7 libffi.so.6 ``` A few dependencies are distro controlled, you can attempt to import the package at your own risk perhaps needing the use of alien or some forced import commands, but its not recommended (turns your system into a dependency hybrid mess), nor supported! ================================================ FILE: wiki/Controller-latency.md ================================================ Controller tracking will always be difficult. There are so many factors that influence the latency and the motion prediction. Its not something like "100ms" constantly, but depends on your movements and even the movement of the headset. There are many parameters that influence the movement that can be changed: - Tracking is currently async to the rendering and running at 3*72=216Hz - Movement prediction is set to 0 to get the latest tracking info -> no prediction on the quest - Tracking info is sent to SteamVR - Tracking info is fed into SteamVR with an offset of 10ms to enable SteamVR pose prediction - The tracking point on the Quest is different that the point on the Rift S. Angular acceleration and linear acceleration of the controller needed to be transformed to the new reference. There is a trade off between fast but wobbly and overshooting controllers and controllers that have a certain latency. For me, the current settings are perfectly playable for games like Skyrim, Fallout or Arizona Sunshine. Games like Beat Saber might be an issue. You can change the 10ms offset for SteamVR in the "Other" tab of ALVR (Controller Pose Offset). The parameter defines how old the data that is fed into SteamVR is and controls the SteamVR pose prediction. Set it to 0 to disable all predictions The default is 0.01=10ms. Its the amount of time I needed to be able swing my sword in Skyrim without feeling weird. Its very possible that this value depends on the game/user, that's why it's exposed in the control panel, and you can change it on the fly ================================================ FILE: wiki/FFmpeg-Hardware-Encoding-Testing.md ================================================ FFmpeg hardware video offloading test commands to validate hardware encoding offloading is working. Learn more at: ## Codecs * **Advanced Video Coding (AVC/h264)** - Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distribution of video content. It supports a maximum resolution of 8K UHD. Hardware encoding support is widely available. * **High Efficiency Video Coding (HEVC/h265)** - High Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H Part 2, is a video compression standard designed as part of the MPEG-H project as a successor to the widely used Advanced Video Coding (AVC, H.264, or MPEG-4 Part 10). In comparison to AVC, HEVC offers from 25% to 50% better data compression at the same level of video quality, or substantially improved video quality at the same bit rate. It supports resolutions up to 8192×4320, including 8K UHD, and unlike the primarily 8-bit AVC, HEVC's higher fidelity Main 10 profile has been incorporated into nearly all supporting hardware. Hardware encoding support is widely available. * **AOMedia Video 1 (AV1)** - AOMedia Video 1 (AV1) is an open, royalty-free video coding format initially designed for video transmissions over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia). The AV1 bitstream specification includes a reference video codec. Hardware encoding support is limited to latest generation hardware. ### Graphics Encoding APIs * **Video Acceleration API** - Video Acceleration API (VA-API) is an open source application programming interface that allows applications such as VLC media player or GStreamer to use hardware video acceleration capabilities, usually provided by the graphics processing unit (GPU). It is implemented by the free and open-source library libva, combined with a hardware-specific driver, usually provided together with the GPU driver. Check your current VA-API status with `vainfo`. * **Vulkan Video** - Vulkan is a low-level low-overhead, cross-platform API and open standard for 3D graphics and computing. It was intended to address the shortcomings of OpenGL, and allow developers more control over the GPU. It is designed to support a wide variety of GPUs, CPUs and operating systems, it is also designed to work with modern multi-core CPUs. Support is upcoming. See: * **NVENC** - Nvidia NVENC is a feature in Nvidia graphics cards that performs video encoding, offloading this compute-intensive task from the CPU to a dedicated part of the GPU. * **AMD Advanced Media Framework**- AMD AMF is a SDK for optimal access to AMD GPUs for multimedia processing. ### Test Source Input Generation Please note that using the test source for input generation induces CPU load. When monitoring for proper GPU offloading, there will still be expected CPU load from FFmpeg. ```sh ffmpeg -hide_banner -f lavfi -i testsrc2=duration=30:size=1280x720:rate=90 ``` * **lavfi** - Libavfilter input virtual device. This input device reads data from the open output pads of a libavfilter filtergraph. For each filtergraph open output, the input device will create a corresponding stream which is mapped to the generated output. The filtergraph is specified through the option graph. * **testsrc2** - The `testsrc2` source generates a test video pattern, showing a color pattern, a scrolling gradient and a timestamp. This is mainly intended for testing purposes. The `testsrc2` source is similar to `testsrc`, but supports more pixel formats instead of just `rgb24`. This allows using it as an input for other tests without requiring a format conversion. 1) duration - how long of a clip in seconds 2) size - dimensions of the video 3) rate - frame rate per second ### Render Playback Use your favorite video player to verify the video was rendered correctly. * MPV - mpv is free and open-source media player software based on MPlayer, mplayer2 and FFmpeg. It runs on several operating systems, including Unix-like operating systems (Linux, BSD-based, macOS) and Microsoft Windows, along with having an Android port called mpv-android. It is cross-platform, running on ARM, PowerPC, x86/IA-32, x86-64, and MIPS architecture. * VLC - VLC media player (previously the VideoLAN Client and commonly known as simply VLC) is a free and open-source, portable, cross-platform media player software and streaming media server developed by the VideoLAN project. VLC is available for desktop operating systems and mobile platforms, such as Android, iOS and iPadOS. VLC is also available on digital distribution platforms such as Apple's App Store, Google Play, and Microsoft Store. ### Nvidia GPU Test the Nvidia hardware encoding pipeline. Only NVENC is supported as the current Nvidia VA-API driver () only supports NVDEC. Check your hardware support at for NVENC support. Monitoring utilities: * **nvtop** - NVTOP stands for Neat Videocard TOP, a (h)top like task monitor for AMD, Intel and NVIDIA GPUs. It can handle multiple GPUs and print information about them in a htop-familiar way. * **nvidia-smi pmon** - The NVIDIA System Management Interface (nvidia-smi) is a command line utility, based on top of the NVIDIA Management Library (NVML), intended to aid in the management and monitoring of NVIDIA GPU devices. The `pmon` command lists the statistics for all the compute and graphics processes running on each device. Nvenc AVC (h264) hardware encoding: ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -c:v h264_nvenc -qp 18 \ nvidia-h264_nvec-90fps-300s.mp4 ``` Nvenc HEVC (h265) hardware encoding: ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -c:v hevc_nvenc -qp 18 \ nvidia-hevc_nvec-90fps-300s.mp4 ``` Nvenc AV1 hardware encoding (Ada Lovelace or newer hardware): ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -c:v av1_nvenc -qp 18 \ nvidia-av1_nvec-90fps-300s.mp4 ``` ### Intel GPU Test the Intel hardware encoding pipeline. Only VA-API is supported with the intel-media-driver () on GEN based graphics hardware. Check your hardware support at for encoding codec support. Monitoring utilities: * **nvtop** - NVTOP stands for Neat Videocard TOP, a (h)top like task monitor for AMD, Intel and NVIDIA GPUs. It can handle multiple GPUs and print information about them in a htop-familiar way. VA-API AVC (h264) hardware encoding: ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -vaapi_device /dev/dri/renderD128 -vf 'format=nv12,hwupload' \ -c:v h264_vaapi -qp 18 \ intel-h264_vaapi-90fps-300s.mp4 ``` VA-API HEVC (h265) hardware encoding: ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -vaapi_device /dev/dri/renderD128 -vf 'format=nv12,hwupload' \ -c:v hevc_vaapi -qp 18 \ intel-hevc_vaapi-90fps-300s.mp4 ``` VA-API AV1 hardware encoding (Arc A-Series only): ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -vaapi_device /dev/dri/renderD128 -vf 'format=nv12,hwupload' \ -c:v av1_vaapi -qp 18 \ intel-av1_vaapi-90fps-300s.mp4 ``` ### AMD GPU Test the AMD hardware encoding pipeline. Only VA-API is supported with the mesa-va-drivers () on AMD based graphics hardware. Check your hardware support at for encoding codec support. Video Core Next (VCN) hardware is required for hardware encoding. Monitoring utilities: * **nvtop** - NVTOP stands for Neat Videocard TOP, a (h)top like task monitor for AMD, Intel and NVIDIA GPUs. It can handle multiple GPUs and print information about them in a htop-familiar way. VA-API AVC (h264) hardware encoding: ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -vaapi_device /dev/dri/renderD128 -vf 'format=nv12,hwupload' \ -c:v h264_vaapi -qp 18 \ amd-h264_vaapi-90fps-300s.mp4 ``` VA-API HEVC (h265) hardware encoding: ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -vaapi_device /dev/dri/renderD128 -vf 'format=nv12,hwupload' \ -c:v hevc_vaapi -qp 18 \ amd-hevc_vaapi-90fps-300s.mp4 ``` VA-API AV1 hardware encoding (VCN 4.0+, Navi 3x only): ```sh ffmpeg -hide_banner \ -f lavfi -i testsrc2=duration=300:size=1280x720:rate=90 \ -vaapi_device /dev/dri/renderD128 -vf 'format=nv12,hwupload' \ -c:v av1_vaapi -qp 18 \ amd-av1_vaapi-90fps-300s.mp4 ``` ================================================ FILE: wiki/Fixed-Foveated-Rendering-(FFR).md ================================================ ## What is it, why do I need it In short: The human eye can only see sharp in a very small area (the fovea). That's why we move our eyes constantly to get the feeling that our whole view is a sharp image. The idea of foveated rendering is, to only render the small portion of the screen where we look at at the highest resolution, and the other parts at a lower resolution. This would massively increase the performance without any noticeable visual impact. But it has one drawback: You need to track the movement of the eyes. While there are already headsets out there that have eyetracking, the quest does not have it. That's why oculus is using Fixed Foveated rendering. There is some research that shows, that if you assume that the user looks at the center of the screen, some parts of the image are more important than others ([Oculus](https://developer.oculus.com/documentation/mobilesdk/latest/concepts/mobile-ffr/)). Many games on the Quest use this to improve rendering performance. ## FFR in ALVR That's not how ALVR is using it :P We don't have any influence on how games get rendered, we only get the final rendered image that should be displayed. What ALVR normally does is: - takes that image - encodes it as a video with the resolution you set at the video tab - transmits it to the Quest - displays the video With FFR: - takes the image - projects the image, keeping the center area at the resolution you set at the video tab, reducing the resolution at the outer regions - encodes it as a video with the new, lower overall resolution - transmits the video to the Quest - reprojects the video to the original size - displays the image There are two implementations of FFR by [zarik5](https://github.com/zarik5) - warp: Uses a radial warping of the video where the center stays the same resolution and the outer regions get "squished" to reduce resolution - slices: Slices the image into parts (center, left/right, top/bottom) and encodes the outer slices with lower resolution. This method produces a much sharper image. **Advantages**: Lower resolution results in faster encoding and decoding of the video which will decrease overall latency. Same bitrate at lower resolution results in higher image quality. On slow networks, bitrate can be reduced resulting in the same quality as without ffr. **Drawbacks**: Using the warped method results in a slightly overall blurry image. You can compensate this by setting the initial video resolution to 125%. Slicing will result in a noticeable border where the high res center ends. Increasing the foveation strength will result in visual artifacts with both methods ## Configuration That depends on your perception. You should try different settings going from strength = 2 up to 5 for both methods. The higher you go, the more visual artifacts you will see at the edges of the screen. For the slice method, you can also set a center offset. This moves the high res center up or down to accommodate games that have more interaction in the upper or lower part of the screen. [wikipedia](https://en.wikipedia.org/wiki/Foveated_rendering) ================================================ FILE: wiki/Hand-tracking-controller-bindings.md ================================================ Current bindings for ALVR 20 === To control state of gestures, you can toggle `Headset -> Controllers -> Gestures -> Only touch` Enabled state means that gestures won't be triggered, disabled would mean all gestures are activated. Gestures --- | Action | Handtracking pinch | | -------------- | ----------------------------------- | | Trigger | Pinch thumb and index | | Joystick click | Curl thumb to palm | | Grip | Curl middle, ring and little | | Y/B | Pinch thumb and middle | | X/A | Pinch thumb and ring | | Menu button | Pinch thumb and little on left hand | Joystick --- Activation is done through curling all 4 fingers and touching top of hand with thumb ================================================ FILE: wiki/Headset-and-ALVR-streamer-on-separate-networks.md ================================================ ## ALVR v14 and Above Here are explained two methods to connect PC and headset remotely, port-forwarding and ZeroTier. The primary purpose of this is connecting the headset to a Cloud PC (like ShadowPC). ## Important notes on security * ALVR protocol does not have any encryption or authentication (apart from ALVR device IP address shown in ALVR streamer and the requirement to add devices on ALVR streamer). * It is recommended to run ALVR via encrypted tunnel (VPN) over the internet. In case VPN is not an option, access to ALVR streamer (UDP ports 9943 and 9944) should be restricted by Windows Firewall (only connections from known IP addresses of ALVR devices should be allowed) and ALVR streamer should not be left running unattended. * **Warning!** SteamVR allows to control desktop from VR headset (i.e. a **malicious ALVR device could take over the PC**). * As the license states ALVR IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND (see the file `LICENSE` in this GitHub repository for legal text/definition). You are on your own (especially if you run ALVR over the Internet without VPN). ## Port-forwarding Port-forwarding allows to connect devices that are behind different NATs, i.e. local networks. You need to have administrator access to your router. This method has the best streaming performance. **IMPORTANT**: ALVR does not use end-to-end encryption of the stream data. By using this method you need to be aware that the connection is vulnerable to "Man In The Middle" attacks. 1. Take note of the public IP of your headset. You can use the online tool [WhatIsMyIP](https://www.whatismyip.com/). 2. Inside your router web interface or app, add a port-forwarding rule for your headset. You need to specify the ports 9943 and 9944 for both TCP and UDP. 3. Connect to the remote PC and open ALVR. In the Devices tab press `Add device manually`. Fill in the fields with a name for your headset (you can use the name you want), the hostname (you can read it in the welcome screen in your headset when you open the ALVR app), the remote IP of the headset (that is the IP you got on step 1.) and then press `Save`. You can now use ALVR to connect to your remote PC. **Note**: The public IP can change often. Every time you want to use ALVR you need to check that your current public IP is the same as the last time. If the IP changed, you can update it using the "Edit connection" interface, accessed with the `Edit` button next to your headset name on the streamer. ## ZeroTier [ZeroTier](https://www.zerotier.com/) is a tunneling software that makes remote devices connect to each other as if they are in the same local network. Comparing this to the port-forwarding method: Pros: * Does not require access to the router interface. * You don't need to update the public IP often on the streamer. * The connection in encrypted. Cons: * The streaming performance is worse. You may experience more glitches and loss of quality in the image and audio. ### Requirements * [ZeroTier](https://www.zerotier.com/) for your PC * ZeroTier APK for your Quest (you can find it online) * SideQuest or some other method to install the ZeroTier APK onto your headset ### Installation Use the "Install APK" function of SideQuest to install the ZeroTier APK to your Quest, and also download and install ZeroTier on your PC. After you've installed ZeroTier, follow Zerotier's official [Getting Started](https://zerotier.atlassian.net/wiki/spaces/SD/pages/8454145/Getting+Started+with+ZeroTier) guide to setup a network for ALVR. Join the network on both the Quest and the PC. On the Quest, make sure that the network is enabled by switching on the slider on the network in the list in the ZeroTier app (you may be prompted to allow ZeroTier to create a VPN connection). After both your PC and your Quest are connected to the same ZeroTier network, we'll need to manually add your quest to the ALVR dashboard. To do so, we'll need to find your Quest's ZeroTier IP. There are two ways to do this. * Go the the ZeroTier network page, find your quest under "Members", and copy the managed IP from there * Or, in the ZeroTier app on your quest, click on the network you created. The IP is under the "Managed IPs" section at the bottom. The IP should look something like this `192.168.143.195`. If there's a `/` at the end with a couple numbers following it, remove them along with the slash. Next, we'll need to add the Quest to the ALVR dashboard. On your headset, launch ALVR. The on the ALVR dashboard on your PC, click the "Add device manually" button, provide a name and hostname (You can get this from the "trust" screen of ALVR on your Quest), then put in the IP address that we got from ZeroTier. At this point, you should be ready to go. Have fun in VR! ### Troubleshooting * If you can't get your Quest to connect to ALVR, and are stuck on the "Trust" screen, try to ping your Quest's managed IP address (the one we got earlier). If it says "no route to host" or something similar, your Quest can't see your PC. Try running through the steps above to make sure you didn't miss anything. ## Tailscale An alternative to ZeroTier with practically the same setup procedure. This could have better latency, depending on your distance to the datacenter. ## n2n [n2n](https://github.com/ntop/n2n) is another P2P VPN solution, just like ZeroTier. You need to run _supernode_ on a server with an IP and port publicly accessible on internet (or at least your PC and Quest can access to it), and run _edge_ node on your PC and Quest. Its pros and cons are similar to ZeroTier, but it's self-hosted and open-source if you care about privacy, though instead you need some knowledge about networking and server deploying. ### Requirements * Compile [n2n](https://github.com/ntop/n2n) from source * Or you can grab pre-built binaries from [here](https://github.com/lucktu/n2n) directly, compiled by lucktu. * Some Linux distribution may have n2n, but be sure you're using the same version. Since the source code is v3, the following steps will also use v3 in the example below. * [TAP-Windows driver](https://community.openvpn.net/openvpn/wiki/GettingTapWindows) or [OpenVPN](https://openvpn.net/community/) (includes TAP-Windows) if you're using Windows PC * [hin2n](https://github.com/switch-iot/hin2n) APK * A server with public IP and allow public ports * SideQuest or some other method to install the hin2n APK onto your headset ### Installation We're going to use n2n v3, and set the port of _supernode_ to `1234` as the example. You can change `1234` to any port, but below `1024` requires root. * Open port `1234` on your server's firewall (usually `iptables`, if you don't know what to do, ask Google). * Upload _supernode_ binary to your server, run `./supernode -p 1234`. * Install TAP-Windows driver or OpenVPN on your PC if you're using Windows. * Upload _edge_ binary to your PC, run `./edge -c [network-name] -k [secret-password] -a 192.168.100.1 -l [your-server-ip]:1234` to connect to the _supernode_, assign the IP `192.168.100.1` to the PC, and use the password you provided for data encryption. * Once you see `[OK] edge <<< ================ >>> supernode`, your PC is done, or you need to follow the error logs to see what's wrong. * Install _hin2n_ on your Quest and open it, click the plus button at the top-right corner to add a new configuration and assign `192.168.100.2` to your Quest: * N2N version: v3 * Supernode: `[your-server-ip]:1234` * Community: `[network-name]` * Encrypt key: `[secret-password]` * IP address: `192.168.100.2` * Subnet mask: `255.255.255.0` * Click "Current Setting" under the connect button, select the configuration we created just now, then click the connect button. If you're asked to allow hin2n to create a VPN connection, allow it. * Once you see `[OK] edge <<< ================ >>> supernode`, your Quest is done. * Open ALVR on your headset, record the hostname it shows. * Open ALVR dashboard on your PC, click "Add device manually" button, put the hostname you just recorded, and set IP address to `192.168.100.2` which is assigned to Quest just now. * Once it's done, you're all set. ### Troubleshooting * Make sure you can access to the supernode, your supernode should be run on a server with public IP, and you can ping it on your PC. * If your Quest cannot connect to ALVR dashboard, ping the IP you assigned to Quest in hin2n. If it fails, try redoing the setup steps. * If the edge binary or hin2n says the IP has already been assigned and not released by supernode, you can set IP address to another one in the same subnet like `192.168.100.123` to reassign a new IP to the device. * If you're playing over WAN, you may see more glitches, higher stream latency, or lagger response with TCP. Use adaptive bitrate and UDP may improve your experience. ================================================ FILE: wiki/Home.md ================================================ ALVR is a vr streaming software that allows you to stream SteamVR games to your standalone VR headset. Use the sidebar to navigate the wiki. ================================================ FILE: wiki/How-ALVR-works.md ================================================ This document details some technologies used by ALVR. If you have any doubt about what is (or isn't) written in here you can contact @zarik5, preferably on Discord. This document was last updated on June 27th 2023 and refers to the master branch. ## Table of contents * Architecture * The packaged application * Programming languages * Source code organization * Logging and error management * The event system * Session and settings * Procedural generation of code and UI * The dashboard * The user interface * Driver communication * Driver lifecycle * The streaming pipeline: Overview * Client-driver communication * Discovery * Streaming * SteamVR driver * Client and driver compositors * Foveated rendering * Color correction * Video transcoding * Audio * Tracking and display timing * Other streams * Upcoming * Phase sync * Sliced encoding ## Architecture ### The packaged application ALVR is made of two applications: the streamer and client. The streamer can be installed on Windows and Linux, while the client is installed on Android VR headsets. The client communicates with the driver through TCP or UDP sockets. The client is a single unified APK, named `alvr_client_android.apk`. It is powered by OpenXR and it is compatible with Quest headsets, recent Pico headsets and HTC Focus 3 and XR Elite. The streamer is made of two main parts: the dashboard and the driver (also known as server). The driver is dynamically loaded by SteamVR. This is the file structure on Windows: * `bin/win64/` * `driver_alvr_server.dll`: The main binary, responsible for client discovery and streaming. Loaded by SteamVR. * `driver_alvr_server.pdb`: Debugging symbols * `openvr_api.dll`: OpenVR SDK used for updating the chaperone. * `vcruntime140_1.dll`: Windows SDK used by C++ code in the driver. * `ALVR Dashboad.exe`: Dashboard binary used to change settings, manage clients, monitor statistics and do installation actions. It can launch SteamVR. * `driver.vrdrivermanifest`: Auxiliary config file used by the driver. At runtime, some other files are created: * `session.json`: This contains unified configuration data used by ALVR, such as settings and client records. * `session_log.txt`: Main log file. Each line is a json structure and represents an event generated by the driver. This gets cleared each time a client connects. * `crash_log.txt`: Auxiliary log file. Same as `session_log.txt`, except only error logs are saved, and does not get cleared. ### Programming languages ALVR is written in multiple languages: Rust, C, C++, HLSL, GLSL. The main language used in the codebase is Rust, which is used for the dashboard, networking, video decoding and audio code. C and C++ are used for graphics, video encoding and SteamVR integration. HLSL is used for graphics shaders on the Windows driver, GLSL is used on the Linux driver and the client. Moving forward, more code will be rewritten from C/C++ to Rust and HLSL code will be moved to GLSL or WGSL. Rust is a system programming language focused on memory safety and ease of use. It is as performant as C++ but Rust code is less likely to be affected by runtime bugs. The prime feature Rust feature used by ALVR is enums, that correspond to tagged unions in C++. Rust's enum is a data type that stores different kinds of data, but only one type can be accessed at a time. For example the type `Result` can contain either an `Ok` value or an `Err` value but not both. Together with pattern matching, this is the foundation of error management in Rust applications. ### Source code organization ALVR code is hosted in a monorepo. This is an overview of the git tree: * `.github/`: Contains scripts used by the GitHub CI. * `alvr/`: Each subfolder is a Rust crate ("crate" means a code library or executable). * `audio/`: Utility crate hosting audio related code shared by client and driver. * `client_core/`: Platform agnostic code for the client. It is used as a Rust library for `alvr_client_openxr` and can also compiled to a C ABI shared library with a .h header for integration with other projects. * `client_mock/`: Client mock implemented as a thin wrapper around `alvr_client_core`. * `client_openxr/`: Client implementation using OpenXR, compiled to a APK binary. * `common/`: Some common code shared by other crates. It contains code for versioning, logging, struct primitives, and OpenXR paths. * `dashboard/`: The dashboard application. * `events/`: Utility crate hosting code related to events. * `filesystem/`: Utility crate hosting code for filesystem abstraction between Windows and Linux. * `packets/`: Utility crate containing packet definitions for communication between client, driver and dashboard. * `server/`: The driver shared library loaded by SteamVR. * `server_io/`: Common functionality shared by dashboard and driver, for interaction with the host system. This allows dashboard and driver to work independently from each other. * `session/`: Utility crate related to session file and data management. * `sockets/`: Utility crate shared by client and driver with socket and protocol implementation. * `vrcompositor_wrapper/`: Small script used on Linux to correctly load the ALVR Vulkan layer by SteamVR. * `vulkan_layer/`: Vulkan WSI layer used on Linux to work around limitations of the OpenVR API on Linux. This is mostly patchwork and hopefully will be removed in the future. * `xtask/`: Utility CLI hosting a variety of scripts for environment setting, building, and packaging ALVR. Should be called with `cargo xtask`. * `resources/`: resources for the README. * `wiki/`: Contains the source for the Github ALVR wiki. Changes are mirrored to the actual wiki once committed. * `about.toml`: Controls what dependency licenses are allowed in the codebase, and helps with generating the licenses file in the packaged ALVR streamer. * `Cargo.lock`: Contains versioning information about Rust dependencies used by ALVR. * `Cargo.toml`: Defines the list of Rust crates contained in the repository, and hosts some other workspace-level Rust configuration. ## Logging and error management Logging is split into interface and implementation. The interface is defined in `alvr/common/src/logging.rs`, the implementations are defined in each binary crate as `logging_backend.rs`. ALVR logging system is based on the crate [log](https://crates.io/crates/log). `log` is already very powerful on its own, since its macros can collect messages, file and line number of the invocation. ALVR defines some structures, macros and functions to ease error management. The base type used for error management is `StrResult` that is an alias for `Result`. Read more about Rust's Result type [here](https://doc.rust-lang.org/std/result/). There are many ways of logging in ALVR, each one for different use-cases. To make use of them you should add `use alvr_common::prelude::*` at the top of the Rust source file. * `error!()`, `warn!()`, `info!()`, `debug!()` (reexported macros from the `log` crate). Log is processed depending on the logging backend. * `show_e()` and `show_w()` are used to log a string message, additionally showing a popup. * `show_err()`, `show_warn()` work similarly to `show_e()` and `show_w()`, but they accept a `Result<>` and log only if the result is `Err()`. * `fmt_e!()` adds tracing information to a message and produces a `Err()`, that can be returned. * `err!()` and `enone!()` used respectively with .`.map_err()` and `.ok_or_else()`, to map a `Result` or `Option` to a `StrResult`, adding tracing information. * Some other similarly named functions and macros with similar functionality ### The event system Events are messages used internally in the driver and sent to dashboard instances. Events are generated with `send_event()` and is implemented on top of the logging system. This is the layout of `Event`, in JSON form ```json { "timestamp": "", "event_type": { "id": "", "content": { } } } ``` Log is a special kind of event: ```json { "timestamp": "", "event_type": { "id": "Log", "content": { "severity": "Error or Warn or Info or Debug", "content": "" } } } ``` The driver logs events in JSON form to `session.json`, one per line. Currently its use is limited, but eventually this will replace the current logging system, and logging will be built on top of the event system. The goal is to create a unified star-shaped network where each client and dashboard instance sends events to the server and the server broadcasts events to all other clients and dashboard instances. This should also unify the way the server communicates with clients and dashboards, making the dashboard just another client. ## Session and settings ALVR uses a unified configuration file, that is `session.json`. It is generated the first time ALVR is launched. This file contains the following top-level fields: * `"server_version"`: the current version of the streamer. It helps during a version upgrade. * `"drivers_backup"`: temporary storage for SteamVR driver paths. Used by the dashboard. * `"openvr_config"`: contains a list of settings that have been checked for a diff. It is used by C++ code inside the driver. * `"client_connections"`: contains entries corresponding to known clients. * `"session_settings"`: all ALVR settings, laid in a tree structure. ### Procedural generation of code and UI ALVR lays out settings in a tree-like structure, in a way that the code itself can efficiently make use of. Settings can contain variants (in `session.json` are specified in PascalCase), which represent mutually exclusive options. ALVR uses the macro `SettingsSchema` in the `settings-schema` crate to generate auxiliary code, ie a schema and the "default representation" of the settings. This is a crate created specifically for ALVR but can be used for other projects too. The schema is made of nested `SchemaNode`s that contain metadata. Some of the metadata is specified directly inside inline attributes in structures and enums. The "default representation" (the type names are generated by concatenating the structure/enum name with `Default`), are structures that can hold settings in a way no not lose information about unselected variants; enums are converted to structs and variants that hold a value are converted to fields. The main goal of this is to meet the user expectation of not losing nested configuration when changing some options. The default representation is exactly what is saved inside `session.json` in `"session_settings"`. Info about the various types of schema nodes can be found [here](https://github.com/zarik5/settings-schema-rs). The dashboard makes use of schema metadata and the default representation to generate the settings UI. The end result is that the settings UI layout closely matches the structures used internally in the code, and this helps understanding the inner workings of the code. When upgrading ALVR, the session might have a slightly different layout, usually some settings might have been added/removed/moved/renamed. ALVR is able to handle this by doing an extrapolation process: it starts from the default session, and replace values taken from the old session file with the help of the settings schema. ## The dashboard The dashboard is the main way of interacting with ALVR. Functionality is organized in tabs. ### The User Interface These are the main components: TODO: Add screenshots * Sidebar: is used to select the tab for the main content page. * Devices tab: used to trust clients or add them manually specifying the IP * Statistics tab: shows graphs for latency and FPS and a summary page * Settings tab: settings page split between `Presets` and `All Settings`. `All Settings` are procedurally generated from a schema. `Presets` are controls that modify other settings. * Installation tab: utilities for installation: setting firewall rules, registering the driver, launching the setup wizard. * Logs tab: shows logs and events in a table. * Debug tab: debugging actions. * About tab: information about ALVR. * Lower sidebar button: can be either "Launch SteamVR" or "Restart SteamVR", depending on the driver connection status * Notification bar: shows log in a non-obstructive way. ### Driver communication The dashboard communicates with the driver in order to update its information and save configuration. This is done through a HTTP API, with base URL `http://localhost:8082`. These are the endpoints: * `/api/dashboard-request`: This is the main URL used by the dashboard to send messages and data to the server. The body contains the specific type and body of the request. * `/api/events`: This endpoint is upgraded to a websocket and is used for listening to events from the driver * `/api/ping`: returns code 200 when the driver is alive. The dashboard retains some functionality when the driver is not launched. It can manage settings, clients and perform installation actions, but clients cannot be discovered. Once The driver is launched all these actions are performed by the server, requested with the HTTP API. This mechanism ensures that there are no data races. ### Driver lifecycle The dashboard is able to launch and restart SteamVR, in order to manage the driver's lifecycle. The driver launch procedure is as follows: * The driver is registered according to the "Driver launch action" setting, if needed. By default, current SteamVR drivers are unregistered and backed up inside `session.json`. * On Linux, the vrcompositor wrapper is installed if needed * SteamVR is launched. Once the drivers shuts down, if there are backed up drivers, these are restored. The driver restart procedure is as follows: * The dashboard notifies the driver that it should be restarted. * The driver sends a request for restart to the dashboard. * The driver asks SteamVR to shutdown, never unregistering drivers. * The dashboard waits for SteamVR to shutdown, otherwise killing it after a timeout. * The dashboard relaunches SteamVR. This might seem unnecessarily complicated. The reason for the first message round trip is to plug-in to the existing restarting system used by settings invalidation, which is invoked from the driver itself. The reason which the driver cannot be autonomous in restarting is because any auxiliary process spawned by the driver will block SteamVR shutdown or leave it in a zombie state. ## The streaming pipeline: Overview The goal of ALVR is to bridge input and output of a PCVR application to a remote headset. In order to do this ALVR implements pipelines to handle input, video and audio. The tracking-video pipeline (as known as the motion-to-photon pipeline) is the most complex one and it can be summarized in the following steps: * Poll tracking data on the client * Send tracking to the driver * Execute the PCVR game logic and render layers * Compose layers into a frame * Encode the video frame * Send the encoded video frame to the client through the network * Decode the video frame on the client * Perform more compositor transformations * Submit the frame to the VR runtime * The runtime renders the frame during a vsync. ## Client-driver communication ALVR uses a custom protocol for client-driver communication. ALVR supports UDP and TCP transports. USB connection is supported although not as a first class feature; you can read more about it [here](https://github.com/alvr-org/ALVR/wiki/ALVR-wired-setup-(ALVR-over-USB)). ### Discovery Usually the first step to establish a connection is discovery. When the server discovers a client it shows it in the "New devices" section in the Devices tab. The user can then trust the client and the connection is established. ALVR uses a UDP socket at 9943 for discovery. The client broadcasts a packet and waits for the driver to respond. It's the client that broadcasts and it's the driver that then asks for a connection: this is because of the balance in responsibility of the two peers. The client becomes the portal though a PC, that can contain sensitive data. For this reason the server has to trust the client before initiating the connection. This is the layout of the discovery packet | Prefix | Protocol ID | Hostname | | :---------------: | :---------: | :------: | | "ALVR" + 0x0 x 12 | 8 bytes | 32 bytes | * The prefix is used to filter packets and ensure a packet is really sent by an ALVR client * The protocol ID is a unique version identifier calculated from the semver version of the client. If the client version is *semver-compatible* with the streamer, the protocol ID will match. * Hostname: the hostname is a unique identifier for a client. When a client is launched for the first time, an hostname is chosen and it persists for then successive launches. It is reset when the app is upgraded or downgraded. The format of the packet can change between major versions, but the prefix must remain unchanged, and the protocol ID must be 8 bytes. ### Streaming ALVR uses two sockets for streaming: the control socket and stream socket. Currently these are implemented with async code; there's a plan to move this back to sync code. The control socket uses the TCP transport; it is used to exchange small messages between client and server, ALVR requires TCP to ensure reliability. The stream socket can use UDP or TCP; it is used to send large packets and/or packets that do not require reliability, ALVR is robust to packet losses and packet reordering. The specific packet format used over the network is not clearly defined since ALVR uses multiple abstraction layers to manipulate the data (bincode, tokio Length Delimited Coding). Furthermore, packets are broken up into shards to ensure they can support the MTU when using UDP. Since the amount of data streamed is large, the socket buffer size is increased both on the driver side and on the client. ## SteamVR driver The driver is the component responsible for most of the streamer functionality. It is implemented as a shared library loaded by SteamVR. It implements the [OpenVR API](https://github.com/ValveSoftware/openvr) in order to interface with SteamVR. Using the OpenVR API, ALVR pushes tracking and button data to SteamVR using `vr::VRServerDriverHost()->TrackedDevicePoseUpdated()`. SteamVR then returns a rendered game frame with associated pose used for rendering. On Windows, frames are retrieved implementing the `IVRDriverDirectModeComponent` interface: SteamVR calls `IVRDriverDirectModeComponent::Present()`. On Linux this API doesn't work, and so ALVR uses a WSI Vulkan layer to intercept display driver calls done by vrcompositor. The pose associated to the frame is obtained from the vrcompositor execution stack with the help of libunwind. ## Client and driver compositors ALVR is essentially a bridge between PC and headset that transmits tracking, audio and video. But it also implements some additional features to improve image quality and streaming performance. To this goal, ALVR implements Fixed Foveated Rendering (FFR) and color correction. The client compositor is implemented in OpenGL, while on the server it's either implemented with DirectX 11 on Windows or Vulkan on Linux. There are plans to move all compositor code to the graphics abstraction layer [wgpu](https://github.com/gfx-rs/wgpu), mainly for unifying the codebase. It's important to note that ALVR's compositors are separate from the headset runtime compositor and SteamVR compositors. The headset runtime compositor is part of the headset operative system and controls compositing between different apps and overlays, and prepares the image for display (with lens distortion correction, chroma aberration correction, mura and ghosting correction). On the driver side, on Windows ALVR takes responsibility for compositing layers returned by SteamVR. The only responsibility of SteamVR is converting the frame into a valid DXGI texture if the game uses OpenGL or Vulkan graphics. On Linux ALVR grabs Vulkan frames that are already composited by vrcompostor. This introduced additional challenges since vrcompositor implements async reprojection which disrupts our head tracking mechanism. ### Foveated encoding Foveated rendering is a technique where frame images are individually compressed in a way that the human eye barely detects the compression. Particularly, the center of the image is kept at original resolution, and the rest is compressed. ALVR refers to foveated rendering as "Foveated encoding" to clarify its scope. In native standalone or PCVR apps, foveated rendering reduces the load on the GPU by rendering parts of the image ar lower resolution. In ALVR's case frames are still rendered at full resolution, but are then "encoded" (compressing the outskirts of the image) before actually encoding and transmitting them. The image is then reexpanded on the client side after decoding and before display. Currently ALVR supports only fixed foveation, but support for tracked eye foveation is planned. In its history, ALVR implemented different algorithms for foveated encoding. The first one is "Warp", where the image is compressed in an elliptical pattern using the tangent function to define the compression ratio radially. A problem with algorithm is that it causes the image to become blurry. [Here](https://www.shadertoy.com/view/3l2GRR) is a demo of this algorithm in action. The second algorithm used was "Slices" where the image is sliced up into 9 sections (center, edges, corners), compressed to different degrees and the re-packed together into a single rectangular frame. The main issue with this algorithm was its complexity. You can find a demo [here](https://www.shadertoy.com/view/WddGz8). The current algorithm in use is reimplementation of Oculus AADT (Axis-Aligned Distorted Transfer), which simply compresses the lateral edges of the image horizontally and the vertical edged vertically. This algorithm has less compression power but it's much simpler and less taxing on the Quest's GPU. ### Color correction Color correction is implemented on the server and adds simple brightness, contrast, saturation, gamma and sharpening controls. It's implemented on the server for performance reasons and to avoid amplifying image artifacts caused by transcoding. ## Video transcoding To be able to send frames from driver to client through the network, they have to be compressed since current WiFi technology doesn't allow to send the amount of data of raw frames. Doing a quick conservative calculation, let's say we have 2 x 2048x2048 eye images, 3 color channels, 8 bits per channel, sent 72 times per second, that would amount to almost 15 Gbps. ALVR uses h264 and HEVC video codecs for compression. These codecs are chosen since they have hardware decoding support on Android and generally hardware encoding support on the PC side. On Windows, the driver uses NvEnc for Nvidia GPUs and AMF for AMD GPUS; on Linux ALVR supports VAAPI, NvEnc and AMF through FFmpeg. In case the GPU doesn't support hardware encoding, on both Windows and Linux ALVR supports software encoding with x264 (through FFmpeg), although the performance is often insufficient for a smooth experience. The client supports only MediaCodec, which is the API to access hardware video codecs on Android. h264 and HEVC codecs compression works on the assumption that consecutive frames are similar to each other. Each frame is reconstructed from past frames + some small additional data. For this reason, packet losses may cause glitches that persist many frames after the missing frame. When ALVR detects packet losses, it requests a new IDR frame from the encoder. A IDR frame is a packet that contains all the information to build a whole frame by itself; the encoder will ensure that successive frames will not rely on older frames than the last requested IDR. ## Audio Game audio is captured on the PC and sent to the client, and microphone audio is captured on the client and sent to the PC. Windows and Linux implementation once again differ. On Windows, game audio is captured from a loopback device; microphone is is sent to virtual audio cable software to expose audio data from a (virtual) input device. On Linux the microphone does not work out-of-the-box, but there is a bash script available for creating and plugging into pipewire audio devices. Unlike for video, audio is sent as a raw PCM waveform and new packets do not rely on old packets. But packet losses may still cause popping, which happens when there is a sudden jump in the waveform. To mitigate this, when ALVR detects a packet loss (or a buffer overflow or underflow) it will render a fadeout or cross-fade. ## Tracking and display timing Handling head and controller tracking is tricky for VR applications, and even more for VR streaming applications. In a normal native VR application, tracking is polled at the beginning of the rendering cycle, it is used to render the eye views from a certain perspective and render the controller or hand models. When the game finished rendering the frame it submits it to the VR runtime which will display it on screen. From the time tracking is polled and the frame is displayed on screen, 1 or more frame durations may have passed (for example at 72fps the frame duration is 13ms). Our eyes are very sensitive to latency, especially for orientation, so VR runtimes implement image reprojection (Oculus calls it Asynchronous Time Warp). Reprojection works by rendering the frame rotated in 3D to compensate for the difference in orientation between the tracking pose polled at the beginning of the rendering cycle and the actual pose of the headset at the time of vsync when the image should be pushed to the display. To be able to correctly rotate the image, the runtime will also need to know the timestamp used for polling tracking, which can be the time of poll, or better, the predicted time of the vsync. If a time in the future is used for tracking poll, the polled tracking will be extrapolated. For VR streaming applications, the pipeline is similar, except that tracking is polled for a more distant point in the future, to compensate for the whole transcoding pipeline, and it's not trivial to decide on how much to predict in the future. ALVR calculates the prediction offset by reading how much time passes between the tracking poll time and the time a frame rendered with the same tracking is submitted. These interval samples are averaged and then used for future tracking polls. (To calculate the correct total latency you also need to add the VR runtime compositor latency, which in the dashboard latency graph is shown as "Client VSync"). On the streamer side, ALVR needs to workaround a OpenVR API limitation. SteamVR returns frames with its pose, but then ALVR is responsible of matching the pose with one of the poses submitted previously and re-match its timestamp. ## Other streams There are some other kinds of data which can be streamed without requiring any special timing. These are button presses and haptics, respectively sent from client to driver and from driver to client. ## Upcoming ### Phase sync Phase sync is not a single algorithm but many that share similar objectives, reducing latency or jitter in the rendering/streaming pipeline. The term "phase sync" comes from Oculus, that describes its algorithm for reducing latency in its OpenXR runtime by starting the rendering cycle as late las possible to reduce waiting time before the vsync. In general, a phase sync algorithm is composed of two parts: a queue that holds data resources or pointers, and a statistical model to predict event times. The statistical model is fed with duration or other kinds of timing samples and as output it returns a refined time prediction for a recurring event. The statistical model could be simple and just aim for a average-controlled event, or more complex that aims for submitting for a deadline; the second case needs to take into account the variance of the timing samples. Unlike Oculus implementation, these statistical models can be highly configurable to tune the target mean or target variance. There are a few phase sync algorithms planned to be implemented: frame submission timing (to reduce frame queueing on the client, controlled by shifting the phase of the driver rendering cycle), SteamVR tracking submission timing (to make sure SteamVR is using exactly the tracking sample we want) and tracking poll timing (to reduce queuing on the server side). ## Sliced encoding Sliced encoding is another algorithm showcased by Oculus and it's about reducing latency by parallelizing work. In a simple streaming pipeline, frames are processed sequentially: rendering, then encoding, then transmission, then decoding. There is already some degree of parallelism, as rendering, encoding, transmission, and decoding can happen at the same time. Sliced encoding can help in reducing encoding and decoding time, as the frames are split into "slices". This allows for more efficient utilization of hardware encoders/decoders, or even use hardware and software codecs in parallel. It's crucial to note that network latency cannot be optimized. Given the constraint of network, sliced encoding can reduce waiting times between encoder/transmission and transmission/decoder as each encoded slice can be transmitted immediately and doesn't have to wait for the rest of the frame to be encoded (and a similar reasoning applies for the decoding side). ================================================ FILE: wiki/Information-and-Recommendations.md ================================================ ## PC - A high-end PC is a requirement; ALVR is not a cheap alternative to a PCVR HMD - ALVR resolution configuration and SteamVR multi-sampling may be used to influence quality in favor of performance or vice-versa. - Frequent dropped frames can cause a poor experience on ALVR; this can be verified using a tool such as [OVR Advanced Settings](https://github.com/OpenVR-Advanced-Settings/OpenVR-AdvancedSettings). - Higher bit-rates will cause higher latency. - Ensure all relevant software is up to date - especially graphics and network drivers. - A good starting point is 100% resolution (`Very low` resolution preset) and 30mbit constant bitrate. In this config it should be very smooth with almost no lag or packet loss; packet loss seen at this point is likely a result of network issues. ## Network - A wired connection from the PC to the network is **strongly recommended**. - A modern mid to high-end router and / or access point supporting at least 802.11ac (ideally 802.11ax) is recommended. ## Wireless ### General WiFi configuration best practices - Any device that can be wired should be - each wireless device slows down the overall wireless network - Devices should have the least amount of obstructions and be as close to the access point or router as possible - Any other wireless networks (ex: a printer's default wireless network) should be disabled; each network slows others down - Any devices that do not need high speeds but support them (ex: a thermostat) should use 2.4Ghz; often middle and higher end access points and routers support methods to "force" clients to use 2.4Ghz, and some can even perform this automatically based on signal strength and connection speed - Only WiFi revisions which are necessary should be enabled; older standards such as 802.11b, 802.11g, and to a lesser extent, 802.11n, will slow down all clients - Devices that require high speeds (such as standalone headset) should use: - 5GHz only - The newest WiFi specifications (802.11ax, followed by 802.11ac) - In most environments, the largest channel width possible (160MHz for 802.11ax, 80MHz in practice for 802.11ac) (**note: some vendors do not set this to the maximum by default**) - The lowest utilization, followed by the lowest channel number (sub-frequency) possible - **Manually selecting channels should only be done in places with extreme noise, or on older, lower quality, or ISP provided access points or routers** - modern mid to high-end routers and access points should optimize their channels fairly well, and as a result of other routers and clients "channel hopping", static settings are often less optimal - If a specific WiFi channel range is absolutely necessary, use a WiFi scanning tool on a phone or PC to determine the least used channels - mid to high-end access points and routers may provide an interface for this as well, however, this sometimes causes a disconnect when scanning - **Manually selecting wifi signal strength should only be done in places with extreme noise** - modern routers and access points do this well, and it is a complex task - If a specific transmit power is necessary, keep in mind that stronger is not always better - as transmit power increases, distortion may increase (leading to *lower* speeds), battery life of clients may increase (due to the higher power requested by the access point or router), and issues with sticky clients (devices which stay connected to wifi even with bad signal) may appear - If you have a significant number of devices, some routers and access points support features such as airtime fairness, which help to limit the amount of airtime slower clients take, improving the performance of higher speed clients ### Things to keep in mind when configuring a wireless network and devices - All devices on the same frequency impact each other (**including other WiFi networks on the same channel**) because only one device can transmit or receive data at a time, meaning that: - If one device utilizes WiFi heavily it will impact the latency and throughput of all other clients - If a slow device is connected, it can still take a significant amount of "airtime" (time for that dedicated client to transmit / receive data to the access point or router), even though it does so at a slower rate than other clients - Each connected device requires additional time, regardless of whether it is actively in use (and often devices send small amounts of data when idle for things such as NTP and DHCP) - WiFi is [half duplex](https://en.wikipedia.org/wiki/Duplex_(telecommunications)#Half_duplex) by nature of it being radio frequency, meaning data can only ever be transmitted **or** received on the same frequency, not both at the same time; twisted pair (copper ethernet cable) is full duplex - Wireless frequency bands (ex: 2.4Ghz, 5Ghz) have separate channels that can be statically assigned if needed, but **these are not mutually exclusive, meaning the channels overlap significantly and interfere with each other** - Different regions of the world support different channels (sub-frequencies); devices sold in these regions are generally locked to those channels (ex: in the US, 2.4Ghz channels 12 - 13 are low power only, and channel 14 is military and EMS use only) - Different wireless devices support different frequencies, standards, speeds, and features; using these to your advantage is key to getting best performance ## Routing / Switching / Firewalling / General Info - Ideally the headset and streamer should exist on the same logical (layer 2) network and subnet - this allows for no routing overhead, and the correct function of device discovery via [mDNS](https://en.wikipedia.org/wiki/Multicast_DNS) - Twisted pair (normal copper ethernet cables) should never be run alongside power cables - this can cause signal noise and result in frame loss and lowered auto-negotiation speeds - High quality CAT5E or higher (ideally CAT6A or CAT7) gigabit+ cabling should be used for modern networks - In some cases firewall, anti-virus, malware, or EDR (enhanced detection and response) software may interfere with network traffic - Windows Defender and Sophos Endpoint Protection are reported to work without issue - Pause frames should be disabled where possible, as these introduce additional latency and buffering *** Someone did a few blog-posts on some of the points: Some points came from [FingrMastr](https://github.com/FingrMastr) ## Linux ### Encoder requirements ALVR uses FFmpeg for all encoders, so you will need to make sure the encoder of your choice works with FFmpeg. Always consult Log tab in dashboard, it will tell you the reason why an encoder failed to initialize. ### VAAPI (AMD/Intel GPUs) Requires *libva* and appropriate driver for your GPU. Check codec support with `vainfo`: ```sh $ vainfo 130 ↵ !10090 Trying display: wayland vainfo: VA-API version: 1.16 (libva 2.16.0) vainfo: Driver version: Mesa Gallium driver 23.0.0-devel for Radeon RX 7900 XTX (gfx1100, LLVM 16.0.0, DRM 3.49, 6.1.1-zen1-1-zen) vainfo: Supported profile and entrypoints VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileJPEGBaseline : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc ``` *VAProfileH264High, VAProfileHEVCMain, VAProfileHEVCMain10* encoders (VAEntrypointEncSlice) required. If you don't see those in your output, your driver install is incorrect or your distribution decided to build *mesa* without non-free codecs. #### Test ffmpeg commands (VAAPI) ```sh # H264 ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc -t 30 -vf 'format=nv12,hwupload' -c:v h264_vaapi vaapi-h264.mp4 # HEVC ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc -t 30 -vf 'format=nv12,hwupload' -c:v hevc_vaapi vaapi-hevc.mp4 ``` ### NVENC (Nvidia) Requires *libcuda*. #### Test ffmpeg commands (Nvidia) ```sh # H264 ffmpeg -f lavfi -i testsrc -t 30 -vf 'format=nv12,hwupload' -c:v h264_nvenc nvenc-h264.mp4 # HEVC ffmpeg -f lavfi -i testsrc -t 30 -vf 'format=nv12,hwupload' -c:v hevc_nvenc nvenc-hevc.mp4 ``` ### Software (any GPUs) Software encoder is mainly used as a fallback and as such should work on all GPUs without any requirements. Only H264 encoding is currently supported. ================================================ FILE: wiki/Installation-guide.md ================================================ ## Launcher (BETA) Launcher will allow you to manage old, current and new installations of ALVR streamer and allow to automatically install and upgrade to specific ALVR app version on headset ### Installation * Download `alvr_launcher_windows.zip` (on Windows) or `alvr_launcher_linux.tar.gz` (on Linux) from the release [download page](https://github.com/alvr-org/ALVR/releases/latest) and extract into a path that contains only ASCII characters (english only) and has edit permissions without administrator or root rights. * Run `ALVR Launcher.exe` (on Windows) or `alvr_launcher_linux/ALVR Launcher` (on Linux) * Press `Add version` button * For default installation keep channel and version as is and press `Install` * Wait until it finishes downloading, installing (depends on your connection) * To install ALVR app on headset, use button `Install APK` * In the list, to open streamer app (PC) press `Launch`. You will be greeted with a setup wizard. Follow the setup to set the firewall rules and other settings. ### Usage * Before launching SteamVR through ALVR, please install it. First time launch will result in steamvr being blank and alvr will not work - close it and start again. It will have registered driver and should work. * Launch ALVR app on your headset. While the headset screen is on, click `Trust` next to the device entry (in the ALVR streamer app on PC, in the `Devices` tab) to start streaming. * You can change settings on the PC in the `Settings` tab. Most of the settings require to restart SteamVR to be applied. Use the apposite button on the bottom right corner. For any problem visit the [Troubleshooting page](https://github.com/alvr-org/ALVR/wiki/Troubleshooting). ## Microphone Setup on Windows To use your microphone in ALVR on Windows you need to install **Virtual Audio Cable** (or equivalent software). However if Virtual Audio Cable is already installed but not working with ALVR **or if you encounter any issues**, it's worth following these steps to reinstall and configure it properly. ### **1. Install or Reinstall Virtual Audio Cable** 1. **Download** the latest Lite version of [Virtual Audio Cable](https://software.muzychenko.net/freeware/vac470lite.zip). 2. **Extract** the ZIP archive. 3. Open the extracted folder and run **"setup64.exe"** as administrator. ### **2. Configure Windows Sound Settings** 1. **Open** Windows Sound Settings (`Win + I` → "Sound"). 2. **Under Output Devices**: - **Do not set any "Virtual Audio Cable" as the default output**, or you’ll hear yourself. Select your headphone or whatever you're using. ### **3. Configure ALVR** 1. **Open ALVR** and go to **Settings**. 2. Set **Headset Speaker** → **System Default**. 3. Set **Headset Microphone** → **Automatic** or **Virtual Audio Cable**. ## Advanced installation ### Installing app using Sidequest * Install SideQuest on your PC and enable developer mode on the headset. You can follow [this guide](https://sidequestvr.com/setup-howto). * Connect your headset to Sidequest. If you have Quest, Pico, and other compatible device download the ALVR app [here](https://sidequestvr.com/app/9) ### Manually installing ALVR streamer There is also a portable version for the PC that requires more manual steps to make it work. #### Windows * Download `alvr_streamer_windows.zip` from the latest release [download page](https://github.com/alvr-org/ALVR/releases/latest). * Unzip into a path that contains only ASCII characters and has edit permissions without administrator rights. * Run #### Linux * Download `alvr_streamer_linux.tar.gz` from the release [download page](https://github.com/alvr-org/ALVR/releases/latest), extract it. * Run `bin/alvr_dashboard` #### Nightly If you want to get new features early or you want to help with testing you can install a nightly version. Download the latest nightly streamer [here](https://github.com/alvr-org/ALVR-nightly/releases/latest). Since nightly releases can be unstable, always use matching versions for PC and headset. They are updated once a day. ### Arch Linux (AUR) * Install `rustup` and a rust toolchain, if you don't have it: . * Install [alvr](https://aur.archlinux.org/packages/alvr)AUR (stable, amdgpu), or [alvr-nvidia](https://aur.archlinux.org/packages/alvr-nvidia)AUR (stable, nvidia), or [alvr-git](https://aur.archlinux.org/packages/alvr-git)AUR(nightly, unstable) * Install SteamVR, **launch it once** then close it. * Run `alvr_dashboard` or ALVR from your DE's application launcher. ### Flatpak For Flatpak users, refer to the instructions [here](https://github.com/alvr-org/ALVR/wiki/Installing-ALVR-and-using-SteamVR-on-Linux-through-Flatpak) ## Advanced usage ### Use ALVR together with third-party drivers By default ALVR disables other SteamVR drivers before starting. Among these drivers there is [Driver4VR](https://www.driver4vr.com/) for full body tracking. ALVR disables these drivers to maximize compatibility with every PC setup. You can disable this behavior by manually registering the ALVR driver. Go to the `installation` tab and click on `Register ALVR driver`. The next time you launch ALVR you will be able to use the other drivers concurrently. ### Launch ALVR together with SteamVR You can skip the ALVR Dashboard and open ALVR automatically together with SteamVR. **Note:** You can only do that while SteamVR is not already running. Otherwise driver might be unregistered on shutdown. Open ALVR, go to the `Installation` tab and click on `Register ALVR driver`. ### Connect headset to PC via a USB Cable Check out the guide [here](https://github.com/alvr-org/ALVR/wiki/ALVR-wired-setup-(ALVR-over-USB)). ================================================ FILE: wiki/Installing-ALVR-and-using-SteamVR-on-Linux-through-Flatpak.md ================================================ ## Disclaimer 1. Flatpak suppport is experimental - but it does seem to work. Some manual steps are needed! 2. Native Linux SteamVR utility applications such as OpenVRAS are not supported nor tested, use at your own risk 3. Firewall configuration does not work 4. Any scripts that affect the host will run within the sandbox 5. Sometimes, a new instance of Steam will launch when launching the dashboard. To fix this, close both ALVR and Steam then launch Steam. As soon as Steam opens to the storefront, launch the ALVR dashboard. 6. User must setup xdg shortcut themselves - see below. Without an xdg entry the launcher has to be run from terminal. ```sh flatpak run --command=alvr_launcher com.valvesoftware.Steam ``` 8. This does seem to work with both steam flatpak and native steam - it calls via xdg-open. But it is not recommended to have both versions of steam installed as this creates ambiguity. ## Dependencies First, flatpak must be installed from your distro's repositories. Refer to [this page](https://flatpak.org/setup/) to find the instructions for your distro. ## Setup Flatpak steam needs extra step compared to native steam. After installing SteamVR, run the following command: ```sh sudo setcap CAP_SYS_NICE+eip ~/.var/app/com.valvesoftware.Steam/data/Steam/steamapps/common/SteamVR/bin/linux64/vrcompositor-launcher ``` This command is normally run by SteamVR, but due to the lack of sudo access within the Flatpak sandbox, it must be run outside of the Flatpak sandbox. After running the command, run SteamVR once then close it. ### steamvr custom launch options At the time of writing steamvr needs special options to work on linux - this applies to both the flatpak version and native. The flatpak uses a slightly different path is the only difference. Paths below assume steam has been installed in the "normal" location - if your steam is in a different place then adjust paths as appropriate. For flatpak steam ``` ~/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command% ``` For native steam ``` ~/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command% ``` ### failed to create pipewire errors This is most likely caused by missing pipewire socket which is required for audio to work. It's also possible to have this error even after applying fixes bellow. Which might suggest issues on host system (mainly - missing pipewire, a requirement for audio to work) To fix the most likely cause, please follow guidance bellow #### Using terminal Open terminal and paste commands from bellow depending on type of your Flatpak Steam installation: On user installation of steam flatpak: `flatpak override --user --filesystem="xdg-run/pipewire-0" com.valvesoftware.Steam` On system installation of steam flatpak: `flatpak override --filesystem="xdg-run/pipewire-0" com.valvesoftware.Steam` Press enter to apply it and restart SteamVR (and close Flatpak Steam) from ALVR to apply the fix. #### Using Flatseal GUI app for managing Flatpak application permissions: [Flathub](https://flathub.org/apps/com.github.tchx84.Flatseal) 1. Find Steam in app's list on the left, left click on it 2. Inside Steam app on the right -> scroll down to Filesystem section 3. In Filesystem section -> find Other files sub-section 4. In Other files subsection -> add new entry with content: `xdg-run/pipewire-0` You should also see some other permissions there `xdg-music:ro`, `xdg-pictures:ro` and maybe more for other integration (like discord). ## Install Download `com.valvesoftware.Steam.Utility.alvr.flatpak` file from one of the latest [nightly](https://github.com/alvr-org/ALVR-nightly/releases) that contains flatpak bundle and install like so: ```sh flatpak install --user com.valvesoftware.Steam.Utility.alvr.flatpak ``` ## Notes ### Running the launcher It's recommended that user sets up an xdg shortcut - but the launcher can also be run from terminal via the following command: ```sh flatpak run --command=alvr_launcher com.valvesoftware.Steam ``` An icon and desktop file named `com.valvesoftware.Steam.Utility.alvr.desktop` is supplied within the `alvr/xtask/flatpak` directory. Move this to where other desktop files are located on your system in order to run the dashboard without the terminal. ```sh # systemwide shortcut # sudo cp com.valvesoftware.Steam.Utility.alvr.desktop /var/lib/flatpak/exports/share/applications/ # users local folder cp com.valvesoftware.Steam.Utility.alvr.desktop $HOME/.local/share/flatpak/exports/share/applications/ # install icon as well xdg-icon-resource install --size 256 alvr_icon.png application-alvr-launcher ``` The shortcut may not appear until desktop session is refreshed (e.g. log off then back on) ### EXPERIMENTAL - APK install via flatpak launcher First need to setup adb on host, and enable usb debugging on device. Verify that devices shows up when you run "adb devices" and is authorised. Script assumes that user has AndroidStudio installed with keys in default location ($HOME/.android/adbkey.pub) - change if necessary Convenience script is provided in git: run_with_adb_keys.sh It's likely one the keys are exposed to the flatpak in the default location it will work without needing more changes. ``` export ADB_VENDOR_KEYS=~/.android/adbkey.pub flatpak override --user --filesystem=~/.android com.valvesoftware.Steam.Utility.alvr flatpak run --env=ADB_VENDOR_KEYS=$ADB_VENDOR_KEYS --command=alvr_launcher com.valvesoftware.Steam ``` ### Wayland variable causes steamvr error: Make sure the QT_QPA_PLATFORM var allows x11 option - or steamvr freaks out. Launch from terminal to see errors. This can be a problem if you have modified this variable globally to force usage of wayland for some program like GameScope. You can fix this by setting the variable passed to steamvr Example custom launch options for steamvr - including both QT_QPA_PLATFORM and vrmonitor fixes: ``` QT_QPA_PLATFORM=xcb ~/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command% ``` ### Hybrid graphics If using desktop it's recommended to disable igpu - makes things simpler. If using laptop then must pass extra options to ensure dgpu is used. These options are in addition to the others already mentioned. #### Amd/Intel integrated gpu + Amd/Intel discrete gpu Put DRI_PRIME=1 %command% into SteamVR's commandline options and in those of all VR games you intend to play with ALVR. ``` DRI_PRIME=1 QT_QPA_PLATFORM=xcb ~/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command% ``` #### Amd/Intel integrated gpu + Nvidia discrete gpu Put __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia %command% into SteamVR's commandline options and in those of all VR games you intend to play with ALVR. Again - in addition to other options. ``` __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia QT_QPA_PLATFORM=xcb ~/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command% ``` ### Other Applications The support for other applications that are not launched via Steam is non-existent due to the Flatpak sandbox. Various SteamVR utilities such as [WlxOverlay](https://github.com/galister/WlxOverlay) and [OpenVR-AdvancedSettings](https://github.com/OpenVR-Advanced-Settings/OpenVR-AdvancedSettings) cannot run within the Flatpak sandbox due to their usage of AppImage. However, unpacking the supplied AppImage or building the utilities from source and running their binaries from within the sandbox similiarly to `alvr_dashboard` could work, but there is no guarantee that they will work properly. (at time of writing it does work properly) Download wlx-overlay-s appimage. Make it executable (chmod +x Wlx-Overlay-xxx.Appimage). Extract it (./Wlx-Overlay-xxx.Appimage --app-image-extract) Use flatseal or terminal to expose a folder to the steam flatpak (e.g. ~/test, should be in same section as the pipewire fix from above) Copy the extracted files into the exposed folder. Test it from terminal: flatpak run --command=bash com.valvesoftware.Steam (cd ~/test/squasroot-fs && ./Apprun) To make a desktop shortcut, use a command like flatpak run --command=~/test/squashroot-fs/Apprun com.valvesoftware.Steam Some applications such as [Godot](https://godotengine.org) support OpenXR. However, unless they are launched within the Steam Flatpak sandbox, they will not work with the Steam Flatpak. See [here](https://github.com/flathub/com.valvesoftware.Steam/issues/1010) for more details. ================================================ FILE: wiki/Linux-Troubleshooting.md ================================================ ## (! Mandatory, apply fix if not applied yet !) Black screen even when SteamVR shows movement, Dashboard not detecting launched ALVR/SteamVR The Steam runtimes SteamVR runs in break the ALVR driver loaded by SteamVR. This causes the screen to stay black on the headset, an error to be reported that the PipeWire device is missing, or can even result in SteamVR crashing. ### Fix Add `~/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command%` to the launch options of SteamVR (SteamVR -> Manage/Right Click -> Properties -> General -> Launch Options). This path might differ based on your Steam installation; in that case, SteamVR will not start at all. If this is the case, you can figure out the actual path by going to Steam Settings -> Storage. Then pick the storage location with the star emoji (⭐) and take the path directly above the usage statistics. Prepend this path to `steamapps/common/SteamVR/bin/vrmonitor.sh`. Finally, put the resulting path into the SteamVR launch options instead of the original one. ### Hyprland/Sway/wlroots Qt fix If you're on Hyprland, Sway, or another wlroots-based Wayland compositor, you might have to prepend `QT_QPA_PLATFORM=xcb` to your launch options. Related issue: [[BUG] No SteamVR UI on wlroots-based wayland compositors (sway, hyprland, ...) with workaround](https://github.com/ValveSoftware/SteamVR-for-Linux/issues/637). ## The ALVR driver doesn't get detected by SteamVR (even after vrmonitor fix) This can be related to the use of an AUR package on Arch. ### Fix If you're using an Nvidia-based system, ensure you are using a package which supports this (e.g. `alvr-nvidia`). Also try using a launcher (e.g. `alvr-launcher-bin`) or portable .tar.gz release from the Releases page. ## Artifacting, no SteamVR Overlay or graphical glitches in streaming view This could be related to the AMD AMDVLK or AMDGPU-PRO drivers being present on your system. AMDVLK overrides other Vulkan drivers and can cause SteamVR to break. Also to note is that AMD has discontinued the AMDVLK driver, so limited support should be expected if using it. ### Fix First check if AMDVLK or AMDGPU-PRO are installed by seeing if `ls /usr/share/vulkan/icd.d/ | grep -e amd_icd -e amd_pro` shows anything. If so, uninstall AMDVLK and/or the AMDGPU-PRO drivers from your system to use the RADV driver instead. (This method may not catch all installations due to distro variations.) On Arch, first install `vulkan-radeon`, then uninstall other drivers. ## "Failed to create VAAPI encoder" error Gameplay stream appears blocky or crashes, then an error window appears on your desktop saying: > Failed to create VAAPI encoder: Cannot open video encoder codec: Function not implemented. Please make sure you have installed VAAPI runtime. ### Fix For Fedora: * Switch from `mesa-va-drivers` to `mesa-va-drivers-freeworld`. [Guide on how to do so](https://fostips.com/hardware-acceleration-video-fedora/) or [the RPM docs](https://rpmfusion.org/Howto/Multimedia). For Arch (don't use VAAPI for Nvidia): * Follow the steps on [this](https://wiki.archlinux.org/title/Hardware_video_acceleration#Installation) page, then reboot your machine. For other distros (e.g. Manjaro): * Install the nonfree version of the Mesa/VAAPI drivers that include the proprietary codecs needed for H264/HEVC encoding. ## Nvidia driver version requirements ALVR requires the Nvidia driver version 535 or newer and CUDA version 12.1 or newer. If your configuration doesn't meet these requirements, SteamVR or the encoder might not work. ### Fix Install the minimum or newer versions of the Nvidia and CUDA drivers. If errors saying CUDA was not detected persist, try using the latest ALVR nightly release. ## Using ALVR with only integrated graphics Beware that using **only** integrated graphics for running ALVR is highly inadvisable, as in most cases it will lead to very poor performance (even on more powerful devices like the Steam Deck, it's still very slow). Don't expect things to work perfectly in this case either, as some older integrated graphics may simply not have the best Vulkan support and might fail to work at all. ## Hybrid graphics advice ### General advice If you're using a PC and can disable your integrated GPU from the BIOS/UEFI, it's highly advised to do so to avoid multiple problems with handling hybrid graphics. If you're using a laptop and it doesn't allow disabling integrated graphics (in most cases), you'll have to resort to the methods below. ### AMD/Intel integrated GPU + AMD/Intel discrete GPU Prepend `DRI_PRIME=1` to the launch options of SteamVR and of all VR games you intend to play with ALVR. ### AMD/Intel integrated GPU + Nvidia discrete GPU Prepend `__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia` to the launch options of SteamVR and of all VR games you intend to play with ALVR. If this results in errors such as `error in encoder thread: Failed to initialize vulkan frame context: Invalid argument`, then try adding `VK_DRIVER_FILES=/usr/share/vulkan/icd.d/nvidia_icd.json` to the above launch options. - Go to `/usr/share/vulkan/icd.d` and ensure `nvidia_icd.json` exists. It may also be under the name `nvidia_icd.x86_64.json`, in which case you should adjust `VK_DRIVER_FILES` accordingly. - On older distributions, `VK_DRIVER_FILES` may not be available, in which case you should use the deprecated but equivalent `VK_ICD_FILENAMES`. ### SteamVR Dashboard not rendering in VR on Nvidia discrete GPU You may need to run the entire Steam client itself via PRIME render offload. First, ensure the Steam client is completely closed. If Steam is already open, you can do so by clicking the Steam dropdown in the top left and choosing "Exit". Then from a terminal run: `__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia steam-runtime`. ## Wayland When using older Gnome versions (older than 47) under Wayland, issues may be caused by DRM leasing not being available. ### Fix Prepend `WAYLAND_DISPLAY=''` to the SteamVR launch options to force XWayland on SteamVR. ## The view shakes when using SlimeVR This might be fixed in future updates of ALVR. ### Fix Start the SlimeVR Server only after you have connected and gotten an image to ALVR at least once. ## Error 109 SteamVR displays 109 or other errors. ### Fix Start Steam first before starting SteamVR through ALVR. If SteamVR is already started, restart it. ## No audio or microphone Audio and/or microphone are enabled in presets, but you still can't hear audio or no one can hear you. ### Fix Make sure you select `ALVR Audio` and/or `ALVR Microphone` in your device list as default **after** connecting the headset. As soon as the headset is disconnected, the devices will be removed. If you set them as default, they will be automatically selected whenever they show up, and you won't need to do it manually ever again. If you don't appear to have the audio devices, or have PipeWire errors in your logs, ensure you have `pipewire` version 0.3.49 or newer installed by using the command `pipewire --version`. For older Debian (version 11 or older) or Ubuntu-based (version 22.04 or older) distributions, you can check the [pipewire-upstream](https://github.com/pipewire-debian/pipewire-debian) page for instructions on installing newer PipeWire versions. ## Low AMDGPU performance and shutters This might be caused by [[PERF] Subpar GPU performance due to wrong power profile mode · Issue #469 · ValveSoftware/SteamVR-for-Linux · GitHub](https://github.com/ValveSoftware/SteamVR-for-Linux/issues/469). ### Fix Using CoreCtrl is highly advised (install it using your distribution's package management system). In its settings, set your GPU to the VR profile, as well as CPU to the performance profile (if it's an old Ryzen CPU). ## OVR Advanced Settings OVR Advanced Settings is incompatible with ALVR, and will produce a ladder-like latency graph with very bad shifting vision. Disable the OVR Advanced Settings driver, and don't use it with ALVR. ## Bindings not working/high CPU usage due to bindings UI SteamVR can't properly update bindings, open menus, and/or eats too much CPU. This issue is caused by SteamVR's webserver spamming requests that stall the Chromium UI and cause it to use a lot of CPU. ### Fix Apply the following patch: `https://github.com/alvr-org/ALVR-Distrobox-Linux-Guide/blob/main/patch_bindings_spam.sh`. One-liner assuming the default Steam path for Arch, Fedora: `curl -s https://raw.githubusercontent.com/alvr-org/ALVR-Distrobox-Linux-Guide/main/patch_bindings_spam.sh | sh -s ~/.steam/steam/steamapps/common/SteamVR`. ================================================ FILE: wiki/My-game-is-not-working-properly!-Help!.md ================================================ While most games do work without any problems, some do only work partially or not at all. This includes - headset not found - warped image - controller not tracking - buttons not working - ... Most of the time its the overly specific initialization of the game towards a specific headset that breaks the game. For example, Vivecraft broke because ALVR reported the headset manufacturer as "Oculus driver 1.38.0" and not as "Oculus". In general, this is a rather bad practice as all relevant data can be accessed trough SteamVR and the game should not make assumptions based on the manufacturer of the hmd. There are many different fields that a game could require to run. Nonetheless, we want to play and support those games. Problem is, that we don't own all games. This is a Open Source without any funding. We can not buy any games just to fix a bug. In the case of Vivecraft, one user (thanks @Avencore) was generous to gift us a copy and the bug could be fixed. There are no guaranties! Neither on the time it will take nor if the bug will ever be fixed! Please contact us before buying anything. ================================================ FILE: wiki/Other-resources.md ================================================ * Hand tracking OSC for VRChat with ALVR support: https://github.com/A3yuu/FingerTruckerOSC * ALVR in the browser using WebXR and WebCodecs: https://github.com/rinsuki-lab/ALVR/tree/research/alvr-web ================================================ FILE: wiki/Real-time-video-upscaling-experiments.md ================================================ ## Why? The Quest can display a resolution close to 4k. Rendering a game, encoding and decoding these kinds of resolutions is very taxing on both the PC and the Quest. So usually a lower resolution image displayed on the Quest. Ideally the output of such an upscaled image should match the screens pixels 1:1. But because of the Asynchronous Timewarp step this is not possible in the Quest. OVR only accepts undistorted frames. Currently ALVR does no upscaling prior to the image being mapped to an OpenGL texture. This texture gets interpolated to match the screen pixels by OVR. For this process video resolutions above 100% it use bilinear interpolation and for resolutions below 100% it uses nearest neighbor. There's a lot of good info on this topic in this issue: ## Lanczos resampling This traditional upscaling method seems like a good step up from basic bilinear interpolation and is relatively light on GPU resources. A GPL 2 implementation of a Lanczos shader can be found here: ## Neural net image super resolution I did some basic investigations on the feasibility of using AI upscalers to get even better results than traditional signal processing methods. ### Hardware acceleration on the XR2 There seem to be 3 paths towards getting fast NNs running on the Quest's SoC. There is the [Qualcomm Neural Processing SDK](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk/tools), which automatically detects what the capabilities of the system are and picks the right hardware to run the NN on (GPU, DSP, AI accelerator). The [TensorFlow Lite NNAPI delegate](https://www.tensorflow.org/lite/performance/nnapi) relies on hardware and driver support for the Android Neural Networks API. Then there is also the [TensorFlow Lite Hexagon delegate](https://www.tensorflow.org/lite/performance/hexagon_delegate) which specifically targets the Snapdragon DSP. I only tested an example image super-resolution app from the [tensorflow respository](https://github.com/tensorflow/examples/tree/master/lite/examples/super_resolution) in CPU and generic GPU (OpenCL) accelerated modes. Even upscaling tiny 50x50 images took around 500ms with this. Even though better hardware acceleration could improve this I do not expect 100x improvements. The only hope for NN super-resolution to be viable would be to find a significantly faster neural net, which leads us into the next topic. ### Existing neural nets A well established real-time upscaler is [Anime4K](https://github.com/bloc97/Anime4K/). It states that it can achieve 1080p to 2160p upscaling in 3ms on a Vega64 GPU. A [rough estimate](https://uploadvr.com/oculus-quest-2-benchmarks/) puts the Quest 2 at a 10x performance disadvantage compared to such high end desktop GPUs. It doesn't seem entirely impossible to get this to work with some optimizations and lowering of the upscaling quality, but there is more bad news. Anime4K has a rather bad Peak signal-to-noise ratio (PSNR). It can get away with this because the stylized look anime is quite forgiving in being heavily filtered. For an upscaler that has a better PSNR there are many options but very few that can run real-time. The smallest neural net that I could find is [SubPixel-BackProjection](https://github.com/supratikbanerjee/SubPixel-BackProjection_SuperResolution). Tt gets nice results but in my testing took 3 seconds to upscale from 720p to 1080p with CUDA acceleration. Way out of the ballpark for XR2 the chip. So in conclusion, it does not seem like there is enough performance to squeeze out of the XR2 to do do real-time NN upscaling at such high resolutions. We will more likely get better results out of classical techniques. ================================================ FILE: wiki/Roadmap.md ================================================ This post will continue to evolve during ALVR development. ## Long-term goal Create a universal bridge between XR devices. ## What is coming next * Compositor rewrite * **Purpose**: add Linux support for FFR and color correction, preparation for sliced encoding * **Status**: FFE and color correction done on all platforms * Encoder rewrite * **Purpose**: support any OS and hardware with a single API, using [Vulkan video extensions](https://www.khronos.org/blog/an-introduction-to-vulkan-video) * **Status**: blocked by adoption by AMD and Intel, landing of the feature on stable Nvidia drivers * Monado Driver * **Purpose**: support other runtimes with the streamer * **Status**: blocked on refactors Due to the low development capacity, no ETA can be provided. New releases will not have a regular cadence and they do not have scheduled features. ================================================ FILE: wiki/Settings-tutorial.md ================================================ Applicable to ALVR v20. This tutorial will help you find optimal settings for your hardware and network as well as give you some pointers to troubleshoot common configuration issues. ## Prerequisites * You have installed the ALVR streamer on your PC, and the ALVR app on your HMD. * You can launch up to the SteamVR void (or up to the SteamVR home) and are able to launch games. ## Step 1: choose resolution, refresh rate, codec To get a sharp image, you need combination of high resolution, enough sharpening, good bitrate with chosen codec. For example, on good wireless router you can use medium resolution preset (default) with 1.0 sharpening (higher than default) with H.264 set at constant 400-500 mbps, or hevc at costant 100-150 mbps. In wired case, you can go all the way to 800-1000 mbps constant bitrate on H.264. Next, choose a refresh rate. Obviously higher is better, but on weaker/older hardware it's often preferable to use a lower setting that gives consistent results. For the Quest 2, 120 Hz has to be enabled in its settings. A few notes on codec choices: * AV1 works only on latest gen gpus (Nvidia RTX 4xxx and AMD Radeon RX 7xxx) and on Quest 3 only. * HEVC/H.265 is usually best for bitrate constrained scenarious. * AVC/H.264 (with CAVLC) may save a few milliseconds of decode latency, but needs a much higher bitrate to reach similar image quality. * Software encoding (x264) can give good results on a beefy high core-count CPU and a very high bitrate. Will require playing with a USB3 cable. The only choice if you don't have a hardware encoder (eg, RX6500). ## Step 2: tweak encoder settings Enable foveated encoding. Go to the SteamVR void and look closely at the framerate graph under the latency graph in the statistics tab. * If the streamer FPS matches the refresh rate you chose in step 1, you can reduce the foveation settings (by increasing the center width/height, or reducing the strength). * If the streamer FPS is lower than the refresh rate you chose in step 1, increase the foveation settings (by decreasing the center width/height, or increasing the strength). Repeat until you are at the maximum of what your encoder can do. ## Step 3: tweak bitrate Slowly increase bitrate until one of two things happen: * The image freezes for half a second or more periodically (on TCP) or you see a glitched image (on UDP): you have gone beyond what your wireless AP is capable of. Lower the bitrate, or consider using a cable. * The controllers stop moving, the image flips upside down, and/or becomes just a solid blinking light: the HMD's decoder is unable to keep up. Lower the bitrate. ## Step 4: tweak frame buffering If you notice micro-stuttering on the headset, especially in busy scenes with fast motion, slowly increase maxBufferingFrames until the playback is smooth. Keep in mind that increasing maxBufferingFrames will linearly increase latency; if the value that gives a smooth playback results in too high of a latency for your use case, try a different codec, a lower bitrate and/or stronger foveation settings. By that point, your latency graph and your playback should be smooth and consistent. Enjoy! ![optimal latency graph](images/latency-graphs/optimal.png) ## Still not satisfied with image quality? * Tweak the color correction sliders, eg slightly increasing sharpening. * If using AMF, enable the pre-processor. * Use the quality encoder preset. * Try a lower refresh rate and start again from step 2. * Try a different codec and start again from step 2. * Try increasing foveation settings (allowing the encoder to use more bits for the center of the image). See also the [Troubleshooting](https://github.com/alvr-org/ALVR/wiki/Troubleshooting#common-performance-related-problems) page for more help. ================================================ FILE: wiki/Troubleshooting.md ================================================ ## If you're looking for Linux troubleshooting, please check [here](https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting) first, and only then this page. For ALVR 20.0.0 and later === First off, please make sure to carefully read the [Installation guide](https://github.com/alvr-org/ALVR/wiki/Installation-guide) and [Usage](https://github.com/alvr-org/ALVR/wiki/Usage) pages. The first thing to try is to delete the file `session.json` located in the ALVR installation folder on the PC. This resets everything to default. If it doesn't work, try reinstalling ALVR. Keep in mind that sometimes a restart of ALVR/SteamVR/PC/Headset will be enough to solve some problems. Having trouble getting ALVR to work? --- [I'm having trouble starting ALVR.](#trouble-starting-alvr) [ALVR starts fine, but says X error.](#alvr-starts-fine-but) [ALVR starts fine and doesn't show any error, but it doesn't see (or connect to) my headset.](#alvr-cant-see-my-headset) If you need more help, come to our [Discord](https://discord.gg/KbKk3UM) and ask in the #help channel. When asking for help, please describe the issue, if you're getting an error message, copy it, and tell us what you already tried to fix it. Trouble starting ALVR === ALVR needs a working graphics driver to be installed in order to work. **On linux**, you also need to make sure you have either `vaapi` on AMD or `cuda` on NVIDIA for hardware encoders to work. ALVR starts launching, but gets stuck on "ALVR is not responding..." === With ALVR versions >= 20.0, some antivirus software can prevent ALVR from launching SteamVR. Try disabling any antivirus other than Windows Defender (McAfee, Norton, etc.), reboot, then try again. If the issue persists, make sure you don't have an instance of ALVR or SteamVR running in the background (check in Task Manager). If you continue having issues, hop in the [ALVR Discord server](https://discord.gg/KbKk3UM), and we'll do our best to help you get it sorted out. ALVR starts fine, but === This section has some advice for when ALVR shows an error (or sometimes warning) pop-up. This could be either a yellow pop-up in the setup window (`ALVR Dashboard.exe`) or a separate pop-up when you connect with a headset. [WARN] clientFoundInvalid --- If you get a warning pop-up inside the `ALVR Dashboard.exe` window saying `clientFoundInvalid`, make sure the version of ALVR you installed on your headset is compatible with the version you're trying to run on your PC. The latest release can be found [here](https://github.com/alvr-org/ALVR/releases/latest) and contains both the `alvr_client.apk` file for your headset and the `alvr_streamer_windows.zip` archive with the application for your PC. The version of ALVR available on the SideQuest store is compatible with the latest release on GitHub (the previous link). Keep in mind that the version on SideQuest might take us a while to update after a new version is released on GitHub. Failed to initialize CEncoder --- ALVR currently needs a recent AMD, Nvidia or Intel GPU to run, since it utilizes hardware video encoding (see [requirements](https://github.com/alvr-org/ALVR#requirements)). If you get an error saying something like ``` Failed to initialize CEncoder. All VideoEncoder are not available. VCE: AMF Error 1. g_AMFFactory.Init(), NVENC: NvEnc NvEncoderD3D11 failed. Code=1 NvEncoder::LoadNvEncApi : NVENC library file is not found. Please ensure NV driver is installed at c:\src\alvr\alvr_server\nvencoder.cpp:70 ``` and you have up-to-date GPU drivers, then your graphics card isn't supported. If you're using a laptop with a powerful enough discrete GPU, you _might_ be able to get ALVR to work by forcing SteamVR to use it in either Windows settings, or the Nvidia control panel. If you have a compatible GPU, you're most likely seeing a different error after either `VCE:`, `VPL:` or `NVENC:` than above. In that case, try using a different video codec in ALVR settings. You can also try lowering your video resolution setting. Failed to start audio capture --- ![Failed to start audio capture](images/ALVR-audio-crash.png) This error can show up when connecting your headset, when SteamVR gets started. Make sure the audio device you have selected in ALVR settings isn't disabled, it should be the device you usually use for games (speakers/headphones). ALVR does not create its own audio device. You can see if you have an "enable audio enhancements" option on your sound device in Windows settings and if so, make sure it's disabled. ALVR can't see my headset === Here is some advice for issues that can come up even though you don't see any error popup from ALVR. ALVR on the headset stuck on `Searching for streamer...` --- This issue can have multiple causes. It is likely that the issue is with the PC ALVR application. See below for more specific issues. ALVR device list is empty --- ![Empty ALVR device list](images/ALVRexe-no-devices.png) Check that the PC app and the headset app run on the latest version of ALVR. If your version is v2.3.1 or v2.4.0-alpha5 then you downloaded ALVR from the wrong link. The correct link is . Make sure ALVR is running both on the PC and on the headset. To be visible in the device list, ALVR on the headset sends broadcast packets which the PC application listens for. These can be blocked by your firewall or possibly your router, if both headset and PC are connected wirelessly, having AP isolation enabled on the router will cause this. To fix this, you can try the following: * Ping the headset to check it's reachable from the PC - you can do this by opening CMD and typing `ping ` without "<>" (you can find the headset's IP in the top left corner of SideQuest) - if ping fails, check that both PC and headset are connected to the same network * You can also try disabling your firewall for testing, but you shouldn't leave it disabled to use ALVR * Open ports 9943 and 9944 on your firewall * Disable the PMF (Protected Management Frames) setting on your Router If pinging works but you still don't see the device on the streamer app, then headset and PC might be on separate subnets. To solve this you can add the device manually. In the Devices tab press `Add device manually`. Fill in the fields with a name for your headset (you can use the name you want), the hostname (you can read it in the welcome screen in your headset when you open the ALVR app), the IP of the headset and then press `Save`. SteamVR says "headset not detected" --- ![SteamVR headset not detected](images/SteamVR-headset-not-detected.png) This message means that the ALVR SteamVR driver isn't loading properly when SteamVR starts. On linux double-check if you have software and hardware encoders installed, without them driver won't load. Check that SteamVR isn't blocking ALVR (see SteamVR settings, enable advanced settings and check `Startup / Shutdown -> Manage Add-ons`). ![SteamVR add-ons](images/SteamVR-add-ons.png) If you're still getting this message (or otherwise not getting a headset icon in the SteamVR window), a SteamVR log (vrserver.txt) will have some information on why the driver isn't loading. You can find it where you installed Steam, in `Steam\logs\vrserver.txt`. ### Some lines to look for and tips for them `Unable to load driver alvr_server because of error VRInitError_Init_FileNotFound(103). Skipping.` - This usually means a library that ALVR needs is missing. Make sure you followed installation instructions carefully, installed the latest Visual C++ Redistributable x64 package and no files are missing where you extracted ALVR (especially in the bin\win64 directory). `Skipping duplicate external driver alvr_server` - This line means another ALVR driver is registered. Go to the installation tab in ALVR and remove all drivers. `Skipping external driver X:\path\to\your\alvr_streamer_windows because it is not a directory` - This can happen if you put ALVR in a OneDrive (or a similar service) directory or the path to ALVR contains characters not in UTF-8. Try putting ALVR elsewhere, preferably so that the path to ALVR contains only ASCII characters. If you have trouble looking through the logs, none of the tips work, or don't apply to you, feel free to ask on our [Discord](https://discord.gg/KbKk3UM) in the #help channel (you may be asked to post the log there). ALVR sees the headset, SteamVR shows headset icon --- ![SteamVR waiting...](images/SteamVR-waiting.png) This is a situation where you have ALVR open on both headset and PC, you can see the headset in the device list and trust it. ALVR then starts SteamVR automatically when you try connecting and SteamVR shows an icon for the headset (and controllers). First make sure that SteamVR (more specifically, vrserver.exe) is allowed incoming connections (UDP, port 9944) in your firewall. You can also try disabling your firewall for testing, but you keep it disabled to use ALVR. You can try restarting ALVR on both the headset and the PC. On the headset, when connecting, you should see the view lagging behind when you turn your head (it drops below 1 fps), this means the headset is getting a response from the streamer when connecting and is waiting for the video stream to start. If you get no lag in the headset, response from the PC isn't reaching the headset. ## Common performance-related problems ### Overloaded encoder ![latency graph of overloaded encoder](images/latency-graphs/overloaded-encoder.png) Symptoms: stuttery playback on the headset, streamer FPS is stable but below the target refresh rate. Solution: increase foveation settings or decrease refresh rate. ### Overloaded decoder ![latency graph of overloaded decoder](images/latency-graphs/overloaded-decoder.png) Symptoms: laggy/frozen controllers, erroneous head tracking, image flipped upside-down, blinking solid colour. Solution: reduce bitrate. ### Overloaded network ![latency graph of overloaded network](images/latency-graphs/overloaded-network.png) Symptoms: stream freezes, image is glitchy. Solution: check that HMD is using 5G frequency and that no other device is connected to the 5G band on your AP, reduce bitrate or use a cable. ### Overloaded streamer ![latency graph of overloaded streamer](images/latency-graphs/overloaded-streamer.png) Symptoms: stuttery playback on the headset, streamer FPS dips or fluctuates below the target refresh rate. Solution: * Decrease the graphics settings in the game * If possible, use the game's native upscaling solution (FSR/NIS/XeSS/DLSS…) * Decrease the target refresh rate in ALVR * Decrease render resolution in SteamVR overlay or ALVR video settings. (This will severely degrade image quality.) ### Micro-stuttering ![latency graph of headset stuttering](images/latency-graphs/not-enough-buffering.png) Symptoms: image is not always smooth especially in high motion or fast scenes. Solution: increase maxBufferingFrames. ### Possible temporary fix for Meta framerate scaling for throttling feature #### Problem The current version of ALVR does not support Meta's framerate scaling for throttling feature. This can cause issues where the framerate between the headset and the streamer application does not align, potentially leading to stuttering or throttling. A future update to ALVR is expected to address this issue, but a workaround is available in the meantime. #### Temporary Fix 1. **Reboot Your Headset** - Start by rebooting your VR headset. This may resolve the issue without further adjustments. 2. **Manually Set the Framerate** - Use the **SideQuest Desktop application** to manually adjust the framerate of the ALVR Android client on your headset to match the framerate set in the ALVR streamer application. - Example: If the ALVR streamer is configured to 90Hz, set the headset's refresh rate to 90Hz in SideQuest. - for more information see issue [#2537] (https://github.com/alvr-org/ALVR/issues/2537). This adjustment bypasses the framerate scaling for throttling feature, ensuring smoother performance. ================================================ FILE: wiki/_Sidebar.md ================================================ #### Start here * [Installation guide](https://github.com/alvr-org/ALVR/wiki/Installation-guide) * [Hand tracking controller bindings](https://github.com/alvr-org/ALVR/wiki/Hand-tracking-controller-bindings) * [Other resources](https://github.com/alvr-org/ALVR/wiki/Other-resources) *** #### Configuration * [Settings tutorial](https://github.com/alvr-org/ALVR/wiki/Settings-tutorial) * [Information and Recommendations](https://github.com/alvr-org/ALVR/wiki/Information-and-Recommendations) * [ALVR headset and streamer on separate networks](https://github.com/alvr-org/ALVR/wiki/Headset-and-ALVR-streamer-on-separate-networks) * [Fixed Foveated Rendering (FFR)](https://github.com/alvr-org/ALVR/wiki/Fixed-Foveated-Rendering-(FFR)) * [ALVR wired setup (ALVR over USB)](https://github.com/alvr-org/ALVR/wiki/ALVR-wired-setup-(ALVR-over-USB)) *** #### Troubleshooting * [Troubleshooting](https://github.com/alvr-org/ALVR/wiki/Troubleshooting) * [Linux Troubleshooting](https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting) * [ALVR Checklist before posting a new Issue](https://github.com/alvr-org/ALVR/wiki/ALVR-Checklist) * [Controller latency](https://github.com/alvr-org/ALVR/wiki/Controller-latency) * [My game is not working properly! Help](https://github.com/alvr-org/ALVR/wiki/My-game-is-not-working-properly!-Help!) * [Hardware Video Encoding Testing](https://github.com/alvr-org/ALVR/wiki/FFmpeg-Hardware-Encoding-Testing) *** #### Development * [Roadmap](https://github.com/alvr-org/ALVR/wiki/Roadmap) * [Building From Source](https://github.com/alvr-org/ALVR/wiki/Building-From-Source) * [How ALVR works](https://github.com/alvr-org/ALVR/wiki/How-ALVR-works) * [Real time video upscaling experiments](https://github.com/alvr-org/ALVR/wiki/Real-time-video-upscaling-experiments)