hz-unity-passthrough-camera-access
Meta Quest Passthrough Camera Access (PCA) for Unity — access the forward-facing RGB cameras on Quest 3 / Quest 3S to feed Computer Vision and Machine Learning pipelines. Use when capturing the passthrough camera image/texture, reading the camera pose and intrinsics, projecting camera pixels into world space via `PassthroughCameraAccess.ViewportPointToRay`, wiring camera frames into ML/CV models, or reasoning about resolution, permissions, vendor tags, and the pinhole/principal-point model. For projecting the image onto a flat world-space surface (frustum-slice quad / image-plane overlay) see references/principal-point-offset.md; for placing 2D ML detections as world-space 3D bounding boxes see references/detection-bounding-boxes.md. Skip if the user only wants to cast the user's POV (use the Media Projection API instead) or is doing screen-space-only overlays.
Install
With your agent
Install OMGSkills skill "meta-quest/agentic-tools:skills/hz-unity-passthrough-camera-access" into this agent's normal skills directory: use the OMGSkills MCP get_skill tool, install only the pinned_install commit and path, verify SKILL.md against pinned_install.skill_md_sha, and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/meta-quest/agentic-tools /tmp/agentic-tools && ln -s /tmp/agentic-tools/skills/hz-unity-passthrough-camera-access ~/.claude/skills/hz-unity-passthrough-camera-accessFrom README
Meta Quest Passthrough Camera Access (PCA) Building on top of the Android Camera2 API, Passthrough Camera Access provides access to the forward-facing RGB cameras on Quest 3 and Quest 3S for the purpose of supporting Computer Vision and Machine Learning. This API is distinct from the Media Projection API, which supports casting from the user's POV (including the UI) and should be used if your purpose is to represent what the user is seeing. Use the Passthrough Camera API to add application-specific computer-vision capabilities that extend the understanding of the user's environment and actions beyond what is provided by the Scene API. Use Cases This API provides an unobstructed view using the forward-facing RGB cameras and can be integrated with ML/CV pipelines. Common use cases: Specially-trained ML/CV models that identify specific objects the application can interact with (e.g. fitness equipment like dumbbells, industrial equipment like audio mixers).
More from this repo
hz-android-2d-porting
Guides porting existing Android 2D apps to Meta VR and Horizon OS — input adaptation, panel layout, and design...
hz-immersive-designer
Guides design of comfortable, intuitive VR/MR experiences for Meta VR and Horizon OS — comfort guidelines, interaction...
hz-iwsdk-webxr
Builds WebXR experiences for Meta VR and Horizon OS using the Immersive Web SDK (IWSDK) — ECS architecture, Three.js...