AR libraries & repos · the open-source we build on
Augmented reality on the open web is built from free, open libraries — the same ones our
Webcam Studio and Photo Editor use. Here is the honest map of 17 of them:
what each does, its repo, and its licence. Everything marked runs in browser is free on the visitor's own device;
GPU / self-host is the heavier, run-it-yourself layer (see GenAI School).
Face / body / hand tracking (5)
Live landmark and pose tracking — the eyes of any AR filter. All run in the browser.
MediaPipe Tasks — Vision · Google runs in browser
Face landmarks, hand/pose tracking, selfie segmentation, gesture recognition — runs live in-browser via WASM/WebGL. This is what our Webcam Studio uses for background replace.
TensorFlow.js Models · Google runs in browser
BlazeFace, FaceMesh (468 points), BodyPix / body-segmentation, hand-pose, MoveNet pose — pretrained, load-and-run in the browser.
JEELIZ FaceFilter · Jeeliz runs in browser
Lightweight deep-learning face tracking for Snapchat-style filters; ships three.js/BabylonJS glue and many ready demos (glasses, masks, makeup).
face-api.js · justadudewhohacks runs in browser
Face detection, landmarks, expressions and recognition in the browser on tfjs — good for filters that react to expression.
Kalidokit · yeemachine runs in browser
Turns MediaPipe/tfjs face+pose+hand data into VRM avatar blendshapes and bone rotations — the math behind VTuber-style live avatars.
Background & segmentation (2)
Cut the person out (or the background off) — keep the subject exact.
@imgly/background-removal · IMG.LY runs in browser
High-quality background removal fully in the browser (ONNX-WASM) — keeps the subject exact. This powers our Photo Editor cutout.
rembg · danielgatis GPU / self-host
Server/desktop background removal (U^2-Net family) — the heavier, batch-friendly option to run yourself on your own machine/GPU.
WebAR — marker, image & world tracking (4)
AR in a plain browser, no app: markers, image targets, WebXR sessions.
AR.js · AR-js-org runs in browser
Marker-based and location-based WebAR that runs on plain phones — no app. Great for print-marker experiences (tie into business cards / shirts).
MindAR · hiukim runs in browser
Image-tracking and face-tracking WebAR with three.js/A-Frame — put 3D content on a tracked image or face, in the browser.
A-Frame · Supermedium runs in browser
HTML-tag VR/AR framework on three.js + WebXR — the fast way to build a scene, pairs with AR.js/MindAR. Powers the Dudael VR side.
WebXR Device API · W3C / Immersive Web runs in browser
The native browser standard for AR/VR sessions and hit-testing on supported devices — the real floor for phone/headset AR.
3D / 2D rendering engines (3)
The layer AR content is drawn with — and the bridge to the shirt → 3D → game pipeline.
three.js · mrdoob runs in browser
The workhorse WebGL 3D engine — render glasses/masks/objects onto tracked faces and scenes; exports/loads glTF for the shirt→3D→game pipeline.
Babylon.js · Microsoft runs in browser
Full-featured 3D/game engine with first-class WebXR — an alternative to three.js when you want physics, GUI and tooling built in.
PixiJS · pixijs runs in browser
Fast 2D WebGL renderer — for 2D overlays, stickers, and shader effects on webcam/photos without the 3D overhead.
Generative face/style edit (GPU) (3)
Identity-preserving likeness, inpainting, style transfer. Heavier — run it yourself or on PRANA GPU.
InsightFace · deepinsight GPU / self-host
Face analysis + the swap/restore models behind identity-preserving edits. The real path to "make the character look like the uploaded person" — needs a GPU.
ComfyUI · comfyanonymous GPU / self-host
Node-graph runner for Stable Diffusion — inpainting (change hair/eyes), ControlNet, IP-Adapter/InstantID for likeness, and img2img. Our GenAI School teaches these graphs.
IP-Adapter · tencent-ailab GPU / self-host
Image-prompt adapter for diffusion models — the standard way to condition on a reference face/style so a generated character resembles your upload.
Want to build one? Start from MediaPipe or tfjs-models for tracking, draw with three.js,
and for marker/image AR use AR.js or MindAR. Our repo's studio code is open too — fork it. Licences are the authors'; check each before commercial use.