MediaPipe / Three.js Shape Creator

[ { "path": ".github/FUNDING.yml", "content": "# These are supported funding model platforms\n\ngithub: [collidingScopes]\npatreon: # Replace with a single Patreon username\nopen_collective: # Replace with a single Open Collective username\nko_fi: # Replace with a single Ko-fi username\ntidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel\ncommunity_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry\nliberapay: # Replace with a single Liberapay username\nissuehunt: # Replace with a single IssueHunt username\nlfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry\npolar: # Replace with a single Polar username\nbuy_me_a_coffee: # Replace with a single Buy Me a Coffee username\nthanks_dev: # Replace with a single thanks.dev username\ncustom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']\n" }, { "path": "README.md", "content": "# 3D Hand Tracking Demo [Shape Creator]\n\nCreate and control 3D shapes using hand gestures in real-time.\n\nProject built with mediapipe computer vision and Threejs.\n\n[Video](https://youtu.be/oE3a0ghsrBk?si=UCcnjHjpWj21bBA0) | [Live Demo](https://collidingscopes.github.io/shape-creator-tutorial/) | [More Code & Tutorials](https://www.funwithcomputervision.com/)\n\n

\n\n\n## Requirements\n\n- Modern web browser with WebGL support\n- Camera access\n\n## Technologies\n\n- **Three.js** for 3D rendering\n- **MediaPipe** for hand tracking and gesture recognition\n- **HTML5 Canvas** for visual feedback\n- **JavaScript** for real-time interaction\n\n## Setup for Development\n\n```bash\n# Clone this repository\ngit clone https://github.com/collidingScopes/shape-creator-tutorial\n\n# Navigate to the project directory\ncd shape-creator-tutorial\n# Serve with your preferred method (example using Python)\npython -m http.server\n```\n\nThen navigate to `http://localhost:8000` in your browser.\n\n## License\n\nMIT License\n\n## Credits\n\n- Three.js - https://threejs.org/\n- MediaPipe - https://mediapipe.dev/\n\n## Related Projects\n\nI've released several computer vision projects (with code + tutorials) here:\n[Fun With Computer Vision](https://www.funwithcomputervision.com/)\n\nYou can purchase lifetime access and receive the full project files and tutorials. I'm adding more content regularly :)\n\nYou might also like some of my other open source projects:\n\n- [Threejs hand tracking tutorial](https://collidingScopes.github.io/threejs-handtracking-101) - Basic hand tracking setup with threejs and MediaPipe computer vision\n- [Particular Drift](https://collidingScopes.github.io/particular-drift) - Turn photos into flowing particle animations\n- [Liquid Logo](https://collidingScopes.github.io/liquid-logo) - Transform logos and icons into liquid metal animations\n- [Video-to-ASCII](https://collidingScopes.github.io/ascii) - Convert videos into ASCII pixel art\n\n## Contact\n\n- Instagram: [@stereo.drift](https://www.instagram.com/stereo.drift/)\n- Twitter/X: [@measure_plan](https://x.com/measure_plan)\n- Email: [stereodriftvisuals@gmail.com](mailto:stereodriftvisuals@gmail.com)\n- GitHub: [collidingScopes](https://github.com/collidingScopes)\n\n## Donations\n\nIf you found this tool useful, feel free to buy me a coffee. \n\nMy name is Alan, and I enjoy building open source software for computer vision, games, and more. This would be much appreciated during late-night coding sessions!\n\n[![Buy Me A Coffee](https://www.buymeacoffee.com/assets/img/custom_images/yellow_img.png)](https://www.buymeacoffee.com/stereoDrift)" }, { "path": "index.html", "content": "\n\n\n \n \n MediaPipe / Three.js Shape Creator\n \n \n\n \n \n \n\n \n \n \n \n \n \n\n \n \n \n \n \n \n \n\n\n\n

\n $\"Recycle$ \n

\n Bring hands close and pinch to create a shape
\n > Move hands apart to make the shape larger
\n Hover over a shape / pinch to move it
\n Move a shape into the recycle bin to delete it\n

\n\n \n

\n 🪬
\n code & tutorials here\n

\n\n \n \n \n\n\n\n\n\n" }, { "path": "main.js", "content": "let video = document.getElementById('webcam');\nlet canvas = document.getElementById('canvas');\nlet ctx = canvas.getContext('2d');\nlet scene, camera, renderer;\nlet shapes = [];\nlet currentShape = null;\nlet isPinching = false;\nlet shapeScale = 1;\nlet originalDistance = null;\nlet selectedShape = null;\nlet shapeCreatedThisPinch = false;\nlet lastShapeCreationTime = 0;\nconst shapeCreationCooldown = 1000;\n\nconst initThree = () => {\n scene = new THREE.Scene();\n camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);\n camera.position.z = 5;\n renderer = new THREE.WebGLRenderer({ alpha: true });\n renderer.setSize(window.innerWidth, window.innerHeight);\n document.getElementById('three-canvas').appendChild(renderer.domElement);\n const light = new THREE.AmbientLight(0xffffff, 1);\n scene.add(light);\n animate();\n};\n\nconst animate = () => {\n requestAnimationFrame(animate);\n shapes.forEach(shape => {\n if (shape !== selectedShape) {\n shape.rotation.x += 0.01;\n shape.rotation.y += 0.01;\n }\n });\n renderer.render(scene, camera);\n};\n\nconst neonColors = [0xFF00FF, 0x00FFFF, 0xFF3300, 0x39FF14, 0xFF0099, 0x00FF00, 0xFF6600, 0xFFFF00];\nlet colorIndex = 0;\n\nconst getNextNeonColor = () => {\n const color = neonColors[colorIndex];\n colorIndex = (colorIndex + 1) % neonColors.length;\n return color;\n};\n\nconst createRandomShape = (position) => {\n const geometries = [\n new THREE.BoxGeometry(),\n new THREE.SphereGeometry(0.5, 32, 32),\n new THREE.ConeGeometry(0.5, 1, 32),\n new THREE.CylinderGeometry(0.5, 0.5, 1, 32)\n ];\n const geometry = geometries[Math.floor(Math.random() * geometries.length)];\n const color = getNextNeonColor();\n const group = new THREE.Group();\n\n const material = new THREE.MeshBasicMaterial({ color: color, transparent: true, opacity: 0.5 });\n const fillMesh = new THREE.Mesh(geometry, material);\n\n const wireframeMaterial = new THREE.MeshBasicMaterial({ color: 0xffffff, wireframe: true });\n const wireframeMesh = new THREE.Mesh(geometry, wireframeMaterial);\n\n group.add(fillMesh);\n group.add(wireframeMesh);\n group.position.copy(position);\n scene.add(group);\n\n shapes.push(group);\n return group;\n};\n\nconst get3DCoords = (normX, normY) => {\n const x = (normX - 0.5) * 10;\n const y = (0.5 - normY) * 10;\n return new THREE.Vector3(x, y, 0);\n};\n\nconst isPinch = (landmarks) => {\n const d = (a, b) => Math.hypot(a.x - b.x, a.y - b.y, a.z - b.z);\n return d(landmarks[4], landmarks[8]) < 0.06;\n};\n\nconst areIndexFingersClose = (l, r) => {\n const d = (a, b) => Math.hypot(a.x - b.x, a.y - b.y);\n return d(l[8], r[8]) < 0.12;\n};\n\nconst findNearestShape = (position) => {\n let minDist = Infinity;\n let closest = null;\n shapes.forEach(shape => {\n const dist = shape.position.distanceTo(position);\n if (dist < 1.5 && dist < minDist) {\n minDist = dist;\n closest = shape;\n }\n });\n return closest;\n};\n\nconst isInRecycleBinZone = (position) => {\n const vector = position.clone().project(camera);\n const screenX = ((vector.x + 1) / 2) * window.innerWidth;\n const screenY = ((-vector.y + 1) / 2) * window.innerHeight;\n\n const binWidth = 160;\n const binHeight = 160;\n const binLeft = window.innerWidth - 60 - binWidth;\n const binTop = window.innerHeight - 60 - binHeight;\n const binRight = binLeft + binWidth;\n const binBottom = binTop + binHeight;\n\n const adjustedX = window.innerWidth - screenX;\n\n return adjustedX >= binLeft && adjustedX <= binRight && screenY >= binTop && screenY <= binBottom;\n};\n\nconst hands = new Hands({ locateFile: file => `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}` });\nhands.setOptions({ maxNumHands: 2, modelComplexity: 1, minDetectionConfidence: 0.7, minTrackingConfidence: 0.7 });\n\nhands.onResults(results => {\n ctx.clearRect(0, 0, canvas.width, canvas.height);\n const recycleBin = document.getElementById('recycle-bin');\n\n for (const landmarks of results.multiHandLandmarks) {\n const drawCircle = (landmark) => {\n ctx.beginPath();\n ctx.arc(landmark.x * canvas.width, landmark.y * canvas.height, 10, 0, 2 * Math.PI);\n ctx.fillStyle = 'rgba(0, 255, 255, 0.7)';\n ctx.fill();\n };\n drawCircle(landmarks[4]); // Thumb tip\n drawCircle(landmarks[8]); // Index tip\n }\n\n // Existing shape interaction and gesture logic...\n if (results.multiHandLandmarks.length === 2) {\n const [l, r] = results.multiHandLandmarks;\n const leftPinch = isPinch(l);\n const rightPinch = isPinch(r);\n const indexesClose = areIndexFingersClose(l, r);\n\n if (leftPinch && rightPinch) {\n const left = l[8];\n const right = r[8];\n const centerX = (left.x + right.x) / 2;\n const centerY = (left.y + right.y) / 2;\n const distance = Math.hypot(left.x - right.x, left.y - right.y);\n\n if (!isPinching) {\n const now = Date.now();\n if (!shapeCreatedThisPinch && indexesClose && now - lastShapeCreationTime > shapeCreationCooldown) {\n currentShape = createRandomShape(get3DCoords(centerX, centerY));\n lastShapeCreationTime = now;\n shapeCreatedThisPinch = true;\n originalDistance = distance;\n }\n } else if (currentShape && originalDistance) {\n shapeScale = distance / originalDistance;\n currentShape.scale.set(shapeScale, shapeScale, shapeScale);\n }\n isPinching = true;\n recycleBin.classList.remove('active');\n return;\n }\n }\n\n isPinching = false;\n shapeCreatedThisPinch = false;\n originalDistance = null;\n currentShape = null;\n\n if (results.multiHandLandmarks.length > 0) {\n for (const landmarks of results.multiHandLandmarks) {\n const indexTip = landmarks[8];\n const position = get3DCoords(indexTip.x, indexTip.y);\n\n if (isPinch(landmarks)) {\n if (!selectedShape) {\n selectedShape = findNearestShape(position);\n }\n if (selectedShape) {\n selectedShape.position.copy(position);\n\n const inBin = isInRecycleBinZone(selectedShape.position);\n selectedShape.children.forEach(child => {\n if (child.material && child.material.wireframe) {\n child.material.color.set(inBin ? 0xff0000 : 0xffffff);\n }\n });\n if (inBin) {\n recycleBin.classList.add('active');\n } else {\n recycleBin.classList.remove('active');\n }\n }\n } else {\n if (selectedShape && isInRecycleBinZone(selectedShape.position)) {\n scene.remove(selectedShape);\n shapes = shapes.filter(s => s !== selectedShape);\n }\n selectedShape = null;\n recycleBin.classList.remove('active');\n }\n }\n } else {\n if (selectedShape && isInRecycleBinZone(selectedShape.position)) {\n scene.remove(selectedShape);\n shapes = shapes.filter(s => s !== selectedShape);\n }\n selectedShape = null;\n recycleBin.classList.remove('active');\n }\n});\n\nconst initCamera = async () => {\n const stream = await navigator.mediaDevices.getUserMedia({ video: { width: 1280, height: 720 } });\n video.srcObject = stream;\n await new Promise(resolve => video.onloadedmetadata = resolve);\n canvas.width = video.videoWidth;\n canvas.height = video.videoHeight;\n new Camera(video, {\n onFrame: async () => await hands.send({ image: video }),\n width: video.videoWidth,\n height: video.videoHeight\n }).start();\n};\n\ninitThree();\ninitCamera();" }, { "path": "styles.css", "content": "body, html {\n margin: 0;\n padding: 0;\n overflow: hidden;\n background: #000;\n}\n#webcam, #canvas, #three-canvas {\n position: absolute;\n width: 100%;\n height: 100%;\n top: 0;\n left: 0;\n object-fit: cover;\n pointer-events: none;\n transform: scaleX(-1);\n}\n#recycle-bin {\n position: absolute;\n bottom: 60px;\n right: 60px;\n width: 160px;\n height: 160px;\n z-index: 20;\n pointer-events: none;\n}\n#recycle-bin.active {\n filter: drop-shadow(0 0 10px #ff0000);\n transform: scale(1.1);\n transition: transform 0.2s, filter 0.2s;\n}\n#instructions {\n position: absolute;\n top: 5px;\n left: 5px;\n text-align: left;\n /* color: white; */\n /* background: rgba(0, 0, 0, 0.5); */\n /* padding: 10px 15px; */\n /* border-radius: 10px; */\n /* font-family: sans-serif; */\n /* font-size: 14px; */\n /* z-index: 30; */\n}\n\n#links-para{\n position: absolute;\n bottom: 5px;\n left: 5px;\n font-family: Helvetica, sans-serif;\n font-size: 16px;\n background-color: rgba(255, 255, 255, 0.5);\n padding: 10px;\n}\n\na {\n color: #0000FF !important;\n}\n\n#coffee-link {\n position: absolute;\n top: 5px;\n right: 5px;\n font-family: Helvetica, sans-serif;\n font-size: 16px;\n background-color: rgba(255, 255, 255, 0.5);\n padding: 10px;\n}\n\n.text-box {\n padding: 8px 15px;\n background-color: rgba(255, 255, 255, 0.9);\n color: black;\n border-radius: 4px;\n font-family: \"Arial\", \"Helvetica Neue\", Helvetica, sans-serif;\n border: 2px solid black;\n box-shadow: 3px 3px 0px black;\n font-size: clamp(13px, 2vw, 15px);\n text-align: center;\n z-index: 200;\n opacity: 1;\n transition: opacity 0.3s ease-in-out, bottom 0.3s ease-in-out, box-shadow 0.2s ease;\n}\n\n\n#social-links {\n position: absolute;\n bottom: 10px;\n left: 10px;\n}\n\n#logo-container {\n position: absolute;\n top: 10px;\n right: 10px;\n}\n\n#video-link {\n position: absolute;\n top: 10px;\n left: 10px;\n}\n\n#logo {\n font-size: 2em;\n}" } ]