Repository: collidingScopes/shape-creator-tutorial
Branch: main
Commit: b401cc849f2c
Files: 5
Total size: 15.2 KB

Directory structure:
gitextract_ays4otfv/

├── .github/
│   └── FUNDING.yml
├── README.md
├── index.html
├── main.js
└── styles.css

================================================
FILE CONTENTS
================================================

================================================
FILE: .github/FUNDING.yml
================================================
# These are supported funding model platforms

github: [collidingScopes]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']


================================================
FILE: README.md
================================================
# 3D Hand Tracking Demo [Shape Creator]

Create and control 3D shapes using hand gestures in real-time.

Project built with mediapipe computer vision and Threejs.

[Video](https://youtu.be/oE3a0ghsrBk?si=UCcnjHjpWj21bBA0) | [Live Demo](https://collidingscopes.github.io/shape-creator-tutorial/) | [More Code & Tutorials](https://www.funwithcomputervision.com/)

<img src="demo.png">


## Requirements

- Modern web browser with WebGL support
- Camera access

## Technologies

- **Three.js** for 3D rendering
- **MediaPipe** for hand tracking and gesture recognition
- **HTML5 Canvas** for visual feedback
- **JavaScript** for real-time interaction

## Setup for Development

```bash
# Clone this repository
git clone https://github.com/collidingScopes/shape-creator-tutorial

# Navigate to the project directory
cd shape-creator-tutorial
# Serve with your preferred method (example using Python)
python -m http.server
```

Then navigate to `http://localhost:8000` in your browser.

## License

MIT License

## Credits

- Three.js - https://threejs.org/
- MediaPipe - https://mediapipe.dev/

## Related Projects

I've released several computer vision projects (with code + tutorials) here:
[Fun With Computer Vision](https://www.funwithcomputervision.com/)

You can purchase lifetime access and receive the full project files and tutorials. I'm adding more content regularly :)

You might also like some of my other open source projects:

- [Threejs hand tracking tutorial](https://collidingScopes.github.io/threejs-handtracking-101) - Basic hand tracking setup with threejs and MediaPipe computer vision
- [Particular Drift](https://collidingScopes.github.io/particular-drift) - Turn photos into flowing particle animations
- [Liquid Logo](https://collidingScopes.github.io/liquid-logo) - Transform logos and icons into liquid metal animations
- [Video-to-ASCII](https://collidingScopes.github.io/ascii) - Convert videos into ASCII pixel art

## Contact

- Instagram: [@stereo.drift](https://www.instagram.com/stereo.drift/)
- Twitter/X: [@measure_plan](https://x.com/measure_plan)
- Email: [stereodriftvisuals@gmail.com](mailto:stereodriftvisuals@gmail.com)
- GitHub: [collidingScopes](https://github.com/collidingScopes)

## Donations

If you found this tool useful, feel free to buy me a coffee. 

My name is Alan, and I enjoy building open source software for computer vision, games, and more. This would be much appreciated during late-night coding sessions!

[![Buy Me A Coffee](https://www.buymeacoffee.com/assets/img/custom_images/yellow_img.png)](https://www.buymeacoffee.com/stereoDrift)

================================================
FILE: index.html
================================================
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>MediaPipe / Three.js Shape Creator</title>
  <script defer src="https://cloud.umami.is/script.js" data-website-id="eb59c81c-27cb-4e1d-9e8c-bfbe70c48cd9"></script>
  <link rel="stylesheet" href="styles.css">

  <!-- Primary Meta Tags -->
  <meta name="title" content="Shape Creator Tutorial">
  <meta name="description" content="create/control 3D shapes with hand gestures">

  <!-- Open Graph / Facebook -->
  <meta property="og:type" content="website">
  <meta property="og:url" content="https://collidingscopes.github.io/shape-creator-tutorial/">
  <meta property="og:title" content="Shape Creator Tutorial">
  <meta property="og:description" content="create/control 3D shapes with hand gestures">
  <meta property="og:image" content="https://raw.githubusercontent.com/collidingScopes/shape-creator-tutorial/main/demo.png">

  <!-- Twitter -->
  <meta property="twitter:card" content="summary_large_image">
  <meta property="twitter:url" content="https://collidingscopes.github.io/shape-creator-tutorial/">
  <meta property="twitter:title" content="Shape Creator Tutorial">
  <meta property="twitter:description" content="create/control 3D shapes with hand gestures">
  <meta property="twitter:image" content="https://raw.githubusercontent.com/collidingScopes/shape-creator-tutorial/main/demo.png">
  
</head>
<body>

  <video id="webcam" autoplay muted playsinline></video>
  <canvas id="canvas"></canvas>
  <div id="three-canvas"></div>
  <img id="recycle-bin" src="recyclebin.png" alt="Recycle Bin" />
  <div id="instructions" class="text-box">
    Bring hands close and pinch to create a shape<br>
     > Move hands apart to make the shape larger<br>
    Hover over a shape / pinch to move it<br>
    Move a shape into the recycle bin to delete it
  </div>

  <div id="social-links" class="text-box">
      <a href="https://www.x.com/measure_plan/" target="_blank">Twitter</a><br>
      <a href="https://www.instagram.com/stereo.drift/" target="_blank">Instagram</a><br>
      <a href="https://www.youtube.com/@funwithcomputervision" target="_blank">Youtube</a>
  </div>
  <div id="logo-container" class="text-box">
      <span id="logo">🪬</span><br>
      <a href="https://www.funwithcomputervision.com/" target="_blank">code & tutorials here</a>
  </div>

  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>

</body>

<script src="main.js"></script>

</html>

================================================
FILE: main.js
================================================
let video = document.getElementById('webcam');
let canvas = document.getElementById('canvas');
let ctx = canvas.getContext('2d');
let scene, camera, renderer;
let shapes = [];
let currentShape = null;
let isPinching = false;
let shapeScale = 1;
let originalDistance = null;
let selectedShape = null;
let shapeCreatedThisPinch = false;
let lastShapeCreationTime = 0;
const shapeCreationCooldown = 1000;

const initThree = () => {
  scene = new THREE.Scene();
  camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
  camera.position.z = 5;
  renderer = new THREE.WebGLRenderer({ alpha: true });
  renderer.setSize(window.innerWidth, window.innerHeight);
  document.getElementById('three-canvas').appendChild(renderer.domElement);
  const light = new THREE.AmbientLight(0xffffff, 1);
  scene.add(light);
  animate();
};

const animate = () => {
  requestAnimationFrame(animate);
  shapes.forEach(shape => {
    if (shape !== selectedShape) {
      shape.rotation.x += 0.01;
      shape.rotation.y += 0.01;
    }
  });
  renderer.render(scene, camera);
};

const neonColors = [0xFF00FF, 0x00FFFF, 0xFF3300, 0x39FF14, 0xFF0099, 0x00FF00, 0xFF6600, 0xFFFF00];
let colorIndex = 0;

const getNextNeonColor = () => {
    const color = neonColors[colorIndex];
    colorIndex = (colorIndex + 1) % neonColors.length;
    return color;
};

const createRandomShape = (position) => {
  const geometries = [
    new THREE.BoxGeometry(),
    new THREE.SphereGeometry(0.5, 32, 32),
    new THREE.ConeGeometry(0.5, 1, 32),
    new THREE.CylinderGeometry(0.5, 0.5, 1, 32)
  ];
  const geometry = geometries[Math.floor(Math.random() * geometries.length)];
  const color = getNextNeonColor();
  const group = new THREE.Group();

  const material = new THREE.MeshBasicMaterial({ color: color, transparent: true, opacity: 0.5 });
  const fillMesh = new THREE.Mesh(geometry, material);

  const wireframeMaterial = new THREE.MeshBasicMaterial({ color: 0xffffff, wireframe: true });
  const wireframeMesh = new THREE.Mesh(geometry, wireframeMaterial);

  group.add(fillMesh);
  group.add(wireframeMesh);
  group.position.copy(position);
  scene.add(group);

  shapes.push(group);
  return group;
};

const get3DCoords = (normX, normY) => {
  const x = (normX - 0.5) * 10;
  const y = (0.5 - normY) * 10;
  return new THREE.Vector3(x, y, 0);
};

const isPinch = (landmarks) => {
  const d = (a, b) => Math.hypot(a.x - b.x, a.y - b.y, a.z - b.z);
  return d(landmarks[4], landmarks[8]) < 0.06;
};

const areIndexFingersClose = (l, r) => {
  const d = (a, b) => Math.hypot(a.x - b.x, a.y - b.y);
  return d(l[8], r[8]) < 0.12;
};

const findNearestShape = (position) => {
  let minDist = Infinity;
  let closest = null;
  shapes.forEach(shape => {
    const dist = shape.position.distanceTo(position);
    if (dist < 1.5 && dist < minDist) {
      minDist = dist;
      closest = shape;
    }
  });
  return closest;
};

const isInRecycleBinZone = (position) => {
  const vector = position.clone().project(camera);
  const screenX = ((vector.x + 1) / 2) * window.innerWidth;
  const screenY = ((-vector.y + 1) / 2) * window.innerHeight;

  const binWidth = 160;
  const binHeight = 160;
  const binLeft = window.innerWidth - 60 - binWidth;
  const binTop = window.innerHeight - 60 - binHeight;
  const binRight = binLeft + binWidth;
  const binBottom = binTop + binHeight;

  const adjustedX = window.innerWidth - screenX;

  return adjustedX >= binLeft && adjustedX <= binRight && screenY >= binTop && screenY <= binBottom;
};

const hands = new Hands({ locateFile: file => `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}` });
hands.setOptions({ maxNumHands: 2, modelComplexity: 1, minDetectionConfidence: 0.7, minTrackingConfidence: 0.7 });

hands.onResults(results => {
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  const recycleBin = document.getElementById('recycle-bin');

  for (const landmarks of results.multiHandLandmarks) {
    const drawCircle = (landmark) => {
      ctx.beginPath();
      ctx.arc(landmark.x * canvas.width, landmark.y * canvas.height, 10, 0, 2 * Math.PI);
      ctx.fillStyle = 'rgba(0, 255, 255, 0.7)';
      ctx.fill();
    };
    drawCircle(landmarks[4]); // Thumb tip
    drawCircle(landmarks[8]); // Index tip
  }

  // Existing shape interaction and gesture logic...
  if (results.multiHandLandmarks.length === 2) {
    const [l, r] = results.multiHandLandmarks;
    const leftPinch = isPinch(l);
    const rightPinch = isPinch(r);
    const indexesClose = areIndexFingersClose(l, r);

    if (leftPinch && rightPinch) {
      const left = l[8];
      const right = r[8];
      const centerX = (left.x + right.x) / 2;
      const centerY = (left.y + right.y) / 2;
      const distance = Math.hypot(left.x - right.x, left.y - right.y);

      if (!isPinching) {
        const now = Date.now();
        if (!shapeCreatedThisPinch && indexesClose && now - lastShapeCreationTime > shapeCreationCooldown) {
          currentShape = createRandomShape(get3DCoords(centerX, centerY));
          lastShapeCreationTime = now;
          shapeCreatedThisPinch = true;
          originalDistance = distance;
        }
      } else if (currentShape && originalDistance) {
        shapeScale = distance / originalDistance;
        currentShape.scale.set(shapeScale, shapeScale, shapeScale);
      }
      isPinching = true;
      recycleBin.classList.remove('active');
      return;
    }
  }

  isPinching = false;
  shapeCreatedThisPinch = false;
  originalDistance = null;
  currentShape = null;

  if (results.multiHandLandmarks.length > 0) {
    for (const landmarks of results.multiHandLandmarks) {
      const indexTip = landmarks[8];
      const position = get3DCoords(indexTip.x, indexTip.y);

      if (isPinch(landmarks)) {
        if (!selectedShape) {
          selectedShape = findNearestShape(position);
        }
        if (selectedShape) {
          selectedShape.position.copy(position);

          const inBin = isInRecycleBinZone(selectedShape.position);
          selectedShape.children.forEach(child => {
            if (child.material && child.material.wireframe) {
              child.material.color.set(inBin ? 0xff0000 : 0xffffff);
            }
          });
          if (inBin) {
            recycleBin.classList.add('active');
          } else {
            recycleBin.classList.remove('active');
          }
        }
      } else {
        if (selectedShape && isInRecycleBinZone(selectedShape.position)) {
          scene.remove(selectedShape);
          shapes = shapes.filter(s => s !== selectedShape);
        }
        selectedShape = null;
        recycleBin.classList.remove('active');
      }
    }
  } else {
    if (selectedShape && isInRecycleBinZone(selectedShape.position)) {
      scene.remove(selectedShape);
      shapes = shapes.filter(s => s !== selectedShape);
    }
    selectedShape = null;
    recycleBin.classList.remove('active');
  }
});

const initCamera = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({ video: { width: 1280, height: 720 } });
  video.srcObject = stream;
  await new Promise(resolve => video.onloadedmetadata = resolve);
  canvas.width = video.videoWidth;
  canvas.height = video.videoHeight;
  new Camera(video, {
    onFrame: async () => await hands.send({ image: video }),
    width: video.videoWidth,
    height: video.videoHeight
  }).start();
};

initThree();
initCamera();

================================================
FILE: styles.css
================================================
body, html {
  margin: 0;
  padding: 0;
  overflow: hidden;
  background: #000;
}
#webcam, #canvas, #three-canvas {
  position: absolute;
  width: 100%;
  height: 100%;
  top: 0;
  left: 0;
  object-fit: cover;
  pointer-events: none;
  transform: scaleX(-1);
}
#recycle-bin {
  position: absolute;
  bottom: 60px;
  right: 60px;
  width: 160px;
  height: 160px;
  z-index: 20;
  pointer-events: none;
}
#recycle-bin.active {
  filter: drop-shadow(0 0 10px #ff0000);
  transform: scale(1.1);
  transition: transform 0.2s, filter 0.2s;
}
#instructions {
  position: absolute;
  top: 5px;
  left: 5px;
  text-align: left;
  /* color: white; */
  /* background: rgba(0, 0, 0, 0.5); */
  /* padding: 10px 15px; */
  /* border-radius: 10px; */
  /* font-family: sans-serif; */
  /* font-size: 14px; */
  /* z-index: 30; */
}

#links-para{
        position: absolute;
        bottom: 5px;
        left: 5px;
        font-family: Helvetica, sans-serif;
        font-size: 16px;
        background-color: rgba(255, 255, 255, 0.5);
        padding: 10px;
}

a {
    color: #0000FF !important;
}

#coffee-link {
  position: absolute;
  top: 5px;
  right: 5px;
  font-family: Helvetica, sans-serif;
  font-size: 16px;
  background-color: rgba(255, 255, 255, 0.5);
  padding: 10px;
}

.text-box {
    padding: 8px 15px;
    background-color: rgba(255, 255, 255, 0.9);
    color: black;
    border-radius: 4px;
    font-family: "Arial", "Helvetica Neue", Helvetica, sans-serif;
    border: 2px solid black;
    box-shadow: 3px 3px 0px black;
    font-size: clamp(13px, 2vw, 15px);
    text-align: center;
    z-index: 200;
    opacity: 1;
    transition: opacity 0.3s ease-in-out, bottom 0.3s ease-in-out, box-shadow 0.2s ease;
}


#social-links {
    position: absolute;
    bottom: 10px;
    left: 10px;
}

#logo-container {
    position: absolute;
    top: 10px;
    right: 10px;
}

#video-link {
    position: absolute;
    top: 10px;
    left: 10px;
}

#logo {
    font-size: 2em;
}