Full Code of dupontgu/pov_pong_mouse for AI

Repository: dupontgu/pov_pong_mouse
Branch: main
Commit: 9de8de0adfa2
Files: 5
Total size: 20.5 KB

Directory structure:
gitextract_tp3htaza/

├── LICENSE
├── README.md
├── game_mouse.ino
├── pong.h
└── usbh_helper.h

================================================
FILE CONTENTS
================================================

================================================
FILE: LICENSE
================================================
MIT License

Copyright (c) 2024 Guy Dupont

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
## WHAT?

A mouse with a game of Pong running inside it's firmware, rendered on screen by moving the cursor really fast!

## Demo

https://github.com/user-attachments/assets/f5d2ad66-0942-43ed-bbec-080b894ee467

Notes:
- The game stops/starts when I _click_ the scroll wheel. Otherwise the mouse acts normally.
- The score is rendered after a point is scored by showing the cursor somewhere between the left and right "paddles". The closer to a given paddle, the more that player is winning by (it's a ratio). If the cursor is in the middle, the score is tied.

## Hardware Details
My code runs on an [Seeed Studio XIAO RP2040](https://wiki.seeedstudio.com/XIAO-RP2040/) dev board, which - using the magic of [TinyUSB](https://docs.tinyusb.org/en/latest/) - can act as both a custom USB device _and_ host.  I detached the USB connection from the inside of a cheap HP mouse and fed it into the XIAO's USB port (this goes to the computer). I then wire up the aux USB connection using the XIAO's power/GPIO pins, which goes back to the original mouse's PCB. This creates a USB interceptor that can be programmed to do whatever! Also see my [Mouse/Keyboard effects pedal](https://github.com/dupontgu/hidden_agenda_pedal) :P.

<img src=".docs/mouse_apart.jpg" style="width: 700px;"/>

## How It Works
Most modern computer mice using relative positioning - they report _changes_ in their movement. If you move the mouse slowly to the left, it might spit out a bunch of packets where the x component is just -1 (meaning the mouse moved 1 "pixel" to the left). However! It is possible to implement a USB mouse that uses _absolute_ positioning. It can send an exact position on your monitor (percentage X, percentage Y) that the cursor should move to instantly. This is commonly used for touchscreen drivers - you want the cursor to appear where the finger touches the screen.  This firmware emulates an absolute positioning mouse and quickly moves the mouse cursor between points of interest. It moves fast enough that the cursor (kinda) appears in all positions at once and gives the impression that it is in multiple places at once. I have implemented a simple game of Pong to run in the firmware, and set the cursor's points of interest to be the 2 paddles and the ball while the game is active. So the game runs completely inside the mouse!

## Setup
1. You'll need hardware! Any RP2040 based board wired up with a second USB port will do, but I recommend Adafruit's [Feather RP2040 with USB Host](https://www.adafruit.com/product/5723).
2. Follow [Adafruit's guide](https://learn.adafruit.com/adafruit-feather-rp2040-with-usb-type-a-host/arduino-ide-setup) for getting set up with the Arduino IDE.
3. Install the Adafruit TinyUSB Library and the Pico PIO USB Library (again, [guide](https://learn.adafruit.com/adafruit-feather-rp2040-with-usb-type-a-host/usb-host-device-info)).
4. Ensure that your USB D+/D- pins are [set correctly](./usbh_helper.h).
5. Build and run!


================================================
FILE: game_mouse.ino
================================================
/*********************************************************************
 Adafruit invests time and resources providing this open source code,
 please support Adafruit and open-source hardware by purchasing
 products from Adafruit!

 MIT license, check LICENSE for more information
 Copyright (c) 2019 Ha Thach for Adafruit Industries
 All text above, and the splash screen below must be included in
 any redistribution
*********************************************************************/

/**
Adapted from Adafruit's TinyUSB sample remapper demo:
https://github.com/adafruit/Adafruit_TinyUSB_Arduino/tree/master/examples/DualRole/HID/hid_remapper

by Guy Dupont, August 2024
**/

// USBHost is defined in usbh_helper.h
#include "usbh_helper.h"
#include "pong.h"

// adapted from https://github.com/jonathanedgecombe/absmouse/blob/master/src/AbsMouse.cpp
uint8_t const abs_mouse_desc[] = {
  0x05, 0x01,        // Usage Page (Generic Desktop Ctrls)
  0x09, 0x02,        // Usage (Mouse)
  0xA1, 0x01,        // Collection (Application)
  0x09, 0x01,        //   Usage (Pointer)
  0xA1, 0x00,        //   Collection (Physical)
  0x85, 0x04,        //     Report ID (4)
  0x05, 0x09,        //     Usage Page (Button)
  0x19, 0x01,        //     Usage Minimum (0x01)
  0x29, 0x03,        //     Usage Maximum (0x03)
  0x15, 0x00,        //     Logical Minimum (0)
  0x25, 0x01,        //     Logical Maximum (1)
  0x95, 0x03,        //     Report Count (3)
  0x75, 0x01,        //     Report Size (1)
  0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
  0x95, 0x01,        //     Report Count (1)
  0x75, 0x05,        //     Report Size (5)
  0x81, 0x03,        //     Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
  0x05, 0x01,        //     Usage Page (Generic Desktop Ctrls)
  0x09, 0x30,        //     Usage (X)
  0x09, 0x31,        //     Usage (Y)
  0x16, 0x00, 0x00,  //     Logical Minimum (0)
  0x26, 0xFF, 0x7F,  //     Logical Maximum (32767)
  0x36, 0x00, 0x00,  //     Physical Minimum (0)
  0x46, 0xFF, 0x7F,  //     Physical Maximum (32767)
  0x75, 0x10,        //     Report Size (16)
  0x95, 0x02,        //     Report Count (2)
  0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
  0xC0,              //   End Collection
  0xC0               // End Collection
};

uint8_t const desc_hid_report[] = {
  TUD_HID_REPORT_DESC_MOUSE()
};

// two HID devices, one for absolute mouse (the game) and one for normal mouse (passthrough)
Adafruit_USBD_HID usb_hid_abs(abs_mouse_desc, sizeof(abs_mouse_desc), HID_ITF_PROTOCOL_NONE, 2, true);
Adafruit_USBD_HID usb_hid(desc_hid_report, sizeof(desc_hid_report), HID_ITF_PROTOCOL_MOUSE, 2, true);
Game game;

long lastFrameTime = 0;

// these numbers are virtual/relative and don't correspond to monitor size
const int SCREEN_WIDTH = 800;
const int SCREEN_HEIGHT = 600;
const int PADDLE_WIDTH = 10;
const int PADDLE_HEIGHT = 100;
const int HALF_PADDLE_HEIGHT = PADDLE_HEIGHT / 2;
const int BALL_SIZE = 10;
const float MAX_Y_VEL = 8.0;

int8_t mouseVelocity;
bool gameRunning = false;

void setup() {
  Serial.begin(115200);
  usb_hid_abs.begin();
  usb_hid.begin();
  initGame(&game, 10.0, SCREEN_HEIGHT, SCREEN_WIDTH, BALL_SIZE, PADDLE_HEIGHT, PADDLE_WIDTH);
}

// renders the cursor on screen in a specific spot.
// x and y should be floats between 0.0 and 1.0
// (0, 0) -> top left of screen, (1, 1) -> bottom right
void renderCursor(float x, float y) {
  static uint8_t reportOut[5] = { 0, 0, 0, 0, 0 };
  uint16_t adjX = (uint16_t)((x * 20000) + 6383);  // padding is (32767 - [max]) / 2
  uint16_t adjY = (uint16_t)((y * 26000) + 3383);
  reportOut[1] = adjX & 0xFF;
  reportOut[2] = (adjX >> 8) & 0xFF;
  reportOut[3] = adjY & 0xFF;
  reportOut[4] = (adjY >> 8) & 0xFF;
  if (usb_hid_abs.ready()) {
    // report id 4 was hardcoded in descriptor above
    usb_hid_abs.sendReport(4, &reportOut, 5);
  }
}

void loop() {
  if (gameRunning) {
    long timeNow = millis();
    if ((timeNow - lastFrameTime) >= 16) {
      lastFrameTime = timeNow;
      float leftPaddleVel = paddleAutoPilot(&game, &(game.lPaddle), 5.0, 2);
      float rightPaddleVel = mouseVelocity;
      mouseVelocity = 0;
      tick(&game, leftPaddleVel, rightPaddleVel);
    }

    Frame frame = nextSubframe(&game);
    renderCursor(frame.x, frame.y);
  }

  // ok to delay in this loop, different values produce different POV artifacts
  delay(7);
}

//------------- Core1 -------------//
void setup1() {
  // configure pio-usb: defined in usbh_helper.h
  rp2040_configure_pio_usb();

  // run host stack on controller (rhport) 1
  // Note: For rp2040 pico-pio-usb, calling USBHost.begin() on core1 will have most of the
  // host bit-banging processing works done in core1 to free up core0 for other works
  USBHost.begin(1);
}

void loop1() {
  USBHost.task();
}

//--------------------------------------------------------------------+
// TinyUSB Host callbacks
//--------------------------------------------------------------------+
extern "C" {

  // Invoked when device with hid interface is mounted
  // Report descriptor is also available for use.
  // tuh_hid_parse_report_descriptor() can be used to parse common/simple enough
  // descriptor. Note: if report descriptor length > CFG_TUH_ENUMERATION_BUFSIZE,
  // it will be skipped therefore report_desc = NULL, desc_len = 0
  void tuh_hid_mount_cb(uint8_t dev_addr, uint8_t instance, uint8_t const *desc_report, uint16_t desc_len) {
    (void)desc_report;
    (void)desc_len;
    uint16_t vid, pid;
    tuh_vid_pid_get(dev_addr, &vid, &pid);

    Serial.printf("HID device address = %d, instance = %d is mounted\r\n", dev_addr, instance);
    Serial.printf("VID = %04x, PID = %04x\r\n", vid, pid);

    uint8_t const itf_protocol = tuh_hid_interface_protocol(dev_addr, instance);
    if (itf_protocol == HID_ITF_PROTOCOL_MOUSE) {
      Serial.printf("HID Mouse\r\n");
      if (!tuh_hid_receive_report(dev_addr, instance)) {
        Serial.printf("Error: cannot request to receive report\r\n");
      }
    }
  }

  // Invoked when device with hid interface is un-mounted
  void tuh_hid_umount_cb(uint8_t dev_addr, uint8_t instance) {
    Serial.printf("HID device address = %d, instance = %d is unmounted\r\n", dev_addr, instance);
  }

  // Invoked when received report from device via interrupt endpoint
  void tuh_hid_report_received_cb(uint8_t dev_addr, uint8_t instance, uint8_t const *report, uint16_t len) {
    static uint8_t reportOut[5] = { 0, 0, 0, 0, 0 };
    static bool scrollButtonDown;
    // different mice might have different lengths here! My shitty HP mouse only reports [buttons, x, y]
    if (len == 3) {
      // scroll wheel click is 3rd bit, use that to start/stop the game
      if (report[0] & 0b100) {
        if (!scrollButtonDown) {
          scrollButtonDown = true;
          gameRunning = !gameRunning;
        }
      } else {
        scrollButtonDown = false;
      }

      if (!gameRunning) {
        // if game's not running, copy mouse report to passthrough
        reportOut[0] = report[0];
        reportOut[1] = report[1];
        reportOut[2] = report[2];
        usb_hid.sendReport(0, reportOut, 5);
      } else {
        // if game is happening, copy latest Y velocity to be used by game logic
        mouseVelocity = (int8_t)report[2];
      }
    }
    // continue to request to receive report
    if (!tuh_hid_receive_report(dev_addr, instance)) {
      Serial.printf("Error: cannot request to receive report\r\n");
    }
  }
}


================================================
FILE: pong.h
================================================
#define RENDER_SCORE_FRAMES_MAX 80

typedef struct
{
  float x, y;
  float velX, velY;
} Ball;

typedef struct
{
  float x, y;
  float velY;
  int score;
} Paddle;

typedef struct
{
  Ball ball;
  Paddle lPaddle;
  Paddle rPaddle;
  float maxYVel;
  int screenHeight;
  int screenWidth;
  int ballSize;
  int paddleHeight;
  int paddleWidth;
  int halfPaddleHeight;
  int renderScoreCounter;
  uint8_t subFrame;
} Game;

typedef struct
{
  float x, y;
} Frame;

void initGame(Game *gameHolder, float maxBallVelocity, float screenHeight, float screenWidth, int ballSize, int paddleHeight, int paddleWidth) {
  Paddle left = { 10, (screenHeight - paddleHeight) / 2, 0, 0 };
  gameHolder->lPaddle = left;
  Paddle right = { screenWidth - 20, (screenHeight - paddleHeight) / 2, 0, 0 };
  gameHolder->rPaddle = right;
  // TODO - pass in ball x velocity as argument
  Ball ball = { .x = screenWidth / 2, .y = screenHeight / 2, .velX = -5, .velY = 1.6 };
  gameHolder->ball = ball;
  gameHolder->maxYVel = maxBallVelocity;
  gameHolder->screenHeight = screenHeight;
  gameHolder->screenWidth = screenWidth;
  gameHolder->ballSize = ballSize;
  gameHolder->paddleHeight = paddleHeight;
  gameHolder->paddleWidth = paddleWidth;
  gameHolder->halfPaddleHeight = paddleHeight / 2;
  gameHolder->renderScoreCounter = 0;
}

int distFromPaddleCenter(Game *game, Ball *ball, Paddle *paddle) {
  return ball->y - (paddle->y + game->halfPaddleHeight);
}

float yVelocityFromPaddleIntersect(Game *game, int distFromPaddleCenter) {
  return ((float)distFromPaddleCenter / (float)game->halfPaddleHeight) * game->maxYVel;
}

void movePaddle(Game *game, Paddle *paddle) {
  paddle->y += paddle->velY;

  // Keep paddle within screen bounds
  if (paddle->y < 0) {
    paddle->y = 0;
  } else if (paddle->y > game->screenHeight - game->paddleHeight) {
    paddle->y = game->screenHeight - game->paddleHeight;
  }
}

void moveBall(Game *game) {
  Ball *ball = &(game->ball);
  Paddle *leftPaddle = &(game->lPaddle);
  Paddle *rightPaddle = &(game->rPaddle);
  ball->x += ball->velX;
  ball->y += ball->velY;

  // Bounce off top and bottom walls
  if (ball->y <= 0 || ball->y >= game->screenHeight - game->ballSize) {
    ball->velY = -ball->velY;
  }

  int leftPaddleYDist = distFromPaddleCenter(game, ball, leftPaddle);
  int rightPaddleYDist = distFromPaddleCenter(game, ball, rightPaddle);

  // Bounce off paddles
  if (ball->x <= leftPaddle->x + game->paddleWidth && ball->y + game->ballSize >= leftPaddle->y && abs(leftPaddleYDist) < game->halfPaddleHeight) {
    ball->velX = -ball->velX;
    ball->x = leftPaddle->x + game->paddleWidth;
    ball->velY = yVelocityFromPaddleIntersect(game, leftPaddleYDist);
  } else if (ball->x + game->ballSize >= rightPaddle->x && ball->y + game->ballSize >= rightPaddle->y && abs(rightPaddleYDist) < game->halfPaddleHeight) {
    ball->velX = -ball->velX;
    ball->x = rightPaddle->x - game->ballSize;
    ball->velY = yVelocityFromPaddleIntersect(game, rightPaddleYDist);
  }

  bool pointScored = false;
  // Reset ball if it goes past the paddles
  if (ball->x < 0) {
    pointScored = true;
    game->rPaddle.score++;
    Serial.println("r score");
  } else if (ball->x > game->screenWidth) {
    pointScored = true;
    game->lPaddle.score++;
    Serial.println("l score");
  }

  if (pointScored) {
    game->renderScoreCounter = RENDER_SCORE_FRAMES_MAX;
    ball->x = game->screenWidth / 2;
    ball->y = game->screenHeight / 2;
    ball->velX = -ball->velX;
    ball->velY = 1;
  }
}

// quick an dirty AI, can be used with either/both paddles
// call with the desired paddle, then pass the returned velocity value back in with the next "tick"
float paddleAutoPilot(Game *game, Paddle *paddle, float maxVelocity, int randomSeed) {
  float guessVelocity = 0.0;
  int scoreSeed = game->lPaddle.score + game->rPaddle.score + randomSeed;
  bool targetOffsetNegative = (scoreSeed % 2) == 1;
  int targetOffset = ((scoreSeed + 1) * 7) % game->halfPaddleHeight;
  if (targetOffsetNegative) {
    targetOffset = -targetOffset;
  }
  float yDistFromBall = game->ball.y - (paddle->y + game->halfPaddleHeight + targetOffset);
  if (yDistFromBall > maxVelocity) {
    guessVelocity = maxVelocity;
  } else if (yDistFromBall < -maxVelocity) {
    guessVelocity = -maxVelocity;
  }
  return guessVelocity;
}

void tick(Game *game, float leftPaddleVel, float rightPaddleVel) {
  if (game->renderScoreCounter > 0) {
    game->renderScoreCounter -= 1;
  } else {
    game->lPaddle.velY = leftPaddleVel;
    game->rPaddle.velY = rightPaddleVel;
    movePaddle(game, &(game->lPaddle));
    movePaddle(game, &(game->rPaddle));
    moveBall(game);
  }
}

Frame nextSubframe(Game *game) {
  Frame frame;
  if (game->renderScoreCounter > 0) {
    // score is rendered by showing the mouse cursor in three spots
    // outer spots show the bounds, inner spot indicates which player is winning
    // the closer the inner spot is to either edge indicates how much that player is up by
    // when the game is tied, the inner spot should sit right in the middle
    float pointRatio = (float)(game->rPaddle.score + 1) / (float)(game->lPaddle.score + 1);
    if (pointRatio < 1.0) {
      pointRatio /= 2.0;
    } else {
      pointRatio = 1.0 - ((1.0 / pointRatio) / 2.0);
    }
    switch (game->subFrame++) {
      case 0:
        frame.x = 0;
        frame.y = 0.5;
        break;
      case 1:
        frame.x = pointRatio;
        frame.y = 0.5;
        break;
      case 2:
        frame.x = 1;
        frame.y = 0.5;
      default:
        game->subFrame = 0;
    }
  } else {
    switch (game->subFrame++) {
      case 0:
        frame.x = game->lPaddle.x / (float)game->screenWidth;
        frame.y = (game->lPaddle.y + game->halfPaddleHeight) / (float)game->screenHeight;
        break;
      case 1:
        frame.x = game->ball.x / (float)game->screenWidth;
        frame.y = game->ball.y / (float)game->screenHeight;
        break;
      case 2:
        frame.x = game->rPaddle.x / (float)game->screenWidth;
        frame.y = (game->rPaddle.y + game->halfPaddleHeight) / (float)game->screenHeight;
      default:
        game->subFrame = 0;
    }
  }
  return frame;
}


================================================
FILE: usbh_helper.h
================================================
/*********************************************************************
 Adafruit invests time and resources providing this open source code,
 please support Adafruit and open-source hardware by purchasing
 products from Adafruit!

 MIT license, check LICENSE for more information
 Copyright (c) 2019 Ha Thach for Adafruit Industries
 All text above, and the splash screen below must be included in
 any redistribution
*********************************************************************/

/**
Adapted from Adafruit's TinyUSB sample remapper demo:
https://github.com/adafruit/Adafruit_TinyUSB_Arduino/tree/master/examples/DualRole/HID/hid_remapper

by Guy Dupont, August 2024
**/

#ifndef USBH_HELPER_H
#define USBH_HELPER_H

#ifdef ARDUINO_ARCH_RP2040
  // pio-usb is required for rp2040 host
  #include "pio_usb.h"

  // Pin D+ for host, D- = D+ + 1
  #ifndef PIN_USB_HOST_DP
  // THIS IS SPECIFIC TO MY BOARD, use 16 for D+ with usb host feather
  #define PIN_USB_HOST_DP  3
  #endif

  // Pin for enabling Host VBUS. comment out if not used
  #ifndef PIN_5V_EN
  #define PIN_5V_EN        18
  #endif

  #ifndef PIN_5V_EN_STATE
  #define PIN_5V_EN_STATE  1
  #endif
#endif // ARDUINO_ARCH_RP2040

#include "Adafruit_TinyUSB.h"

#if defined(CFG_TUH_MAX3421) && CFG_TUH_MAX3421
  // USB Host using MAX3421E: SPI, CS, INT
  #include "SPI.h"

  #if defined(ARDUINO_METRO_ESP32S2)
    Adafruit_USBH_Host USBHost(&SPI, 15, 14);
  #elif defined(ARDUINO_ADAFRUIT_FEATHER_ESP32_V2)
    Adafruit_USBH_Host USBHost(&SPI, 33, 15);
  #else
    // Default CS and INT are pin 10, 9
    Adafruit_USBH_Host USBHost(&SPI, 10, 9);
  #endif
#else
  // Native USB Host such as rp2040
  Adafruit_USBH_Host USBHost;
#endif

//--------------------------------------------------------------------+
// Helper Functions
//--------------------------------------------------------------------+

#ifdef ARDUINO_ARCH_RP2040
static void rp2040_configure_pio_usb(void) {
  //while ( !Serial ) delay(10);   // wait for native usb
  Serial.println("Core1 setup to run TinyUSB host with pio-usb");

  // Check for CPU frequency, must be multiple of 120Mhz for bit-banging USB
  uint32_t cpu_hz = clock_get_hz(clk_sys);
  if (cpu_hz != 120000000UL && cpu_hz != 240000000UL) {
    while (!Serial) {
      delay(10);   // wait for native usb
    }
    Serial.printf("Error: CPU Clock = %lu, PIO USB require CPU clock must be multiple of 120 Mhz\r\n", cpu_hz);
    Serial.printf("Change your CPU Clock to either 120 or 240 Mhz in Menu->CPU Speed \r\n");
    while (1) {
      delay(1);
    }
  }

#ifdef PIN_5V_EN
  pinMode(PIN_5V_EN, OUTPUT);
  digitalWrite(PIN_5V_EN, PIN_5V_EN_STATE);
#endif

  pio_usb_configuration_t pio_cfg = PIO_USB_DEFAULT_CONFIG;
  pio_cfg.pin_dp = PIN_USB_HOST_DP;

#if defined(ARDUINO_RASPBERRY_PI_PICO_W)
  // For pico-w, PIO is also used to communicate with cyw43
  // Therefore we need to alternate the pio-usb configuration
  // details https://github.com/sekigon-gonnoc/Pico-PIO-USB/issues/46
  pio_cfg.sm_tx      = 3;
  pio_cfg.sm_rx      = 2;
  pio_cfg.sm_eop     = 3;
  pio_cfg.pio_rx_num = 0;
  pio_cfg.pio_tx_num = 1;
  pio_cfg.tx_ch      = 9;
#endif

  USBHost.configure_pio_usb(1, &pio_cfg);
}
#endif

#endif