docs: 添加 CLAUDE.md 项目指南
为 Claude Code 提供项目概览和常用命令指南
This commit is contained in:
242
CLAUDE.md
Normal file
242
CLAUDE.md
Normal file
@@ -0,0 +1,242 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
CRAIC (Camera-Robot AI Control System) — Competition code for the China Robot and Artificial Intelligence Competition (中国机器人及人工智能大赛), Robot Task Challenge (Small Desktop Level).
|
||||
|
||||
**Hardware**: ESP32-S3-WROOM-1-N16R8 with OV2640 camera, 6-DOF mechanical arm with Feettech SCS/STS serial servos.
|
||||
|
||||
**Architecture**: Three-tier system communicating via UDP port 8888:
|
||||
1. **ESP32-S3 firmware** (`jxbeye/`) — Dual-core camera streaming + UDP command receiver
|
||||
2. **ROS 2 teleop** (`ros2/src/udp_teleop/`) — Keyboard control node for chassis and arm
|
||||
3. **Python tools** (`tools/`) — Standalone control scripts with inverse kinematics
|
||||
|
||||
## Build & Run Commands
|
||||
|
||||
### ESP32-S3 Firmware
|
||||
|
||||
```bash
|
||||
cd jxbeye
|
||||
pio run -t upload # Build and flash firmware
|
||||
pio device monitor # Serial monitor (1000000 baud)
|
||||
```
|
||||
|
||||
**First boot**: ESP32 creates AP `ESP32-S3-Camera` (password `12345678`), access web UI at `http://192.168.4.1`.
|
||||
|
||||
**WiFi configuration via serial**: Send `WIFI:SSID:PASSWORD` to configure station mode.
|
||||
|
||||
### ROS 2 Teleop
|
||||
|
||||
```bash
|
||||
# Build (from ros2/ directory)
|
||||
conda activate ros2_humble
|
||||
colcon build --symlink-install --packages-select udp_teleop
|
||||
source install/setup.bash
|
||||
|
||||
# Run keyboard control
|
||||
ros2 run udp_teleop keyboard_control \
|
||||
--ros-args --params-file src/udp_teleop/config/params.yaml
|
||||
|
||||
# Override target IP
|
||||
ros2 run udp_teleop keyboard_control \
|
||||
--ros-args -p udp_ip:=192.168.4.1 -p udp_port:=8888
|
||||
```
|
||||
|
||||
**Keyboard mappings**:
|
||||
- Chassis: W/S (forward/back), A/D (strafe), Q/E (rotate)
|
||||
- Arm: ↑/↓ (height), 2-6 (select joint), ←/→ (adjust angle)
|
||||
|
||||
**Important**: Must use `ros2 run`, not `ros2 launch` — the `stdin` keyboard backend requires an interactive terminal.
|
||||
|
||||
### Python Arm Control Tools
|
||||
|
||||
```bash
|
||||
# Direct joint command with interpolation
|
||||
python tools/udp_control.py joints \
|
||||
--height -100 --j2 10 --j3 20 --j4 30 \
|
||||
--duration 1.0 --rate 20
|
||||
|
||||
# Cartesian pose mode (uses inverse kinematics)
|
||||
python tools/udp_control.py pose \
|
||||
--x 150 --y 50 --z -100 --phi 45 \
|
||||
--duration 1.0
|
||||
|
||||
# Dry run (print commands without sending)
|
||||
python tools/udp_control.py pose --x 200 --y 0 --z -50 --phi 0 --dry-run
|
||||
|
||||
# Camera frame capture
|
||||
python tools/camera_capture.py --ip 192.168.4.1
|
||||
python tools/camera_capture.py --scan # Auto-detect camera on subnet
|
||||
```
|
||||
|
||||
### UDP Testing
|
||||
|
||||
```bash
|
||||
# Start echo server
|
||||
python tools/udp_server.py
|
||||
|
||||
# Send test commands
|
||||
echo 'XYW:100:0:0:XZHY' | nc -u 192.168.4.1 8888
|
||||
echo 'JXB:-100:10:20:30:0:0:0:0:EZHY' | nc -u 192.168.4.1 8888
|
||||
```
|
||||
|
||||
## Architecture Details
|
||||
|
||||
### ESP32-S3 Dual-Core Design
|
||||
|
||||
- **Core 0**: Camera capture loop (OV2640 → JPEG)
|
||||
- **Core 1**: WiFi streaming (MJPEG HTTP server on port 80)
|
||||
- **AsyncUDP**: Non-blocking UDP command receiver runs on Core 1, handled via interrupt callbacks
|
||||
|
||||
The dual-core split ensures camera capture never blocks on WiFi transmission. UDP commands are processed asynchronously and do not interfere with streaming.
|
||||
|
||||
### UDP Protocol
|
||||
|
||||
All commands are ASCII text ending with terminator (varies by command type):
|
||||
|
||||
```
|
||||
# Chassis control (XYZ cartesian velocity)
|
||||
XYW:<X_speed>:<Y_speed>:<W_angular>:XZHY\n
|
||||
|
||||
# Arm control (6 motors: height, J2-J6)
|
||||
JXB:<height>:<J2>:<J3>:<J4>:<J5>:<J6>:0:0:EZHY\n
|
||||
|
||||
# Laser control
|
||||
LASERON\n
|
||||
LASEROFF\n
|
||||
|
||||
# Serial passthrough (any payload with ZHY or \n terminator)
|
||||
<payload>ZHY\n
|
||||
```
|
||||
|
||||
**Critical**: The ESP32 firmware parses based on terminator suffix, not command prefix.
|
||||
|
||||
### Mechanical Arm Coordinate System & Kinematics
|
||||
|
||||
**Coordinate frame**: Base frame with Z-axis pointing UP (not down). Origin at the bottom of the J1 linear slide.
|
||||
|
||||
**Height coordinate (d1)**:
|
||||
- User-facing coordinate: `-290 mm` (bottom) to `0 mm` (top)
|
||||
- Physical meaning: vertical position of J2 relative to base origin
|
||||
- **Z-up convention**: positive d1 = higher position
|
||||
|
||||
**Planar joints (J2, J3, J4)**:
|
||||
- All three rotate around vertical Z-axis in the XY plane
|
||||
- J2 is base rotation, J3/J4 are elbow/wrist
|
||||
- TCP yaw angle `phi = J2 + J3 + J4` (additive)
|
||||
|
||||
**Geometry parameters** (see `docs/arm.md` for full derivation):
|
||||
- `L1 = 125 mm`: J2-J3 link length
|
||||
- `L2 = 125 mm`: J3-J4 link length
|
||||
- `x4 = 110 mm`: J4-to-TCP horizontal offset
|
||||
- `z4 = 80 mm`: J4-to-TCP vertical offset (variable: 55mm when gripper down, -100mm when up)
|
||||
|
||||
**Zero offsets**: Physical mechanical zero does not align with math zero (straight line):
|
||||
- `J2_zero = 3°`, `J3_zero = 7°`, `J4_zero = 25°`
|
||||
- UDP command angles = math angles + zero offsets
|
||||
|
||||
**Inverse kinematics** (`tools/udp_control.py`):
|
||||
- Solves for joint angles given TCP pose `(x, y, z, phi)`
|
||||
- Uses standard 2-link planar arm solution (atan2-based)
|
||||
- Two solutions: `--elbow-up` vs default elbow-down
|
||||
- Validates workspace limits and singularity checks
|
||||
- See `docs/arm.md` for full mathematical derivation
|
||||
|
||||
**State persistence**: `tools/.udp_control_state.json` caches last sent joint command. This enables smooth interpolated motion from previous position without re-homing. Use `--no-state-cache` to disable.
|
||||
|
||||
### ROS 2 Keyboard Backend Selection
|
||||
|
||||
The `udp_teleop` node supports three keyboard input backends:
|
||||
|
||||
- `stdin` (default on Linux/macOS): Terminal raw mode, zero dependencies, **requires interactive terminal**
|
||||
- `pynput`: Cross-platform library, works in background
|
||||
- `win_poll`: Windows-specific Win32 API polling
|
||||
|
||||
Backend auto-selected by platform. Override with `keyboard_backend` parameter in `config/params.yaml`.
|
||||
|
||||
**Limitation**: `stdin` backend fails when launched via `ros2 launch` because child processes lack TTY. Always use `ros2 run` for interactive keyboard control.
|
||||
|
||||
### Camera Capture Tool Auto-Detection
|
||||
|
||||
`camera_capture.py` implements subnet scanning and ESP32 identification:
|
||||
|
||||
1. Probes common DHCP IPs on local subnet (`x.x.x.1`, `x.x.x.100-110`, etc.)
|
||||
2. Verifies ESP32 by checking `/status` endpoint for JSON keys `capture_fps` and `has_client`
|
||||
3. Falls back to full subnet scan if not found in common range
|
||||
4. Connects to `/stream` MJPEG endpoint, parses multipart frames, extracts first valid JPEG (SOI `0xFFD8` to EOI `0xFFD9`)
|
||||
|
||||
Use `--scan` to force full subnet scan, or `--ip` to skip detection.
|
||||
|
||||
## Environment Setup
|
||||
|
||||
### ROS 2 Humble via Conda (robostack)
|
||||
|
||||
```bash
|
||||
# Create environment (one-time)
|
||||
conda create -n ros2_humble -c robostack-staging -c conda-forge ros-humble-desktop
|
||||
conda activate ros2_humble
|
||||
conda install -c robostack-staging -c conda-forge colcon-common-extensions
|
||||
pip install pynput
|
||||
|
||||
# Every session
|
||||
conda activate ros2_humble
|
||||
cd ros2
|
||||
source install/setup.bash # After first colcon build
|
||||
```
|
||||
|
||||
**Alternative**: Native apt installation on Ubuntu 22.04 — see `ros2/README.md`.
|
||||
|
||||
### PlatformIO ESP32
|
||||
|
||||
```bash
|
||||
pip install platformio
|
||||
cd jxbeye
|
||||
pio pkg install # Install dependencies
|
||||
```
|
||||
|
||||
**Board configuration** (`platformio.ini`):
|
||||
- Custom board definition: `esp32-s3-wroom-1-n16r8`
|
||||
- PSRAM: Octal mode (`board_build.psram_type = octal`)
|
||||
- Flash: 16MB QIO mode at 80MHz
|
||||
- Partition table: `default_16MB.csv`
|
||||
|
||||
## Important Files
|
||||
|
||||
### Configuration
|
||||
|
||||
- `jxbeye/platformio.ini` — ESP32 build config (PSRAM settings critical)
|
||||
- `ros2/src/udp_teleop/config/params.yaml` — ROS node parameters (IP, speeds, steps)
|
||||
- `tools/.udp_control_state.json` — Cached arm joint state for interpolation
|
||||
|
||||
### Documentation
|
||||
|
||||
- `docs/arm.md` — Full inverse kinematics derivation with LaTeX equations
|
||||
- `README.md` — Project overview and quick start
|
||||
- `ros2/src/udp_teleop/README.md` — ROS package details and keyboard mappings
|
||||
|
||||
### Core Implementations
|
||||
|
||||
- `jxbeye/src/main.cpp` — ESP32 dual-core firmware entry point
|
||||
- `ros2/src/udp_teleop/udp_teleop/keyboard_control.py` — ROS keyboard node
|
||||
- `tools/udp_control.py` — Standalone arm controller with full IK solver
|
||||
- `tools/camera_capture.py` — MJPEG stream frame extractor
|
||||
|
||||
## Common Tasks
|
||||
|
||||
**Change arm geometry parameters**: Edit constants in `tools/udp_control.py` (`DEFAULT_L1`, `DEFAULT_L2`, `DEFAULT_X4`, `DEFAULT_Z4`) or pass as CLI args.
|
||||
|
||||
**Modify joint limits**: Edit `DEFAULT_*_MIN/MAX` in `tools/udp_control.py` or use `--height-min`, `--j2-max`, etc.
|
||||
|
||||
**Adjust interpolation smoothness**: Change `--duration` (total time) and `--rate` (Hz) in `udp_control.py`. Default is 1.0s at 20Hz = 20 steps.
|
||||
|
||||
**Debug UDP protocol**: Use `tools/udp_server.py` as echo server, point ROS/tools at `127.0.0.1:8888` to inspect raw commands.
|
||||
|
||||
**Test kinematics without hardware**: Use `--dry-run` flag with `udp_control.py` to print UDP commands without sending.
|
||||
|
||||
**Verify IK correctness**: Use `--show-fk` flag to compute forward kinematics of the solved joint angles and compare to target pose.
|
||||
|
||||
## Dataset
|
||||
|
||||
The `dataset.zip` and `dataset/` directory contain competition-specific training data (exact format unknown from structure alone).
|
||||
Reference in New Issue
Block a user