User Manual

1. Before You Start

1.1 Required Items

1.2 Notes

• Use a USB cable that supports data transfer. Charge-only cables cannot flash firmware or show serial logs.
• Format the MicroSD card as FAT32.
• ESP32-S3 supports 2.4G Wi-Fi only. Do not use a 5G-only network.
• Camera, audio, display, and Wi-Fi features all consume memory. During development, verify basic features first, then enable advanced features one by one.
• This manual uses the BigSmart Xiaozhi firmware under E:\RYMCU\xiaozhi as the software reference. If you use other firmware, button behavior and menu flow may differ.

2. Development Environment

2.1 Recommended Environment

3. Firmware Flashing

3.1 Flash the Repository Merged Firmware Image

This repository provides BigSmart merged firmware images:

firmware/rymcu-V2.3.28-merged.bin
firmware/rymcu-V2.3.19-merged.bin
firmware/xiaozhi-esp32-merged.bin
firmware/espressif-brookesia-merged.bin

The RYMCU official firmware rymcu-V2.3.28-merged.bin is recommended as the default. rymcu-V2.3.19-merged.bin remains available as a legacy fallback.

The V2.3.28 prebuilt image includes Weather, Calendar, Codex Status, Smart Home MQTT/Endpoint, the NES launcher, and NES dedicated boot mode. It does not include the later NES display, audio, and volume-control changes from the 2026-07-28 commit b1d1c08.

You can flash it with ESP-IDF, esptool.py, or a GUI flashing tool. A common command-line method is:

esptool.py --chip esp32s3 -p COM_PORT -b 460800 write_flash 0x0 firmware\rymcu-V2.3.28-merged.bin

3.2 Enter Download Mode

If automatic flashing fails, enter download mode manually:

1. Hold the Boot button.
2. Reset or power-cycle the board.
3. Release the Boot button.
4. Run the flashing command again.

4. First Boot and Wi-Fi Provisioning

4.1 Startup Check

After flashing, the device restarts. In a normal startup:

• The screen turns on and shows the Xiaozhi UI.
• Serial logs print board initialization information.
• If an SD card is present, logs show it mounted at /sdcard.
• If Wi-Fi is not configured, the device enters provisioning.

4.2 Wi-Fi Provisioning

The Xiaozhi reference project includes BluFi provisioning documentation. A typical flow:

1. Enable WiFi Configuration Method -> Esp Blufi during build configuration.
2. On first boot without saved Wi-Fi credentials, the device enters provisioning automatically.
3. Use the EspBlufi app or another compatible BluFi client to find the device.
4. Connect to the device and enter the 2.4G Wi-Fi SSID and password.
5. After connection succeeds, the device saves the configuration and connects automatically on later boots.

To reprovision Wi-Fi after the firmware is running, use one of these methods:

• Click the Boot button during startup to enter provisioning.
• Request reprovisioning through the MCP tool self.system.reconfigure_wifi.
• Clear NVS or erase data during flashing, then reboot.

5. Buttons and Basic Interaction

6. Voice Assistant

6.1 Conversation

1. Make sure the device is connected to Wi-Fi.
2. Click Boot to switch to conversation/listening state, or hold GPIO10/PTT to start speaking.
3. Speak your question or command.
4. Release PTT or wait for the device to stop listening.
5. The device plays the response through the speaker and updates the screen state.

6.2 AEC Switching

Device-side AEC suppresses speaker echo. When enabled in firmware, double-click Boot while idle to toggle AEC. If the device frequently captures its own speaker output during recognition, try enabling AEC. In simpler acoustic environments, you may disable it to reduce processing load.

7. SD Card Usage

7.1 Format and Directories

Format the MicroSD card as FAT32 and create directories as needed:

/sdcard
├── music
│   ├── song1.mp3
│   └── song2.mp3
├── test.mp3
└── videos
    ├── demo.mjpg
    ├── demo.mp3
    └── demo.fps

7.2 Check SD Card Mounting

If the startup log contains SD card mounted at /sdcard, the card mounted successfully. You can also call self.media.list_mp3_files to check whether MP3 files can be read.

8. Local MP3 Playback

8.1 Supported Tools

8.2 Examples

Play music from the SD card:

{
  "tool": "self.media.play_mp3",
  "arguments": {
    "filepath": "/sdcard/music/song1.mp3"
  }
}

List MP3 files under /sdcard:

{
  "tool": "self.media.list_mp3_files",
  "arguments": {
    "directory": "/sdcard"
  }
}

9. Internet Radio

After the device is online, internet radio tools can play network audio streams.

Example:

{
  "tool": "self.radio.play_url",
  "arguments": {
    "station_name": "Hit FM"
  }
}

10. Weather, Calendar, Codex, and NES

The V2.3.28 firmware adds or improves several Launcher app entries:

The Codex app only provides the device-side status and bridge settings entry; PC-side bridge scripts are not published in the BigSmart-Open repository. NES behavior is limited to the V2.3.28 prebuilt image and does not include the later NES display, audio, and volume-control changes from the 2026-07-28 commit b1d1c08.

11. RGB LED and Smart Home MQTT/Endpoint

11.1 Direct RGB Control

Set the RGB LED:

{
  "tool": "self.light.set_rgb_color",
  "arguments": {
    "red": 255,
    "green": 100,
    "blue": 50
  }
}

Turn off the RGB LED:

{
  "tool": "self.light.turn_off",
  "arguments": {}
}

11.2 Smart Home MQTT Overview

The V2.3.28 firmware uses the RYMCU Smart Home MQTT protocol to control third-party smart home devices. The MQTT topic root is rymcu/home/v1; devices interact with BigSmart through pair codes, discovery announcements, state reports, and command topics.

There are two upper-level control paths:

• Local Xiaozhi session MCP: the voice service calls self.smart_home.* tools on the device.
• External MCP Endpoint Bridge: an endpoint written during provisioning calls the same self.smart_home.* tools, and Smart Home MQTT still sends the final commands.

Common tools:

Set the broker:

{
  "tool": "self.smart_home.set_mqtt_endpoint",
  "arguments": {
    "endpoint": "broker-cn.emqx.io:1883",
    "confirm": true
  }
}

Connect and scan devices:

{
  "tool": "self.smart_home.connect",
  "arguments": {}
}

{
  "tool": "self.smart_home.scan_devices",
  "arguments": {}
}

Control a discovered device:

{
  "tool": "self.smart_home.control_device",
  "arguments": {
    "device_id": "lamp-bedroom-01",
    "action": "turn_on",
    "entity_id": "main",
    "params": "{}"
  }
}

Provisioning custom data also supports two prefixes:

• mqtt:<host[:port]>: save the Smart Home MQTT broker.
• mcp:<endpoint>: save the external MCP Endpoint address.

12. IMU Attitude and Shake Detection

BigSmart includes a QMI8658 six-axis sensor. After firmware startup, sensor data is read periodically and shake detection is started. You can read attitude angles, acceleration, and gyroscope data through MCP tools:

{
  "tool": "self.imu.get_attitude_angles",
  "arguments": {}
}

Typical uses:

• Shake to wake or switch UI.
• Motion-controlled mini games.
• Device attitude display.
• Interactive installation triggers.

13. Camera Usage

BigSmart uses a GC0308 camera with 640 x 480 @ 16 FPS hardware support. The reference firmware uses lazy initialization: the camera is not initialized during startup and is only initialized on the first camera request, reducing startup memory pressure.

Suggestions:

• Make sure GC0308-related firmware configuration is enabled.
• Call the camera only when image capability is needed to avoid competing with audio, Wi-Fi, and large UI tasks for memory.
• If the image orientation is wrong, check mirror and flip settings in firmware.

14. Troubleshooting

15. Hardware Maintenance

• Power off before plugging or unplugging the display, camera, microphone board, or other flex cables.
• Pay attention to battery polarity and charging safety when using battery power.
• Enclosure files are in enclosure/. When modifying the structure, consider screen, button, microphone openings, and speaker acoustic cavity.
• Before adding new peripherals, check Hardware Configuration to avoid GPIO conflicts.

16. References

• Project hardware documentation: hardware.md