# Client-sdk-esp32 **Repository Path**: wang-jie_12/Client-sdk-esp32 ## Basic Information - **Project Name**: Client-sdk-esp32 - **Description**: LiveKit Client SDK for ESP32 series chips. Easily enable real-time audio, video, and data for embedded projects. - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-10-21 - **Last Updated**: 2026-11-01 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README The LiveKit icon, the name of the repository and some sample code in the background. # ESP32 SDK for LiveKit [![API Reference](https://img.shields.io/badge/API_Reference-blue)](https://livekit.github.io/client-sdk-esp32/) [![Component Registry](https://components.espressif.com/components/livekit/livekit/badge.svg)](https://components.espressif.com/components/livekit/livekit) [![CI Status](https://img.shields.io/github/actions/workflow/status/livekit/client-sdk-esp32/ci.yml?label=CI)](https://github.com/livekit/client-sdk-esp32/actions/workflows/ci.yml) Use this SDK to add realtime video, audio and data features to your ESP32 projects. By connecting to [LiveKit](https://livekit.io/) Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI, live streaming, or video calls with minimal setup. > [!IMPORTANT] > This SDK is currently in Developer Preview mode and not ready for production use. > There may be bugs and APIs are subject to change during this period. ## Features - **Supported chips**: ESP32-S3 and ESP32-P4 - **Bidirectional audio**: Opus encoding, acoustic echo cancellation (AEC) - **Video publishing**: H.264 encoding, subscribing coming soon - **AI Agents**: interact with agents in the cloud built with [LiveKit Agents](https://docs.livekit.io/agents/) - **Real-time data**: data streams, data packets, remote method calls (RPC) ## Getting Started Before you begin, please ensure you have ESP IDF v5.4 or higher installed on your system (see installation guide for [macOS/Linux](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/linux-macos-setup.html) or [Windows](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/windows-setup.html)). ### Examples One of the best ways to get started with LiveKit is by reviewing the [examples](./components/livekit/examples/) and choosing one as a starting point for your project: - [**Voice AI Agent**](./components/livekit/examples/voice_agent/README.md): Conversational AI voice agent that interacts with hardware based on user requests. - [**Minimal**](./components/livekit/examples/minimal/README.md): Basic example of connecting to a LiveKit room with bidirectional audio. - [**Minimal Video**](./components/livekit/examples/minimal_video/README.md): Equivalent to the minimal example with video publishing. - [**Custom Hardware**](./components/livekit/examples/custom_hardware/README.md): Bidirectional audio on custom ESP32-S3 hardware with manual codec initialization (ES8311 + ES7210). Once you have chosen an example to be your starting point, create a fresh project from it locally using the following command: ```sh idf.py create-project-from-example "livekit/livekit=0.3.10:" ``` Substitute *\* for the example's directory name. ### Installation If you would like to add LiveKit to your existing application, add it as a dependency using IDF: ```sh idf.py add-dependency "livekit/livekit=0.3.10" ``` > [!IMPORTANT] > Be sure to pin to a specific version as shown in the command above, as subsequent v0.x.x releases may have breaking changes. ## API Overview With LiveKit added as a dependency to your application, include the LiveKit header and invoke `livekit_system_init` early in your application's main function: ```c #include "livekit.h" void app_main(void) { livekit_system_init(); // Your application code... } ``` ### Configure media pipeline LiveKit for ESP32 puts your application in control of the media pipeline; your application configures a capturer and/or renderer and provides their handles when creating a room. #### Capturer: input from camera/microphone - Required for rooms which will publish media tracks - Created using the Espressif [*esp_capture*](https://components.espressif.com/components/espressif/esp_capture/) component - Capture audio capture over I2S, video from MIPI CSI or DVI cameras - After configuration, you will provide the `esp_capture_handle_t` when creating a room #### Renderer: output to display/speaker - Required for rooms which will subscribe to media tracks - Created using the Espressif [*av_render*](https://components.espressif.com/components/tempotian/av_render/) component - Playback audio over I2S, video on LCD displays supported by *esp_lcd* - After configuration, you will provide the `av_render_handle_t` when creating a room ### Create room Create a room object, specifying your capturer, renderer, and handlers for room events: ```c static livekit_room_handle_t room_handle = NULL; livekit_room_options_t room_options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .capturer = my_capturer }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = my_renderer }, .on_state_changed = on_state_changed, .on_participant_info = on_participant_info }; if (livekit_room_create(&room_handle, &room_options) != LIVEKIT_ERR_NONE) { ESP_LOGE(TAG, "Failed to create room object"); } ``` This example does not show all available fields in room options—please refer to the [API reference](https://livekit.github.io/client-sdk-esp32/group__Lifecycle.html#structlivekit__room__options__t) for an extensive list. Typically, you will want to create the room object early in your application's lifecycle, and connect/disconnect as necessary based on user interaction. ### Connect room With a room room handle, connect by providing a server URL and token: ```c livekit_room_connect(room_handle, "", ""); ``` The connect method is asynchronous; use your `on_state_changed` handler provided in room options to get notified when the connection is established or fails (e.g. due to an expired token, etc.). Once connected, media exchange will begin: 1. If a capturer was provided, video and/or audio tracks will be published. 2. If a renderer was provided, the first video and/or audio tracks in the room will be subscribed to. ### Real-time data In addition to real-time audio and video, LiveKit offers several methods for exchange real-time data between participants in a room. #### Remote method call (RPC) Define an RPC handler: ```c static void get_cpu_temp(const livekit_rpc_invocation_t* invocation, void* ctx) { float temp = board_get_temp(); char temp_string[16]; snprintf(temp_string, sizeof(temp_string), "%.2f", temp); livekit_rpc_return_ok(temp_string); } ``` Register the handler on the room to allow it to be invoked by remote participants: ```c livekit_room_rpc_register(room_handle, "get_cpu_temp", get_cpu_temp); ``` > [!TIP] > In the [*voice_agent*](./components/livekit/examples/voice_agent/) example, RPC is used to allow an AI agent to interact > with hardware by defining a series of methods for the agent to invoke. #### Data streams Data streams deliver ordered sequences of chunks (Header → Chunks → Trailer) over the reliable data channel. There are two kinds: - **Text streams**: Each chunk is independently valid UTF-8 and can be used directly as a string. When sending, data is automatically chunked at UTF-8 character boundaries so multi-byte characters are never split. Used for chat messages, transcriptions, etc. - **Byte streams**: Chunks are raw binary fragments of a larger payload, split without delimiters. Chunks must be aggregated by the application to reconstruct the original data. Used for file transfers, binary payloads, etc. **Receiving** — register a handler for a topic: ```c static void on_text_chunk(const livekit_data_stream_chunk_t* chunk, void* ctx) { ESP_LOGI(TAG, "%.*s", (int)chunk->content_size, (const char*)chunk->content); } livekit_data_stream_handler_t handler = { .on_recv = on_text_chunk, }; livekit_room_data_stream_topic_register(room_handle, "lk.chat", &handler); ``` The optional `on_open` and `on_close` callbacks in the handler can be set for additional behavior such as logging stream metadata or detecting abnormal closure. **Sending a text stream:** ```c livekit_data_stream_options_t opts = { .topic = "lk.chat", .is_text = true }; livekit_data_stream_handle_t stream; livekit_room_data_stream_open(room_handle, &opts, &stream); livekit_room_data_stream_write(room_handle, stream, (const uint8_t*)"hello world", 11); livekit_room_data_stream_close(room_handle, stream); ``` **Sending a byte stream:** ```c uint8_t image_data[4096]; size_t image_size = read_image(image_data, sizeof(image_data)); livekit_data_stream_options_t opts = { .topic = "image", .is_text = false, .total_length = image_size, .has_total_length = true, }; livekit_data_stream_handle_t stream; livekit_room_data_stream_open(room_handle, &opts, &stream); livekit_room_data_stream_write(room_handle, stream, image_data, image_size); livekit_room_data_stream_close(room_handle, stream); ``` #### User packets Publish a user packet containing a raw data payload under a specific topic: ```c const char* command = "G5 I0 J3 P0 Q-3 X2 Y3"; livekit_payload_t payload = { .bytes = (uint8_t*)command, .size = strlen(command) }; livekit_data_publish_options_t options = { .payload = &payload, .topic = "gcode", .lossy = false, .destination_identities = (char*[]){ "printer-1" }, .destination_identities_count = 1 }; livekit_room_publish_data(room_handle, &options); ``` ## Documentation Please refer to the [LiveKit Docs](https://docs.livekit.io/home/) for an introduction to the platform and its features. For more detail about the APIs available in this SDK, please refer to the [API Reference](https://livekit.github.io/client-sdk-esp32/). ## Known Issues - In some cases, a remote participant leaving the room can lead to a disconnect. ## Getting Help & Contributing We invite you to join our [developer community](https://community.livekit.io/c/robotics) to ask questions, suggest improvements, or discuss how you can best contribute to this SDK.
LiveKit Ecosystem
Agents SDKsPython · Node.js
LiveKit SDKsBrowser · Swift · Android · Flutter · React Native · Rust · Node.js · Python · Unity · Unity (WebGL) · ESP32 · C++
Starter AppsPython Agent · TypeScript Agent · React App · SwiftUI App · Android App · Flutter App · React Native App · Web Embed
UI ComponentsReact · Android Compose · SwiftUI · Flutter
Server APIsNode.js · Golang · Ruby · Java/Kotlin · Python · Rust · PHP (community) · .NET (community)
ResourcesDocs · Docs MCP Server · CLI · LiveKit Cloud
LiveKit Server OSSLiveKit server · Egress · Ingress · SIP
CommunityDeveloper Community · Slack · X · YouTube