# EasyCarla-RL **Repository Path**: drone1024/EasyCarla-RL ## Basic Information - **Project Name**: EasyCarla-RL - **Description**: No description available - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2025-06-09 - **Last Updated**: 2025-06-09 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README 🚀 [**New Dataset Released !**](#-download-dataset) # EasyCarla-RL: A lightweight and beginner-friendly OpenAI Gym environment built on the CARLA simulator ## Overview EasyCarla-RL provides a lightweight and easy-to-use Gym-compatible interface for the CARLA simulator, specifically tailored for reinforcement learning (RL) applications. It integrates essential observation components such as LiDAR scans, ego vehicle states, nearby vehicle information, and waypoints. The environment supports safety-aware learning with reward and cost signals, visualization of waypoints, and customizable parameters including traffic settings, number of vehicles, and sensor range. EasyCarla-RL is designed to help both researchers and beginners efficiently train and evaluate RL agents without heavy engineering overhead.

## Installation Clone the repository: ```bash git clone https://github.com/silverwingsbot/EasyCarla-RL.git cd EasyCarla-RL ``` Install the required dependencies: ```bash pip install -r requirements.txt ``` Install EasyCarla-RL as a local Python package: ```bash pip install -e . ``` Make sure you have a running [CARLA simulator](https://carla.org/) server compatible with your environment. For detailed installation instructions, please refer to the [official CARLA docs](https://carla.readthedocs.io/en/0.9.13/start_quickstart/) ## Quick Start Run a simple demo to interact with the environment: ```bash python easycarla_demo.py ``` This script demonstrates how to: - Create and reset the environment - Select random or autopilot actions - Step through the environment and receive observations, rewards, costs, and done signals Make sure your CARLA server is running before executing the demo. ## Advanced Example: Evaluation with Diffusion Q-Learning For a more advanced usage, you can run a pre-trained [Diffusion Q-Learning](https://github.com/Zhendong-Wang/Diffusion-Policies-for-Offline-RL) agent in the EasyCarla-RL environment: ```bash cd example python run_dql_in_carla.py ``` Make sure you have downloaded or prepared a trained model checkpoint under the `example/params_dql/` directory. This example demonstrates: - Loading a pre-trained RL agent - Interacting with EasyCarla-RL for evaluation - Evaluating the performance of a real RL model on a simulated autonomous driving task ## 📥 Download Dataset This repository provides an offline dataset for training and evaluating RL agents in the EasyCarla-RL environment. This dataset includes over **7,000 trajectories** and **1.1 million timesteps**, collected from a mix of expert and random policies (with an **8:2 ratio** of expert to random), recorded in the Town03 map. The data is stored in **HDF5 format**. You can download it from either of the following sources: * [Download from Hugging Face (direct link)](https://huggingface.co/datasets/silverwingsbot/easycarla/resolve/main/easycarla_offline_dataset.hdf5) * [Download from 百度网盘 (提取码: 2049)](https://pan.baidu.com/s/1yhCFzl4RFHzxfszebYnOIg?pwd=2049) Filename: `easycarla_offline_dataset.hdf5` Size: \~2.76 GB Format: HDF5 ### Dataset Structure (HDF5) Each sample in the dataset includes the following fields: ``` / (root) ├── observations → shape: [N, 307] # concatenated: ego_state + lane_info + lidar + nearby_vehicles + waypoints ├── actions → shape: [N, 3] # [throttle, steer, brake] ├── rewards → shape: [N] # scalar reward per step ├── costs → shape: [N] # safety-related cost signal per step ├── done → shape: [N] # 1 if episode ends ├── next_observations → shape: [N, 307] # next-step observations, same format as observations ├── info → dict containing: │ ├── is_collision → shape: [N] # 1 if collision occurs │ └── is_off_road → shape: [N] # 1 if vehicle leaves the road ``` * `N` is the number of total timesteps across all episodes (\~1.1 million). * `observations` and `next_observations` are 307-dimensional vectors formed by concatenating: * `ego_state` (9) + `lane_info` (2) + `lidar` (240) + `nearby_vehicles` (20) + `waypoints` (36) ### Observation Format Each observation in the dataset is stored as a **307-dimensional flat vector**, constructed by concatenating several components in the following order: ```python # Flattening function used during data generation def flatten_obs(obs_dict): return np.concatenate([ obs_dict['ego_state'], # 9 dimensions obs_dict['lane_info'], # 2 dimensions obs_dict['lidar'], # 240 dimensions obs_dict['nearby_vehicles'], # 20 dimensions obs_dict['waypoints'] # 36 dimensions ]).astype(np.float32) # Total: 307 dimensions ``` This format allows for efficient training of neural networks while preserving critical spatial and semantic information. ### How to Load and Train with HDF5 Dataset？ This example shows how to load the offline dataset and use it in a typical RL training loop. The model here is a placeholder — you can plug in any behavior cloning, Q-learning, or actor-critic model. ```python import h5py import torch import numpy as np # === Load dataset from HDF5 === with h5py.File('easycarla_offline_dataset.hdf5', 'r') as f: observations = torch.tensor(f['observations'][:], dtype=torch.float32) actions = torch.tensor(f['actions'][:], dtype=torch.float32) rewards = torch.tensor(f['rewards'][:], dtype=torch.float32) next_observations = torch.tensor(f['next_observations'][:], dtype=torch.float32) dones = torch.tensor(f['done'][:], dtype=torch.float32) # === (Optional) check shape info === print("observations:", observations.shape) print("actions:", actions.shape) # === Placeholder model example === class YourModel(torch.nn.Module): def __init__(self, obs_dim, act_dim): super().__init__() # define your model here pass def forward(self, obs): # define forward pass return None # === Training setup === model = YourModel(obs_dim=observations.shape[1], act_dim=actions.shape[1]) optimizer = torch.optim.Adam(model.parameters(), lr=3e-4) loss_fn = torch.nn.MSELoss() # === Offline RL training loop === for epoch in range(1, 11): # e.g. 10 epochs for step in range(100): # e.g. 100 steps per epoch # sample random batch idx = np.random.randint(0, len(observations), size=256) obs_batch = observations[idx] act_batch = actions[idx] rew_batch = rewards[idx] next_obs_batch = next_observations[idx] done_batch = dones[idx] # forward, compute loss pred = model(obs_batch) # e.g. predict action or Q-value loss = loss_fn(pred, act_batch) # just an example # backward and update optimizer.zero_grad() loss.backward() optimizer.step() print(f"[Epoch {epoch}] Loss: {loss.item():.4f} # Replace with your own logging or evaluation") ``` ## Project Structure ``` EasyCarla-RL/ ├── easycarla/ # Main environment module (Python package) │ ├── envs/ │ │ ├── __init__.py │ │ └── carla_env.py # Carla environment wrapper following the Gym API │ └── __init__.py ├── example/ # Advanced example │ ├── agents/ │ ├── params_dql/ │ ├── utils/ │ └── run_dql_in_carla.py # Script to run a pretrained RL model ├── easycarla_demo.py # Quick Start demo script (basic Gym-style environment interaction) ├── requirements.txt ├── setup.py └── README.md ``` ## License This project is licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ## Author Created by [SilverWings](https://github.com/silverwingsbot) ## 💓 Acknowledgement This project is made possible thanks to the following outstanding open-source contributions: - [CARLA](https://github.com/carla-simulator/carla) - [gym-carla](https://github.com/cjy1992/gym-carla) - [Diffusion Q-Learning](https://github.com/Zhendong-Wang/Diffusion-Policies-for-Offline-RL)