# leju_robot_rl **Repository Path**: leju-robot/leju_robot_rl ## Basic Information - **Project Name**: leju_robot_rl - **Description**: No description available - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 2 - **Created**: 2025-09-01 - **Last Updated**: 2025-09-18 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Template for Isaac Lab Projects ## Kuavo-S42，S46 [![IsaacSim](https://img.shields.io/badge/IsaacSim-4.2.0-silver.svg)](https://docs.isaacsim.omniverse.nvidia.com/latest/index.html) [![Python](https://img.shields.io/badge/python-3.10-blue.svg)](https://docs.python.org/3/whatsnew/3.10.html) [![Linux platform](https://img.shields.io/badge/platform-linux--64-orange.svg)](https://releases.ubuntu.com/20.04/) [![Windows platform](https://img.shields.io/badge/platform-windows--64-orange.svg)](https://www.microsoft.com/en-us/) [![pre-commit](https://img.shields.io/github/actions/workflow/status/isaac-sim/IsaacLab/pre-commit.yaml?logo=pre-commit&logoColor=white&label=pre-commit&color=brightgreen)](https://github.com/isaac-sim/IsaacLab/actions/workflows/pre-commit.yaml) [![docs status](https://img.shields.io/github/actions/workflow/status/isaac-sim/IsaacLab/docs.yaml?label=docs&color=brightgreen)](https://github.com/isaac-sim/IsaacLab/actions/workflows/docs.yaml) [![License](https://img.shields.io/badge/license-BSD--3-yellow.svg)](https://opensource.org/licenses/BSD-3-Clause) [![License](https://img.shields.io/badge/license-Apache--2.0-yellow.svg)](https://opensource.org/license/apache-2-0) 基于NVIDIA Isaacsim平台的人形机器人强化学习训练框架 ### 📥 安装指南 ### LFS安装 ```bash # 安装Git LFS # Ubuntu/Debian系统 sudo apt update sudo apt install git-lfs # CentOS/RHEL系统 sudo yum install git-lfs # 或者通过官方脚本安装 curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash sudo apt install git-lfs # 初始化Git LFS git lfs install # 验证安装 git lfs version # 如果需要追踪大文件（例如模型文件、数据集等） git lfs track "*.onnx" git lfs track "*.pth" git lfs track "*.bin" git add .gitattributes git commit -m "Track large files with Git LFS" ``` ### 环境要求 - Ubuntu 20.04/22.04 LTS - Isaac Sim 4.2 - Isaac Lab 1.4.1 - rsl-rl[仓库地址](https://www.lejuhub.com/liuzhijie/rsl_rl) 建议使用该commit：386875591808cfd1462a42446b1fa0a22ac161d0 ### Ubuntu20.04 源码安装isaaclab - 安装isaac-sim-4.2版本 下载isaac-sim4.2版本的源码，解压到主目录，并添加路径 export ISAACSIM_PATH="${HOME}/isaacsim" export ISAACSIM_PYTHON_EXE="${ISAACSIM_PATH}/python.sh" - 验证isaac-sim4.2版本是否安装成功 步骤1：${ISAACSIM_PATH}/isaac-sim.sh 步骤2：${ISAACSIM_PYTHON_EXE} ${ISAACSIM_PATH}/standalone_examples/api/omni.isaac.core/add_cubes.py - 下载isaac-lab-1.4.1版本源码，并解压到主目录 - 构建isaac-lab与isaac-sim的链接 cd IsaacLab ln -s path_to_isaac_sim _isaac_sim - 使用isaac_lab命令创建conda环境 ./isaaclab.sh --conda my_env - 升级pip /home/yours_user/isaacsim42/kit/python/bin/python3 -m pip install --upgrade pip - 安装isaac-lab： ./isaaclab.sh --install rl_games 安装过程中git clone 有时候会出错，设置下git配置： 1. 增加网络缓存： git config --global http.postBuffer 524288000 2. 设置连接超时： git config --global http.timeout 600 ， git config --global https.timeout 600 ### 训练配置 使用VS Code调试配置进行训练： ```json { "name": "Train Flat-Kuavo-S42", "type": "debugpy", "request": "launch", "args": [ "--task", "Legged-Isaac-Velocity-Flat-Kuavo-S42-v0", "--num_envs", "4096", "--headless" ], "program": "${workspaceFolder}/scripts/rsl_rl/train.py", "console": "integratedTerminal", } ``` 或通过命令行启动： ```bash python scripts/rsl_rl/train.py \ --task Legged-Isaac-Velocity-Flat-Kuavo-S42-v0 \ --num_envs 4096 \ --headless ``` ### 可视化测试 播放模式配置： ```json { "name": "Play Flat-Kuavo-S42", "type": "debugpy", "request": "launch", "args": [ "--task", "Legged-Isaac-Velocity-Flat-Kuavo-S42-Play-v0", "--num_envs", "32" ], "program": "${workspaceFolder}/scripts/rsl_rl/play.py", "console": "integratedTerminal", } ``` 命令行启动： ```bash python scripts/rsl_rl/play.py \ --task Legged-Isaac-Velocity-Flat-Kuavo-S42-Play-v0 \ --num_envs 32 ``` isaaclab修改： 需要修改isaaclab内关于加速度更新的逻辑，在update处不更新motor的加速度，减低更新频率。但若需要观测电机加速度，不建议屏蔽此处代码。 ```python #/{isaaclab_path}source/extensions/omni.isaac.lab/omni/isaac/lab/assets/articulation/articulation_data.py:82 -- def update(self, dt: float): # update the simulation timestamp self._sim_timestamp += dt # Trigger an update of the joint acceleration buffer at a higher frequency # since we do finite differencing. self.joint_acc ++ def update(self, dt: float): # update the simulation timestamp self._sim_timestamp += dt # Trigger an update of the joint acceleration buffer at a higher frequency # since we do finite differencing. #self.joint_acc ``` 对于 ‘leju_robot_rl/exts/ext_template/ext_template/tasks/locomotion/velocity/config/s42/rough_env_cfg.py’ 和 ‘leju_robot_rl/exts/ext_template/ext_template/tasks/locomotion/velocity/mdp/rewards.py’的修改： ```python ++ gravity_aligned_when_stopping = RewTerm( func=mdp.gravity_aligned_when_stopping, weight=0.1, params={ "command_name": "base_velocity", }, ) ++ def gravity_aligned_when_stopping( env: ManagerBasedRLEnv, command_name: str, asset_cfg: SceneEntityCfg = SceneEntityCfg("robot") ) -> torch.Tensor: is_zero_cmd = torch.norm(env.command_manager.get_command(command_name)[:, :2], dim=1) < 0.05 asset: Articulation = env.scene[asset_cfg.name] # static_flag = getattr(gravity_aligned_when_stopping, "printed", False) # if not static_flag and env.num_envs > 0: # print("\n============ DEBUG: Robot Properties ============") # print("asset.data attributes:") # for attr in dir(asset.data): # if not attr.startswith('_'): # try: # value = getattr(asset.data, attr) # if isinstance(value, torch.Tensor): # print(f" - {attr}: Tensor with shape {value.shape}") # else: # print(f" - {attr}: {type(value)}") # except Exception as e: # print(f" - {attr}: Error accessing - {e}") # print("\nChecking specific properties for COM and orientation:") # if hasattr(asset.data, 'root_link_quat_w'): # print(f" - root_link_quat_w: {asset.data.root_link_quat_w.shape}") # if hasattr(asset.data, 'root_com_pos_w'): # print(f" - root_com_pos_w: {asset.data.root_com_pos_w.shape}") # elif hasattr(asset.data, 'body_com_pos_w'): # print(f" - body_com_pos_w: {asset.data.body_com_pos_w.shape}") # gravity_aligned_when_stopping.printed = True # print("===============================================\n") # # 获取躯干的姿态四元数 root_quat = asset.data.root_link_quat_w # 计算pitch角度 w, x, y, z = root_quat[:, 0], root_quat[:, 1], root_quat[:, 2], root_quat[:, 3] pitch = torch.asin(2.0 * (w * y - x * z)) #pitch接近0时奖励最高 reward = torch.exp(-5.0 * torch.square(pitch)) masked_reward = torch.zeros_like(reward) masked_reward[is_zero_cmd] = reward[is_zero_cmd] return masked_reward ``` 加入这个奖励是避免在rough环境中训练后机器人无法从行走停止，‘leju_robot_rl/logs/rsl_rl/Kuavo/s42/rough/2025-05-26_17-58-26/exported/policy_s42_5_27.onnx’是可以直接部署的文件，对kuavo_s45,kuavo_s46通用。 训练日志可以用tensorboard查看‘leju_robot_rl/logs/rsl_rl/Kuavo/s42/rough/2025-05-26_17-58-26/events.out.tfevents.1748253526.zelin.444980.0’ 当前训练日志是续训练的记录，在添加新的reward的情况下 ‘‘‘bash python scripts/rsl_rl/train.py \ --task Legged-Isaac-Velocity-Rough-Kuavo-S42-v0 \ --num_envs 4096 \ --headless ’’’ 可以得到和‘leju_robot_rl/logs/rsl_rl/Kuavo/s42/rough/2025-05-26_17-58-26/exported/policy_s42_5_27.onnx’同样的效果 ### 目前的配置是简单示例，只是能实现行走和抗推。缺点包括转向很差、踏地重、对称性差等等。 ##################################### Isaaclab安装与显卡型号、CPU型号、linux内核版本号等相关，需要查找网络资源自行安装，以下内容为早期版本安装教程，仅供参考 ##################################### [![IsaacSim](https://img.shields.io/badge/IsaacSim-4.2.0-silver.svg)](https://docs.omniverse.nvidia.com/isaacsim/latest/overview.html) [![Isaac Lab](https://img.shields.io/badge/IsaacLab-1.4.1-silver)](https://isaac-sim.github.io/IsaacLab) [![Python](https://img.shields.io/badge/python-3.10-blue.svg)](https://docs.python.org/3/whatsnew/3.10.html) [![Linux platform](https://img.shields.io/badge/platform-linux--64-orange.svg)](https://releases.ubuntu.com/20.04/) [![Windows platform](https://img.shields.io/badge/platform-windows--64-orange.svg)](https://www.microsoft.com/en-us/) [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/) [![License](https://img.shields.io/badge/license-MIT-yellow.svg)](https://opensource.org/license/mit) ## Overview This repository serves as a template for building projects or extensions based on Isaac Lab. It allows you to develop in an isolated environment, outside of the core Isaac Lab repository. **Key Features:** - `Isolation` Work outside the core Isaac Lab repository, ensuring that your development efforts remain self-contained. - `Flexibility` This template is set up to allow your code to be run as an extension in Omniverse. **Keywords:** extension, template, isaaclab ## Installation - Install Isaac Lab by following the [installation guide](https://isaac-sim.github.io/IsaacLab/source/setup/installation/index.html). We recommend using the conda installation as it simplifies calling Python scripts from the terminal. - Clone the repository separately from the Isaac Lab installation (i.e. outside the `IsaacLab` directory): ```bash # Option 1: HTTPS git clone https://github.com/isaac-sim/IsaacLabExtensionTemplate.git # Option 2: SSH git clone git@github.com:isaac-sim/IsaacLabExtensionTemplate.git ``` - Throughout the repository, the name `ext_template` only serves as an example and we provide a script to rename all the references to it automatically: ```bash # Enter the repository cd IsaacLabExtensionTemplate # Rename all occurrences of ext_template (in files/directories) to your_fancy_extension_name python scripts/rename_template.py your_fancy_extension_name ``` - Using a python interpreter that has Isaac Lab installed, install the library ```bash python -m pip install -e exts/ext_template ``` - Verify that the extension is correctly installed by running the following command: ```bash python scripts/rsl_rl/train.py --task=Template-Isaac-Velocity-Rough-Anymal-D-v0 ``` ### Set up IDE (Optional) To setup the IDE, please follow these instructions: - Run VSCode Tasks, by pressing `Ctrl+Shift+P`, selecting `Tasks: Run Task` and running the `setup_python_env` in the drop down menu. When running this task, you will be prompted to add the absolute path to your Isaac Sim installation. If everything executes correctly, it should create a file .python.env in the `.vscode` directory. The file contains the python paths to all the extensions provided by Isaac Sim and Omniverse. This helps in indexing all the python modules for intelligent suggestions while writing code. ### Setup as Omniverse Extension (Optional) We provide an example UI extension that will load upon enabling your extension defined in `exts/ext_template/ext_template/ui_extension_example.py`. For more information on UI extensions, enable and check out the source code of the `omni.isaac.ui_template` extension and refer to the introduction on [Isaac Sim Workflows 1.2.3. GUI](https://docs.omniverse.nvidia.com/isaacsim/latest/introductory_tutorials/tutorial_intro_workflows.html#gui). To enable your extension, follow these steps: 1. **Add the search path of your repository** to the extension manager: - Navigate to the extension manager using `Window` -> `Extensions`. - Click on the **Hamburger Icon** (☰), then go to `Settings`. - In the `Extension Search Paths`, enter the absolute path to `IsaacLabExtensionTemplate/exts` - If not already present, in the `Extension Search Paths`, enter the path that leads to Isaac Lab's extension directory directory (`IsaacLab/source/extensions`) - Click on the **Hamburger Icon** (☰), then click `Refresh`. 2. **Search and enable your extension**: - Find your extension under the `Third Party` category. - Toggle it to enable your extension. ## Docker setup ### Building Isaac Lab Base Image Currently, we don't have the Docker for Isaac Lab publicly available. Hence, you'd need to build the docker image for Isaac Lab locally by following the steps [here](https://isaac-sim.github.io/IsaacLab/source/deployment/index.html). Once you have built the base Isaac Lab image, you can check it exists by doing: ```bash docker images # Output should look something like: # # REPOSITORY TAG IMAGE ID CREATED SIZE # isaac-lab-base latest 28be62af627e 32 minutes ago 18.9GB ``` ### Building Isaac Lab Template Image Following above, you can build the docker container for this project. It is called `isaac-lab-template`. However, you can modify this name inside the [`docker/docker-compose.yaml`](docker/docker-compose.yaml). ```bash cd docker docker compose --env-file .env.base --file docker-compose.yaml build isaac-lab-template ``` You can verify the image is built successfully using the same command as earlier: ```bash docker images # Output should look something like: # # REPOSITORY TAG IMAGE ID CREATED SIZE # isaac-lab-template latest 00b00b647e1b 2 minutes ago 18.9GB # isaac-lab-base latest 892938acb55c About an hour ago 18.9GB ``` ### Running the container After building, the usual next step is to start the containers associated with your services. You can do this with: ```bash docker compose --env-file .env.base --file docker-compose.yaml up ``` This will start the services defined in your `docker-compose.yaml` file, including isaac-lab-template. If you want to run it in detached mode (in the background), use: ```bash docker compose --env-file .env.base --file docker-compose.yaml up -d ``` ### Interacting with a running container If you want to run commands inside the running container, you can use the `exec` command: ```bash docker exec --interactive --tty -e DISPLAY=${DISPLAY} isaac-lab-template /bin/bash ``` ### Shutting down the container When you are done or want to stop the running containers, you can bring down the services: ```bash docker compose --env-file .env.base --file docker-compose.yaml down ``` This stops and removes the containers, but keeps the images. ## Code formatting We have a pre-commit template to automatically format your code. To install pre-commit: ```bash pip install pre-commit ``` Then you can run pre-commit with: ```bash pre-commit run --all-files ``` ## Troubleshooting ### Pylance Missing Indexing of Extensions In some VsCode versions, the indexing of part of the extensions is missing. In this case, add the path to your extension in `.vscode/settings.json` under the key `"python.analysis.extraPaths"`. ```json { "python.analysis.extraPaths": [ "/exts/ext_template" ] } ``` ### Pylance Crash If you encounter a crash in `pylance`, it is probable that too many files are indexed and you run out of memory. A possible solution is to exclude some of omniverse packages that are not used in your project. To do so, modify `.vscode/settings.json` and comment out packages under the key `"python.analysis.extraPaths"` Some examples of packages that can likely be excluded are: ```json "/extscache/omni.anim.*" // Animation packages "/extscache/omni.kit.*" // Kit UI tools "/extscache/omni.graph.*" // Graph UI tools "/extscache/omni.services.*" // Services tools ... ```