# Fortune

**Repository Path**: mirrors_microsoft/Fortune

## Basic Information

- **Project Name**: Fortune
- **Description**: [Preprint'25] Code and data repo of the paper "Fortune: Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models"
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-09-05
- **Last Updated**: 2025-09-13

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Fortune: Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models

<p align="center">
  <a href="https://arxiv.org/abs/2505.23667">
    <img src="https://img.shields.io/badge/arXiv-2505.23667-b31b1b.svg">
  </a>
</p>

<div align="center">
  <img width="80%" src="https://arxiv.org/html/2505.23667v2/x1.png">
</div>

## Introduction 📖

**Fortune** is a framework for symbolic table reasoning using large language models (LLMs). Unlike prior approaches that rely heavily on supervised fine-tuning (SFT) with annotated formulas, Fortune introduces a **reinforcement learning (RL) framework** that learns directly from binary answer supervision. This formula-driven design leads to stronger generalization, higher reasoning accuracy, and more robust table understanding capabilities.

---

Notice: This repository provides only the core Fortune component implementations. It is not a complete standalone training framework. These components must be integrated into the original Verl codebase and placed in the corresponding Verl project directories to function correctly.

## Quick Start ⚡

### Step 1: Set Up Environment 🛠️

```bash
conda create -n fortune python=3.11
conda activate fortune

# Install torch (adjust CUDA version if needed)
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu124

# Flash attention (for efficient training)
pip install flash-attn --no-build-isolation

# vLLM inference engine
pip install vllm==0.8.3

# Excel file support
pip install openpyxl

# External formula executor
pip install formulas

# Additional utilities
pip install nvitop beautifulsoup4

```

### Step 2: Configure Verl ⚙️

Our implementation builds upon the [Verl](https://github.com/volcengine/verl) framework.

**Note: Our codebase cannot be executed directly. It first requires the installation of Verl, along with additions or modifications to the original Verl framework code. Please be aware that different releases of Verl may require different implementation changes.**

```shell
# 1. Clone Verl (our implementation is based on Verl v0.3.0post1, released on 04/04/2025)
git clone https://github.com/volcengine/verl

# 2. Apply our patches to Verl
## 2.1 Manually update the following files as instructed
# e.g., modify the contents of verl/verl/trainer/ppo/metric_utils.py 
# by applying the changes provided in verl_patches/trainer/ppo/metric_utils_u.py
verl_patches/trainer/ppo/metric_utils_u.py
verl_patches/trainer/ppo/ray_trainer_u.py
verl_patches/utils/dataset/rl_dataset_u.py
verl_patches/utils/dataset/sft_dataset_u.py
verl_patches/utils/reward_score/__init__u.py

## 2.2 Add the remaining files from verl_patches/*
cp -r verl_patches/* verl/verl   # overwrite the original files

# 3. Install Verl
cd verl
pip install -e .
```

### Step 3: Prepare Dataset 📂

Follow the instructions in: `data/raw_data/README.md`.

### Step 4: Run Fortune 🚀

Baseline Evaluation

```bash
# Evaluate GPT-based baseline
sh scripts/eval/run_eval_gpt.sh

# Evaluate local LLM baseline
sh scripts/eval/run_eval_local_llm.sh
```

Training

```bash
# Supervised Fine-Tuning (SFT)
sh scripts/train/run_sft_all.sh

# Reinforcement Learning (RL)
sh scripts/train/run_rl_all.sh

# Cold-Start Reinforcement Learning
sh scripts/train/run_rl_all_cold_start.sh
```

## Citation 📑

If you find this project helpful, please consider citing:

```
@article{cao2025fortune,
  title = {Fortune: Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models},
  author = {Cao, Lang and Xu, Jingxian and Liu, Hanbing and Wang, Jinyu and Zhou, Mengyu and Dong, Haoyu and Han, Shi and Zhang, Dongmei},
  journal = {arXiv preprint arXiv:2505.23667},
  year = {2025}
}
```

## Contributing

This project welcomes contributions and suggestions.  Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

## Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.