# eloss-final

**Repository Path**: discover304/eloss-final

## Basic Information

- **Project Name**: eloss-final
- **Description**: No description available
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-03-31
- **Last Updated**: 2026-03-31

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# eloss-final

A lightweight multi-modal 3D object detection playground implemented with PyTorch Lightning. The project uses synthetic data to exercise the full pipeline (preprocess, load, train, evaluate, test, inference, logging, plotting, and reporting) without relying on external datasets or OpenMMLab dependencies.

## Features
- Synthetic multi-modal dataset (RGB images + LiDAR-style points) with deterministic splits.
- Simple dual-backbone detector combining CNN image features and PointNet-style point features.
- Training/validation/test/inference entrypoints using the latest Lightning API.
- CSV logging, automated plotting, and markdown reporting utilities.
- Dataset download/extraction helpers for KITTI and nuScenes placeholders.

## Quickstart

1. **Install dependencies**

```bash
pip install -e .
```

2. **Generate synthetic data (optional)**

```bash
uv run -m data_utils.kitti_dataset_preprocess --output data/kitti_synthetic --num-samples 256
```

3. **Train (two-stage pretrain + fine-tune)**

```bash
# Download checkpoints used for encoder warm-start
./scripts/download_checkpoints.sh

# Launch staged pretrain + fine-tune
uv run -m model_utils.train \
  --pretrain-epochs 1 \
  --finetune-epochs 3 \
  --batch-size 8 \
  --lr 3e-4 \
  --image-checkpoint checkpoints/eva02_openclip.bin \
  --pc-checkpoint checkpoints/second_car.pth
```
The trainer automatically enables DDP when multiple GPUs (e.g., dual 3090s) are available, freezes the pretrained encoders during pretraining, then unfreezes for full fine-tuning. WandB logging is enabled by default with `log_model=False` to avoid checkpoint uploads. Checkpoints are saved under `outputs/logs/<run_name>/checkpoints/{pretrain,finetune}/`.

4. **Validate/Test**

```bash
uv run -m model_utils.eval --checkpoint <ckpt_path>
uv run -m model_utils.test --checkpoint <ckpt_path>
```

5. **Inference**

```bash
uv run -m model_utils.inference --checkpoint <ckpt_path>
```

6. **Plot and report**

```bash
uv run -m exp_utils.plot --log-dir outputs/logs
uv run -m exp_utils.report --log-dir outputs/logs
```

Scripts in `scripts/` pin the same arguments and invoke everything through `uv run` for reproducibility.

## Research framing and expansion plan

1. We model the neural network as an information system.
2. The same neural network layer has similar compressing power.
3. The compression rate of consecutive repeating neural network layers in a block follows an increase-then-decrease pattern, which is ceilinged by information bottleneck theory and theoretically can be calculated in numbers.
4. Information quantity can be measured as the log of entropy, where entropy of a distribution of features can be calculated algorithmically.
5. We can monitor the information flow in a network and perform test-time adaptation to tackle out-of-distribution and inference robustness challenges.

Next steps to expand the work:

1. Run a baseline task.
2. Apply our idea to the baseline task.
3. Make the baseline better.
4. Repeat with different baseline tasks and different models to prove generalizability.