# FISHER

**Repository Path**: wuqiang1987/FISHER

## Basic Information

- **Project Name**: FISHER
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-08-20
- **Last Updated**: 2026-08-20

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<h1 align="center">
  <img src="assets/icon.jpg" alt="icon" style="width:6%; max-width: 100px">
  <span style="margin-left: 10px">FISHER</span>
</h1>

<div align="center">
  <a href="https://www.python.org/">
    <img src="https://img.shields.io/badge/Python-3.10%2B-orange?logo=python" alt="Python">
  </a>
  <a href="https://pytorch.org/">
    <img src="https://img.shields.io/badge/PyTorch-2.1%2B-brightgreen?logo=pytorch" alt="PyTorch">
  </a>
  <a href="https://arxiv.org/abs/2507.16696">
    <img src="https://img.shields.io/badge/arXiv-2507.16696-blueviolet?logo=arxiv" alt="arXiv">
  </a>
  <a href="https://huggingface.co/collections/jiangab/fisher">
    <img src="https://img.shields.io/badge/HuggingFace-FISHER-yellow?logo=huggingface" alt="huggingface">
  </a>
</div>

<br>

<div align="center">
    <img src="assets/rmis_curve.png" alt="Model Performances on the RMIS Benchmark" style="width:85%; max-width: 2000px">
</div>

## 🔥🔥🔥 Updates

- [2026.6.23] FISHER has been accepted by IEEE TII. We have largely enlarged the experiment part. Please read the [final version](https://ieeexplore.ieee.org/abstract/document/11563541) for recent updates.

- [2026.2.13] We are excited to release the [RMIS](https://github.com/jianganbai/RMIS) codebase.

- [2025.7.25] FISHER is now integrated on [HuggingFace🤗](https://huggingface.co/collections/jiangab/fisher).

- [2025.7.23] We release the inference code and checkpoints for tiny, mini and small.

## Introduction

<div align="center">
    <img src="assets/model_pipe.png" alt="Model Performances on the RMIS Benchmark" style="width:80%; max-width: 1000px">
</div>

FISHER is a **F**oundation model for **I**ndustrial **S**ignal compre**HE**nsive **R**epresentation, which models heterogeneous industrial signals (sound, vibration, voltage, etc.) in a unified manner. FISHER accepts arbitrary sampling rates and models the increment of sampling rate as the concatenation of sub-band information, which first splits a STFT spectrogram into sub-bands before processsing it by the ViT encoder. FISHER is trained by teacher student EMA self-distillation.

To evaluate the model, we have developed the [RMIS benchmark](https://jianganbai.github.io/RMIS), where FISHER achieves the SOTA performances with much more efficient scaling properties.

## Checkpoints

We release the checkpoints of FISHER-tiny, FISHER-mini and FISHER-small.

| Version| ☁️ Tsinghua Cloud | 🤗 HuggingFace | wisemodel
|------------| :------------: | :--------: | :--------: |
| FISHER-tiny | [Link](https://cloud.tsinghua.edu.cn/f/630a4b1b2962481a9150/?dl=1) | [Link](https://huggingface.co/jiangab/FISHER-tiny-0723) | [Link](https://wisemodel.cn/models/jiangab/FISHER-tiny-0723)
| FISHER-mini | [Link](https://cloud.tsinghua.edu.cn/f/60b3bfc0977f45f48dff/?dl=1) | [Link](https://huggingface.co/jiangab/FISHER-mini-0723) | [Link](https://wisemodel.cn/models/jiangab/FISHER-mini-0723)
| FISHER-small | [Link](https://cloud.tsinghua.edu.cn/f/f997a6932b614046915e/?dl=1) | [Link](https://huggingface.co/jiangab/FISHER-small-0723) | [Link](https://wisemodel.cn/models/jiangab/FISHER-small-0723)

## Inference

Please use the following code to infer the signal representation by FISHER.

```python
import torch
import torchaudio
import torch.nn.functional as F
from models.fisher import FISHER

wav, sr = torchaudio.load('/path/to/local/signal.wav')
# You can replace it with your custom loading function for other signals

wav = wav - wav.mean()
STFT = torchaudio.transforms.Spectrogram(
    n_fft=25 * sr // 1000,
    win_length=None,
    hop_length=10 * sr // 1000,
    power=1,
    center=False
)
spec = torch.log(torch.abs(STFT(wav)) + 1e-10)
spec = spec.transpose(-2, -1)  # [1, time, freq]
spec = (spec + 3.017344307886898) / (2.1531635155379805 * 2)

model_path = '/path/to/local/fisher/model.pt'  # Please download the checkpoint in advance.
model = FISHER.from_pretrained(model_path)
model = model.cuda()
model.eval()

# time-wise cutoff
if spec.shape[-2] > 1024:
    spec = spec[:, :1024]
# freq-wise padding
if spec.shape[-1] < model.cfg.band_width:
    spec = F.pad(spec, (0, model.cfg.band_width - spec.shape[-1]))
spec = spec.unsqueeze(1).cuda()

with torch.no_grad():
    # Use autocast for mixed precision inference. You can disable it for full precision.
    with torch.autocast('cuda'):
        repre = model.extract_features(spec)
print(repre.shape)
```

## Acknowledgements

FISHER is developed based on [EAT](https://github.com/cwx-worst-one/EAT) and [fairseq](https://github.com/facebookresearch/fairseq). We thank these authors for open-sourcing their works.

## Citation

If you find FISHER useful, please cite the following paper.

```bibtex
@ARTICLE{11563541,
  author={Fan, Pingyi and Jiang, Anbai and Zhang, Shuwei and Zheng, Xinhu and Lv, Zhiqiang and Han, Bing and Liang, Wenrui and Li, Junjie and Zhang, Wei-Qiang and Qian, Yanmin and Chen, Xie and Liu, Jia},
  journal={IEEE Transactions on Industrial Informatics}, 
  title={FISHER: A Foundation Model for Multimodal Industrial Signal Comprehensive Representation}, 
  year={2026},
  volume={},
  number={},
  pages={1-12},
  keywords={Modeling;Educational institutions;Fault diagnosis;Training;Foundation models;Timing;Speech;Machining;Signal detection;Cloning;Anomaly detection;fault diagnosis;foundation model;multimodal},
  doi={10.1109/TII.2026.3698554}}
```