# server_info

**Repository Path**: qizhi_proj/server_info

## Basic Information

- **Project Name**: server_info
- **Description**: 配置远程服务器的相关信息汇总
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-06-09
- **Last Updated**: 2026-06-10

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Server Info

This repository records the shared CPU LLM research environment on
`172.18.226.106`. Large binary artifacts live under
`/data/share/wanhaochuan`; scripts, manifests, and usage notes live here.

## Server

- SSH: `ssh wanhaochuan@172.18.226.106`
- OS: CentOS Stream 8
- CPU: Intel Xeon Gold 6148, 80 threads
- Memory: about 376 GiB
- GPU: no NVIDIA GPU detected; workflows are CPU-oriented
- Shared artifact root: `/data/share/wanhaochuan`

## Shared Layout

Current shared layout:

```text
/data/share/wanhaochuan/
  envs/
    llm-ops-py311/
      llm-ops-py311-20260609.tar.gz
  models/
    Qwen3-0.6B/
    Qwen3-4B/
    Qwen2.5-Coder-3B/
    Llama-3.2-3B/
    gemma-3-4b-pt/
    Qwen3-Embedding-0.6B/
    Qwen3-Reranker-0.6B/
    routellm-bert/
    ModernBERT-base/
```

Each model has a single directory directly under `models/`; do not add vendor
subdirectories, `latest` symlinks, or date-versioned model directories.

The shared tree is read-only for other users:

```bash
chmod -R u+rwX,go+rX,go-w /data/share/wanhaochuan
```

Use `hf-mirror.com` when downloading Hugging Face models from the server:

```bash
HF_ENDPOINT=https://hf-mirror.com hf download <repo_id> --local-dir /home/wanhaochuan/models/<model-name>
```

## Personal Setup

Users should copy shared assets into their own home directory before running
experiments. Do not run directly from `/data/share/wanhaochuan`.

### Environment

```bash
mkdir -p ~/envs/llm-ops-py311
tar -xzf /data/share/wanhaochuan/envs/llm-ops-py311/llm-ops-py311-20260609.tar.gz \
  -C ~/envs/llm-ops-py311

PATH=~/envs/llm-ops-py311/bin:$PATH ~/envs/llm-ops-py311/bin/conda-unpack
~/envs/llm-ops-py311/bin/python -c "import torch, transformers; print(torch.__version__, transformers.__version__)"
```

The helper script can bootstrap the same environment:

```bash
bash scripts/bootstrap-llm-ops.sh
```

### Models

Copy only the models needed for your experiment:

```bash
mkdir -p ~/models
rsync -a --info=progress2 /data/share/wanhaochuan/models/Qwen3-0.6B/ ~/models/Qwen3-0.6B/
rsync -a --info=progress2 /data/share/wanhaochuan/models/Qwen3-4B/ ~/models/Qwen3-4B/
rsync -a --info=progress2 /data/share/wanhaochuan/models/Qwen2.5-Coder-3B/ ~/models/Qwen2.5-Coder-3B/
rsync -a --info=progress2 /data/share/wanhaochuan/models/Llama-3.2-3B/ ~/models/Llama-3.2-3B/
rsync -a --info=progress2 /data/share/wanhaochuan/models/gemma-3-4b-pt/ ~/models/gemma-3-4b-pt/
rsync -a --info=progress2 /data/share/wanhaochuan/models/Qwen3-Embedding-0.6B/ ~/models/Qwen3-Embedding-0.6B/
rsync -a --info=progress2 /data/share/wanhaochuan/models/Qwen3-Reranker-0.6B/ ~/models/Qwen3-Reranker-0.6B/
rsync -a --info=progress2 /data/share/wanhaochuan/models/routellm-bert/ ~/models/routellm-bert/
rsync -a --info=progress2 /data/share/wanhaochuan/models/ModernBERT-base/ ~/models/ModernBERT-base/
```

## Running Models

`run-causal-lm.sh` runs text-only causal language models:

```bash
bash scripts/run-causal-lm.sh --model 0.6b --prompt "Hello" --max-new-tokens 32
bash scripts/run-causal-lm.sh --model 4b --prompt "Hello" --max-new-tokens 32
bash scripts/run-causal-lm.sh --model qwen2.5-coder-3b --prompt "def add(a, b):" --max-new-tokens 32
bash scripts/run-causal-lm.sh --model llama3.2-3b --prompt "Attention maps" --max-new-tokens 32
```

Supported causal LM aliases:

```text
0.6b
4b
qwen2.5-coder-3b
llama3.2-3b
```

`run-image-text-to-text.sh` runs Gemma 3 through the image-text-to-text model
class. The current helper supports text-only prompts and can be extended for
image inputs when vision operators are studied.

```bash
bash scripts/run-image-text-to-text.sh --prompt "A tensor is" --max-new-tokens 32
```

Override model paths when needed:

```bash
bash scripts/run-causal-lm.sh \
  --model llama3.2-3b \
  --model-path ~/my_models/Llama-3.2-3B \
  --prompt "Explain attention in one sentence."
```

From Windows, use the PowerShell helpers:

```powershell
powershell -ExecutionPolicy Bypass -File .\scripts\run-causal-lm.ps1 -Model llama3.2-3b -Prompt "Attention maps" -MaxNewTokens 32
powershell -ExecutionPolicy Bypass -File .\scripts\run-image-text-to-text.ps1 -Prompt "A tensor is" -MaxNewTokens 32
```

## Encoder And Scoring Models

These models are not chat or text-generation runners:

```text
Qwen3-Embedding-0.6B   text -> normalized 1024-d embedding
Qwen3-Reranker-0.6B    query/document prompt -> CausalLM logits for scoring
routellm-bert          text -> 3-class logits
ModernBERT-base        text -> encoder states; sequence-classification head is untrained unless fine-tuned
```

Quick verification command:

```bash
source /home/wanhaochuan/tools/miniforge3/etc/profile.d/conda.sh
conda activate /home/wanhaochuan/envs/llm-ops-py311

python - <<'PY'
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoModelForCausalLM, AutoModelForSequenceClassification, AutoTokenizer

embedding_dir = "/home/wanhaochuan/models/Qwen3-Embedding-0.6B"
reranker_dir = "/home/wanhaochuan/models/Qwen3-Reranker-0.6B"
routellm_dir = "/home/wanhaochuan/models/routellm-bert"
modernbert_dir = "/home/wanhaochuan/models/ModernBERT-base"

tok = AutoTokenizer.from_pretrained(embedding_dir, trust_remote_code=True)
model = AutoModel.from_pretrained(embedding_dir, dtype=torch.float32, low_cpu_mem_usage=True, trust_remote_code=True).eval()
inputs = tok(["hello world", "operator research"], padding=True, return_tensors="pt")
with torch.no_grad():
    out = model(**inputs)
last_indices = inputs["attention_mask"].sum(dim=1) - 1
emb = out.last_hidden_state[torch.arange(out.last_hidden_state.size(0)), last_indices]
emb = F.normalize(emb, p=2, dim=1)
print("embedding ok", tuple(emb.shape))
del model, tok, inputs, out, emb

tok = AutoTokenizer.from_pretrained(reranker_dir, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(reranker_dir, dtype=torch.float32, low_cpu_mem_usage=True, trust_remote_code=True).eval()
inputs = tok("Query: tensor operators\nDocument: matrix multiplication implements a tensor operator.\nRelevant:", return_tensors="pt")
with torch.no_grad():
    out = model(**inputs)
print("reranker ok", tuple(out.logits.shape))
del model, tok, inputs, out

tok = AutoTokenizer.from_pretrained(routellm_dir)
model = AutoModelForSequenceClassification.from_pretrained(routellm_dir).eval()
inputs = tok("Please solve this routing decision example.", return_tensors="pt", truncation=True)
with torch.no_grad():
    out = model(**inputs)
print("routellm ok", tuple(out.logits.shape))
del model, tok, inputs, out

tok = AutoTokenizer.from_pretrained(modernbert_dir)
model = AutoModelForSequenceClassification.from_pretrained(modernbert_dir, num_labels=2).eval()
inputs = tok("Classify this short sentence.", return_tensors="pt", truncation=True)
with torch.no_grad():
    out = model(**inputs)
print("modernbert classifier ok", tuple(out.logits.shape))
PY
```

## Validation

Current manifest:

```text
manifests/20260610-embeddings-rerankers.sha256
```

Verify all shared artifacts from the server:

```bash
cd /data/share/wanhaochuan
sha256sum -c /path/to/server_info/manifests/20260610-embeddings-rerankers.sha256
```

`manifests/20260609.sha256` and `manifests/20260610.sha256` are kept as
historical manifests for earlier shared layouts.

## Maintenance Rules

1. Keep large binary artifacts under `/data/share/wanhaochuan`.
2. Keep scripts, manifests, and documentation in Git.
3. Put each model directly under `/data/share/wanhaochuan/models/<model-name>/`.
4. Do not use `current`, `latest`, or date symlinks for shared models.
5. Shared artifacts are immutable after publishing; add a new explicit model
   directory if a different artifact is needed.
6. Keep shared files readable by other users and not writable by group/other.