# cicd

**Repository Path**: jungle/cicd

## Basic Information

- **Project Name**: cicd
- **Description**: No description available
- **Primary Language**: Python
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-04-21
- **Last Updated**: 2026-06-07

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Simple CI/CD Service (Built with FastAPI)

## Core Features

- **Automated Git Repository Build & Deploy**: Submit a Git repository URL and the service automatically clones, builds, and deploys it (supports Dockerfile or custom build commands/ci_build.sh)
- **Single Sign-On (SSO)**: Integrates with a centralized authentication service using Cookie-based authentication
- **User Permission Control**:
  - **Regular Users**: Subject to job creation quota limit (default 2), can only operate on tasks they created
  - **Administrator Users**: Can operate on all tasks, no quota limit
- **User Management System**: Supports user profiles, avatar display, and quota management
- **Real-time Task Status**: Frontend automatically polls task status every 5 seconds without manual page refresh
- **Real-time Logs**: Incremental log streaming via WebSocket
- **Task History**: Records task execution history, runtime duration, and other metadata

## Recent Updates (April 2026)

- ? **User Profile Management**: UserProfile table with avatar URL, role, quota, registration time support
- ? **Task Creator Tracking**: Each task records created_by field, displaying creator name and avatar
- ? **Job Quota System**: Regular users have default quota limit (2 jobs), exceeding returns "insufficient quota" error
- ? **Fine-grained Permissions**: owner_or_admin authorization model - task creator or admin can edit/execute/delete tasks
- ? **Auto Status Refresh**: Task list status automatically refreshes every 5 seconds without manual page reload
- ? **English UI Localization**: User list, buttons, tooltips all switched to English
- ? **Service Footer Info**: Displays application version, uptime duration, and copyright information
- ? **Avatar Display**: Shows creator avatar in task list, supports parsing from multiple auth center fields

## Quick Start

### 1. Install Dependencies

\\\ash
python -m pip install -r requirements.txt
\\\

### 2. Start Service (Development Mode)


### Artifacts storage layout and permissions

- **Storage path**: artifacts are saved under the path configured by the `CICD_ARTIFACTS_ROOT` environment variable. Files uploaded by agents are placed under:

  - `CICD_ARTIFACTS_ROOT/host_{hostname}/task_{task_id}/` — each artifact is stored inside its task folder.

- **Stored path in DB**: The application stores artifact paths relative to `CICD_ARTIFACTS_ROOT` (for portability). API responses may return absolute paths when helpful, but service-internal references use the relative path.

- **Permissions**: For production Linux deployments, ensure the service user running the API (for example `cicd` or the container user) owns the artifacts directory and has read/write permissions. Example:

```bash
# create and set ownership (replace USER and GROUP as needed)
mkdir -p /data/cicd/artifacts
chown -R cicd:cicd /data/cicd/artifacts
chmod -R 750 /data/cicd/artifacts
```

- **Retention / cleanup**: Large deployments should run periodic cleanup to reclaim disk space. A helper script is included at `scripts/cleanup_artifacts.py` — it deletes files older than N days and can be run as a cron job or scheduled task. Example (daily cron):

```bash
# run daily at 03:30, delete artifacts older than 30 days
30 3 * * * /usr/bin/python3 /path/to/cicd/scripts/cleanup_artifacts.py --root /data/cicd/artifacts --days 30
```

On Windows, schedule `python scripts/cleanup_artifacts.py --root D:\\ci_workspace\\artifacts --days 30` in Task Scheduler.

### Resumable upload (agent -> server)

Agents can upload large artifacts using a simple resumable flow when network is unreliable. Endpoints:

- `POST /hosts/artifacts/initiate` — body: `{"task_id": <int>, "filename": "out.tar.gz"}` → returns `{upload_id}`.
- `PUT /hosts/artifacts/chunk?upload_id=<id>` — send form `file` chunks (append-only). Each call appends data to the temporary upload file.
- `POST /hosts/artifacts/complete` — body: `{"upload_id": "..."}` → finalizes upload, moves file to `host_{hostname}/task_{task_id}/` and records artifact relative path in DB.

Temporary uploads are stored under `CICD_ARTIFACTS_ROOT/tmp_uploads/<upload_id>/` until completed. If an upload is interrupted, the agent may resume by re-using the same `upload_id` and sending subsequent chunks.

Example agent sequence (pseudo-code):

```python
# 1. initiate
# POST /hosts/artifacts/initiate {task_id, filename} -> {upload_id}
# 2. for each chunk: PUT /hosts/artifacts/chunk?upload_id=... with form file chunk
# 3. POST /hosts/artifacts/complete {upload_id}
```

Refer to `app/routers/hosts.py` for implementation details.

### Running cleanup regularly

Choose one of the following approaches:

- Systemd (one-shot run): copy `scripts/cleanup_artifacts.service` to `/etc/systemd/system/`, edit `ExecStart` and `User`, then enable/run via `systemctl enable --now cleanup_artifacts.service`.
- Container (docker-compose): use `docs/cleanup-docker-compose.yml` as a template and schedule a job (e.g., via host cron or external scheduler) to run the container when needed. The example runs the script in `--dry-run` mode by default — remove `--dry-run` to actually delete files.


It's recommended to use \python -m uvicorn\ in a virtual environment to avoid issues with uvicorn executable recognition on PowerShell/Windows:

\\\ash
# Install dependencies (if not already installed)
python -m pip install -r requirements.txt
# Or install uvicorn only
python -m pip install "uvicorn[standard]"

# Start service (using python -m ensures module availability)
python -m uvicorn app.main:app --reload --reload-exclude "repos/*" --reload-exclude "artifacts/*" --host 0.0.0.0 --port 8000
\\\

**Note (PowerShell)**: To prevent shell wildcard expansion, wrap \--reload-exclude\ paths in quotes as shown above.

### 3. Production Deployment with Redis + RQ

\\\ash
docker-compose up --build
\\\

This starts \
edis\, \web\ (FastAPI), and \worker\ (RQ worker) services.

## Environment Configuration

The service loads \.env\ from the project root on startup. Restart the API and worker after modifying \.env\.

### Common Environment Variables

\\\dotenv
CICD_REPOS_ROOT=D:\\ci_workspace\\repos
CICD_ARTIFACTS_ROOT=D:\\ci_workspace\\artifacts

# Auth center URL for server-side /api/auth/session calls
AUTH_CENTER_URL=http://127.0.0.1

# Login page URL for unauthenticated redirects
AUTH_LOGIN_URL=http://127.0.0.1/enter
\\\

## Authentication & Authorization

- When accessing the CICD page without login, the browser is redirected to \AUTH_LOGIN_URL?next=current_url\
- The CICD backend forwards \cookieOnlineJudgeToken\ and \cookieOnlineJudgeUsername\ to the auth center's \/api/auth/session\ endpoint for validation
- Auth center returning \dmin=true\ indicates admin privileges; in current implementation, either \HEAD\ or \ADMIN\ permission grants admin status
- **Non-admin users**: Read-only access to task lists, details, logs, and artifacts
- **Admin users**: Can create, edit, re-execute, cancel, delete, restore, and clean up tasks

**本地开发：禁用 SSO（仅用于测试）**

- **说明**：为便于本地开发或自动化测试，可通过环境变量临时绕过外部认证中心，不会调用 `/api/auth/session`。切勿在生产环境启用该模式。
- **环境变量**：在项目根目录的 `.env` 中添加（或在运行环境中设置）：

```dotenv
AUTH_DISABLED=1
AUTH_BYPASS_USERNAME=dev
AUTH_BYPASS_ADMIN=1
```

- **行为**：当 `AUTH_DISABLED` 为真时，应用会向每个请求注入一个伪造用户（由 `AUTH_BYPASS_USERNAME` 与 `AUTH_BYPASS_ADMIN` 控制），中间件与直接调用的认证函数都会返回该用户信息。
- **注意**：修改 `.env` 后需要重启 API 和 worker 进程以使配置生效。

## Usage Examples

### Submit a Build Job

\\\ash
curl -X POST "http://localhost:8000/jobs" \\
  -H "Content-Type: application/json" \\
  -d '{"git_url":"https://github.com/your/repo.git"}'

Notes:
- You can optionally set `branch` to define which branch will be checked out when the job runs (default: `master`).
- To restrict which branches can re-queue a job via webhook, set `branch_filter` when creating/editing the job to a comma-separated list of patterns (e.g. `main,release/*`). An empty `branch_filter` means all branches are accepted.
\\\

### Query Job Status

\\\ash
curl http://localhost:8000/jobs/1
\\\

### Notes

- If the repository contains a \Dockerfile\, the service attempts to build and run the Docker image (requires Docker on the host)
- Custom build commands can be specified via the \uild_command\ field (e.g., \./build.sh\, \
pm ci && npm run build\)
- If auth center is enabled, write operations require an authenticated browser session with admin privileges or appropriate Cookie headers

### WebSocket Log Subscription

\\\
ws://127.0.0.1:8000/jobs/{id}/ws
\\\

### Cancel a Running Job

\\\ash
curl -X POST http://localhost:8000/jobs/1/cancel
\\\

## Production Deployment Guide

### 1. Auth Center & Cookie Domain

If CICD and auth center are deployed in different subdomains under the same parent domain, set Cookie Domain to the parent domain itself (e.g., \happyoj.com\), not using the leading-dot notation (\.happyoj.com\).

### 2. CICD SSO Configuration

Configure CICD to integrate with the auth center via \.env\:

\\\dotenv
AUTH_CENTER_URL=https://happyoj.com
AUTH_LOGIN_URL=https://happyoj.com/enter
\\\

### 3. Reverse Proxy & HTTPS

- Production deployments should run FastAPI and auth center behind Nginx / Caddy / Apache
- All browser traffic should use HTTPS consistently
- The reverse proxy must pass through the real Host header

### 4. Redis & Worker

## Multi-Agent Roadmap (Design + Implementation Plan)

目标：构建一个可扩展的多 Agent 调度与执行体系，支持标签路由、并发限制、失败重试、artifact 上传与断点续传、以及前端可视化和监控告警。

分阶段计划（按优先级执行）

1. 基础数据模型与注册契约（已部分实现）
  - `Host` 拥有：`hostname`, `name`, `owner`, `info`（JSON string）, `capacity`, `current_running`, `api_key`, `status`, `last_heartbeat`。
  - Agent 在注册时发送结构化 `info`，包含 `labels`/`capabilities`（数组）和可选 `capacity`（并发执行槽位）。

2. 任务实体与路由（进行中）
  - `AgentTask` 支持：`hostname`（可选目标），`required_labels`（可选，逗号或数组），`priority`，`retries`，`status`（pending→running→done/failed）以及关联 `job_id`/`run_id`。
  - 调度器：优先分配给指定 `hostname`，否则从 pending 任务池中按创建时间、priority 与标签匹配 Agent `info.labels` 选择任务。

3. 并发控制与容量（下一步）
  - 在分配任务前检查 `Host.current_running < Host.capacity`；分配后增加 `current_running`；任务完成或失败时减少 `current_running`。
  - 支持 `capacity=0/1/多`，默认 `capacity=1`。

4. 重试策略与超时
  - 为任务添加 `max_retries`、`attempts`、`timeout_seconds`；任务失败时按指数退避重试并记录日志。

5. 前端与可视化
  - Agents 面板显示 `name`、`owner`、`status`、`capacity/current_running`、`labels`，支持按 owner/状态/label 过滤。
  - 任务队列视图显示 pending/running 列表和路由匹配信息。

6. E2E 覆盖与 CI
  - 编写 E2E: 多 Agent 同时拉取任务、标签匹配、超过 capacity 的排队与恢复、artifact 上传与回退、失败重试。
  - 在 CI 中加入这些测试并对 PR 触发运行。

7. 运营与监控
  - 心跳丢失报警（Prometheus/Alertmanager 或简单告警邮件）、任务执行耗时、失败率、吞吐量指标与日志审计。

时间线（建议）
- 周 1: 完成数据模型、Agent 注册契约、基本路由（priority/labels）并完成单节点 E2E。  
- 周 2: 实现并发 capacity 管理、重试策略与超时，补充单元测试。  
- 周 3: 前端展示与过滤、E2E 覆盖、CI 集成与文档完善。

如果你同意，我将立即开始第3步（并发控制与容量），并在完成后跑回归 E2E 测试。否则请指定优先级或调整时间线。

- Development: Optional Redis, service falls back to local thread execution
- Production: Enable Redis + dedicated worker

### 5. Directories & Persistence

\
epos\ and \rtifacts\ should be placed outside the project directory and mounted to persistent storage:

\\\dotenv
CICD_REPOS_ROOT=/data/cicd/repos
CICD_ARTIFACTS_ROOT=/data/cicd/artifacts
\\\

### 6. Permission Model

- **Authenticated Regular Users**: Read-only
- **Users with \dmin=true\**: Full write access
- All write endpoints validated server-side

### 7. Troubleshooting

- **\.env\ changes not taking effect**: Check if API/worker have been restarted
- **Cookie domain errors**: Verify auth center is not using \.happyoj.com\
- **Stuck on login page**: Verify auth center's \/api/auth/session\ endpoint is accessible

### 8. WebSocket Handshake (Important for Live Logs)

If browser console shows `WebSocket connection ... failed` and a direct check to `/jobs/{id}/ws` returns `HTTP 404`, the reverse proxy is likely forwarding the request as plain HTTP instead of WebSocket upgrade.

Use Nginx HTTP/1.1 upgrade forwarding:

```nginx
map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;
}

server {
  listen 443 ssl;
  server_name cicd.happyoj.com;

  location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_read_timeout 3600;
  }
}
```

After updating Nginx, reload it and verify handshake:

```bash
curl -i -N \

  -H "Connection: Upgrade" \
  -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" \
  -H "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
  https://cicd.happyoj.com/jobs/6/ws
```

Expected: status `101 Switching Protocols`.