# pyds

**Repository Path**: boxigg/pyds

## Basic Information

- **Project Name**: pyds
- **Description**: python data service
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-12-16
- **Last Updated**: 2026-01-12

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# PyDS - Python Data Service for Quantitative Trading

A Python data service library designed for quantitative trading applications, providing efficient Tick and Bar data processing with TDengine integration.

## Features

- **Tick Data Processing**: High-performance tick data handling and storage
- **Bar Data Management**: K-line data aggregation and storage
- **TDengine Integration**: Native support for TDengine time-series database
- **Redis Integration**: Real-time data streaming via Redis
- **Type Safety**: Full type hinting support
- **Easy Configuration**: Simple YAML-based configuration

## Installation

### Basic Installation
```bash
pip install pyds
```

### Development Installation
```bash
pip install pyds[dev]
```

### With All Optional Dependencies
```bash
pip install pyds[dev,test,docs]
```

## Quick Start

### Basic Usage

```python
from pyds import TaosDatabase, TickData, BarData
from datetime import datetime

# Initialize database connection
db = TaosDatabase()

# Create tick data
tick = TickData(
    symbol="cu2401",
    exchange="SHFE",
    datetime=datetime.now(),
    last_price=75000.0,
    volume=100,
    # ... other fields
)

# Save tick data
db.save_tick(tick)

# Query tick data
ticks = db.load_tick("cu2401", "SHFE", 
                   start_time=datetime(2024, 12, 14, 9, 0),
                   end_time=datetime(2024, 12, 14, 15, 0))
```

### Redis Integration

```python
from pyds import TickEngine
from pyds.constant import EVENT_TICK

# Create tick engine
engine = TickEngine()

# Subscribe to tick data
engine.subscribe(EVENT_TICK, "cu2401.SHFE", on_tick_received)

def on_tick_received(tick):
    print(f"Received tick: {tick.symbol} {tick.last_price}")

# Start the engine
engine.start()
```

## Configuration

Create a `settings.yaml` file:

```yaml
database:
  host: localhost
  port: 6030
  user: root
  password: taosdata
  database: pyds_db
  timezone: Asia/Shanghai

redis:
  host: localhost
  port: 6379
  password: ""
  db: 0
```

## Project Structure

```
pyds/
├── __init__.py          # Package initialization
├── constant.py          # Constants and enums
├── db.py              # Database base classes
├── engine.py          # Event engine
├── object.py          # Data structures (TickData, BarData, etc.)
├── setting.py         # Configuration management
├── taosdb.py         # TDengine database implementation
├── taos_script.py    # SQL scripts
├── tq_tick.py        # Tick data processing
├── utility.py        # Utility functions
└── locale.py         # Internationalization
```

## API Reference

### Core Classes

#### `TaosDatabase`
Main database interface for TDengine operations.

- `save_tick(tick)`: Save tick data
- `save_bar(bar)`: Save bar data
- `load_tick(symbol, exchange, start_time, end_time)`: Query tick data
- `load_bar(symbol, exchange, interval, start_time, end_time)`: Query bar data

#### `TickData`
Tick data structure with the following fields:
- `symbol`: Trading symbol
- `exchange`: Exchange name
- `datetime`: Tick timestamp
- `last_price`: Last price
- `volume`: Trading volume
- `...`: Other tick-related fields

#### `BarData`
Bar (K-line) data structure with OHLCV fields:
- `symbol`: Trading symbol
- `exchange`: Exchange name
- `interval`: Time interval (1m, 5m, 1h, etc.)
- `datetime`: Bar timestamp
- `open_price`: Opening price
- `high_price`: Highest price
- `low_price`: Lowest price
- `close_price`: Closing price
- `volume`: Trading volume

### Utilities

#### `convert_tz(dt)`: Convert timezone of datetime object to database timezone.

## Development

### Setup Development Environment

```bash
# Clone the repository
git clone https://github.com/quant/pyds.git
cd pyds

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install in development mode
pip install -e .[dev]
```

### Running Tests

```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=pyds --cov-report=html

# Run specific test
pytest tests/test_taosdb.py::test_save_tick
```

### Code Quality

```bash
# Format code
black pyds tests
isort pyds tests

# Lint code
flake8 pyds tests

# Type checking
mypy pyds
```

## Dependencies

### Core Dependencies
- **redis**: Redis client for data streaming
- **taos**: TDengine Python client
- **pandas**: Data analysis and manipulation
- **typing-extensions**: Enhanced type support
- **numpy**: Numerical computing (performance)
- **pyyaml**: Configuration file parsing

### Development Dependencies
- **pytest**: Testing framework
- **black**: Code formatter
- **isort**: Import sorter
- **flake8**: Linter
- **mypy**: Type checker
- **sphinx**: Documentation generator

## Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Add tests for new functionality
5. Run the test suite and ensure all tests pass
6. Commit your changes (`git commit -m 'Add some amazing feature'`)
7. Push to the branch (`git push origin feature/amazing-feature`)
8. Open a Pull Request

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Changelog

### v0.0.1
- Initial release
- Basic tick and bar data support
- TDengine integration
- Redis streaming support

## Support

- **Documentation**: [https://pyds.readthedocs.io/](https://pyds.readthedocs.io/)
- **Issues**: [GitHub Issues](https://github.com/quant/pyds/issues)
- **Discussions**: [GitHub Discussions](https://github.com/quant/pyds/discussions)

## Performance Considerations

- Use batch insert operations for better performance
- Configure appropriate TDengine buffer size
- Consider data compression for historical data
- Use Redis pipelining for high-frequency data

## Security

- Never commit sensitive credentials to version control
- Use environment variables for production configuration
- Implement proper access controls for database connections
- Regular security updates for dependencies