# mcclib

**Repository Path**: the_machine/mcclib

## Basic Information

- **Project Name**: mcclib
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2019-11-15
- **Last Updated**: 2021-04-08

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Mcclib: 1.8.1

### Table of content
**[Installation](#installation)**<br>
**[T24 related](#T24-related)**</br>
**[Logger](#logger)**</br>
**[Database](#database)**</br>
**[DingTalk Robot](#dingtalk-robot)**</br>
**[Utility](#aws)**</br>
**[AWS](#aws)**</br>
**[Email](#email)**</br>
**[Autobuild](#autobuild)**</br>

## Installation

### From git

```python
pip insatll git+https://github.com/MicroCred/mcclib.git
```

or

```python
git clone https://github.com/MicroCred/mcclib.git
cd mcclib/mcclib
python setup install
# or
pip install .
```

### From AWS ECR

This should be helpful when building other docker images that requires mcclib.

Build target image with multi-stage build:

Example:

```docker
FROM ubuntu

# copy codes from ecr images and install package
COPY --from=078579596025.dkr.ecr.cn-north-1.amazonaws.com.cn/data-analytics/mcclib:latest /mcclib /mcclib
RUN cd /mcclib && pip install .

# do your stuff
...
```

## Usage

### T24 related

1. workday

    To get latest workday/holiday from T24.

    ```python
    from mcclib.t24 import workday
    from mcclib.utils import config_parser

    url = 'T24_SERVICE_URL'
    port = 'T24_UT_PORT'
    configs = config_parser.get_env_vars([url, port])
    t24cal = workday.get_latest_t24cal(
        service_url=configs[url],
        ut_port=configs[port],
    raise_error=True)
    ```

### Logger

```python
from mcclib.logger import custom_logger

logger = custom_logger.CustomLogger(name='test')
logger.debug('debug')
logger.info('info')
logger.warn('warn')
logger.error('error')


from mcclib.logger import logutils
import logging

# create logger
logger = logutils.get_logger('test-v2', reset=True)
logger.setLevel(logging.DEBUG)

# create handler
ch = logutils.get_console_handler(level=logging.INFO)
fh = logutils.get_file_handler(file_name='test.log', level=logging.DEBUG, kind='timerotating', reset=True)
logger.addHandler(ch)
logger.addHandler(fh)

# log
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')

# MultiLogger
logger_config = [
    {'name': 'console', 'kind': 'default', 'logger_level': 'debug'},
    {'name': 'text', 'kind': 'text-file', 'logger_level': 'debug'}
]
logger = MutiLogger(logger_list=logger_config)
logger.info('test info')
logger.debug('test debug')
logger.to_string()
```

### Database

1. database connection

    ```python
    from mcclib.database.database_connection import DatabaseConnection

    redshift = {
        'host': '127.0.0.1',
        'port': 5439,
        'dbname': 'test',
        'user': 'admin',
        'password': '123'
    }
    db = DatabaseConnection(db_config=redshift, database='redshift')
    df = db.read("select * from table")
    db.write("drop table tmp1 if exists")
    ```

### DingTalk Robot

Send message to DingTalk Robot. See [official document](https://ding-doc.dingtalk.com/doc#/serverapi2/qf2nxq) for detail.

Set the following variables for default configurations

```shell
DINGTALK_URL=https://oapi.dingtalk.com/robot/send?access_token=<TOKEN>
DINGTALK_KW=keyword
```

```python
from mcclib.dingtalk_robot import dtrobot

dtr = dtrobot.DingTalkRobot()
# or
dtr = dtrobot.DingTalkRobot(url=<URL>, keyword=<KEYWORD>)
dtr.send_text('Msg sent to DingTalk')
```

### AWS

1. s3client

    ```python
    from mcclib.s3 import s3client, utils

    client = s3client.get_client()
    client.upload(bucket='test_bucket',
                  key='test_folder/test_file',
                  to_upload='local_file.txt')

    client.download(bucket='test_bucket',
                    key='test_folder/test_file',
                    save_as='local_saved.txt')

    client.delete(bucket='test_bucket',
                  key='test_folder/test_file')

    client.list_s3_objects(bucket='test_bucket',
                           prefix='test_folder/test_file')

    # copy s3 file to redshift
    utils.s3_to_redshift(
        ignore_header=1,
        sep=',',
        table='etl_test.s3_to_redshift',
        bucket='bucket',
        db_config=db_config,
        key='test/test.txt',
        region='cn-northwest-1',
        aws_access_key_id='abc123',
        aws_secret_access_key='abc123',
    )
    ```

### Utility

1. config_parser

    ```python
    from mcclib.utils import config_parser

    url = 'T24_SERVICE_URL'
    port = 'T24_UT_PORT'
    configs = config_parser.get_env_vars([url, port])
    ```

### Email
```python
## read email
from mcclib.email import read_email

email_config = {
    'host': 'imap.mail.com',
    'port': 993,
    'username': 'youremail@mail.com',
    'password': 'yourpassword'
}

eid = read_email.list_email_id(
    email_config=email_config,
    tag='Unseen'
)

mail = read_email.fetch_mail(
    email_config=email_config,
    mail_id='508'
)

mail.set_seen(
    email_config=email_config,
    mail_id='508'
)

## Email object
mail = read_email.fetch_email_object(mconfig=email_config, mid='508')
# set flag
mail.set_unseen()        
# save attachment
att_fp = mail.save_attachment(dir='./', name='测试.txt', rename='test.txt')[0]

## Send email (# SSL connection only)

from mcclib.email import send_email

stmp_config = {
    'host': '123.com.cn',
    'port': 465, 
    'username': '123@123.com',
    'password': '123'
}
email_content = {
    'from': 'zhangxian@microcredchina.com',
    'to': ['zhangxian@microcredchina.com', 'joker0414zx@163.com'],
    'cc': None,
    'subject': '测试发送邮件',
    'text-content': '这是测试邮件',
    'attachment': ['test.txt', 'test_att.xlx']
}
send_email.send_email(email_content=email_content, stmp_config=stmp_config)

```

### Autobuild

Automatically build docker image and push to ecr repo.</br>

```bash
autobuild -f <Dockerfile> -r <REPO_TO_BUILD>
```

Requirements: `Dockerfile` and `autobuild.ini` must be included in the repository to build</br>

- `Dockerfile`
    ```docker
    # version: 1.0.0

    # Do your stuff
    FROM ...
    COPY ...
    ```
- `autobuild.ini`
    ```ini
    [autobuild]
    ecr_repo = <ECR_REPO>
    ```

###

### Test

```bash
# build image
docker build -t mcclib-test -f Dockerfile-test

# run test
docker run --rm --network='host' --env test_name='t24_workday' --env test_args='{"url": "http://127.0.0.1", "port": "8011"}' mcclib-test
```