# TS-TCC-Plus

**Repository Path**: askuasen/ts-tcc-plus

## Basic Information

- **Project Name**: TS-TCC-Plus
- **Description**: A modified version based on Time-Series Representation Learning via Temporal and Contextual Contrasting (TS-TCC)
- **Primary Language**: Python
- **License**: BSD-4-Clause
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2024-11-01
- **Last Updated**: 2025-01-14

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# TS-TCC-Plus
A modified version based on Time-Series Representation Learning via Temporal and Contextual Contrasting (TS-TCC)[[Paper](https://www.ijcai.org/proceedings/2021/0324.pdf)], which implements the lithofacies calssification by using well logs<sup><a herf="#ref2">2</a></sup><sup>,</sup><sup><a herf="#ref3">3</a></sup>..

This work is based on Emadeldeen Eldele's code (TS-TCC) [[code](https://github.com/emadeldeen24/TS-TCC)], and we also take the loss function strategy that named
total coding rate(TCR), which is built by Yi Ma's team[[Paper](https://arxiv.org/abs/2306.01129)],


## Abstract
 
![workflow_chart](figure/TS-TCC-en_us.png "arch")  
**Figure 1 TS-TCC arch**    
![workflow_chart](figure/TS-TCC-Plus-en.png "arch")  
**Figure 1 TS-TCC-Plus arch**  


## Requirmenets:
- Python3.10.8
- Pytorch==2.5.0+cu11.8
- Numpy==1.26.4
- scikit-learn>=1.5.2
- Pandas==1.5.3
- openpyxl (for classification reports)
- mne=='0.20.7' (For Sleep-EDF preprocessing)
- mat4py (for Fault diagnosis preprocessing)
## Datasets
### Download datasets
We used four public datasets in this study:
- [LithoFacies] We save the dataset in data folder.
- [HAR](https://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones)  


### Preparing datasets
The data should be in a separate folder called "data" inside the project folder.
Inside that folder, you should have a separate folders; one for each dataset. Each subfolder should have "train.pt", "val.pt" and "test.pt" files.
The structure of data files should in dictionary form as follows:
`train.pt = {"samples": data, "labels: labels}`, and similarly `val.pt`, and `test.pt`

The details of preprocessing is as follows:
#### 1- Lithofacies dataset:
Create a folder named `data_files` in the path `data_preprocessing/faciesdata/`.
Download the dataset files and place them in this folder. 

Run the script `preprocess_facies2.py` to generate the numpy files, and it will automatically place
them in the `data/FACIES` folder.

#### 2- UCI HAR dataset
When you dowload the dataset and extract the zip file, you will find the data in a folder named
`UCI HAR Dataset` ... place it in `data_preprocessing/uci_har/` folder and run `preprocess_har.py` file.



### Configurations
The configuration files in the `config_files` folder should have the same name as the dataset folder name.
For example, for HAR dataset, the data folder name is `HAR` and the configuration file is `HAR_Configs.py`.
From these files, you can update the training parameters.

## Training TS-TCC-Plus 
You can select one of several training modes:
 - Random Initialization (random_init)
 - Supervised training (supervised)
 - Self-supervised training (self_supervised)
 - Fine-tuning the self-supervised model (fine_tune)
 - Training a linear classifier (train_linear)

The code allows also setting a name for the experiment, and a name of separate runs in each experiment.
It also allows the choice of a random seed value.

To use these options:
```
python main.py --experiment_description exp1 --run_description run_1 --arch TS-TCC-Plus --encoder_backbone BaseCNN --seed 123  --training_mode random_init --selected_dataset FACIES
```


modify main.py to obtain the pretrain model and obtain the features   
Note that the name of the dataset should be the same name as inside the "data" folder, and the training modes should be
the same as the ones above.

To train the model for the `fine_tune` and `train_linear` modes, you have to run `self_supervised` first.


## Results
- The experiments are saved in "experiments_logs" directory by default (you can change that from args too).
- Each experiment will have a log file and a final classification report in case of modes other that "self-supervised".

## Reference

```
@inproceedings{ijcai2021-324,
  title     = {Time-Series Representation Learning via Temporal and Contextual Contrasting},
  author    = {Eldele, Emadeldeen and Ragab, Mohamed and Chen, Zhenghua and Wu, Min and Kwoh, Chee Keong and Li, Xiaoli and Guan, Cuntai},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, {IJCAI-21}},
  pages     = {2352--2359},
  year      = {2021},
}
```
```
@article{yu2024white,
  title={White-Box Transformers via Sparse Rate Reduction},
  author={Yu, Yaodong and Buchanan, Sam and Pai, Druv and Chu, Tianzhe and Wu, Ziyang and Tong, Shengbang and Haeffele, Benjamin and Ma, Yi},
  journal={Advances in Neural Information Processing Systems},
  volume={36},
  year={2024}
}
```
### Cite  

1. <p name = "ref1">BinSen Xu ,Ning Li, LiZhi , et al. Serial structure multi-task learning method for predicting reservoir parameters[J]. APPLIED GEOPHYSICS, 2022, 19(4): 513-527. DOI: 10.1007/s11770-022-0931-9</p>  
2. <p name = "ref2">BinSen Xu, LiZhi Xiao. 2024. Comparison of well logging formation evaluation using serial and parallel multi-task learning networks. Chinese Journal of Geophysics (in Chinese), 67(4): 1613-1626, doi: 10.6038/cjg2023R0584</p>  
3. <p name = "ref3">徐彬森, 肖立志. 基于串行及并行多任务学习网络的储层参数评价研究[J]. 地球物理学报. 2024, 67(4): 1613-1626. DOI: 10.6038/cjg2023R0584</p>

## Contact
For any issues/questions regarding the reproducing the results, please contact me.   
Ethan Askua  
School of Geophysics,   
China University of Petroleum(Beijing) (CUPB), Beijing.   
Email: xbs150@163.com