# Transformer_code

**Repository Path**: whiteeat/Transformer_code

## Basic Information

- **Project Name**: Transformer_code
- **Description**: For a better understanding of Transformer_code
- **Primary Language**: Python
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2022-03-23
- **Last Updated**: 2022-04-28

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Transformer_code
For a better understanding of Transformer_code

为了更好的理解Transformer的代码，我查阅相关资料，借鉴多位大佬内容进行整合。

主要内容按 Transformer 的架构分块进行编写，可在 Jupyter 上进行查阅和修改。

相关图示如下(具体内容查看Jupyter)：

![image](https://user-images.githubusercontent.com/85781642/159620753-a5e06f55-3fd6-40a3-9821-79e0c6bc4129.png)

主要参考内容：

 1.[How to code The Transformer in Pytorch](https://towardsdatascience.com/how-to-code-the-transformer-in-pytorch-24db27c8f9ec#b0ed)

 2.[Transformer 的 PyTorch 实现](https://wmathor.com/index.php/archives/1455/)
 
 3.[The Annotated Transformer](nlp.seas.harvard.edu/2018/04/03/attention.html)