代码拉取完成，页面将自动刷新
Better Language Models and Their Implications
Our model, called GPT-2 (a successor to GPT), was trained simply to predict the next word in 40GB of Internet text. Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper. from openAI Blog
This repository is simple implementation GPT-2 about text-generator in Pytorch with compress code
The original repertoire is openai/gpt-2. Also You can Read Paper about gpt-2, "Language Models are Unsupervised Multitask Learners". To Understand more detail concept, I recommend papers about Transformer Model.
Good implementation GPT-2 in Pytorch which I referred to, huggingface/pytorch-pretrained-BERT, You can see more detail implementation in huggingface repository.
Transformer(Self-Attention) Paper : Attention Is All You Need(2017)
First OpenAi-GPT Paper : Improving Language Understanding by Generative Pre-Training(2018)
See OpenAI Blog about GPT-2 and Paper
$ git clone https://github.com/graykode/gpt-2-Pytorch && cd gpt-2-Pytorch
# download huggingface's pytorch model
$ curl --output gpt2-pytorch_model.bin https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-pytorch_model.bin
# setup requirements, if using mac os, then run additional setup as descibed below
$ pip install -r requirements.txt
$ python main.py --text "It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him."
--text : sentence to begin with.--quiet : not print all of the extraneous stuff like the "================"--nsamples : number of sample sampled in batch when multinomial function use--unconditional : If true, unconditional generation.--batch_size : number of batch size--length : sentence length (< number of context)--temperature: the thermodynamic temperature in distribution (default 0.7)
--top_k : Returns the top k largest elements of the given input tensor along a given dimension. (default 40)
See more detail option about temperature and top_k in here
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install torch tqdm
$ brew install libomp
$ export LC_ALL=en_US.UTF-8
$ export LANG=en_US.UTF-8
$ pip install -r requirements.txt
Jeff Wu(@WuTheFWasThat), Thomas Wolf(@thomwolf) for allowing referring code.
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
马建仓 AI 助手
