What is minGPT?
minGPT is a PyTorch re-implementation of GPT, designed to be small, clean, interpretable, and educational. It focuses on being minimal and educational, making GPT more accessible and easier to understand.
Key Features
- Minimal Implementation: Around 300 lines of code, focusing on the core aspects of GPT.
- Educational: Designed to be interpretable and easy to learn from.
- Training and Inference: Covers both aspects for a comprehensive understanding.
- Library Installation: Simple installation and integration into projects.
- Usage Examples: Includes demos and projects to illustrate usage.
Installation
To install minGPT:
git clone https://github.com/karpathy/minGPT.git
cd minGPT
pip install -e .
Usage
Instantiate a GPT-2 (124M param version):
from mingpt.model import GPT
model_config = GPT.get_default_config()
model_config.model_type = 'gpt2'
model_config.vocab_size = 50257
model_config.block_size = 1024
model = GPT(model_config)
Train the model:
train_dataset = YourDataset()
from mingpt.trainer import Trainer
train_config = Trainer.get_default_config()
train_config.learning_rate = 5e-4
train_config.max_iters = 1000
train_config.batch_size = 32
trainer = Trainer(train_config, model, train_dataset)
trainer.run()
Additional Information
- Unit Tests: Run
python -m unittest discover tests
for testing. - Todo List: Includes planned features and improvements.
- References: Links to related code repositories and papers.
Contributors
License
MIT License.