MiniGPT
A decoder-only, GPT-style Transformer built entirely from scratch in PyTorch — no nn.Transformer shortcuts — with a live self-attention visualizer.
Overview
MiniGPT is a decoder-only, GPT-style Transformer I implemented entirely from scratch in PyTorch — token and positional embeddings, multi-head causal self-attention, and pre-norm residual blocks — deliberately without the nn.Transformer shortcut, to understand every piece.
It’s a character-level language model with autoregressive sampling (temperature / top-k), wrapped in a Gradio interface for in-browser training, text generation, and a live self-attention heatmap so you can watch the model attend.
What I built
Transformer internals
Implemented embeddings, multi-head causal self-attention, and pre-norm residual blocks by hand.
Autoregressive generation
Built a character-level LM with temperature and top-k sampling for controllable text generation.
Interactive learning UI
A Gradio app for in-browser training, generation, and live self-attention heatmap visualization.
Built with
- Python
- PyTorch
- Gradio
Have a project in mind?
Whether it’s a business website, an app, or something AI-powered — tell me about it and I’ll reply the same day.