 GPT-3

ID: gpt-3

 Top articles  Latest articles New article in topic

GPT-3 by

Ciro Santilli 37 2025-08-08

Vocabulary size (V): 50,257
Hidden size (d_model): 12,288
Context length 2048
Q V size: (d_head): 128
Attention heads (h): 96
FFN inner size (d_ff) 4 × 12,288 = 49,152
Layers (L): 96

 Read the full article

 New to topics? Read the docs here!