Llama 3.1 405B

Llama 3.1 70B

Llama 3.1 8B

GPT-2 XL

GPT-2 large

GPT-2 medium

{c}

https://github.com/karpathy/nanoGPT


nanoGPT

Llama 3.1

Llama 2 7B

{c}

= ChatGPT 4 Turbo
{synonym}

https://platform.openai.com/docs/models/gpt-4-turbo


GPT 4 Turbo

GPT-2 variant

GPT-2 implementation in <PyTorch>

GPT-2 implementation in PyTorch

GPT-2 implementation

{c}

= GPT-2 paper
{c}
{synonym}
{title2}

https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf


Language Models are Unsupervised Multitask Learners

{c}

= GPT-1 paper
{c}
{synonym}
{title2}


Improving Language Understanding by Generative Pre-Training

{title2=2024}

= Llama3
{synonym}

https://www.llama.com/models/llama-3/


Llama 3

{title2=2023}

= Llama2
{synonym}

Page: https://www.llama.com/llama2/


Llama 2

{tag=LLM inference batching}

As of <llama.cpp> 79e0b68c178656bb0632cb8602d2940b755077f8 there is a `--parallel` option but not sure what it does.

Bibliography:
* https://github.com/ggml-org/llama.cpp/discussions/3222
* https://www.reddit.com/r/LocalLLaMA/comments/12aj0ze/what_is_batchsize_in_llamacpp_also_known_as_n/
* https://www.reddit.com/r/LocalLLaMA/comments/12gtanv/batch_queries/
* related for server:
  * https://www.reddit.com/r/LocalLLaMA/comments/1f19t2l/parallel_requests_using_llamaserver


<llama-cli> inference batching

llama-cli inference batching

<llama cli> inference batching

GPT-4

{c}
{title2=175 B parameters}
{title2=2020-06}

* Vocabulary size (V): 50,257
* Hidden size (d_model): 12,288
* Context length	2048
* Q V size: (d_head): 128
* Attention heads	(h): 96
* FFN inner size (d_ff)	4 × 12,288 = 49,152
* Layers (L): 96


Ciro Santilli @cirosantilli 40