GPT-J

  • Link: https://huggingface.co/EleutherAI/gpt-j-6B

  • Family: GPT

  • Pretraining Architecture: Decoder

  • Pretraining Task: LM

  • Extension: GPT-J 6B is a Transformer model trained using Mesh Transformer JAX and same tokenizer as GPT2/3

  • Application: Same as GPT-3

  • Date (of first known publication): 05/2021

  • Num. Params: 6B

  • Corpus: Pile corpus, a large-scale curated dataset created by EleutherAI.

  • License: Open, Apache-2.0

  • Lab: EleutherAI

PreviousGPT-NeoX-20BNextPythia

Last updated