Transformers from scratch

Workshop

Coding Transformers, the framework that ChatGPT is built on, from Scratch

Thursday, 16. May 2024 | 09:00 - 17:00

Description

This workshop explored how to code, train, and use transformers from scratch in Python with PyTorch and Lightning. Participants then leveraged pre-made transformer classes to build, train, and use a Large Language Model (LLM) Transformer capable of answering basic questions.

Topics

Step-by-step overviews of how transformers work
An introduction to Lightning Studios for easy access cloud computing
Coding a transformer from scratch, including word embedding, positional encoding, self-attention, masked self-attention, and encoder-decoder attention.
Training our transformer with a simple dataset to translate English to Spanish.
Using our transformer to translate user input.
Downloading and fine-tuning an open source large language model.
Using the large language model to answer basic questions.

Requirements

Basic coding skills in Python.

Target audience

Anyone who wants to learn how artificial intelligence (AI) like ChatGPT works. No previous experience with AI is necessary.

Speaker

Joshua Starmer

Founder and CEO at StatQuest and Lead AI Educator at Lightning AI