Lectures | CS324.
Percy Liang; Tatsunori Hashimoto; Christopher Ré.
stanford-cs324.github.io .
2022.
Accessed Dec 14, 2023.
- ✅ Introduction: What is an LM? A brief history . Why does CS 324 exist?
- Capabilities: What are the capabilities of GPT-3?
- Harms I & II: Performance disparities, social biases and stereotypes, toxicity, and misinformation.
- Data: Data behind LLMs; documentation of datasets; data ecosystems.
- Security & Privacy: Security implications of LLMs; data poisoning; privacy risks and opportunities.
- Legality: The law on development and deployment of LLMs; distinction between law and ethics.
- Modeling: Tokenization; model architecture.
- Training: Objective functions; optimization algorithms.
- Parallelism: Key goal of hardware, systems, and for more than a decade, the only way to get performance.
- Scaling Laws: Motivating problem: hyper-parameter costs.
- Selective Architectures: Raising the ceiling of how big the models can get.
- Adaptation: Why adapt the LM? Probing; fine-tuning; lightweight fine-tuning.
- Environmental Impact: What is the environmental impact of LLMs?