Learning Resources

A comprehensive collection of learning resources organized to match the LLM development roadmap structure.

Part 1: The Foundations 🔍

🎯 Focus: Core ML concepts, neural networks, traditional models, tokenization, embeddings, transformers
📈 Difficulty: Beginner to Intermediate
🎓 Outcome: Solid foundation in ML/NLP fundamentals and transformer architecture

Prerequisites

Mathematics & Statistics:

Programming & Python:

Books:

Machine Learning Fundamentals:

Deep Learning Basics:

1. Neural Networks Foundations for LLMs

📈 Difficulty: Intermediate 🎯 Prerequisites: Calculus, linear algebra

Core Textbooks & Courses:

Mathematical Foundations:

Essential Papers & Articles:

2. Traditional Language Models

📈 Difficulty: Intermediate 🎯 Prerequisites: Probability, statistics

Core Textbooks:

N-gram Models:

RNN & LSTM Resources:

Foundational Papers:

Historical Context:

Dependency Parsing:

3. Tokenization

📈 Difficulty: Beginner 🎯 Prerequisites: Python basics

Core Concepts & Posts:

Hands-On Implementations:

Interactive Tools:

Libraries & Documentation:

Research Papers:

4. Embeddings

📈 Difficulty: Beginner-Intermediate 🎯 Prerequisites: Linear algebra, Python

Core Concepts & Posts:

Hands-On Implementations:

Foundational Papers:

Advanced Topics:

5. The Transformer Architecture

📈 Difficulty: Advanced 🎯 Prerequisites: Neural networks, linear algebra

Foundational Paper:

Visual Explanations:

Technical Deep Dives:

Implementation Posts:

Textbook Resources:

Applications & Extensions:

Part 2: Building & Training Models 🧬

🎯 Focus: Data preparation, pre-training, fine-tuning, preference alignment
📈 Difficulty: Intermediate to Advanced
🎓 Outcome: Ability to train and fine-tune language models from scratch

🎯 Learning Objectives: Learn to prepare high-quality datasets, implement distributed pre-training, create instruction datasets, perform supervised fine-tuning, and align models with human preferences using advanced techniques like RLHF and DPO.

6. Data Preparation

📈 Difficulty: Intermediate 🎯 Prerequisites: Python, SQL

Data Collection & Scraping:

Data Processing Libraries:

Data Quality & Ethics:

Text Preprocessing:

Version Control & Management:

LLM-Specific Resources:

7. Pre-Training Large Language Models

📈 Difficulty: Expert 🎯 Prerequisites: Transformers, distributed systems

Foundational Understanding:

Video Resources:

Key Research Papers:

Training Frameworks & Tools:

8. Post-Training Datasets (for Fine-Tuning)

📈 Difficulty: Intermediate 🎯 Prerequisites: Data preparation

Instruction Datasets:

Conversation Datasets:

Preference & RLHF Datasets:

Question Answering:

Resources:

9. Supervised Fine-Tuning (SFT)

📈 Difficulty: Advanced 🎯 Prerequisites: Pre-training basics

Libraries & Tools:

Research Papers:

Implementation Examples:

Posts:

Parameter-Efficient Methods:

10. Preference Alignment (RL Fine-Tuning)

📈 Difficulty: Expert 🎯 Prerequisites: Reinforcement learning basics

Libraries & Frameworks:

Core RLHF Papers:

Constitutional AI & Safety:

Scaling & Evaluation:

Learning Resources:

Part 3: Advanced Topics & Specialization ⚙️

🎯 Focus: Evaluation, reasoning, optimization, architectures, enhancement
📈 Difficulty: Expert/Research Level
🎓 Outcome: Research credentials, publications, and ability to lead theoretical advances

🎯 Learning Objectives: This advanced track develops research-grade expertise in LLM evaluation, reasoning enhancement, model optimization, novel architectures, and model enhancement techniques for cutting-edge research and development.

11. Model Evaluation

📈 Difficulty: Intermediate 🎯 Prerequisites: Statistics, model training

Standard Benchmarks:

Evaluation Frameworks:

Specialized Evaluation:

LLM-as-Judge:

Research & Methodology:

12. Reasoning

📈 Difficulty: Intermediate 🎯 Prerequisites: Prompt engineering

Core Reasoning Papers:

Tool Use & Action:

Evaluation Datasets:

Advanced Reasoning Systems:

Resources:

13. Quantization

📈 Difficulty: Intermediate 🎯 Prerequisites: Model optimization

Quantization Libraries:

Advanced Quantization Methods:

Formats & Standards:

Learning Resources:

14. Inference Optimization

📈 Difficulty: Advanced 🎯 Prerequisites: Model deployment

High-Performance Inference Engines:

Attention Optimization:

Advanced Techniques:

Learning Resources:

15. Model Architecture Variants

📈 Difficulty: Advanced 🎯 Prerequisites: Transformer architecture

Sparse & Efficient Architectures:

State Space Models:

Long Context Models:

Positional Encodings:

16. Model Enhancement

📈 Difficulty: Advanced 🎯 Prerequisites: Model training, optimization

Context Window Extension:

Model Merging & Composition:

Knowledge Transfer:

Learning Resources:

Part 4: Engineering & Applications 🚀

🎯 Focus: Production deployment, RAG, agents, multimodal, security, ops
📈 Difficulty: Intermediate to Advanced
🎓 Outcome: Production-ready LLM applications and systems at scale

🎯 Learning Objectives: This production-focused track teaches deployment optimization, inference acceleration, application development with RAG systems and agents, multimodal integration, LLMOps implementation, and responsible AI practices for scalable LLM solutions.

17. Running LLMs & Building Applications

📈 Difficulty: Intermediate 🎯 Prerequisites: Web development, APIs

Web Frameworks:

LLM APIs:

Local LLM Tools:

Development Tools:

Technologies:

Educational Platforms:

Learning Resources:

18. Retrieval Augmented Generation (RAG)

📈 Difficulty: Advanced 🎯 Prerequisites: Embeddings, databases

RAG Frameworks:

Vector Databases:

Graph RAG:

Foundational RAG Papers:

Advanced RAG Research:

Question Answering:

Learning Resources:

19. Tool Use & AI Agents

📈 Difficulty: Advanced 🎯 Prerequisites: Function calling, planning

Agent Frameworks:

Function Calling & Tools:

Microsoft Frameworks:

Learning Resources:

20. Multimodal LLMs

📈 Difficulty: Advanced 🎯 Prerequisites: Computer vision, audio processing

Vision-Language Models:

Audio Processing:

Image Generation:

Processing Libraries:

Learning Resources:

21. Securing LLMs & Responsible AI (Optional)

📈 Difficulty: Advanced 🎯 Prerequisites: Security fundamentals, ethical AI

Security Frameworks:

Attack Vectors & Defense:

Safety & Evaluation:

Privacy Protection:

Learning Resources:

Interpretability Research:

22. Large Language Model Operations (LLMOps)

📈 Difficulty: Advanced 🎯 Prerequisites: DevOps, MLOps, cloud platforms

MLOps Platforms:

Infrastructure & Orchestration:

Monitoring & Observability:

Data Processing:

CI/CD & Model Management:

Back to top

Copyright © 2025 Mohammad Shojaei. All rights reserved. You may copy and distribute this work, but please note that it may contain other authors' works which must be properly cited. Any redistribution must maintain appropriate attributions and citations.