Abstract:

The course presents the theoretical and practical foundations of modern Natural Language Processing. We will begin with text representation, tokenization, language models, embeddings, and classical methods for text classification and retrieval. We will then study neural language models, Attention, and the Transformer architecture, which forms the basis for large language models such as GPT and BERT.

The course emphasizes contemporary applications of large language models, including prompting, instruction following, model adaptation using PEFT/LoRA, the construction of Retrieval Augmented Generation systems, and the use of Agents that operate external tools.

In addition, the course will cover the evaluation of NLP and LLM systems using a variety of metrics, including BLEU, ROUGE, Perplexity, BERTScore, Precision@k, Faithfulness, and Task Success.