This document is relevant for: Trn1, Trn2

Tutorials

This section will go over tutorials to help users get started with NxD Training library.

• Megatron GPT Pretraining

• HuggingFace Llama3.1/Llama3-8B Pretraining

• HuggingFace Llama3.1/Llama3-8B Supervised Fine-tuning

• HuggingFace Llama3.1/Llama3-8B Efficient Supervised Fine-tuning with LoRA (Beta)

• HF Llama3.1/Llama3-8B Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO) based Fine-tuning (Beta)

• HuggingFace Llama3.1/Llama3-70B Pretraining

• Checkpoint Conversion

This document is relevant for: Trn1, Trn2