Train Examples
Below are examples for using Ray Train with a variety of frameworks and use cases.
Beginner
Framework | Example |
---|---|
PyTorch | |
Lightning | |
Accelerate, PyTorch, Hugging Face | Distributed Data Parallel Training with Hugging Face Accelerate |
DeepSpeed, PyTorch | |
TensorFlow | |
Horovod | |
PyTorch | |
Transformers |
Intermediate
Framework | Example |
---|---|
PyTorch | |
Lightning | |
Transformers | Fine-tune a Text Classifier on GLUE Benchmark with Hugging Face Accelerate |
Advanced
Framework | Example |
---|---|
Accelerate, DeepSpeed, Hugging Face | Fine-tune Llama-2 series models with Deepspeed, Accelerate, and Ray Train TorchTrainer |
Hugging Face, DeepSpeed | |
Lightning, DeepSpeed | |
Lightning |