Serve Examples

Below are tutorials for exploring Ray Serve capabilities and learning how to integrate different modeling frameworks.

Beginner

Serve ML Models

Serve a Stable Diffusion Model

Serve a Text Classification Model

Serve an Object Detection Model

Intermediate

Serve an Inference Model on AWS NeuronCores Using FastAPI

Serve an Inference with Stable Diffusion Model on AWS NeuronCores Using FastAPI

Serve a model on Intel Gaudi Accelerator

Scale a Gradio App with Ray Serve

Serve a Text Generator with Request Batching

Serve a Chatbot with Request and Response Streaming

Serving models with Triton Server in Ray Serve

Advanced

Serve a Java App