2 posts tagged with "deployment"

The first AMD Developer Cloud deployment guide showed how to put vLLM Semantic Router in front of a balance-oriented ROCm backend. The maintained multi-objective recipe takes the next step: clients choose the optimization objective they want, while the router keeps each objective's signals, projections, decisions, algorithms, and plugins isolated.

This guide deploys six physical open models across seven serving GPUs, reserves the eighth GPU for router classifiers, and presents five stable Mixture-of-Models entrypoints. Requests move between checkpoints with different architectures, latency, tool-use, and quality profiles instead of simulating those differences with aliases on one backend.

Running vLLM Semantic Router on AMD Developer Cloud is not just about bringing up one more inference endpoint. It is about turning it into a routed multi-tier system that can classify requests, choose a semantic lane, and make replay and Insights immediately useful.

This post walks through the practical path: start the ROCm backend on an AMD Developer Cloud instance, install vLLM-SR, import the reference profile, and validate the deployment end to end.