3 posts tagged with "amd"

The first AMD Developer Cloud deployment guide showed how to put vLLM Semantic Router in front of a balance-oriented ROCm backend. The maintained multi-objective recipe takes the next step: clients choose the optimization objective they want, while the router keeps each objective's signals, projections, decisions, algorithms, and plugins isolated.

This guide deploys six physical open models across seven serving GPUs, reserves the eighth GPU for router classifiers, and presents five stable Mixture-of-Models entrypoints. Requests move between checkpoints with different architectures, latency, tool-use, and quality profiles instead of simulating those differences with aliases on one backend.

Most agent systems start with a simple idea: call model: auto and let the inference layer pick the right model. That is useful, but it is not enough for long-running agents.

A coding agent can begin with architecture work, call tools, receive short tool outputs, continue with "fix that", then ask a privacy-sensitive question in the same user session. The latest message may look simple, but the route cannot be chosen from the latest message alone. The router also has to know whether this is a safe moment to switch models.

This guide shows how to deploy that pattern on AMD ROCm with vLLM Semantic Router. You will start one ROCm vLLM backend, serve the agentic routing recipe, open the dashboard, validate the OpenAI-compatible API, and use Inferoa to experience route decisions and Router Learning behavior from an agent client.

Agentic routing is not only choosing a model. It is choosing when to keep one.

Running vLLM Semantic Router on AMD Developer Cloud is not just about bringing up one more inference endpoint. It is about turning it into a routed multi-tier system that can classify requests, choose a semantic lane, and make replay and Insights immediately useful.

This post walks through the practical path: start the ROCm backend on an AMD Developer Cloud instance, install vLLM-SR, import the reference profile, and validate the deployment end to end.