AMD GPUs and ROCm

Select AMD GPU classes and run ROCm-compatible containers on SaladCloud.

Last Updated: September 10, 2026

Overview

SaladCloud supports selected AMD GPU classes alongside its NVIDIA classes. AMD workloads use ROCm and HIP where NVIDIA workloads use CUDA.

Salad Container Engine runs every container inside WSL2, so AMD GPUs are reached through the WSL2 GPU path rather than the native Linux ROCm path. The device inside your container is /dev/dxg. There is no /dev/kfd and no /dev/dri; do not use either as a health check.

Selecting an AMD class does not make a CUDA image portable. A CUDA-only image cannot use an AMD GPU, and a ROCm image cannot use an NVIDIA GPU, unless that exact image was built and tested for both.

Image Requirements

Ship ROCm 7.1 or newer. Anything older, including 7.0, fails GPU enumeration with HSA_STATUS_ERROR_OUT_OF_RESOURCES. AMD’s own host-side requirements, such as an Adrenalin or host ROCm version, are not your image requirement — SaladCloud manages the host.

Include rocminfo. It is the readiness check for AMD on SaladCloud. Selecting a GPU class does not add missing executables to your container.

Build for the right gfx target. Every compiled extension needs code for the architecture of the class you selected. One image serving several AMD classes must cover all of their targets.

Never Assign LD_LIBRARY_PATH or PYTHONPATH

SaladCloud injects host library paths into both variables. An entrypoint that assigns rather than appends erases them:

export LD_LIBRARY_PATH=/opt/rocm/lib             # WRONG - erases the injected paths
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH   # correct

This is the most common self-inflicted failure on AMD, and it is hard to spot because it produces HSA_STATUS_ERROR_OUT_OF_RESOURCES — the same error as an image whose ROCm is too old. Clobbering PYTHONPATH instead breaks the injected amdsmi module, which breaks vLLM’s ROCm detection.

python -I and python -E entrypoints have the same effect on PYTHONPATH, silently. If you need isolated mode, add the injected paths to sys.path yourself.

Reference Images

These run on SaladCloud AMD GPU classes. We recommend starting from one, then pinning the digest you tested.

• rocm/pytorch:latest
• rocm/dev-ubuntu-24.04 at tags 7.1, 7.2, and latest
• rocm/vllm:rocm7.14.0_rdna_ubuntu24.04_py3.14_pytorch_2.11.0_vllm_0.23.0

What the Host Provides

SaladCloud injects the following into every container on an AMD node. You configure none of it, but knowing it exists is what keeps your image from breaking it.

Path	Contents
/dev/dxg	The WSL kernel GPU device. It brokers access to every GPU on the node; there is no per-GPU node.
/usr/lib/wsl/lib	Microsoft’s WSL-to-GPU shim (libdxcore.so) and the libd3d12* libraries it loads
/opt/rocm-host/lib	The host librocdxg ROCm-to-DXG bridge, and nothing else
/opt/rocm-wsl	The host’s DXG-capable AMD SMI (lib, bin, share/amd_smi, libexec/amdsmi_cli)

Three environment variables carry those paths, with your image’s own value preserved in the middle:

Variable	Value
LD_LIBRARY_PATH	/opt/rocm-host/lib : yours : /usr/lib/wsl/lib:/opt/rocm-wsl/lib
PYTHONPATH	/opt/rocm-wsl/share/amd_smi : yours : /opt/rocm-wsl/libexec/amdsmi_cli
PATH	/opt/rocm-wsl/bin : yours

HSA_ENABLE_DXG_DETECTION=1 is also set. Without it some ROCm builds enumerate zero agents.

/opt/rocm-host/lib is prepended so the host bridge beats any copy your image bundles; librocdxg is version-matched to the host driver, much as libcuda.so is on NVIDIA. Everything else is appended, so your image’s ROCm runtime, HIP, and math libraries still win. Nothing mounts at /opt/rocm, so an image shipping its own ROCm there is untouched.

The host does not provide a ROCm runtime. Your image brings its own ROCm, HIP, and math libraries. The host supplies only what no image can: the bridge and the WSL driver libraries. Do not install a Linux GPU kernel driver, do not install rocdxg-amd-smi-lib, and do not replace the injected libraries from inside the container.

AMD’s local WSL2 Docker instructions include device and library-mount flags for a host you administer. Those are host configuration; SaladCloud already performs the equivalent injection. Do not translate them into container-group settings.

Checking That the GPU Works

Run rocminfo. It enumerates the ROCr agents and reports a specific error when it cannot, which makes it the check to build on. Confirm that the expected gfx target appears, then run a real GPU operation — enumeration and a successful framework import are only smoke tests.

rocm-smi exits 0 even when it has not reached a GPU. Never use its exit code as a health signal, startup probe, or readiness check.

The injected amd-smi works. SaladCloud puts the host’s DXG-capable build first on PATH, and nothing is required of you. This matters beyond telemetry: vLLM decides whether it is on ROCm by calling amdsmi.amdsmi_init(), and some PyTorch versions report a phantom extra GPU when that call fails.

Your image’s own amd-smi and rocm-smi cannot work. They check /proc/modules for an amdgpu kernel driver that does not exist under WSL2, and fail with AMDSMI_STATUS_DRIVER_NOT_LOADED or a suggestion to run sudo modprobe amdgpu. Ignore that suggestion; use the injected build on PATH, or /opt/rocm-wsl/bin/amd-smi directly.

PyTorch

PyTorch reuses the torch.cuda namespace on ROCm, so torch.cuda.is_available() alone does not tell you which backend you have. Check torch.version.hip, which is populated on a ROCm build while torch.version.cuda is None:

import torch

print("PyTorch:", torch.__version__)
print("ROCm/HIP:", torch.version.hip)
print("CUDA:", torch.version.cuda)
print("GPU available:", torch.cuda.is_available())
print("GPU count:", torch.cuda.device_count())

Then run a real tensor operation and call torch.cuda.synchronize() so asynchronous errors surface. The PyTorch AMD/ROCm tutorial has a worked example.

Integrated GPUs

Some AMD nodes have an integrated GPU beside the discrete card. AMD’s ROCDXG bridge has a known limitation here: only the first enumerated ROCr agent can create a context. If the integrated GPU enumerates first, the discrete card is unreachable.

SaladCloud avoids this by injecting a librocdxg that does not enumerate integrated GPUs at all, leaving the discrete card as agent 0. But rocm/pytorch and rocm/vllm both bundle their own copy under _rocm_sdk_core, and an entrypoint that clobbers LD_LIBRARY_PATH hands the decision back to it. You then get [CreateContext] fail c000000d with a segfault, or hipErrorInvalidImage.

No environment variable fixes this. HIP_VISIBLE_DEVICES, ROCR_VISIBLE_DEVICES, and addressing cuda:1 directly all fail the same way. The fix is to stop overriding LD_LIBRARY_PATH.

Choosing AMD GPU Classes

Use the Portal or the List GPU Classes API for the classes and IDs currently available to your organization; inventory and pricing change, so do not copy a UUID from an unrelated example. Before a large deployment, check the GPU Availability API with the CPU, memory, storage, and country requirements you intend to use.

Start with a single class and one replica. A container group spanning several classes can place an instance on any of them, so your image has to satisfy every gfx target in the set. Keep AMD and NVIDIA classes in separate container groups: an instance allocated from the wrong vendor cannot run a vendor-specific image.

Treat the class, the pinned image digest, and your application configuration as one tested unit, and revalidate when any of them changes.

Troubleshooting

HSA_STATUS_ERROR_OUT_OF_RESOURCES

Usually preceded by WSL environment detected and an hsa api call failure line. ROCr could not enumerate the GPU. Two causes produce identical output:

1. The image’s ROCm is older than 7.1. See Image Requirements.
2. An entrypoint erased LD_LIBRARY_PATH, so /opt/rocm-host/lib is gone and librocdxg cannot be found. Print the variable inside the container and check. See Never Assign LD_LIBRARY_PATH or PYTHONPATH.

This is not a class allocation problem and not a ROCR_VISIBLE_DEVICES problem.

[CreateContext] fail c000000d, Segfault, or hipErrorInvalidImage

The node has an integrated GPU and your image’s bundled librocdxg is winning over the injected one. No environment variable will fix it. See Integrated GPUs.

AMD SMI Reports That amdgpu Is Not Loaded

You are running your image’s AMD SMI rather than the injected host build. Do not run modprobe. Confirm /opt/rocm-wsl/bin is still first on PATH, or call /opt/rocm-wsl/bin/amd-smi directly. See Checking That the GPU Works.

No AMD Device Is Visible

Run rocminfo and read its error. HSA_STATUS_ERROR_OUT_OF_RESOURCES means the ROCm version or LD_LIBRARY_PATH problem above, not a visibility problem.

If rocminfo does enumerate agents but your application sees nothing, check ROCR_VISIBLE_DEVICES, HIP_VISIBLE_DEVICES, and CUDA_VISIBLE_DEVICES — each can restrict what a HIP application sees. If it is still unavailable, send Salad support the class ID, image digest, rocminfo output, the container’s LD_LIBRARY_PATH, Container Logs, and System Events. Do not try to create or mount GPU devices yourself.

torch.cuda.is_available() Is False

If torch.version.hip is None, the installed PyTorch is not a ROCm build. If it is populated, run rocminfo to see whether the GPU enumerates at all, then work back through the ROCm version and device visibility.

hipErrorNoBinaryForGPU

No compatible kernel image for this device — something was compiled for a different gfx target. Identify the target with rocminfo, rebuild the affected code, and retest on the same class.

The Replica Repeatedly Exits or Reallocates

Check Container Logs and System Events before touching the ROCm stack. A failure before ROCm initializes is usually an ordinary container configuration problem — entrypoint, CPU and memory, or exit code.

A Model Feature, Precision, or Quantization Fails

Upstream ROCm support does not mean every feature works on every architecture. Check the application’s own ROCm support notes, then validate that exact configuration on the class you selected.

Performance Is Lower Than Expected

Confirm the GPU is actually in use, run a warm-up, and measure the real workload rather than a smoke test. See Build high-performance applications.

AMD64 Appears in an Image or Binary Name

That is the x86-64 CPU architecture. It says nothing about ROCm or AMD GPU support.

Related SaladCloud Documentation

• Run PyTorch on AMD GPUs with ROCm
• List GPU Classes API
• GPU Availability API
• Container group resource selection
• Startup Probes and Health Probes
• System Events
• Troubleshooting Salad Container Engine workloads
• Build high-performance applications

Related AMD and Framework Documentation

• AMD: ROCm compatibility matrix
• AMD: WSL compatibility matrix
• AMD: PyTorch on ROCm for WSL
• AMD: PyTorch compatibility
• AMD: HIP environment variables
• AMD: HIP performance guidelines
• PyTorch: HIP semantics