Project-HAMi / HAMi

English version | 中文版 | 日本語版
# HAMi
**Kubernetes GPU virtualization and heterogeneous accelerator scheduling for AI infrastructure.**
HAMi stands for **Heterogeneous AI Computing Virtualization Middleware**. Formerly known as `k8s-vGPU-scheduler`, HAMi helps platform teams share expensive GPUs and other AI accelerators across Kubernetes workloads, isolate device memory and compute, and schedule pods with device-aware policies without changing application code.
HAMi is a CNCF Incubating and CNCF Landscape project. It is also listed in the CNAI Landscape.
## Why HAMi?
AI infrastructure teams often run into the same Kubernetes accelerator problems: whole GPUs are allocated to small jobs, teams compete for scarce devices, different accelerator vendors expose different operational models, and schedulers lack enough device context to place workloads efficiently.
HAMi provides a Kubernetes-native layer for:
- **Device sharing**: allocate a fraction of a physical accelerator by memory, core, or device count.
- **Resource isolation**: enforce per-workload accelerator memory and compute limits where the device backend supports it.
- **Device-aware scheduling**: place pods with topology-aware, binpack, spread, and device-specific scheduling policies.
- **Heterogeneous AI clusters**: manage NVIDIA GPUs, NPUs, HCUs, MLUs, and other accelerator types through one scheduling and allocation workflow.
- **Zero application changes**: keep using standard Kubernetes resource requests and limits.
- **Production operations**: expose metrics, dashboards, WebUI, Helm installation, and community-supported deployment guidance.
## Use Cases
- Increase GPU utilization in shared Kubernetes AI clusters.
- Run multi-tenant notebook, training, and inference workloads on the same accelerator pool.
- Build private cloud AI platforms with fair device allocation and quota control.
- Operate heterogeneous accelerator clusters across NVIDIA, Ascend, Cambricon, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, and other vendors.
- Combine HAMi with Kubernetes schedulers such as kube-scheduler and Volcano for batch AI workloads.
## How It Works
HAMi is composed of a mutating webhook, scheduler extender, device plugins, and device-specific in-container virtualization components.
 [code] 
## Device Virtualization
HAMi lets workloads request only the accelerator resources they need. For example, the following pod asks for one physical NVIDIA GPU with 3 GiB of GPU memory:
 [code] 
The workload sees the allocated device resources inside the container, while HAMi coordinates scheduling, allocation, and isolation.
> Notes:
>
> 1. After installing HAMi, the value of `nvidia.com/gpu` registered on the node defaults to the number of vGPUs.
> 2. When requesting resources in a pod, `nvidia.com/gpu` refers to the number of physical GPUs required by the current pod.
## Supported Devices
HAMi supports multiple heterogeneous accelerator backends, including GPUs, NPUs, HCUs, MLUs, GCUs, XPUs, and more.