Skip to content

Abstract vGPU devices behind a framework dispatch - #366

Open
yummybomb wants to merge 14 commits into
mainfrom
hypeship/vgpu-framework-abstraction
Open

Abstract vGPU devices behind a framework dispatch#366
yummybomb wants to merge 14 commits into
mainfrom
hypeship/vgpu-framework-abstraction

Conversation

@yummybomb

@yummybomb yummybomb commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Bottom layer of the vendor VFIO vGPU stack (this#322#363#364#321). Behavior-preserving refactor only — no lifecycle semantics change in this layer.

Kernel 6.8 hosts assign NVIDIA vGPUs through a vendor-specific VFIO interface instead of mdev, so the mdev-shaped seams get generalized before the new backend lands above:

  • VGPUDevice / VGPUFramework abstraction — mdev moves behind a framework dispatch (CreateVGPU / DestroyVGPU / DiscoverVGPU), with assignments described by a framework + device path instead of a bare mdev UUID.
  • Hypervisor selection — vGPU instances require QEMU; the policy moves to callers instead of being implied by mdev.
  • Metadata — VF allocation tracked with a single field; instance metadata carries GPUFramework / GPUDevicePath alongside the mdev UUID.
  • QEMU args — vGPU attaches via sysfsdev, and the dead mdev branch is dropped from PCI passthrough args.

Testing

  • go build ./..., go vet clean
  • go test -race ./lib/devices/ ./lib/hypervisor/... and targeted lib/instances suites pass (TestSocketCacheKeyChangesWhenSocketIsRecreated and the network/image-dependent instances tests fail identically on the unmodified stack head in this environment)

Note

Medium Risk
Changes GPU allocation, teardown, and how VMs attach vGPU devices across instance lifecycle and QEMU/CH config, though current behavior still routes through mdev on Linux.

Overview
Introduces a framework-agnostic vGPU layer so instance lifecycle and hypervisors no longer talk to mdev directly. Linux still creates/destroys mdev via CreateVGPU / DestroyVGPU; macOS stubs return the existing unsupported error.

Devices & metadata: Adds VGPUDevice, VGPUFramework, and renames VF has_mdevallocated. Instance metadata now stores GPUFramework and GPUDevicePath (with GPUMdevUUID kept for mdev); helpers in vgpu.go centralize set/clear/release and legacy path resolution from UUID-only records.

VM attachment: hypervisor.VMConfig gains VGPUDevicePath, separate from whole-GPU PCIDevices. QEMU always attaches vGPU with vfio-pci,sysfsdev=… from that field and no longer treats mdev paths inside the PCI passthrough loop. Cloud Hypervisor appends the vGPU sysfs path to its device list the same way.

Lifecycle: Create, start, stop, and delete call the new APIs and releaseStoredVGPU instead of inline CreateMdev / DestroyMdev.

Reviewed by Cursor Bugbot for commit 4499016. Bugbot is set up for automated code reviews on this repo. Configure here.

@yummybomb yummybomb changed the title hypeship/vgpu framework abstraction Abstract vGPU devices behind a framework dispatch Aug 6, 2026
@yummybomb
yummybomb marked this pull request as ready for review August 6, 2026 19:20
@yummybomb
yummybomb force-pushed the hypeship/vgpu-framework-abstraction branch 2 times, most recently from c3d6a2f to f677357 Compare August 6, 2026 19:40

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f677357. Configure here.

Comment thread lib/instances/start.go Outdated
@yummybomb
yummybomb force-pushed the hypeship/vgpu-framework-abstraction branch from a903554 to 4499016 Compare August 7, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant