Includes all pending changes needed for nixos-anywhere: - fiberlb: L7 policy, rule, certificate types - deployer: New service for cluster management - nix-nos: Generic network modules - Various service updates and fixes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
181 lines
7.2 KiB
YAML
181 lines
7.2 KiB
YAML
id: T054
|
|
name: PlasmaVMC Operations & Resilience
|
|
goal: Implement missing VM lifecycle operations (Update, Reset, Hotplug) and ChainFire state watch
|
|
status: complete
|
|
priority: P1
|
|
owner: peerB
|
|
created: 2025-12-12
|
|
depends_on: []
|
|
blocks: [T039]
|
|
|
|
context: |
|
|
**Findings from T049 Audit:**
|
|
- `vm_service.rs` TODOs: Update, Reset, Disk/NIC attachment/detachment.
|
|
- "Implement VM watch via ChainFire watch" is pending.
|
|
|
|
**Strategic Value:**
|
|
- Required for production operations (resizing VMs, recovering stuck VMs, attaching volumes).
|
|
- ChainFire watch is critical for multi-node state synchronization (HA).
|
|
|
|
acceptance:
|
|
- VM Update (CPU/RAM) changes persisted and applied (next boot or live if supported)
|
|
- VM Reset (Hard/Soft) functional
|
|
- Disk/NIC hot-plug/unplug functional via QMP
|
|
- PlasmaVMC servers watch ChainFire for state changes (external coordination)
|
|
|
|
steps:
|
|
- step: S1
|
|
name: VM Lifecycle Ops
|
|
done: Implement Update and Reset APIs
|
|
status: complete
|
|
completed: 2025-12-12 18:00 JST
|
|
owner: peerB
|
|
priority: P1
|
|
outputs:
|
|
- path: plasmavmc/crates/plasmavmc-server/src/vm_service.rs
|
|
note: Implemented update_vm and reset_vm methods
|
|
notes: |
|
|
Implemented:
|
|
- reset_vm: Hard reset via QMP system_reset command (uses existing reboot backend method)
|
|
- update_vm: Update VM spec (CPU/RAM), metadata, and labels
|
|
* Updates persisted to storage
|
|
* Changes take effect on next boot (no live update)
|
|
* Retrieves current status if VM is running
|
|
|
|
Implementation details:
|
|
- reset_vm follows same pattern as reboot_vm, calls backend.reboot() for QMP system_reset
|
|
- update_vm uses proto_spec_to_types() helper for spec conversion
|
|
- Properly handles key ownership for borrow checker
|
|
- Returns updated VM with current status
|
|
|
|
- step: S2
|
|
name: Hotplug Support
|
|
done: Implement Attach/Detach APIs for Disk/NIC
|
|
status: complete
|
|
completed: 2025-12-12 18:50 JST
|
|
owner: peerB
|
|
priority: P1
|
|
outputs:
|
|
- path: plasmavmc/crates/plasmavmc-kvm/src/lib.rs
|
|
note: QMP-based disk/NIC attach/detach implementation
|
|
- path: plasmavmc/crates/plasmavmc-server/src/vm_service.rs
|
|
note: Service-level attach/detach methods
|
|
|
|
- step: S3
|
|
name: ChainFire Watch
|
|
done: Implement state watcher for external events
|
|
status: complete
|
|
started: 2025-12-12 18:05 JST
|
|
completed: 2025-12-12 18:15 JST
|
|
owner: peerA
|
|
priority: P1
|
|
outputs:
|
|
- path: plasmavmc/crates/plasmavmc-server/src/watcher.rs
|
|
note: State watcher module (280+ lines) for ChainFire integration
|
|
notes: |
|
|
Implemented:
|
|
- StateWatcher: Watches /plasmavmc/vms/ and /plasmavmc/handles/ prefixes
|
|
- StateEvent enum: VmUpdated, VmDeleted, HandleUpdated, HandleDeleted
|
|
- StateSynchronizer: Applies watch events to local state via StateSink trait
|
|
- WatcherConfig: Configurable endpoint and buffer size
|
|
- Exported WatchEvent and EventType from chainfire-client
|
|
|
|
Integration pattern:
|
|
- Create (StateWatcher, event_rx) = StateWatcher::new(config)
|
|
- watcher.start().await to spawn watch tasks
|
|
- StateSynchronizer processes events via StateSink trait
|
|
|
|
evidence:
|
|
- item: S2 Hotplug Support
|
|
desc: |
|
|
Implemented QMP-based disk and NIC hotplug for PlasmaVMC:
|
|
|
|
KVM Backend (plasmavmc-kvm/src/lib.rs):
|
|
- attach_disk (lines 346-399): Two-step QMP process
|
|
* blockdev-add: Adds block device backend (qcow2 driver)
|
|
* device_add: Adds virtio-blk-pci frontend
|
|
* Resolves image_id/volume_id to filesystem paths
|
|
- detach_disk (lines 401-426): device_del command removes device
|
|
- attach_nic (lines 428-474): Two-step QMP process
|
|
* netdev_add: Adds TAP network backend
|
|
* device_add: Adds virtio-net-pci frontend with MAC
|
|
- detach_nic (lines 476-501): device_del command removes device
|
|
|
|
Service Layer (plasmavmc-server/src/vm_service.rs):
|
|
- attach_disk (lines 959-992): Validates VM, converts proto, calls backend
|
|
- detach_disk (lines 994-1024): Validates VM, calls backend with disk_id
|
|
- attach_nic (lines 1026-1059): Validates VM, converts proto, calls backend
|
|
- detach_nic (lines 1061-1091): Validates VM, calls backend with nic_id
|
|
- Helper functions:
|
|
* proto_disk_to_types (lines 206-221): Converts proto DiskSpec to domain type
|
|
* proto_nic_to_types (lines 223-234): Converts proto NetworkSpec to domain type
|
|
|
|
Verification:
|
|
- cargo check --package plasmavmc-server: Passed in 2.48s
|
|
- All 4 methods implemented (attach/detach for disk/NIC)
|
|
- Uses QMP blockdev-add/device_add/device_del commands
|
|
- Properly validates VM handle and hypervisor backend
|
|
files:
|
|
- plasmavmc/crates/plasmavmc-kvm/src/lib.rs
|
|
- plasmavmc/crates/plasmavmc-server/src/vm_service.rs
|
|
timestamp: 2025-12-12 18:50 JST
|
|
|
|
- item: S1 VM Lifecycle Ops
|
|
desc: |
|
|
Implemented VM Update and Reset APIs in PlasmaVMC:
|
|
|
|
Files modified:
|
|
- plasmavmc/crates/plasmavmc-server/src/vm_service.rs
|
|
|
|
Changes:
|
|
- reset_vm (lines 886-917): Hard reset via QMP system_reset command
|
|
* Loads VM and handle
|
|
* Calls backend.reboot() which issues QMP system_reset
|
|
* Updates VM status and persists state
|
|
* Returns updated VM proto
|
|
|
|
- update_vm (lines 738-792): Update VM spec, metadata, labels
|
|
* Validates VM exists
|
|
* Updates CPU/RAM spec using proto_spec_to_types()
|
|
* Updates metadata and labels if provided
|
|
* Retrieves current status before persisting (fixes borrow checker)
|
|
* Persists updated VM to storage
|
|
* Changes take effect on next boot (documented in log)
|
|
|
|
Verification: cargo check --package plasmavmc-server succeeded in 1.21s (warnings only, unrelated to changes)
|
|
files:
|
|
- plasmavmc/crates/plasmavmc-server/src/vm_service.rs
|
|
timestamp: 2025-12-12 18:00 JST
|
|
|
|
- item: S3 ChainFire Watch
|
|
desc: |
|
|
Implemented ChainFire state watcher for multi-node PlasmaVMC coordination:
|
|
|
|
Files created:
|
|
- plasmavmc/crates/plasmavmc-server/src/watcher.rs (280+ lines)
|
|
|
|
Files modified:
|
|
- plasmavmc/crates/plasmavmc-server/src/lib.rs - Added watcher module
|
|
- chainfire/chainfire-client/src/lib.rs - Exported WatchEvent, EventType
|
|
|
|
Components:
|
|
- StateWatcher: Spawns background tasks watching ChainFire prefixes
|
|
- StateEvent: Enum for VM/Handle update/delete events
|
|
- StateSynchronizer: Generic event processor with StateSink trait
|
|
- WatcherError: Error types for connection, watch, key parsing
|
|
|
|
Key features:
|
|
- Watches /plasmavmc/vms/ for VM changes
|
|
- Watches /plasmavmc/handles/ for handle changes
|
|
- Parses key format to extract org_id, project_id, vm_id
|
|
- Deserializes VirtualMachine and VmHandle from JSON values
|
|
- Dispatches events to StateSink implementation
|
|
|
|
Verification: cargo check --package plasmavmc-server succeeded (warnings only)
|
|
files:
|
|
- plasmavmc/crates/plasmavmc-server/src/watcher.rs
|
|
- plasmavmc/crates/plasmavmc-server/src/lib.rs
|
|
- chainfire/chainfire-client/src/lib.rs
|
|
timestamp: 2025-12-12 18:15 JST
|
|
notes: |
|
|
Depends on QMP capability of the underlying hypervisor (KVM/QEMU).
|