- Created T026-practical-test task.yaml for MVP smoke testing - Added k8shost-server to flake.nix (packages, apps, overlays) - Staged all workspace directories for nix flake build - Updated flake.nix shellHook to include k8shost Resolves: T026.S1 blocker (R8 - nix submodule visibility)
619 lines
18 KiB
Markdown
619 lines
18 KiB
Markdown
# PlasmaVMC Integration Design
|
||
|
||
**Date:** 2025-12-08
|
||
**Task:** T015 S4
|
||
**Status:** Design Complete
|
||
|
||
## 1. Overview
|
||
|
||
PlasmaVMC VmServiceとOverlay Network Serviceの統合設計。VM作成時にネットワークポートを自動的に作成・アタッチし、IPアドレス割り当てとセキュリティグループ適用を行う。
|
||
|
||
## 2. Integration Architecture
|
||
|
||
### 2.1 Service Dependencies
|
||
|
||
```
|
||
VmService (plasmavmc-server)
|
||
│
|
||
├──→ NetworkService (overlay-network-server)
|
||
│ ├──→ ChainFire (network state)
|
||
│ └──→ OVN (logical network)
|
||
│
|
||
└──→ HypervisorBackend (KVM/FireCracker)
|
||
└──→ OVN Controller (via OVS)
|
||
└──→ VM TAP Interface
|
||
```
|
||
|
||
### 2.2 Integration Flow
|
||
|
||
```
|
||
1. User → VmService.create_vm(NetworkSpec)
|
||
2. VmService → NetworkService.create_port()
|
||
└── Creates OVN Logical Port
|
||
└── Allocates IP (DHCP or static)
|
||
└── Applies security groups
|
||
3. VmService → HypervisorBackend.create()
|
||
└── Creates VM with TAP interface
|
||
└── Attaches TAP to OVN port
|
||
4. OVN → Updates network state
|
||
└── Port appears in Logical Switch
|
||
└── DHCP server ready
|
||
```
|
||
|
||
## 3. VmConfig Network Schema Extension
|
||
|
||
### 3.1 Current NetworkSpec
|
||
|
||
既存の`NetworkSpec`は以下のフィールドを持っています:
|
||
|
||
```rust
|
||
pub struct NetworkSpec {
|
||
pub id: String,
|
||
pub network_id: String, // Currently: "default" or user-specified
|
||
pub mac_address: Option<String>,
|
||
pub ip_address: Option<String>,
|
||
pub model: NicModel,
|
||
pub security_groups: Vec<String>,
|
||
}
|
||
```
|
||
|
||
### 3.2 Extended NetworkSpec
|
||
|
||
`network_id`フィールドを拡張して、subnet_idを明示的に指定できるようにします:
|
||
|
||
```rust
|
||
pub struct NetworkSpec {
|
||
/// Interface identifier (unique within VM)
|
||
pub id: String,
|
||
|
||
/// Subnet identifier: "{org_id}/{project_id}/{subnet_name}"
|
||
/// If not specified, uses default subnet for project
|
||
pub subnet_id: Option<String>,
|
||
|
||
/// Legacy network_id field (deprecated, use subnet_id instead)
|
||
/// If subnet_id is None and network_id is set, treated as subnet name
|
||
#[deprecated(note = "Use subnet_id instead")]
|
||
pub network_id: String,
|
||
|
||
/// MAC address (auto-generated if None)
|
||
pub mac_address: Option<String>,
|
||
|
||
/// IP address (DHCP if None, static if Some)
|
||
pub ip_address: Option<String>,
|
||
|
||
/// NIC model (virtio-net, e1000, etc.)
|
||
pub model: NicModel,
|
||
|
||
/// Security group IDs: ["{org_id}/{project_id}/{sg_name}", ...]
|
||
/// If empty, uses default security group
|
||
pub security_groups: Vec<String>,
|
||
}
|
||
```
|
||
|
||
### 3.3 Migration Strategy
|
||
|
||
**Phase 1: Backward Compatibility**
|
||
- `network_id`が設定されている場合、`subnet_id`に変換
|
||
- `network_id = "default"` → `subnet_id = "{org_id}/{project_id}/default"`
|
||
- `network_id = "{subnet_name}"` → `subnet_id = "{org_id}/{project_id}/{subnet_name}"`
|
||
|
||
**Phase 2: Deprecation**
|
||
- `network_id`フィールドを非推奨としてマーク
|
||
- 新規VM作成では`subnet_id`を使用
|
||
|
||
**Phase 3: Removal**
|
||
- `network_id`フィールドを削除(将来のバージョン)
|
||
|
||
## 4. VM Creation Integration
|
||
|
||
### 4.1 VmService.create_vm() Flow
|
||
|
||
```rust
|
||
impl VmService {
|
||
async fn create_vm(&self, request: CreateVmRequest) -> Result<VirtualMachine> {
|
||
let req = request.into_inner();
|
||
|
||
// 1. Validate network specs
|
||
for net_spec in &req.spec.network {
|
||
self.validate_network_spec(&req.org_id, &req.project_id, net_spec)?;
|
||
}
|
||
|
||
// 2. Create VM record
|
||
let mut vm = VirtualMachine::new(
|
||
req.name,
|
||
&req.org_id,
|
||
&req.project_id,
|
||
Self::proto_spec_to_types(req.spec),
|
||
);
|
||
|
||
// 3. Create network ports
|
||
let mut ports = Vec::new();
|
||
for net_spec in &vm.spec.network {
|
||
let port = self.network_service
|
||
.create_port(CreatePortRequest {
|
||
org_id: vm.org_id.clone(),
|
||
project_id: vm.project_id.clone(),
|
||
subnet_id: self.resolve_subnet_id(
|
||
&vm.org_id,
|
||
&vm.project_id,
|
||
&net_spec.subnet_id,
|
||
)?,
|
||
vm_id: vm.id.to_string(),
|
||
mac_address: net_spec.mac_address.clone(),
|
||
ip_address: net_spec.ip_address.clone(),
|
||
security_group_ids: if net_spec.security_groups.is_empty() {
|
||
vec!["default".to_string()]
|
||
} else {
|
||
net_spec.security_groups.clone()
|
||
},
|
||
})
|
||
.await?;
|
||
ports.push(port);
|
||
}
|
||
|
||
// 4. Create VM via hypervisor backend
|
||
let handle = self.hypervisor_backend
|
||
.create(&vm)
|
||
.await?;
|
||
|
||
// 5. Attach network ports to VM
|
||
for (net_spec, port) in vm.spec.network.iter().zip(ports.iter()) {
|
||
self.attach_port_to_vm(port, &handle, net_spec).await?;
|
||
}
|
||
|
||
// 6. Persist VM and ports
|
||
self.store.save_vm(&vm).await?;
|
||
for port in &ports {
|
||
self.network_service.save_port(port).await?;
|
||
}
|
||
|
||
Ok(vm)
|
||
}
|
||
|
||
fn resolve_subnet_id(
|
||
&self,
|
||
org_id: &str,
|
||
project_id: &str,
|
||
subnet_id: Option<&String>,
|
||
) -> Result<String> {
|
||
match subnet_id {
|
||
Some(id) if id.starts_with(&format!("{}/{}", org_id, project_id)) => {
|
||
Ok(id.clone())
|
||
}
|
||
Some(name) => {
|
||
// Treat as subnet name
|
||
Ok(format!("{}/{}/{}", org_id, project_id, name))
|
||
}
|
||
None => {
|
||
// Use default subnet
|
||
Ok(format!("{}/{}/default", org_id, project_id))
|
||
}
|
||
}
|
||
}
|
||
|
||
async fn attach_port_to_vm(
|
||
&self,
|
||
port: &Port,
|
||
handle: &VmHandle,
|
||
net_spec: &NetworkSpec,
|
||
) -> Result<()> {
|
||
// 1. Get TAP interface name from OVN port
|
||
let tap_name = self.network_service
|
||
.get_port_tap_name(&port.id)
|
||
.await?;
|
||
|
||
// 2. Attach TAP to VM via hypervisor backend
|
||
match vm.hypervisor {
|
||
HypervisorType::Kvm => {
|
||
// QEMU: Use -netdev tap with TAP interface
|
||
self.kvm_backend.attach_nic(handle, &NetworkSpec {
|
||
id: net_spec.id.clone(),
|
||
network_id: port.subnet_id.clone(),
|
||
mac_address: Some(port.mac_address.clone()),
|
||
ip_address: port.ip_address.clone(),
|
||
model: net_spec.model,
|
||
security_groups: port.security_group_ids.clone(),
|
||
}).await?;
|
||
}
|
||
HypervisorType::Firecracker => {
|
||
// FireCracker: Use TAP interface in network config
|
||
self.firecracker_backend.attach_nic(handle, &NetworkSpec {
|
||
id: net_spec.id.clone(),
|
||
network_id: port.subnet_id.clone(),
|
||
mac_address: Some(port.mac_address.clone()),
|
||
ip_address: port.ip_address.clone(),
|
||
model: net_spec.model,
|
||
security_groups: port.security_group_ids.clone(),
|
||
}).await?;
|
||
}
|
||
_ => {
|
||
return Err(Error::Unsupported("Hypervisor not supported".into()));
|
||
}
|
||
}
|
||
|
||
Ok(())
|
||
}
|
||
}
|
||
```
|
||
|
||
### 4.2 NetworkService Integration Points
|
||
|
||
**Required Methods:**
|
||
```rust
|
||
pub trait NetworkServiceClient: Send + Sync {
|
||
/// Create a port for VM network interface
|
||
async fn create_port(&self, req: CreatePortRequest) -> Result<Port>;
|
||
|
||
/// Get port details
|
||
async fn get_port(&self, org_id: &str, project_id: &str, port_id: &str) -> Result<Option<Port>>;
|
||
|
||
/// Get TAP interface name for port
|
||
async fn get_port_tap_name(&self, port_id: &str) -> Result<String>;
|
||
|
||
/// Delete port
|
||
async fn delete_port(&self, org_id: &str, project_id: &str, port_id: &str) -> Result<()>;
|
||
|
||
/// Ensure VPC and default subnet exist for project
|
||
async fn ensure_project_network(&self, org_id: &str, project_id: &str) -> Result<()>;
|
||
}
|
||
```
|
||
|
||
## 5. Port Creation Details
|
||
|
||
### 5.1 Port Creation Flow
|
||
|
||
```
|
||
1. VmService.create_vm() called with NetworkSpec
|
||
└── subnet_id: "{org_id}/{project_id}/{subnet_name}" or None (default)
|
||
|
||
2. NetworkService.create_port() called
|
||
├── Resolve subnet_id (use default if None)
|
||
├── Ensure VPC and subnet exist (create if not)
|
||
├── Create OVN Logical Port
|
||
│ └── ovn-nbctl lsp-add <logical_switch> <port_name>
|
||
├── Set port options (MAC, IP if static)
|
||
│ └── ovn-nbctl lsp-set-addresses <port> <mac> <ip>
|
||
├── Apply security groups (OVN ACLs)
|
||
│ └── ovn-nbctl acl-add <switch> <direction> <priority> <match> <action>
|
||
├── Allocate IP address (if static)
|
||
│ └── Update ChainFire IPAM state
|
||
└── Return Port object
|
||
|
||
3. HypervisorBackend.create() called
|
||
└── Creates VM with network interface
|
||
|
||
4. Attach port to VM
|
||
├── Get TAP interface name from OVN
|
||
├── Create TAP interface (if not exists)
|
||
├── Bind TAP to OVN port
|
||
│ └── ovs-vsctl add-port <bridge> <tap> -- set Interface <tap> type=internal
|
||
└── Attach TAP to VM NIC
|
||
```
|
||
|
||
### 5.2 Default Subnet Creation
|
||
|
||
プロジェクトのデフォルトサブネットが存在しない場合、自動作成:
|
||
|
||
```rust
|
||
async fn ensure_project_network(
|
||
&self,
|
||
org_id: &str,
|
||
project_id: &str,
|
||
) -> Result<()> {
|
||
// Check if VPC exists
|
||
let vpc_id = format!("{}/{}", org_id, project_id);
|
||
if self.get_vpc(org_id, project_id).await?.is_none() {
|
||
// Create VPC with auto-allocated CIDR
|
||
self.create_vpc(CreateVpcRequest {
|
||
org_id: org_id.to_string(),
|
||
project_id: project_id.to_string(),
|
||
name: "default".to_string(),
|
||
cidr: None, // Auto-allocate
|
||
}).await?;
|
||
}
|
||
|
||
// Check if default subnet exists
|
||
let subnet_id = format!("{}/{}/default", org_id, project_id);
|
||
if self.get_subnet(org_id, project_id, "default").await?.is_none() {
|
||
// Get VPC CIDR
|
||
let vpc = self.get_vpc(org_id, project_id).await?.unwrap();
|
||
let vpc_cidr: IpNet = vpc.cidr.parse()?;
|
||
|
||
// Create default subnet: first /24 in VPC
|
||
let subnet_cidr = format!("{}.0.0/24", vpc_cidr.network().octets()[1]);
|
||
|
||
self.create_subnet(CreateSubnetRequest {
|
||
org_id: org_id.to_string(),
|
||
project_id: project_id.to_string(),
|
||
vpc_id: vpc_id.clone(),
|
||
name: "default".to_string(),
|
||
cidr: subnet_cidr,
|
||
dhcp_enabled: true,
|
||
dns_servers: vec!["8.8.8.8".to_string(), "8.8.4.4".to_string()],
|
||
}).await?;
|
||
|
||
// Create default security group
|
||
self.create_security_group(CreateSecurityGroupRequest {
|
||
org_id: org_id.to_string(),
|
||
project_id: project_id.to_string(),
|
||
name: "default".to_string(),
|
||
description: "Default security group".to_string(),
|
||
ingress_rules: vec![
|
||
SecurityRule {
|
||
protocol: Protocol::All,
|
||
port_range: None,
|
||
source_type: SourceType::SecurityGroup,
|
||
source: format!("{}/{}/default", org_id, project_id),
|
||
},
|
||
],
|
||
egress_rules: vec![
|
||
SecurityRule {
|
||
protocol: Protocol::All,
|
||
port_range: None,
|
||
source_type: SourceType::Cidr,
|
||
source: "0.0.0.0/0".to_string(),
|
||
},
|
||
],
|
||
}).await?;
|
||
}
|
||
|
||
Ok(())
|
||
}
|
||
```
|
||
|
||
## 6. IP Address Assignment
|
||
|
||
### 6.1 DHCP Assignment (Default)
|
||
|
||
```rust
|
||
// Port creation with DHCP
|
||
let port = network_service.create_port(CreatePortRequest {
|
||
subnet_id: subnet_id.clone(),
|
||
vm_id: vm_id.clone(),
|
||
ip_address: None, // DHCP
|
||
// ...
|
||
}).await?;
|
||
|
||
// IP will be assigned by OVN DHCP server
|
||
// Port.ip_address will be None until DHCP lease is obtained
|
||
// VmService should poll or wait for IP assignment
|
||
```
|
||
|
||
### 6.2 Static Assignment
|
||
|
||
```rust
|
||
// Port creation with static IP
|
||
let port = network_service.create_port(CreatePortRequest {
|
||
subnet_id: subnet_id.clone(),
|
||
vm_id: vm_id.clone(),
|
||
ip_address: Some("10.1.0.10".to_string()), // Static
|
||
// ...
|
||
}).await?;
|
||
|
||
// IP is allocated immediately
|
||
// Port.ip_address will be Some("10.1.0.10")
|
||
```
|
||
|
||
### 6.3 IP Assignment Tracking
|
||
|
||
```rust
|
||
// Update VM status with assigned IPs
|
||
vm.status.ip_addresses = ports
|
||
.iter()
|
||
.filter_map(|p| p.ip_address.clone())
|
||
.collect();
|
||
|
||
// Persist updated VM status
|
||
store.save_vm(&vm).await?;
|
||
```
|
||
|
||
## 7. Security Group Binding
|
||
|
||
### 7.1 Security Group Resolution
|
||
|
||
```rust
|
||
fn resolve_security_groups(
|
||
org_id: &str,
|
||
project_id: &str,
|
||
security_groups: &[String],
|
||
) -> Vec<String> {
|
||
if security_groups.is_empty() {
|
||
// Use default security group
|
||
vec![format!("{}/{}/default", org_id, project_id)]
|
||
} else {
|
||
// Resolve security group IDs
|
||
security_groups
|
||
.iter()
|
||
.map(|sg| {
|
||
if sg.contains('/') {
|
||
// Full ID: "{org_id}/{project_id}/{sg_name}"
|
||
sg.clone()
|
||
} else {
|
||
// Name only: "{sg_name}"
|
||
format!("{}/{}/{}", org_id, project_id, sg)
|
||
}
|
||
})
|
||
.collect()
|
||
}
|
||
}
|
||
```
|
||
|
||
### 7.2 OVN ACL Application
|
||
|
||
```rust
|
||
async fn apply_security_groups(
|
||
&self,
|
||
port: &Port,
|
||
security_groups: &[String],
|
||
) -> Result<()> {
|
||
for sg_id in security_groups {
|
||
let sg = self.get_security_group_by_id(sg_id).await?;
|
||
|
||
// Apply ingress rules
|
||
for rule in &sg.ingress_rules {
|
||
let acl_match = build_acl_match(rule, &sg.id)?;
|
||
ovn_nbctl.acl_add(
|
||
&port.subnet_id,
|
||
"to-lport",
|
||
1000,
|
||
&acl_match,
|
||
"allow-related",
|
||
).await?;
|
||
}
|
||
|
||
// Apply egress rules
|
||
for rule in &sg.egress_rules {
|
||
let acl_match = build_acl_match(rule, &sg.id)?;
|
||
ovn_nbctl.acl_add(
|
||
&port.subnet_id,
|
||
"from-lport",
|
||
1000,
|
||
&acl_match,
|
||
"allow-related",
|
||
).await?;
|
||
}
|
||
}
|
||
|
||
Ok(())
|
||
}
|
||
```
|
||
|
||
## 8. VM Deletion Integration
|
||
|
||
### 8.1 Port Cleanup
|
||
|
||
```rust
|
||
impl VmService {
|
||
async fn delete_vm(&self, request: DeleteVmRequest) -> Result<()> {
|
||
let req = request.into_inner();
|
||
|
||
// 1. Get VM and ports
|
||
let vm = self.get_vm(&req.org_id, &req.project_id, &req.vm_id).await?;
|
||
let ports = self.network_service
|
||
.list_ports(&req.org_id, &req.project_id, Some(&req.vm_id))
|
||
.await?;
|
||
|
||
// 2. Stop VM if running
|
||
if matches!(vm.state, VmState::Running | VmState::Starting) {
|
||
self.stop_vm(request.clone()).await?;
|
||
}
|
||
|
||
// 3. Delete VM via hypervisor backend
|
||
if let Some(handle) = self.handles.get(&TenantKey::new(
|
||
&req.org_id,
|
||
&req.project_id,
|
||
&req.vm_id,
|
||
)) {
|
||
self.hypervisor_backend.delete(&handle).await?;
|
||
}
|
||
|
||
// 4. Delete network ports
|
||
for port in &ports {
|
||
self.network_service
|
||
.delete_port(&req.org_id, &req.project_id, &port.id)
|
||
.await?;
|
||
}
|
||
|
||
// 5. Delete VM from storage
|
||
self.store.delete_vm(&req.org_id, &req.project_id, &req.vm_id).await?;
|
||
|
||
Ok(())
|
||
}
|
||
}
|
||
```
|
||
|
||
## 9. Error Handling
|
||
|
||
### 9.1 Network Creation Failures
|
||
|
||
```rust
|
||
// If network creation fails, VM creation should fail
|
||
match network_service.create_port(req).await {
|
||
Ok(port) => port,
|
||
Err(NetworkError::SubnetNotFound) => {
|
||
// Try to create default subnet
|
||
network_service.ensure_project_network(org_id, project_id).await?;
|
||
network_service.create_port(req).await?
|
||
}
|
||
Err(e) => return Err(VmError::NetworkError(e)),
|
||
}
|
||
```
|
||
|
||
### 9.2 Port Attachment Failures
|
||
|
||
```rust
|
||
// If port attachment fails, clean up created port
|
||
match self.attach_port_to_vm(&port, &handle, &net_spec).await {
|
||
Ok(()) => {}
|
||
Err(e) => {
|
||
// Clean up port
|
||
let _ = self.network_service
|
||
.delete_port(&vm.org_id, &vm.project_id, &port.id)
|
||
.await;
|
||
return Err(e);
|
||
}
|
||
}
|
||
```
|
||
|
||
## 10. Configuration
|
||
|
||
### 10.1 VmService Configuration
|
||
|
||
```toml
|
||
[vm_service]
|
||
network_service_endpoint = "http://127.0.0.1:8081"
|
||
network_service_timeout_secs = 30
|
||
|
||
[network]
|
||
auto_create_default_subnet = true
|
||
default_security_group_name = "default"
|
||
```
|
||
|
||
### 10.2 Environment Variables
|
||
|
||
| Variable | Default | Description |
|
||
|----------|---------|-------------|
|
||
| `PLASMAVMC_NETWORK_SERVICE_ENDPOINT` | `http://127.0.0.1:8081` | NetworkService gRPC endpoint |
|
||
| `PLASMAVMC_AUTO_CREATE_NETWORK` | `true` | Auto-create VPC/subnet for project |
|
||
|
||
## 11. Testing Considerations
|
||
|
||
### 11.1 Unit Tests
|
||
|
||
- Mock NetworkService client
|
||
- Test subnet_id resolution
|
||
- Test security group resolution
|
||
- Test port creation flow
|
||
|
||
### 11.2 Integration Tests
|
||
|
||
- Real NetworkService + OVN
|
||
- VM creation with network attachment
|
||
- IP assignment verification
|
||
- Security group enforcement
|
||
|
||
### 11.3 Test Scenarios
|
||
|
||
1. **VM creation with default network**
|
||
- No NetworkSpec → uses default subnet
|
||
- Default security group applied
|
||
|
||
2. **VM creation with custom subnet**
|
||
- NetworkSpec with subnet_id
|
||
- Custom security groups
|
||
|
||
3. **VM creation with static IP**
|
||
- NetworkSpec with ip_address
|
||
- IP allocation verification
|
||
|
||
4. **VM deletion with port cleanup**
|
||
- Ports deleted on VM deletion
|
||
- IP addresses released
|
||
|
||
## 12. Future Enhancements
|
||
|
||
1. **Hot-plug NIC**: Attach/detach network interfaces to running VMs
|
||
2. **Network migration**: Move VM between subnets
|
||
3. **Multi-NIC support**: Multiple network interfaces per VM
|
||
4. **Network QoS**: Bandwidth limits and priority
|
||
5. **Network monitoring**: Traffic statistics per port
|