centra
a7ec7e2158
- Created T026-practical-test task.yaml for MVP smoke testing - Added k8shost-server to flake.nix (packages, apps, overlays) - Staged all workspace directories for nix flake build - Updated flake.nix shellHook to include k8shost Resolves: T026.S1 blocker (R8 - nix submodule visibility)
468 lines
22 KiB
Markdown
468 lines
22 KiB
Markdown
# MVP-Beta Tenant Path Architecture
|
|
|
|
## Overview
|
|
|
|
This document describes the architecture of the PlasmaCloud MVP-Beta tenant path, which enables end-to-end multi-tenant cloud infrastructure provisioning with complete isolation between tenants.
|
|
|
|
The tenant path spans three core components:
|
|
1. **IAM** (Identity and Access Management): User authentication, RBAC, and tenant scoping
|
|
2. **NovaNET**: Network virtualization with VPC overlay and tenant isolation
|
|
3. **PlasmaVMC**: Virtual machine provisioning and lifecycle management
|
|
|
|
## Architecture Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ User / API Client │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
↓ Authentication Request
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ IAM (Identity & Access) │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌────────────────────┐ ┌──────────────────┐ │
|
|
│ │ IamTokenService │────────▶│ IamAuthzService │ │
|
|
│ │ │ │ │ │
|
|
│ │ • Authenticate │ │ • RBAC Eval │ │
|
|
│ │ • Issue JWT Token │ │ • Permission │ │
|
|
│ │ • Scope: org+proj │ │ Check │ │
|
|
│ └────────────────────┘ └──────────────────┘ │
|
|
│ │
|
|
│ Data Stores: │
|
|
│ • PrincipalStore (users, service accounts) │
|
|
│ • RoleStore (system, org, project roles) │
|
|
│ • BindingStore (principal → role assignments) │
|
|
│ │
|
|
│ Tenant Scoping: │
|
|
│ • Principals belong to org_id │
|
|
│ • Tokens include org_id + project_id │
|
|
│ • RBAC enforces resource.org_id == token.org_id │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
↓ JWT Token {org_id, project_id, permissions}
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ API Gateway / Service Layer │
|
|
│ • Validates JWT token │
|
|
│ • Extracts org_id, project_id from token │
|
|
│ • Passes tenant context to downstream services │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌───────────────┴───────────────┐
|
|
↓ ↓
|
|
┌─────────────────────────────────┐ ┌─────────────────────────────────┐
|
|
│ NovaNET │ │ PlasmaVMC │
|
|
│ (Network Virtualization) │ │ (VM Provisioning) │
|
|
├─────────────────────────────────┤ ├─────────────────────────────────┤
|
|
│ │ │ │
|
|
│ ┌────────────────────────┐ │ │ ┌────────────────────────┐ │
|
|
│ │ VpcServiceImpl │ │ │ │ VmServiceImpl │ │
|
|
│ │ • Create VPC │ │ │ │ • Create VM │ │
|
|
│ │ • Scope: org_id │ │ │ │ • Scope: org_id, │ │
|
|
│ │ • VPC ID generation │ │ │ │ project_id │ │
|
|
│ └────────────────────────┘ │ │ │ • Network attach │ │
|
|
│ ↓ │ │ └────────────────────────┘ │
|
|
│ ┌────────────────────────┐ │ │ │ │
|
|
│ │ SubnetServiceImpl │ │ │ │ │
|
|
│ │ • Create Subnet │ │ │ ┌────────────────────────┐ │
|
|
│ │ • CIDR allocation │ │ │ │ NetworkAttachment │ │
|
|
│ │ • DHCP config │ │ │ │ • Attach port to VM │ │
|
|
│ │ • Gateway config │ │ │ │ • Update port.device │ │
|
|
│ └────────────────────────┘ │ │ │ • TAP interface │ │
|
|
│ ↓ │ │ └────────────────────────┘ │
|
|
│ ┌────────────────────────┐ │ │ ↑ │
|
|
│ │ PortServiceImpl │◀────┼───┼──────────────┘ │
|
|
│ │ • Create Port │ │ │ port_id in NetworkSpec │
|
|
│ │ • IP allocation │ │ │ │
|
|
│ │ • MAC generation │ │ │ Hypervisor: │
|
|
│ │ • Port status │ │ │ • KvmBackend │
|
|
│ │ • device_id tracking │ │ │ • FirecrackerBackend │
|
|
│ └────────────────────────┘ │ │ │
|
|
│ │ │ Storage: │
|
|
│ Metadata Store: │ │ • NetworkMetadataStore │
|
|
│ • NetworkMetadataStore │ │ • ChainFire (planned) │
|
|
│ • In-memory (dev) │ │ │
|
|
│ • FlareDB (production) │ └─────────────────────────────────┘
|
|
│ │
|
|
│ Data Plane (OVN): │
|
|
│ • Logical switches per VPC │
|
|
│ • Logical routers per subnet │
|
|
│ • Security groups │
|
|
│ • DHCP server │
|
|
│ │
|
|
└─────────────────────────────────┘
|
|
```
|
|
|
|
## Component Boundaries
|
|
|
|
### IAM: Tenant Isolation + RBAC Enforcement
|
|
|
|
**Responsibilities**:
|
|
- User authentication and token issuance
|
|
- Organization and project hierarchy management
|
|
- Role-based access control (RBAC) enforcement
|
|
- Cross-tenant access denial
|
|
|
|
**Tenant Scoping**:
|
|
- Each `Principal` (user/service account) belongs to an `org_id`
|
|
- Tokens include both `org_id` and `project_id` claims
|
|
- Resources are scoped as: `org/{org_id}/project/{project_id}/{resource_type}/{id}`
|
|
|
|
**Key Types**:
|
|
```rust
|
|
struct Principal {
|
|
id: String,
|
|
org_id: Option<String>, // Primary tenant boundary
|
|
project_id: Option<String>, // Sub-tenant boundary
|
|
// ...
|
|
}
|
|
|
|
struct Scope {
|
|
System, // Global access
|
|
Org(String), // Organization-level
|
|
Project { org, project }, // Project-level
|
|
}
|
|
|
|
struct Permission {
|
|
action: String, // e.g., "compute:instances:create"
|
|
resource_pattern: String, // e.g., "org/acme-corp/project/*/instance/*"
|
|
conditions: Vec<Condition>, // e.g., resource.owner == principal.id
|
|
}
|
|
```
|
|
|
|
**Integration Points**:
|
|
- Issues JWT tokens consumed by all services
|
|
- Validates authorization before resource creation
|
|
- Enforces `resource.org_id == token.org_id` at policy evaluation time
|
|
|
|
### NovaNET: Network Isolation per Tenant VPC
|
|
|
|
**Responsibilities**:
|
|
- VPC (Virtual Private Cloud) provisioning
|
|
- Subnet management with CIDR allocation
|
|
- Port creation and IP/MAC assignment
|
|
- Security group enforcement
|
|
- Port lifecycle management (attach/detach)
|
|
|
|
**Tenant Scoping**:
|
|
- Each VPC is scoped to an `org_id`
|
|
- VPC provides network isolation boundary
|
|
- Subnets and ports inherit VPC tenant scope
|
|
- Port device tracking links to VM IDs
|
|
|
|
**Key Types**:
|
|
```rust
|
|
struct Vpc {
|
|
id: String,
|
|
org_id: String, // Tenant boundary
|
|
project_id: String,
|
|
cidr: String, // e.g., "10.0.0.0/16"
|
|
// ...
|
|
}
|
|
|
|
struct Subnet {
|
|
id: String,
|
|
vpc_id: String, // Parent VPC (inherits tenant)
|
|
cidr: String, // e.g., "10.0.1.0/24"
|
|
gateway: String,
|
|
dhcp_enabled: bool,
|
|
// ...
|
|
}
|
|
|
|
struct Port {
|
|
id: String,
|
|
subnet_id: String, // Parent subnet (inherits tenant)
|
|
ip_address: String,
|
|
mac_address: String,
|
|
device_id: String, // VM ID when attached
|
|
device_type: DeviceType, // Vm, LoadBalancer, etc.
|
|
// ...
|
|
}
|
|
```
|
|
|
|
**Integration Points**:
|
|
- Accepts org_id/project_id from API tokens
|
|
- Provides port IDs to PlasmaVMC for VM attachment
|
|
- Receives port attachment/detachment events from PlasmaVMC
|
|
- Uses OVN (Open Virtual Network) for overlay networking data plane
|
|
|
|
### PlasmaVMC: VM Scoping by org_id/project_id
|
|
|
|
**Responsibilities**:
|
|
- Virtual machine lifecycle management (create, start, stop, delete)
|
|
- Hypervisor abstraction (KVM, Firecracker)
|
|
- Network interface attachment to NovaNET ports
|
|
- VM metadata persistence (ChainFire)
|
|
|
|
**Tenant Scoping**:
|
|
- Each VM belongs to an `org_id` and `project_id`
|
|
- VM metadata includes tenant identifiers
|
|
- Network attachments validated against tenant scope
|
|
|
|
**Key Types**:
|
|
```rust
|
|
struct Vm {
|
|
id: String,
|
|
name: String,
|
|
org_id: String, // Tenant boundary
|
|
project_id: String,
|
|
spec: VmSpec,
|
|
state: VmState,
|
|
// ...
|
|
}
|
|
|
|
struct NetworkSpec {
|
|
id: String, // Interface name (e.g., "eth0")
|
|
network_id: String, // VPC ID from NovaNET
|
|
subnet_id: String, // Subnet ID from NovaNET
|
|
port_id: String, // Port ID from NovaNET
|
|
mac_address: String,
|
|
ip_address: String,
|
|
// ...
|
|
}
|
|
```
|
|
|
|
**Integration Points**:
|
|
- Accepts org_id/project_id from API tokens
|
|
- Fetches port details from NovaNET using port_id
|
|
- Notifies NovaNET when VM is created (port attach)
|
|
- Notifies NovaNET when VM is deleted (port detach)
|
|
- Uses hypervisor backends (KVM, Firecracker) for VM execution
|
|
|
|
## Data Flow: Complete Tenant Path
|
|
|
|
### Scenario: User Creates VM with Network
|
|
|
|
```
|
|
Step 1: User Authentication
|
|
──────────────────────────────────────────────────────────────
|
|
User IAM
|
|
│ │
|
|
├──── Login ──────────▶│
|
|
│ ├─ Validate credentials
|
|
│ ├─ Lookup Principal (org_id="acme")
|
|
│ ├─ Generate JWT token
|
|
│◀─── JWT Token ───────┤ {org_id: "acme", project_id: "proj-1"}
|
|
│ │
|
|
|
|
|
|
Step 2: Create Network Resources
|
|
──────────────────────────────────────────────────────────────
|
|
User NovaNET
|
|
│ │
|
|
├── CreateVPC ────────▶│ (JWT token in headers)
|
|
│ {org: acme, ├─ Validate token
|
|
│ project: proj-1, ├─ Extract org_id="acme"
|
|
│ cidr: 10.0.0.0/16} ├─ Create VPC(id="vpc-123", org="acme")
|
|
│◀─── VPC ─────────────┤ {id: "vpc-123"}
|
|
│ │
|
|
├── CreateSubnet ─────▶│
|
|
│ {vpc: vpc-123, ├─ Validate VPC belongs to token.org_id
|
|
│ cidr: 10.0.1.0/24} ├─ Create Subnet(id="sub-456")
|
|
│◀─── Subnet ──────────┤ {id: "sub-456"}
|
|
│ │
|
|
├── CreatePort ───────▶│
|
|
│ {subnet: sub-456, ├─ Allocate IP: 10.0.1.10
|
|
│ ip: 10.0.1.10} ├─ Generate MAC: fa:16:3e:...
|
|
│◀─── Port ────────────┤ {id: "port-789", device_id: ""}
|
|
│ │
|
|
|
|
|
|
Step 3: Create VM with Network Attachment
|
|
──────────────────────────────────────────────────────────────
|
|
User PlasmaVMC NovaNET
|
|
│ │ │
|
|
├─ CreateVM ──────▶│ (JWT token) │
|
|
│ {name: "web-1", ├─ Validate token │
|
|
│ network: [ ├─ Extract org/project │
|
|
│ {port_id: │ │
|
|
│ "port-789"} ├─ GetPort ─────────────▶│
|
|
│ ]} │ ├─ Verify port.subnet.vpc.org_id
|
|
│ │ │ == token.org_id
|
|
│ │◀─── Port ──────────────┤ {ip: 10.0.1.10, mac: fa:...}
|
|
│ │ │
|
|
│ ├─ Create VM │
|
|
│ ├─ Attach network: │
|
|
│ │ TAP device → port │
|
|
│ │ │
|
|
│ ├─ AttachPort ──────────▶│
|
|
│ │ {device_id: "vm-001"}│
|
|
│ │ ├─ Update port.device_id="vm-001"
|
|
│ │ ├─ Update port.device_type=Vm
|
|
│ │◀─── Success ───────────┤
|
|
│ │ │
|
|
│◀─── VM ──────────┤ {id: "vm-001", state: "running"}
|
|
│ │
|
|
|
|
|
|
Step 4: Cross-Tenant Access Denied
|
|
──────────────────────────────────────────────────────────────
|
|
User B PlasmaVMC IAM
|
|
(org: "other") │ │
|
|
│ │ │
|
|
├─ GetVM ────────▶│ (JWT token: org="other")
|
|
│ {vm_id: ├─ Authorize ─────────▶│
|
|
│ "vm-001"} │ {action: "vm:read", ├─ Evaluate RBAC
|
|
│ │ resource: "org/acme/..."}
|
|
│ │ ├─ Check resource.org_id="acme"
|
|
│ │ ├─ Check token.org_id="other"
|
|
│ │ ├─ DENY: org mismatch
|
|
│ │◀─── Deny ────────────┤
|
|
│◀── 403 Forbidden ┤
|
|
│ │
|
|
```
|
|
|
|
## Tenant Isolation Mechanisms
|
|
|
|
### Layer 1: IAM Policy Enforcement
|
|
|
|
**Mechanism**: Resource path matching with org_id validation
|
|
|
|
**Example**:
|
|
```
|
|
Resource: org/acme-corp/project/proj-1/instance/vm-001
|
|
Token: {org_id: "acme-corp", project_id: "proj-1"}
|
|
Policy: Permission {action: "compute:*", resource: "org/acme-corp/*"}
|
|
|
|
Result: ALLOW (org_id matches)
|
|
```
|
|
|
|
**Cross-Tenant Denial**:
|
|
```
|
|
Resource: org/acme-corp/project/proj-1/instance/vm-001
|
|
Token: {org_id: "other-corp", project_id: "proj-2"}
|
|
|
|
Result: DENY (org_id mismatch)
|
|
```
|
|
|
|
### Layer 2: Network VPC Isolation
|
|
|
|
**Mechanism**: VPC provides logical network boundary
|
|
|
|
- Each VPC has a unique overlay network (OVN logical switch)
|
|
- Subnets within VPC can communicate
|
|
- Cross-VPC traffic requires explicit routing (not implemented in MVP-Beta)
|
|
- VPC membership enforced by org_id
|
|
|
|
**Isolation Properties**:
|
|
- Tenant A's VPC (10.0.0.0/16) is isolated from Tenant B's VPC (10.0.0.0/16)
|
|
- Even with overlapping CIDRs, VPCs are completely isolated
|
|
- MAC addresses are unique per VPC (no collision)
|
|
|
|
### Layer 3: VM Scoping
|
|
|
|
**Mechanism**: VMs are scoped to org_id and project_id
|
|
|
|
- VM metadata includes org_id and project_id
|
|
- VM list operations filter by token.org_id
|
|
- VM operations validated against token scope
|
|
- Network attachments validated against VPC tenant scope
|
|
|
|
## Service Communication
|
|
|
|
### gRPC APIs
|
|
|
|
All inter-service communication uses gRPC with Protocol Buffers:
|
|
|
|
```
|
|
IAM: :50080 (IamAdminService, IamAuthzService)
|
|
NovaNET: :50081 (VpcService, SubnetService, PortService, SecurityGroupService)
|
|
PlasmaVMC: :50082 (VmService)
|
|
FlashDNS: :50083 (DnsService) [Future]
|
|
FiberLB: :50084 (LoadBalancerService) [Future]
|
|
LightningStor: :50085 (StorageService) [Future]
|
|
```
|
|
|
|
### Environment Configuration
|
|
|
|
Services discover each other via environment variables:
|
|
|
|
```bash
|
|
# PlasmaVMC configuration
|
|
NOVANET_ENDPOINT=http://novanet:50081
|
|
IAM_ENDPOINT=http://iam:50080
|
|
|
|
# NovaNET configuration
|
|
IAM_ENDPOINT=http://iam:50080
|
|
FLAREDB_ENDPOINT=http://flaredb:50090 # Metadata persistence
|
|
```
|
|
|
|
## Metadata Persistence
|
|
|
|
### Development: In-Memory Stores
|
|
|
|
```rust
|
|
// NetworkMetadataStore (NovaNET)
|
|
let store = NetworkMetadataStore::new_in_memory();
|
|
|
|
// Backend (IAM)
|
|
let backend = Backend::memory();
|
|
```
|
|
|
|
### Production: FlareDB
|
|
|
|
```
|
|
IAM: PrincipalStore, RoleStore, BindingStore → FlareDB
|
|
NovaNET: NetworkMetadataStore → FlareDB
|
|
PlasmaVMC: VmMetadata → ChainFire (immutable log) + FlareDB (mutable state)
|
|
```
|
|
|
|
## Future Extensions (Post MVP-Beta)
|
|
|
|
### S3: FlashDNS Integration
|
|
|
|
```
|
|
User creates VM → PlasmaVMC creates DNS record in tenant zone
|
|
VM hostname: web-1.proj-1.acme-corp.cloud.internal
|
|
DNS resolution within VPC
|
|
```
|
|
|
|
### S4: FiberLB Integration
|
|
|
|
```
|
|
User creates LoadBalancer → FiberLB provisions LB in tenant VPC
|
|
LB backend pool: [vm-1, vm-2, vm-3] (all in same project)
|
|
LB VIP: 10.0.1.100 (allocated from subnet)
|
|
```
|
|
|
|
### S5: LightningStor Integration
|
|
|
|
```
|
|
User creates Volume → LightningStor allocates block device
|
|
Volume attachment to VM → PlasmaVMC attaches virtio-blk
|
|
Snapshot management → LightningStor + ChainFire
|
|
```
|
|
|
|
## Testing & Validation
|
|
|
|
**Integration Tests**: 8 tests validating complete E2E flow
|
|
|
|
| Test Suite | Location | Tests | Coverage |
|
|
|------------|----------|-------|----------|
|
|
| IAM Tenant Path | iam/.../tenant_path_integration.rs | 6 | Auth, RBAC, isolation |
|
|
| Network + VM | plasmavmc/.../novanet_integration.rs | 2 | VPC lifecycle, VM attach |
|
|
|
|
**Key Validations**:
|
|
- ✅ User authentication and token issuance
|
|
- ✅ Organization and project scoping
|
|
- ✅ RBAC policy evaluation
|
|
- ✅ Cross-tenant access denial
|
|
- ✅ VPC, subnet, and port creation
|
|
- ✅ Port attachment to VMs
|
|
- ✅ Port detachment on VM deletion
|
|
- ✅ Tenant-isolated networking
|
|
|
|
See [E2E Test Documentation](../por/T023-e2e-tenant-path/e2e_test.md) for detailed test descriptions.
|
|
|
|
## Conclusion
|
|
|
|
The MVP-Beta tenant path provides a complete, production-ready foundation for multi-tenant cloud infrastructure:
|
|
|
|
- **Strong tenant isolation** at IAM, network, and compute layers
|
|
- **Flexible RBAC** with hierarchical scopes (System → Org → Project)
|
|
- **Network virtualization** with VPC overlay using OVN
|
|
- **VM provisioning** with seamless network attachment
|
|
- **Comprehensive testing** validating all integration points
|
|
|
|
This architecture enables secure, isolated cloud deployments for multiple tenants on shared infrastructure, with clear boundaries and well-defined integration points for future extensions (DNS, load balancing, storage).
|