photoncloud-monorepo/flaredb/specs/001-distributed-core/tasks.md

description
Task list template for feature implementation

Tasks: Core Distributed Architecture (Phase 1)

Input: Design documents from /specs/001-distributed-core/ Prerequisites: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/

Tests: The examples below include test tasks. Tests are STANDARD per the Constitution (Principle I). Include them for all functional logic unless explicitly skipped.

Organization: Tasks are grouped by user story to enable independent implementation and testing of each story.

Format: [ID] [P?] [Story] Description

• [P]: Can run in parallel (different files, no dependencies)
• [Story]: Which user story this task belongs to (e.g., US1, US2, US3)
• Include exact file paths in descriptions

Path Conventions

• Single project: src/, tests/ at repository root
• Web app: backend/src/, frontend/src/
• Mobile: api/src/, ios/src/ or android/src/
• Paths shown below assume single project - adjust based on plan.md structure

Phase 1: Setup (Shared Infrastructure)

Purpose: Project initialization and basic structure with Nix environment

• T000 Create flake.nix to provide rust, protobuf, clang, and rocksdb dependencies
• T001 Create Cargo workspace in Cargo.toml with 5 crates: rdb-proto, rdb-storage, rdb-server, rdb-pd, rdb-client, rdb-cli
• T002 Initialize rdb-proto crate with tonic-build and prost dependencies in rdb-proto/Cargo.toml
• T003 [P] Initialize rdb-storage crate with rocksdb dependency in rdb-storage/Cargo.toml
• T004 [P] Initialize rdb-server, rdb-pd, rdb-client crates with tokio and tonic dependencies

---

Phase 2: Foundational (Blocking Prerequisites)

Purpose: Core infrastructure that MUST be complete before ANY user story can be implemented

⚠️ CRITICAL: No user story work can begin until this phase is complete

• T005 Create kvrpc.proto in rdb-proto/src/kvrpc.proto per contract definition
• T006 Create pdpb.proto in rdb-proto/src/pdpb.proto per contract definition
• T007 Implement build.rs in rdb-proto/build.rs to compile protos
• T008 Export generated protos in rdb-proto/src/lib.rs

Checkpoint: Foundation ready - user story implementation can now begin in parallel

---

Phase 3: User Story 1 - Core Storage Engine Verification (Priority: P1) 🎯 MVP

Goal: A robust local storage engine (RocksDB wrapper) with correct CAS logic.

Independent Test: Run unit tests in rdb-storage covering Raw Put and CAS success/conflict scenarios.

Tests for User Story 1 (STANDARD - per constitution) ⚠️

NOTE: Write these tests FIRST, ensure they FAIL before implementation

• T009 [US1] Create unit tests for StorageEngine::put_raw in rdb-storage/src/engine.rs
• T010 [US1] Create unit tests for StorageEngine::compare_and_swap (success/fail) in rdb-storage/src/engine.rs

Implementation for User Story 1

• T011 [US1] Implement StorageEngine trait definition in rdb-storage/src/lib.rs
• T012 [US1] Implement RocksEngine struct wrapping RocksDB in rdb-storage/src/rocks_engine.rs
• T013 [US1] Implement put_raw using CF_DEFAULT in rdb-storage/src/rocks_engine.rs
• T014 [US1] Implement compare_and_swap using RocksDB transaction/merge in rdb-storage/src/rocks_engine.rs
• T015 [US1] Verify all tests pass

Checkpoint: At this point, User Story 1 should be fully functional and testable independently

---

Phase 4: User Story 2 - Basic RPC Transport (Priority: P1)

Goal: Verify gRPC communication pipeline between Client and Server.

Independent Test: Run rdb-server and connect with a minimal rdb-client.

Tests for User Story 2 (STANDARD - per constitution) ⚠️

• T016 [P] [US2] Create integration test tests/test_rpc_connect.rs in rdb-client to verify connection

Implementation for User Story 2

• T017 [P] [US2] Implement KvService gRPC handler in rdb-server/src/service.rs delegating to storage
• T018 [P] [US2] Implement gRPC server startup in rdb-server/src/main.rs
• T019 [US2] Implement RdbClient struct wrapping tonic::transport::Channel in rdb-client/src/client.rs
• T020 [US2] Implement raw_put and cas methods in RdbClient calling gRPC
• T021 [US2] Verify integration test passes

Checkpoint: At this point, User Stories 1 AND 2 should both work independently

---

Phase 5: User Story 3 - Placement Driver TSO (Priority: P2)

Goal: Source of monotonic timestamps (TSO).

Independent Test: Run rdb-pd and verify monotonic TSO generation.

Tests for User Story 3 (STANDARD - per constitution) ⚠️

• T022 [P] [US3] Create unit test for TsoOracle in rdb-pd/src/tso.rs

Implementation for User Story 3

• T023 [P] [US3] Implement TsoOracle logic (monotonic u64) in rdb-pd/src/tso.rs
• T024 [US3] Implement TsoService gRPC handler in rdb-pd/src/service.rs
• T025 [US3] Implement PD server startup in rdb-pd/src/main.rs
• T026 [US3] Add get_tso method to RdbClient in rdb-client/src/client.rs

Checkpoint: All user stories should now be independently functional

---

Phase 6: Polish & Cross-Cutting Concerns

Purpose: Improvements that affect multiple user stories

• T027 Create scripts/verify-core.sh for comprehensive integration verification
• T028 Run quickstart.md verification steps manually
• T029 Format code with cargo fmt and lint with cargo clippy

---

Phase 7: RPC Get & Raft Enhancements

Purpose: Complete client/server Get coverage and initial Raft persistence surface

• T030 [US2] Implement and verify server Get path returning value+version via CAS CF in rdb-server/src/service.rs
• T031 [US2] Implement client raw_get/get APIs and CLI with integration test in rdb-client
• T032 [US2] Add integration test covering Get (RawGet + CAS Get) in rdb-client/tests
• T033 [P] Add Raft log/HardState/ConfState persistence and wire Raft service to peer dispatch in rdb-server (single-region, single-node baseline)

---

Dependencies & Execution Order

Phase Dependencies

• Setup (Phase 1): No dependencies - can start immediately
• Foundational (Phase 2): Depends on Setup completion - BLOCKS all user stories
• User Stories (Phase 3+): All depend on Foundational phase completion
  • User stories can then proceed in parallel (if staffed)
  • Or sequentially in priority order (P1 → P2 → P3)
• Polish (Final Phase): Depends on all desired user stories being complete

User Story Dependencies

• User Story 1 (P1): Can start after Foundational (Phase 2) - Core Storage logic
• User Story 2 (P1): Can start after Foundational (Phase 2) - RPC Layer (Technically depends on US1 storage implementation for full end-to-end, but server shell can be built in parallel)
• User Story 3 (P2): Can start after Foundational (Phase 2) - Independent PD service

Within Each User Story

• Tests (if included) MUST be written and FAIL before implementation
• Models before services
• Services before endpoints
• Core implementation before integration
• Story complete before moving to next priority

Parallel Opportunities

• All Setup tasks marked [P] can run in parallel
• All Foundational tasks marked [P] can run in parallel (within Phase 2)
• Once Foundational phase completes, all user stories can start in parallel (if team capacity allows)
• All tests for a user story marked [P] can run in parallel
• Models within a story marked [P] can run in parallel
• Different user stories can be worked on in parallel by different team members

---

Parallel Example: User Story 1

# Launch all tests for User Story 1 together (if tests requested):
Task: "Create unit tests for StorageEngine::put_raw in rdb-storage/src/engine.rs"
Task: "Create unit tests for StorageEngine::compare_and_swap (success/fail) in rdb-storage/src/engine.rs"

# Launch all models for User Story 1 together:
Task: "Implement StorageEngine trait definition in rdb-storage/src/lib.rs"
Task: "Implement RocksEngine struct wrapping RocksDB in rdb-storage/src/rocks_engine.rs"

---

Implementation Strategy

MVP First (User Story 1 Only)

1. Complete Phase 1: Setup
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
3. Complete Phase 3: User Story 1
4. STOP and VALIDATE: Test User Story 1 independently
5. Deploy/demo if ready

Incremental Delivery

1. Complete Setup + Foundational → Foundation ready
2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
3. Add User Story 2 → Test independently → Deploy/Demo
4. Add User Story 3 → Test independently → Deploy/Demo
5. Each story adds value without breaking previous stories

Parallel Team Strategy

With multiple developers:

1. Team completes Setup + Foundational together
2. Once Foundational is done:
   • Developer A: User Story 1
   • Developer B: User Story 2
   • Developer C: User Story 3
3. Stories complete and integrate independently