--- description: "Task list template for feature implementation" --- # Tasks: Core Distributed Architecture (Phase 1) **Input**: Design documents from `/specs/001-distributed-core/` **Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/ **Tests**: The examples below include test tasks. Tests are STANDARD per the Constitution (Principle I). Include them for all functional logic unless explicitly skipped. **Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story. ## Format: `[ID] [P?] [Story] Description` - **[P]**: Can run in parallel (different files, no dependencies) - **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3) - Include exact file paths in descriptions ## Path Conventions - **Single project**: `src/`, `tests/` at repository root - **Web app**: `backend/src/`, `frontend/src/` - **Mobile**: `api/src/`, `ios/src/` or `android/src/` - Paths shown below assume single project - adjust based on plan.md structure ## Phase 1: Setup (Shared Infrastructure) **Purpose**: Project initialization and basic structure with Nix environment - [X] T000 Create `flake.nix` to provide rust, protobuf, clang, and rocksdb dependencies - [X] T001 Create Cargo workspace in `Cargo.toml` with 5 crates: `rdb-proto`, `rdb-storage`, `rdb-server`, `rdb-pd`, `rdb-client`, `rdb-cli` - [X] T002 Initialize `rdb-proto` crate with `tonic-build` and `prost` dependencies in `rdb-proto/Cargo.toml` - [X] T003 [P] Initialize `rdb-storage` crate with `rocksdb` dependency in `rdb-storage/Cargo.toml` - [X] T004 [P] Initialize `rdb-server`, `rdb-pd`, `rdb-client` crates with `tokio` and `tonic` dependencies --- ## Phase 2: Foundational (Blocking Prerequisites) **Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented **⚠️ CRITICAL**: No user story work can begin until this phase is complete - [X] T005 Create `kvrpc.proto` in `rdb-proto/src/kvrpc.proto` per contract definition - [X] T006 Create `pdpb.proto` in `rdb-proto/src/pdpb.proto` per contract definition - [X] T007 Implement `build.rs` in `rdb-proto/build.rs` to compile protos - [X] T008 Export generated protos in `rdb-proto/src/lib.rs` **Checkpoint**: Foundation ready - user story implementation can now begin in parallel --- ## Phase 3: User Story 1 - Core Storage Engine Verification (Priority: P1) 🎯 MVP **Goal**: A robust local storage engine (RocksDB wrapper) with correct CAS logic. **Independent Test**: Run unit tests in `rdb-storage` covering Raw Put and CAS success/conflict scenarios. ### Tests for User Story 1 (STANDARD - per constitution) ⚠️ > **NOTE**: Write these tests FIRST, ensure they FAIL before implementation - [X] T009 [US1] Create unit tests for `StorageEngine::put_raw` in `rdb-storage/src/engine.rs` - [X] T010 [US1] Create unit tests for `StorageEngine::compare_and_swap` (success/fail) in `rdb-storage/src/engine.rs` ### Implementation for User Story 1 - [X] T011 [US1] Implement `StorageEngine` trait definition in `rdb-storage/src/lib.rs` - [X] T012 [US1] Implement `RocksEngine` struct wrapping RocksDB in `rdb-storage/src/rocks_engine.rs` - [X] T013 [US1] Implement `put_raw` using `CF_DEFAULT` in `rdb-storage/src/rocks_engine.rs` - [X] T014 [US1] Implement `compare_and_swap` using RocksDB transaction/merge in `rdb-storage/src/rocks_engine.rs` - [X] T015 [US1] Verify all tests pass **Checkpoint**: At this point, User Story 1 should be fully functional and testable independently --- ## Phase 4: User Story 2 - Basic RPC Transport (Priority: P1) **Goal**: Verify gRPC communication pipeline between Client and Server. **Independent Test**: Run `rdb-server` and connect with a minimal `rdb-client`. ### Tests for User Story 2 (STANDARD - per constitution) ⚠️ - [X] T016 [P] [US2] Create integration test `tests/test_rpc_connect.rs` in `rdb-client` to verify connection ### Implementation for User Story 2 - [X] T017 [P] [US2] Implement `KvService` gRPC handler in `rdb-server/src/service.rs` delegating to storage - [X] T018 [P] [US2] Implement gRPC server startup in `rdb-server/src/main.rs` - [X] T019 [US2] Implement `RdbClient` struct wrapping `tonic::transport::Channel` in `rdb-client/src/client.rs` - [X] T020 [US2] Implement `raw_put` and `cas` methods in `RdbClient` calling gRPC - [X] T021 [US2] Verify integration test passes **Checkpoint**: At this point, User Stories 1 AND 2 should both work independently --- ## Phase 5: User Story 3 - Placement Driver TSO (Priority: P2) **Goal**: Source of monotonic timestamps (TSO). **Independent Test**: Run `rdb-pd` and verify monotonic TSO generation. ### Tests for User Story 3 (STANDARD - per constitution) ⚠️ - [X] T022 [P] [US3] Create unit test for `TsoOracle` in `rdb-pd/src/tso.rs` ### Implementation for User Story 3 - [X] T023 [P] [US3] Implement `TsoOracle` logic (monotonic u64) in `rdb-pd/src/tso.rs` - [X] T024 [US3] Implement `TsoService` gRPC handler in `rdb-pd/src/service.rs` - [X] T025 [US3] Implement PD server startup in `rdb-pd/src/main.rs` - [X] T026 [US3] Add `get_tso` method to `RdbClient` in `rdb-client/src/client.rs` **Checkpoint**: All user stories should now be independently functional --- ## Phase 6: Polish & Cross-Cutting Concerns **Purpose**: Improvements that affect multiple user stories - [X] T027 Create `scripts/verify-core.sh` for comprehensive integration verification - [X] T028 Run `quickstart.md` verification steps manually - [X] T029 Format code with `cargo fmt` and lint with `cargo clippy` --- ## Phase 7: RPC Get & Raft Enhancements **Purpose**: Complete client/server Get coverage and initial Raft persistence surface - [X] T030 [US2] Implement and verify server Get path returning value+version via CAS CF in `rdb-server/src/service.rs` - [X] T031 [US2] Implement client `raw_get`/`get` APIs and CLI with integration test in `rdb-client` - [X] T032 [US2] Add integration test covering Get (RawGet + CAS Get) in `rdb-client/tests` - [X] T033 [P] Add Raft log/HardState/ConfState persistence and wire Raft service to peer dispatch in `rdb-server` (single-region, single-node baseline) --- ## Dependencies & Execution Order ### Phase Dependencies - **Setup (Phase 1)**: No dependencies - can start immediately - **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories - **User Stories (Phase 3+)**: All depend on Foundational phase completion - User stories can then proceed in parallel (if staffed) - Or sequentially in priority order (P1 → P2 → P3) - **Polish (Final Phase)**: Depends on all desired user stories being complete ### User Story Dependencies - **User Story 1 (P1)**: Can start after Foundational (Phase 2) - Core Storage logic - **User Story 2 (P1)**: Can start after Foundational (Phase 2) - RPC Layer (Technically depends on US1 storage implementation for full end-to-end, but server shell can be built in parallel) - **User Story 3 (P2)**: Can start after Foundational (Phase 2) - Independent PD service ### Within Each User Story - Tests (if included) MUST be written and FAIL before implementation - Models before services - Services before endpoints - Core implementation before integration - Story complete before moving to next priority ### Parallel Opportunities - All Setup tasks marked [P] can run in parallel - All Foundational tasks marked [P] can run in parallel (within Phase 2) - Once Foundational phase completes, all user stories can start in parallel (if team capacity allows) - All tests for a user story marked [P] can run in parallel - Models within a story marked [P] can run in parallel - Different user stories can be worked on in parallel by different team members --- ## Parallel Example: User Story 1 ```bash # Launch all tests for User Story 1 together (if tests requested): Task: "Create unit tests for StorageEngine::put_raw in rdb-storage/src/engine.rs" Task: "Create unit tests for StorageEngine::compare_and_swap (success/fail) in rdb-storage/src/engine.rs" # Launch all models for User Story 1 together: Task: "Implement StorageEngine trait definition in rdb-storage/src/lib.rs" Task: "Implement RocksEngine struct wrapping RocksDB in rdb-storage/src/rocks_engine.rs" ``` --- ## Implementation Strategy ### MVP First (User Story 1 Only) 1. Complete Phase 1: Setup 2. Complete Phase 2: Foundational (CRITICAL - blocks all stories) 3. Complete Phase 3: User Story 1 4. **STOP and VALIDATE**: Test User Story 1 independently 5. Deploy/demo if ready ### Incremental Delivery 1. Complete Setup + Foundational → Foundation ready 2. Add User Story 1 → Test independently → Deploy/Demo (MVP!) 3. Add User Story 2 → Test independently → Deploy/Demo 4. Add User Story 3 → Test independently → Deploy/Demo 5. Each story adds value without breaking previous stories ### Parallel Team Strategy With multiple developers: 1. Team completes Setup + Foundational together 2. Once Foundational is done: - Developer A: User Story 1 - Developer B: User Story 2 - Developer C: User Story 3 3. Stories complete and integrate independently