diff --git a/docs/en/SUMMARY.md b/docs/en/SUMMARY.md index 39816bf4..02e38866 100644 --- a/docs/en/SUMMARY.md +++ b/docs/en/SUMMARY.md @@ -12,6 +12,7 @@ * [Built from source](installation/install-from-source.md) * [Deploy to production](installation/deploy-to-production.md) * [Version upgrade](installation/upgrade.md) +* [Migrating from legacy deals](installation/migrating-from-legacy-deals.md) ## Data Preparation @@ -31,6 +32,7 @@ * [Inline Preparation](topics/inline-preparation.md) * [Benchmark](topics/benchmark.md) * [PDP Calibnet E2E Runbook](topics/pdp-calibnet-e2e.md) +* [DDO Contract Deal Guide](topics/ddo-contract-deal-guide.md) ## 💻 CLI Reference diff --git a/docs/en/deal-making/create-a-deal-schedule.md b/docs/en/deal-making/create-a-deal-schedule.md index b0ae0ea2..a073abcb 100644 --- a/docs/en/deal-making/create-a-deal-schedule.md +++ b/docs/en/deal-making/create-a-deal-schedule.md @@ -12,6 +12,20 @@ For PDP (`--deal-type pdp`) schedules, run deal pusher with FEVM RPC configured: singularity run deal-pusher --eth-rpc "$ETH_RPC_URL" ``` +For DDO (`--deal-type ddo`) schedules, run deal pusher with DDO contract configuration and run deal tracker with DDO tracking enabled: + +```sh +singularity run deal-pusher \ + --eth-rpc "$ETH_RPC_URL" \ + --ddo-contract "$DDO_CONTRACT_ADDRESS" \ + --ddo-payments-contract "$DDO_PAYMENTS_CONTRACT_ADDRESS" \ + --ddo-payment-token "$DDO_PAYMENT_TOKEN" + +singularity run deal-tracker \ + --eth-rpc "$ETH_RPC_URL" \ + --ddo-contract "$DDO_CONTRACT_ADDRESS" +``` + ## Send all deals at once With smaller dataset, you could send all deals to your storage providers all at once. To achieve this, you can use below command @@ -38,3 +52,14 @@ singularity deal schedule create -h For a full real-network walkthrough with concrete transactions, see: - [PDP Calibnet End-to-End Runbook](../topics/pdp-calibnet-e2e.md) + +## DDO caveats + +- DDO schedules use deal type `ddo`. +- DDO schedules require a non-empty `urlTemplate`. +- The provider must be active in the DDO contract and support the configured payment token. +- DDO deals stay in `proposed` state until `deal-tracker` observes allocation activation on-chain. + +For a full DDO setup guide, see: + +- [DDO Contract Deal Guide](../topics/ddo-contract-deal-guide.md) diff --git a/docs/en/installation/migrating-from-legacy-deals.md b/docs/en/installation/migrating-from-legacy-deals.md new file mode 100644 index 00000000..8c6f2974 --- /dev/null +++ b/docs/en/installation/migrating-from-legacy-deals.md @@ -0,0 +1,131 @@ +# Migrating from legacy deals + +This guide is for operators moving from legacy market deals (`market`) to the contract-backed deal types now supported by Singularity: + +- PDP deals (`pdp`) +- DDO allocation deals (`ddo`) + +If your program refers to the second path as PoRep onboarding or allocation-backed deals, Singularity exposes it as `ddo`. + +## What `admin init` does + +Run: + +```bash +singularity admin init +``` + +This upgrades the schema and backfills missing `deal_type` values on older rows: + +- legacy deals are marked as `market` +- legacy schedules are marked as `market` + +This is a metadata migration only. It does **not** convert existing market deals into PDP or DDO deals. + +## What stays the same + +- Existing market deals remain market deals. +- Existing market schedules continue to run as market schedules unless you change them. +- Existing prep data, wallets, providers, and CAR generation do not need to be recreated just because deal types were added. + +## Recommended migration path + +The safest migration is to create a new schedule for the new deal type rather than mutating an active legacy schedule in place. + +1. Run `singularity admin init`. +2. List your existing schedules and identify the legacy market schedule you want to replace. +3. Confirm the preparation has a wallet attached. +4. Pause the legacy schedule before enabling the new one. +5. Create a fresh PDP or DDO schedule against the same preparation. +6. Run the workers needed for that deal type. +7. Verify new deals are being created and tracked. +8. Remove the old schedule only after the replacement path is behaving as expected. + +## Migrating to PDP + +Use PDP when you want proof-set based contract deals. + +Before creating a PDP schedule: + +- run `deal-pusher` with `--eth-rpc` +- attach a wallet that can sign FEVM transactions +- make sure the preparation's piece sizes fit the current PDP limit + +Current PDP caveats: + +- accepted piece CID formats are legacy CommP and CommPv2 +- the current proof-set piece-size cap is `1,065,353,216` bytes + +Example: + +```bash +singularity run deal-pusher --eth-rpc "$ETH_RPC_URL" + +singularity run pdp-tracker --eth-rpc "$ETH_RPC_URL" + +singularity deal schedule create \ + --preparation 1 \ + --provider t01000 \ + --deal-type pdp \ + --piece-cid bafkzcibd... \ + --max-pending-deal-number 1 \ + --total-deal-number 1 +``` + +For a real-network walkthrough, see [PDP Calibnet E2E Runbook](../topics/pdp-calibnet-e2e.md). + +## Migrating to DDO + +Use DDO when you want allocation-based contract deals. + +Before creating a DDO schedule: + +- run `deal-pusher` with `--eth-rpc`, `--ddo-contract`, `--ddo-payments-contract`, and `--ddo-payment-token` +- run `deal-tracker` with `--eth-rpc` and `--ddo-contract` +- attach a wallet that can sign FEVM transactions +- make sure the provider is registered and active in the DDO contract +- make sure the provider supports the payment token you configured +- provide a non-empty `--url-template` so the storage provider can fetch each piece + +Example: + +```bash +singularity run deal-pusher \ + --eth-rpc "$ETH_RPC_URL" \ + --ddo-contract "$DDO_CONTRACT_ADDRESS" \ + --ddo-payments-contract "$DDO_PAYMENTS_CONTRACT_ADDRESS" \ + --ddo-payment-token "$DDO_PAYMENT_TOKEN" + +singularity run deal-tracker \ + --eth-rpc "$ETH_RPC_URL" \ + --ddo-contract "$DDO_CONTRACT_ADDRESS" + +singularity deal schedule create \ + --preparation 1 \ + --provider t01000 \ + --deal-type ddo \ + --url-template "https://downloads.example.com/piece/{PIECE_CID}.car" \ + --max-pending-deal-number 10 \ + --total-deal-number 100 +``` + +For DDO, the most important operational difference is that `deal-tracker` is responsible for moving deals from `proposed` to `active` once the allocation is activated on-chain. + +## In-place schedule updates + +Singularity does allow `deal_type` updates on schedules, but use that path carefully. + +- Existing deal rows keep their original `deal_type`. +- DDO schedules require a non-empty `url_template`. +- Contract-backed deal types have different runtime dependencies than legacy market deals. + +If you need a low-risk migration, create a new schedule instead of editing the old one. + +## Rollback + +If the new path is not behaving correctly: + +1. Pause the PDP or DDO schedule. +2. Re-enable the legacy market schedule. +3. Fix worker configuration or contract settings. +4. Retry with a new replacement schedule once the environment is correct. diff --git a/docs/en/installation/upgrade.md b/docs/en/installation/upgrade.md index 126b6a9b..cdf7112f 100644 --- a/docs/en/installation/upgrade.md +++ b/docs/en/installation/upgrade.md @@ -6,3 +6,5 @@ singularity admin init ``` For major version upgrade, e.g. 2.4.0 to 3.0.0, please refer to migration guide for each major version release. + +If you are moving existing schedules from legacy market deals to PDP or DDO, see [Migrating from legacy deals](migrating-from-legacy-deals.md). diff --git a/docs/en/topics/ddo-contract-deal-guide.md b/docs/en/topics/ddo-contract-deal-guide.md new file mode 100644 index 00000000..0c818c6c --- /dev/null +++ b/docs/en/topics/ddo-contract-deal-guide.md @@ -0,0 +1,136 @@ +# DDO Contract Deal Guide + +This guide covers Singularity's allocation-based contract deal path, exposed as deal type `ddo`. + +If your program refers to this path as PoRep onboarding or allocation-backed contract usage, use `ddo` in Singularity commands and API payloads. + +## What DDO changes + +Unlike legacy market deals: + +- Singularity submits allocations on-chain instead of sending a boost proposal +- the storage provider fetches data from the download URL attached to each allocation +- payment setup happens before allocation submission +- deals become `active` only after `deal-tracker` sees the allocation activate on-chain + +## Prerequisites + +You need: + +- a preparation with CAR files ready to schedule +- a wallet attached to that preparation +- an FEVM RPC endpoint +- the DDO Diamond proxy contract address +- the DDO Payments contract address +- an ERC20 payment token address supported by the target provider +- a reachable download URL template for each piece + +The target provider must also be: + +- resolvable by Lotus +- active in the DDO contract +- configured to accept the payment token you pass to `deal-pusher` + +## Required worker configuration + +Start `deal-pusher` with the DDO contract settings: + +```bash +singularity run deal-pusher \ + --eth-rpc "$ETH_RPC_URL" \ + --ddo-contract "$DDO_CONTRACT_ADDRESS" \ + --ddo-payments-contract "$DDO_PAYMENTS_CONTRACT_ADDRESS" \ + --ddo-payment-token "$DDO_PAYMENT_TOKEN" +``` + +Start `deal-tracker` with DDO tracking enabled: + +```bash +singularity run deal-tracker \ + --eth-rpc "$ETH_RPC_URL" \ + --ddo-contract "$DDO_CONTRACT_ADDRESS" +``` + +Without the DDO-enabled `deal-tracker`, DDO deals will be created in `proposed` state but will not transition to `active` when the allocation is sealed. + +## Creating a DDO schedule + +Create a schedule with: + +- `--deal-type ddo` +- a provider address that Lotus can resolve +- a non-empty `--url-template` + +Example: + +```bash +singularity deal schedule create \ + --preparation 1 \ + --provider t01000 \ + --deal-type ddo \ + --url-template "https://downloads.example.com/piece/{PIECE_CID}.car" \ + --max-pending-deal-number 10 \ + --total-deal-number 100 +``` + +`--url-template` should include `{PIECE_CID}` so each allocation points at the correct CAR file. + +## How scheduling works + +For each batch, Singularity: + +1. validates the provider against the DDO contract +2. checks that each piece size fits the provider's configured min and max limits +3. builds per-piece download URLs from `url_template` +4. ensures deposit and operator approval through the payments contract +5. submits the allocation transaction +6. waits for confirmation depth +7. parses allocation IDs from the receipt +8. writes DDO deal rows into the database + +## Runtime flags + +The main DDO tuning flags on `deal-pusher` are: + +- `--ddo-batch-size` +- `--ddo-confirmation-depth` +- `--ddo-poll-interval` +- `--ddo-term-min` +- `--ddo-term-max` +- `--ddo-expiration-offset` + +Defaults today are: + +- batch size: `10` +- confirmation depth: `5` +- poll interval: `30s` +- min term: `518400` epochs +- max term: `5256000` epochs +- expiration offset: `172800` epochs + +## Observing state + +You can inspect DDO deals with the normal deal list APIs and CLI filters: + +```bash +singularity deal list --deal-type ddo +``` + +Expected state flow: + +- `proposed`: allocation submitted, waiting for activation +- `active`: allocation activated on-chain and picked up by `deal-tracker` + +## Updating an existing DDO schedule + +DDO schedules can be updated, but keep these invariants in mind: + +- `deal_type` must remain a valid deal type +- `url_template` cannot be empty while the schedule is `ddo` + +If you are migrating from a legacy market schedule, creating a fresh DDO schedule is safer than editing the old one in place. + +## Current limitations + +- DDO activation tracking is implemented, but expiry or terminal-state tracking is not yet defined. +- The current implementation waits for tx confirmation before writing deal rows; crash recovery for the gap between confirmation and DB insert is not yet hardened.