make Migration Overview MOLT-centric #19532
Conversation
✅ Deploy Preview for cockroachdb-api-docs canceled.
|
✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.
|
taroface
force-pushed
the
migration-refactor-v3
branch
from
745768a to
778a2d8
Compare
❌ Deploy Preview for cockroachdb-docs failed. Why did it fail? →
|
❌ Deploy Preview for cockroachdb-docs failed. Why did it fail? →
|
❌ Deploy Preview for cockroachdb-docs failed. Why did it fail? →
|
✅ Netlify Preview
To edit notification comments on pull requests, go to your Netlify site configuration. |
taroface
force-pushed
the
migration-refactor-v3
branch
8 times, most recently
from
e12f37f to
57bbdcf
Compare
|
Temporarily removing from review while I do some deeper reorganization (prompted by many broken links). |
taroface
force-pushed
the
migration-refactor-v3
branch
3 times, most recently
from
3fc3401 to
51c454f
Compare
…verview and Strategy
taroface
force-pushed
the
migration-refactor-v3
branch
2 times, most recently
from
b35e8aa to
4140338
Compare
taroface
force-pushed
the
migration-refactor-v3
branch
from
4140338 to
12b29dd
Compare
|
|
||
| - Define an explicit primary key on every table. For more information, refer to [Primary key best practices]({% link {{ site.current_cloud_version }}/schema-design-table.md %}#primary-key-best-practices). | ||
|
|
||
| - Do **not** use a sequence to define a primary key column. Instead, Cockroach Labs recommends that you use [multi-column primary keys]({% link {{ site.current_cloud_version }}/performance-best-practices-overview.md %}#use-multi-column-primary-keys) or [auto-generating unique IDs]({% link {{ site.current_cloud_version }}/performance-best-practices-overview.md %}#use-functions-to-generate-unique-ids) for primary key columns. |
There was a problem hiding this comment.
@Jeremyyang920 should this explain why exactly? Mainly because of the hot spotting problem. Might be worthwhile to provide reasoning for this statement instead of a blanket statement
| SET CLUSTER SETTING kv.rangefeed.enabled = true; | ||
| ~~~ | ||
|
|
||
| - Ensure that your CockroachDB deployment has a valid [Enterprise license]({% link {{ site.current_cloud_version }}/licensing-faqs.md %}). |
There was a problem hiding this comment.
Should we show the sample command of how to do this?
SET CLUSTER SETTING cluster.organization = 'org';
SET CLUSTER SETTING enterprise.license = 'license';
|
|
||
| {% include molt/migration-modify-target-schema.md %} | ||
|
|
||
| ## Step 6. Replicate changes to CockroachDB |
There was a problem hiding this comment.
@rohan-joshi @Jeremyyang920 should we add the section about how to verify replication has caught up and the next step (cutover) can happen?
| 1. Convert the source schema: Use the [Schema Conversion Tool]({% link cockroachcloud/migrations-page.md %}) to generate CockroachDB-compatible [DDL]({% link {{ site.current_cloud_version }}/sql-statements.md %}#data-definition-statements). Apply the converted schema to the target database. Drop constraints and indexes to facilitate data load. | ||
| 1. Load data into CockroachDB: Use [MOLT Fetch]({% link molt/molt-fetch.md %}) to bulk-ingest your source data. | ||
| 1. (Optional) Verify consistency before replication: Use [MOLT Verify]({% link molt/molt-verify.md %}) to confirm that the data loaded into CockroachDB is consistent with the source. | ||
| 1. Replicate ongoing changes. Enable continuous replication to keep CockroachDB in sync with the source. |
There was a problem hiding this comment.
Nit: seems like we explicitly say which tool is used for the above convert schema, load data, and verify steps. Should we also note which tool will replicate ongoing changes? Or at least provide examples (i.e. Replicator or DMS). I guess the complexity here is that we don't refer to "Replicator" directly in these docs as a standalone tool
| 1. Load data into CockroachDB: Use [MOLT Fetch]({% link molt/molt-fetch.md %}) to bulk-ingest your source data. | ||
| 1. (Optional) Verify consistency before replication: Use [MOLT Verify]({% link molt/molt-verify.md %}) to confirm that the data loaded into CockroachDB is consistent with the source. | ||
| 1. Replicate ongoing changes. Enable continuous replication to keep CockroachDB in sync with the source. | ||
| 1. Verify consistency before cutover. Use MOLT Verify to confirm that the CockroachDB data is consistent with the source. |
There was a problem hiding this comment.
Nit: maybe link to MOLT Verify again
| <tr> | ||
| <td class="comparison-chart__feature"><a href="#fetch"><b>Fetch</b></a></td> | ||
| <td>Initial data load; optional continuous replication</td> | ||
| <td>PostgreSQL 11-14, MySQL 5.7-8.0+, CockroachDB</td> |
There was a problem hiding this comment.
We actually support PG 15 too (just checked our CI and we run tests again this nightly)
| <tr> | ||
| <td class="comparison-chart__feature"><a href="#verify"><b>Verify</b></a></td> | ||
| <td>Schema and data validation</td> | ||
| <td>PostgreSQL 12-14, MySQL 5.7-8.0+, CockroachDB</td> |
|
|
||
| - [Multiple migration modes](#migration-modes) via `IMPORT INTO` or `COPY FROM` | ||
| - Data movement via [cloud storage, local file servers, or direct copy]({% link molt/molt-fetch.md %}#data-path). | ||
| - [Concurrent data export]({% link molt/molt-fetch.md %}#best-practices) from multiple source tables and shards |
There was a problem hiding this comment.
Is it worth calling out here that this is limited to certain PK types? Or is that too detailed here?
| - Table structure verification | ||
| - Column definition verification | ||
| - Row-level data verification | ||
| - Continuous, live, or one-time verification |
There was a problem hiding this comment.
Though we support this, want to say this is not an oft-tested path and we've had issues with customers workign with this mode. CC @Jeremyyang920 and @rohan-joshi
DOC-12110
DOC-12134
DOC-12697
DOC-12951
moltsubdirectory, as this is easier to maintain & ties the docs to MOLT tooling, which is already there (link changes are the bulk of the files changed)These changes apply to 23.1-25.2 docs. Please review: