跳至主要内容
小龙虾小龙虾AI
🤖

surreal-sync

从 MongoDB、PostgreSQL、MySQL、Neo4j、Kafka 和 JSONL 进行数据迁移和同步到 SurrealDB。完整和增量 CDC 同步。属于 surreal-...

下载1.1k
星标2
版本1.2.1
数据库
安全通过

技能说明


name: surreal-sync description: "Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-skills collection." license: MIT metadata: version: "1.0.4" author: "24601" parent_skill: "surrealdb" snapshot_date: "2026-02-19" upstream: repo: "surrealdb/surreal-sync" release: "v0.3.4" sha: "8166b2b041b1"

Surreal-Sync -- Data Migration and Synchronization

Surreal-Sync is a CLI tool for migrating data from various database sources to SurrealDB with full and incremental synchronization via Change Data Capture (CDC).

Supported Sources

SourceFull SyncIncremental CDCMethod
MongoDBYesYesChange streams
MySQLYesYesTrigger-based CDC + sequence checkpoints
PostgreSQL (triggers)YesYesTrigger-based CDC + sequence checkpoints
PostgreSQL (wal2json)YesYesLogical replication with wal2json plugin
Neo4jYesYesTimestamp-based tracking
JSONL FilesYesN/ABatch import from JSON Lines
Apache KafkaYesYesConsumer subscriptions with deduplication

Quick Start

# Install surreal-sync (Rust binary)
cargo install surreal-sync

# Full sync from PostgreSQL (trigger-based)
surreal-sync from postgres trigger-full \
  --connection-string "postgresql://user:pass@localhost/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Incremental CDC from PostgreSQL (wal2json)
surreal-sync from postgres wal2json \
  --connection-string "postgresql://user:pass@localhost/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Full sync from MongoDB
surreal-sync from mongo full \
  --connection-string "mongodb://localhost:27017/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Batch import from JSONL
surreal-sync from jsonl import \
  --file data.jsonl \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Consume from Kafka
surreal-sync from kafka consume \
  --bootstrap-servers "localhost:9092" \
  --topic my-events \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

CLI Pattern

surreal-sync from <SOURCE> <COMMAND> \
  --connection-string [CONNECTION STRING] \
  --surreal-endpoint [SURREAL ENDPOINT] \
  --surreal-username [SURREAL USERNAME] \
  --surreal-password [SURREAL PASSWORD] \
  --to-namespace <NS> \
  --to-database <DB>

Key Features

  • Automatic schema inference and SurrealDB table creation
  • Record ID mapping from source primary keys
  • Relationship extraction and graph edge creation
  • Configurable batch sizes and parallelism
  • Resumable sync with checkpoint tracking
  • Deduplication for Kafka consumers

Full Documentation

See the main skill's rule file for complete guidance:

如何使用「surreal-sync」?

  1. 打开小龙虾AI(Web 或 iOS App)
  2. 点击上方「立即使用」按钮,或在对话框中输入任务描述
  3. 小龙虾AI 会自动匹配并调用「surreal-sync技能完成任务
  4. 结果即时呈现,支持继续对话优化

相关技能