pgrundev / pgsync

pgsync 
 Memory-bounded logical replication for PostgreSQL, in Rust. 
 One binary takes a transactionally consistent snapshot of the tables you name,
 then streams every committed change through native pgoutput into a second
 PostgreSQL — one target transaction per source transaction, checkpoint committed with the data. 
 No extension on the source, no broker, no memory that grows with your database.
 Quickstart ·
 Install ·
 Setup ·
 Commands ·
 Configuration ·
 Correctness ·
 Benchmarks ·
 Troubleshooting ·
 Architecture 
> **Status: controlled technical beta (v0.1.1, Beta 1.1).** Synthetic or explicitly approved
> non-sensitive data only. There is no data masking yet, so pgsync must not be pointed at
> production customer data. What is and is not supported is spelled out in
> `docs/features.md`; the evidence behind every claim below is in
> `docs/test-report.md` and `docs/benchmarks.md`.
---
## Quickstart
**1. Install**
 [code] 
**2. Prepare the source** (once, as an admin; needs `wal_level = logical`)
 [code] 
**3. Configure** — `pgsync.yaml` (full annotated version: `examples/pgsync.yaml`)
 [code] 
 [code] 
**4. Run**
 [code] 
`pgsync` reads `pgsync.yaml` from the current directory (or `--config `). Ctrl-C
finishes the transaction in flight and exits cleanly.
### Try it locally in 90 seconds
Two PostgreSQL 18 containers with a demo publication already set up:
 [code] 
In a second shell, write to the source and read it back from the target:
 [code] 
Connection URLs come from the environment variables named in the config — never from the
file, never from the command line — so they stay out of `ps`, logs, and git.
## Why pgsync
| | |
|---|---|
| **Correct before fast** | Each source transaction becomes exactly one target transaction, applied in commit order. The checkpoint row (`pgsync.job`) is updated *inside* that transaction, so data and position are durable together or not at all. WAL is acknowledged to the source only after the commit returns. |
| **Crash-proof by construction** | After a SIGKILL, a dropped connection, or a server restart, pgsync reloads the checkpoint from the target and resumes. Redelivered transactions are recognised by LSN and skipped; nothing is applied twice. A 15-round random-kill loop under continuous writes converges to identical content hashes. |
| **Memory does not scale with data** | One bounded, byte-accounted channel between reader and applier, bounded statement batches, fixed-size COPY buffers, and no transaction buffering. Measured RSS: 33 MB peak snapshotting 1 GB, ~15 MB steady during CDC, 9.5 MB per process with 20 jobs. |
| **No silent skipping, ever** | If a change cannot be applied safely — a column the target lacks, an UPDATE that matches no row, a slot someone else advanced — the job stops visibly (`ERROR` or `RESYNC_REQUIRED`) with a reason. There is no "start from current LSN" fallback in the code. |
| **Gentle with the source** | Native `pgoutput`, no extensions, no event triggers, no DDL, no DML