What PeerDB-io/peerdb shipped
Generated by FoxPlug from public activity — not affiliated with Peerdb.
Week of September 14, 2026
What shipped
- BigQuery connector now uses storage read API client, improving throughput from 1.8 MB/s to ~18 MB/s for sources with heavy data volumes. Pull request #4797
- BigQuery CDC scheduling now uses last_synced_at to determine when a sync completed successfully, improving reliability of scheduled pulls. Pull request #4810
- BigQuery connector now selects only configured columns via table mappings instead of SELECT * EXCEPT, reducing unnecessary data transfer. Pull request #4740
- Normalization now properly handles Nullable(JSON) columns, required for structured ingestion workflows. Pull request #4813
- MongoDB BSON to JSON conversion moved to flow/pkg, enabling ClickPipes' Discovery MongoDB schema inference. Pull request #4825
- Structured ingestion now accepts NaN and Inf float values, recording them as strings in intermediate raw events. Pull request #4798
- PostgreSQL CDC no longer performs unnecessary JSON marshal roundtrips for JSON columns, reducing CDC latency for large documents. Pull request #4759
- DropFlowSource cleanup retry alerts downgraded from errors to warnings to prevent alert spam during stuck cleanups. Pull request #4807
- MongoDB BSON to QValue converters refactored behind BsonToQValueConverter interface for better type dispatch. Pull request #4802
- ClickHouse allowed domains configuration now trims whitespace and ignores empty entries in comma-separated lists. Pull request #4795
Why it matters
This week includes significant performance improvements for BigQuery and PostgreSQL connectors that reduce latency and increase throughput for high-volume syncs. Structured ingestion support expanded with better JSON handling and MongoDB schema inference. Several reliability fixes address edge cases in CDC scheduling and error reporting.
Changelog entry
- BigQuery: Storage read API client now enabled by default, improving throughput approximately 10x for large datasets Pull request #4797
- BigQuery: CDC scheduling now uses last_synced_at to detect successful syncs instead of just last_attempt_at Pull request #4810
- BigQuery: Table pulls now select only configured columns via mappings instead of SELECT * EXCEPT Pull request #4740
- PostgreSQL: CDC JSON column handling no longer performs unnecessary marshal/unmarshal roundtrips Pull request #4759
- Normalization: Nullable(JSON) columns now properly handled in normalize queries Pull request #4813
- Structured ingestion: NaN and Inf float values now accepted and recorded as strings Pull request #4798
- MongoDB: BSON to JSON conversion moved to flow/pkg for ClickPipes Discovery schema inference Pull request #4825
- MongoDB: BSON to QValue conversion refactored with per-type converters behind BsonToQValueConverter Pull request #4802
- Alerts: DropFlowSource cleanup retry failures downgraded from error to warning level Pull request #4807
- ClickHouse: Allowed domains configuration now trims whitespace and ignores empty entries Pull request #4795
- ClickHouse: Type combinations with Nullable and LowCardinality no longer require explicit listing Pull request #4799
- MongoDB: CDC now reports receive/process/addRecord timing metrics Pull request #4788
This week: BigQuery storage read API now live (18MB/s throughput), CDC scheduling reliability improved with last_synced_at tracking, PostgreSQL JSON handling optimized, and structured ingestion gains Nullable(JSON) support.
This week's improvements focus on performance and reliability. BigQuery connector throughput increased 10x with storage read API integration. PostgreSQL CDC latency reduced by eliminating unnecessary JSON parsing. BigQuery CDC scheduling now tracks successful syncs separately from attempts. Structured ingestion gains support for Nullable(JSON) columns and NaN/Inf float values. MongoDB schema inference enabled via BSON conversion refactoring.