What databendlabs/databend shipped

This week: segment zone maps for fuse tables, TopN pruning across UNION ALL, recluster unified under OPTIMIZE TABLE, UNION ALL over stage files fixed, CSV empty values preserved, and snapshot hints published only after commit.

This week in Databend: fuse tables gained a segment zone map built by recluster so segments can be pruned before reading; runtime TopN pruning now reaches UNION ALL branches; ordinary-table and materialized-view reclustering are unified under OPTIMIZE TABLE. Fixes include data loss in UNION ALL when a branch reads stage files, CSV empty string and binary values, and snapshot hints being published before an explicit transaction commits. Materialized views can now be created from a source table in a different database, direct MV reads use the view's own SELECT grant, Python UDF egress is controlled through network rules and external access integrations, and the admin API exposes node-local cache statistics.

This week: exact float-to-decimal casts, exact decimal arithmetic and rounding, dictionary-encoded function evaluation, balanced hash shuffle memory, safer runtime filter placement, vector index REFRESH and VACUUM, and sandboxed Python UDFs.

This week's work focused on correctness and predictability. Float-to-decimal casts and decimal arithmetic, casts and rounding are now exact near precision limits and raise errors instead of panicking. Numeric text loaded into integer columns is parsed exactly, and timestamp boundaries and text round trips were fixed. On the execution side, functions, grouping and aggregates can evaluate on encoded dictionary columns, thread-level hash shuffle memory is bounded regardless of cluster size, row fetch decodes each segment once, and runtime filters are no longer pushed below row-sensitive paths like LIMIT and windows. Vector indexes gained a new storage format with optional FP16 refine data plus REFRESH and VACUUM, and Python UDF workers are now sandboxed.

This week in Databend: remote UDF access managed with SQL integrations, S3 storage-class overrides for external tables, S3 region detection via AWS_REGION, a new min_scale for decimals, and fixes for REPLACE INTO duplicate keys and expression-key anti joins.

This week in Databend: remote UDF server access can be managed with SQL integrations, including network rules and USAGE authorization. S3 storage classes now support database defaults and table-level overrides for external tables, and locations without a region can detect it via the AWS_REGION regional endpoint. A min_scale function was added for decimals. Fixes include REPLACE INTO detecting duplicated keys before pruning, the optimizer avoiding incorrect anti joins for expression keys, and isolated bounded workers for scalar Python UDFs. time_slice and single-key hash shuffle were also made faster.

This week in Databend: experimental zero-copy CREATE TABLE... CLONE, ALTER TABLE MATERIALIZE TTL, send_notification SQL, ATTACH TABLE privilege, Iceberg name mapping fix, exact decimal modulo, bounded window restore memory, faster VACUUM TEMPORARY FILES.

This week in Databend: experimental zero-copy CREATE TABLE... CLONE for managed FUSE tables across databases; ALTER TABLE MATERIALIZE TTL to synchronously clean up expired rows through the DELETE mutation pipeline; send_notification for delivery through an existing notification integration; and a required ATTACH TABLE privilege alongside CREATE. Fixes include Iceberg reads with name mapping and file physical schemas, exact decimal modulo instead of Float64, view names no longer resolving to an outer CTE of the same name, bounded memory when restoring spilled window partitions, and transaction statistics merged on commit retry. VACUUM TEMPORARY FILES now processes expired spill directories concurrently.

Tonight's build: horizontal recluster and inverted-index multiway merge, hash write layout for replace/insert all/merge, and DROP PARTITION KEY. Fixes landed for top-N pruning, rank limit ordering, grouping sets, singleton group keys, and Python UDF worker starvation.

This week's work covers storage and query correctness. The storage side adds horizontal recluster and inverted-index multiway merge, applies the hash write layout to REPLACE INTO, INSERT ALL and MERGE INTO, supports ALTER TABLE DROP PARTITION KEY for FUSE tables, and reduces metadata overhead when compacting recent segments on write-heavy tables. The query side fixes results that could be wrong: static value-statistics TopN pruning no longer drops top-N blocks, rank limit pushdown follows the ORDER BY key order, ordered aggregates in ROLLUP, CUBE and GROUPING SETS use original grouping values, singleton group keys hash correctly, and the final sort run spills under memory pressure. Context functions are rejected in persisted expressions, scalar number formats are cached per batch, and Pyth

V1.2.951-nightly: column scope validation in planner, distributed ALTER TABLE MODIFY COLUMN rewrites, Azure Blob presigning, Iceberg on Azure support, and multiple query optimizer fixes.

V1.2.951-nightly ships this week with improvements to query correctness and Azure support. Planner now validates column scope after each optimizer pass to catch bugs earlier. ALTER TABLE MODIFY COLUMN rewrites now distribute work across nodes. Iceberg tables on Azure and PRESIGN with Azure Workload Identity are now supported. Multiple query optimizer fixes address scalar aggregates and rank limit pushdown issues.

V1.2.949-nightly ships tuple IN multi-column subqueries, float canonicalization in hashes, row-level TTL, and webhook body templates. Float zero and NaN fixes prevent silent mismatches in joins and aggregates.

Databend v1.2.949-nightly includes critical correctness fixes and new capabilities. Float zero and NaN values are now properly canonicalized in hash operations, preventing silent mismatches in Bloom indexes and hash joins. Tuple IN now supports multi-column subqueries with SQL row-comparison semantics. Row-level TTL provides automatic data lifecycle management for FUSE tables. Webhook notifications gain templating support for integration with Feishu, Slack, and DingTalk. Meta clients now support gRPC TLS endpoints. Infrastructure improvements include virtual column defaults and fixes to correlation key matching.

V1.2.947-nightly ships indexes on materialized views, streaming inverted index construction, and efficient index merging without re-tokenizing. Window functions now respect session timezones.

V1.2.947-nightly brings substantial improvements to indexing infrastructure. Inverted indexes now stream construction out-of-memory, merge without re-tokenizing, and work on materialized views. Query fixes ensure correct behavior in window RANGE INTERVAL frames and nested column statistics. Storage optimizations reshuffle block metadata post-pruning for better distributed read parallelism. Dynamic tables now supported as Fuse-backed multi-table derived results.