DuckDB v1.5.0 "Variegata" Release

This release of DuckDB is named "Variegata" after the paradise shelduck Tadorna Variegata, also known as the paradise duck, a species of Shelduck (a group of goose-like ducks), endemic to New Zealand.

Please also refer to the announcement blog post: https://duckdb.org/2026/03/09/announcing-duckdb-150.html

What's Changed

• Optimize LAST aggregate to iterate backward within batches by @xe-nvdk
• Allow user type with schema defined to have array bounds by @Dtenwolde in #18342
• Remove case-insensitive character from grammar files by @Dtenwolde in #18948
• Fix query() function error message for PIVOT statements without explicit IN clauses by @shivampr in #18900
• Merge v1.4-andium into main by @c-herrewijn in #18952
• [Dev] Some light code cleanup in RemoveUnusedColumns::VisitOperator by @Tishj in #18964
• Support renaming a database - ALTER DATABASE <> RENAME TO <> by @abramk in #18970
• V1.4 andium by @Mytherin in #18982
• Fix field id uniqueness for alter database by @abramk in #18988
• Fix unused variable warnings by @mlafeldt in #18913
• Avoid adding a TupleDataChunkPart with only one row. by @lance5401 in #18904
• Tiny fix for ColumnDataAllocator::ColumnDataAllocator by @baolinhuang in #18990
• Fixes for ALTER DATABASE - use serialization framework, and fix stringification of identifiers by @Mytherin in #18994
• Expected errors 2053 by @hmeriann in #18960
• Fix union semantics in recursive CTEs by @cryoEncryp in #19017
• Multifile parquet_metadata and friends parallelism by @J-Meyers in #18854
• [Optimize] use StorageLock for CompressionFunction by @baolinhuang in #19028
• PEG grammar updates by @Dtenwolde in #19031
• Rework Compression Function registration to avoid grabbing locks by @Mytherin in #19045
• Merge v1.4 into main by @Mytherin in #19048
• [C-API] default to adding overload on conflict when registering functions by @Maxxen in #19036
• Destroy row groups in parallel by @Mytherin in #19053
• Switching core extension upload to dedicated credentials by @hannes in #19060
• CI: Increase stale bot timeout to 1 year by @szarnyasg in #19000
• Revert "set default value of MAIN_BRANCH_VERSIONING to false" by @c-herrewijn in #19057
• Fix extension load by @dentiny in #19079
• Change core extensions bucket name by @hannes in #19082
• Merge v1.4 into main by @Mytherin in #19098
• Allow directory override for extension build by @Mytherin in #19110
• Skip serialization of row_start in DataPointer when targeting latest storage by @Mytherin in #19111
• [C-API] Initial support for accessing file system by @Maxxen in #19086
• Rework UNION (ALL) operators (SetOperationNode/LogicalSetOperation/PhysicalUnion) to have multiple children by @Mytherin in #19109
• pushdown filter when doing cross or lateral join with unnest by @DinosL in #19085
• Fix unholy interaction between CREATE VIEW and CTE definitions by @kryonix in #19116
• Add support for read_duckdb - a function that allows reading / globbing DuckDB database files by @Mytherin in #19108
• Fix typo by @szarnyasg in #19130
• fix: check tree depth and flatten dep join for custom operator extension by @Wal8800 in #19094
• During optimistic writing, gather N row groups and write them at the same time column-at-a-time by @Mytherin in #19118
• Remove unused code by @szarnyasg in #19134
• Add support for late materialization and filter pushdown to read_duckdb by @Mytherin in #19145
• Move storage block prefetching to an option instead of baking in testing during ALTERNATIVE_VERIFY + add a test config for this option by @Mytherin in #19146
• Introduce parser_override_function_t by @Dtenwolde in #19126
• Perform deduplication based on join_keys when building ht by @tianjq16 in #19097
• Print help to stdout by @staticlibs in #19170
• Validate filesystem registration by @dentiny in #19168
• Fix missing replacement scan code for CTEs by @kryonix in #19159
• Correctly set row start in WAL replay by @Mytherin in #19192
• Use a PartialBlockManager per column, instead of sharing one over the entire row group by @Mytherin in #19194
• Internal #5380: AsOf Join Pipeline by @hawkfish in #19169
• Cross and lateral join pushdown with unnest or json - update by @DinosL in #19152
• Missing error messages for test cases by @hmeriann in #19129
• [Profiling] Move EXTRA_INFO into the metrics map by @maiadegraaf in #19124
• Common Subplan Elimination by @lnkuiper in #19080
• GEOMETRY Rework: Part 1 - Logical Type by @Maxxen in #19136
• [Parquet] Implement (unshredded) VARIANT support for COPY TO parquet by @Tishj in #19125
• Add more missing error messages to test cases by @hmeriann in #19210
• [CI] - Fix failing Extension updating test NightlyTests job by cleaning up some space on the runner by @hmeriann in #19214
• Inconsistent semicolon handling by @Dtenwolde in #19199
• Internal #5383: Range Join Sorting by @hawkfish in #19174
• [Profiling] Add QueryContext to more functions, for Reads by @maiadegraaf in #18726
• Merge v1.4-andium into main by @Maxxen in #19225
• [shell] Fix tilde parsing for .once statement by @dentiny in #19254
• Partially remove BoundQueryNode - have Bind directly emit a BoundStatement by @Mytherin in #19289
• Binder clean-up by @Mytherin in #19293
• test: add test for CREATE TABLE with my_schema.my_enum[] by @NickCrews in #19296
• Fix uncaught exception in test_empty_profiling_settings.test by @maiadegraaf in #19286
• GEOMETRY Rework: Part 2 - Statistics by @Maxxen in #19203
• Internal #5380: AsOf Sorting Conversion by @hawkfish in #19278
• Add option for opt-in to parser override by @Dtenwolde in #19181
• Remove BoundTableRef and make binding a TableRef directly return a BoundStatement by @Mytherin in #19294
• Rewrite Grouped Top-N Window Functions To Aggregations by @d-justen in #19280
• Perform optimistic reads while filling gaps in CachingFileSystem by @carlopi in #19122
• Issue #19027: AsOf Join Predicates by @hawkfish in #19316
• Enable experimental metadata re-use by default by @Mytherin in #19317
• Binder: BoundQueryNode cleanup by @Mytherin in #19322
• Remove old sort code by @lnkuiper in #19321
• [Parquet] Implement shredded VARIANT support for COPY TO parquet by @Tishj in #19219
• Merge andium into main by @Tishj in #19325
• [C API] Expose column count and type for table description by @taniabogatsch in #19334
• Relocate the python sqllogic code to the dedicated repository by @Tishj in #19340
• Binder: Rework the way set operations work so that we can use the standard binding flow by @Mytherin in #19342
• Rework CTE binding: remove CTENode, and bind CommonTableExpressionMap directly instead by @Mytherin in #19351
• Avoid keeping shared ptr to ClientContext in QueryContext by @Mytherin in #19353
• Internal #6190: AsOf Threading by @hawkfish in #19328
• [Indexes] Buffer Managed Indexes Part 4: Segment Handles for Base Leaf by @artjomPlaunov in #19303
• Provide force assertion macros by @dentiny in #19212
• Parse result for PEG transformer by @Dtenwolde in #19229
• Allow * NOT SIMILAR TO 'pattern' by @Dtenwolde in #19232
• [C API] Improve documentation on list vectors by @taniabogatsch in #19349
• Make Binding CTEs lazy, and other CTE binding refactors by @Mytherin in #19372
• Internal #6210: IEJoin Threading by @hawkfish in #19361
• [CI] Move two test cases from duckdb-httpfs repo by @hmeriann in #19368
• [C API] Update misleading "use instead" by @taniabogatsch in #19371
• Enable -Wtype-limits warning and fix warnings by @hannes in #19382
• [Profiling] Add metrics for ATTACH / CHECKPOINT / WAL replay by @taniabogatsch in #19367
• Update issue template by @szarnyasg in #19384
• Not building imdb all the time by @hannes in #19388
• CSV Reader: make line reconstruction deal with buffer_pos == buffer_size by @Mytherin in #19390
• [Dev] variant_extract BindData copy fix by @Tishj in #19394
• [Dev] Add a VARIANT visitor class by @Tishj in #19387
• Switch RowVersionManager to using the FixedSizeAllocator by @Mytherin in #19392
• [Profiling][CAPI] Add appender profiling test by @maiadegraaf in #19383
• Use statement for PEG Parser transformer by @Dtenwolde in #19302
• old_transactions is no longer necessary by @Captain32 in #19381
• Add Null Support for TopNWindowElimination Rule by @d-justen in #19370
• Use const references in exceptions varargs by @maiadegraaf in #18723
• Merge v1.4-andium into main by @Mytherin in #19402
• [Variant] Implement comparison operation logic for Variant columns by @Tishj in #19337
• Implement std::allocator that uses ArenaAllocator by @lnkuiper in #19412
• [CI] Correct file path in test case by @hmeriann in #19429
• [C API] Patch C API query progress documentation by @taniabogatsch in #19430
• Fixes .dump command failing with ROLLBACK when dumping tables in non-default schemas. by @shivampr in #19363
• [C API] Extend documentation around header generation by @taniabogatsch in #19433
• Internal #6270: Fix small binary fallback values for arg_min/max_nulls_last by @d-justen in #19434
• Merge v1.4 into main by @Mytherin in #19438
• SET statement for PEG Transformer by @Dtenwolde in #19320
• Add DuckDB::String to operators by @maiadegraaf in #19423
• Make cached hashes thread-safe and emit dictionary vectors from Perfect Hash Join by @lnkuiper in #19332
• fix: make GetCachedHashes thread safe by @0ax1 in #19274
• [Profiling] Enable ATTACH / CHECKPOINT / WAL replay metrics by default by @maiadegraaf in #19436
• Update / simplify issue template by @szarnyasg in #19442
• Move FORCE_ASYNC_SINK_SOURCE from Nightly to Main by @carlopi in #19452
• Fix WITH ORDINALITY virtual columns by @niykko in #19315
• Internal #6196: AsOf Prefix Comparisons by @hawkfish in #19456
• Add safe queue and deque by @dentiny in #19273
• 💝 duckdb now has its endoflife.date page ⏳ by @adriens in #19450
• Fix wording in README by @lnkuiper in #19462
• [C-API] Add support for defining config options (settings) by @Maxxen in #19473
• Internal #6196: AsOf Prefix Comparisons by @hawkfish in #19466
• Internal #6196: AsOf Prefix Comparisons by @hawkfish in #19484
• [TPCH] fix load.sql to produce valid test data by @hmeriann in #19441
• [minor] Add catalog type to exception by @dentiny in #19488
• GEOMETRY Rework: Part 3 - Filter pushdown by @Maxxen in #19439
• [minor] Address test compilation warnings by @dentiny in #19492
• Fix issue #17941: Cryptic error message when dropping a column part of a unique constraint. by @arjanssuri in #19465
• [Optimizer] Support constant order normalization by @doublefish5106 in #19494
• Use Late Materialization in TopNWindowElimination Optimizer by @d-justen in #19463
• Internal #6326: AsOf Join Copying by @hawkfish in #19505
• [swift] fix microseconds conversion to seconds by @kokluch in #19486
• Fix bug initializing std::vector for column names in ADBC API by @CurtHagenlocher in #19444
• Remove empty lines at the start of code blocks by @lnkuiper in #19519
• Merge v1.4 into main by @Mytherin in #19528
• Fix failing python replacement scan by @evertlammerts in #19509
• Adds parquet_full_metadata function by @J-Meyers in #19148
• [Profiling] Add WAL_REPLAY_ENTRY_COUNT and COMMIT_WRITE_WAL_LATENCY metrics by @maiadegraaf in #19524
• Convert CLI to use C++ API, and remove sqlite API wrappers by @Mytherin in #19536
• Adding release name for 1.5 by @hannes in #19540
• CLI: Generate ".help" menu from the list of commands, and add syntax highlighting to .help by @Mytherin in #19542
• CLI: Refactor command-line parameters into a struct similar to MetadataCommands, and add better errors + syntax highlighting by @Mytherin in #19544
• Delete Statement and expression associativity for PEG Transformer by @Dtenwolde in #19507
• Attach statement for PEG transformer by @Dtenwolde in #19460
• Augment Table API function via context stored in AsyncResult object by @carlopi in #19422
• Add COLUMNS() expansion support to DISTINCT ON by @shivampr in #19550
• Retain recursive unnest column names by @DinosL in #19310
• Adds struct variant to regexp_extract_all by @J-Meyers in #19078
• CLI: Make last query result queryable through the _ token by @Mytherin in #19553
• CLI: rework .import to use built-in readers by @Mytherin in #19563
• Count also time blocked waiting for the lock in BLOCKED_THREADS_TIME by @carlopi in #19547
• Change format_version to 2 when specify V2 for PARQUET_VERSION by @koron in #18855
• Fix: list_concat(NULL) returns NULL by @maiadegraaf in #19565
• CLI: More shell code refactoring, remove global variables and split commands into separate files by @Mytherin in #19568
• Consdier offset in topn optimizer by @ZENOTME in #19561
• Fix Issue #19487: array_to_string on empty list return NULL instead of empty string by @arjanssuri in #19548
• CLI: Add support for dynamic prompts by @Mytherin in #19579
• Custom Order Row Group Scans by @d-justen in #19537
• Added missing return code checks for ADBC test cases by @hannes in #19576
• CLI: Make startup-text display configurable using the .startup_text by @Mytherin in #19584
• CLI: Unify all highlighting color code, add support for extended (8-bit) colors and add .display_colors command by @Mytherin in #19587
• Replace deprecated header by @krlmlr in #19602
• const-safe: Add const handling for list_entry_t by @krlmlr in #19603
• Fix uninitialized physical_type in move constructor by @krlmlr in #19604
• Fix uninitialized variable in multi-file progress tracking by @krlmlr in #19597
• Fix uninitialized members in CachedFile constructor by @krlmlr in #19598
• Fix uninitialized members in HashJoinGlobalSourceState by @krlmlr in #19599
• [C API] Generate description, deprecated, and use_instead for function groups by @taniabogatsch in #19586
• Merge v1.4-andium into main by @carlopi in #19589
• CLI: Sort .display_colors by group / hue / luminosity by @Mytherin in #19610
• Use std::remove_cv for physical type id fetching by @dentiny in #19614
• [C-API] Initial support for defining COPY functions by @Maxxen in #19186
• Apply RemoveUnusedColumns below table in out functions by @geoffxy in #19474
• Fix uninitialized sorted_tuples member by @krlmlr in #19593
• Array comparison optimization by @EslamAhmed171 in #19591
• Python SQLLogicTest - show results in a consistent order by @DinosL in #19562
• Add missing description to copy CAPI by @lnkuiper in #19617
• Speed up Dictionary and DICT_FSST compression by replacing std::unordered_map with PrimitiveDictionary by @yan-alex in #19577
• CLI: Display auto-complete suggestions when tabbing through them by @Mytherin in #19626
• Allow Table function to opt-in to return arbitrary combinations of [0, output.size()] x [HAVE_MORE_OUTPUT, FINISHED] by @carlopi in #19566
• Add a Mermaid Flowchart transformer for EXPLAIN output by @tobilg in #19581
• Add recovery mode to ATTACH options and enable NO_WAL mode by @taniabogatsch in #19612
• feat: add --sql option to .open in the cli client by @hello-world-bfree in #19445
• [ART] Pretty Printer + Verify by @artjomPlaunov in #19558
• Internal #6341: AsOf Parallelism by @hawkfish in #19605
• Add window functions to duckdb_functions by @Tishj in #19532
• CLI: Fix cursor rendering in multi-line editing mode by @Mytherin in #19649
• CLI: Eagerly open database also if it does not exist by @Mytherin in #19627
• CLI: Make the progress bar configurable by @Mytherin in #19650
• Encapsulate BaseScalarFunction properties by @Maxxen in #19632
• Buffer-managed query results by @lnkuiper in #19437
• [ART] Iterative Iterator by @artjomPlaunov in #19521
• CI Fuzzer runtime error by @DinosL in #19630
• Guard printing for bignum by @krlmlr in #19600
• Don't truncate test names to 80 characters by @JelteF in #19642
• [Parquet] Emit VARIANT from Parquet VARIANT columns, instead of JSON by @Tishj in #18996
• CLI: Make Linenoise work on Windows by @Mytherin in #19659
• Issue #19499: HashedSort Sorting Memory by @hawkfish in #19664
• Fix incorrect partition calculation due to missing vector_type by @tianjq16 in #19638
• CLI - store action as just a char to avoid sanitizer complaints by @Mytherin in #19679
• Merge v1.4 into main by @Mytherin in #19670
• More arena_stl_allocator integration by @lnkuiper in #19677
• Fix z/OS-specific issues by @DNikolaevAtRocket in #19641
• CLI - Add paging support for output by @Mytherin in #19676
• GEOMETRY Rework: Part 4 - Fixup Parquet Extension + Add Arrow Support by @Maxxen in #19476
• SegmentTree: move next and index out of SegmentBase and into SegmentNode by @Mytherin in #19686
• Add Clear method to BaseAppender by @EtgarDev in #19623
• Handle edge case for date_trunc optimization by @rcurtin in #19628
• fix checkpointing on files that dont exist by @dioptre in #19344
• Fix: Correct typos in comments and error messages by @feichai0017 in #19257
• Fix compiler warnings on main by @lnkuiper in #19690
• Issue #19499: HashedSort Sorting Memory by @hawkfish in #19687
• Internal #6519: HashedSort Memory Footprint by @hawkfish in #19702
• Fix LENGTH(NULL) with GROUP BY NULL binding issue by @shivampr in #19269
• New EXTENSION_RELATION type by @krlmlr in #19596
• array intersect by @krleonid in #19667
• Internal #6341: AsOf Deadlock by @hawkfish in #19705
• Add options for how invalid utf-8 should be handled in decode() by @casperhart in #18168
• CommonSubplan improvements by @lnkuiper in #19710
• Fix #18653: remove unnecessary delim_join by @flashmouse in #19235
• Internal #6523: AsOf Exception Handling by @hawkfish in #19703
• CLI: Refactor BoxRenderer, add wrapping support for large values, and add pretty printing / highlighting for nested types, JSON and variant by @Mytherin in #19721
• Fix empty result pullup for ANTI joins by @feichai0017 in #19701
• search activer binders in table function binder during binding column… by @Damon07 in #14486
• Move formatting logic for tests and benchmarks into a standalone script by @adsharma in #17350
• Exclude Row Groups in Top-N Queries by @d-justen in #19655
• fix bug in generate_grammar script by @OrangieLou in #19751
• Internal #4490: DENSE_RANK Performance by @hawkfish in #19757
• Internal #6521: HashedSort Scan Lock by @hawkfish in #19758
• Issue #19647: Constant Window Allocators by @hawkfish in #19752
• BoxRenderer: Fix rendering of empty results by @Mytherin in #19746
• Bump ducklake main by @J-Meyers in #19756
• Fix initialization when all values are NULL in SMALLER_BINARY aggregation by @DinosL in #19755
• Clean up unused headers in parser/ and subfolders by @Schwarf in #18422
• Fix #17251: Eliminate duplicates after a set operation using set sema… by @tianjq16 in #17267
• Implement outer join elimination by @flashmouse in #18043
• [C-API] Add support for catalog entry lookup by @Maxxen in #19684
• Add Stats API to filesystem interfaces by @dentiny in #19743
• Add sleep scalar function by @krleonid in #19736
• Add flush memory threshold for appender to prevent OOM by @p1p1bear in #19007
• add statement debug and debug_skip by @samansmink in #18472
• Storage: Remove start from RowGroup, ColumnData and ColumnSegment, and rework the way that start is defined in the ColumnSegmentTree by @Mytherin in #19761
• Add Seek() to TupleDataCollection for IEJoin optimization by @EslamAhmed171 in #19753
• Fix TopN Pruning Test by @d-justen in #19768
• Support reading pipes with read_text by @lnkuiper in #19765
• Support TopN dynamic filter for NULLS FIRST by @feichai0017 in #19744
• fix(#18481): Avoid to overwrite secret_directory configuration by it default value by @quentingodeau in #19448
• Change back default strategy for debug_physical_table_scan_execution_strategy by @carlopi in #19778
• Fix major big-endian issues by @DNikolaevAtRocket in #19748
• Disable sleep() function if DuckDB isn't compiled with threads by @lnkuiper in #19783
• Remove unused parameter from arrow by @pdet in #19784
• Block Allocator by @lnkuiper in #19678
• Merge v1.4-andium into main by @Mytherin in #19771
• CLI: Add table metadata rendering, and use this rendering for .tables and DESCRIBE queries by @Mytherin in #19792
• CLI: Improve array rendering by no longer adding newlines after every array entry unless there are complex objects in the array by @Mytherin in #19795
• CLI: For .tables - group tables by database/schema, and render the database name and schema of the tables as a header above the tables by @Mytherin in #19798
• Skip test block_allocator_memory.test on 32bit platforms by @carlopi in #19794
• [ADBC] Fix bug in StatementGetParameterSchema when no params are set by @amoeba in #19790
• Specify shared deletion option on windows by @dentiny in #19782
• Fix flaky sleep test by @krleonid in #19803
• Internal #6541: Window ArenaAllocator Usage by @hawkfish in #19770
• CLI: Several auto-complete improvements, make auto-complete behave more like zsh by @Mytherin in #19805
• [Profiling] Split COMMIT_WRITE_WAL_LATENCY into COMMIT_LOCAL_STORAGE_LATENCY and WRITE_TO_WAL_LATENCY by @maiadegraaf in #19808
• Move GetData as GetDataInternal, and have GetData as wrapper by @carlopi in #19791
• [swift] fix compiler warnings for deprecated @_implementationOnly by @kokluch in #19800
• CLI: Silence auto-complete failures instead of exiting by @Mytherin in #19813
• Random small nits by @taniabogatsch in #19828
• Add test to exclusion list by @Maxxen in #19827
• Create statement for PEG Transformer by @Dtenwolde in #19807
• Add struct_keys method to StructVector by @EtgarDev in #19673
• [Dev] Disallow aggregates used in the TRY expression child by @Tishj in #16285
• CI Cleanup by @Mytherin in #19840
• Fix positional scan and restore task executor by @carlopi in #19825
• [CLI] fix escaping sequence for ctrl/alt arrow key shortcuts by @tobwen in #19261
• Enable macros in DuckLake by @pdet in #19836
• Make RowGroup::Checkpoint and ColumnData::Checkpoint generate a new RowGroup/ColumnData - instead of checkpointing them in-place by @Mytherin in #19843
• Fix incorrect results with NOT EXISTS and IS DISTINCT FROM operator (#19680) by @henry8128 in #19811
• #6429 Roaring Booleans by @yan-alex in #19785
• GEOMETRY Rework: Part 4 - Step 2 - Adjust WKB conversion functions by @Maxxen in #19848
• Internal #6607: IEJoin Task Locking by @hawkfish in #19852
• Add TOTAL_MEMORY_ALLOCATED profiling metric by @krleonid in #19834
• Delete unused block header size by @ccfelius in #19855
• [CAPI] Expose basic custom log storage to CAPI by @maiadegraaf in #19810
• Enable reads to run concurrently while another connection is checkpointing by @Mytherin in #19867
• CLI: Enable support for mouse clicks in Linenoise - pressing Ctrl+Q allows a single mouse-click to be consumed to change the cursor position by @Mytherin in #19869
• Add alias reference support (alias.<name>) by @EtgarDev in #19747
• [Compression] Fix an existing and a new bug in Roaring compression by @Tishj in #19878
• PEG Transformer template script by @Dtenwolde in #19812
• [Testing][Dev] Extend tests covered by the storage_compatibility test config by @Tishj in #19860
• [swift] decode boolean primitive type by @kokluch in #19890
• [Parquet] Implement automatic shredding for VARIANT columns in COPY TO parquet by @Tishj in #19336
• Offset Pruning for Pagination Queries by @d-justen in #19806
• Internal #6568: SortStrategy Refactoring by @hawkfish in #19814
• Refactor hash join filter scan helper by @feichai0017 in #19802
• [Optimizer] Fix cardinality estimates getting lost through struct_extract expressions by @Tishj in #19829
• [Dev] Fix assertion failure in AlterType by @Tishj in #19850
• Internal #6653: WindowExpression Construction by @hawkfish in #19885
• Fix nested sleep_ms calls and add tests by @krleonid in #19837
• fix: make row_start an optional_idx in PartitionStatistics by @rustyconover in #19903
• CLI: Refactor shell rendering code, move everything to ShellRenderer, and perform various fixes by @Mytherin in #19897
• [Storage] Implement storage for VARIANT columns by @Tishj in #19674
• Encapsulate scalar/aggregate function callbacks by @Maxxen in #19707
• Merge V1.4 -> Main by @pdet in #19876
• Checkpoint: prevent adding blocks to the free_list that are still being used by concurrent readers by @Mytherin in #19928
• Merge V1.4 -> Main by @pdet in #19935
• Avoid eagerly grabbing the checkpoint lock when performing inserts, instead grab a shared table lock by @Mytherin in #19937
• [Checkpoint] Fully rewrite stats on Checkpoint by @Tishj in #19932
• fix: update aggregate docs regarding combine, add guards around calling combine callback. by @rustyconover in #19938
• [Wal]ReplayDelete in batches instead of rows by @artjomPlaunov in #19904
• CheckZonemapSegments: iterate on filters, then jump further by @carlopi in #19865
• Tweak Relation::Query() by @krlmlr in #19594
• More TASK_EXECUTOR opt-in by @carlopi in #19940
• [Stats] has_no_null is no longer initialized to true in BaseStatistics::CreateEmpty by @Tishj in #19948
• [Stats] Improve the VariantStats class to support merging and better visualization by @Tishj in #19950
• Handle empty table name on CREATE by @yan-alex in #19955
• Fix ColumnAliasBinder by @d-justen in #19956
• CLI: Add support for dark / light mode that is automatically detected by @Mytherin in #19985
• CLI: Change return value from -1 to false in TryGetBackgroundColor by @c8ef in #19992
• CLI: Fully unify result rendering, and fix automatic pager mode by @Mytherin in #19988
• Adding extension alias lookup in GetDatabaseType by @tomas-karnagel in #19980
• CLI: Various auto-complete fixes by @Mytherin in #19995
• ALP uncompressed mode by @yan-alex in #19902
• Disable block allocator test on non-64bit by @lnkuiper in #19999
• Add comments to cardinality estimator code by @Tmonster in #19973
• [fix] ClientBufferManager::Pin to also propagate context when available by @carlopi in #19986
• Sideways Information Passing for Joins using Bloom Filters by @gropaul in #19502
• Don't require icu unless extension is enabled by @dg0yt in #19963
• Add z/OS support for block allocator by @DNikolaevAtRocket in #19977
• Add a secondary extension directory for package managers by @xhochy in #17499
• #15867 - Make it possible to hide query in profiling output by setting query_name metric to false. by @ormandi in #15870
• Internal #6607: IEJoin Task Locking by @hawkfish in #20028
• Fix #19984: shouldn't use outer table distinct info to eliminate join by @flashmouse in #20023
• fix: allow table in/out functions to indicate if they support parallelism by @rustyconover in #19951
• Fix #20029: accept "\e\" as termination sequence, and add a timeout to TryGetBackgroundColor by @Mytherin in #20042
• Fix #20034: correctly move correlated columns for lazy CTEs by @Mytherin in #20038
• Bloom filter row group skipping by @lnkuiper in #20035
• CLI: Optimize and clean-up result rendering code by @Mytherin in #20021
• fix alter table rename / alter on temp table losing temporary flag by @diwu-sf in #20011
• Improve catalog error message for missing schemas by @krleonid in #19922
• feat(adbc): partial supporting ADBC 1.1.0 by @eitsupi in #19833
• [Profiling] Generate more metric code and add metric subgroups by @maiadegraaf in #19483
• Filesystem-compatible wrapper for CachingFileSystem by @dentiny in #19972
• #Fix 18158: Deliver child column info by @flashmouse in #18327
• Allow concurrent commits to happen while a checkpoint is running by @Mytherin in #20052
• CLI: Improve rendering of wide values, and improve auto-complete by @Mytherin in #20074
• Checkpoint: Allow vacuuming of rows for tables that have indexes if the vacuum does not change the rowids by @Mytherin in #20073
• Eagerly Execute Min/Max Aggregates on Statistics by @d-justen in #19906
• Late extension linking by @hannes in #19369
• ADBC: Set append data error in Ingest() by @kou in #19991
• Don't throw permission error during binder when LocalFileSystem is disabled by @JelteF in #20077
• Add struct_values function to return field values of STRUCT by @EtgarDev in #19968
• TupleDataCollection + MemoryTag fix by @lnkuiper in #20089
• Enable Parquet reader support for uniformly encrypted Parquet files written by Arrow C++ API by @ccfelius in #20046
• Fix source and bin dirs to duckdb's root by @evertlammerts in #20088
• Allow CTEs to emit NULL types by @Mytherin in #20097
• Change CLI highlighting colors and error message rendering by @Mytherin in #20099
• Roaring boolean FetchRow fix by @yan-alex in #20101
• Fix race condition in piecewise merge join by @lnkuiper in #20059
• Improve join filter pushdown for ON clause conditions by @EslamAhmed171 in #19888
• Merge V1.4 -> Main by @pdet in #20111
• Fix parsing of dollar quoted strings in various places by @JelteF in #20121
• Do not autocomplete within dollar-quoted strings by @JelteF in #20122
• Fix concurrent test failure by @Mytherin in #20129
• Fix over-eager constraint checking in the presence of primary keys/unique constraints by @Mytherin in #20119
• When shortening columns in the BoxRenderer, avoid applying the min col width twice by @Mytherin in #20126
• Unify column renaming behavior of CTEs with other places in the system by @kryonix in #20116
• Fix vector sizes CI by @lnkuiper in #20130
• [Logging] Add assertions for LogType type by @maiadegraaf in #20064
• [Profiling] Check that file type set by profiling_output matches output type by @maiadegraaf in #19946
• Add flag to use pre-built static library by @hannes in #20092
• Create Shell Log Storage to Print Warnings in CLI by @maiadegraaf in #19974
• [Doc] Add ccache into contributing doc build section by @dentiny in #20139
• Fix column lifetime for DISTINCT ON by @feichai0017 in #19786
• documentation for extension patch workflow by @artjomPlaunov in #19905
• Art pretty printer options by @artjomPlaunov in #20079
• feat: add OperatorOrder() to table functions to indicate ordering needs. by @rustyconover in #19954
• Internal #6777: IEJoin Unified L1/2 by @hawkfish in #20083
• fix: link error on linux with multiple definition of LogicalType::VARCHAR in shell_renderer.cpp by @rustyconover in #20096
• Remove pointer indirection in ExtensionAccess by @staticlibs in #19529
• [Dev] Fix failure on extension_entries.hpp on main by @Tishj in #20138
• [Dev] Fix concurrent test failure (followup) by @Tishj in #20140
• HashCSVStateMachineConfig to return hash_t by @carlopi in #20137
• Provide option disable cache validation by @dentiny in #19886
• Parallel TupleDataCollection destructor by @lnkuiper in #20159
• Internal #6778: IEJoin Parallel L1/L2 by @hawkfish in #20152
• Allow concurrent insertions into tables with indexes while checkpointing by @Mytherin in #20160
• Revert linux toolchain downgrade by @hannes in #20170
• Do not exit shell on Ctrl+C by @staticlibs in #20155
• CLI: Several rendering fixes / improvements by @Mytherin in #20175
• fix: remove extra space in GetName() for PhysicalTableScan in profiler output by @rustyconover in #20179
• Internal #6779: 64-Bit Range Joins by @hawkfish in #20176
• More CommonSubplanOptimizer improvements by @lnkuiper in #20168
• Profiling: Fix rows_scanned by @d-justen in #20161
• [Optimizer] Clean up the cross-over point from ColumnIndex -> StorageIndex by @Tishj in #20087
• Issue #19629: DATE_TRUNC Return Type by @hawkfish in #20120
• Move to macos-latest for newer compiler by @hannes in #20169
• [minor] Avoid divide by 0 for json progress report by @dentiny in #20196
• Allow Late Materialization for Parquet in TopNWindowElimination by @d-justen in #20066
• Link MSVC runtime library statically by @staticlibs in #19415
• fix ci run - 'no space left on device' by @c-herrewijn in #20203
• fix: simplify logic by removing empty else branch by @darion-yaphet in #20184
• fix: simplify platform detection in CMakeLists.txt by @darion-yaphet in #20183
• Add days_in_month internal macro for retrieving number of days in a month by @ArNine in #20180
• Fix try cast for variant to incompatible struct types by @wmTJc9IK0Q in #20211
• fix: add schema and catalog to table scan explain output by @rustyconover in #20181
• feat: add filenames to MultiFileDynamicToString by @rustyconover in #20182
• [C-API] Add support for scalar function local states by @Maxxen in #20109
• CLI: Add .last to re-render the last query result, and make several (performance) improvements to .mode duckbox by @Mytherin in #20223
• Throw exception if there is ARTKey mismatch by @artjomPlaunov in #20225
• Fix Windows shell input: correct variable buffering and character order by @EslamAhmed171 in #20217
• [minor] Address compilation warnings by @dentiny in #20213
• Refactor PhysicalCreateIndex and introduce index build abstraction by @ccfelius in #20158
• Refactor PhysicalCreateIndex and introduce index build abstraction by @Maxxen in #19461
• Internal #6837: Streaming IGNORE NULLS by @hawkfish in #20226
• [Optimizer] Push down struct_extract expression to the storage. by @Tishj in #20063
• Friendly SWITCH case by @Dtenwolde in #20001
• Parallel SortedRunMerger destructor by @lnkuiper in #20200
• Internal #6462: Implement AGO Macro by @hawkfish in #20228
• Do not call UnregisterBlock from destructor by @staticlibs in #20242
• [minor] Address more compile warnings by @dentiny in #20232
• Re-enable aws tests by @Tmonster in #20249
• Fix wrong error message when add primary key to a view(#20239) by @ArNine in #20253
• Fix last parallel destructor race condition (hopefully) by @lnkuiper in #20251
• [Optimizer] Pushdown a CAST expression on top of a struct_extract by @Tishj in #20250
• [ART] Prefix handles by @artjomPlaunov in #19658
• Fix: CHECK constraint internal error on UPDATE with missing columns (#20199) by @AestheticAkhmad in #20243
• Internal #6462: Implement AGO Macro by @hawkfish in #20257
• feat: preserve ATTACH options and return via duckdb_databases() by @rustyconover in #20186
• support reading parquet files with empty groups by @achille-roussel in #20135
• Move checkpointing outside of the attached database destructor by @taniabogatsch in #19930
• Issue #20076: Implement CountWindowElimination Logic by @hawkfish in #20273
• Minor python fixes by @c-herrewijn in #20269
• Fix concat(null) error by @d-justen in #20266
• Big-endian patches (FSST, arrow, types, md5) by @DNikolaevAtRocket in #20237
• Merge v1.4-andium into main by @carlopi in #20231
• Allow concurrent deletes while checkpointing by @Mytherin in #20286
• Correctly handle default expressions in WAL when doing ALTER TABLE ... ADD COLUMN ... DEFAULT ... by @yan-alex in #20224
• [Fix] Use the current schema and catalog during data creation for TPCH and TPCDS by @taniabogatsch in #20290
• peg grammar: start rule first by @jraymakers in #20275
• return_stats for boolean and numeric (128 bits) during copy by @aykut-bozkurt in #20222
• Fixes raised by cppcheck by @carlopi in #20323
• Fixup QueryProfiler::GetBytesRead and Written by @carlopi in #20318
• Internal #6974: Window Self-Join Files by @hawkfish in #20317
• Internal #6999: Window TopN Comparisons by @hawkfish in #20316
• Internal #6943: IEJoin Code Cleanup by @hawkfish in #20315
• Use eager min/max aggregation on parquet statistics by @d-justen in https://github.com/duckdb/duckdb/pull/20301
• c-api: adding out_file against NULL check by @ProjectMutilation in https://github.com/duckdb/duckdb/pull/20303
• Apply LRU cache to parquet metadata by @dentiny in https://github.com/duckdb/duckdb/pull/20157
• [Profiling] Add duckdb_profiling_settings() macro by @maiadegraaf in https://github.com/duckdb/duckdb/pull/20234
• Fix some compiler warnings on apple clang 17 by @hannes in https://github.com/duckdb/duckdb/pull/20337
• feat(adbc): support the uri option of ADBC 1.1.0 by @eitsupi in https://github.com/duckdb/duckdb/pull/20312
• feat(adbc): support ADBC_INFO_DRIVER_ADBC_VERSION (new in ADBC 1.1.0) by @eitsupi in https://github.com/duckdb/duckdb/pull/20344
• GEOMETRY Rework: Part 5 - Coordinate Reference System Support by @Maxxen in https://github.com/duckdb/duckdb/pull/20143
• Remove two unnecessary test exclusions by @yan-alex in https://github.com/duckdb/duckdb/pull/20347
• Preserve uppercase chars in DB filename on Windows by @staticlibs in https://github.com/duckdb/duckdb/pull/20298
• Eager aggregate support filter by @flashmouse in https://github.com/duckdb/duckdb/pull/20198
• [Profiling] Add enable_profiling() Table Function by @maiadegraaf in https://github.com/duckdb/duckdb/pull/20151
• Fix: Potential Unexpected Result when Using TRY Expression #20006 by @AestheticAkhmad in https://github.com/duckdb/duckdb/pull/20238
• Fix #20308: Added collation binding to the join conditions by @gemy26 in https://github.com/duckdb/duckdb/pull/20349
• Pick up stale PR 19271 by @lnkuiper in https://github.com/duckdb/duckdb/pull/20359
• Support reading FIXED_LEN_BYTE_ARRAY fields with DELTA_BYTE_ARRAY encoding by @arouel in https://github.com/duckdb/duckdb/pull/20270
• Parquet: Populate the column_orders field and expose it in parquet_file_metadata by @lasanaka-jumptrading in https://github.com/duckdb/duckdb/pull/20241
• Add RemoveFiles to FileSystem by @utay in https://github.com/duckdb/duckdb/pull/20333
• Perform cleanup when COPY function throws an error by @yan-alex in https://github.com/duckdb/duckdb/pull/20345
• Pick up stale PR 19268 by @lnkuiper in https://github.com/duckdb/duckdb/pull/20358
• CachingOperator: handle also ORDERED, and consider swapping chunks by @carlopi in https://github.com/duckdb/duckdb/pull/20259
• fix binder: handle errors with over in where inside subquery by @Pranav2612000 in https://github.com/duckdb/duckdb/pull/20310
• Add interruption check to optimizer execution by @EtgarDev in https://github.com/duckdb/duckdb/pull/20377
• Out of core tests by @lnkuiper in https://github.com/duckdb/duckdb/pull/20383
• Use forward-compatible std::shuffle by @philippmd in https://github.com/duckdb/duckdb/pull/20395
• fix compilation with FORCE_QUERY_LOG=1 by @zhanglei1949 in https://github.com/duckdb/duckdb/pull/20387
• [minor] Fix a few compilation warnings by @dentiny in https://github.com/duckdb/duckdb/pull/20385
• Add parse_formatted_bytes function to parse human-readable byte size by @EtgarDev in https://github.com/duckdb/duckdb/pull/20107
• Fix issue #20187: Fix PIVOT aggregate mismatch for ranges >=21 by @gemy26 in https://github.com/duckdb/duckdb/pull/20272
• Fix caching filesystem CanSeek by @dentiny in https://github.com/duckdb/duckdb/pull/20330
• Improve format script compatibility on Windows by @xuboying in https://github.com/duckdb/duckdb/pull/20398
• Add way to skip version checks in scripts/format.py by @carlopi in https://github.com/duckdb/duckdb/pull/20388
• Fix function chain in qualify(#20233) by @ArNine in https://github.com/duckdb/duckdb/pull/20262
• Fix concat(null) is null return false, it should be true(issue: #20306) by @ArNine in https://github.com/duckdb/duckdb/pull/20320
• NightlyTests fixes by @lnkuiper in https://github.com/duckdb/duckdb/pull/20408
• Storage cleanup: remove unused headers and fix transitive includes by @Schwarf in https://github.com/duckdb/duckdb/pull/20407
• Avoid frequent checkpoints triggered by optimistic insertions by @Captain32 in https://github.com/duckdb/duckdb/pull/20336
• No catalog found Nightly test fix by @ccfelius in https://github.com/duckdb/duckdb/pull/20409
• TIME_NS Support + Arrow by @pdet in https://github.com/duckdb/duckdb/pull/20361
• feat(adbc): support the experimental adbc.ingest.target_catalog option and improve ingest name resolution by @eitsupi in https://github.com/duckdb/duckdb/pull/20369
• Fix file opener propagation by @dentiny in https://github.com/duckdb/duckdb/pull/20414
• Cleanup RowGroup Scan code, and make BoundIndex::Delete throw an error if the rows are not present in the index by @Mytherin in https://github.com/duckdb/duckdb/pull/20430
• [Optimizer] Support NOT elimination by @doublefish5106 in https://github.com/duckdb/duckdb/pull/20394
• JSON Reader: exit scan loop if we have found an error that we can't yet throw by @Mytherin in https://github.com/duckdb/duckdb/pull/20434
• Merge V1.4 -> Main by @pdet in https://github.com/duckdb/duckdb/pull/20350
• Internal #6975: Window Inner Self-Join by @hawkfish in https://github.com/duckdb/duckdb/pull/20459
• Issue #20413: ASOF Arbitrary Predicates by @hawkfish in https://github.com/duckdb/duckdb/pull/20456
• [C API] Expose safe string assign function by @taniabogatsch in https://github.com/duckdb/duckdb/pull/20467
• Internal #6976: Window Self-Join Predicate by @hawkfish in https://github.com/duckdb/duckdb/pull/20473
• CMake: export also duckdb_generated_extension_loader by @carlopi in https://github.com/duckdb/duckdb/pull/20449
• Chore: merge back main into v1.5-variegata by @carlopi in https://github.com/duckdb/duckdb/pull/20469
• add_third_party(skiplist) into top level CMake by @carlopi in https://github.com/duckdb/duckdb/pull/20494
• Add SHOW schemas by @c-herrewijn in https://github.com/duckdb/duckdb/pull/20492
• [SQLLogicTest] Add the CONTINUE statement by @Tishj in https://github.com/duckdb/duckdb/pull/20489
• Remove fs limitation: OVERWRITE is not supported for remote file systems by @carlopi in https://github.com/duckdb/duckdb/pull/20487
• [Encryption] Adding random IV and tag for canary encryption by @ccfelius in https://github.com/duckdb/duckdb/pull/20208
• [Optimizer] Push down variant_extract + cast into the storage layer by @Tishj in https://github.com/duckdb/duckdb/pull/20268
• Internal #6977: Window Self-Join Aggregate by @hawkfish in https://github.com/duckdb/duckdb/pull/20497

Full Changelog: v1.4.4...v1.5.0