# DB-DuckDB

**Repository Path**: skyter10086/DB-DuckDB

## Basic Information

- **Project Name**: DB-DuckDB
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Artistic-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-10-23
- **Last Updated**: 2026-11-02

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# DB::DuckDB

A DuckDB driver for Raku's [DB](https://raku.land/zef:raku-community-modules/DB) interface,
implemented as a direct [NativeCall](https://docs.raku.org/language/nativecall) binding over
the DuckDB C API — no external process, no hand-maintained FFI layer. The C in the
distribution is three small self-contained files under `resources/`, none of which includes
`duckdb.h` or links libduckdb: a ~45-line pointer bridge (the two remaining by-value
`duckdb_result` accessors cannot be called safely from NativeCall on SysV platforms; built on
first use, skipped entirely on Windows), a `VARCHAR` bulk decoder, and an Arrow release
bridge. The two shims are optional — without a compiler the driver uses slower pure Raku
paths.

```raku
use DB::DuckDB;

my $db = DB::DuckDB.new;                              # in-memory
# my $db = DB::DuckDB.new(path => 'data.duckdb');     # file backed

say $db.query('select 42').value;                     # 42

$db.execute('create table foo (id integer, name varchar)');
$db.execute('insert into foo values (?, ?)', 1, 'a');

for $db.query('select * from foo').hashes -> %row {
    say %row<id>, %row<name>;
}
```

## Status

1.0.0. Everything below is implemented and covered by tests. The query path has not
changed since 0.9.2 - 1.0 adds the packaging and the compatibility promise, not features:

* driver: `new`, `connect`, `query`, `execute`, `close`, `library-version`
* connection: `prepare`, `prepare-nocache`, `query`, `execute`, `begin`, `commit`, `rollback`, `ping`, `finish`
* statement: positional `?` and named `$name` placeholders (DuckDB syntax; `:name` is not supported), binding and rebinding, `execute`
* result: `value`, `row`, `array`, `hash`, `arrays`, `hashes`, `column`, `column-view`, `column-views`, `column-arrays`, `:parallel`, `:degree`, `:stream`, `chunks`, `is-streaming`, `names`, `rows`, `rows-changed`, `free`
* writes: `appender`, `insert-rows`, `execute-many`, and the `DB::DuckDB::Appender` handle
  (`append-row` / `append-rows` / `flush` / `close` / `rows` / `width`), all routed through
  DuckDB's appender
* parquet: `read-parquet` (a file, a glob or a list; `:union-by-name`, `:filename`,
  `:hive-partitioning`, `:columns`, `:stream`), `write-parquet` (a table or a query, with
  compression / row-group-size / partitioning), `parquet-metadata`, `parquet-schema`,
  `parquet-row-groups`
* `DB::DuckDB::Arrow`: `arrow-chunk`, `with-arrow-chunks` — Arrow C data interface views, no Arrow extension and no NanoArrow

Results are read chunk by chunk out of the vectors and decoded in Raku (`DB::DuckDB::Decoder`),
because 2.0 stopped rendering several types through `duckdb_value_varchar`. `VARCHAR` columns
are unpacked in bulk by a small C shim (see *Design notes*) and fall back to the per-value
decoder when the shim cannot be built.

Every chunk the driver fetches is destroyed again once its vector has been consumed or copied,
including the chunks `column-views` holds for lazy decoding (released in `free`). Chunks that
were never destroyed cost ~235 KB each, which is what made a 200k-row query leak ~23 MB per
round; the whole suite now runs flat across hundreds of rounds.

Out of scope for 1.x: callbacks/UDFs, extension loading, and anything that needs a by-value
struct return (see *Design notes*). These are not half-finished features waiting for a patch
release -- the 1.x line will not grow them; a 2.0 is where that kind of change would land.

## Stability

1.0.0 freezes the documented surface for the 1.x series: a patch release fixes bugs, a
minor release adds functionality without breaking what exists, and anything that breaks
the public API waits for 2.0. `api` in `META6.json` stays `"1"` while the `DB` role
contract is unbroken and moves independently of the distribution version.

**Public API, covered by that promise**

* `DB::DuckDB`, `DB::DuckDB::Connection`, `DB::DuckDB::Statement`, `DB::DuckDB::Result`,
  `DB::DuckDB::Appender`, `DB::DuckDB::ColumnView`, `DB::DuckDB::TextColumn`,
  `DB::DuckDB::Chunk`, `DB::DuckDB::Parquet`, `DB::DuckDB::Arrow` and
  `DB::DuckDB::Error`, with the methods listed under *Status*.
* The switches documented in *Environment switches*: their names and meanings are part of
  1.x. A switch may gain values and may be documented later than it appears in code, but it
  will not be renamed or dropped before 2.0.
* The `DB` role contract.

**Internal, no promise, may change in any release**

* `DB::DuckDB::Native`, `DB::DuckDB::Bridge`, `DB::DuckDB::Decoder`,
  `DB::DuckDB::VarcharShim`, `DB::DuckDB::ArrowShim`, `DB::DuckDB::ShimCache` and the
  `resources/*.c` shim sources. They are listed in `provides` only because Rakudo needs
  them on disk. Their `SHIM-ABI` number is what keeps a library cached by an older release
  from being reused: a bump forces a rebuild.

## Requirements

* Raku 6.d / Rakudo (tested on Rakudo 2026.09, Windows 11 and WSL Ubuntu)
* `DB`, plus `Alien::Libduckdb`, which puts a libduckdb where this module can
  find it: a prebuilt archive in seconds, a cmake source build as fallback
  (5-20 minutes). `zef` resolves both automatically.
* *Optional:* a C compiler for the `VARCHAR` and Arrow shims. It is not required - without
  one the driver just logs a note and uses the slower pure Raku decoders (and the pure Raku
  Arrow release callback, which caches its callable per release pointer since 0.8.1). Set
  `DB_DUCKDB_CC` if the compiler is not called `cc`/`gcc`/`clang` and not in one
  of the usual Windows toolchain directories.

### The VARCHAR shim

`resources/duckdb-varchar-shim.c` is a self-contained C file: it does not
include `duckdb.h` and does not link libduckdb, it only knows the byte layout of
`duckdb_string_t`. It is compiled on first use (one `gcc -O2 -shared`) into
`~/.local/share/DB-DuckDB` and reloaded from there afterwards; a cached library
is reused only when the per-file stamp next to it records the ABI this driver
expects, so a newer source or a bumped ABI means a rebuild.

| variable | effect |
|---|---|
| `DB_DUCKDB_VARCHAR_SHIM` | path of a prebuilt shim library; skips the build |
| `DB_DUCKDB_SHIM_CACHE` | where the built library is cached |
| `DB_DUCKDB_CC` / `CC` | compiler to use |
| `DB_DUCKDB_NO_VARCHAR_SHIM=1` | never use the shim (pure Raku decoder) |
| `DB_DUCKDB_NO_CACHE_SWEEP=1` | leave the shim cache exactly as it is |
| `DB_DUCKDB_SHIM_TRACE` | path of a log file; the shim appends a line per step (survives a hard crash, unlike stderr) |

The two paths are cross-checked in `t/04-varchar-shim.rakutest`: both digests
of the same table have to match byte for byte.

The cache directory is swept once per process, after a shim has been located:
`.bak` files from older releases, stamps whose library is gone, per-process
builds older than an hour, and a shared-name library with no stamp that
predates a stamped one. Everything in that directory is rebuildable from
`resources/`, so the sweep is safe; it still refuses to empty a cache that has
no usable library left, and never touches the library the process resolved.
`DB::DuckDB::ShimCache` holds the logic, `t/15-shim-cache.rakutest` pins it.

### Why the Alien dependency is pinned

The DuckDB C API is not ABI-stable across majors: `duckdb_result` is passed by
value, the `DUCKDB_TYPE` enum was renumbered, and the by-value accessors moved.
This binding is written and tested against **DuckDB 2.x**, so `META6.json`
asks for exactly `Alien::Libduckdb:ver<1.1.0>` - the release that ships
libduckdb 2.0.0 - instead of whatever is newest. Bumping that pin is a
deliberate act: read `duckdb.h`, check the result struct layout, run `t/`, then
bump.

As a second line of defence, the driver calls `duckdb_library_version()` right
before it opens its first database handle and refuses to continue when the
loaded library is not 2.x:

```
DB::DuckDB: libduckdb v3.0.0 is not ABI-compatible with this binding, which
targets DuckDB 2.x; library in use: C:\rakudo\bin\duckdb.dll. ...
```

That catches what the pin cannot: a `DB_DUCKDB_LIB` override, a stale
`duckdb.dll` sitting next to `raku`, or a system libduckdb.

### How the library is located

1. `Alien::Libduckdb::shared-lib()` - the private prefix that `zef install`
   set up (this is the normal path)
2. `%DB_DUCKDB_LIB` environment variable, if it points at an existing file
3. `resources/libraries/duckdb` inside the distribution, if installed with one
4. next to the running `raku` executable - this covers `C:\rakudo\bin\duckdb.dll`
5. `/usr/local/lib`, `/usr/lib`, `/opt/homebrew/lib`
6. bare `duckdb`, letting the OS loader search

```powershell
$env:DB_DUCKDB_LIB = 'C:\rakudo\bin\duckdb.dll'
```
## Install

```sh
zef install .
```

Or run straight from a checkout:

```sh
raku -Ilib -e 'use DB::DuckDB; say DB::DuckDB.new.query("select version()").value'
```

## Bulk writes

Multi-row `INSERT` statements pay DuckDB's parser for every row. The appender API
does not, and the driver exposes it at three levels:

```raku
my $db = DB::DuckDB.new;

# 1. the low-level handle
my $ap = $db.appender('measurements');
$ap.append-row(1, 'x', 1.5);
$ap.append-row([2, 'y', 2.5]);    # a single Positional is unpacked
$ap.free;                         # flush + close; raises on a failed cast

# 2. whole batches (transactional by default)
$db.insert-rows('measurements', ((3, 'z', 3.5), (4, Any, 4.5)));

# 3. reuse an existing INSERT statement
$db.execute-many('insert into measurements values (?, ?, ?)',
                 ((5, 'w', 5.5), (6, 'v', 6.5)));
```

`insert-rows` and `execute-many` wrap the load in a transaction, so a batch that
fails halfway leaves nothing behind. When the caller already has a transaction
open (an explicit `begin`), the batch joins it instead of opening its own and
nothing is committed early:

```raku
$db.execute('begin');
$db.execute('insert into measurements values (7, NULL, 7.5)');
$db.insert-rows('measurements', ((8, 'u', 8.5),));   # joins the transaction
$db.execute('rollback');                              # both rows are gone
```

`execute-many` only routes to the appender when the statement is a plain
`INSERT INTO t [(cols)] VALUES (?...)`. A batch that adds `ON CONFLICT` is
*staged* instead: from 32 rows up, the values go into a TEMP table through the
appender and are replayed with one `INSERT ... SELECT` that keeps the clause, so
a 20k-row upsert costs 0.25 s rather than 55.6 s. `DO NOTHING` always takes that
path; `DO UPDATE` takes it only when the clause names its conflict target and
the batch has no duplicate key -- the bulk form updates a key once where the
prepared path updates it per row, so anything else stays sequential. Everything
left (`RETURNING`, expressions, sub-selects, `ON CONSTRAINT`, narrow batches)
still runs as one prepared statement per row inside the same transaction, and a
failure in a staged batch rolls back with no staging table left behind.

## Session options

`threads`, `memory_limit` and `preserve_insertion_order` are pushed to every
connection the driver opens, and can be changed on a live session:

```raku
my $db = DB::DuckDB.new(:threads(8), :memory-limit('2GB'),
                        :preserve-insertion-order(False));

$db.set-session-option('threads', 4);        # applied immediately
```

`preserve_insertion_order => False` is only sent when it is turned off: `True`
is DuckDB's own default, so the driver stays out of the engine's way. Set any
other option with `set-session-option`.

## Zero-boxing column views

`column()` always builds a plain `Array` and boxes every value into a Raku
scalar. When you only want to scan or aggregate a column, `column-view` keeps
DuckDB's own vector buffer and reads through it in place:

```raku
my $r = $conn.query('SELECT id, amount, note FROM sales');
my $ids = $r.column-view(0);

say $ids.type;      # BIGINT
say $ids.elems;     # 5000
say $ids.int64(0);  # no Mu allocation per value
say $ids.sum;       # walks the raw buffer, skips NULLs
say $ids.min;       # same
say $ids.count;     # non-NULL values
say $ids.to-list;   # List, built with a typed loop (fastest full read)
say $dates.at(0, :typed);  # Date, for DATE / TIMESTAMP columns
say $ids.slice(0, 100);   # a narrower view, no copy
```

`int64` / `num64` / `is-null` / `sum` / `min` / `max` / `count` / `mean` /
`slice` work on the numeric types without boxing. `at($i)` returns a proper
Raku value for every type, but for `VARCHAR`, dates, wide `DECIMAL` and nested
types it goes through the same decoder the boxed path uses, so `to-list` is the
better choice there. `$r.column-views` returns all columns at once.

Temporal columns can skip the text form: `at($i, :typed)` and `to-list(:typed)`
answer DATE as a `Date` and every `TIMESTAMP_*` as a UTC `DateTime` (fractional
seconds kept as a `Rat`), built from the integer payload rather than formatted
and re-parsed. TIME and TIMESTAMPTZ keep their text, because there is no
lossless Raku value for a bare clock or an explicitly zoned stamp. A sliced or
materialised view rebuilds the value from DuckDB's own text, so `(BC)` and
`infinity` come back as the text they went in as instead of a wrong date.

The view borrows the result's memory: it is only valid while `$r` is alive, and
reading it after `$result.finish` raises.

### Zero-copy text columns

Text is the one type the view above still boxes, because `at($i)` on a
`VARCHAR` has to return a `Str`. `.text` gives the column back as bytes
instead: one payload buffer, plus a byte offset and a length per row, filled by
the shim in a single copy per chunk.

```raku
my $t = $conn.query('SELECT note FROM sales').column-view(0).text;

say $t.type;          # VARCHAR
say $t.elems;         # 1_000_000
say $t.byte-size;     # 41_896_900 -- one buffer, one allocation
$t.bytes;             # the whole payload, back to back (Buf, borrowed)
$t.offset(7);         # byte offset of row 7 inside .bytes
$t.length(7);         # byte length (UTF-8 bytes, not characters)
$t.bytes-at(7);       # Buf slice -- no decode, no Str
$t.at(7);             # Str, one allocation, only when you ask for it
$t.is-null(7);        # signals NULL; an empty string is length 0 but not NULL
$t.to-list;           # List of Str/Any, for compatibility with column()
$t.slice(100, 500);   # rows 100..599 as another TextColumn -- no copy
```

`slice` is the cheap window: the new column borrows the same payload buffer and
the same three tables and only remembers where it starts, so it costs one small
object however big the parent is -- a thousand 100k-row slices of a million-row
column take 23 ms in total. `.bytes` on the slice still reports the whole parent
payload (trimming it would be the copy the view avoids); `.byte-size` reports
the window's own extent. Slices of slices add their start rows, an out-of-range
window raises, and a NULL inside the window still reads back as `Any`.

Use it for anything that does not need a string object per row: grouping,
hashing, length filters, writing the column back out, feeding a parser. On one
million 40-byte strings the export costs 67 ms against 1.9 s for `to-list`
(3.9 s before it learned to decode whole ASCII windows), and the rows read back
byte for byte identical across chunk seams (`t/16-text-column.rakutest`).

The tables are borrowed: `offset-buffer` / `length-buffer` / `null-buffer`
expose the raw `CArray`s so a hot loop can index them without a method call per
row. Treat them as read-only, and keep the result alive while the column is in
use -- a streamed result recycles its chunks, so `.text` on a streaming view is
a detached copy.

`BLOB` columns work the same way; the payload is the bytes as stored. Other
types are refused: numeric columns already have `int64` / `num64` / `sum`, and
temporal and `DECIMAL` values are rendered from their integer payload. When the
C shim is unavailable, `.text` refuses with an explanation rather than
silently falling back to the slow path -- check
`DB::DuckDB::TextColumn::text-column-available()` first if you want to branch.

## Parallel decoding

`column-arrays` decodes the columns of a materialised result on several threads
by default, once the result is big enough to pay for them:

```raku
my $r = $conn.query('SELECT * FROM wide_table');

my @cols = $r.column-arrays;                        # parallel when it pays
my @four = $r.column-arrays(:degree(4));            # capped at 4 workers
my @one  = $r.column-arrays(:!parallel);            # force the serial path
```

A worker is only used if it gets at least 20k rows, so a 25k-row result stays on
one thread and a million-row one spreads over the pool; `:degree` caps the count
on top of that. `DB_DUCKDB_NO_COLUMN_PARALLEL=1` switches the default off for a
whole process.

The work is cut into `(column, chunk-span)` tasks and each worker writes into
that column's preallocated array, so no memory is shared between workers and no
value is copied twice. A result with more columns than workers spreads the
columns over the workers; a result with fewer columns splits each column's
chunks into contiguous spans and stitches the spans back in chunk order, so row
order is preserved and the output is identical to the serial path (checked
value by value in `t/08-parallel.rakutest`).

`DB_DUCKDB_DEGREE` in the environment sets the default worker count.

Measured on this machine, 1M rows, wall clock for the whole decode (median of
four runs per shape; this is a 24-core box and the spread between runs is wide
because GC dominates). The numbers predate the weight-based split below; the
one- and two-column rows are exactly the lopsided cases that motivated it, so
treat them as a lower bound:

| shape | serial | `:parallel` | speedup |
|---|---|---|---|
| 1 x `VARCHAR` | 4.7 s | 1.9 s | 2.4x |
| 5 numeric columns | 8.5 s | 3.4 s | 2.2x |
| 4 numeric columns | 5.2 s | 2.6 s | 2.0x |
| 7 mixed columns | 22.5 s | 10.5 s | 2.1x |
| 3 mixed columns | 9.2 s | 6.5 s | 1.3x |
| 2 mixed columns | 5.5 s | 4.8 s | 1.1x |

Parallelism pays where the per-value cost is high (`VARCHAR` unpacking,
`DECIMAL` formatting) or where there are many columns to spread. Results with
only one or two columns are bound by the boxing step and by Raku's GC -- both
shared between threads -- so the gain there is small. Below 20 000 rows the
driver stays serial, because setting the workers up costs more than the decode.

### Splitting by column weight

Cutting every column into the same number of spans spreads the *work* evenly
only if the columns cost the same. They do not: a `VARCHAR` value costs about
four BIGINTs to decode, a nested value about three, `DECIMAL`/`HUGEINT` about
two. With two such columns and four workers, an even split of spans leaves the
cheap column's worker idle while the expensive one still runs, and a lopsided
result can end up *slower* than the serial path.

So the span count per column is `ceil(nt * w / sum-of-weights)`, capped at both
`nt` and the chunk count, with weights `VARCHAR` 4, `LIST`/`STRUCT`/`MAP`/
`ARRAY`/`UNION` 3, `DECIMAL`/`HUGEINT` 2, everything else 1. Plain numeric
columns are left whole unless they are the only column, because stitching a
numeric column back costs a copy that cancels the gain.

Measured on 200k rows of `BIGINT + VARCHAR(40)`, 20 interleaved rounds in one
process, paired median against the serial path:

| workers | even split | by weight |
|---|---|---|
| `:degree(2)` | 1.22-1.24x | 1.20-1.38x |
| `:degree(4)` | 0.92-0.93x | 1.29-1.32x |

The point is the second row: adding workers no longer costs speed. Absolute
ratios move by up to 20% with machine load.

Two cases cannot be split and fall back to the serial path automatically: a
streaming result (its chunks belong to the stream, not to a chunk list) and a
result that was already partially read.

## Streaming results

By default a result is materialised -- DuckDB runs the query to completion
before the first row is available. `:stream` keeps the query pulling row
batches on demand:

```raku
my $r = $conn.query('SELECT * FROM huge', :stream);

$r.is-streaming;         # True

for $r.chunks -> $chunk {
    # $chunk.size         -- rows in this vector (<= 2048, the last one shorter)
    # $chunk.column-count
    my $view = $chunk.column-view(0);
    ... $view.sum ...
    my @row  = $chunk.row(0);      # boxed values, for the odd row
    my @rows = $chunk.to-rows;     # boxed, for small chunks
}
```

**A chunk is valid only until the next one is pulled.** DuckDB recycles the
vector, so the driver expires the previous chunk (and every view it handed out)
before fetching the next one; any read through a stale chunk or view raises
instead of returning another row's data. That is the trade-off for not copying:
consume inside the loop body and let the chunk go. `$chunk.is-valid` reports
whether the driver has moved on; `.size` and `.column-count` stay readable
either way, because they are plain numbers rather than pointers.

Measured on this machine, 1M rows x 3 columns (`bigint`, `double`, `varchar`):
first usable chunk after **12 ms**, versus **5.8 s** to materialise the whole
result through `column-arrays`; summing the `bigint` column while streaming
takes 3.9 s and agrees exactly with `SELECT sum(a)`.

### Prefetching

`chunks` used to take turns with the engine: fetch, decode, fetch. The engine
computes while you decode nothing, and sits idle while you decode. A worker
thread now pulls the next chunk while the caller works on the current one.

The queue is two slots deep, not more: a chunk is ~235 KB and decoding it costs
several times what fetching it costs, so a deeper queue would only raise time
to first chunk and memory. Two slots are enforced with a token channel -- the
worker takes a token before fetching, the caller returns one when it drops the
previous chunk -- so at most two chunks are alive.

Nothing changes at the API: `free` stops the worker, joins it and destroys
anything still queued before the result goes away, so a stream abandoned after
one chunk is safe. Measured 300k rows, 7-9 interleaved rounds: **1.06-1.13x**
on a decode-bound full scan, **1.03-1.18x** on an engine-bound streaming
aggregation, time to first chunk unchanged, RSS plateau +17 MB.
`DB_DUCKDB_NO_PREFETCH=1` restores the one-at-a-time path.

## Arrow views

`DB::DuckDB::Arrow` exposes a chunk of a materialised result through the Arrow C
data interface, so the buffers can be handed to another component instead of
being boxed into Raku values here:

```raku
use DB::DuckDB::Arrow;

my $r = $conn.query('SELECT id, tags FROM items');

my $chunk = arrow-chunk($r, 0);          # 0-based chunk index, 2048 rows each
my $col   = $chunk.column(1);            # LIST<VARCHAR>
say $chunk.length;                       # rows in this chunk
say $chunk.columns;
say $col.null-count;
say $col.children[0].strs;               # the child vector of the list
$chunk.free;                             # idempotent

with-arrow-chunks($r, -> $c, $i { ... });  # frees every chunk for you
```

`with-arrow-chunks` hands each chunk and its index to the block and frees the
chunk on the way out, so nothing leaks if the block dies. Chunks are numbered
the same way as `arrow-chunk`.

`Column` gives typed access without boxing: `.ints`, `.int32s`, `.nums`,
`.bools`, `.strs` (each returns a `List`), `.is-null($i)`, `.children` for
recursing into `LIST`/`STRUCT`, plus the raw `.buffer($j)` pointer and
`.length` for talking to something that wants the buffers themselves. `Chunk`
also exposes `.addr` for the `ArrowArray*` struct.

There is no `LOAD arrow` and no NanoArrow dependency: the Arrow C data interface
is two ten-field structs, so `ArrowArray` / `ArrowSchema` are declared directly
as `repr('CStruct')`.

This is not a faster read path. `strs` has to follow the offset buffer and copy
each value out, which on 200k rows is 6.3 s against 1.7 s for the bulk
`join-varchar` decoder used by `column()` -- about 3.8x slower. Use it for
nested types, for passing data to other libraries, or when you want the Arrow
buffers as they are.

Releasing an Arrow array means calling a callback pointer stored inside the
struct. Doing that from Raku naively leaked on Windows (~174 MB to ~711 MB over
30k releases, because each `nativecast` to a callable builds a stub the VM
never reclaims). Two things fix it: `resources/duckdb-arrow-shim.c` moves the
indirect call to C, and -- for machines without a compiler -- the in-Raku path
now caches one callable per release pointer instead of building one per chunk.
DuckDB uses a single release function for every chunk it produces, so that
cache holds exactly one entry; the no-shim path measured +69 MB over ~1500
releases before and +2 MB after (GC noise). It is built on demand next to the
VARCHAR shim, under a per-process name that is renamed into place only when the
shared name is free, so several processes can build at once (same hardening the
VARCHAR shim received in 0.7.1).

`DB_DUCKDB_ARROW_SHIM` points at a prebuilt library, `DB_DUCKDB_NO_ARROW_SHIM=1`
forces the in-Raku path, `DB_DUCKDB_ARROW_DEBUG=1` logs which release path is
taken. When no compiler is available the driver checks that a shim library
actually exists before trusting it -- an earlier version only `require`d the
binding module, which compiles fine without the library and then failed on the
first release call.

A chunk fetched from the result is owned by the caller: `arrow-chunk` destroys
it right after the conversion has copied the data into Arrow-owned buffers, so
the Arrow view does not keep the DuckDB result alive.

## Parquet

DuckDB 2.0's C API has no Parquet entry point -- there is nothing named
`parquet` in `duckdb.h`. The format is reachable only through SQL, so the
driver wraps `read_parquet(...)` and `COPY ... TO ... (FORMAT PARQUET)` instead
of pretending a binding exists. Rows read from a file travel through the same
chunk/vector path a table scan uses, which is why the VARCHAR, temporal,
DECIMAL and ENUM C channels apply unchanged.

```raku
# read: a file, a glob, or a list of files
for $db.read-parquet('sales.parquet').hashes -> %row { ... }
$db.read-parquet('data/*.parquet', :union-by-name, :filename).column-views[0].sum;
$db.read-parquet(<a.parquet b.parquet>, :union-by-name);

# write: a table, or a query
$db.write-parquet('out.parquet', 'events');                    # returns rows written
$db.write-parquet('out.parquet', 'select * from events', :sql);
$db.write-parquet('out.z.parquet', 'events', :compression<zstd>, :row-group-size(250_000));
$db.write-parquet('out/part', 'events', :partition-by<day region>);

# footer only
$db.parquet-metadata('out.parquet')[0]<num_rows num_row_groups format_version>;
$db.parquet-schema('out.parquet');
$db.parquet-row-groups('out.parquet');
```

Read paths bind the file names as parameters, one placeholder per path, so a
file whose name contains a quote needs no escaping. Writes have to interpolate
the target and the table name, so both go through the same literal and
identifier quoting used internally, and `compression` is validated against the
codec list before the statement is built.

Options: `:union-by-name`, `:filename`, `:hive-partitioning`, `:columns`,
`:stream` when reading; `:sql`, `:schema`, `:compression`, `:row-group-size`,
`:partition-by`, `:overwrite-or-ignore` when writing. `COPY` replaces an
existing file by default; `:overwrite-or-ignore` matters for partitioned
directories that already hold files. `:row-group-size` is only honoured at
2048-row vector boundaries, so the real groups come out as multiples of that.

Measured on 1M rows x 4 columns (best of 3, single process, zstd):

| operation | time | notes |
|---|---:|---|
| `write-parquet` (COPY) | 0.033 s | 2.05 MiB, 0.03 µs/row |
| `execute-many`, 20k rows | 0.226 s | 11.3 µs/row, ~400x slower per row |
| read `BIGINT`, 1M values | 0.58 s | table scan 0.58 s |
| read `VARCHAR`, 1M values | 1.15 s | table scan 1.20 s |
| read all four columns | 5.08 µs/row | table scan 5.05 µs/row |
| first streamed chunk | 19.8 ms | table scan 37.7 ms |
| `parquet-metadata` (footer) | 0.6 ms | does not read data pages |

The reader is not a separate decode path, and does not need to be: reading a
Parquet file costs what reading the equivalent table costs, within noise.

## Environment switches

Every switch is read once per process (or per call, where noted) and is a plain
environment variable — no config file. `=1` means "any non-empty value".

**Output shape**

| variable | effect |
|---|---|
| `DB_DUCKDB_DEGREE` | default worker count for `column-arrays(:parallel)` |
| `DB_DUCKDB_NO_COLUMN_PARALLEL=1` | make the *default* for `column-arrays` single-threaded; per-call `:parallel` still wins |
| `DB_DUCKDB_NO_PREFETCH=1` | one chunk at a time in `chunks`, no prefetch worker |
| `DB_DUCKDB_NO_COLUMN_VIEW=1` | `column-views` falls back to plain `column-arrays` (decodes eagerly, holds nothing) |

**C decoding shims** (all of them fall back to the pure Raku path, which is slower but
tested against DuckDB's own textual rendering)

| variable | effect |
|---|---|
| `DB_DUCKDB_NO_VARCHAR_SHIM=1` | no C shim at all, pure Raku `VARCHAR` decoder |
| `DB_DUCKDB_NO_VARCHAR_JOIN=1` | per-value `VARCHAR` decoding, no join buffer |
| `DB_DUCKDB_NO_TEMPORAL_SHIM=1` | render `DATE` / `TIME` / `TIMESTAMP*` in Raku instead of the C renderer |
| `DB_DUCKDB_NO_DECIMAL_SHIM=1` | render `DECIMAL` in Raku instead of the C renderer |
| `DB_DUCKDB_NO_TEXT_WHOLE=1` | `TextColumn.to-list` back to the per-row ASCII path instead of the whole-window decode |

**Libraries and build**

| variable | effect |
|---|---|
| `DB_DUCKDB_LIB` | path of the libduckdb to load |
| `DB_DUCKDB_SHIM` | path of a prebuilt `VARCHAR` bridge library, skips the build |
| `DB_DUCKDB_ARROW_SHIM` | path of a prebuilt Arrow release shim |
| `DB_DUCKDB_NO_ARROW_SHIM=1` | never build or use the Arrow shim; release through the cached in-Raku callback |
| `DB_DUCKDB_ARROW_DEBUG=1` | log which Arrow release path is taken |
| `DB_DUCKDB_VARCHAR_SHIM` | path of the compiled `VARCHAR` shim to reuse |
| `DB_DUCKDB_SHIM_CACHE` | directory holding built shims (default: under the dist cache) |
| `DB_DUCKDB_NO_CACHE_SWEEP=1` | never delete stale shim libraries from the cache |
| `DB_DUCKDB_CC` | compiler to build the shims with, when `cc` / `gcc` / `clang` is not it |
| `DB_DUCKDB_SHIM_TRACE=1` | log shim build decisions (`VARCHAR` shim section) |

## Type mapping (DuckDB → Raku)

| DuckDB type | Raku value |
|---|---|
| `BOOLEAN` | `Bool` |
| `TINYINT` / `SMALLINT` / `INTEGER` / `BIGINT` | `Int` |
| `UTINYINT` / `USMALLINT` / `UINTEGER` / `UBIGINT` | `Int` |
| `FLOAT` / `DOUBLE` | `Num` |
| `VARCHAR` | `Str` |
| `NULL` (any type) | `Any` (undefined) |
| `DATE`, `TIME`, `TIMESTAMP`, `TIMESTAMP_S/_MS/_NS`, `TIME_NS` | `Str` in DuckDB's ISO format, rendered from the integer payload (byte identical to `CAST(x AS VARCHAR)`) |
| `TIME_TZ`, `TIMESTAMP_TZ`, `TIMESTAMP_TZ_NS` | `Str` in DuckDB's own ISO rendering (session time zone honoured) |
| `DECIMAL` | `Str`, exact and lossless, rendered from the unscaled integer |
| `ENUM` | `Str`, looked up in the type's dictionary (one `Str` per label) |
| `HUGEINT` / `UHUGEINT` | `Str`, exact and lossless |
| `BLOB`, `UUID`, `INTERVAL`, `LIST`, `STRUCT`, `MAP`, `UNION`, `BIT`, `BIGNUM`, `ARRAY` | `Str` in DuckDB's textual rendering |
| `SQLNULL` type | `Any` |

Raku → DuckDB (bind): `Bool` → `BOOLEAN`, `Int` → `BIGINT`, `Num` / `Rat` / `FatRat` → `DOUBLE`,
`Str` → `VARCHAR`, `Blob` → `BLOB`, undefined → `NULL`, anything else → its `.Str`.

Dates and decimals are deliberately *not* auto-coerced into Raku `Date`/`DateTime`/`Rat`:
DuckDB's own textual rendering is exact for every type, while the `duckdb_value_*` family that
would return real date/decimal structures is deprecated and partly by-value. Exactness beats
convenience here — convert at the call site when you need it.

## Design notes

* **One database instance per driver.** `connect` creates connections that all share a single
  `duckdb_database`. Opening `:memory:` twice would silently create two independent databases,
  and opening a file path twice trips DuckDB's instance/file locking, so the driver opens once
  and guards it with `$!opened`.
* **`duckdb_result` is mapped as a fixed-layout struct** (48 bytes on LP64: three `idx_t` plus
  three pointers) and its size is asserted at load time, so an ABI change fails loudly instead
  of corrupting memory.
* **Results are read through the chunk/vector API.** `duckdb_result_chunk_count` /
  `duckdb_fetch_chunk` hand back a `duckdb_data_chunk` pointer, and each vector's data
  buffer is read in place; the structs living inside those buffers are dissected field by field
  through `uint32`/`uint64` views, since NativeCall faults on `CStruct` array element access on
  Windows. No struct is ever returned or taken by value, with one deliberate exception: the two
  accessors that still declare a by-value `duckdb_result` are called through
  `DB::DuckDB::Bridge`, which passes the same 48 bytes as a pointer. NativeCall's by-value
  marshalling happens to work under MS x64 but is miscompiled under SysV, where it surfaced as
  `unique_ptr` NULL crashes inside DuckDB.
* **Values are rendered exactly like DuckDB's own `CAST(x AS VARCHAR)`**, not coerced into Raku
  `Date`/`DateTime`/`Rat`. `TIMESTAMP_TZ` is the one piece of session state involved: the vector
  holds UTC microseconds, so the instant is shifted by the session zone's offset (synced from
  `strftime(..., '%z')` on construction and refreshed after `SET TimeZone`) and only then
  labelled with that offset.
* **Prepared statements are cached per connection** by the `DB` role; statements are destroyed
  when their connection is dropped. `close` also tears down connections the caller forgot to
  `finish`, because DuckDB will not release the file lock until every connection and prepared
  statement is gone.
* **`VARCHAR` is unpacked in bulk by a C shim.** Decoding a string column one
  `duckdb_string_t` at a time and `Buf.push`-ing each payload was the slowest
  thing in result reading (and pure-Raku batching was slower still, because
  every payload still had to be copied byte by byte). `duckdb-varchar-shim.c`
  snapshots the vector's `duckdb_string_t` array once, then copies every payload
  into one flat buffer while writing an offsets table, a lengths table and a
  NULL bitmap; Raku decodes that buffer once and slices values out with
  `substr`. Measured 2.7-3.4x on `column()` and 1.5-1.8x on `arrays()`/`hashes()`
  at 100k-1M rows.
  The shim never trusts the pointers it is handed: the `string_t` array is
  copied into memory it owns before being walked, out-of-line payloads are read
  through a probing copy, a payload longer than 16 MiB (the signature of a
  selection buffer mistaken for a string vector) and any unreadable pointer make
  it return `DKD_NOT_FLAT`, which is the signal to fall back to the per-value
  decoder for that chunk.
* **Temporal columns are formatted by the driver, not by DuckDB.** A `DATE` is an int32 day
  count and the `TIMESTAMP*` / `TIME*` types are int64 counters, so the text can be computed
  directly from the payload instead of asking DuckDB to `CAST` each value. `dkd_temporal_join`
  in the same shim renders a whole chunk into one `0x01`-separated buffer (a `NULL` becomes
  `0x02`) that Raku splits once, and a pure Raku renderer produces exactly the same text when
  no compiler is around. Measured on 1M rows against DuckDB's own cast: `DATE` 8.38 → 1.38,
  `TIMESTAMP` 23.35 → 1.22, `TIME` 4.58 → 1.27 µs/value, and a three column `arrays()` read
  35.88 → 4.78 µs/row (the cast itself costs 1.30 µs/value, so this is the floor). The text
  matches the `CAST` to the byte, including the `(BC)` suffix on
  pre-epoch dates, `24:00:00`, and fractional seconds with trailing zeros removed.
  `TIME_TZ` / `TIMESTAMP_TZ` / `TIMESTAMP_TZ_NS` stay on DuckDB's own rendering because their
  text carries the session zone. Set `DB_DUCKDB_NO_TEMPORAL_SHIM=1` to force the Raku renderer.
* **DECIMAL is rendered from its unscaled integer, in C.** A `DECIMAL` is an integer plus a
  scale, and DuckDB prints exactly `scale` digits, trailing zeros included. Doing that in Raku
  turned out to cost more per value than asking DuckDB to format it (1.71 µs just for the
  formatting), so `dkd_decimal_join` does the digit loop instead, for the 64 bit and the 128 bit
  widths alike (the latter behind `__SIZEOF_INT128__`; without it the Raku path stays). On 1M
  rows: `DECIMAL(9,2)` 2.45 → 1.23, `DECIMAL(18,4)` 3.52 → 1.24, `DECIMAL(38,10)` 9.42 → 1.48
  µs/value, against 1.24–1.42 for `CAST(x AS VARCHAR)`. One quirk is reproduced on purpose: the
  all-fraction widths print without an integer digit (`DECIMAL(9,9)` → `.5`), where
  `DECIMAL(10,9)` → `0.5`. Set `DB_DUCKDB_NO_DECIMAL_SHIM=1` to force the Raku renderer.
* **ENUM is a dictionary lookup, not a render.** The keys are read as the narrowest unsigned
  integer that fits the dictionary (`duckdb_enum_internal_type` says which) and the text comes
  from `duckdb_enum_dictionary_value`, read once per chunk. 2.71 → 0.61 µs/value on 1M rows,
  which is faster than the VARCHAR channel because rows sharing a label share one `Str` and
  nothing is allocated per value.
* **Works on Windows without an import library**: the DLL is loaded at runtime, so the MSVC
  `.lib` that ships with the C API is not needed.

## Tests

```sh
raku -Ilib t/01-basic.rakutest    # types, placeholders, transactions, errors
raku -Ilib t/02-file.rakutest     # file database, multiple connections, reopen
raku -Ilib t/03-types.rakutest    # every 2.0 type vs. DuckDB's own text rendering,
                                  # NULLs, chunked results, session time zone, uuid
raku -Ilib t/04-varchar-shim.rakutest   # the VARCHAR shim, including its fallbacks
raku -Ilib t/05-appender.rakutest       # appender, insert-rows, type mapping
raku -Ilib t/06-column-view.rakutest    # column-views vs column-arrays, all types
raku -Ilib t/07-streaming.rakutest      # chunks, 2048-row vectors, chunk lifetime,
                                        # prefetch vs one-at-a-time
raku -Ilib t/08-parallel.rakutest       # parallel column decode vs the serial path,
                                        # including lopsided results at 2/3/4/8 workers
raku -Ilib t/09-lazy-view.rakutest      # lazily decoded views and their cleanup
raku -Ilib t/10-varchar-join.rakutest   # the join-buffer VARCHAR decoder and its fallbacks
raku -Ilib t/11-arrow.rakutest          # Arrow C data interface views, nested types, release
raku -Ilib t/12-temporal.rakutest       # date/time/timestamp rendering vs DuckDB's own cast,
                                        # pre-epoch and extreme values, NULLs, lazy views
raku -Ilib t/13-decimal-enum.rakutest   # decimal widths 4..38 incl. the 128 bit slot and the
                                        # all-fraction quirk; enum dictionaries and NULLs
raku -Ilib t/14-parquet.rakutest        # parquet round trip, path shapes (quotes, unicode,
                                        # glob, list), options, footer metadata, partitioning
raku -Ilib t/15-shim-cache.rakutest     # cache sweep: what is removed, what is kept,
                                        # and that it never empties a usable cache
raku -Ilib t/16-text-column.rakutest    # the zero-copy text column: byte buffer, offsets,
                                        # slice windows, whole-window decode, NULLs
```

16 files, 526 top-level assertions (623 counting subtests), green on Rakudo 2026.09,
Windows 11 and WSL Ubuntu.

No `prove` harness is required -- each file is a standalone `Test` script. To run the whole
gate in one go and get a verdict:

```sh
pwsh -File tools/release-check.ps1          # all 16, per-file ok / not ok, PASS or FAIL
pwsh -File tools/release-check.ps1 -Filter 16-text   # one file
```

### The `libunwind: pc not in table` line on Windows

If the `libduckdb` you load was built with llvm-mingw, an error raised inside the library can
leave one or two lines of unwinder chatter on stderr:

```
libunwind: pc not in table, pc=0x7ffc...1d0e
```

Those lines come from `libunwind.dll`, not from this driver: C++ exception handling inside the
library walks the stack on its way out, runs into the MoarVM frame the call came through, and
that frame carries no unwind table. Nothing about the result changes - the call still fails with
`DB::DuckDB::Error`, the exit code stays 0 and every assertion still passes. The trace strings
are compiled into llvm-mingw's libunwind (its static `libunwind.a` carries them too), and no
environment variable turns them off. A libduckdb built with MSVC - the official Windows binaries,
for example - links neither libc++ nor libunwind and stays silent, so point `DB_DUCKDB_LIB` at
such a `duckdb.dll` if the chatter gets in the way. `tools/release-check.ps1` strips the lines
before counting assertions and says how many it removed.

A runnable five-line tour lives in `examples/quickstart.raku`:

```sh
raku -Ilib examples/quickstart.raku
```

## License

Artistic-2.0 - the full text is in [LICENSE](LICENSE).