# HTTP-Core **Repository Path**: skyter10086/raku-HTTP-Core ## Basic Information - **Project Name**: HTTP-Core - **Description**: No description available - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-11-04 - **Last Updated**: 2026-11-04 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README --- AIGC: Label: "1" ContentProducer: 001191440300708461136T1XGW3 ProduceID: aed9bd2c2f3c6f83a54981e0ddb92ca2_2ff46853d75a11f18947525400de85a5 ReservedCode1: aCy5yttkD9A2gcPJBug8DKlMg8u9pcZZNTrT80qVrEaBhbmAj0FBt7ImjkXwsgd8v5ISlUYZ5Dmx1KDIG1ZNnVMN69icYZ5WsE72c+D+GEZv15zi1TJxqdJvR02xEFSZNRrgKTgjz3mKdPEnfDOYnYDX128/TMLCvPZL42SukyScjkoWULPJ+OhJweQ= ContentPropagator: 001191440300708461136T1XGW3 PropagateID: aed9bd2c2f3c6f83a54981e0ddb92ca2_2ff46853d75a11f18947525400de85a5 ReservedCode2: aCy5yttkD9A2gcPJBug8DKlMg8u9pcZZNTrT80qVrEaBhbmAj0FBt7ImjkXwsgd8v5ISlUYZ5Dmx1KDIG1ZNnVMN69icYZ5WsE72c+D+GEZv15zi1TJxqdJvR02xEFSZNRrgKTgjz3mKdPEnfDOYnYDX128/TMLCvPZL42SukyScjkoWULPJ+OhJweQ= --- # HTTP::Core A minimal HTTP/1.x **request-head parser** for Raku: a hand-written C kernel bound through NativeCall with **no shim layer**. This is a probe, not a framework. It exists to answer one question — *how far does a raw NativeCall binding get before you need an intermediate layer?* — and then to be the parsing primitive a real server can build on. On top of that kernel sit a codec, a router, a server and a client, so the answer is measured rather than asserted: the whole stack, from `parse-request` to a keep-alive connection pool, is built out of the same nine exported C functions. Suite status: **17 test files, 101 tests, all passing** on Rakudo 2026.09 / MoarVM. ## Contents | Section | What is in it | | --- | --- | | [Install](#install) | Clone, build the kernel, point the binding at it | | [Quick start](#quick-start) | A server and a client in five lines each | | [What it does](#what-it-does) | The parser's contract, and what it refuses to do | | [Why no shim](#why-no-shim) | The NativeCall hazards the kernel design avoids | | [Byte offsets, not graphemes](#byte-offsets-not-graphemes) | The one trap that silently shifts every offset | | [Server](#server) | Routing, dispatch, keep-alive policy | | [Client](#client) | Requests, redirects, pooling, TLS | | [Examples](#examples) | Runnable scripts, each with `--help` | | [Tests](#tests) | How to run the suite, and what each file proves | | [Layout](#layout) | Where everything lives | | [API reference](docs/API.md) | Every public method, one table per module | | [Troubleshooting](#troubleshooting) | The failures that actually happen | | [Benchmarks](#benchmarks) | Numbers, and the post-mortems behind them | | [Limits](#limits) | What is deliberately missing | ## Install Rakudo 2026.09 (MoarVM) or newer. Two dependencies, `Compress::Zlib` and `JSON::Fast`; the test suite additionally uses `IO::Socket::Async::SSL`. ```sh git clone https://gitee.com/skyter10086/raku-HTTP-Core.git cd raku-HTTP-Core raku -I. Build.rakumod # build the kernel; CC=... overrides the compiler raku -Ilib -e 'use HTTP::Core; say core-version(); say library-path()' ``` `Build.rakumod` writes `http_wire.dll`, `libhttp_wire.so` or `libhttp_wire.dylib` into `resources/libraries/`. That directory is ignored by git on purpose — a compiled kernel belongs to whoever compiled it — so a fresh clone has no library until you run the build. The binding searches, in order: `%*ENV`, then `resources/libraries/`, then the platform loader path. If you keep the library somewhere else, name it explicitly: ```sh HTTP_CORE_LIB=/opt/http-core/libhttp_wire.so raku -Ilib -e 'use HTTP::Core' ``` Using it from this tree needs nothing else: `-Ilib` is the whole install. `zef install .` additionally ships the `provides` and `resources` entries from `META6.json`, which is convenient once the kernel is built but no substitute for it — build first, install second. ### Windows No Visual Studio, no WSL, no MSYS shell: Rakudo's Windows build runs the script directly and `zig cc` emits a plain `.dll` that NativeCall loads as-is. ```powershell git clone https://gitee.com/skyter10086/raku-HTTP-Core.git cd raku-HTTP-Core raku -I. Build.rakumod # picks zig cc / cc / gcc / clang raku -Ilib -e "use HTTP::Core; say core-version(); say library-path()" zef install . # builds, tests, installs raku -e "use HTTP::Core; say core-version(); say library-path()" # now from the site repo ``` Four differences from the POSIX recipe above: - **`CC` is an environment variable, not a command prefix.** `CC="gcc" raku …` is POSIX shell syntax and does not exist in PowerShell. Set it first, then run the same command: `$env:CC = 'gcc'` (`set CC=gcc` in cmd.exe, `Remove-Item Env:\CC` to get the default back). - **The compiler is picked, not assumed.** With `CC` unset, `Build` tries `zig cc`, `cc`, `gcc`, `clang` and keeps the first one that actually produces a library, so a machine with only llvm-mingw and one with only zig both build without being told anything. It compiles rather than probing with `--version`: WSL inherits the Windows `PATH`, so a `zig.exe` sitting there answers `zig version` cheerfully and then cannot build an ELF target — the build falls through to `cc` instead of stopping on the first candidate. `zig cc` needs nothing else installed: a single `zig.exe`, no Visual Studio and no runtime redistributable; the DLL imports nothing beyond `KERNEL32.dll` and the UCRT that ships with Windows 10/11. `gcc` and `clang` from llvm-mingw work too and produce a smaller DLL with the same imports. MSVC `cl` is untested and unnecessary — it wants a developer prompt and its DLL would add a `vcruntime` dependency. - **The kernel is `http_wire.dll`, no `lib` prefix.** That is the PE convention and `_find-lib` follows the platform (stem *and* extension), so a `libhttp_wire.so` left in `resources/libraries/` by a WSL session is never picked. `zig cc` also drops `http_wire.lib` and `http_wire.pdb` next to it; both are ignored by git and neither is packaged. - **A Windows checkout is byte-identical to a WSL one.** `.gitattributes` pins `* text=auto eol=lf`, so `core.autocrlf` does not matter and the kernel never sees a stray `\r`: the 14 `\r\n` sequences in `http_wire.c` are spelled as escapes, and the file itself has no CR bytes. If `zef install .` finishes with `Testing [OK]` and the last command prints an absolute path under `C:\rakudo\share\perl6\site\resources\`, the install is complete. Anything else — see [Troubleshooting](#troubleshooting). ## Quick start Serve something: ```sh raku -Ilib examples/serve.raku --port=8080 curl -s http://127.0.0.1:8080/health ``` Request something: ```sh raku -Ilib examples/client.raku --url=http://127.0.0.1:8080/health --verbose raku -Ilib examples/client.raku --self-test # no network, no second terminal ``` Both scripts parse their own arguments; `--help` prints the generated usage. Option values may be written either way: `--port 8080` and `--port=8080` are equivalent, and so are `--timeout 5` and `--timeout=5`. Raku's `MAIN` binds a space-separated value only to a `Str` option, so the numeric options are declared `Str` and converted by a small `int-option` / `num-option` helper; a value that is not a number is reported with its own name instead of the usage block. ## What it does - Parses a request line and header block out of a flat byte buffer. - Allocates nothing: every returned string is a copy taken from *your* buffer. - Reports the head length so the body can be sliced from the same buffer. - Distinguishes **`Incomplete`** (feed more bytes) from **`ParseError`** (give up). - Runs on MoarVM safely because it does no callbacks and owns no event loop. It does *not* do chunked transfer decoding, HTTP/2, URL decoding, or header coalescing. Those belong above this layer. ## Why no shim | NativeCall hazard | How the kernel avoids it | | --- | --- | | Struct returned by value | Never returns a struct; fills flat `size_t` quadruples | | `CArray[CStruct]` element reads | Unknown length → pointer math; nothing is read as a CStruct | | Dangling pointers into Raku memory | Kernel never stores a pointer past the call | | Foreign-thread callbacks | No callbacks at all; strictly pull-based | | `-1` / `-2` sentinel confusion | Explicit `PARSE-ERROR` / `PARSE-INCOMPLETE` constants | | Buffers shared across calls | Every parse owns its nine out-buffers; `state` reuse is banned (see below) | The one hazard that did bite: NativeCall refuses to pass a native `size_t @array` to a `CArray` parameter, so every out-parameter is declared as a bare `Pointer` and fed with `nativecast(Pointer, @native-array)`. See `lib/HTTP/Core/Native.rakumod`. The second hazard, learned the hard way: **do not hoist those buffers into `state`.** Doing so made the single-threaded benchmark 17% faster and silently corrupted 47% of concurrent parses, because `state` is not thread-local in Rakudo 2026.09. `t/03-threads.rakutest` exists to catch a regression. The full post-mortem is in `bench/RESULTS.md` §8. ## Byte offsets, not graphemes `Str.index` counts **grapheme clusters**, and `"\r\n"` is a single cluster under UAX #29. So `$text.index("\r\n\r\n")` on a wire image of 152 bytes returns 141, not 146 — seven CRLFs, seven missing positions. Every offset in this module is a byte offset, so any search for a terminator must run on the `Buf`, not on a decoded `Str`. `t/06-codec.rakutest` has the byte-level helper. ## Server ```raku use HTTP::Core::Server; use HTTP::Core::Response; my $app = HTTP::Core::Server.new(port => 8080); $app.router.get: '/', -> $req, %p { HTTP::Core::Response.html('

hi

') }; $app.router.get: '/users/:id', -> $req, %p { 'user ' ~ %p }; $app.router.post: '/echo', -> $req, %p { $req.body }; $app.start; ``` A handler may return a `Response`, a `Str` (sent as `text/plain`), or a `Blob` (sent as octet-stream). Anything else is stringified. A handler that dies turns into a 500 and the connection stays consistent. What the loop gives you for free: | Concern | Behaviour | | --- | --- | | Keep-alive | `Connection: close` honoured, otherwise the socket is reused | | Pipelining | Multiple requests in one packet are drained in order | | Partial bodies | Dispatch waits for the full `Content-Length` before running a handler | | Oversized head | 431 once the buffer passes `max-head` (64 KiB) | | Oversized body | 413 once `Content-Length` passes `max-body` (8 MiB) | | Malformed bytes | 400, then the connection closes | | Unknown path | 404 | | Known path, wrong verb | 405 with an `Allow` header | `start` only arms the listener and returns immediately. A connect issued in the same tick can still be refused, so scripts either retry the first exchange or `sleep` one short tick after `start` — every example here does one of the two. `$app.connections` reports live entries in the connection registry, and `$app.drop-connections` closes everything outstanding. The registry is keyed on a monotonic `Connection.serial`, not on a socket address, because an address is recycled the moment the old connection is collected — the bug, the fix and the 192/192/0 measurement are in `bench/RESULTS-CRO.md`. Deliberately missing: chunked request bodies, TLS, `Expect: 100-continue`, HTTP/2. For TLS, put this behind a reverse proxy or wrap the accepted socket — do not bind OpenSSL by hand. ## Client The same kernel read backwards, so a client needs one `use` and two calls: ```raku use HTTP::Core; use HTTP::Core::Client; my $bytes = encode-request( method => 'POST', target => '/items?page=2', headers => ('Host' => 'example.com', 'Content-Type' => 'application/json').List, body => '{"a":1}', ); my $res = parse-response($incoming); # or decode-response, same thing say $res.status; say $res.body; say $res.framing; # no-body / content-length / chunked / until-close say response-extent($res); # bytes on the wire, or -1 if unknown from the head ``` And the pooled form, which is what the client is for: ```raku my $client = HTTP::Core::Client.new(:timeout(5)); my $res = await $client.get('http://127.0.0.1:8080/health'); say $res.status, ' ', $res.body.decode; $client.close; ``` Headers are an ordered `List` of pairs, not a hash: a response may repeat `Set-Cookie`, and the order of those repeats is meaningful (RFC 6265 §3). `header-pairs` returns the wire verbatim; `header-values('set-cookie')` returns every value, while `header('X')` returns the first. **`Content-Length` is the one header the codec derives.** It is arithmetic on the bytes you handed over, not policy, so: a body with no framing header gets the length added; a caller-supplied `Transfer-Encoding` is left alone; a caller-supplied `Content-Length` must match the body or `encode-request` dies rather than put an unparseable request on the wire. `Host` and `Date` remain yours. Response parsing tells you more than a request parse needs: | Field | Meaning | | --- | --- | | `.framing` | How the body is delimited: `no-body`, `content-length`, `chunked`, `until-close` | | `.consumed` | Bytes of head consumed; the body starts here | | `.is-complete` | Whether every body byte is present in the buffer yet | | `.close` | Whether this response forbids connection reuse | `parse-response($blob, :head)` is the HEAD-request form: the head is parsed but the body is never expected. Failures are values, not die messages: `await` gives you an `HTTP::Core::X` exception (`Timeout`, `Network`, `Truncated`, `Redirect`, `Protocol`, `InvalidUrl` …), all descending from `X::Base`, so one `CATCH` can classify the lot. ## Examples | Script | What it demonstrates | Run it | | --- | --- | --- | | `examples/serve.raku` | Routes, path params, wildcards, JSON, graceful shutdown | `raku -Ilib examples/serve.raku --help` | | `examples/client.raku` | Pooled client, verbs, bodies, headers, self-test | `raku -Ilib examples/client.raku --self-test` | | `examples/hello-server.raku` | The forty-line server, `PORT=` / `DEMO=` driven | `DEMO=3 raku -Ilib examples/hello-server.raku` | | `examples/parse-demo.raku` | A parser walkthrough, no sockets | `raku -Ilib examples/parse-demo.raku` | | `examples/registry-watch.raku` | The connection registry filling and emptying | `DEBUG=1 raku -Ilib examples/registry-watch.raku` | `registry-watch.raku` is the executable form of the leak fix: it opens one keep-alive connection and prints the expected and observed count at four points, each line ending in `ok` or `FAIL`. With `DEBUG=1` each accept line and its drop line carry the same `conn#N`; the `id=` field differs between them for the same connection, which is exactly why the address could not be the registry key. ## Tests ```sh prove -e 'raku -Ilib' t/*.rakutest # the whole suite: 17 files, 101 tests raku -Ilib t/01-parse.rakutest # 37 assertions, parser only raku -Ilib t/02-server.rakutest # 13 subtests, every request issued by Client raku -Ilib t/06-codec.rakutest # 11 subtests: both directions, all four framings raku -Ilib t/07-live-client.rakutest # 3 subtests: client codec against a real socket raku -Ilib t/17-registry-release.rakutest # 4 subtests: the registry gives its entries back ``` Test files use the `.rakutest` extension so a harness (`prove -e 'raku -Ilib' t/*.rakutest`, or anything that globs `t/**/*.rakutest`) discovers them on its own; `.rakumod` is reserved for modules. The helpers under `t/fixtures/` (`CurlCaptures`, `CurlResponses`) stay `.rakumod` — they are libraries, not tests. `prove6` works too if it is installed; plain `prove` needs no extra distribution. ## Layout ``` Build.rakumod compiles src/http_wire.c via %*ENV src/http_wire.c the kernel: pure function, zero allocation src/http_wire.h C ABI + phr_header layout lib/HTTP/Core.rakumod facade: parse-request + self-check lib/HTTP/Core/Native.rakumod the only file that knows about pointers lib/HTTP/Core/Request.rakumod userspace object, lazily materialised strings lib/HTTP/Core/Response.rakumod serialisation, status phrases, constructors lib/HTTP/Core/Codec.rakumod the four directions: encode/decode, both sides lib/HTTP/Core/Codec/Content.rakumod content-encoding negotiation (gzip/deflate) lib/HTTP/Core/Codec/Raku.rakumod the string-building oracle the C path is tested against lib/HTTP/Core/Bytes.rakumod Str -> UTF-8 without the per-call transcoder lib/HTTP/Core/Router.rakumod :params and * wildcards lib/HTTP/Core/Framing.rakumod message boundaries: buf in, Outcome out lib/HTTP/Core/Connection.rakumod buffer + parked promises, event-driven lib/HTTP/Core/Transport.rakumod role: incoming / write / close / id lib/HTTP/Core/Transport/TCP.rakumod IO::Socket::Async implementation lib/HTTP/Core/Transport/TLS.rakumod the same, behind a handshake lib/HTTP/Core/Transport/InMemory.rakumod a socket-less transport for tests lib/HTTP/Core/Server.rakumod routing, dispatch, keep-alive policy only lib/HTTP/Core/Client.rakumod URL, redirects, deadlines, pooling lib/HTTP/Core/Pool.rakumod leases, idle/active accounting, reuse metrics lib/HTTP/Core/X.rakumod Incomplete / ParseError / Network / Timeout / ... tools/ scratch helpers used while developing the parser t/ .rakutest files, one concern each t/fixtures/ CurlCaptures / CurlResponses: capture fixtures (modules, not tests) examples/ runnable scripts, each with --help docs/API.md every public method, one table per module bench/ RESULTS.md, RESULTS-CRO.md: numbers and post-mortems ``` ## API reference The full surface — every public method, argument and default, one table per module — lives in **[docs/API.md](docs/API.md)**. The short version: | Module | Use it for | | --- | --- | | `HTTP::Core` | `parse-request`, `encode-request`, `parse-response`, `core-version` | | `HTTP::Core::Request` | Method, target, headers, body, re-emission | | `HTTP::Core::Response` | Constructors (`text` / `html` / `json` / `bytes`), parsing | | `HTTP::Core::Router` | Patterns, `:params`, wildcards, 405 support | | `HTTP::Core::Server` | Bind, dispatch, keep-alive, connection accounting | | `HTTP::Core::Client` | Verbs, redirects, deadlines, pooling | | `HTTP::Core::Pool` | Leases, counters, reuse rate | | `HTTP::Core::X` | The exception hierarchy | ## Troubleshooting **`Cannot locate native library 'http_wire'`** — the kernel was never built, or it sits outside the search path. Build it, or point at it: `HTTP_CORE_LIB=$PWD/resources/libraries/libhttp_wire.so raku -Ilib -e 'use HTTP::Core'`. **`invalid ELF header` from inside a precomp** — a Windows `http_wire.dll` and a Linux `libhttp_wire.so` are sitting in the same directory and the loader grabbed the wrong one. The binding orders candidates by platform (stem and extension both), but an explicit `HTTP_CORE_LIB` removes all doubt. **`'zig' is not recognized` / `The term 'CC=zig cc' is not recognized`** — two different Windows mistakes. The first is harmless if any other compiler is present: the probe falls through to `cc`, `gcc`, `clang` and the build carries on. With no compiler at all, install zig (`winget install zig.zig`, or unzip a release and put `zig.exe` on `PATH`), or point `CC` at one you have. The second is POSIX shell syntax pasted into PowerShell: `CC="zig cc" raku …` assigns nothing and tries to run a command literally named `CC=zig cc`; set the variable on its own line. **`Could not find Compress::Zlib` while installing** — `HTTP::Core::Codec::Content` loads it at the top of the file, and `zef` compiles every module in `provides` before it calls the install done, so the module has to be resolvable at that moment. `META6.json` declares it, which means a modern `zef` will fetch it; if your `zef` is old enough to ignore `depends`, or the dependency lives in the user repository (`~/.raku`) while `zef` is staging into `site` — its throwaway repository chain is staging + site + vendor + core and does not include `~/.raku` — install into the user repository as well: `zef install --install-to=home .`, or `zef install --install-to=site Compress::Zlib` first. **`Cannot locate native library` right after `zef install .`** — the installed copy resolves its DLL through the distribution resource table, which zef fills in while staging. A binding that froze the path at `BEGIN` (a `constant LIB`) recorded a path inside zef's staging directory, which is deleted the moment the install finishes, so it works during `Testing` and fails afterwards. Current code resolves on first call (`is native(&_find-lib)`); if you still see this, a stale `lib/HTTP/Core/Native.rakumod` is installed — `zef uninstall HTTP::Core` and install again. **`Saw 1 occurrence of deprecated code` on every run** — Rakudo 2026.09 deprecates every path accessor on `Distribution::Resource` (`.IO`, `.Str`, `.absolute`, `.resolve`) and suggests "using `%?RESOURCES` directly", but that value is a resource object, not a path, and `NativeCall` needs a string — so the installed-distribution lookup still calls `.absolute`. The alternatives were tried and do not work: `$?DISTRIBUTION.content('resources/libraries/…')` returns `Any` for an installed distribution, and `.open` just reaches the same deprecated `Str` from the inside. The notice is noise once per process, it comes from `lib/HTTP/Core/Native.rakumod`, and it goes away when the replacement accessor lands. To silence it now, tell Rakudo not to report deprecations — `RAKUDO_NO_DEPRECATIONS=1` in the environment: ```sh RAKUDO_NO_DEPRECATIONS=1 raku -Ilib -e 'use HTTP::Core; say library-path()' ``` ```powershell $env:RAKUDO_NO_DEPRECATIONS = '1'; raku -e "use HTTP::Core; say library-path()" ``` Nothing is hidden by this beyond the notice: the deprecated call is one line whose behaviour is unchanged, and an empty setting restores the message. **Which library actually loaded?** `raku -Ilib -e 'use HTTP::Core; say library-path(); say assert-abi()'`. `assert-abi` compares the kernel's ABI constant against the binding's and dies instead of limping along after a partial rebuild. **`EADDRINUSE` on `start`** — read `$app.bind-error` for the reason, or pass a port you did not choose: `my $port = free-port()`. **The first request after `start` is refused** — `start` arms the listener and returns; a connect in the same tick can race the bind. Retry the first exchange (every example does) rather than assuming the server is down. **Offsets look seven bytes short** — you searched a decoded `Str` for `"\r\n\r\n"`. Use a `Buf`; see [Byte offsets, not graphemes](#byte-offsets-not-graphemes). **Concurrent parses return the wrong values** — a scratch buffer was hoisted into `state`, which is not thread-local. Keep buffers per call; `t/03-threads.rakutest` is the guard. **`connections` never goes down** — that was the registry leak fixed in `55c8bae` / `c8d9af9`: `!drop` used a `Str` key against an `Int` id, threw inside the frame callback, and the entry leaked forever. Run `raku -Ilib examples/registry-watch.raku`; every line should read `ok`. **Stale precompilation** — `rm -rf lib/.precomp` and run again. A rebuild of the kernel changes nothing Raku-side, but a partially written precomp after an interrupted run will keep failing until it is removed. ## Benchmarks `bench/RESULTS.md` holds the parse microbenchmarks — including §8, the `state` post-mortem, and §12, the cost of `encode` versus rebuilding a head by hand. `bench/RESULTS-CRO.md` holds the connection-registry investigation: root cause, the 192/192/0 measurement, and why the key had to change. ## Limits The transport is `IO::Socket::Async`, i.e. libuv, which MoarVM already owns, so there is exactly one event loop and no thread juggling. Anything that needs C to call back into Raku (a server framework that owns its own loop) still hits the MoarVM limitation and needs a sidecar process instead. Also out of scope, on purpose: chunked request bodies, `Expect: 100-continue`, HTTP/2, WebSocket upgrades, and an HTTP client that hides the response bytes. This library is the layer those would be built on, not the layer that hides it. ## License MIT, as declared in `META6.json`. The full text is in [LICENSE](LICENSE).