Guide

Loading the extension

-- SQLite shell
.load ./msgpack

-- Application code (C)
sqlite3_load_extension(db, "./msgpack", NULL, &zErr);

Once loaded, all msgpack_* functions (including typed constructors, timestamp helpers, and schema validation) and the two table-valued functions (msgpack_each, msgpack_tree) are available in every database connection.

Building from source

Requires CMake ≥ 3.16 and a C11 and C++17 compiler (GCC, Clang, or MSVC). The SQLite extension itself compiles with C99; C++17 is required for the standalone C++ Blob library and its tests.

cmake -B build
cmake --build build
ctest --test-dir build

The default build produces:

Build options

Quick start

-- Encode a scalar value
SELECT hex(msgpack_quote(42));         -- 2A
SELECT hex(msgpack_quote('hello'));    -- A568656C6C6F
SELECT hex(msgpack_quote(NULL));       -- C0  (nil)

-- Build a map and extract from it
SELECT msgpack_extract(
  msgpack_object('name', 'Alice', 'age', 30),
  '$.name'
);  -- Alice

-- Build an array and query its length
SELECT msgpack_array_length(msgpack_array(10, 20, 30));  -- 3

-- Update a value (returns a new BLOB, original is unchanged)
SELECT msgpack_to_json(
  msgpack_set(msgpack_object('a', 1), '$.b', 2)
);  -- {"a":1,"b":2}

-- Convert to/from JSON
SELECT msgpack_to_json(msgpack_from_json('[1,true,"hi"]'));  -- [1,true,"hi"]

-- Aggregate rows into a msgpack array
CREATE TABLE t(v INTEGER);
INSERT INTO t VALUES (1),(2),(3);
SELECT msgpack_to_json(msgpack_group_array(v)) FROM t;  -- [1,2,3]

Path syntax

Path expressions follow the same conventions as SQLite's JSON1:

A path that does not exist returns NULL from scalar functions and is treated as a missing element in multi-path and mutation operations.

Type system

Each MessagePack element has a type string returned by msgpack_type():

BLOB auto-embedding

When a BLOB value is passed to any construction or mutation function, the extension checks whether it is valid msgpack:

• Valid msgpack BLOB → embedded directly as a nested element (transparent nesting)
• Invalid / raw BLOB → stored as msgpack bin type

This makes composition natural without extra wrapping:

-- Works just like JSON1's json() wrapper for nested values
SELECT msgpack_to_json(
  msgpack_object(
    'tags',   msgpack_array('sqlite', 'msgpack'),
    'meta',   msgpack_object('version', 1)
  )
);
-- {"tags":["sqlite","msgpack"],"meta":{"version":1}}

Spec compliance

This extension implements the MessagePack specification in full:

• Smallest encoding rule — values are always encoded in the most compact valid format (e.g., 42 uses positive fixint 0x2a, not uint8 0xcc 0x2a). Use the typed constructors (msgpack_int8, msgpack_uint32, etc.) when you need a specific wire width.
• All 36 format families — nil, bool, positive fixint, negative fixint, uint8/16/32/64, int8/16/32/64, float32/64, fixstr, str8/16/32, bin8/16/32, fixarray, array16/32, fixmap, map16/32, fixext1/2/4/8/16, ext8/16/32.
• Extension types — arbitrary ext types via msgpack_ext(type_code, data).
• Timestamp extension — built-in support for the standard timestamp extension type (type code −1) with ts32, ts64, and ts96 encodings via msgpack_timestamp().
• Copy-on-write mutation — no in-place modification; every mutation returns a new BLOB.
• Never-used byte — 0xc1 is treated as an error by msgpack_valid and msgpack_error_position.

Limits & robustness

The implementation enforces the following hard limits to keep operations on untrusted blobs bounded:

Inputs that exceed these limits are rejected as invalid rather than processed (validation returns false, mutating operations raise an SQL error). All decoding paths are bounds-checked: malformed, truncated, or adversarially deeply-nested blobs cannot crash the process or read out of bounds. The fuzz corpus exercises these adversarial cases (see Testing). For the security model and reporting policy see SECURITY.md.