The fields of the error, never null: Pg.error_fields(error).constraint.
#import "Extensions/Postgres";
A PostgreSQL client over Io fibers.
Io :: #import "Extensions/Io";
Pg :: #import "Extensions/Postgres";
main :: () {
Io.io_run(() {
config, error := Pg.parse_url("postgres://app@localhost/app_dev");
db:, error = Pg.connect(config);
if error.kind != .NONE { print("%\n", Pg.error_text(error)); return; }
defer Pg.close(db);
result:, error = Pg.query_dynamic(db, "SELECT id, name FROM users WHERE id > $1", 10);
defer Pg.free_result(*result);
for row: 0..result.row_count - 1 {
id := Pg.get(*result, row, 0, s64);
name := Pg.get(*result, row, 1, string);
}
});
}This one is only checked to compile, not run.
Every call that talks to the server runs on a fiber inside Io.io_run and blocks only that
fiber. Io.with_deadline bounds any of them.
Set the module parameters when importing:
Pg :: #import "Extensions/Postgres"(MIGRATIONS = "sql/migrations", SCHEMA = "sql/schema.jai", QUERY_CACHE = "sql/cache", MIN_SERVER_VERSION = 15);Doesn't run in the browser (skipped on wasm).
Module parameters#
- TLS := true
- Connects over TLS, which needs Extensions/Tls (mbedTLS). Set
falseto leave it out of programs that only use Unix sockets or cleartext TCP. - MIGRATIONS := "db/migrations"
- The migrations folder, relative to the project root.
- SCHEMA := "db/schema.jai"
- The schema file, relative to the project root.
- QUERY_CACHE := "db/queries"
- The folder of the typed-query cache, relative to the project root.
- MIN_SERVER_VERSION := 0
- The oldest server major version accepted; an older server fails every describe.
Procedures#
Error#
One line describing the error, in temporary storage.
Frees what the error owns.
The five characters of a code, in temporary storage.
The code with only its class kept: sqlstate_class(.UNIQUE_VIOLATION) == .INTEGRITY_CONSTRAINT_VIOLATION.
The code written as text.
Config#
postgres://user:password@host:port/database?sslmode=verify-full&application_name=api.
postgresql:// works too. A Unix socket directory goes in host, percent-encoded or as a
parameter: postgres:///app?host=/var/run/postgresql. It is parse_conninfo for a URI.
libpq's connection strings: a URI (see parse_url) or host=localhost port=5432 dbname=app,
where a value with spaces is quoted, 'a value', and \ escapes the next character
(\' and \\ inside quotes). Parameters (keywords): host, hostaddr, port, user, password, passfile, dbname,
sslmode, sslnegotiation (direct), sslrootcert (a PEM file, or system), sslcert, sslkey,
sslpassword, sslcertmode, sslcrl, sslcrldir, sslsni, sslkeylogfile, ssl_min_protocol_version,
ssl_max_protocol_version, channel_binding,
application_name, fallback_application_name, options, connect_timeout (seconds), client_encoding,
replication and require_auth. libpq's other keywords are accepted and ignored. Errors read like
libpq's.
The Config for the options of a connection string, as parse_conninfo makes it. Programs
that combine several sources (a command line and a connection string) set the options
themselves with conninfo_set first.
A config that owns copies of its strings, for keeping beyond the original (free_config).
Frees the strings of a config from parse_conninfo, parse_url or copy_config; does nothing
for one the program built.
Conninfo#
Parses a URI (postgresql:// or postgres://) or a key=value list. The strings in the
result belong to it: free_conninfo.
Frees the strings of the options.
Whether text is something a program should hand to parse_conninfo: a URI or a list with
an = in it. psql and libpq use this to tell a connection string from a database name.
The value of keyword and whether it was set.
Sets keyword to a copy of value, replacing what was there. False for a keyword libpq does
not know.
Saslprep#
The prepared password and true, or password and false when it cannot be prepared. ASCII
passwords come back unchanged. The result is in temporary storage unless it is password.
Auth#
Reads a require_auth value (libpq's syntax: see Config.require_auth). The errors are
libpq's.
Fails the login when the server's way of authenticating is not one require_auth allows.
code is the AuthenticationRequest the server sent.
The password for a login from the password file, or an empty string. host is the name
connected to; a socket directory or no name searches for localhost.
Connection#
Opens a connection and authenticates. On failure the connection is null and nothing needs
closing. Must run on a fiber (Io.io_run); config.connect_timeout bounds each attempt.
Ends the session (Terminate) and frees the connection. now skips the goodbye.
A ParameterStatus value, or "" if the server did not report it.
Writes a server notice to the log.
Query#
Runs one statement with parameters $1, $2, ... taken from args, and returns its rows in
text format. Arguments may be integers, floats, bools, strings, [] u8 (bytea), Uuid,
Timestamp, Local_Timestamp, Date, Time, Interval, Numeric, Nullable of any of
those, and one-dimensional arrays of them.
query_dynamic with binary results: smaller and faster to decode, but every column's type must
be one get can decode in binary (see Oid).
Parses sql as the unnamed prepared statement and describes it (Parse, Describe, Sync). Free
the result with free_statement_info.
Frees what describe_statement returned.
Runs SQL that may hold several statements, without parameters (the simple query protocol), and discards any rows. For migrations and scripts. Without an explicit BEGIN the whole string runs as one implicit transaction, so the first error stops it and undoes the statements before it.
Runs SQL that may hold several statements (the simple query protocol) and reports each
statement's outcome to hooks. The returned error is the one that stopped the script, after the
results of the statements before it were reported. Without an explicit BEGIN the whole string
runs as one implicit transaction.
The value at row and column as a T. NULL is an error unless T is a Nullable.
Whether the value is NULL. False when the row or column does not exist.
The first column with this name, or -1.
The undecoded bytes of a value, in the column's format; is_null for NULL.
Drops the rows of a result and what was decoded from them, keeping its columns.
Frees a result and everything decoded from it. It is zeroed.
Statements#
Runs one statement like query_dynamic, prepared under a name and kept in the connection's cache
(unless plan is .CUSTOM or the cache is off): the first run parses it, later runs with the
same text and parameter types only bind. Typed queries do this on their own.
How many statements the connection has prepared on the server and remembers.
The server name of the statement a query with this text and these parameter types is prepared
as (j and 16 hex digits), for looking it up in pg_prepared_statements.
Copy#
Sends rows through sql, a copy <table> (<columns>) from stdin (format binary) statement,
and returns how many rows the server stored. Each row sends the members named in columns, in
that order, or every member when columns is empty. The columns' types must match the members'
binary forms exactly (an s32 for int4, an s64 for int8): COPY casts nothing. A Json(T)
member goes as jsonb unless it has the @pg_json note, which sends it as json.
stored, error := Pg.copy_in(db, "copy users (email, score) from stdin (format binary)", users, .["email", "score"]);Not run: the example is marked ignore.
Runs a COPY <table or (query)> TO STDOUT statement and calls proc with each piece of its
data, as it arrives, so a table of any size streams through. Returns how many rows the server
sent. Any format works: text, CSV or binary.
count, error := Pg.copy_out(db, "copy (select email from users) to stdout (format csv)", write_chunk, file);Not run: the example is marked ignore.
Frees the rows of a binary COPY and everything they point to.
Types#
The value, or default when it is NULL.
A non-null Nullable holding value.
value as a json/jsonb parameter.
Decodes a document into a T.
The document as a Json_Value tree, which the caller releases with free_json (from
Extensions/Json); any JSON can be read this way, whatever T a column would map it to.
Microseconds since 1970-01-01 00:00:00 UTC.
The current time from the system clock.
The date of a proleptic Gregorian calendar day. Year 0 is 1 BC, as in ISO 8601.
The year, month and day of a date.
The instant at that wall-clock reading, taken as UTC.
Spans of time as an Interval, for arithmetic on timestamps: Pg.now() - Pg.days(7). A day is
a calendar day, not 24 hours, as in Postgres; the rest are exact.
An interval of exact hours.
An interval of exact minutes.
An interval of exact seconds.
Every field negated.
Months, days and microseconds added field by field.
Uuid from "0f8fad5b-d9cb-469f-a165-70867728950e" (dashes optional, any case).
"0f8fad5b-d9cb-469f-a165-70867728950e", in temporary storage.
The integer part of a numeric; false for NaN, the infinities, or one out of range. A fraction is truncated toward zero.
The numeric as a float, and false if its text does not read as a number.
Pool#
The config is copied. Nothing connects until the first acquire, or min_idle asks for it.
Closes the idle connections and frees the pool once every borrowed one is back.
A connection for this fiber alone, until release. Waits for one when max_size are in use,
up to acquire_timeout (and any Io.with_deadline around the call).
Gives a connection back. One left inside a transaction is rolled back first; a broken one, or
one past max_lifetime, is closed instead.
A round trip that runs nothing: an empty Sync. Fails if the connection is dead.
Runs body in a transaction on a connection from the pool. The transaction commits when body
returns no error, and rolls back when it returns one (which comes out of transaction) or a
statement in it failed. A serialization failure or deadlock (40001, 40P01) runs body again, up
to retries more times, so body must be safe to repeat. data is passed through, as Jai
procedures capture nothing.
Transfer :: struct { from: s64; to: s64; amount: s64; }
error := Pg.transaction(pool, Transfer.{ 1, 2, 10 }, (tx: *Pg.Connection, t: Transfer) -> Pg.Pg_Error {
_, error := Pg.query_dynamic(tx, "update accounts set balance = balance - $1 where id = $2", t.amount, t.from);
if error.kind != .NONE return error;
_, error = Pg.query_dynamic(tx, "update accounts set balance = balance + $1 where id = $2", t.amount, t.to);
return error;
}, .SERIALIZABLE, retries = 3);Not run: the example is marked ignore.
Pipeline#
The answer to a statement queued with queue or queue_execute: its rows, or the count of rows
it changed. The statements queued so far get a sync point first if they have none, so the batch
ends. Pendings may be taken in any order; results read on the way are kept for their own take.
Starts pipeline mode on a connection. Entering again returns the same pipeline.
Ends pipeline mode. It fails, and the pipeline stays, while results are unread.
Statements waiting for their results, sync points included.
Queues a statement with parameters $1, $2, ... from args (see query_dynamic for the types),
results in text format.
pipeline_query with results in binary format.
Queues a statement whose parameters are already text. types are the parameter OIDs to declare
(0, or fewer than values, leaves the rest to the server).
Prepares a named statement. types are the parameter OIDs to declare.
Runs a prepared statement with args (see pipeline_query), results in text format.
Runs a prepared statement with parameters that are already text.
The result of these four is .RESULT; a describe also fills result.columns, and a statement
describe parameter_types.
Queues a describe of a portal.
Closing a statement or portal that does not exist is not an error.
Queues a close of a portal.
Queues a sync point and sends everything queued.
Queues a sync point without sending.
Asks the server to send the answers so far without ending the batch, and sends everything queued. Without a sync point, a batch that fails is not rolled back until the next one.
Sends everything queued.
Makes the next statement to be read deliver its rows count at a time (.ROWS; the last piece
may have fewer), then end with a .RESULT that has no rows but the command tag, so a large
result never has to fit in memory. It only works before the statement's first result is read.
Reverts at the statement's end.
pipeline_chunked_rows with one row per result.
Reads the result of the oldest queued statement or sync point, sending what is still unsent
first. Blocks the fiber until the answer is in, within any Io.with_deadline: after a deadline
the pipeline keeps what it had read, and the call can be repeated. The result's kind is NONE
when nothing is queued. The returned error is for failures of the connection; the server's
errors come as .ERROR results. Free the result with free_pipeline_result.
Frees a result from pipeline_next_result.
Gives up on everything queued: asks the server to cancel what runs, reads and drops all answers
up to the last sync point (queuing one if needed), and leaves pipeline mode. The connection
breaks if the server does not finish within cancel_timeout.
Describes a prepared statement by name, without a pipeline (Describe, Sync).
Describes a portal (a cursor, say) by name: the columns it returns.
Drops a prepared statement by name. One that does not exist is not an error.
Drops a portal by name. One that does not exist is not an error.
Notify#
Frees a notification's strings. It is zeroed.
The oldest queued notification, without waiting. found is false when none is queued. Free it
with free_notification.
Frees every queued notification.
The oldest queued notification, or the next one to arrive: blocks the fiber until then, within
any Io.with_deadline or Io.cancel (the error is .TIMED_OUT or .CANCELED, and the
connection stays usable). Free the notification with free_notification. Nothing else may use
the connection while a fiber waits on it. With on_notification set, notifications never queue,
so there is nothing to wait for.
LISTEN on a channel. The name is quoted, so any text is a channel.
Stops listening to one channel.
Stops listening to every channel.
NOTIFY a channel, with a payload of up to 7999 bytes. Delivered to the listeners when the transaction commits; inside none, at once. A connection also hears its own notifications.
The text as an SQL identifier: in double quotes, with embedded quotes doubled.
Listener#
The next notification, in order. Blocks the fiber until one arrives, within any Io.with_deadline
or Io.cancel (.TIMED_OUT or .CANCELED); .CLOSED once the listener is closed and what it
held is taken. Free the notification with free_notification.
Stops listening: the connection goes back to the pool without its subscriptions, and the listener is freed. Notifications not taken yet are dropped.
Trace#
Starts tracing: proc gets a line per message from now on (null stops). With regress, the
output is libpq's PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE: OIDs, process ids and the
server's source locations are replaced by fixed text, and an error's length by NN, so a trace
is the same from one server run to the next. Without it they show as they are.
Migrate#
The migrations in folder, sorted by version, for embedding: #run load_migrations(...).
A relative folder starts at the calling file's folder. A badly named file or a repeated version
stops the compile.
load_migrations for tools: the migrations, or a message saying what is wrong with the folder.
relative_to is a file whose folder relative paths start at; empty for the working directory.
The schema hash: SHA-256 chained over each migration's name and digest, in order, so it changes when any file is renamed, edited, added or removed. Hex.
The schema hash of each prefix of migrations: entry i covers migrations 0 to i.
Applies the pending migrations in order, each in its own transaction, and returns how many it
applied. An advisory lock keeps two processes from migrating at once. It refuses (applying
nothing) when an applied migration changed or disappeared, or a pending one is numbered below
the newest applied one, unless options allows it.
migrate, and when a migration fails, failure says which one and where in its file.
Each migration's state, in the list's order, then the applied ones missing from the list. Creates the migrations table if it does not exist.
Splits a script into its statements, skipping quoted text, dollar quotes and comments, so a ;
inside them does not end a statement. Empty statements are dropped. The pieces point into sql.
The line and column of a 1-based character position in text, or 0 and 0 for no position.
Characters between two byte offsets of text.
Where error stopped in the file whole of migration name. statement is the text the
server was sent: the whole file, or one statement inside it (a slice of whole) for a migration
without a transaction.
For an error that came without a position: the first statement of the migration whole that
fails the same way, as the failure's line. Runs inside the caller's open transaction, in the
state the migration started from, and leaves it in that state (a savepoint is rolled back to).
Gives failure back unchanged when no single statement reproduces the error, which happens when
the file's ; splitting is not how the server saw it.
How many newlines text has.
"migration 0007_add_teams.sql failed at line 3: relation ... does not exist".
Managed#
The settings the compile-time describe service uses: the defaults, with JACKET_PG_MANAGED_VERSION
(a pinned release such as "16.15.1") and JACKET_PG_MANAGED_CACHE (the cache folder) applied.
Managed mode as the project's jacket.toml sets it, under [postgres]: managed = true turns it
on (-1 when the file or key is missing) and managed_version = "16.11.1" picks a release.
JACKET_PG_MANAGED and JACKET_PG_MANAGED_VERSION win over both.
The settings for the project at root: the environment, then its jacket.toml.
The release's name for a platform, or "" when it publishes nothing for it. musl only matters
on Linux.
The triple for the machine this runs on. "" when there is none (an unsupported OS or CPU).
The pinned SHA-256 for a release and triple, or "".
The per-user folder for downloaded binaries, from the platform's conventions. The arguments are the environment variables it reads, so the choice can be checked without them.
Downloads the release for this machine into the cache unless it is there, and returns its
bin folder. Safe to run in several processes at once: each unpacks into its own folder and
the first to rename it into place wins.
Makes sure the binaries are there, creates the cluster if needed, and starts a server on it,
or reuses one already running on the same data folder (found through postmaster.pid). The
returned config connects as postgres.
The cluster's folder: the setting, else <cache>/cluster-<major version>.
Everything start_managed does before starting the server: the download, and the cluster.
These are the slow parts (seconds to tens of seconds, the first time only), so code with a
time limit on its requests calls this first, somewhere without one, and start_managed is
then quick. Safe to call again and from several processes; it returns an error text.
Stops a server start_managed started. A server it found already running is left alone unless
force is set. Returns an error text, empty when it stopped (or was not asked to).
JSON options#
A Json_Columns entry's three options as the second parameter of Json(T, OPTIONS): two bits
each, in the order of the enums.
The options pack_json_options packed.
Wire#
Starts a message in the output buffer, with its length left to fill in.
Fills in the length of the message begin_message started.
Adds a Parse message to the output.
Parameters go as text unless one needs another format; the format codes are left out when every parameter is text.
Adds a Describe message to the output.
Adds a Close message to the output.
Adds an Execute message to the output.
Adds a Sync message to the output.
Adds a Flush message to the output.
Writes the output buffer to the server.
Room for total items, growing by doubling so a long run of small appends copies little.
Appends bytes.
A C string: the text and a zero. Text holding a zero would end early, so callers check first.
Reads a big-endian number.
Reads a big-endian number.
Reads a big-endian number.
Makes at least count unread bytes available.
The bytes left in the cursor's data.
Startup#
Connects c to the server: transport, TLS, the startup message and authentication.
Opens the TCP or Unix socket and sets up the reader and writer.
Asks for TLS and starts it, as ssl_mode requires. Does nothing for a Unix socket.
Sets up the TLS configuration from the settings and runs the handshake.
Appends a line to the connection's key log file, which only its owner may read. A file that cannot be opened or written is reported, as libpq does, and the connection goes on.
Prints a line to standard error.
The handshake over the connection's reader and writer, the stream becoming theirs.
What libpq prints for a failed handshake, or a TLS failure afterwards.
The alerts OpenSSL words as "ssl/tls alert ..." (those of SSL 3.0); the others it calls TLS 1.0's.
Whether host is written as an address (only digits and dots, or with a colon), which libpq
never sends as SNI.
Asks the server to stop what c is running: a CancelRequest on a connection of its own, over
TLS when c uses it. The server answers nothing and closes that connection once it has acted.
Whether a statement was stopped shows on c (QUERY_CANCELED), not here.
Reads the server's replies to the startup message until ReadyForQuery, answering authentication requests.
Answers one AuthenticationRequest: sends the password or the next SCRAM step.
The transaction status ReadyForQuery reports.
A .PROTOCOL error for what.
An .AUTH error for what.
Whether text has a zero byte, which a startup parameter cannot hold.
Asks the server to cancel the statement c is running, as libpq's PQcancel does. Meant for
another fiber than the one waiting for the statement (a signal handler's, say): the waiting
call then returns the server's "canceling statement due to user request". Does nothing useful
when no statement is running.
TLS name#
"" when the certificate is for host, else why not.
The address host stands for and true, or false when host is a name.
Whether c is a to f in either case.
Whether the certificate entry is address, byte for byte.
The text of a raw address, as libpq shows it in its messages.
A name from the certificate against the host: equal without regard to case, or a wildcard
*.rest for a host with exactly one more label in front of .rest.
SCRAM#
Frees what a SCRAM exchange holds.
Picks a mechanism from the server's offer and sends the client-first message.
Answers the server-first message with the client-final message.
PBKDF2 of the password, taken from the cache when it was computed before.
Checks the server-final message: its signature proves the server knows the password.
The password as SCRAM uses it: SASLprep (saslprep.jai), or its own bytes when that cannot be done, like libpq.
The hash of the server's certificate for tls-server-end-point: the certificate's own signature hash, with MD5 and SHA-1 raised to SHA-256. Empty when the signature uses no hash (Ed25519) or one this client cannot compute.
The SCRAM-SHA-256 secret the server stores for a password, to put in ALTER ROLE ... PASSWORD
without the server seeing the password (libpq's PQencryptPasswordConn does the same). The
password is prepared with SASLprep. iterations is what the server reports as
scram_iterations, 4096 by default.
The client-final-message for a server-first-message, and the signature the server must prove it
knows in its final message. binding is the GS2 header (plus the certificate hash for -PLUS),
salt the decoded salt. Exported so the RFC 7677 example can be checked.
How many times a SCRAM exchange ran PBKDF2 in this process: cache hits do not count.
The MD5 secret for a password and a role name: md5 and the hash of the two joined.
SQL scan#
Rewrites % to $1, $2, ... in order and %% to %. A $n of Postgres' own in the source
is an error: one style only. Text inside quotes and comments is copied as it is.
The offset in the source of a byte of the rewritten statement. Inside a replaced % it is
that %.
The byte offset in text of a 1-based character position (what an ErrorResponse's P field
counts, in characters, not bytes). Past the end, the end.
A 64-bit statement id: the first eight bytes of SHA-256 of the rewritten statement.
The full SHA-256 of a text, as lower-case hex.
The 16 lower-case hex digits of a statement id.
Lower-case hex of count bytes at data, twice as long.
Entry#
The JSON document of a cache entry.
Reads an entry back from its JSON. False when the document is not shaped like one.
Describe#
The complaint for a server older than the request allows, or an empty string.
The request as JSON text.
Describes request.sql on db and returns the answer as JSON text:
{"kind":"ok","entry":{...}} the cache entry (see entry_to_json)
{"kind":"sql_error","error":{...}} the server refused the statement
{"kind":"error","message":"..."} anything else (the connection died, say)
The answer to a fingerprints request: {"kind":"ok","server_major":17,"fingerprints":{key: hash}}.
The answer for migrations that do not build and leave no earlier schema to describe against.
Schema databases#
The name of the database for a schema hash.
Picks (and builds when it is time) the schema database for the migrations in folder. db is
connected to the server's own database and config is how it was reached.
Drops schema databases beyond the newest SCHEMA_DATABASES_KEPT. One still in use (by another
editor, say) is left for next time.
Resolve#
The project's migrations folder, schema file and query cache, from the module parameters MIGRATIONS, SCHEMA and QUERY_CACHE (paths relative to the project root).
The path of the schema file.
The path of the query cache folder.
MIN_SERVER_VERSION: the oldest server major version the project supports; 0 accepts any.
The folder that holds the project's files: from the file with the query, the nearest folder up
the tree with a jacket.toml or the first folder of MIGRATIONS (db, by default); if there is
none, that file's own folder. JACKET_PG_ROOT overrides it.
The compile-time database's URL: JACKET_DATABASE_URL, else DATABASE_URL, from the environment;
when it has neither, the same two names in the project's .env file. Empty when there is none.
The project's migrations folder and their schema hash (see schema_hash), or empty strings when
there are none.
id_hex read back.
The path of the cache file for a statement.
The entry in a cache file and true, or false when there is none, it cannot be read, or it is for another statement.
Writes a statement's cache entry, leaving the file alone if it would not change.
Dotenv#
The value key gets in the text of a .env file, and whether it is assigned at all. The last
assignment wins. The result lives in temporary storage.
The database URL the project's .env file gives: JACKET_DATABASE_URL, else DATABASE_URL. Empty
when there is no file or neither is assigned.
ORM#
The rows of T's table that pass where, in order_by order (desc for descending), at most
limit of them (-1: all) after skipping offset. from_sql replaces the table with a query of
your own that returns its columns, with %name / %(expression) parameters:
from_sql = "select * from users_visible_to(%viewer)". fields = "id, email" selects only
those columns, and the rows are then Partial(T, "id, email").
Sets fields of the rows that pass where: set = #code it.score = score + 1 (or a block of such
assignments). Returns the number of rows changed.
Deletes the rows that pass where and returns how many there were. where is required: pass
#code true to delete every row.
Inserts value and returns the row the table now holds (with its defaults and generated
columns). Fields with a database default (@pg_default) or a generated value
(@pg_generated) are left to the database.
Inserts each value in turn (one statement each), stopping at the first error. Returns the rows inserted so far, from context.allocator.
A select whose columns come from the loop that reads it: only the fields of T the body reads,
plus the primary key, are fetched; the other fields of it stay zero. A json document read only
for some top-level fields comes back with just those keys, and it.author.email (a to-one
relation) is fetched in the same statement, making it an Orm_Item that usings a T. When it is used whole
(passed to a procedure, stored, *it), every column is fetched. The query runs when the loop
starts, and a failure lands in error (which is cleared first). Strings in it read from the
result, which is freed when the loop ends: copy what you keep.
error: Pg.Pg_Error;
for Pg.each(db, User, *error, where = #code it.score > min) print("%\n", it.email); // select "id", "email" ...Not run: the example is marked ignore.
Makes Pg.each loopable.
Stores values with one binary COPY and returns how many rows the server stored. Much faster
than insert_many for many rows, but it reads nothing back: fields marked @pg_default or
@pg_generated are left to the database, as in insert, and their values are not returned.
Loads a has-many relation for several rows; see Loaded_Many.
Loads a to-one relation for several rows; see Loaded_One.
load_many with the related struct given, for Pg.each, whose loop knows it already.
Frees what load_many returned.
like: Pg.like(it.email, "%@example.com").
ilike: case-insensitive like.
= any(values): Pg.in(it.id, ids).
lower(value).
upper(value).
(value is null or condition): an optional filter, Pg.if_some(team, #code it.team_id == team)
with team: Nullable(s64), where team stands for its value inside the condition.
?: the jsonb document has the top-level key: Pg.has_key(it.settings, "beta").
@?: the jsonpath matches: Pg.json_path(it.settings, "$.tags[*] ? (@ == \"x\")").
Typed#
Every row decoded and copied, with the strings, arrays and documents it points to, into one
block from allocator. The rows stay valid after free_rows; array_free(items) frees all of
it (with temp, nothing needs freeing). Check rows.error afterwards.
Not run: the example is marked ignore.
Frees the result data of rows.
Runs the statement and returns its rows. Rows(Row(...)) has one field per column; when the
columns are exactly one table's, it is Rows(Table) with that table's struct from db/schema.jai
(row type identity, see row_type).
The only row, or a .NOT_FOUND error when there is none. More rows are a .USAGE error. The row
lives in temporary storage (until reset_temporary_storage).
The first row and whether there was one.
Runs a statement and returns the number of rows it affected.
query on a pipeline: the statement is queued, checked like query, and its rows come later
from take. Queue several, then take them: they share one round trip.
p := Pg.pipeline_enter(c);
users := Pg.queue(p, "select id, name from users where id = $1", id);
count := Pg.queue_execute(p, "update users set seen = now() where id = $1", id);
rows, error := Pg.take(p, users);
changed, error = Pg.take(p, count);
Pg.pipeline_exit(p);Not run: the example is marked ignore.
execute on a pipeline: take gives the count of rows the statement changed.
query for results too big to hold: the rows arrive batch at a time and for decodes them
one by one, so at most one batch is in memory. The statement is sent here, and the connection is
busy until the loop ends; leaving the loop early cancels the rest. A failure, before or during the
loop, lands in error (cleared first). Strings in it live until the next batch: copy what you
keep.
error: Pg.Pg_Error;
for Pg.stream(c, *error, "select id, body from events where day = %", day, batch = 5000) {
total += it.body.count;
}
if error.kind != .NONE ...Not run: the example is marked ignore.
The first column of the only row.
query into your own struct. The columns are matched to fields by name, and checked: a column
that can be NULL needs a Nullable field, the types must be assignable, and a Json(T) field
says which type a json column holds. project_file is as for execute.
Schema#
The Jai type a json or jsonb column holds, from the Json_Columns visible where this is
called (a macro, so the generated file sees the user's list), else Json_Raw. A mapping can
change without generating the schema again.
The options of the Json_Columns entry for column, packed as the second parameter of
Json(T, OPTIONS); 0 when it has none.
Does a file called name sit in the folder of file? For the generated file to load
json_columns.jai when there is one.
Stops the compile when file (the generated schema) was made from other migrations than the
ones in the project now. Needs no server: it compares hashes of db/migrations/*.sql.
Why file no longer matches the migrations, or an empty string when it does.
The hash a generated file records in its header, or an empty string.
Stops the compile when a Json_Columns entry names something that is not a json or jsonb
column. known is every json column of the schema, as the generated file lists them.
How a Postgres enum label is spelled as a Jai enum member, and back. A label that is a plain
name is used as it is, so 'pending' is .pending. Any other (spaces, dashes, a leading
digit, a Jai keyword, a double underscore) becomes __ followed by its letters and digits,
with every other byte as _ and two hex digits: 'in progress' is __in_20progress.
The label an enum member name stands for: the inverse of enum_member_name.
Generate#
Reads the catalog of the migrations' schema database (see schema_databases.jai) on the server
config names, building it first when there is none, and returns the text of db/schema.jai;
or what went wrong. Runs on a fiber, like every call that talks to the server.
The catalog of the migrations in migrations_folder and their schema hash, read from the
schema database on the server config names (built first when there is none); or what went
wrong. Runs on a fiber, like every call that talks to the server.
The text of db/schema.jai for a catalog made from migrations with this schema hash, or a
problem such as a column that cannot be a field name. Laid out as the formatter would.
Everything the generator needs from a migrated database.
The Jai names of the catalog's types. One whose preferred name is taken falls back to a longer one, so two tables never collapse into one struct.
The type of a column's field, or a problem: an enum the catalog does not list.
Diff#
Reads the catalog of the migrations' schema database (see schema_databases.jai), compares it
with the structs in schema_file, and returns the proposed SQL; or what went wrong. An empty
sql means nothing differs. Runs on a fiber, like every call that talks to the server.
The structs and enums of db/schema.jai, which must have the shape generate writes: one
declaration per line of the form Name :: struct @note(...) {, then name: Type; @note per
field line, then a } in the first column. Structs without @pg_table are not tables and are
skipped. Anything else in a table struct or an enum is a problem naming the line.
Compares the structs with the catalog and writes the SQL that would bring the migrated schema to what the structs say. See docs/postgres/diff.md for what it covers.
The Jai type text of a field as the Postgres type that would hold it, or an empty string for a
type that has no mapping. refs resolves the names of enums.
Main#
jaic runs the tool from the tool's own folder, so a relative path is taken from the folder the shell was in (PWD), when the shell says.
The compile-time database's settings from the environment, or a message on problem.
Commands that only read a schema built from the migrations, so a managed server can stand in
for DATABASE_URL. migrate and status work on a real database and need its URL.
The managed server (docs/postgres/managed.md), downloaded and started on first use and left running for the next build, as the describe service does.
Describes one entry again. True when it differs (and, unless checking, was rewritten).
The project's migrations, or a problem reported.
Creates db/migrations/NNNN_name.sql.
Writes the migration that would bring the migrated schema to the structs of db/schema.jai, unless nothing differs.
The path of db/migrations/NNNN_name.sql, numbered after the newest migration and as wide as
it; or a problem reported.
"Add users table" as "add_users_table".
Describe service#
The service's answer to one request.
The URL of the managed server, starting it if needed, or why it cannot.
The slot holding the connection for this pair, added if there is none.
Closes the held connection for this pair, if any.
The held connection for this pair, connecting when there is none, or why it cannot. Failures are remembered for a while.
Foreign#
Types#
What sort of failure a Pg_Error is. SERVER errors also carry the server's fields.
- NONE
- CONFIG
- a bad URL or setting, or a call made outside
Io.io_run - IO
- the connection failed;
iosays how - TIMED_OUT
- an
Io.with_deadlinelimit passed - CANCELED
- the fiber was canceled
- CLOSED
- the connection was closed, or broken by an earlier failure
- TLS
- the TLS handshake or a TLS record failed
- AUTH
- authentication was refused or could not be done safely
- PROTOCOL
- the server sent something this client does not understand
- SERVER
- an ErrorResponse:
code,messageand the rest are set - DECODE
- a value did not decode into the type asked for
- USAGE
- the call does not fit the result (a missing column, more rows than one, say)
- NOT_FOUND
query_onematched no row- HOST_REJECTED
- a server answered, but is not the kind
target_session_attrsasks for - ABORTED
- a pipelined statement was skipped because an earlier one of its batch failed
The five-character SQLSTATE, packed into a number so that it compares and switches like an
enum: if error.code == .UNIQUE_VIOLATION. Codes not listed here still compare and print
(sqlstate_text). Its class is the first two characters: sqlstate_class(code).
- NONE
- SUCCESSFUL_COMPLETION
- 00000
- WARNING
- 01000
- NO_DATA
- 02000
- CONNECTION_EXCEPTION
- 08000
- CONNECTION_DOES_NOT_EXIST
- 08003
- CONNECTION_FAILURE
- 08006
- PROTOCOL_VIOLATION
- 08P01
- FEATURE_NOT_SUPPORTED
- 0A000
- CARDINALITY_VIOLATION
- 21000
- DATA_EXCEPTION
- 22000
- STRING_DATA_RIGHT_TRUNCATION
- 22001
- NUMERIC_VALUE_OUT_OF_RANGE
- 22003
- NULL_VALUE_NOT_ALLOWED
- 22004
- INVALID_DATETIME_FORMAT
- 22007
- DATETIME_FIELD_OVERFLOW
- 22008
- DIVISION_BY_ZERO
- 22012
- CHARACTER_NOT_IN_REPERTOIRE
- 22021
- INVALID_PARAMETER_VALUE
- 22023
- INVALID_TEXT_REPRESENTATION
- 22P02
- INVALID_BINARY_REPRESENTATION
- 22P03
- INVALID_JSON_TEXT
- 22P05
- INTEGRITY_CONSTRAINT_VIOLATION
- 23000
- RESTRICT_VIOLATION
- 23001
- NOT_NULL_VIOLATION
- 23502
- FOREIGN_KEY_VIOLATION
- 23503
- UNIQUE_VIOLATION
- 23505
- CHECK_VIOLATION
- 23514
- EXCLUSION_VIOLATION
- 23P01
- INVALID_CURSOR_STATE
- 24000
- INVALID_TRANSACTION_STATE
- 25000
- ACTIVE_SQL_TRANSACTION
- 25001
- READ_ONLY_SQL_TRANSACTION
- 25006
- IN_FAILED_SQL_TRANSACTION
- 25P02
- IDLE_IN_TRANSACTION_SESSION_TIMEOUT
- 25P03
- INVALID_SQL_STATEMENT_NAME
- 26000
- INVALID_AUTHORIZATION_SPECIFICATION
- 28000
- INVALID_PASSWORD
- 28P01
- TRANSACTION_ROLLBACK
- 40000
- SERIALIZATION_FAILURE
- 40001
- STATEMENT_COMPLETION_UNKNOWN
- 40003
- DEADLOCK_DETECTED
- 40P01
- SYNTAX_ERROR_OR_ACCESS_RULE_VIOLATION
- 42000
- SYNTAX_ERROR
- 42601
- INSUFFICIENT_PRIVILEGE
- 42501
- CANNOT_COERCE
- 42846
- GROUPING_ERROR
- 42803
- DATATYPE_MISMATCH
- 42804
- WRONG_OBJECT_TYPE
- 42809
- UNDEFINED_COLUMN
- 42703
- UNDEFINED_FUNCTION
- 42883
- UNDEFINED_TABLE
- 42P01
- UNDEFINED_PARAMETER
- 42P02
- UNDEFINED_OBJECT
- 42704
- DUPLICATE_COLUMN
- 42701
- DUPLICATE_DATABASE
- 42P04
- DUPLICATE_FUNCTION
- 42723
- DUPLICATE_PREPARED_STATEMENT
- 42P05
- DUPLICATE_SCHEMA
- 42P06
- DUPLICATE_TABLE
- 42P07
- DUPLICATE_OBJECT
- 42710
- AMBIGUOUS_COLUMN
- 42702
- AMBIGUOUS_FUNCTION
- 42725
- INDETERMINATE_DATATYPE
- 42P18
- INVALID_COLUMN_REFERENCE
- 42P10
- INVALID_OBJECT_DEFINITION
- 42P17
- INSUFFICIENT_RESOURCES
- 53000
- DISK_FULL
- 53100
- OUT_OF_MEMORY
- 53200
- TOO_MANY_CONNECTIONS
- 53300
- PROGRAM_LIMIT_EXCEEDED
- 54000
- OBJECT_NOT_IN_PREREQUISITE_STATE
- 55000
- OBJECT_IN_USE
- 55006
- LOCK_NOT_AVAILABLE
- 55P03
- OPERATOR_INTERVENTION
- 57000
- QUERY_CANCELED
- 57014
- ADMIN_SHUTDOWN
- 57P01
- CRASH_SHUTDOWN
- 57P02
- CANNOT_CONNECT_NOW
- 57P03
- DATABASE_DROPPED
- 57P04
- IDLE_SESSION_TIMEOUT
- 57P05
- SYSTEM_ERROR
- 58000
- RAISE_EXCEPTION
- P0001
- NO_DATA_FOUND
- P0002
- TOO_MANY_ROWS
- P0003
- ASSERT_FAILURE
- P0004
- INTERNAL_ERROR
- XX000
- DATA_CORRUPTED
- XX001
What a call returns. It is small on purpose (the hot calls hand it back by value): the kind, the
SQLSTATE and the primary message. The rest of an ErrorResponse is in error_fields(error).
- kind: Error_Kind
- io: Io.Io_Error
- IO, TIMED_OUT, CANCELED: the Io error behind it
- code: Sqlstate
- SERVER
- message: string
- the primary message; for client errors, what happened
- owner: *Error_Fields
- Owns
messageand the strings of the fields: release it withfree_error. Null when the error holds nothing (no error).
The other fields of an ErrorResponse or NoticeResponse. Empty strings and zeros for the ones the server did not send, and for errors that did not come from the server.
- severity: string
- "ERROR", "FATAL", "PANIC" (never localized), "NOTICE", ...
- detail: string
- hint: string
- position: s32
- 1-based character in the query the error points at, or 0
- internal_position: s32
- internal_query: string
- where: string
- the call stack of a PL/pgSQL function, say
- schema: string
- table: string
- column: string
- data_type: string
- constraint: string
- file: string
- the server's source location
- line: string
- routine: string
- storage: [] u8
- The bytes every string of the error points into, stored after this struct in the same block.
How connect uses TLS, as libpq's sslmode.
- DISABLE
- never TLS
- PREFER
- TLS when the server offers it, no certificate check (the default, like libpq)
- REQUIRE
- TLS, no certificate check
- VERIFY_CA
- TLS, the chain must lead to a trusted root
- VERIFY_FULL
- TLS, the chain and the host name are checked
- ALLOW
- a plain connection first; TLS (no certificate check) only when the server refuses that
Which of several servers connect accepts, as libpq's target_session_attrs.
- ANY
- READ_WRITE
- transactions are not read-only (a primary, by default)
- READ_ONLY
- PRIMARY
- not in hot standby
- STANDBY
- in hot standby
- PREFER_STANDBY
- a standby if any host is one, else any
The order connect tries the hosts in, as libpq's load_balance_hosts.
- DISABLE
- as listed
- RANDOM
- shuffled for each connection
Whether connect sends a client certificate.
- DISABLE
- never send a client certificate
- ALLOW
- send the certificate in
ssl_cert_pemwhen there is one (the default) - REQUIRE
- fail unless a certificate was sent
The TLS versions libpq names in ssl_min_protocol_version and ssl_max_protocol_version. The
TLS library behind this client speaks 1.2 and 1.3 only, so a range that ends below 1.2 cannot
be used, and a minimum below it is raised to 1.2.
- TLS_1_0
- TLS_1_1
- TLS_1_2
- TLS_1_3
Whether SCRAM authentication binds to the TLS channel, as libpq's channel_binding.
- DISABLE
- plain SCRAM-SHA-256 only
- PREFER
- SCRAM-SHA-256-PLUS over TLS when the server offers it
- REQUIRE
- fail unless SCRAM-SHA-256-PLUS is used
Everything connect needs: servers, login, TLS and session settings. The defaults
suit a local server; parse_url fills one from a connection string.
- host: := "localhost"
- A host name or address, or a directory holding the server's Unix socket (starts with
/). - port: u16
- hostaddr: string
- An address to connect to instead of looking
hostup.hostis still the name the server's certificate is checked against and the password file is searched by. - hosts: [] Host_Port
- Every host of a connection string such as
host1:5432,host2:5433, the first one included. Empty for one host (hostandport). With several,connecttries them in turn until one accepts the login and suitstarget_session_attrs;host,hostaddrandportare then only the first entry's copy. - target_session_attrs: := Target_Session_Attrs.ANY
- load_balance_hosts: := Load_Balance_Hosts.DISABLE
- user: string
- empty: $USER
- password: string
- database: string
- empty: the user name
- application_name: := "jai"
- options: string
- sent as the
optionsstartup parameter: "-c search_path=app" - parameters: [] Startup_Parameter
- More startup parameters, or replacements for the client's own (
client_encoding,DateStyle,IntervalStyle), matched by name without regard to case. Text-format dates, times and intervals only decode with DateStyle ISO and IntervalStyle postgres. - send_styles: := true
- Send DateStyle ISO and IntervalStyle postgres. false leaves both to the server's settings (as psql does); text-format dates, times and intervals may then fail to decode.
- on_notice: Notice_Proc
- The connection's
on_noticeanduser_datafrom the start, so notices sent while logging in (a login event trigger's, say) reach them too. Null keeps the default, which logs. - user_data: *void
- json_options: Json_Options
- How
jsonandjsonbcolumns decode intoJson(T): unknown and missing fields, duplicate keys, depth and size limits. Each result starts with these, and itsjson_optionscan be changed beforeget. - ssl_mode: := Ssl_Mode.PREFER
- ssl_direct: := false
- Start TLS at once, with ALPN
postgresql, instead of asking first (PostgreSQL 17 and later): saves a round trip. Needsssl_modeREQUIRE or stricter. - ssl_root_cert_pem: string
- The roots to trust, as PEM. With
ssl_modeVERIFY_CA or VERIFY_FULL, empty means the system's roots. With REQUIRE or PREFER, roots given are checked all the same (the chain, not the name, as libpq does with a root.crt), and empty means no check. The chain must end in a self-signed certificate from this list. - ssl_crl_pem: string
- Certificate revocation lists (PEM, any number, or one DER) for the chain: if the server's certificate is listed, or none of them was issued by its issuer, the connection fails.
- ssl_cert_pem: string
- The client certificate chain (PEM, the client's certificate first) and its private key (PEM or
DER), for servers that ask for one.
ssl_key_passworddecrypts an encrypted key. Bothssl_cert_pemandssl_keyare needed to send a certificate;ssl_key_fileis only the name error messages use for the key. - ssl_key: string
- ssl_key_password: string
- ssl_key_file: string
- ssl_cert_mode: := Ssl_Cert_Mode.ALLOW
- ssl_sni: := true
- Send the host name as SNI, unless it is an IP address or a socket directory.
- ssl_key_log_file: string
- Appends the TLS session's secrets to this file, in the key log format Wireshark reads, for debugging. A file that cannot be opened is reported on stderr; the connection goes on.
- ssl_min_protocol_version: := Ssl_Protocol_Version.TLS_1_2
- ssl_max_protocol_version: := Ssl_Protocol_Version.TLS_1_3
- channel_binding: := Channel_Binding.PREFER
- min_protocol_version: := Protocol_Version.V3_0
- The protocol versions to use, like libpq's
min_protocol_versionandmax_protocol_version. The startup message asks for the maximum; a server that speaks less answers with the version it does, which the connection uses unless it is below the minimum (then connecting fails). The default is 3.0, as libpq's:.LATESTasks for 3.2, which PostgreSQL 18 and later speak. - max_protocol_version: := Protocol_Version.V3_0
- statement_cache: := Statement_Cache_Mode.ON
- Named prepared statements, kept per connection for typed queries (
Pg.query,execute, ...): the first run of a query parses it, later runs only bind..OFFuses unnamed statements only, for poolers that do not track named ones (PgBouncer before 1.21). The cache holdsstatement_cache_sizestatements; the least recently used is closed to make room. - statement_cache_size: := 256
- verify_statements: := false
- The first time a connection prepares a statement, ask the server what it returns and fail with an error if its columns are not the ones the query was compiled against, instead of reading the rows wrongly after a migration under a running program. Costs one round trip per statement and connection.
- statement_timeout: := 0
- The server's
statement_timeoutfor the session, in milliseconds; 0 keeps the server's own. Sent as a startup parameter, so a pooler that rejects unknown parameters needsoptions=-c statement_timeout=...or a role setting instead. - connect_timeout: := 10_000
- The whole connect (TCP, TLS, authentication), in milliseconds; negative: no limit.
- cancel_timeout: := 2_000
- After a deadline or cancel ends a query: how long to spend asking the server to stop it and reading what is left of its answer, in milliseconds. The connection breaks if that fails.
- max_scram_iterations: := 1_000_000
- A server could ask for any number of SCRAM iterations to waste the client's time.
- allow_cleartext_password: := false
- Send the password in clear text when the server asks, without TLS.
- passfile: string
- The password file to look in when
passwordis empty: lines ofhost:port:database:user:password,*matching anything,\escaping:and\. Empty:$PGPASSFILE, then~/.pgpass. A file others can read is ignored, as in libpq. - require_auth: string
- The authentication methods the server may use, as libpq's
require_auth: a comma separated list ofpassword,md5,gss,sspi,scram-sha-256andnone, or the same each with a!in front to forbid them. Empty: any. - owned: bool
- set by
parse_url: the strings belong to the config (free_config)
One run-time setting sent in the startup message (work_mem, search_path, ...).
- name: string
- value: string
One server of a multi-host connection string.
- host: string
- hostaddr: string
- port: u16
The options a connection string set. A keyword that was not given has no value; one given empty has an empty value, which is not the same.
- values: [] string
- is_set: [] bool
Authentication methods a server can ask for; a set of them is what require_auth allows.
- PASSWORD
- the server asked for the password in clear text
- MD5
- GSS
- SSPI
- SCRAM_SHA_256
- SASL
- NONE
- the server accepted the login without asking for anything
The methods require_auth allows. restricted is false when it is empty: anything goes.
- restricted: bool
- allowed: Auth_Method
A minor version of protocol 3, for Config.min_protocol_version and max_protocol_version.
Version 3.2 (PostgreSQL 18 and later) allows cancel keys of up to 256 bytes, where 3.0 has four.
- V3_0
- V3_2
- LATEST
- the newest this client speaks: 3.2
Where the session is in a transaction, as the server last reported it.
- IDLE
- outside a transaction block
- IN_TRANSACTION
- inside BEGIN ... COMMIT
- FAILED
- inside a transaction block that failed: only ROLLBACK works
A name and value, as the server reports run-time settings.
- name: string
- value: string
Called for each NoticeResponse (warnings, RAISE NOTICE). The notice is freed after the call.
Called for each NotificationResponse (a NOTIFY on a channel the connection LISTENs to), in the
middle of whatever request is reading. The notification is freed after the call. Without one,
notifications queue on the connection (next_notification, wait_for_notification).
An open session. Made by connect, released with close. One connection runs one request at a
time; use a Pool to share a database between fibers.
- status: Transaction_Status
- backend_pid: s32
- server_version: s64
- From
server_version: 170011 for 17.11, 150004 for 15.4. - parameters: [..] Parameter
- What the server reported with ParameterStatus: server_version, TimeZone, client_encoding,
standard_conforming_strings, ... (
parameter(connection, name)looks one up). - on_notice: Notice_Proc
- user_data: *void
- for the program: on_notice can find its state here
- on_notification: Notification_Proc
- notifications: [..] Notification
- Notifications received and not yet taken, oldest first, when
on_notificationis not set. - on_trace: Trace_Proc
set_trace- trace_data: *void
- protocol_minor: u32
- The minor protocol version in use: 2 (PostgreSQL 18 and later) or 0.
- protocol_requested: u32
- the minor version the startup message asked for
- protocol_required: u32
- the lowest minor version
min_protocol_versionaccepts - tls_used: bool
- channel_binding_used: bool
- Authentication used SCRAM-SHA-256-PLUS: the password exchange is bound to this TLS session.
- broken: bool
- Set when the connection can no longer be used: a failed read or write, a protocol error, or a FATAL error from the server.
- busy: bool
- a request is in flight
- pool: *Pool
- the pool it belongs to, if any
- host: string
- Below: the client's own state.
- tls_name: string
- The name the server's certificate is for (the host, not the address connected to), the name sent as SNI ("" for none), whether verify-full checks the name, and whether the handshake sent a client certificate.
- tls_sni: string
- key_log_file: string
- sslkeylogfile, for the handshake.
- tls_check_name: bool
- tls_cert_sent: bool
- port: u16
- unix_socket: bool
- ssl_direct: bool
- cancel_timeout: s64
- opened_at: s64
- monotonic ms, for a pool's max_lifetime
- cancel_key: [] u8
- json_options: Json_Options
- copied to each result
- tcp: Io.Tcp_Stream
- reader: Io.Reader
- writer: Io.Writer
- out: [..] u8
- message_start: s64
- message_type: u8
- of the message being written; 0 for the startup messages
- trace_regress: bool
- pipeline: Pipeline
- pipeline mode, when
pipeline.active - statements: Statement_Cache
- prepared statements, by query (statements.jai)
- input: [..] u8
- input_start: s64
One result column: its name and the type the server gave it.
- name: string
- table_oid: u32
- the table the column comes from; 0 for computed columns
- column_number: s16
- its attribute number in that table; 0 for computed columns
- type_oid: u32
- see
Oid - type_size: s16
- pg_type.typlen: negative for variable-length types
- type_modifier: s32
- pg_attribute.atttypmod: varchar(n), numeric(p, s), ...
- format: Format
- origin: string
- "table.column" when a typed query knows it (names decode errors); else empty
The rows of one statement. Strings and byte slices that get returns point into the result,
and arrays and numerics it builds belong to it: all stay valid until free_result.
- columns: [] Column
- row_count: s64
- command: string
- The command tag ("SELECT 3", "INSERT 0 1", "UPDATE 7", "CREATE TABLE") and the row count in it (0 when it has none).
- rows_affected: s64
- returns_rows: bool
- The statement returns rows (it sent a row description), even if there were none or it has
no columns:
selectreturns one row of zero columns. - partial: bool
- More rows of the same statement follow (
Script_Hooks.chunk_rows);commandis not set yet. - values: [..] Value_Span
- row_count * columns.count of them, row by row
- data: [..] u8
- json_options: Json_Options
- for
Json(T)columns; the connection's to begin with - json_query: Json_Options
- the query's own (
json = ...on a typed query), overjson_options - allocations: [..] *void
- allocator: Allocator
- context.allocator when the query ran: it owns everything above
- offset: s64
- length: s64
- -1 for NULL
What the server makes of a statement without running it: the types it infers for $1, $2,
... and the columns it would return (none for a statement that returns no rows).
- parameter_types: [] u32
- columns: [] Column
What run_script does with each statement's outcome. Every hook is optional.
- on_result: (hooks: *Script_Hooks, result: *Result)
- Called once per statement that completes, in order, with its rows (when it returns rows) and
command tag. The result is freed after the call unless the hook moves it out (and leaves
result.*zeroed). Without this hook rows are not kept at all. - chunk_rows: s64
- When positive, rows reach
on_resultin pieces of this many as they arrive, markedpartial, so a large result never has to fit in memory. The statement's last call has the remaining rows (maybe none) and the command tag. A partial result must not be moved out: its rows are dropped after the call and its columns kept for the next piece. - on_copy: (hooks: *Script_Hooks, copy: *Copy_Info)
- Called when the server starts a COPY, before any data moves.
- copy_in: (hooks: *Script_Hooks, copy: *Copy_Info) -> data : string, done: bool, failure: string
- COPY ... FROM STDIN: called until it says
done, each time for the next piece of data (any split; rows need not line up with calls). A non-emptyfailureaborts the COPY with that message instead, and the server answers with an error. Without this hook the COPY fails. - copy_out: (hooks: *Script_Hooks, copy: *Copy_Info, data: [] u8)
- COPY ... TO STDOUT: called with each piece of data the server sends (one row in text and CSV formats). Without this hook the data is dropped.
- user_data: *void
The shape of a COPY the server has started.
Whether a connection keeps prepared statements (Config.statement_cache).
- ON
- OFF
- unnamed statements only, for poolers that do not track named ones
How the server plans a typed query (plan = on query, execute, ...).
- AUTO
- a named statement: the server may switch to a generic plan after five runs
- CUSTOM
- the unnamed statement, planned again with the actual values on every run
Gets the data of a COPY ... TO STDOUT in the pieces the server sends it in. A piece is one row
in the text and CSV formats, and any slice of the stream in binary; it is valid during the call.
The rows of a binary COPY. Strings, byte slices and what arrays and numerics hold belong to
result: free them all with free_copy_rows.
The type OIDs this client knows by number. Other types (enums, domains, composites) get their OIDs when they are created, so they are only known by name.
- BOOL
- BYTEA
- CHAR
- NAME
- INT8
- INT2
- INT4
- TEXT
- OID
- JSON
- JSON_ARRAY
- FLOAT4
- FLOAT8
- UNKNOWN
- BPCHAR
- VARCHAR
- DATE
- TIME
- TIMESTAMP
- TIMESTAMPTZ
- INTERVAL
- NUMERIC
- UUID
- JSONB
- BOOL_ARRAY
- BYTEA_ARRAY
- CHAR_ARRAY
- NAME_ARRAY
- INT2_ARRAY
- INT4_ARRAY
- TEXT_ARRAY
- BPCHAR_ARRAY
- VARCHAR_ARRAY
- INT8_ARRAY
- OID_ARRAY
- FLOAT4_ARRAY
- FLOAT8_ARRAY
- TIMESTAMP_ARRAY
- DATE_ARRAY
- TIME_ARRAY
- TIMESTAMPTZ_ARRAY
- INTERVAL_ARRAY
- NUMERIC_ARRAY
- UUID_ARRAY
- JSONB_ARRAY
How a value travels on the wire: server text, or its binary form.
- TEXT
- BINARY
A value that may be SQL NULL. null is true for NULL, and value is then zero.
- value: T
- null: bool
A json or jsonb document holding a T. get decodes it with the Json module's decoder for
T, under the result's json_options and the query's json (json_options.jai); as a parameter
it is encoded with write_json. OPTIONS is a Json_Columns entry's options (packed by
pack_json_options), which the generated schema puts in the type; 0 says nothing.
write lets a parameter passed as Any find its encoder, so leave it as it is. Json(Json_Raw)
keeps the document undecoded: it is what a generated schema gives a json column nobody mapped.
- value: T
- write: (value: *T, builder: *String_Builder)
A json or jsonb document, undecoded: its text, which points into the result like any string.
decode_json turns it into a T later. Also a parameter, sent as it is.
- text: string
uuid: the 16 bytes in network order.
- bytes: [16] u8
timestamptz: an instant, in microseconds since 2000-01-01 00:00:00 UTC (Postgres' own epoch).
TIMESTAMP_INFINITY and TIMESTAMP_MINUS_INFINITY are infinity and -infinity.
- microseconds: s64
timestamp (without time zone): a wall-clock reading, in microseconds since 2000-01-01 00:00:00.
- microseconds: s64
date: days since 2000-01-01. DATE_INFINITY and DATE_MINUS_INFINITY are infinity and
-infinity.
- days: s32
time (without time zone): microseconds since midnight.
- microseconds: s64
interval: months, days and microseconds are kept apart, as Postgres keeps them, because a
month has no fixed number of days and a day has no fixed number of microseconds across a
daylight-saving change. From PostgreSQL 17 infinity is every field at S32_MAX / S64_MAX and
-infinity every field at the minimum.
- months: s32
- days: s32
- microseconds: s64
numeric: an exact decimal, as its text: "-12.3400", "0", "NaN", "Infinity", "-Infinity".
numeric_to_s64 and numeric_to_float64 convert it.
- text: string
Sizing and recycling rules for a Pool. Times are in milliseconds.
- max_size: := 10
- min_idle: := 0
- Kept open even when idle; opened in the background, so the first borrowers need not wait.
- idle_timeout: := 10 * 60_000
- An idle connection older than this is closed (ms; negative: never).
- max_lifetime: := 30 * 60_000
- A connection is closed after this long, less up to a tenth so that a fleet started together does not reconnect together (ms; negative: never).
- health_check_after: := 30_000
- A connection idle for longer is checked with an empty Sync before it is handed out (ms; negative: never).
- acquire_timeout: := 30_000
- How long
acquirewaits for a free connection (ms; negative: no limit).
Belongs to one scheduler: create it, and use it, on the fibers of one io_run (with
io_run_threads, one pool per thread). No locks are involved.
- config: Config
- settings: Pool_Config
- idle: [..] Idle_Connection
- Idle connections, the most recently used last: they are handed out first, so the rest go
stale and are closed by
idle_timeoutwhen the load drops. - open: s64
- Connections open or being opened, idle or borrowed: at most
max_size. - changed: Io.Condition
- A connection came back or a slot freed up.
- closed: bool
- maintaining: bool
- the background fiber runs: the pool is not freed under it
- thread_index: u32
- maintainer: Io.Fiber_Handle
- connection: *Connection
- since: s64
- milliseconds, monotonic
Transaction isolation level for transaction.
- DEFAULT
- the server's default_transaction_isolation
- READ_COMMITTED
- REPEATABLE_READ
- SERIALIZABLE
What a pipeline reported next.
- NONE
- nothing is waiting for a result
- RESULT
- a statement finished: its rows and command tag, or what a prepare/describe/close did
- ROWS
- chunked rows mode: some rows of a statement that goes on (
result.partial) - ERROR
- the server rejected the statement:
error - ABORTED
- the statement was skipped because an earlier one of its batch failed
- SYNC
- a sync point was reached: the batch is over and the server is ready again
What one queued statement or sync point turned into.
- kind: Pipeline_Kind
- result: Result
- For RESULT and ROW: the columns, rows and command tag. After a describe, only the columns.
- parameter_types: [] u32
- After
pipeline_describe_statement: the types the server inferred for$1,$2, ... - error: Pg_Error
- for ERROR
The kind of request queued in a pipeline.
- QUERY
- Parse (unless prepared), Bind, Describe portal, Execute
- PREPARE
- DESCRIBE_STATEMENT
- DESCRIBE_PORTAL
- CLOSE
- SYNC
The pipeline of a connection (pipeline_enter). While it is active, the connection answers no
other request.
- connection: *Connection
- active: bool
- aborted: bool
- A statement failed and its batch has not reached its sync point yet: the rest of the batch is skipped.
- steps: [..] Pipeline_Step
- queued, whose results are not all read; from
headon - head: s64
- unsynced: s64
- statements queued since the last sync point
- chunk_rows: s64
- the statement being read delivers this many rows per result (0: all at once)
- tag_ready: bool
- the statement's last rows were delivered, and its command tag is in
current - current: Pipeline_Result
- the result being assembled: a deadline in the middle loses nothing
- queued: s64
- statements queued since
pipeline_enter, sync points not counted - read: s64
- of those, how many have their result read
- early: [..] Early_Result
- read on the way to a later typed result (
take), not taken yet
A queued typed statement whose answer is still to come: take it after queuing the rest. R is
what take returns.
- serial: s64
A message from NOTIFY on a channel the connection LISTENs to.
- channel: string
- payload: string
- empty when NOTIFY gave none
- backend_pid: s32
- the session that sent it
A dedicated, self-healing LISTEN on a pool's database. listen(pool, "jobs") takes a connection
out of the pool for as long as the listener lives, LISTENs on it, and reads notifications on a
fiber of its own into a queue. If the connection fails (the server restarts, the backend is
terminated, the network drops) the listener returns it to the pool, takes another, LISTENs
again and carries on. Postgres keeps no notifications for a session that is not connected, so
those sent while the listener is reconnecting are lost; code that must not miss any should read
its table again after reconnects changes.
- queue: [..] Notification
- Notifications received and not yet taken, oldest first (
receive). The reader waits whenLISTENER_QUEUEof them are queued. - arrived: Io.Condition
- taken: Io.Condition
- closed: bool
- no more will arrive
- reconnects: s64
- How many times the connection was replaced after a failure.
- pool: *Pool
- channels: [..] string
- connection: *Connection
- fiber: Io.Fiber_Handle
- closing: bool
- finished: bool
- ended: Io.Condition
listener, error := Pg.listen(pool, "jobs", "mail");
while true {
n, error := Pg.receive(listener);
if error.kind != .NONE break;
defer Pg.free_notification(*n);
print("% on %\n", n.payload, n.channel);
}
Pg.close_listener(listener);Not run: the example is marked ignore.
Called with each traced line, without its newline. The line is in temporary storage.
One migration file, embedded at compile time by load_migrations.
- version: s64
- the number the file name starts with
- name: string
- the file name without
.sql - sql: string
- sha256: string
- hex digest of
sql - transaction: := true
- false for
-- jacket: no-transaction
Relaxes the checks migrate makes before applying anything.
- allow_modified: := false
- Apply even when an applied migration's file changed or is missing from the list.
- allow_out_of_order: := false
- Apply migrations numbered below the newest applied one (merged from another branch). Without it they are refused, since they were written against an older schema.
Where a migration stopped, for tools that point at the file.
- name: string
- the file name without
.sql - file: string
- its path, when the caller knows the folder
- index: s64
- in the list, from 0
- line: s64
- 1-based line of the file the server pointed at, or 0 when it gave no position
- column: s64
- message: string
- the server's message
Where one migration stands against a database.
- PENDING
- not applied yet
- APPLIED
- MODIFIED
- applied, but the file changed since
- MISSING
- applied, but no longer in the list (the entry has only version and name)
- OUT_OF_ORDER
- pending, but numbered below the newest applied migration
A migration and whether it is applied, changed, missing or out of order.
- version: s64
- name: string
- state: Migration_State
What decoding does with a JSON key the target struct has no field for.
- INHERIT
- IGNORE
- the built-in default: a key the struct has no field for is skipped
- DENY
- it is a decode error
What decoding does when a required field is absent from the JSON.
- INHERIT
- ERROR
- the built-in default: a field that is not optional must be there
- DEFAULT
- it keeps its declared initializer
What decoding does when a JSON object repeats a key.
- INHERIT
- LAST
- the built-in default: what
jsonbkeeps, and whatjsontext may contain - ERROR
How json and jsonb columns decode into structs. Layers of these (connection, query)
merge; INHERIT defers to the layer below.
- unknown_fields: := Json_Unknown_Fields.INHERIT
- missing_fields: := Json_Missing_Fields.INHERIT
- duplicate_keys: := Json_Duplicate_Keys.INHERIT
- max_depth: := 256
- nesting of arrays and objects
- max_size: := 1 << 30
- bytes of text
Reading inside a message body. A read past the end sets bad and returns zeros, so a parser
checks bad once at the end.
- data: [] u8
- at: s64
- bad: bool
- started: bool
- verified: bool
- the server proved it knows the password
- plus: bool
- channel binding is in use
- nonce: string
- client_first_bare: string
- binding: string
- the gs2 header, then the certificate hash for -PLUS
- server_signature: [32] u8
- key: [32] u8
- salted: [32] u8
- used: s64
- full: bool
One place where the rewritten text differs from the source: a % that became $n (hole is
n), or a %% that became % (hole is 0).
- source_offset: s64
- source_length: s64
- rewritten_offset: s64
- rewritten_length: s64
- hole: s64
- sql: string
- the statement the server sees
- edits: [..] Rewrite_Edit
- holes: s64
- how many
%holes the source has - error: string
- empty when the source scanned cleanly
- error_offset: s64
- where in the source the error is
- NONE
- NON_NULL
- the alias said
name! - NULLABLE
- the alias said
name?
- type: string
- how Postgres shows it: "bigint", "character varying(20)", "mood"
- base: string
- what decides the Jai type: domains resolved, arrays as "int4[]", enums as "enum"
- name: string
- without the alias suffixes
- type: string
- base: string
- nullable: bool
- origin: string
- schema.table.column, for a column read straight from a table
- jai: string
- the alias's Jai type, or empty
- forced: Forced_Nullability
- oid: u32
- in the database that described it; not stored
- key: string
- fingerprint: string
- format: s64
- ENTRY_FORMAT when written by this compiler; 0 when the file has none
- sql: string
- schema_hash: string
- server_major: s64
- search_path: string
- deps: [..] Query_Dep
- deps_complete: bool
- params: [..] Query_Type
- columns: [..] Query_Column
- op: string
- "describe", "fingerprints" or "ping"
- url: string
- the database; empty with
managed: the service's own managed server - managed: bool
- no URL: describe in the managed server (needs
migrations; see managed.jai) - managed_version: string
- which release the managed server runs; empty: the default
- search_path: string
- set before describing; empty: the server's own
- sql: string
- the rewritten statement (
$1,$2, ...) - schema_hash: string
- of the migrations; copied into the entry
- keys: [] string
- "fingerprints": the dependency keys to fingerprint
- migrations: string
- the migrations folder: answer from its schema database
- setup: string
- set by the service: SQL run first inside the describe's transaction
- setup_marks: [] Setup_Mark
- set by the service: where each migration lies in
setup - stale: Migration_Failure
- set by the service: the schema is the one before this failed migration
- min_server_version: s64
- the oldest server major version the project supports; 0: any
- first_seen: [..] Seen_Hash
- When each set of migrations (by schema hash) was first asked about without a database.
- failures: [..] Known_Failure
- Sets of migrations (by schema hash) whose build failed, so the failure is answered again without building again.
- hash: string
- failure: Migration_Failure
- hash: string
- milliseconds: s64
Where to describe: database on the same server (empty: the URL's own database) after running
setup inside the transaction. built names a database this call created, so the caller can
let go of connections to older ones and then prune_schema_databases.
- database: string
- setup: string
- setup_marks: [] Setup_Mark
- built: bool
- failed: bool
- A migration failed while building the database for the whole set.
databaseis then the schema before it (empty when no migration comes before it): answers from it are stale. - failure: Migration_Failure
Where one migration's SQL lies in setup, so a failure in the transaction can be traced to its
file.
- name: string
- index: s64
- start: s64
- byte offset in the setup text
- length: s64
Some fields of a table struct, with their types and notes: the rows of select(..., fields = ...).
fields is a comma-separated list of T's field names.
Contact :: Pg.Partial(User, "id, email");Doesn't run in the browser (skipped on wasm).
How Pg.each loads a has-many relation its loop body reads (for post: it.posts).
- PRELOAD
- a second query for every row's related rows, as
load_manydoes - LATERAL
- one query: each related column comes back as an array from a lateral join
- db: D
- error: *Pg_Error
The rows parents have through the has-many relation relation (a @pg_has_many note of
T), in one query: items[i] belongs to parents[i]. Free it with free_loaded; the items
read from its result.
- items: [][] Item
- rows: [] Item
- Every item once;
itemsare views into it. - result: Result
posts, error := Pg.load_many(db, users, "posts");
for users print("% wrote % posts\n", it.email, posts.items[it_index].count);Not run: the example is marked ignore.
The row each of parents points to through the to-one relation relation (a
@pg_belongs_to note of T), in one query: items[i] is null when parents[i] points nowhere.
A typed SQL NULL for any parameter: Pg.query(db, "update t set x = %", Pg.NULL).
The row of a described statement: one field per column, named by the column. shape is the
struct body the describe step wrote (doc comments included), so equal columns give one type.
The result of query: it owns the received data, and for rows hands out one row at a time.
Strings and byte slices in a row point into that data and stay valid until free_rows.
Rows of a streamed typed query, read one at a time with a for loop.
A one-field struct whose field has the type of a column: for query_scalar's return type.
One line of the hand-written db/json_columns.jai:
- column: string
- type: Type
- options: Json_Options
Json_Columns :: Pg.Json_Column.[ .{ "users.settings", Settings, .{ unknown_fields = .DENY } } ];Doesn't run in the browser (skipped on wasm).
A Postgres enum.
- key: string
"schema"."name", asDescribed_Type.qualified- schema: string
- name: string
- labels: [] string
- in sort order
- name: string
- sql_type: string
format_type, for the doc comment- base: string
- what
jai_type_oftakes: "int8", "text[]", "json", or "enum" / "enum[]" - enum_key: string
- the
Catalog_Enum.keyof an enum column or enum array - nullable: bool
- primary_key: bool
- has_default: bool
- a default, identity or generated: an insert may leave it out
- generated: bool
- always generated: an insert may not give it a value
A table, view, materialized view or foreign table.
- schema: string
- name: string
- view: bool
- columns: [] Catalog_Column
- schema: string
- the referencing table
- table: string
- columns: [] string
- target_schema: string
- target_table: string
- target_columns: [] string
- enums: [] Catalog_Enum
- relations: [] Catalog_Relation
- foreign_keys: [] Catalog_Foreign_Key
- enums: [] string
- the Jai type of each enum, in the catalog's order
- relations: [] string
- and of each relation
One field of a struct in the schema file.
- name: string
- type_text: string
- as written,
Pg.Nullable(...)included - primary_key: bool
- has_default: bool
- generated: bool
- json_text: bool
@pg_json: ajsoncolumn rather thanjsonb
A struct with an @pg_table note.
- type_name: string
- schema: string
- table: string
- belongs_to: [] string
- the arguments of the
@pg_belongs_tonotes - fields: [] Schema_Field
An enum of the schema file: a Postgres enum.
- type_name: string
- schema: string
- name: string
- labels: [] string
- in declaration order
What parse_schema_text reads from the file: the table structs and the enums.
- structs: [] Schema_Struct
- enums: [] Schema_Enum
The migration diff proposes.
- sql: string
- changes: s64
- statements that change the schema
- todos: s64
- review comments for changes it could not express
How an enum of the file is called in Postgres.
- type_name: string
- schema: string
- name: string
A connection kept between requests, and the backoff for reaching it again after a failure.
database empty is the URL's own database; otherwise a schema database on the same server.
- url: string
- database: string
- db: *Pg.Connection
- failed_until: s64
- monotonic ms: until then the last failure is the answer
- backoff: s64
- failure: string
The managed server this service process owns (managed.jai). It is started by the first request
that needs it and then reused; start_managed takes a lock, so two services (two compiles, or
a compile and the editor) never start the same data folder at once. The server keeps running
when the service exits, so the next build finds it ready.
- ready: bool
- url: string
- failed_until: s64
- monotonic ms: until then the last failure is the answer
- failure: string
- version: := MANAGED_VERSION
- Which release to use; it needs an entry in
MANAGED_ARCHIVES. - cache_dir: string
- Where the downloaded binaries live. Empty:
managed_cache_dir(). - data_dir: string
- The cluster's folder. Empty:
<cache_dir>/cluster-<major version>. - socket_dir: string
- Where the Unix socket goes (ignored on Windows). Empty:
<data_dir>/sock, or a short folder under /tmp when that path is too long for a socket. - port: u16
- 0: 5432 for a Unix socket (it only names the socket file), a free-looking port on Windows.
- timeout_seconds: := 60
- Seconds
pg_ctlwaits for the server to start or stop. - extra_conf: string
- Lines appended to the managed settings file, e.g. "max_connections = 50".
- config: Config
- connect with this; user
postgres, databasepostgres, no password - url: string
- the same as a
postgresql://URL - bin_dir: string
- data_dir: string
- socket_dir: string
- empty on Windows
- port: u16
- started_here: bool
- false: a server was already running on
data_dirand was reused
- MACOS
- LINUX
- WINDOWS
- version: string
- triple: string
- sha256: string
- of the
.tar.gz, as committed here
- command: string
- root: string
- build: string
- prune: bool
- out: string
- name: string
- the argument of
new
Constants#
Every keyword libpq accepts, in the order it lists them.
infinity and -infinity of timestamps and dates, as the server stores them.
Microseconds from 1970-01-01 (the Unix epoch) to 2000-01-01.
notifications held before the reader waits for the program
ms before the first reconnect attempt, doubling
Messages the server may send are never larger than this; a bigger length means the stream is out of step or hostile.
A message starts with its type byte and its four-byte length.
3 << 16
PostgreSQL 18 and later; older servers answer with version 3.0
AuthenticationRequest codes.
SaltedPassword (the PBKDF2 of the password) for the last few (password, salt, iterations)
triples of the process, so a pool that reconnects, or opens several connections at once, runs
the iterations once. The key is a hash of the triple: the cache holds no password text. The
salted value is as good as the password for logging in with SCRAM, and lives as long as the
process, as the password does in the connection's Config.
The version of what an entry holds and how its statement is rewritten. Raise it when the description changes shape, or when the placeholder rewrite, the alias syntax or anything else that decides what a statement means does: every entry written before is then a miss, described again when a database is there (or by the pg tool's prepare), and an error offline.
How long a set of migrations must stay unchanged before its database is built;
JACKET_PG_BUILD_DELAY_MS overrides it.
Schema databases kept on the server, newest first.
Defaults for select: no filter, no order.
The default release. Managed_Settings.version may name any other release the checksum table
below pins: the latest of each supported major version (15, 16, 17 and 18).
SHA-256 of every release archive the managed mode accepts, taken from the release's .sha256
files and cross-checked against the digests GitHub reports for the assets. A download is
compared with these, never with a checksum fetched next to it.
The port a Unix socket server uses by default (it only names the socket file), and the range Windows picks from (the dynamic ports).