vfs

Go SQLite VFS API

This package implements the SQLite OS Interface (aka VFS).

It replaces the default SQLite VFS with a pure Go implementation, and exposes interfaces that should allow you to implement your own custom VFSes.

Since it is a from scratch reimplementation, there are naturally some ways it deviates from the original.

The main differences are file locking and WAL mode support.

File Locking

POSIX advisory locks, which SQLite uses on Unix, are broken by design. Instead, on Linux and macOS, this package uses OFD locks to synchronize access to database files.

This package can also use BSD locks, albeit with reduced concurrency (BEGIN IMMEDIATE behaves like BEGIN EXCLUSIVE). BSD locks are the default on BSD and illumos, but you can opt into them with the sqlite3_flock build tag.

On Windows, this package uses LockFileEx and UnlockFileEx, like SQLite.

You can also opt into a cross platform locking implementation with the sqlite3_dotlk build tag.

Otherwise, file locking is not supported, and you must use nolock=1 (or immutable=1) to open database files. To use the database/sql driver with nolock=1 you must disable connection pooling by calling db.SetMaxOpenConns(1).

You can use vfs.SupportsFileLocking to check if your build supports file locking.

Write-Ahead Logging

On Unix, this package may use mmap to implement shared-memory for the WAL-index, like SQLite.

With BSD locks a WAL database can only be accessed by a single proccess. Other processes that attempt to access a database locked with BSD locks, will fail with the SQLITE_PROTOCOL error code.

You can also opt into a cross platform, in-process, memory sharing implementation with the sqlite3_dotlk build tag.

Otherwise, WAL support is limited, and EXCLUSIVE locking mode must be set to create, read, and write WAL databases. To use EXCLUSIVE locking mode with the database/sql driver you must disable connection pooling by calling db.SetMaxOpenConns(1).

You can use vfs.SupportsSharedMemory to check if your build supports shared memory.

Batch-Atomic Write

On Linux, this package may support batch-atomic writes on the F2FS filesystem.

Checksums

This package can be configured to add an 8-byte checksum to the end of every page in an SQLite database. The checksum is added as each page is written and verified as each page is read.
The checksum is intended to help detect database corruption caused by random bit-flips in the mass storage device.

The implementation is compatible with SQLite's Checksum VFS Shim.

Build Tags

The VFS can be customized with a few build tags:

• sqlite3_flock forces the use of BSD locks.
• sqlite3_dotlk forces the use of dot-file locks.
• sqlite3_nosys prevents importing x/sys.

Important

The default configuration of this package is compatible with the standard Unix and Windows SQLite VFSes; sqlite3_flock builds are compatible with the unix-flock VFS; sqlite3_dotlk builds are compatible with the unix-dotfile VFS. If incompatible file locking is used, accessing databases concurrently with other SQLite libraries will eventually corrupt data.

Custom VFSes

• github.com/ncruces/go-sqlite3/vfs/memdb implements an in-memory VFS.
• github.com/ncruces/go-sqlite3/vfs/readervfs implements a VFS for immutable databases.
• github.com/ncruces/go-sqlite3/vfs/adiantum wraps a VFS to offer encryption at rest.
• github.com/ncruces/go-sqlite3/vfs/xts wraps a VFS to offer encryption at rest.