Crate cuprate_database

Cuprate’s database abstraction.

This documentation is mostly for practical usage of cuprate-database.

For a high-level overview, see the database section in Cuprate’s architecture book.

If you need blockchain specific capabilities, consider using the higher-level cuprate-blockchain crate which builds upon this one.

Purpose

This crate abstracts various database backends with traits.

All backends have the following attributes:

• Embedded
• Multiversion concurrency control
• ACID
• Are (key, value) oriented and have the expected API (get(), insert(), delete())
• Are table oriented ("table_name" -> (key, value))
• Allows concurrent readers

The currently implemented backends are:

• heed (LMDB)
• redb

Terminology

To be more clear on some terms used in this crate:

Term	Meaning
Env	The 1 database environment, the “whole” thing
DatabaseR{o,w}	A actively open readable/writable key/value store
Table	Solely the metadata of a Database (the key and value types, and the name)
TxR{o,w}	A read/write transaction
Storable	A data type that can be stored in the database

The flow is Env -> Tx -> Database

Which reads as:

1. You have a database Environment
2. You open up a Transaction
3. You open a particular Table from that Environment, getting a Database
4. You can now read/write data from/to that Database

Concrete types

You should not rely on the concrete type of any abstracted backend.

For example, when using the heed backend, Env’s associated TxRw type is RefCell<heed::RwTxn<'_>>. In order to ensure compatibility with other backends and to not create backend-specific code, you should not refer to that concrete type.

Use generics and trait notation in these situations:

• impl<T: TxRw> Trait for Object
• fn() -> impl TxRw

ConcreteEnv

This crate exposes ConcreteEnv, which is a non-generic/non-dynamic, concrete object representing a database Environment.

The actual backend for this type is determined via feature flags.

This object existing means E: Env doesn’t need to be spread all through the codebase, however, it also means some small invariants should be kept in mind.

As ConcreteEnv is just a re-exposed type which has varying inner types, it means some properties will change depending on the backend used.

For example:

• size_of::<ConcreteEnv>
• align_of::<ConcreteEnv>

Things like these functions are affected by the backend and inner data, and should not be relied upon. This extends to any struct/enum that contains ConcreteEnv.

ConcreteEnv invariants you can rely on:

• It implements Env
• Upon Drop::drop, all database data will sync to disk

Note that ConcreteEnv itself is not a cloneable type, it should be wrapped in std::sync::Arc.

Defining tables

Most likely, your crate building on-top of cuprate_database will want to define all tables used at compile time.

If this is the case, consider using the define_tables macro to bulk generate zero-sized marker types that implement Table.

This macro also generates other convenient traits specific to your tables.

Feature flags

Different database backends are enabled by the feature flags:

• heed (LMDB)
• redb

The default is heed.

tracing is always enabled and cannot be disabled via feature-flag.

Examples

The below is an example of using cuprate-database.

use cuprate_database::{
    ConcreteEnv,
    config::ConfigBuilder,
    Env, EnvInner,
    DatabaseRo, DatabaseRw, TxRo, TxRw,
};

// Create a configuration for the database environment.
let tmp_dir = tempfile::tempdir()?;
let db_dir = tmp_dir.path().to_owned();
let config = ConfigBuilder::new(db_dir.into()).build();

// Initialize the database environment.
let env = ConcreteEnv::open(config)?;

// Define metadata for a table.
struct Table;
impl cuprate_database::Table for Table {
    // The name of the table is "table".
    const NAME: &'static str = "table";
    // The key type is a `u8`.
    type Key = u8;
    // The key type is a `u64`.
    type Value = u64;
}

// Open up a transaction + tables for writing.
let env_inner = env.env_inner();
let tx_rw = env_inner.tx_rw()?;
// We must create the table first or the next line will error.
env_inner.create_db::<Table>(&tx_rw)?;
let mut table = env_inner.open_db_rw::<Table>(&tx_rw)?;

// Write data to the table.
table.put(&0, &1)?;

// Commit the data written.
drop(table);
TxRw::commit(tx_rw)?;

// Read the data, assert it is correct.
let tx_ro = env_inner.tx_ro()?;
let table = env_inner.open_db_ro::<Table>(&tx_ro)?;
assert_eq!(table.first()?, (0, 1));

Modules

• config
  Database Env configuration.
• resize
  Database memory map resizing algorithms.

Macros

• define_tables
  Define all table types.

Structs

• ConcreteEnv
  A strongly typed, concrete database environment, backed by heed.
• StorableBytes
  A Storable version of Bytes.
• StorableStr
  A Storable string.
• StorableVec
  A Storable vector of T: Storable.

Enums

• InitError
  Errors that occur during (Env::open).
• KeyCompare
  Comparison behavior for Keys.
• RuntimeError
  Errors that occur after successful Env::open.

Constants

• DATABASE_BACKEND
  Static string of the crate being used as the database backend.
• DATABASE_CORRUPT_MSG
  Corrupt database error message.
• DATABASE_DATA_FILENAME
  Cuprate’s database filename.
• DATABASE_LOCK_FILENAME
  Cuprate’s database lock filename.

Traits

• DatabaseIter
  Database (key-value store) read-only iteration abstraction.
• DatabaseRo
  Database (key-value store) read abstraction.
• DatabaseRw
  Database (key-value store) read/write abstraction.
• Env
  Database environment abstraction.
• EnvInner
  The inner Env type.
• Key
  Database Table key metadata.
• Storable
  A type that can be stored in the database.
• Table
  Database table metadata.
• TxRo
  Read-only database transaction.
• TxRw
  Read/write database transaction.