Docker

Containerized PostgreSQL database with all batteries included backup solution powered by pgBackRest

RDS PostgreSQL aims at being easy to use while keeping data a safe as possible. Based on the conventions of the official PostgreSQL docker image it can be configured by tuning different environment variables.

Architecture

The naming conventions for the data directories are implemented according to the common storage conventions. Configuration and backup operations are encoded in the startup script run.sh that implements the following flow

graph TD
    startup[container startup] -->|get postgres major version| set_data_dir["`set postgres_data_dir to 
    **<postgres_data_base_dir>/<major_version>**`"]
    set_data_dir -->|get previous postgres major version| has_old_data{"`has is old 
    data present in 
    **<postgres_data_base_dir>/
    <previous_version>**`"}
    has_old_data -->|no| data_dir_empty{"` is **<postgres_data_dir>** 
    is empty`"}
    has_old_data -->|yes| migrate_data["`migrate data from **<postgres_data_base_dir>/
    <previous_version>** to **<postgres_data_base_dir>/
    <version>**`"]
    backup_exists -->|no| init_db
    data_dir_empty -->|no| init_db
    data_dir_empty -->|yes| backup_exists{backup exists?}
    backup_exists -->|yes| restore[restore from backup]
    migrate_data --> init_users
    restore --> init_users[initialize / update schemas and users]
    init_db[initialize database] --> initial_backup
    initial_backup[initial backup] --> init_users
    init_users --> start_database[start database]

Versions

The PostgreSQL version can be selected via specific docker image tags

  • ghcr.io/pellepelster/solidblocks-rds-postgresql:14-v0.4.12
  • ghcr.io/pellepelster/solidblocks-rds-postgresql:15-v0.4.12
  • ghcr.io/pellepelster/solidblocks-rds-postgresql:16-v0.4.12
  • ghcr.io/pellepelster/solidblocks-rds-postgresql:17-v0.4.12
  • ghcr.io/pellepelster/solidblocks-rds-postgresql:18-v0.4.12

Each docker image includes at least the previous PostgreSQL version for version migration purposes. The current version can be shown with pg_versions and the versions are stored at /usr/libexec/postgresql/${postgresql_major_version}

Version upgrades

The database can be updated simply by switching to antother docker image tag, where all versions from the official PostgreSQL repository are available as described in the general documentation.

For details of the update process please refer to this part of the documentation, the following steps are recommended for a version upgrade:

  • execute a full backup (backup-full.sh)
  • stop container with currently running version
  • start new container with the same configuration but a new PostgreSQL version

Configuration

Global

configurationtypedescription
DB_INSTANCE_NAMEenvironmentunique name of this database instance
DB_ADMIN_PASSWORDenvironmentPassword for the db superuser, if not set a random password will be assigned. Username for the superuser is rds
DB_POSTGRES_EXTRA_CONFIGenvironmentExtra postgres configurations options for the postgresql.conf
DB_BACKUP_ENCRYPTION_PASSPHRASEenvironmentPassphrase to use for backup encryption. If no passphrase is provided backups will be stored unencrypted
DB_BACKUP_FULL_SCHEDULEenvironmentCRON expression specifying when should full backups be executed. If empty, full backups are disabled. Default: empty string.
DB_BACKUP_DIFF_SCHEDULEenvironmentCRON expression specifying when should differential backups be executed. If empty, differential backups are disabled. Default: empty string.
DB_BACKUP_INCR_SCHEDULEenvironmentCRON expression specifying when should incremental backups be executed. If empty, incremental backups are disabled. Default: empty string.
/some/data/dir:/storage/datamountContainer volume mount for the PostgreSQL data directory. The docker image uses a user with uid 10000, which needs to be reflected in the directory permissions
/some/backup/dir:/storage/backupmountContainer volume mount for the pgBackRest backup repository directory. The docker image uses a group with gid 10000, which needs to be reflected in the directory permissions

Based on the functionality of pgBackRest three types of backup repositories are supported. Local filesystem (local), n S3 compatible object storage (s3) or Google cloud storage based (gcs). Those can be configured individually, but at least one type has to be configured.

Local Backup

configurationtypedefaultdescription
DB_BACKUP_LOCALenvironment0Flag to enable local filesystem as backup repository
DB_BACKUP_LOCAL_RETENTION_FULL_TYPEenvironmentcountRetention type for full backups, see retention type documentation
DB_BACKUP_LOCAL_RETENTION_FULLenvironment7Retention for full backups, see retention full documentation
DB_BACKUP_LOCAL_RETENTION_DIFFenvironment4Retention for diff backups, see retention diff documentation

S3 Backup

The S3 backup target works with an S3 compatible service, so apart from AWS you can for example also use Hetzner Cloud Storage. In most cases you should only need to point DB_BACKUP_S3_HOST to your specific provider.

configurationtypedescription
DB_BACKUP_S3environmentFlag to enable S3 object storage as backup repository
DB_BACKUP_S3_HOSTenvironmentHostname of the S3 object storage service, can be used to use different S3 providers. E.g. to use Hetzer Object storage in fsn use fsn1.your-objectstorage.com. Defaults to AWS if empty
DB_BACKUP_S3_BUCKETenvironmentBucket for the backup repository
DB_BACKUP_S3_ACCESS_KEYenvironmentAccess key for the backup bucket
DB_BACKUP_S3_SECRET_KEYenvironmentSecret key for the backup bucket
DB_BACKUP_S3_CA_PUBLIC_KEYenvironmentPublic key for the CA that issued the certificates for the DB_BACKUP_S3_HOST. Useful when a non SaaS solution like minIO is used.
DB_BACKUP_S3_RETENTION_FULL_TYPEenvironmentRetention type for full backups, see retention type documentation
DB_BACKUP_S3_RETENTION_FULLenvironmentRetention for full backups, see retention full documentation
DB_BACKUP_S3_RETENTION_DIFFenvironmentRetention for diff backups, see retention diff documentation

Google storage bucket backup

configurationtypedefaultdescription
DB_BACKUP_GCSenvironment0Flag to enable Google Cloud storage as backup repository
DB_BACKUP_GCS_BUCKETenvironment<none>Name of the Google Cloud storage bucket
DB_BACKUP_GCS_SERVICE_KEY_BASE64environment<none>Base64 encoded service key file with appropriate permissions to write to the DB_BACKUP_GCS_BUCKET bucket
DB_BACKUP_GCS_RETENTION_FULL_TYPEenvironmentcountRetention type for full backups, see retention type documentation
DB_BACKUP_GCS_RETENTION_FULLenvironment7Retention for full backups, see retention full documentation
DB_BACKUP_GCS_RETENTION_DIFFenvironment4Retention for diff backups, see retention diff documentation

Restore

Point-In-Time-Recovery (PITR) is supported by providing a specific time to restore. See pgBackRest restore documentation for more details on the configuration options.

configurationtypedefaultdescription
RESTORE_PITRenvironment<none>Point in time to recover to, in the format YYYY-MM-dd HH:mm:ssz. Please be aware that the server hosting the database might be in a different timezone, so always include the timezone when specifying PITR times

Databases

Multiple databases can automatically be provisioned by providing configurations for multiple distinct unique ${database_id}s

per database configurationtypedescriptionexample
DB_DATABASE_${database_id}environmentname of the database that will be crated when the PostgreSQL is initialized. The databaase id must adhere to the limitations of shell environment variable naming ([a-zA-Z_])-
DB_USERNAME_${database_id}environmentname of the user who will be granted full access to DB_DATABASE_${database_id}-
DB_PASSWORD_${database_id}environmentpassword for the database user-
DB_CREATE_OPTIONS_${database_id}environmentoptions for database creation, see documentation, WITH will be automatically added of an option is providedENCODING=‘UTF8’ LC_COLLATE=‘de_DE.UTF8’ LC_CTYPE=‘de_DE.UTF8’ TEMPLATE=‘template0’
Tip

DB_USERNAME_${database_id} and DB_PASSWORD_${database_id} can be changed at any time and will be re-provisioned on start to allow for easy password rotation or username change. Changing DB_DATABASE_${database_id} is currently not supported.

Extensions

The following PostgreSQL extensions are available by default

nameversiondescription
adminpack2.1administrative functions for PostgreSQL
amcheck1.3functions for verifying relation integrity
autoinc1.0functions for autoincrementing fields
bloom1.0bloom access method - signature file based index
btree_gin1.3support for indexing common datatypes in GIN
btree_gist1.6support for indexing common datatypes in GiST
citext1.6data type for case-insensitive character strings
cube1.5data type for multidimensional cubes
dblink1.2connect to other PostgreSQL databases from within a database
dict_int1.0text search dictionary template for integers
dict_xsyn1.0text search dictionary template for extended synonym processing
earthdistance1.1calculate great-circle distances on the surface of the Earth
file_fdw1.0foreign-data wrapper for flat file access
fuzzystrmatch1.1determine similarities and distance between strings
hstore1.8data type for storing sets of (key, value) pairs
insert_username1.0functions for tracking who changed a table
intagg1.1integer aggregator and enumerator (obsolete)
intarray1.5functions, operators, and index support for 1-D arrays of integers
isn1.2data types for international product numbering standards
lo1.1Large Object maintenance
ltree1.2data type for hierarchical tree-like structures
moddatetime1.0functions for tracking last modification time
old_snapshot1.0utilities in support of old_snapshot_threshold
pageinspect1.9inspect the contents of database pages at a low level
pg_buffercache1.3examine the shared buffer cache
pg_cron1.6Job scheduler for PostgreSQL
pg_freespacemap1.2examine the free space map (FSM)
pg_prewarm1.2prewarm relation data
pg_remote_exec1.0remote shell execution for non-superusers
pg_stat_statements1.9track planning and execution statistics of all SQL statements executed
pg_surgery1.0extension to perform surgery on a damaged relation
pg_trgm1.6text similarity measurement and index searching based on trigrams
pg_visibility1.2examine the visibility map (VM) and page-level visibility info
pgaudit1.6.3provides auditing functionality
pgcrypto1.3cryptographic functions
pgrowlocks1.2show row-level locking information
pgstattuple1.5show tuple-level statistics
plpgsql1.0PL/pgSQL procedural language
postgres_fdw1.1foreign-data wrapper for remote PostgreSQL servers
refint1.0functions for implementing referential integrity (obsolete)
seg1.4data type for representing line segments or floating-point intervals
sslinfo1.2information about SSL certificates
tablefunc1.0functions that manipulate whole tables, including crosstab
tcn1.0Triggered change notifications
tsm_system_rows1.0TABLESAMPLE method which accepts number of rows as a limit
tsm_system_time1.0TABLESAMPLE method which accepts time in milliseconds as a limit
unaccent1.1text search dictionary that removes accents
uuid-ossp1.1generate universally unique identifiers (UUIDs)
xml21.1XPath querying and XSLT