RDS PostgreSQL

Based on the RDS PostgreSQL docker image this Terraform module provides a ready to use PostgreSQL server that is backed up to a S3 compatible object store.

The module supports two backup methods either local storage (Hetzner cloud Volume) or a S3 compatible object store. To avoid data loss it is not possible to run the database without one of the backup methods enabled.

Design Goals

Acknowledging that there are a lot of available solutions to run PostgreSQL in the cloud it is important to emphasize the design goals of this solution to be able to make an informed decision on if and how to use it

  • First and foremost goal is to preserve the data stored in the PostgreSQL instance. This is ensured by an extensive set of full integration tests replicating the intended usage scenarios
  • Reduced complexity through single node operations. Running a PostgreSQL server with replication or in multi master comes with its own set of diverse failure scenarios and edge-cases. This solution aims at workloads that can easily handled by a single node and if in doubt favours MTTR over MTBF
  • Reduced complexity through less moving parts. Features like autoscaling, automatic recovery after zone outages, user interfaces, etc. like they are provided by the Zalando PostgreSQL Operator provide a lot of value but also add multiple layers between the VM and the actual PostgreSQL instance. This solutions explicitly tries to avoid these features trying to keep the stack as simple as running a docker container on a VM.

Operations

To access the created instance use one of SSH keys that was provided via ssh_keys. All linux system resources include the provided name to make them distinguishable.

Show Logs

journalctl -u rds@rds-postgresql-${name}.service

Start/Stop Databaase

systemctl [start|stop] rds@rds-postgresql-${name}.service 

Maintenance

In case of errors or the need for manual intervention a maintenance mode is available. Triggering the maintenance mode will set up the VM like it would be set up for database startup, but without actually starting the database. This allows to exec into the container to debug issues.

main.tf

module "rds-postgresql" {
  source  = "pellepelster/solidblocks-rds-postgresql/hcloud"
  # [...]
  mode = "maintenance"
  # [...]
}
docker exec -ti rds-postgresql-${name}_postgresql /bin/bash

Terraform Module

Requirements

Name Version
hcloud >=1.48.0
http >= 3.3.0

Providers

Name Version
hcloud >=1.48.0
http >= 3.3.0

Resources

Name Type
hcloud_server.rds resource
hcloud_volume_attachment.backup resource
hcloud_volume_attachment.data resource
hcloud_volume.backup data source
hcloud_volume.data data source
http_http.cloud_init_bootstrap_solidblocks data source

Inputs

Name Description Type Default Required
backup_encryption_passphrase If set the backups will be encrypted using this passphrase string null no
backup_full_calendar systemd timer spec for full backups string "*-*-* 20:00:00" no
backup_incr_calendar systemd timer spec for incremental backups string "*-*-* *:00:55" no
backup_local_retention_diff Local backup number of differential backups to retain. See https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-diff number 4 no
backup_local_retention_full Local backups full backup retention count/time. See https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-full number 7 no
backup_local_retention_full_type Local backups retention policy type [count, time]. See https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-full string "count" no
backup_s3_access_key AWS access key for S3 backups. To enable S3 backups backup_s3_bucket, backup_s3_access_key and backup_s3_secret_key have to be provided. string null no
backup_s3_bucket AWS bucket name for S3 backups. To enable S3 backups backup_s3_bucket, backup_s3_access_key and backup_s3_secret_key have to be provided. string null no
backup_s3_host AWS host S3 backups. string "s3.eu-central-1.amazonaws.com" no
backup_s3_region AWS region for S3 backups. string "eu-central-1" no
backup_s3_retention_diff AWS S3 backup number of differential backups to retain. See https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-diff number 4 no
backup_s3_retention_full AWS S3 backups full backup retention count/time. See https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-full number 7 no
backup_s3_retention_full_type AWS S3 backups retention policy type [count, time]. See https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-full string "count" no
backup_s3_secret_key AWS secret key for S3 backups. To enable S3 backups backup_s3_bucket backup_s3_access_key and backup_s3_secret_key have to be provided. string null no
backup_volume backup volume id string 0 no
data_volume data volume id number n/a yes
databases A list of databases to create when the instance is initialized, for example: { id : "database1", user : "user1", password : "password1" }. Changing user and password is supported at any time, the provided config is translated into an config for the Solidblocks RDS PostgreSQL module (https://pellepelster.github.io/solidblocks/rds/index.html), please see https://pellepelster.github.io/solidblocks/rds/index.html#databases for more details of the database configuration. list(object({ id : string, user : string, password : string })) n/a yes
db_admin_password The database admin password. Username is always rds string "" no
db_backup_gcs_bucket Name of the Google Cloud storage bucket string null no
db_backup_gcs_service_key content of the service key json file with appropriate permissions to write to the db_backup_gcs_bucket bucket. string null no
environment_variables A list environment variables to pass to the PostgreSQL docker container map(string) {} no
extra_user_data deprecated, please use pre_script/post_script string "" no
firewall_disable disable automatic firewall configuration bool false no
labels A list of labels to be attached to the server instance. map(any) {} no
location Hetzner location to use for provisioned resources string n/a yes
mode startup mode for the database, can be empty to start the database or ‘maintenance’ to enable the maintenance mode of the underlying docker container to debug issues see also https://pellepelster.github.io/solidblocks//rds/#maintenance string null no
name Unique name for the PostgreSQL instance string n/a yes
network_id network the created sever should be attached to, network_ip also needs to bet set in that case number null no
network_ip ip address in the attached network. when an ip address is provided the database server will automatically be bound to this ip and will not be exposed on any other network interfaces string null no
post_script shell script that will be executed after the server configuration is executed string "" no
postgres_extra_config Extra postgres configurations options for the postgresql.conf, see also https://pellepelster.github.io/solidblocks/rds/index.html#global -> DB_POSTGRES_EXTRA_CONFIG string null no
postgres_major_version PostgreSQL major version to use. Upgrading the version will trigger auto migration based on the underlying RDS PostgreSQL docker image, see also https://pellepelster.github.io/solidblocks/rds/index.html#versions. Please be aware that depending on the amount of data to migrate the migration may Terraforms timeouts, see https://pellepelster.github.io/solidblocks/hetzner/rds-postgresql/index.html#operations for debugging options. number 14 no
postgres_stop_timeout shutdown timeout for the postgres database in seconds, see also https://www.postgresql.org/docs/current/app-pg-ctl.html number 60 no
pre_script shell script that will be executed before the server configuration is executed string "" no
public_net_ipv4_enabled enable/disable public ip addresses, see also https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/server#public_net bool true no
public_net_ipv6_enabled enable/disable public ip addresses, see also https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/server#public_net bool true no
restore_pitr Point in time to recover to, using the recovery type time as defined in https://pgbackrest.org/command.html#command-restore. Format should be YYYY-MM-dd HH:mm:ssz Please be aware that the server hosting the database might be in a different timezone, so always include the timezone when specifying PITR times date +"%Y-%m-%d %H:%M:%S%z" string null no
server_type Hetzner cloud server type, supports x86 and ARM architectures string "cx22" no
solidblocks_base_url override base url for testing purposes string "https://github.com" no
solidblocks_cloud_init_version Solidblocks cloud-init version to use string "v0.3.0" no
solidblocks_rds_version Solidblocks rds-postgresql version to use string "v0.3.0" no
ssh_keys ssh keys to provision for instance access list(number) n/a yes
ssl_acme_server The URL of the ACME Server to use. Defaults to Let’s Encrypt production environment. string "https://acme-v02.api.letsencrypt.org/directory" no
ssl_dns_provider provider type to use for LetsEncrypt DNS challenge, see https://go-acme.github.io/lego/dns/ for available options string "" no
ssl_dns_provider_config environment configuration variable(s) to use for DNS provider selected via ssl_dns_provider, see documentation of selected provider for required configuration variables map(string) {} no
ssl_domains domains to use for generated certificates list(string) [] no
ssl_email email address to use for LetsEncrypt account creation string "" no
ssl_enable enable automatic ssl certificate creation using LetsEncrypt bool false no

Outputs

Name Description
ipv4_address IPv4 address of the created server if applicable
ipv4_address_private private IPv4 address of the created server if applicable
ipv6_address IPv6 address of the created server if applicable
this_server_id Hetzner ID of the created server
user_data n/a