Backing Up AtScale

AtScale System Back-ups

As part of your regular system back-ups, it good practice to backup AtScale as a fundamental part of IT and business continuity planning. Below you’ll find details regarding the common back-up procedures that can be part of your process. Each company’s backup and recovery plans are unique, so you might use this as a example. Not included here is frequency, offsite storage, encryption or recovery testing/validation procedures.

PostgreSQL DB

At the heart of the AtScale deployment is the PostgreSQL database. There are several approaches for backing up PostgreSQL (e.g. pg dump, volume mount backup, etc). If you are running AtScale with an external PostgreSQL DB, you can also use options/utilities available with that managed service (e.g. snapshots).

A very typical PostgreSQL backup process would be PG Dump. The following command can be used:

PostgreSQL instances are not exposed by default, therefore it is required to port forward the PostgreSQL Connection Pool Load Balancer port to the instance where the backup will be made. This can be achieved by

kubectl port-forward -n <NAMESPACE> svc/postgres-pgpool <DESTINATION_PORT>:10518

where <DESTINATION_PORT> should be replaced by a free port on the instance. The recommended port is 10518.

After exposing the port, in a separate terminal with pg_dumpall tool installed (Ref: pg_dumpall ) , the data can be backed up by:

1. Setting the environment variable PGPASSWORD with the PostgreSQL default user password
   export PGPASSWORD=$(kubectl get secret -n <NAMESPACE> postgres-default-user -o json | jq -r '.data.password' | base64 -d

2. Running pg_dumpall in order to extract the data
   pg_dumpall -h 127.0.0.1 -p <DESTINATION_PORT> -U postgres --no-password > backup.dump
   where <DESTINATION_PORT> should be set to the same port specified in the kubectl port-forward command.

The data on the AtScale Internal PostgreSQL instance will be exported to the backup.dump file, and it can be restored as needed with the use of the pg_restore (ref: pg_restore ) tool.

Values Override file

The values override file has all of your environment’s details and any customization required for a successful install. Consider versioning and storing the value override files.

SML

Since SML is stored in your git repository, there is little need to backup SML. However note that saving a file in Design Center does not save into git. While in Design Center → Source Control, be sure to commit SML changes to git to ensure your SML is stored and protected.

Configuration

The following (not exhaustive list) should be backed-up in case you want to stand-up a new environment with tested configuration. (Note: Restoring PostgreSQL will restore most of this. But backing up the configuration is good practice in case you want to standup a clean environment with existing configuration.)

• License

• Identity Provider set up

• Git repositories

• Data Warehouse configuration

  • Connection names

  • Data platform details (server, URL, extra flags, etc)

  • Aggregate Database and Schema

  • Impersonation details

  • Access Controls

  • Query Mapping

• Aggregate Definitions

• API token

• Integrations

  • Tableau Server

  • Looker Server/Repo

  • Alation or Collibra connector set-up *

*Data Catalog connector set-up is outside of AtScale and not found in a PostgreSQL back-up