How to Backup PostgreSQL Database to Exoscale Using Bash

Exoscale is an S3-compatible object storage service that helps users to store and serve large amounts of data over the internet. It is an alternate solution to AWS S3 and offers simple, affordable, and predictable pricing. With Exoscale, you can backup or serve your data from any Exoscale zone with no hidden fees.

This post will show you how to backup PostgreSQL databases to Exoscale using Bash.

Create a Bucket on Exoscale
Create Exoscale Credentials
Install S3cmd Tool
Configure S3cmd to Connect PostgreSQL Server to Exoscale
Use S3cmd to Backup PostgreSQL Database to Exoscale Bucket
Conclusion

Create a Bucket on Exoscale

First, you will need to create a bucket on Exoscale where you want to store your PostgreSQL database. Follow the below steps to create a bucket.

Step 1 - First, sign in to your Exoscale account.

Step 2 - Click on the STORAGE on the left pane, you should see the following screen:

Step 3 - Click on the ADD button. You should see the bucket creation screen:

Step 4 - Provide your bucker name, choose your Zone and click on the Create button. Once the bucket is created, you should see the following screen:

Create Exoscale Credentials

Next, you will need to create an Exoscale credential to connect your bucket from the remote server. Follow the below steps to create an Exoscale credential:

Step 1 - On the Exoscale dashboard, click on the AMI in the left pane. You should see the following screen:

Step 2 - Click on ADD KEY button. You should see the following screen:

Step 3 - Provide your key name and click on the CREATE button. You should see your created access key and secret key on the following screen:

Install S3cmd Tool

Next, you will need to install the S3cmd tool on your PostgreSQL database server from where you want to back up your PostgreSQL database.

S3cmd is a free and open-source command-line tool that allows you to upload, download and manage data in Exoscale bucket and other cloud storage service providers. Follow the below steps to install the S3cmd to your server.

First, install the Python and other dependencies by running the following command:

apt-get install python3 python3-setuptools curl -y

Next, download the latest version of S3cmd using the following command:

curl -LO https://github.com/s3tools/s3cmd/releases/download/v2.2.0/s3cmd-2.2.0.tar.gz

Next, extract the downloaded file with the following command:

tar -xvzf s3cmd-2.2.0.tar.gz

Next, navigate to the extracted directory and install it using the following command:

cd s3cmd-2.2.0
python3 setup.py install

Once the S3cmd is installed, verify the S3cmd version using the following command:

s3cmd --version

You will get the following output:

s3cmd version 2.2.0

Configure S3cmd to Connect PostgreSQL Server to Exoscale

Next, you will need to configure the S3cmd using the Exoscale Access key and Secret key. You will require the following details to configure S3cmd:

Access Key: FEO2c60fbb123a7c263cc53751d
Secret Key: KhnddL-AA8DpYRG8o_nlnsdngPosi5KfsaoSEdA1s9k
Region: Frankfurt-de-fra-1
S3 Endpoint: sos-de-fra-1.exo.io
host_bucket: %(bucket)s.sos-de-fra-1.exo.io

Next, run the following command to configure S3cmd tool:

s3cmd --configure

You will be asked to provide your Exoscale Access Key, Secret Key, Region, and Endpoint as shown below:

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
 
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: FEO2c60fbb123a7c263cc53751d
Secret Key: KhnddL-AA8DpYRG8o_nlnsdngPosi5KfsaoSEdA1s9k
Default Region [US]: Frankfurt-de-fra-1
 
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]: sos-de-fra-1.exo.io
 
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: %(bucket)s.sos-de-fra-1.exo.io
 
Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program:
 
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]:
 
On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:
 
New settings:
Access Key: FEO2c60fbb123a7c263cc53751d
Secret Key: KhnddL-AA8DpYRG8o_nlnsdngPosi5KfsaoSEdA1s9k
Default Region: Frankfurt-de-fra-1
S3 Endpoint: sos-de-fra-1.exo.io
DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.sos-de-fra-1.exo.io
Encryption password:
Path to GPG program: None
Use HTTPS protocol: True
HTTP Proxy server name:
HTTP Proxy server port: 0
 
Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)
 
Now verifying that encryption works...
Not configured. Never mind.
 
Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'

Once the S3cmd is configured, you can verify your Exoscale Bucket information using the following command:

s3cmd info s3://postgresql-exoscale/

You will get the following output:

s3://postgresql-exoscale/ (bucket):
Location: de-fra-1
Payer: BucketOwner
Expiration Rule: none
Policy: none
CORS: <?xml version="1.0" encoding="UTF-8"?><CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"></CORSConfiguration>
ACL: snapshooter: FULL_CONTROL

Use S3cmd to Backup PostgreSQL Database to Exoscale Bucket

First, log in to your server and connect to the PostgreSQL shell with the following command:

su - postgresql
psql

Once you are connected, list all PostgreSQL databases using the following command:

postgres=# \l

You will get all databases in the following output:

List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
testdb | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
(7 rows)

Next, use the pg_dumpall tool to backup all PostgreSQL databases:

pg_dumpall > dump.sql

If you want to backup a single database, run the following command:

pg_dump -U username -W -F t -d testdb > testdb.tar

Next, run the S3cmd command to copy the dump.sql file to the Exoscale Bucket:

s3cmd put dump.sql s3://postgresql-exoscale/

You will get the following output:

upload: 'dump.sql' -> 's3://postgresql-exoscale/dump.sql' [1 of 1]
8388608 of 8388608 100% in 1s 5.99 MB/s done
upload: 'dump.sql' -> 's3://postgresql-exoscale/dump.sql' [1 of 1]
8388608 of 8388608 100% in 0s 12.10 MB/s done

If you want to copy a single backup file, run the following command:

s3cmd put testdb.tar s3://postgresql-exoscale/

You will get the following output:

upload: 'testdb.tar' -> 's3://postgresql-exoscale/testdb.tar' [1 of 1]
2097152 of 2097152 100% in 0s 2.01 MB/s done
upload: 'testdb.tar' -> 's3://postgresql-exoscale/testdb.tar' [1 of 1]
2097152 of 2097152 100% in 0s 6.12 MB/s done

You can now verify all backups on the Exoscale Bucket using the following command:

s3cmd ls s3://postgresql-exoscale

You will get the following output:

2022-07-28 06:18 8388608 s3://postgresql-exoscale/dump.sql
2022-07-28 06:18 2097152 s3://postgresql-exoscale/testdb.tar

You can also log in to the Exoscale Bucket and verify your PostgreSQL backup files as shown below:

Conclusion

In this post, you learned how to backup the PostgreSQL database to the Exoscale bucket using bash. I hope this post will help you to backup and manage your PostgreSQL database to and from the Exoscale bucket.