Backup and recovery

Backup

The beCPG application backup is fully automated: a dedicated service runs permanently on the server and produces the application backups every night.

The backup includes:

• The backup of the search index;
• The backup of the databases;
• The backup of the contents and of the settings.

The backup order is important. By default it is guaranteed by the beCPG scripts and carried out as follows:

• 00:00 Statistics update (OLAP module, if installed)
• 03:00 Search index backup
• 03:30 Database backup
• 04:00 Content and configuration backup

All these backups are written on the data disk /mnt/becpg-data (/dev/sdb). The database backups are kept there for 7 days.

Customer responsibility

The data disk holds both the application data and its backups. It must be backed up once a day, around 5 a.m., using your hypervisor, once the application backups are complete.

Points to watch:

• Also keep a backup of the configured virtual machine: this documentation does not cover the reinstallation of the VM.
• Regularly check that the hypervisor backups complete successfully and that the retention applied matches your requirements.
• The available disk space must remain sufficient to hold the daily backups (see Server administration).

The beCPG monitoring checks every day that the application backups ran successfully and alerts the support team in case of anomaly.

In the rest of this document, $BACKUP_DIR refers to the data disk backup once restored on the machine from your hypervisor.

Restore

There are two cases, depending on the incident.

In every case, the restore is performed with the application stopped. The contents, the database and the search index must stay consistent with each other: never restore one without taking the others into account.

Case 1: full restore from the disk backup (recommended)

This is the safest method: the hypervisor backup holds the contents, the databases and the search index taken at the same moment, hence naturally consistent with each other.

1. Stop the application:

    cd /opt/becpg-srv-instances/inst1
    docker compose stop
   

2. Restore the /dev/sdb data disk from your hypervisor, at the wanted date, then mount it back on /mnt/becpg-data.

3. Start the application again:

    cd /opt/becpg-srv-instances/inst1
    docker compose up -d
   

4. Check the state of the search index:

    ./tools/solr.sh check
   

   If it differs, run ./tools/solr.sh fix (see Server administration).

Case 2: database only restore

To be used when only the application data must be rolled back to an earlier state (handling mistake, bulk import to cancel) and the documents are intact.

The daily database backups are available on the server, in the becpg_backups volume, under the becpg-db folder. They are named db_<yy>_<mm>_<dd>.gz and kept for 7 days. To locate the folder:

docker volume inspect becpg_backups --format '{{.Mountpoint}}'

Beyond 7 days, get the file from the hypervisor backup, under $BACKUP_DIR/docker/volumes/becpg_backups/_data/becpg-db/.

1. Stop the application services, leaving the database running:

    cd /opt/becpg-srv-instances/inst1
    docker compose stop becpg becpg-share solr becpg-report
   

2. Copy and uncompress the chosen backup:

    BACKUPS=$(docker volume inspect becpg_backups --format '{{.Mountpoint}}')
    cp $BACKUPS/becpg-db/db_26_08_17.gz /tmp/
    gunzip /tmp/db_26_08_17.gz
   

3. Restore the database. The supplied tool recreates the database then imports the backup; it reads the connection credentials itself, no password has to be typed:

    ./tools/db.sh restore /tmp/db_26_08_17
   

4. Start the application again:

    docker compose up -d
   

5. Rebuild the search index. The index reflects the state of the database before the restore: it must be rebuilt, otherwise the search returns wrong results.

    ./tools/solr.sh reindex
   

   Warning: reindexing can take more than 24 hours on a large instance. This is why the full disk restore (case 1) should be preferred whenever it is possible.

The case of the optional modules

• beCPG OLAP: the analysis cubes are rebuilt automatically from the PLM data on the next synchronisation. There is nothing to restore; the dashboards refill on their own within a few hours.
• beCPG AI: the module stores no business data, there is nothing to restore.

After the restore

• Check that every service is started (see Server administration).
• Log into the application and check a few recent products as well as the access to their documents.
• Report the incident to the support team through the incident manager, stating the date of the restored backup: this allows the consistency of the instance to be verified and, if needed, for you to be guided on the follow-up.