Docker Volume Backups

Dugite-Code — Wed, 19 Feb 2020 16:00:00 GMT

Backups are always an issue. I plan to switch to ZFS for the snapshotting and remote sync features, until then I have take the useful volume-backup and broken it until it works with rdiff-backup

Build the container

First you have to clone the repo and build the container

git clone https://github.com/dugite-code/volume-backup.git
cd volume-backup
docker image build -t vbackup:1.0 .

Backup

Now you can run the container mounting the [volume-name] at /volume and your [backup-dir] at /backup

docker run -v [volume-name]:/volume -v [backup-dir]:/backup --rm vbackup:1.0 backup

Additional rdiff-backup options and be passed via the -o switch and a quoted option for example -o "--exclude ignore.me"

Restore

To restore you must supply some form of options i.e. -o "-r 10D" or restore backup from 10 Days ago

docker run -v [volume-name]:/volume -v [backup-dir]:/backup --rm vbackup:1.0 restore -o "-r 10D"

Trimming old files

With incremental backups it's important to occasionally trim old files that just don't exist anymore. Like Restore you must provide some form of option.

Prior to doing a backup I will run this command to remove files older than 20 Backups

docker run -v [backup-dir]:/backup --rm vbackup:1.0 remote -v -o "--remove-older-than 20B"

I hope you found this helpful. It's not a very clean script, I had to hack apart the reference script quite a bit in order to get it all working. But it serves it's purpose quite well.

I am a Docker Convert

Dugite-Code — Mon, 20 Jan 2020 16:00:00 GMT

I've changed my mind quite a bit when it comes to docker. I used to be a big believer in virtual machines, I still am, but for individual 'applications' Docker makes a fair bit of sense.

Reasons to I use Docker

Simplicity

Docker is the simplest way to replicate a developer's environment on your own computer. No more dealing with differing distro's varying update cycles and the conflicting packages causing edge case issues, because everything is in it's own little box. Nice and predictable.

This saves you time setting things up because at least all the components are included. Configuration is still a pain on some projects, but at least your not missing any metaphorical screws.

The biggest example for this was my mailserver. I used modoboa a great simple mailserver package. The issues were having things brake from system package updates and just updating the package itself was damned complicated. I learnt a lot from these breakages, so much so when I switched to docker I switched to using docker-mailserver a image that has no Web GUI for configuration.

Updates, while problematic to monitor in docker are now a simple painless affair.

Lightweight

Unlike a virtual machine you don't need to replicate everything in a container. This makes it easier to have more services that conflict with each other running side by side. I used to have one dedicated NUC for my mailserver and another for all my other services. I've now condensed it all onto the single NUC with better overall performance thanks to docker.

Portability

One of the biggest advantages to docker is portability. If you take your raw data and docker-compose files throw them onto a completely separate machine and within a few minutes you are up and running again. For virtual machines this would take significant work and, in my experience often fails.

The Issues I have with docker

The pre-built images

The Alpine image root issue last year, where the base image used to build a large number of docker images shipped with a vulnerability, made it obvious you need an actively maintained update cycle.

If the project you are using doesn't provide a docker image or even a dockerfile you will often find pre-built images on docker-hub. The big question you need to ask is if you can trust these images. Check the source repository and decide if it would make more sense to build the image yourself.

Keeping pre-built images up-to date

One of the biggest issues people have with docker is the lack of update tracking. Thankfully this can be overcome using the Watchtower image.

I set watchtower to monitor only mode because automatic updates are sometimes a terrible idea.

watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped
    environment:
      - WATCHTOWER_POLL_INTERVAL=86400 #Poll every 24 hours
      - WATCHTOWER_MONITOR_ONLY=true
      - WATCHTOWER_NOTIFICATIONS=gotify
      - WATCHTOWER_NOTIFICATION_GOTIFY_URL=https://example.tld/gotify/
      - WATCHTOWER_NOTIFICATION_GOTIFY_TOKEN=###########

Importantly for locally built images add the disable label to their docker-compose files, or you will constantly get notifications saying (info): Unable to update container /examplecontainer. Proceeding to next.

  labels:
   - com.centurylinklabs.watchtower.enable=false

Peek Read Info (Posts about docker)