Developers Club geek daily blog

3 years, 5 months ago
Bareos (Backup Archiving Recovery Open Sourced) is the centralized, client-server and krossplatformerny system for backup, an archiving and data recovery.

Backup by means of Bareos and Relax-and-Recover

Is a forok of Bacula and actively develops. Has clients under Linux and Windows. The complete list of the supported features on Github.

The principles of work and configuration completely match Bacula therefore I will not stop in detail on the description of all parameters. Articles about Bacula are on Habré ("Step-by-step modification of the preset Bacula setup", "Bacula: for those to whom it is necessary quickly and in pictures", "Setup and understanding of Bacula", "The centralized data backup of Windows and * to nix of servers means of Bacula").

Hosts:
testbackup-bareos – the Bareos server at which bconsole, bareos-director, file daemon, storage daemon work. The IP address – 10.11.0.150
testbackup-client-03 – the client who we bekapit. OS Dew Cobalt 1.0. file-daemon works at it. The IP address – 10.11.0.153

Setup of the Bareos server


I selected Bareos 15. X, as here work with sockets is improved. But this instruction will be suitable also for a branch 14.X.
We set Bareos from a repository and we configure PostgreSQL:

wget -O /etc/yum.repos.d/bareos.repo http://download.bareos.org/bareos/experimental/nightly/CentOS_6/bareos:master.repo
yum install bareos bareos-common bareos-database-postgresql bareos-client bareos-bconsole bareos-storage bareos-filedaemon bareos-director
su postgres -c /usr/lib/bareos/scripts/create_bareos_database
su postgres -c /usr/lib/bareos/scripts/make_bareos_tables
su postgres -c /usr/lib/bareos/scripts/grant_bareos_privileges

The configuration of the client is in directory/etc/bareos/bareos-dir.d:

testbackup-client-03.conf:
Job {
  Name = "testbackup-client-03-backup"
  Type = Backup
  Level = Incremental
  Client = testbackup-client-03-fd
  FileSet = "testbackup-client-03-fileset"
  Schedule = "WeeklyCycle"
  Storage = File
  Messages = Standard
  Pool = testbackup-client-03-pool-Incremental
  Priority = 10
  Write Bootstrap = "/var/lib/bareos/%c.bsr"
  Full Backup Pool = testbackup-client-03-pool-Full
  Differential Backup Pool = testbackup-client-03-pool-Differential
  Incremental Backup Pool = testbackup-client-03-pool-Incremental
}
Job {
  Name = "testbackup-client-03-restore"
  Type = Restore
  Client = testbackup-client-03-fd
  FileSet = "testbackup-client-03-fileset"
  Storage = File
  Pool = testbackup-client-03-pool-Incremental
  Messages = Standard
  Where = /
}

Job {
  Name = "testbackup-client-03-backup-mysql"
  Type = Backup
  Level = Incremental
  Client = testbackup-client-03-fd
  FileSet = "testbackup-client-03-fileset-mysql"
  Schedule = "WeeklyCycle"
  Storage = File
  Messages = Standard
  Pool = testbackup-client-03-pool-Incremental
  Priority = 10
  Write Bootstrap = "/var/lib/bareos/%c.bsr"
  Full Backup Pool = testbackup-client-03-pool-Full
  Differential Backup Pool = testbackup-client-03-pool-Differential
  Incremental Backup Pool = testbackup-client-03-pool-Incremental
}
Job {
  Name = "testbackup-client-03-restore-mysql"
  Type = Restore
  Client = testbackup-client-03-fd
  FileSet = "testbackup-client-03-fileset-mysql"
  Storage = File
  Pool = testbackup-client-03-pool-Incremental
  Messages = Standard
  Where = /
}

FileSet {
  Name = "testbackup-client-03-fileset"
  Include {
    Options {
        compression = GZIP
        Signature = MD5 # calculate md5 checksum per file
    }
    File = "/"
    File = "/boot"
  }
}

FileSet {
  Name = "testbackup-client-03-fileset-mysql"
  Include {
    Plugin = "bpipe:file=/MYSQL/dump.sql:reader=mysqldump -uzabbix -p12345 --single-transaction --add-drop-database -B zabbix:writer=mysql -uzabbix -p12345"
    Options {
        Signature = MD5 # calculate md5 checksum per file
        compression = GZIP
    }
  }
}

Client {
  Name = testbackup-client-03-fd
  Address = testbackup-client-03
  Password = "Msmn8GRO0CnMyUljLmB9u1wgrbYXoV1rI8MLugI6JJqc "
  File Retention = 30 days # 30 days
  Job Retention = 6 months # six months
  AutoPrune = no # Prune expired Jobs/Files
}

Pool {
  Name = testbackup-client-03-pool-Full
  Pool Type = Backup
  Recycle = yes # Bareos can automatically recycle Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 365 days
  Maximum Volume Bytes = 30G
  Maximum Volumes = 100
  Label Format = "testbackup-client-03-pool-Full-"
}
Pool {
  Name = testbackup-client-03-pool-Differential
  Pool Type = Backup
  Recycle = yes # Bareos can automatically recycle Volumes
  AutoPrune = yes
  Volume Retention = 90 days
  Maximum Volume Bytes = 10G
  Maximum Volumes = 100
  Label Format = "testbackup-client-03-pool-Differential-"
}
Pool {
  Name = testbackup-client-03-pool-Incremental
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 30 days
  Maximum Volume Bytes = 1G
  Maximum Volumes = 100
  Label Format = "testbackup-client-03-pool-Incremental-"
}

Here 4 tasks – 2 Job for backup/recovery of all system (/and / boot) and 2 Job for backup/recovery of a DB of MySQL Zabbix with use of a payp (bpipe) are created. In our example the backup of a DB becomes only for demonstration of work of bpipe. The dump of base does not remain locally at the client, it is given on a network at once.

Structure of bpipe:

Plugin = "<plugin>:file=<filepath>:reader=<readprogram>:writer=<writeprogram>"

plugin – a plug-in name without prefix - fd.so;
filepath – a pseudo-way to Volume to the file of a backup. This way and the file name are necessary only for logical representation in a directory tree at recovery;
readprogram – the program which stdout reads out bpipe and transfers to the server;
writeprogram – the program to which stdin bpipe transfers data from the server.

Director configuration in directory/etc/bareos:

bareos-dir.conf
@/etc/bareos/bareos-dir.d/testbackup-client-03.conf  #включаем конфигурацию клиента

Director {
  Name = testbackup-bareos-dir
  QueryFile = "/usr/lib/bareos/scripts/query.sql"
  Maximum Concurrent Jobs = 10
  Password = "lY/wVlXuC25qk18i15iw10MpOVxLwDLFgOUMtSMpFOC6"
  Messages = Daemon
  Auditing = yes
  Maximum Concurrent Jobs = 20
}
Schedule {
  Name = "WeeklyCycle"
  Run = Full 1st sat at 21:00
}
Storage {
  Name = File
  Address = testbackup-bareos
  Password = "k4dkFWMwJh3zBpnAOpa9+oule1z/OT0GIcE4wZYySgbU"
  Device = FileStorage
  Media Type = File
}
Catalog {
  Name = MyCatalog
  dbdriver = "postgresql"
  dbname = "bareos"
  dbuser = "bareos"
  dbpassword = ""
}
Messages {
  Name = Standard
  mailcommand = "/usr/bin/bsmtp -h localhost -f \"\(Bareos\) \<%r\>\" -s \"Bareos: %t %e of %c %l\" %r"
  mailcommand = "/etc/bareos/scripts/bareoszabbix.bash %i"
 operatorcommand = "/usr/bin/bsmtp -h localhost -f \"\(Bareos\) \<%r\>\" -s \"Bareos: Intervention needed for %j\" %r"
  mail = root@localhost = all, !skipped, !audit
  operator = root@localhost = mount
  console = all, !skipped, !saved, !audit
  append = "/var/log/bareos/bareos.log" = all, !skipped, !audit
  catalog = all, !audit
}
Messages {
  Name = Daemon
  mailcommand = "/usr/bin/bsmtp -h localhost -f \"\(Bareos\) \<%r\>\" -s \"Bareos daemon message\" %r"
  mail = root@localhost = all, !skipped, !audit
  console = all, !skipped, !saved, !audit
  append = "/var/log/bareos/bareos.log" = all, !skipped, !audit
  append = "/var/log/bareos/bareos-audit.log" = audit
}

bconsole configuration in directory/etc/bareos:
bconsole.conf
Director {
  Name = testbackup-bareos-dir
  DIRport = 9101
  address = testbackup-bareos
  Password = "lY/wVlXuC25qk18i15iw10MpOVxLwDLFgOUMtSMpFOC6"
}

Storage Daemon configuration in directory/etc/bareos:

bareos-sd.conf
Storage {
  Name = testbackup-bareos-sd
  Maximum Concurrent Jobs = 20
}
Director {
  Name = testbackup-bareos-dir
  Password = "k4dkFWMwJh3zBpnAOpa9+oule1z/OT0GIcE4wZYySgbU"
}
Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /backup
  LabelMedia = yes;
  Random Access = yes;
  AutomaticMount = yes;
  RemovableMedia = no;
  AlwaysOpen = no;
}
Messages {
  Name = Standard
  director = testbackup-bareos-dir = all
}

If the customer name does not rezolvitsya, then we register in / etc/hosts:
echo "10.11.0.153 testbackup-client-03">> / etc/hosts

We start Director, File Daemon and Storage:

service bareos-dir start
service bareos-sd start
service bareos-fd start

On it setup of the Bareos server is finished. We pass to setup of the client of testbackup-client-03.

Setup of the client of Bareos.


We set a repository of bareos and we put File Daemon:

yum -y install bareos-filedaemon

File Daemon configuration in directory/etc/bareos:

bareos-fd.conf
Director {
  Name =  testbackup-bareos-dir
  Password = "Msmn8GRO0CnMyUljLmB9u1wgrbYXoV1rI8MLugI6JJqc"
}

FileDaemon {
  Name = testbackup-client-03-fd
  Maximum Concurrent Jobs = 20
  Compatible = no #совместимость с Bacula отключена, чтобы использовать все фичи bareos-fd
}

Messages {
  Name = Standard
  director = testbackup-client-03-dir = all, !skipped, !restored
}

On it setup of the client is finished.

Backup/recovery start


We go on the testbackup-bareos server and we start a backup of the MySQL database and a complete backup:

#bconsole
*run testbackup-client-03-backup-mysql
*run testbackup-client-03-backup

For recovery of the database it is necessary to start the corresponding task:
#bconsole
*restore

And to follow on the offered points.
Or, if we know JobId (for example, 158), then at once to start:

*restore restorejob=testbackup-client-03-restore-mysql jobid=158

Installation and Relax-and-Recover setup


REAR (Relax-and-Recover) is the solution Linux BMDR (Bare Metal Disaster Recovery), system recovery on pure iron.

Backup by means of Bareos and Relax-and-Recover

Main features:
  • Recovery on pure iron: physical-to-virtual (P2V), virtual-to-physical (V2P), physical-to-physical (P2P) and virtual-to-virtual (V2V), systems of virtualization of KVM, Xen, VMware
  • Types of loading devices: ISO, USB, eSATA, OBDR/bootable tape, PXE
  • Protocols for data transmission: HTTP, HTTPS, FTP, SFTP, NFS, CIFS (SMB)
  • Types of breakdown of a disk: HWRAID (HP SmartArray), SWRAID, LVM, multipathing, DRBD, iSCSI, LUKS (encrypted partitions and filesystems)
  • Backends: CommVault Galaxy, EMC NetWorker (Legato), HP DataProtector, IBM Tivoli Storage Manager (TSM), SEP Sesam, Symantec NetBackup, Bacula, Bareos, duplicity/duply

Other features are on the website.

In our example we will consider creation of a boot image ISO and a complete recovery of system of the client of testbackup-client-03, using as Bareos backend.

The boot ISO image is created on the client and transferred on NFS to the server (in our case on the Bareos server) to the nfs://10.11.0.150/backup folder
For this purpose it is previously necessary to configure on the server access on NFS.
Later we pass to setup of the client of testbackup-client-03:

We set bconsole:

yum -y install bareos-bconsole

/ etc/bareos/bconsole.conf
Director {
  Name = testbackup-bareos-dir
  DIRport = 9101
  address = testbackup-bareos
  Password = "lY/wVlXuC25qk18i15iw10MpOVxLwDLFgOUMtSMpFOC6"
}

For our Cobalt DEW distribution kit we connect EPEL repository:

rpm -ivh http://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm

Let's comment out mirrorlist and we will uncomment baseurl in / etc/yum.repos.d/epel.repo:

epel.repo
[epel]
name=Extra Packages for Enterprise Linux 6 - $basearch
baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch;=$basearch
failovermethod=priority
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

Let's connect CentOS repository in / etc/yum.repos.d/CentOS-Base.repo:
CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
baseurl=http://mirror.centos.org/centos/6.6/os/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

Also we import a GPG key:

rpm --import http://mirror.centos.org/centos/6.6/os/x86_64/RPM-GPG-KEY-CentOS-6

We install REAR and necessary components:

yum -y install nfs-utils genisoimage syslinux rear

We add information on ip, the interface and the default gateway to the ip_addresses and route files:

mkdir /etc/rear/mappings
echo "eth0 10.11.0.153/24" > /etc/rear/mappings/ip_addresses
echo "default 10.11.0.1 eth0" > /etc/rear/mappings/route

In order to avoid problems with name resolution of Job, Client I recommend that the name File Daemon of the client precisely corresponded to its hostname. I.e. if testbackup-client-03 server hostname, then in the section FileDaemon has to be written to bareos-fd.conf:
Name = testbackup-client-03-fd

We configure local.conf from a template:

cp /usr/share/rear/conf/default.conf /etc/rear/local.conf

I do not publish a complete configuration of local.conf since I changed only the following parameters in comparison with default.conf:

local.conf
OS_VENDOR=CentOS
OS_VERSION=6.6
BACKUP=BAREOS
OUTPUT=ISO
BAREOS_CLIENT=$(grep $(hostname -s) /etc/bareos/bareos-fd.conf | awk '/-fd/ {print $3}' )
OUTPUT_URL=nfs://10.11.0.150/backup
USE_STATIC_NETWORKING=y

As I use the Cobalt DEW distribution kit (version 6.6 rhel-based) and therefore I set OS_VENDOR and OS_VERSION as for CentOS 6.6.

For descriptive reasons I use the template of names for Job, File Set, Client therefore it is necessary to edit a script of recovery of REAR, otherwise the script will not find all necessary data.

vi /usr/share/rear/restore/BAREOS/default/40_restore_backup.sh

We find a line:

BAREOS_CLIENT=$(grep $(hostname -s) /etc/bareos/bareos-fd.conf | awk '/-fd/ {print $3}' )

And in the same place (before it, for example) we set a new variable:

BAREOS_CLIENT_1=$(grep "Name =.*-fd" /etc/bareos/bareos-fd.conf | awk '{print $3}' | sed -e 's/-fd//g' )

Instead of:

echo "restore client=$BAREOS_CLIENT where=/mnt/local select all done

We write:

echo "restore client=$BAREOS_CLIENT_1-fd restorejob=$BAREOS_CLIENT_1-restore fileset=$BAREOS_CLIENT_1-fileset where=/mnt/local select all done

We do boot ISO:

rear -v -d mkrescue

For information: the received ISO size makes about 30-40 MB.
For recovery we are loaded from this image and we select Recover testbackup-client-03.

Backup by means of Bareos and Relax-and-Recover

We start recovery process:

rear -v recover


Backup by means of Bareos and Relax-and-Recover

Upon termination of process of recovery we check our system which is mounted in/MNT / local.
We are convinced that all on site and we reboot:

Backup by means of Bareos and Relax-and-Recover

The system is completely recovered.

Before using a boot image for any OS, I recommend it to test previously.

This article is a translation of the original post at habrahabr.ru/post/260955/
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here: sysmagazine.com@gmail.com.

We believe that the knowledge, which is available at the most popular Russian IT blog habrahabr.ru, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus