Backup with Bareos and ZFS - by Christian Reiß
0 likes226 views
The document presents a talk by Christian Reiß on backup solutions using Bareos, ZFS, and Puppet at Symgenius, a mid-sized hosting company. It emphasizes the challenges faced with previous backup methods, the advantages of using Bacula and ZFS for fast, efficient, and reliable backups, and the implementation of Puppet for automated system management. The speaker provides insights into the features of ZFS and the configuration of backups through the Puppet module.
![19
ZFS - Features
The features of ZFS include:
● […] protection against data
corruption,
● continuous integrity checking and
automatic repair [...]](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-19-320.jpg)
![20
ZFS - Features
The features of ZFS include:
● […] RAID-Z [...]](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-20-320.jpg)
![21
ZFS - Features
The features of ZFS include:
● […] support for high storage
capacities, [...]](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-21-320.jpg)
![22
ZFS - Features
The features of ZFS include:
● […] efficient data compression, [...]](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-22-320.jpg)
![23
ZFS - Features
The features of ZFS include:
● […] snapshots and copy-on-write
clones, [...]](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-23-320.jpg)
![29
ZFS – Live Example
[root@bertha d125.powernetz.de]# pwd
/bacula/storage/d125.powernetz.de
[root@bertha d125.powernetz.de]# df h .
Filesystem Size Used Avail Use% Mounted on
bacula/d125.powernetz.de 4.0T 992G 3.1T 25%
/bacula/storage/d125.powernetz.de](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-29-320.jpg)
![30
ZFS – Live Example
[root@bertha d125.powernetz.de]# ls lhat
total 992G
drwxrx 268 bacula bacula 268 Jul 27 15:05 ..
rwr 1 bacula bacula 12G Jul 27 01:43 d125daily1420
rwr 1 bacula bacula 13G Jul 26 01:20 d125daily1274
rwr 1 bacula bacula 11G Jul 25 01:56 d125daily1119
rwr 1 bacula bacula 44G Jul 24 02:17 d125weekly1932
rwr 1 bacula bacula 14G Jul 23 01:26 d125daily0980
[...]
rwr 1 bacula bacula 31G Jul 17 01:38 d125weekly0828
rwr 1 bacula bacula 13G Jul 16 01:09 d125daily0309
[...]
rwr 1 bacula bacula 362G Jul 11 05:10 d125monthly5778
rwr 1 bacula bacula 64G Jul 9 01:56 d125weekly4000
rwr 1 bacula bacula 56G Jul 2 01:33 d125weekly3312
rwr 1 bacula bacula 45G Jun 24 01:13 d125weekly2846
rwr 1 bacula tape 359G Jun 11 04:38 d125monthly3633](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-30-320.jpg)
![31
ZFS – Live Example
[root@bertha ~]# zfs get compressratio
NAME PROPERTY VALUE
[...]
bacula/d136.powernetz.de compressratio 1.79x
bacula/d137.powernetz.de compressratio 1.69x
bacula/d138.powernetz.de compressratio 2.10x
bacula/d139.powernetz.de compressratio 1.57x
[...]](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-31-320.jpg)
![32
ZFS – Live Example
[root@bertha d125.powernetz.de]# du h d125monthly3633 apparentsize
359G d125monthly3633
[root@bertha d125.powernetz.de]# du h d125monthly3633
264G d125monthly
Net Save of 95GB, that’s ~30%!](https://image.slidesharecdn.com/backupwithbareoszfsandpuppet-christianrei-161004082702/85/Backup-with-Bareos-and-ZFS-by-Christian-Reiss-32-320.jpg)
Backup with Bareos and ZFS - by Christian Reiß
- 1. Bareos, ZFS and Puppet Christian Reiß Symgenius
- 2. 2 About Me ● Christian Reiß ● 37 years old ● System Administrator at Symgenius
- 3. 3 About Me ● Linux since 1995'ish (Kernel 1.3) ● B* evangelist ● Love Puppet
- 4. 4 Agenda ● Motivation and a little(!) history, ● Bacula (Alternatives), ● ZFS Introduction, ● The Puppet Module.
- 5. 5 A Promise ● I’ll try to do an ‘easy listening’, ● No code snippets.
- 6. 6 About Symgenius ● Symgenius ● Brand ‘Power-Netz.de’ ● Mid sized hosting company, ● Usual web-stack, ● Project hosting and housing
- 7. 7 Motivation – The situation ● We do webhosting. ● Several hundred Servers, ● Several hundred Customers each, ● Thousands of files per webhost, ● Thousands of files per Maildir, ● And of course: mysql.
- 8. 8 Motivation - History ● We used to backup with tar, ● Full backups of root into a tar located on a nfs share, ● Incrementals with a “find -ctime 1”. ● Everything piped through compression, ie. pigz or pbzip2.
- 9. 9 Motivation - History ● It did it’s job. ● Compression on client-side, ● Monitoring was a nightmare, ● But Restoring was hell.
- 10. 10 Motivation - Alternatives We tried several alternatives: ● BackupPC ● Rsnapshot ● Rsnapshot + ZFS Snaps ● Bacula
- 11. 11 Motivation - Alternatives ● BackupPC and rsnapshot failed. ● Backup times were 13+ hours due to amount of files, ● Job deletion even took longer. ● Backup + Job deletion exceeded 24 hours → Daily jobs piled up.
- 12. 12 Enter Bacula Bacula solved several issues: ● Backups are blazing fast even with high file counts, ● Backups are written inside a single archive volume, ● Restoring is bliss.
- 13. 13 Enter Bacula … But also: ● No heavy CPU load on client side (think compression) ● You could do Backups during work hours without Clients noticing (Well, except for MySQL Dumps…).
- 14. 14 Enter Bacula Client issue is solved, but what about the server side?
- 15. 15 Enter Bacula The Server needs: ● Protection against Hardware faults: - RAID(-alike) - Bit-Rot ● Other things: - Transparent compression - Quotas per Server
- 16. 16 Enter Bacula Also: ● Should be hassle- and maintenance free (mostly). ● Needs to perform fast on ‘normal’ Hardware.
- 17. 17 ZFS - Overview All is solved with ZFS
- 18. 18 ZFS - Overview All is solved with ZFS and Bareos.
- 19. 19 ZFS - Features The features of ZFS include: ● […] protection against data corruption, ● continuous integrity checking and automatic repair [...]
- 20. 20 ZFS - Features The features of ZFS include: ● […] RAID-Z [...]
- 21. 21 ZFS - Features The features of ZFS include: ● […] support for high storage capacities, [...]
- 22. 22 ZFS - Features The features of ZFS include: ● […] efficient data compression, [...]
- 23. 23 ZFS - Features The features of ZFS include: ● […] snapshots and copy-on-write clones, [...]
- 24. 24 ZFS - Availability Apart from Sun OS ZFS is available ● In Ubuntu 16.04 and up, ● In (future) Debian releases, ● And for all other Distributions via zfsonlinux.org. ● FreeNAS using ZFS as main storage
- 25. 25 ZFS - Install ● You install the package for your OS, ● Reboot, ● You do a “zpool create”, ● You might want to “zfs set” some stuff, ● Done. This does not count as ‘code snippet’.
- 26. 26 ZFS – Live Example Our SD (also Dir) consists out of: ● Intel Xeon CPU L5420 @ 2.50GHz ● 8 total cores ● 32gb Memory ● SSD for System and MySQL Data, ● 22x 4TB ‘NAS Grade’ WD Reds. ● (currently) 1Gb NIC
- 27. 27 ZFS – Live Example We backup 35 servers simultaneously. Both the CPU and NIC are maxed. That’s ~900Mbit of data transparently compressed, RAID’ed and checksummed right there. With 8 cores.
- 28. 28 ZFS – System Load
- 29. 29 ZFS – Live Example [root@bertha d125.powernetz.de]# pwd /bacula/storage/d125.powernetz.de [root@bertha d125.powernetz.de]# df h . Filesystem Size Used Avail Use% Mounted on bacula/d125.powernetz.de 4.0T 992G 3.1T 25% /bacula/storage/d125.powernetz.de
- 30. 30 ZFS – Live Example [root@bertha d125.powernetz.de]# ls lhat total 992G drwxrx 268 bacula bacula 268 Jul 27 15:05 .. rwr 1 bacula bacula 12G Jul 27 01:43 d125daily1420 rwr 1 bacula bacula 13G Jul 26 01:20 d125daily1274 rwr 1 bacula bacula 11G Jul 25 01:56 d125daily1119 rwr 1 bacula bacula 44G Jul 24 02:17 d125weekly1932 rwr 1 bacula bacula 14G Jul 23 01:26 d125daily0980 [...] rwr 1 bacula bacula 31G Jul 17 01:38 d125weekly0828 rwr 1 bacula bacula 13G Jul 16 01:09 d125daily0309 [...] rwr 1 bacula bacula 362G Jul 11 05:10 d125monthly5778 rwr 1 bacula bacula 64G Jul 9 01:56 d125weekly4000 rwr 1 bacula bacula 56G Jul 2 01:33 d125weekly3312 rwr 1 bacula bacula 45G Jun 24 01:13 d125weekly2846 rwr 1 bacula tape 359G Jun 11 04:38 d125monthly3633
- 31. 31 ZFS – Live Example [root@bertha ~]# zfs get compressratio NAME PROPERTY VALUE [...] bacula/d136.powernetz.de compressratio 1.79x bacula/d137.powernetz.de compressratio 1.69x bacula/d138.powernetz.de compressratio 2.10x bacula/d139.powernetz.de compressratio 1.57x [...]
- 32. 32 ZFS – Live Example [root@bertha d125.powernetz.de]# du h d125monthly3633 apparentsize 359G d125monthly3633 [root@bertha d125.powernetz.de]# du h d125monthly3633 264G d125monthly Net Save of 95GB, that’s ~30%!
- 33. 33 ZFS – Live Example On a sidenote: Deduplication does not work.
- 34. 34 ZFS - Advantages ● No more RAID controllers, ● Defense against Bit-Rot, ● Defense against Drive Faults*, ● Compression is done on Server, ● Quotas are possible, ● No hacks or workarounds. ● Everything is done transparently!
- 36. 36 Puppet Module The module does the usual: ● It configures the director, ● The storage daemon ● And of course the FD’s. → Backups work :)
- 37. 37 Puppet Each Server has: ● An independent Pool for Full Backups, ● An independent Pool for Differentials, ● An independent Pool for Incrementals. In its own ZFS-Dataset.
- 38. 38 Puppet Each Server: ● Has limited bconsole shell access (using ACL's) ● Has its own device in bacula-sd ● Is fully monitored by Icinga/Nagios
- 39. 39 Puppet ● We wanted a fully automated System ● Non-destructive Installation ● Managing new Servers, adding ZFS ● Soft removal of retired Servers
- 40. 40 Puppet Module In addition to that: ● One-File-per-Job (Volume) ● Monitor last Backup, ● Monitor MySQL Dumps, ● We check for a valid full-backup, ● We check for /bacula-restores dir.
- 41. 41 Puppet Module … Also: ● Random Passwords generated ● bareos-fd.conf copy saved on dir ● Option to override IPs
- 42. 42 Puppet Module Set & override by hiera: ● Accurate flag (default yes), ● Exclude of files, ● Include of files, ● Onefs flag (default yes).
- 43. 43 Puppet
- 44. 44 Puppet Module … and of course it will install the webui!
- 45. 45 The End. Thank you for your attention! o/
- 46. 46 The End. As promised: Bareos-Puppet Module available: https://github.com/christianreiss/al_bareos Contact me via: [email protected] Questions?