Difference between revisions of "Bacula and Bareos"
(→Manually Managing Volumes) |
|||
(100 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | =CentOS 6.5= | + | =Lawsuit= |
+ | Watch out, bacula users hate bareos...possibly with good reason (not the users just the project): | ||
+ | *http://blog.bacula.org/category/bareos/ | ||
+ | =CentOS 6.5 Install= | ||
yum install mysql-server mysql-devel | yum install mysql-server mysql-devel | ||
service mysqld start | service mysqld start | ||
chkconfig mysqld on | chkconfig mysqld on | ||
− | mysqladmin -u root password | + | mysqladmin -u root password Y0uR3l173P455w0rd |
yum install bareos-database-mysql | yum install bareos-database-mysql | ||
yum install bareos | yum install bareos | ||
+ | |||
+ | |||
+ | /usr/lib/bareos/scripts/create_bareos_database -u root -p | ||
+ | /usr/lib/bareos/scripts/make_bareos_tables -u root -p | ||
+ | /usr/lib/bareos/scripts/grant_bareos_privileges -u root -p | ||
+ | |||
+ | |||
+ | service bareos-dir start | ||
+ | service bareos-sd start | ||
+ | service bareos-fd start | ||
+ | |||
+ | ==Links== | ||
+ | *http://www.unixmen.com/install-and-configure-bacula-server-in-centos-6-4-rhel-6-4/ | ||
+ | *http://www.bareos.org/en/documentation.html | ||
+ | *http://doc.bareos.org/master/html/bareos-manual-main-reference.html | ||
+ | |||
+ | =Archlinux GUI Admin= | ||
+ | yaourt bareos-bat | ||
+ | |||
+ | =Configure On CentOS 6.5= | ||
+ | While I was trying to figure this out I was running into issues and one of the steps I took was to disable selinux because some of the posts recommended checking selinux perms. Instead of doing that I disabled. You can try without disabling. I made it most way through my guide but at the moment I do not need selinux (https://www.centos.org/docs/5/html/5.1/Deployment_Guide/sec-sel-enable-disable.html) so: | ||
+ | sestatus | ||
+ | nano -w /etc/sysconfig/selinux | ||
+ | Set | ||
+ | SELINUX=disabled | ||
+ | |||
+ | One of the other bigger problems I was having was big problems with vchanger permissions. While I know I had them all set correctly I still ended up doing more then I think I needed to do. I changed the owner of /usr/bin/vchanger | ||
+ | # ls -al /usr/bin/vchanger | ||
+ | -rwxr-xr-x. 1 bareos bareos 48512 Sep 28 2012 /usr/bin/vchanger | ||
+ | that did not fix the issue I was having though. I was still not able to use the vchanger command with -u bareos and -g bareos. I have put the real fix in the directions below but I wanted to leave this info here in case the next setup still does not work with the directions below. The fix was to make sure that the actual device link was owned by bareos:bareos. That is (obviously using the correct backup drive): | ||
+ | chown bareos:bareos /dev/sda1 | ||
+ | and just for cause | ||
+ | chown bareos:bareos /dev/sda | ||
+ | One of the last problems I was having was with the config of bacula and how it interfaces with vchanger. You need to specify the correct paths and I was having trouble understanding really how the paths work. In bareos-sd.conf in the Director part I had | ||
+ | ArchiveDevice = /var/lib/bareos/c0/drive0 | ||
+ | that is not what you need, in fact, I do not know what that file really is, I think it is a reference to what the label of the hard drive is or something. Actually I do not see a link and it might be a garbage file from my bad configuration that bareos tried to output. The correct line was: | ||
+ | ArchiveDevice = /var/lib/bareos/c0/0/drive0 | ||
+ | and everything seems to be working great. | ||
+ | |||
+ | One more note as if I do not get to update below you would want to know this. Since this was all new to me I was using 69 magazines just to mess around. I did not understand how magazines where associated with vchanger. They are just files on the drive. You could have 10000 or just one. In the end though, to fill the device you want to create x amount of files and in your bareos-dir.conf Pool directive have an appropriate | ||
+ | Maximum Volume Bytes = Xgb | ||
+ | or mb or whatever. | ||
+ | |||
+ | If I was sending files to a remote server I may choose smaller files in case of disconnection as I wonder if bareos can resume files. | ||
+ | |||
+ | One of the other things that I have noticed that even when I specified Maximum Volume Bytes = 1gb I was only getting 954MB files but that is because bacula writes in blocks, so it can only write a multiple of the block size defaults to 64512 (quoted from optiz0r_@freenode#baculairc) | ||
+ | |||
+ | |||
+ | ==Bareos and vchanger== | ||
+ | |||
+ | Since I use disks I start with installing vchanger | ||
+ | *http://wiki.bacula.org/doku.php?id=removable_disk | ||
+ | '''You should also download the tar.gz from the vchanger sourceforge and look in the docs for vchangerHowto.html as it has the best info. I followed it while making this guide.''' | ||
+ | |||
+ | The link above has a script but it looks like the one on sourceforge (which is referenced in the link) is newer and compiles. Even then, I decided to pull the rpm from pbone | ||
+ | wget ftp5.gwdg.de/pub/opensuse/repositories/home:/pstorz/CentOS_CentOS-6/x86_64/vchanger-0.8.6-10.3.x86_64.rpm | ||
+ | yum install vchanger-0.8.6-10.3.x86_64.rpm | ||
+ | *I am going to start with one drive so first I am going to format it (create a partition first (cfdisk)): | ||
+ | *Quote: | ||
+ | Assuming the drive was assigned node /dev/sdc, the partition can now be formatted and labeled using: | ||
+ | |||
+ | mke2fs -t ext4 -O large_file /dev/sdc1 | ||
+ | We also have the change the permission of the actual drive so vchanger has the right access to it. | ||
+ | *make sure to change the actual permissions of the drive link https://wiki.zmanda.com/index.php/Changer_problems | ||
+ | chown bareos:bareos /dev/sdc1 | ||
+ | chown bareos:bareos /dev/sdc | ||
+ | |||
+ | |||
+ | This will create a new ext4 file system on the partition. Now unplug the drive, wait a few seconds, then plug it back in. After re-attaching the drive, the name of the symlink in /dev/disk/by-uuid pointing to the filesystem partition's device node, (/dev/sdc1 in the example above), will reveal the UUID that can be used in a magazine directive in a vchanger configuration file to assign the removable drive to an autochanger. On many systems, the UUID of the filesystem can also be determined with the blkid command. | ||
+ | |||
+ | ls -al /dev/disk/by-uuid/ | ||
+ | |||
+ | The Bacula Storage Daemon does not usually run as root. Since vchanger will be invoked by the Storage Daemon, and so will run as the same user it does, permissions for the magazine's mounted filesystem must be set to allow write access to the user that the Storage Daemon runs as. This can be done by mounting the new partition somewhere, then using chown and chmod to set the appropriate permissions to allow write access for the user that the Storage Daemon runs as. | ||
+ | |||
+ | mkdir /media/temp | ||
+ | mount /dev/xvdc1 /media/temp | ||
+ | chown bareos:bareos -R /media/temp | ||
+ | |||
+ | Need to get autofs up and running so it mounts the disks and stuff: | ||
+ | |||
+ | yum install autofs | ||
+ | |||
+ | nano /etc/auto.vchanger | ||
+ | *add | ||
+ | # /etc/auto.vchanger | ||
+ | * -fstype=auto,rw,sync :/dev/disk/by-uuid/& | ||
+ | # eof | ||
+ | *Then: | ||
+ | nano /etc/auto.master | ||
+ | *add | ||
+ | /mnt/vchanger /etc/auto.vchanger --timeout=30 | ||
+ | *then | ||
+ | service autofs restart | ||
+ | ls -al /mnt/vchanger/UUID so | ||
+ | ls -al /mnt/vchanger/3c22af77-4881-4a7f-bcfc-517744c964db | ||
+ | Note: After restarting the autofs daemon, whenever a file or directory with a full path beginning with '/mnt/vchanger' is accessed, the autofs filesystem will search the map defined in the /etc/auto.vchanger file for a key matching the path being accessed. In this case, the only key in /etc/auto.vchanger is the wildcard '*', meaning any path name beginning with '/mnt/vchanger' will match. Autofs then automatically mounts the device mapped to the wildcard key at the path being accessed. The /etc/auto.vchanger map file specifies the device to be mounted as /dev/disk/by-uuid/&. The '&' is substituted for the key value. For example, when a program attempts to access /mnt/vchanger/9667f83c-6150-44c7-b0d4-57564f174b35 (or any files or directories below it), the autofs daemon will look at the auto.vchanger map for the key 9667f83c-6150-44c7-b0d4-57564f174b35 and discover that it should mount /dev/disk/by-uuid/9667f83c-6150-44c7-b0d4-57564f174b35 at /mnt/vchanger/9667f83c-6150-44c7-b0d4-57564f174b35 with mount options '-fstype=auto,rw,sync'. After a period of 30 seconds of no activity, autofs will automatically unmount the device. | ||
+ | |||
+ | Always make sure autofs is working properly before continuing with setting up vchanger. If using the above autofs config files, when you plug in a USB drive with filesystem UUID of, say, 9667f83c-6150-44c7-b0d4-57564f174b35, you should be able to list its contents with the commandS | ||
+ | |||
+ | *This should create some working directories | ||
+ | mkdir /var/lib/bareos/vchanger | ||
+ | mkdir /var/lib/bareos/vchanger/c0 | ||
+ | chown bareos:bareos -R /var/lib/bareos/vchanger | ||
+ | *Next we have to create our config: | ||
+ | nano /etc/bareos/vchanger.conf | ||
+ | |||
+ | *Put in (hint this is where you would add all the uuid lines for all your drives): | ||
+ | <pre> | ||
+ | changer_name = "c0" | ||
+ | work_dir = /var/lib/bareos/c0 | ||
+ | logfile = /var/lib/bareos/c0.log | ||
+ | log_level = LOG_ERR | ||
+ | virtual_drives = 1 | ||
+ | slots_per_magazine = 69 | ||
+ | magazine_bays = 1 | ||
+ | automount_dir = /mnt/vchanger | ||
+ | |||
+ | magazine = "UUID:3c22af77-4881-4a7f-bcfc-517744c964db" | ||
+ | |||
+ | </pre> | ||
+ | |||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf initmag 1 | ||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf list | ||
+ | |||
+ | Should work, if not check all permissions: working dirs, drives, mountpoints | ||
+ | *magazine bay 1 has no magazine mounted: check permissions and ownershipt of /dev/drive (chown that thing!) | ||
+ | |||
+ | *More: | ||
+ | |||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf listmags | ||
+ | *Output: | ||
+ | 1:c0:1:/mnt/vchanger/3c22af77-4881-4a7f-bcfc-517744c964db | ||
+ | |||
+ | *Test load and unload | ||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf load 1 /var/lib/bareos/c0/drive0 0 | ||
+ | ls -al /var/lib/bareos/c0/0/ | ||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf unload 1 /var/lib/bareos/c0/drive0 0 | ||
+ | |||
+ | *See what mag is installed in the bay: | ||
+ | cat /var/lib/bareos/c0/bay1 | ||
+ | |||
+ | Write your config files (http://karellen.blogspot.com/2012/01/creating-new-bacula-job.html , with a little from: http://www.revpol.com/offsitebackups?page=9 ): | ||
+ | |||
+ | bareos-sd.conf | ||
+ | Add | ||
+ | {{Note| Only later did I add the -u bareos and -g bareos to the Changer Command directive. You really should not need it}} | ||
+ | **Info about the above note: http://osdir.com/ml/bacula/2013-05/msg00261.html | ||
+ | About the Removable Media Directive Below: http://sourceforge.net/p/bacula/mailman/message/30914547/ | ||
+ | <pre> | ||
+ | Autochanger { | ||
+ | Name = esata-changer | ||
+ | Device = esata-changer-drive-0 | ||
+ | Changer Command = "/usr/bin/vchanger -u bareos -g bareos %c %o %S %a %d" | ||
+ | Changer Device = "/etc/bareos/vchanger.conf" | ||
+ | } | ||
+ | Device { | ||
+ | Name = esata-changer-drive-0 | ||
+ | DriveIndex = 0 | ||
+ | Autochanger = yes; | ||
+ | DeviceType = File | ||
+ | MediaType = File | ||
+ | ArchiveDevice = /var/lib/bareos/c0/0/drive0 | ||
+ | RemovableMedia = no; | ||
+ | RandomAccess = yes; | ||
+ | } | ||
+ | </pre> | ||
+ | |||
+ | bareos-dir.conf | ||
+ | Add | ||
+ | <pre> | ||
+ | Storage { | ||
+ | Name = c0 | ||
+ | # Do not use "localhost" here | ||
+ | Address = SYSTEM-NAME # N.B. Use a fully qualified name here | ||
+ | Password = "YOUR_PASSSWORD!" | ||
+ | Device = esata-changer | ||
+ | Media Type = File | ||
+ | Autochanger = yes; | ||
+ | } | ||
+ | |||
+ | Pool { | ||
+ | Name = incremental | ||
+ | Pool Type = backup | ||
+ | Storage = c0 | ||
+ | Maximum Volume Bytes = 1gb | ||
+ | Recycle Pool = Scratch | ||
+ | } | ||
+ | |||
+ | Job { | ||
+ | Name = "drobo_data" | ||
+ | Type = Backup | ||
+ | Level = Incremental | ||
+ | Client = NETT-AUTOBACK-fd | ||
+ | Schedule = "WeeklyIncremental" | ||
+ | Storage = c0 | ||
+ | Pool = incremental | ||
+ | Messages = Standard | ||
+ | Priority = 10 | ||
+ | FileSet="drobo_data" | ||
+ | } | ||
+ | |||
+ | Schedule { | ||
+ | Name = "WeeklyIncremental" | ||
+ | Run = Full 1st sun at 04:00 | ||
+ | Run = Incremental mon-sat at 04:00 | ||
+ | } | ||
+ | |||
+ | FileSet { | ||
+ | Name = "drobo_data" | ||
+ | Include { | ||
+ | Options { | ||
+ | signature = MD5 | ||
+ | } | ||
+ | File = "/media/drobo/DATA" | ||
+ | } | ||
+ | } | ||
+ | |||
+ | </pre> | ||
+ | |||
+ | service bareos-dir restart | ||
+ | service bareos-sd restart | ||
+ | This will make the services pull the config files. You can then bconsole in and try and do things this is an example of what I had to do to get my first test backup up and running once I got the config right: | ||
+ | |||
+ | bconsole | ||
+ | help is a good command btw | ||
+ | help | ||
+ | help status | ||
+ | in bconsole | ||
+ | label barcodes | ||
+ | make the right selections and choose right pool that your backup destination is setup to use | ||
+ | run | ||
+ | choose the backup to run in my case drobo_data and then I checked on it with: | ||
+ | status director | ||
+ | status storage=c0 | ||
+ | |||
+ | ==Links== | ||
+ | *http://rpm.pbone.net/index.php3 | ||
+ | **http://rpm.pbone.net/index.php3/stat/4/idpl/23671919/dir/centos_6/com/vchanger-0.8.6-10.3.i686.rpm.html | ||
+ | *http://wiki.bacula.org/doku.php?id=removable_disk#vchanger | ||
+ | *http://superuser.com/questions/573816/how-do-i-configure-yum-to-use-additional-repositories | ||
+ | *disk-changer vs vchanger: https://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg55905.html | ||
+ | *http://www.revpol.com/node/140?page=show | ||
+ | *'''THE HOWTO DOCUMENT IN THE TAR.GZ IS NEWEST OFFICIAL DOC!!!!!''' | ||
+ | |||
+ | =Using the software= | ||
+ | |||
+ | ==Starting Over== | ||
+ | Once I got the test setup working I needed to erase the drives, catalog, etc and get it resetup so the backup would fill the entire drive with the right config directives and number of slots. I also need to setup auto recycling: http://www.bacula.org/en/dev-manual/main/main/Automatic_Volume_Recycling.html and append parms, make sure my incremental is setup right. | ||
+ | |||
+ | *Erase Volumes From Catalog | ||
+ | My volume names are pretty standard. So I created a quick script, you may want to figure out how to output the volume names and go through them with the script. Just set the c values and move forward. | ||
+ | |||
+ | erase_volume_script | ||
+ | |||
+ | <pre> | ||
+ | #!/bin/bash | ||
+ | |||
+ | for (( c=2; c<=9; c++ )) | ||
+ | do | ||
+ | echo "Deleting Volume $c" | ||
+ | davar=`printf "%04.0f" $c` | ||
+ | echo "delete volume=c0_0005_$davar yes" | bconsole | ||
+ | done | ||
+ | </pre> | ||
+ | |||
+ | Changing number of magazines and magazine max size: | ||
+ | nano /etc/bareos/vchanger.conf | ||
+ | slots_per_magazine = 10 | ||
+ | |||
+ | nano /etc/bareos/bareos-dir.conf | ||
+ | Pool { } section | ||
+ | Maximum Volume Bytes = 100gb | ||
+ | |||
+ | I then go to the disk via uuid and delete everything but the lost and found dir. | ||
+ | rm files etc | ||
+ | |||
+ | Since I am starting over I need to change which mag number vchanger will start at to 1: | ||
+ | |||
+ | nano -w /var/lib/bareos/c0/nextmag | ||
+ | |||
+ | I also commented out the Schedule {} directive and the schedule directive in Job {} in bareos-dir.conf because I am still testing and will run the job manually for now. | ||
+ | |||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf initmag 1 | ||
+ | |||
+ | Should state: | ||
+ | created magazine 1 in bay 1 [/mnt/vchanger/3c22af77-4881-4a7f-bcfc-517744c964db] | ||
+ | |||
+ | Pop into bconsole and label barcodes with reference to the pool, run the job (if you job is too big for this mag, create the other mags first) | ||
+ | bconsole | ||
+ | label barcodes | ||
+ | run | ||
+ | |||
+ | ==Links and Notes== | ||
+ | *http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/bacula-25/vchanger-and-label-barcodes-issue-97621/ | ||
+ | *http://adsm.org/lists/html/Bacula-users/2009-07/msg00149.html | ||
+ | *Some bad/weird but sometimes helpful advice: http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/bacula-25/vchanger-and-label-barcodes-issue-97621/ | ||
+ | |||
+ | ==Initialize pool="File" and device="FileStorage" for default job BackupCatalog== | ||
+ | This should also work for the default defined job "BackupCatalog" which also uses pool="File" and device="FileStorage" | ||
+ | *http://www.bacula.org/manuals/en/concepts/concepts/Brief_Tutorial.html | ||
+ | bconsole | ||
+ | *bconsole commands: | ||
+ | label | ||
+ | *Select File resource | ||
+ | fileVol001 (Anything you want really) | ||
+ | If you are like me, and had the jobs in the queue, the job should start running. else issue a | ||
+ | run | ||
+ | |||
+ | ==Restoring Data== | ||
+ | *http://www.bacula.org/en/dev-manual/main/main/Restore_Command.html | ||
+ | **Quote: | ||
+ | ***Item 5 will automatically select the most recent Full backup and all subsequent incremental and differential backups for a specified Client. These are the Jobs and Files which, if reloaded, will restore your system to the most current saved state. It automatically enters the JobIds found into the directory tree in an optimal way such that only the most recent copy of any particular file found in the set of Jobs will be restored. This is probably the most convenient of all the above options to use if you wish to restore a selected Client to its most recent state. | ||
+ | ***Item 6 allows you to specify a date and time, after which Bacula will automatically select the most recent Full backup and all subsequent incremental and differential backups that started before the specified date and time. | ||
+ | |||
+ | *http://www.technowut.com/2012/07/23/how-to-do-a-restore-using-bacula/ | ||
+ | |||
+ | bconsole | ||
+ | bconsole commands: | ||
+ | |||
+ | restore | ||
+ | *I found it really nice to use 2 to find a job with a file I was looking for, I did select 1 which lists the last 20 jobs run but I did not get the name of the job with my query. So it was time prohibitive as I would have to refernce the job id from a different file. | ||
+ | *I then selected a JobId with #3. Once you select a job you should fall into a prompt to browse and restore files like it was a filesystem. | ||
+ | |||
+ | help | ||
+ | *Works great..for figuring out how to select files and such. | ||
+ | add file_or_dir_recurse | ||
+ | lsmark is neat | ||
+ | To finish and get prompted to run | ||
+ | done | ||
+ | |||
+ | so yes/mod/no | ||
+ | |||
+ | Just for my test, I modded the job, and changed the Where, it looks like you may have to also depending on what you are doing. It wanted to restore to: "Where: /tmp/bareos-restores". I created a temp dir that I could control, set the right parms, and restored to it for my test. Worked great. | ||
+ | |||
+ | I am sure if you defined a proper restore job, things would be better. | ||
+ | |||
+ | Worked Great... | ||
+ | |||
+ | ==Restore Files without bacula== | ||
+ | *http://www.bacula.org/en/dev-manual/main/main/Restore_Command.html | ||
+ | bextract | ||
+ | If you find yourself using bextract, you probably have done something wrong. | ||
+ | |||
+ | Looks like to use bextract, at least the easyway, you create a file with a config against the job. Somewhere it states that it is easier if you have the list of files so you can and boot/run with it or whatnot. You do not have to do it that way but it is the easiest way, they say. | ||
+ | |||
+ | Bacula creates/can use // bextract can use bsr files or bootstrap files to restore files, and db entries. | ||
+ | |||
+ | *http://www.bacula.org/en/dev-manual/main/main/Bootstrap_File.html | ||
+ | |||
+ | Quote from above: | ||
+ | |||
+ | If you want to extract or copy a single Job, you can do it by selecting by JobId (code not tested) or better yet, if you know the VolSessionTime and the VolSessionId (printed on Job report and in Catalog), specifying this is by far the best. Using the VolSessionTime and VolSessionId is the way Bacula does restores. A bsr file might look like the following: | ||
+ | |||
+ | Volume="Vol001" | ||
+ | VolSessionId=10 | ||
+ | VolSessionTime=1080847820 | ||
+ | |||
+ | If you know how many files are backed up (on the job report), you can enormously speed up the selection by adding (let's assume there are 157 files): | ||
+ | |||
+ | FileIndex=1-157 | ||
+ | Count=157 | ||
+ | |||
+ | Finally, if you know the File number where the Job starts, you can also cause bcopy to forward space to the right file without reading every record: | ||
+ | |||
+ | VolFile=20 | ||
+ | |||
+ | There is nothing magic or complicated about a BSR file. Parsing it and properly applying it within Bacula *is* magic, but you don't need to worry about that. | ||
+ | |||
+ | If you want to see a *real* bsr file, simply fire up the restore command in the console program, select something, then answer no when it prompts to run the job. Then look at the file restore.bsr in your working directory. | ||
+ | |||
+ | ==Recycling Volumes== | ||
+ | |||
+ | If you have scheduled your job, typically nightly, there will come a time when the tape fills up and Bacula cannot continue. In this case, Bacula will send you a message similar to the following: | ||
+ | |||
+ | rufus-sd: block.c:337 === Write error errno=28: ERR=No space left | ||
+ | on device | ||
+ | |||
+ | This indicates that Bacula got a write error because the tape is full. Bacula will then search the Pool specified for your Job looking for an appendable volume. In the best of all cases, you will have properly set your Retention Periods and you will have all your tapes marked to be Recycled, and Bacula will automatically recycle the tapes in your pool requesting and overwriting old Volumes. For more information on recycling, please see the Recycling chapter of this manual. If you find that your Volumes were not properly recycled (usually because of a configuration error), please see the Manually Recycling Volumes section of the Recycling chapter. | ||
+ | |||
+ | |||
+ | *http://www.bacula.org/manuals/en/concepts/concepts/Automatic_Volume_Recycling.html#RecyclingChapter | ||
+ | *http://www.bacula.org/manuals/en/concepts/concepts/Automatic_Volume_Recycling.html#manualrecycling | ||
+ | |||
+ | ===Scratch=== | ||
+ | *http://bacula.10910.n7.nabble.com/Don-t-understant-recycling-and-Scratch-pool-td45115.html | ||
+ | |||
+ | You can use "RecyclePool = Scratch" in the pool resource: | ||
+ | |||
+ | RecyclePool = <pool-resource-name> On versions 2.1.4 or greater, | ||
+ | this directive defines to which pool the Volume will be placed (moved) | ||
+ | when it is recycled. | ||
+ | |||
+ | The scratch pool is only used for newly labelled volumes. If there are | ||
+ | no spare tapes in the pool that the current backup is taking place on, | ||
+ | Bacula will look to see if it can take one from the scratch pool. It | ||
+ | won't look for volumes which may be recycled but are part of another | ||
+ | pool, nor will it move volumes back to scratch when they become | ||
+ | eligible for recycling. Though that's an interesting idea..... | ||
+ | |||
+ | If the variable RecyclePoolId is not set for the media execute "update volume" for all the volumes of the pool, the RecyclePoolId field is well set, and recycling works fine. | ||
+ | |||
+ | |||
+ | In fact, my initial configuration was not correct. | ||
+ | The RecyclePool=Scratch directive must be specified in the Scratch pool section. | ||
+ | If I purge a tape, it is automatically moved to the Scratch pool. | ||
+ | But the tape is never moved to the Scratch pool after a retention period. | ||
+ | I have to purge it manually for that. | ||
+ | |||
+ | ==Volume Management, Job Retention, Backup Job Tree== | ||
+ | *http://www.bacula.org/manuals/en/concepts/concepts/Basic_Volume_Management.html | ||
+ | Look at: Restricting the Number of Volumes and Recycling | ||
+ | |||
+ | **Volume Retention | ||
+ | |||
+ | **Volume Use Duration | ||
+ | To limit the use time (i.e. write the Volume for a maximum of five days), use: | ||
+ | Volume Use Duration = ttt. | ||
+ | |||
+ | '''Since what I am using this for at is simple, and I have a specific use case that I need to meet, I am manually purging volumes. I just need a full backup, with manual incrementals to transport.''' | ||
+ | |||
+ | purge volume=volname | ||
+ | |||
+ | Bacula likes to use append volumes before purge volumes btw | ||
+ | |||
+ | Backups job trees go like this (http://bacula.10910.n7.nabble.com/differential-vs-incremental-td3349.html): | ||
+ | *full, incremental, incremental, incremental, forever... | ||
+ | *full, incremental, (wipe the previous jobs), differential, incremental incremental, forever....differential | ||
+ | |||
+ | *Changing the Retention settings (http://bacula.10910.n7.nabble.com/How-to-re-use-existing-volumes-in-pool-td56384.html) | ||
+ | maybe change VolumeRetention to 5 days and do an "update volumes from resource" (only changing *.conf won't do the trick). | ||
+ | update volumes from resource | ||
+ | Must write the new settings... | ||
+ | |||
+ | ==vchanger initilize a mag quick== | ||
+ | chown bareos:bareos /dev/drive | ||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf initmag 1 | ||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf list | ||
+ | bconsole | ||
+ | label barcodes | ||
+ | |||
+ | ==status== | ||
+ | In bconsole | ||
+ | status director | ||
+ | status storage=c0 | ||
+ | status client (shows filename and job details | ||
+ | =Manually Managing Volumes= | ||
+ | I use a system to do some once in a while backups. That is, one time ever 6-8 months. When I go to backup again, I do not need Bacula to manage anything. Instead, I just want to delete the old data, and make new data. | ||
+ | |||
+ | bconsole | ||
+ | list jobs | ||
+ | Delete the jobs associated with the data: | ||
+ | delete | ||
+ | <jobid> | ||
+ | list jobs | ||
+ | etc, etc, until all the jobs are deleted. | ||
+ | |||
+ | prune | ||
+ | Select the volume you want to prune | ||
+ | |||
+ | I then have to move the volume back to the right pool if I want. | ||
+ | prune volume= | ||
+ | |||
+ | =Removing Recycle Directive from Pool= | ||
+ | Instead of having the volumes moved to a recycle pool I just wanted them to stay so I can update them later. | ||
+ | *Change the config to remove the recycle directive. | ||
+ | You then have to have Bacula update the volumes because the info is stored in the definition of each volume. | ||
+ | update pool, from resources. update volume, all volumes from all pools | ||
+ | then run list volume=XXX to check it's been applied properly | ||
+ | |||
+ | |||
+ | =Moving Volumes Around In Pools= | ||
+ | update | ||
+ | Volume parameters | ||
+ | Pool | ||
+ | |||
+ | =Switching a Drive Out= | ||
+ | |||
+ | ==Unload== | ||
+ | To unload a magazine, first check the status of the director with the command 'status dir' from within bconsole to make sure no currently running jobs are using a volume on the autochanger, then simply unplug the USB drive. | ||
+ | |||
+ | ==Load== | ||
+ | chown bareos:bareos /dev/xvdb* | ||
+ | *Can test with: | ||
+ | vchanger -u bareos -g bareos /etc/bareos/vchanger.conf list | ||
+ | bconsole | ||
+ | update slots | ||
+ | mount | ||
+ | |||
+ | =Updating Pool Defs= | ||
+ | <pre> | ||
+ | 6:32 < optiz0r> Just the latter is needed, however you need to bear in mind that the Scratch Pool option is copied from the pool to the | ||
+ | volume itself at the time the volume is labelled, so after changing the pool definition you need to reload the pool | ||
+ | parameters from the config file (using update pool) then reapply the pool parameters to the volume (with update volume) | ||
+ | 16:34 < webdawg> So remove the Recycle Pool directive in the config, and then update pool, and update volume | ||
+ | 16:34 < webdawg> Just update the recycle flag when I update the volume? | ||
+ | 16:35 < webdawg> or Volume from Pool | ||
+ | 16:36 < optiz0r> update pool, from resources. update volume, all volumes from all pools | ||
+ | 16:37 < optiz0r> then run llist volume=XXX to check it's been applied properly | ||
+ | </pre> | ||
+ | |||
+ | =Kill all waiting Jobs= | ||
+ | |||
+ | <pre> | ||
+ | kill_waiting_jobs.bash | ||
+ | |||
+ | #!/bin/bash | ||
+ | jobIds=`echo 'status dir running' | bconsole | fgrep 'is waiting' | awk '{print $1}'` | ||
+ | for i in $jobIds | ||
+ | do | ||
+ | if [ -z `echo "$i" | grep '^[0-9]\+$'` ] | ||
+ | then | ||
+ | echo "Error: job ID $i is not a number!" | ||
+ | else | ||
+ | echo "Killing waiting Bacula job $i" | ||
+ | echo "cancel jobid=$i" | bconsole | ||
+ | fi | ||
+ | done | ||
+ | </pre> | ||
+ | |||
+ | =Delete all Old Jobs= | ||
+ | THIS SCRIPT IS UNFINISHED (fgrep 'B' heh) | ||
+ | |||
+ | <pre> | ||
+ | #!/bin/bash | ||
+ | jobIds=`echo 'list jobs' | bconsole | fgrep 'B' | awk '{print $2}'` | ||
+ | for i in $jobIds | ||
+ | do | ||
+ | if [ -z `echo "$i" | grep '^[0-9]\+$'` ] | ||
+ | then | ||
+ | echo "Error: job ID $i is not a number!" | ||
+ | else | ||
+ | echo "Deleting Backup Bacula job $i" | ||
+ | echo "delete jobid=$i" | bconsole | ||
+ | fi | ||
+ | done | ||
+ | </pre> | ||
+ | |||
+ | =Truncate= | ||
+ | |||
+ | |||
+ | =FileVol= | ||
+ | /var/lib/bareos/storage |
Latest revision as of 12:05, 13 October 2015
Contents
- 1 Lawsuit
- 2 CentOS 6.5 Install
- 3 Archlinux GUI Admin
- 4 Configure On CentOS 6.5
- 5 Using the software
- 6 Manually Managing Volumes
- 7 Removing Recycle Directive from Pool
- 8 Moving Volumes Around In Pools
- 9 Switching a Drive Out
- 10 Updating Pool Defs
- 11 Kill all waiting Jobs
- 12 Delete all Old Jobs
- 13 Truncate
- 14 FileVol
Lawsuit
Watch out, bacula users hate bareos...possibly with good reason (not the users just the project):
CentOS 6.5 Install
yum install mysql-server mysql-devel
service mysqld start chkconfig mysqld on mysqladmin -u root password Y0uR3l173P455w0rd
yum install bareos-database-mysql yum install bareos
/usr/lib/bareos/scripts/create_bareos_database -u root -p /usr/lib/bareos/scripts/make_bareos_tables -u root -p /usr/lib/bareos/scripts/grant_bareos_privileges -u root -p
service bareos-dir start service bareos-sd start service bareos-fd start
Links
- http://www.unixmen.com/install-and-configure-bacula-server-in-centos-6-4-rhel-6-4/
- http://www.bareos.org/en/documentation.html
- http://doc.bareos.org/master/html/bareos-manual-main-reference.html
Archlinux GUI Admin
yaourt bareos-bat
Configure On CentOS 6.5
While I was trying to figure this out I was running into issues and one of the steps I took was to disable selinux because some of the posts recommended checking selinux perms. Instead of doing that I disabled. You can try without disabling. I made it most way through my guide but at the moment I do not need selinux (https://www.centos.org/docs/5/html/5.1/Deployment_Guide/sec-sel-enable-disable.html) so:
sestatus nano -w /etc/sysconfig/selinux
Set
SELINUX=disabled
One of the other bigger problems I was having was big problems with vchanger permissions. While I know I had them all set correctly I still ended up doing more then I think I needed to do. I changed the owner of /usr/bin/vchanger
# ls -al /usr/bin/vchanger -rwxr-xr-x. 1 bareos bareos 48512 Sep 28 2012 /usr/bin/vchanger
that did not fix the issue I was having though. I was still not able to use the vchanger command with -u bareos and -g bareos. I have put the real fix in the directions below but I wanted to leave this info here in case the next setup still does not work with the directions below. The fix was to make sure that the actual device link was owned by bareos:bareos. That is (obviously using the correct backup drive):
chown bareos:bareos /dev/sda1
and just for cause
chown bareos:bareos /dev/sda
One of the last problems I was having was with the config of bacula and how it interfaces with vchanger. You need to specify the correct paths and I was having trouble understanding really how the paths work. In bareos-sd.conf in the Director part I had
ArchiveDevice = /var/lib/bareos/c0/drive0
that is not what you need, in fact, I do not know what that file really is, I think it is a reference to what the label of the hard drive is or something. Actually I do not see a link and it might be a garbage file from my bad configuration that bareos tried to output. The correct line was:
ArchiveDevice = /var/lib/bareos/c0/0/drive0
and everything seems to be working great.
One more note as if I do not get to update below you would want to know this. Since this was all new to me I was using 69 magazines just to mess around. I did not understand how magazines where associated with vchanger. They are just files on the drive. You could have 10000 or just one. In the end though, to fill the device you want to create x amount of files and in your bareos-dir.conf Pool directive have an appropriate
Maximum Volume Bytes = Xgb
or mb or whatever.
If I was sending files to a remote server I may choose smaller files in case of disconnection as I wonder if bareos can resume files.
One of the other things that I have noticed that even when I specified Maximum Volume Bytes = 1gb I was only getting 954MB files but that is because bacula writes in blocks, so it can only write a multiple of the block size defaults to 64512 (quoted from optiz0r_@freenode#baculairc)
Bareos and vchanger
Since I use disks I start with installing vchanger
You should also download the tar.gz from the vchanger sourceforge and look in the docs for vchangerHowto.html as it has the best info. I followed it while making this guide.
The link above has a script but it looks like the one on sourceforge (which is referenced in the link) is newer and compiles. Even then, I decided to pull the rpm from pbone
wget ftp5.gwdg.de/pub/opensuse/repositories/home:/pstorz/CentOS_CentOS-6/x86_64/vchanger-0.8.6-10.3.x86_64.rpm yum install vchanger-0.8.6-10.3.x86_64.rpm
- I am going to start with one drive so first I am going to format it (create a partition first (cfdisk)):
- Quote:
Assuming the drive was assigned node /dev/sdc, the partition can now be formatted and labeled using:
mke2fs -t ext4 -O large_file /dev/sdc1
We also have the change the permission of the actual drive so vchanger has the right access to it.
- make sure to change the actual permissions of the drive link https://wiki.zmanda.com/index.php/Changer_problems
chown bareos:bareos /dev/sdc1 chown bareos:bareos /dev/sdc
This will create a new ext4 file system on the partition. Now unplug the drive, wait a few seconds, then plug it back in. After re-attaching the drive, the name of the symlink in /dev/disk/by-uuid pointing to the filesystem partition's device node, (/dev/sdc1 in the example above), will reveal the UUID that can be used in a magazine directive in a vchanger configuration file to assign the removable drive to an autochanger. On many systems, the UUID of the filesystem can also be determined with the blkid command.
ls -al /dev/disk/by-uuid/
The Bacula Storage Daemon does not usually run as root. Since vchanger will be invoked by the Storage Daemon, and so will run as the same user it does, permissions for the magazine's mounted filesystem must be set to allow write access to the user that the Storage Daemon runs as. This can be done by mounting the new partition somewhere, then using chown and chmod to set the appropriate permissions to allow write access for the user that the Storage Daemon runs as.
mkdir /media/temp mount /dev/xvdc1 /media/temp chown bareos:bareos -R /media/temp
Need to get autofs up and running so it mounts the disks and stuff:
yum install autofs
nano /etc/auto.vchanger
- add
# /etc/auto.vchanger * -fstype=auto,rw,sync :/dev/disk/by-uuid/& # eof
- Then:
nano /etc/auto.master
- add
/mnt/vchanger /etc/auto.vchanger --timeout=30
- then
service autofs restart
ls -al /mnt/vchanger/UUID so
ls -al /mnt/vchanger/3c22af77-4881-4a7f-bcfc-517744c964db
Note: After restarting the autofs daemon, whenever a file or directory with a full path beginning with '/mnt/vchanger' is accessed, the autofs filesystem will search the map defined in the /etc/auto.vchanger file for a key matching the path being accessed. In this case, the only key in /etc/auto.vchanger is the wildcard '*', meaning any path name beginning with '/mnt/vchanger' will match. Autofs then automatically mounts the device mapped to the wildcard key at the path being accessed. The /etc/auto.vchanger map file specifies the device to be mounted as /dev/disk/by-uuid/&. The '&' is substituted for the key value. For example, when a program attempts to access /mnt/vchanger/9667f83c-6150-44c7-b0d4-57564f174b35 (or any files or directories below it), the autofs daemon will look at the auto.vchanger map for the key 9667f83c-6150-44c7-b0d4-57564f174b35 and discover that it should mount /dev/disk/by-uuid/9667f83c-6150-44c7-b0d4-57564f174b35 at /mnt/vchanger/9667f83c-6150-44c7-b0d4-57564f174b35 with mount options '-fstype=auto,rw,sync'. After a period of 30 seconds of no activity, autofs will automatically unmount the device.
Always make sure autofs is working properly before continuing with setting up vchanger. If using the above autofs config files, when you plug in a USB drive with filesystem UUID of, say, 9667f83c-6150-44c7-b0d4-57564f174b35, you should be able to list its contents with the commandS
- This should create some working directories
mkdir /var/lib/bareos/vchanger mkdir /var/lib/bareos/vchanger/c0 chown bareos:bareos -R /var/lib/bareos/vchanger
- Next we have to create our config:
nano /etc/bareos/vchanger.conf
- Put in (hint this is where you would add all the uuid lines for all your drives):
changer_name = "c0" work_dir = /var/lib/bareos/c0 logfile = /var/lib/bareos/c0.log log_level = LOG_ERR virtual_drives = 1 slots_per_magazine = 69 magazine_bays = 1 automount_dir = /mnt/vchanger magazine = "UUID:3c22af77-4881-4a7f-bcfc-517744c964db"
vchanger -u bareos -g bareos /etc/bareos/vchanger.conf initmag 1 vchanger -u bareos -g bareos /etc/bareos/vchanger.conf list
Should work, if not check all permissions: working dirs, drives, mountpoints
- magazine bay 1 has no magazine mounted: check permissions and ownershipt of /dev/drive (chown that thing!)
- More:
vchanger -u bareos -g bareos /etc/bareos/vchanger.conf listmags
- Output:
1:c0:1:/mnt/vchanger/3c22af77-4881-4a7f-bcfc-517744c964db
- Test load and unload
vchanger -u bareos -g bareos /etc/bareos/vchanger.conf load 1 /var/lib/bareos/c0/drive0 0 ls -al /var/lib/bareos/c0/0/ vchanger -u bareos -g bareos /etc/bareos/vchanger.conf unload 1 /var/lib/bareos/c0/drive0 0
- See what mag is installed in the bay:
cat /var/lib/bareos/c0/bay1
Write your config files (http://karellen.blogspot.com/2012/01/creating-new-bacula-job.html , with a little from: http://www.revpol.com/offsitebackups?page=9 ):
bareos-sd.conf
Add
- Info about the above note: http://osdir.com/ml/bacula/2013-05/msg00261.html
About the Removable Media Directive Below: http://sourceforge.net/p/bacula/mailman/message/30914547/
Autochanger { Name = esata-changer Device = esata-changer-drive-0 Changer Command = "/usr/bin/vchanger -u bareos -g bareos %c %o %S %a %d" Changer Device = "/etc/bareos/vchanger.conf" } Device { Name = esata-changer-drive-0 DriveIndex = 0 Autochanger = yes; DeviceType = File MediaType = File ArchiveDevice = /var/lib/bareos/c0/0/drive0 RemovableMedia = no; RandomAccess = yes; }
bareos-dir.conf
Add
Storage { Name = c0 # Do not use "localhost" here Address = SYSTEM-NAME # N.B. Use a fully qualified name here Password = "YOUR_PASSSWORD!" Device = esata-changer Media Type = File Autochanger = yes; } Pool { Name = incremental Pool Type = backup Storage = c0 Maximum Volume Bytes = 1gb Recycle Pool = Scratch } Job { Name = "drobo_data" Type = Backup Level = Incremental Client = NETT-AUTOBACK-fd Schedule = "WeeklyIncremental" Storage = c0 Pool = incremental Messages = Standard Priority = 10 FileSet="drobo_data" } Schedule { Name = "WeeklyIncremental" Run = Full 1st sun at 04:00 Run = Incremental mon-sat at 04:00 } FileSet { Name = "drobo_data" Include { Options { signature = MD5 } File = "/media/drobo/DATA" } }
service bareos-dir restart service bareos-sd restart
This will make the services pull the config files. You can then bconsole in and try and do things this is an example of what I had to do to get my first test backup up and running once I got the config right:
bconsole
help is a good command btw
help help status
in bconsole
label barcodes
make the right selections and choose right pool that your backup destination is setup to use
run
choose the backup to run in my case drobo_data and then I checked on it with:
status director status storage=c0
Links
- http://rpm.pbone.net/index.php3
- http://wiki.bacula.org/doku.php?id=removable_disk#vchanger
- http://superuser.com/questions/573816/how-do-i-configure-yum-to-use-additional-repositories
- disk-changer vs vchanger: https://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg55905.html
- http://www.revpol.com/node/140?page=show
- THE HOWTO DOCUMENT IN THE TAR.GZ IS NEWEST OFFICIAL DOC!!!!!
Using the software
Starting Over
Once I got the test setup working I needed to erase the drives, catalog, etc and get it resetup so the backup would fill the entire drive with the right config directives and number of slots. I also need to setup auto recycling: http://www.bacula.org/en/dev-manual/main/main/Automatic_Volume_Recycling.html and append parms, make sure my incremental is setup right.
- Erase Volumes From Catalog
My volume names are pretty standard. So I created a quick script, you may want to figure out how to output the volume names and go through them with the script. Just set the c values and move forward.
erase_volume_script
#!/bin/bash for (( c=2; c<=9; c++ )) do echo "Deleting Volume $c" davar=`printf "%04.0f" $c` echo "delete volume=c0_0005_$davar yes" | bconsole done
Changing number of magazines and magazine max size:
nano /etc/bareos/vchanger.conf slots_per_magazine = 10
nano /etc/bareos/bareos-dir.conf
Pool { } section
Maximum Volume Bytes = 100gb
I then go to the disk via uuid and delete everything but the lost and found dir.
rm files etc
Since I am starting over I need to change which mag number vchanger will start at to 1:
nano -w /var/lib/bareos/c0/nextmag
I also commented out the Schedule {} directive and the schedule directive in Job {} in bareos-dir.conf because I am still testing and will run the job manually for now.
vchanger -u bareos -g bareos /etc/bareos/vchanger.conf initmag 1
Should state:
created magazine 1 in bay 1 [/mnt/vchanger/3c22af77-4881-4a7f-bcfc-517744c964db]
Pop into bconsole and label barcodes with reference to the pool, run the job (if you job is too big for this mag, create the other mags first)
bconsole label barcodes run
Links and Notes
- http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/bacula-25/vchanger-and-label-barcodes-issue-97621/
- http://adsm.org/lists/html/Bacula-users/2009-07/msg00149.html
- Some bad/weird but sometimes helpful advice: http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/bacula-25/vchanger-and-label-barcodes-issue-97621/
Initialize pool="File" and device="FileStorage" for default job BackupCatalog
This should also work for the default defined job "BackupCatalog" which also uses pool="File" and device="FileStorage"
bconsole
- bconsole commands:
label
- Select File resource
fileVol001 (Anything you want really)
If you are like me, and had the jobs in the queue, the job should start running. else issue a
run
Restoring Data
- http://www.bacula.org/en/dev-manual/main/main/Restore_Command.html
- Quote:
- Item 5 will automatically select the most recent Full backup and all subsequent incremental and differential backups for a specified Client. These are the Jobs and Files which, if reloaded, will restore your system to the most current saved state. It automatically enters the JobIds found into the directory tree in an optimal way such that only the most recent copy of any particular file found in the set of Jobs will be restored. This is probably the most convenient of all the above options to use if you wish to restore a selected Client to its most recent state.
- Item 6 allows you to specify a date and time, after which Bacula will automatically select the most recent Full backup and all subsequent incremental and differential backups that started before the specified date and time.
- Quote:
bconsole
bconsole commands:
restore
- I found it really nice to use 2 to find a job with a file I was looking for, I did select 1 which lists the last 20 jobs run but I did not get the name of the job with my query. So it was time prohibitive as I would have to refernce the job id from a different file.
- I then selected a JobId with #3. Once you select a job you should fall into a prompt to browse and restore files like it was a filesystem.
help
- Works great..for figuring out how to select files and such.
add file_or_dir_recurse lsmark is neat
To finish and get prompted to run
done
so yes/mod/no
Just for my test, I modded the job, and changed the Where, it looks like you may have to also depending on what you are doing. It wanted to restore to: "Where: /tmp/bareos-restores". I created a temp dir that I could control, set the right parms, and restored to it for my test. Worked great.
I am sure if you defined a proper restore job, things would be better.
Worked Great...
Restore Files without bacula
bextract
If you find yourself using bextract, you probably have done something wrong.
Looks like to use bextract, at least the easyway, you create a file with a config against the job. Somewhere it states that it is easier if you have the list of files so you can and boot/run with it or whatnot. You do not have to do it that way but it is the easiest way, they say.
Bacula creates/can use // bextract can use bsr files or bootstrap files to restore files, and db entries.
Quote from above:
If you want to extract or copy a single Job, you can do it by selecting by JobId (code not tested) or better yet, if you know the VolSessionTime and the VolSessionId (printed on Job report and in Catalog), specifying this is by far the best. Using the VolSessionTime and VolSessionId is the way Bacula does restores. A bsr file might look like the following:
Volume="Vol001" VolSessionId=10 VolSessionTime=1080847820
If you know how many files are backed up (on the job report), you can enormously speed up the selection by adding (let's assume there are 157 files):
FileIndex=1-157 Count=157
Finally, if you know the File number where the Job starts, you can also cause bcopy to forward space to the right file without reading every record:
VolFile=20
There is nothing magic or complicated about a BSR file. Parsing it and properly applying it within Bacula *is* magic, but you don't need to worry about that.
If you want to see a *real* bsr file, simply fire up the restore command in the console program, select something, then answer no when it prompts to run the job. Then look at the file restore.bsr in your working directory.
Recycling Volumes
If you have scheduled your job, typically nightly, there will come a time when the tape fills up and Bacula cannot continue. In this case, Bacula will send you a message similar to the following:
rufus-sd: block.c:337 === Write error errno=28: ERR=No space left on device
This indicates that Bacula got a write error because the tape is full. Bacula will then search the Pool specified for your Job looking for an appendable volume. In the best of all cases, you will have properly set your Retention Periods and you will have all your tapes marked to be Recycled, and Bacula will automatically recycle the tapes in your pool requesting and overwriting old Volumes. For more information on recycling, please see the Recycling chapter of this manual. If you find that your Volumes were not properly recycled (usually because of a configuration error), please see the Manually Recycling Volumes section of the Recycling chapter.
- http://www.bacula.org/manuals/en/concepts/concepts/Automatic_Volume_Recycling.html#RecyclingChapter
- http://www.bacula.org/manuals/en/concepts/concepts/Automatic_Volume_Recycling.html#manualrecycling
Scratch
You can use "RecyclePool = Scratch" in the pool resource:
RecyclePool = <pool-resource-name> On versions 2.1.4 or greater, this directive defines to which pool the Volume will be placed (moved) when it is recycled.
The scratch pool is only used for newly labelled volumes. If there are no spare tapes in the pool that the current backup is taking place on, Bacula will look to see if it can take one from the scratch pool. It won't look for volumes which may be recycled but are part of another pool, nor will it move volumes back to scratch when they become eligible for recycling. Though that's an interesting idea.....
If the variable RecyclePoolId is not set for the media execute "update volume" for all the volumes of the pool, the RecyclePoolId field is well set, and recycling works fine.
In fact, my initial configuration was not correct.
The RecyclePool=Scratch directive must be specified in the Scratch pool section.
If I purge a tape, it is automatically moved to the Scratch pool.
But the tape is never moved to the Scratch pool after a retention period.
I have to purge it manually for that.
Volume Management, Job Retention, Backup Job Tree
Look at: Restricting the Number of Volumes and Recycling
- Volume Retention
- Volume Use Duration
To limit the use time (i.e. write the Volume for a maximum of five days), use: Volume Use Duration = ttt.
Since what I am using this for at is simple, and I have a specific use case that I need to meet, I am manually purging volumes. I just need a full backup, with manual incrementals to transport.
purge volume=volname
Bacula likes to use append volumes before purge volumes btw
Backups job trees go like this (http://bacula.10910.n7.nabble.com/differential-vs-incremental-td3349.html):
- full, incremental, incremental, incremental, forever...
- full, incremental, (wipe the previous jobs), differential, incremental incremental, forever....differential
- Changing the Retention settings (http://bacula.10910.n7.nabble.com/How-to-re-use-existing-volumes-in-pool-td56384.html)
maybe change VolumeRetention to 5 days and do an "update volumes from resource" (only changing *.conf won't do the trick).
update volumes from resource
Must write the new settings...
vchanger initilize a mag quick
chown bareos:bareos /dev/drive vchanger -u bareos -g bareos /etc/bareos/vchanger.conf initmag 1 vchanger -u bareos -g bareos /etc/bareos/vchanger.conf list bconsole label barcodes
status
In bconsole
status director status storage=c0 status client (shows filename and job details
Manually Managing Volumes
I use a system to do some once in a while backups. That is, one time ever 6-8 months. When I go to backup again, I do not need Bacula to manage anything. Instead, I just want to delete the old data, and make new data.
bconsole list jobs
Delete the jobs associated with the data:
delete <jobid> list jobs
etc, etc, until all the jobs are deleted.
prune
Select the volume you want to prune
I then have to move the volume back to the right pool if I want.
prune volume=
Removing Recycle Directive from Pool
Instead of having the volumes moved to a recycle pool I just wanted them to stay so I can update them later.
- Change the config to remove the recycle directive.
You then have to have Bacula update the volumes because the info is stored in the definition of each volume.
update pool, from resources. update volume, all volumes from all pools then run list volume=XXX to check it's been applied properly
Moving Volumes Around In Pools
update Volume parameters Pool
Switching a Drive Out
Unload
To unload a magazine, first check the status of the director with the command 'status dir' from within bconsole to make sure no currently running jobs are using a volume on the autochanger, then simply unplug the USB drive.
Load
chown bareos:bareos /dev/xvdb*
- Can test with:
vchanger -u bareos -g bareos /etc/bareos/vchanger.conf list bconsole update slots mount
Updating Pool Defs
6:32 < optiz0r> Just the latter is needed, however you need to bear in mind that the Scratch Pool option is copied from the pool to the volume itself at the time the volume is labelled, so after changing the pool definition you need to reload the pool parameters from the config file (using update pool) then reapply the pool parameters to the volume (with update volume) 16:34 < webdawg> So remove the Recycle Pool directive in the config, and then update pool, and update volume 16:34 < webdawg> Just update the recycle flag when I update the volume? 16:35 < webdawg> or Volume from Pool 16:36 < optiz0r> update pool, from resources. update volume, all volumes from all pools 16:37 < optiz0r> then run llist volume=XXX to check it's been applied properly
Kill all waiting Jobs
kill_waiting_jobs.bash #!/bin/bash jobIds=`echo 'status dir running' | bconsole | fgrep 'is waiting' | awk '{print $1}'` for i in $jobIds do if [ -z `echo "$i" | grep '^[0-9]\+$'` ] then echo "Error: job ID $i is not a number!" else echo "Killing waiting Bacula job $i" echo "cancel jobid=$i" | bconsole fi done
Delete all Old Jobs
THIS SCRIPT IS UNFINISHED (fgrep 'B' heh)
#!/bin/bash jobIds=`echo 'list jobs' | bconsole | fgrep 'B' | awk '{print $2}'` for i in $jobIds do if [ -z `echo "$i" | grep '^[0-9]\+$'` ] then echo "Error: job ID $i is not a number!" else echo "Deleting Backup Bacula job $i" echo "delete jobid=$i" | bconsole fi done
Truncate
FileVol
/var/lib/bareos/storage