Backblaze
Contents
Backups Retention Policy
- Daily: 3 days
- Weekly (Mondays): 31 days
- Monthly (first of month): 365 days
- Yearly (Jan 1): Indefinite
Cost
- Aug 2019 monthly cost (Storage only, no downloads) is at $2.57 for a total of 160 GB with ~ 20 GB added monthly
- 2018-12 estimate: $100-$200 per year.
Useful commands
Here we will list useful commands for our backups on B2
List Backup Contents
The following command will output a list of files stored on our 'ose-server-backups' bucket in our b2 account.
[root@opensourceecology ~]# sudo su - b2user Last login: Sat Aug 3 05:57:40 UTC 2019 on pts/0 [b2user@opensourceecology ~]$ ~/virtualenv/bin/b2 ls ose-server-backups daily_hetzner2_20190813_072001.tar.gpg daily_hetzner2_20190814_072001.tar.gpg monthly_hetzner2_20181001_091809.tar.gpg monthly_hetzner2_20181101_091810.tar.gpg monthly_hetzner2_20181201_091759.tar.gpg monthly_hetzner2_20190201_072001.tar.gpg monthly_hetzner2_20190301_072001.tar.gpg monthly_hetzner2_20190401_072001.tar.gpg monthly_hetzner2_20190501_072001.tar.gpg monthly_hetzner2_20190601_072001.tar.gpg monthly_hetzner2_20190701_072001.tar.gpg weekly_hetzner2_20190812_072001.tar.gpg yearly_hetzner2_20190101_111520.tar.gpg [b2user@opensourceecology ~]$
Install CLI
Backblaze has an API for B2, and they maintain a python CLI tool for interacting with your storage buckets. Unfortunately, this is not in the centos yum repositories, and the suggested way to install it via easy_install may break other python packages, such as certbot. The result of breaking certbot is that our website becomes inaccessible because our https certificates fail to autorenew. Therefore, it's critical that b2 is installed with care using a virtualenv.
sudo su - # we must install the b2 command in a virtualenv from the binary. if we do it via easy_install, it will # break other critical python packages, such as certbot (yielding our websites down when the https certs # fail to auto-renew properly) # for more info: https://wiki.opensourceecology.org/wiki/Maltfield_Log/2018_Q3#Wed_Aug_08.2C_2018 yum install python-virtualenv # and install other depends yum install python-setuptools git adduser b2user sudo su - b2user mkdir virtualenv pushd virtualenv/ virtualenv . source ~/virtualenv/bin/activate popd mkdir sandbox pushd sandbox git clone https://github.com/Backblaze/B2_Command_Line_Tool.git pushd B2_Command_Line_Tool/ python setup.py install popd popd # test it ~/virtualenv/bin/b2 version
For more information about the b2 command, please see the backblaze docs for the b2 command-line tool
Update CLI
As stated above, it's critical that this is done in a python virtualenv to prevent these steps from breaking the OS and therefore the websites, etc.
sudo su - b2user source ~/virtualenv/bin/activate cd ~/sandbox/B2_Command_Line_Tool/ git pull python setup.py install ~/virtualenv/bin/b2 version
Restore from backups
This section will describe how to extract data from an encrypted backup file stored in Backblaze B2.
Download from WUI
One of the significant benefits of Backblaze over similar cloud storage solutions is how easy it is to download archives. Indeed, you can simply do it from their Web User Interface.
To download one of or encrypted backup files from Backblaze B2, click on the "Upload/Download" button for the "ose-server-backups" bucket on the screen immediately after logging into the wui (see the OSE shared Keepass for the credentials)
Next, determine which file you need and click on it.
Note that, while storage is relatively cheap for B2, download bandwidth can be pricy. Before downloading a ton of files from our Backblaze B2 account, do your due diligence to determine _which_ archive you need from _which_ day to prevent excessive fees.
In the file's Details pop-up, click the "Download" button.
Hint: Because [a] it can take a _very_ long time to download these full backups and [b] an intermittent internet connection on your computer could cause the download to fail and [c] download bandwidth from B2 is not cheap, it may be wise to download from the cli instead
Download from CLI (Command Line Interface)
You can extract data from a backup on Backblaze B2 using their command line utility `b2`.
Because b2 wasn't installed from a trusted OS repository, we execute it as an unprivliged user `b2user`. The following commands shows how to become the `b2user`, list the backups available on the Backblaze B2 `ose-server-backups` bucket, and download the encrypted backup file.
Note: Don't download any archives within the /home/ directory tree because the entire '/home/' directory is backed-up. Instead, download to someplace in '/var/tmp/' as shown below.
[maltfield@hetzner2 backblaze]$ sudo su - b2user [sudo] password for maltfield: Last login: Sat Nov 24 13:10:42 UTC 2018 on pts/124 [b2user@hetzner2 ~]$ ~/virtualenv/bin/b2 ls ose-server-backups daily_hetzner2_20181221_111534.tar.gpg daily_hetzner2_20181222_111516.tar.gpg daily_hetzner2_20181223_111501.tar.gpg monthly_hetzner2_20181001_091809.tar.gpg monthly_hetzner2_20181101_091810.tar.gpg monthly_hetzner2_20181201_091759.tar.gpg weekly_hetzner2_20181126_091805.tar.gpg weekly_hetzner2_20181203_091823.tar.gpg weekly_hetzner2_20181210_091822.tar.gpg weekly_hetzner2_20181217_111545.tar.gpg [b2user@hetzner2 tmp]$ tmpDir="/var/tmp/backblazeRestore_`date +%Y%m%d_%H%M%S`" [b2user@hetzner2 tmp]$ cd $tmpDir [b2user@hetzner2 backblazeRestore_20181223_135712]$ ~/virtualenv/bin/b2 download-file-by-name ose-server-backups daily_hetzner2_20181223_111501.tar.gpg daily_hetzner2_20181223_111501.tar.gpg daily_hetzner2_20181223_111501.tar.gpg: 100%|| 17.5G/17.5G [19:29<00:00, 15.0MB/s] File name: daily_hetzner2_20181223_111501.tar.gpg File id: 4_z5605817c251dadb96e4d0118_f206daef4188682f6_d20181223_m113926_c001_v0001106_t0057 File size: 17509827199 Content type: application/octet-stream Content sha1: none INFO src_last_modified_millis: 1545565162536 [b2user@hetzner2 backblazeRestore_20181223_135712]$ du -sh * 17G daily_hetzner2_20181223_111501.tar.gpg [b2user@hetzner2 backblazeRestore_20181223_135712]$
For more information about the b2 command, please see the backblaze docs for the b2 command-line tool
Decrypt
OSE's backup data holds very sensitive content (ie; passwords, logs, etc), so they're encrypted before being uploaded to 3rd parties.
Use gpg and the 4K 'ose-backups-cron.key' keyfile (which lives in '/root/backups/' on the server and can also be found in the shared ose keepass) to decrypt this data as follows:
Note: Depending on the version of `gpg` installed, you may need to omit the '--batch' option.
[maltfield@hetzner2 backblaze]$ sudo su - [sudo] password for maltfield: Last login: Sun Dec 23 12:09:45 UTC 2018 on pts/113 [root@hetzner2 ~]# cd /var/tmp/back backblaze/ backblazeRestore_20181223_135712/ backups_for_migration_from_hetzner1/ [root@hetzner2 ~]# cd /var/tmp/backblazeRestore_20181223_135712/ [root@hetzner2 backblazeRestore_20181223_135712]# du -sh * 17G daily_hetzner2_20181223_111501.tar.gpg [root@hetzner2 backblazeRestore_20181223_135712]# gpg --batch --passphrase-file /root/backups/ose-backups-cron.key --output daily_hetzner2_20181223_111501.tar daily_hetzner2_20181223_111501.tar.gpg gpg: AES256 encrypted data gpg: encrypted with 1 passphrase [root@hetzner2 backblazeRestore_20181223_135712]# du -sh * 17G daily_hetzner2_20181223_111501.tar 17G daily_hetzner2_20181223_111501.tar.gpg [root@hetzner2 backblazeRestore_20181223_135712]#
There should now be a decrypted file. You can extract it to view the contents using `tar`.
