Backblaze
Backups Retention Policy
- Daily: 3 days
- Weekly (Mondays): 31 days
- Monthly (first of month): 365 days
- Yearly (Jan 1): Indefinite
Cost
- Aug 2019 monthly cost (Storage only, no downloads) is at $2.57 for a total of 160 GB with ~ 20 GB added monthly
- 2018-12 estimate: $100-$200 per year.
Useful commands
Here we will list useful commands for our backups on B2
List Backup Contents
The following command will output a list of files stored on our 'ose-server-backups' bucket in our b2 account.
[root@opensourceecology backups]# rclone --b2-versions ls b2:ose-server-backups 21812118477 daily_hetzner2_20240921_072001.tar.gpg 21349753244 monthly_hetzner2_20231001_072001.tar.gpg 21360808568 monthly_hetzner2_20231101_072001.tar.gpg 21360301269 monthly_hetzner2_20231201_072001.tar.gpg 21820017340 monthly_hetzner2_20240201_072001.tar.gpg 21683700909 monthly_hetzner2_20240301_072001.tar.gpg 21660296728 monthly_hetzner2_20240401_072001.tar.gpg 21790035424 monthly_hetzner2_20240501_072001.tar.gpg 21603737883 monthly_hetzner2_20240601_072001.tar.gpg 21663769333 monthly_hetzner2_20240701_072001.tar.gpg 21991147307 monthly_hetzner2_20240801_072001.tar.gpg 21896377523 monthly_hetzner2_20240901_072001.tar.gpg 21942660432 weekly_hetzner2_20240826_072001.tar.gpg 21902006508 weekly_hetzner2_20240902_072001.tar.gpg 21873908566 weekly_hetzner2_20240909_072001.tar.gpg 21830987241 weekly_hetzner2_20240916_072001.tar.gpg 17516124812 yearly_hetzner2_20190101_111520.tar.gpg 18872422001 yearly_hetzner2_20200101_072001.tar.gpg 19827971632 yearly_hetzner2_20210101_072001.tar.gpg 21079942509 yearly_hetzner2_20230101_072001.tar.gpg 21541199047 yearly_hetzner2_20240101_072001.tar.gpg [root@opensourceecology backups]#
Note: The API keys will need the listFiles permission in order to execute the above command successfully. This permission is given to the root user, but not to the b2user user.
Install CLI
Backblaze has an API for B2, and they maintain a python CLI tool for interacting with your storage buckets. Unfortunately, this is not in the centos yum repositories, and the suggested way to install it via easy_install may break other python packages, such as certbot. The result of breaking certbot is that our website becomes inaccessible because our https certificates fail to autorenew. Therefore, it's critical that b2 is installed with care using a virtualenv.
sudo su - # we must install the b2 command in a virtualenv from the binary. if we do it via easy_install, it will # break other critical python packages, such as certbot (yielding our websites down when the https certs # fail to auto-renew properly) # for more info: https://wiki.opensourceecology.org/wiki/Maltfield_Log/2018_Q3#Wed_Aug_08.2C_2018 yum install python-virtualenv # and install other depends yum install python-setuptools git adduser b2user sudo su - b2user mkdir virtualenv pushd virtualenv/ virtualenv . source ~/virtualenv/bin/activate popd mkdir sandbox pushd sandbox git clone https://github.com/Backblaze/B2_Command_Line_Tool.git pushd B2_Command_Line_Tool/ python setup.py install popd popd # test it ~/virtualenv/bin/b2 version
For more information about the b2 command, please see the backblaze docs for the b2 command-line tool
Update CLI
As stated above, it's critical that this is done in a python virtualenv to prevent these steps from breaking the OS and therefore the websites, etc.
sudo su - b2user source ~/virtualenv/bin/activate cd ~/sandbox/B2_Command_Line_Tool/ git pull python setup.py install ~/virtualenv/bin/b2 version
Restore from backups
This section will describe how to extract data from an encrypted backup file stored in Backblaze B2.
Download from WUI
One of the significant benefits of Backblaze over similar cloud storage solutions is how easy it is to download archives. Indeed, you can simply do it from their Web User Interface.
To download one of or encrypted backup files from Backblaze B2, click on the "Upload/Download" button for the "ose-server-backups" bucket on the screen immediately after logging into the wui (see the OSE shared Keepass for the credentials)
Next, determine which file you need and click on it.
Note that, while storage is relatively cheap for B2, download bandwidth can be pricy. Before downloading a ton of files from our Backblaze B2 account, do your due diligence to determine _which_ archive you need from _which_ day to prevent excessive fees.
In the file's Details pop-up, click the "Download" button.
Hint: Because [a] it can take a _very_ long time to download these full backups and [b] an intermittent internet connection on your computer could cause the download to fail and [c] download bandwidth from B2 is not cheap, it may be wise to download from the cli instead
Download from CLI (Command Line Interface)
You can extract data from a backup on Backblaze B2 using their command line utility `b2`.
Because b2 wasn't installed from a trusted OS repository, we execute it as an unprivliged user `b2user`. The following commands shows how to become the `b2user`, list the backups available on the Backblaze B2 `ose-server-backups` bucket, and download the encrypted backup file.
Note: Don't download any archives within the /home/ directory tree because the entire '/home/' directory is backed-up. Instead, download to someplace in '/var/tmp/' as shown below.
[maltfield@hetzner2 backblaze]$ sudo su - b2user [sudo] password for maltfield: Last login: Sat Nov 24 13:10:42 UTC 2018 on pts/124 [b2user@hetzner2 ~]$ ~/virtualenv/bin/b2 ls ose-server-backups daily_hetzner2_20181221_111534.tar.gpg daily_hetzner2_20181222_111516.tar.gpg daily_hetzner2_20181223_111501.tar.gpg monthly_hetzner2_20181001_091809.tar.gpg monthly_hetzner2_20181101_091810.tar.gpg monthly_hetzner2_20181201_091759.tar.gpg weekly_hetzner2_20181126_091805.tar.gpg weekly_hetzner2_20181203_091823.tar.gpg weekly_hetzner2_20181210_091822.tar.gpg weekly_hetzner2_20181217_111545.tar.gpg [b2user@hetzner2 tmp]$ tmpDir="/var/tmp/backblazeRestore_`date +%Y%m%d_%H%M%S`" [b2user@hetzner2 tmp]$ cd $tmpDir [b2user@hetzner2 backblazeRestore_20181223_135712]$ ~/virtualenv/bin/b2 download-file-by-name ose-server-backups daily_hetzner2_20181223_111501.tar.gpg daily_hetzner2_20181223_111501.tar.gpg daily_hetzner2_20181223_111501.tar.gpg: 100%|| 17.5G/17.5G [19:29<00:00, 15.0MB/s] File name: daily_hetzner2_20181223_111501.tar.gpg File id: 4_z5605817c251dadb96e4d0118_f206daef4188682f6_d20181223_m113926_c001_v0001106_t0057 File size: 17509827199 Content type: application/octet-stream Content sha1: none INFO src_last_modified_millis: 1545565162536 [b2user@hetzner2 backblazeRestore_20181223_135712]$ du -sh * 17G daily_hetzner2_20181223_111501.tar.gpg [b2user@hetzner2 backblazeRestore_20181223_135712]$
For more information about the b2 command, please see the backblaze docs for the b2 command-line tool
Decrypt
OSE's backup data holds very sensitive content (ie; passwords, logs, etc), so they're encrypted before being uploaded to 3rd parties.
Use gpg and the 4K 'ose-backups-cron.key' keyfile (which lives in '/root/backups/' on the server and can also be found in the shared ose keepass) to decrypt this data as follows:
Note: Depending on the version of `gpg` installed, you may need to omit the '--batch' option.
[maltfield@hetzner2 backblaze]$ sudo su - [sudo] password for maltfield: Last login: Sun Dec 23 12:09:45 UTC 2018 on pts/113 [root@hetzner2 ~]# cd /var/tmp/back backblaze/ backblazeRestore_20181223_135712/ backups_for_migration_from_hetzner1/ [root@hetzner2 ~]# cd /var/tmp/backblazeRestore_20181223_135712/ [root@hetzner2 backblazeRestore_20181223_135712]# du -sh * 17G daily_hetzner2_20181223_111501.tar.gpg [root@hetzner2 backblazeRestore_20181223_135712]# gpg --batch --passphrase-file /root/backups/ose-backups-cron.key --output daily_hetzner2_20181223_111501.tar daily_hetzner2_20181223_111501.tar.gpg gpg: AES256 encrypted data gpg: encrypted with 1 passphrase [root@hetzner2 backblazeRestore_20181223_135712]# du -sh * 17G daily_hetzner2_20181223_111501.tar 17G daily_hetzner2_20181223_111501.tar.gpg [root@hetzner2 backblazeRestore_20181223_135712]#
There should now be a decrypted file. You can extract it to view the contents using `tar`.