Backblaze: Difference between revisions

From Open Source Ecology
Jump to navigation Jump to search
No edit summary
Line 2: Line 2:
Daily (3 day storage), weekly (first day of week), Monthly (first of month, 12 month delete), yearly (Jan 1)
Daily (3 day storage), weekly (first day of week), Monthly (first of month, 12 month delete), yearly (Jan 1)


=Communication=
=Cost=
Hey Marcin,
 
Can you please add our billing information to our blackblaze account?
 
Upon further research, I've decided to use Backblaze B2 for storing our
backups instead of Amazon Glacier. Though Glacier is slightly cheaper
per byte, they require they have a minimum lifetime of 90 days. And S3
(a more expensive & hot cloud storage service from Amazon) has a minimum
lifetime of 30 days. Consequently, our bill would be much, much higher
than B2 for daily backups (which we'd want to delete after a few days).
 
*https://wiki.opensourceecology.org/wiki/Maltfield_Log/2018_Q3#Mon_Jul_16.2C_2018
* https://www.backblaze.com/b2/cloud-storage-pricing.html
 
I'm still estimating the cost of our backups to be about $100/year.
 
Backblaze appears to be our best option, so I created an account with
them. We get 10G of free space on B2, so I've been able to test it over
the past few weeks. As of today, I've forked our backup script to just
upload the contents of our '/etc' directory (encrypted) to B2 on a daily
basis. I created lifecycle rules to delete daily backups after 3 days,
weekly backups after 1 month, and monthly backups after 1 year. I'll let
this run for probably a couple months to ensure everything is working
with the lifecycle rules before we cut over from dreamhost to Backblaze B2.
 
But for us to actually use B2 for our backups (cutting out dreamhost),
we need to give them our billing information (we get only 10G free, but
our daily backup is actually >15G).
 
The credentials for our blackblaze account can be found in our shared
ose keepass. In addition to username & password, you'll need to import
the 2FA key into your android app. The 2FA secret key is also in keepass.
 
After logging in, you should be able to click the "Billing" link on the
left-hand navigation pane to enter the information.
 
* https://secure.backblaze.com/billing.htm
 
Thank you,
 
Michael Altfield
Senior System Administrator
PGP Fingerprint: 8A4B 0AF8 162F 3B6A 79B7  70D2 AA3E DF71 60E2 D97B
 
Open Source Ecology
www.opensourceecology.org


2018-12 estimate: $100-$200 per year.


=Restore from backups=
=Restore from backups=

Revision as of 14:55, 23 December 2018

Usage

Daily (3 day storage), weekly (first day of week), Monthly (first of month, 12 month delete), yearly (Jan 1)

Cost

2018-12 estimate: $100-$200 per year.

Restore from backups

This section will describe how to extract data from an encrypted backup file stored in Backblaze B2.

Download from WUI

TODO

Download from CLI

You can extract data from a backup on Backblaze B2 using their command line utility `b2`.

TODO: how to install the cli, per https://wiki.opensourceecology.org/wiki/Maltfield_Log/2018_Q3#Wed_Aug_08.2C_2018

Because b2 wasn't installed from a trusted OS repository, we execute it as an unprivliged user `b2user`. The following commands shows how to become the `b2user`, list the backups available on the Backblaze B2 `ose-server-backups` bucket, and download the encrypted backup file.

Note: Don't download any archives within the /home/ directory tree because the entire '/home/' directory is backed-up. Instead, download to someplace in '/var/tmp/' as shown below.

[maltfield@hetzner2 backblaze]$ sudo su - b2user
[sudo] password for maltfield: 
Last login: Sat Nov 24 13:10:42 UTC 2018 on pts/124
[b2user@hetzner2 ~]$ ~/virtualenv/bin/b2 ls ose-server-backups
daily_hetzner2_20181221_111534.tar.gpg
daily_hetzner2_20181222_111516.tar.gpg
daily_hetzner2_20181223_111501.tar.gpg
monthly_hetzner2_20181001_091809.tar.gpg
monthly_hetzner2_20181101_091810.tar.gpg
monthly_hetzner2_20181201_091759.tar.gpg
weekly_hetzner2_20181126_091805.tar.gpg
weekly_hetzner2_20181203_091823.tar.gpg
weekly_hetzner2_20181210_091822.tar.gpg
weekly_hetzner2_20181217_111545.tar.gpg
[b2user@hetzner2 tmp]$ tmpDir="/var/tmp/backblazeRestore_`date +%Y%m%d_%H%M%S`"
[b2user@hetzner2 tmp]$ cd $tmpDir
[b2user@hetzner2 backblazeRestore_20181223_135712]$ ~/virtualenv/bin/b2 download-file-by-name ose-server-backups daily_hetzner2_20181223_111501.tar.gpg daily_hetzner2_20181223_111501.tar.gpg
daily_hetzner2_20181223_111501.tar.gpg: 100%|| 17.5G/17.5G [19:29<00:00, 15.0MB/s]
File name:    daily_hetzner2_20181223_111501.tar.gpg
File id:      4_z5605817c251dadb96e4d0118_f206daef4188682f6_d20181223_m113926_c001_v0001106_t0057
File size:    17509827199
Content type: application/octet-stream
Content sha1: none
INFO src_last_modified_millis: 1545565162536
[b2user@hetzner2 backblazeRestore_20181223_135712]$ du -sh *
17G     daily_hetzner2_20181223_111501.tar.gpg
[b2user@hetzner2 backblazeRestore_20181223_135712]$ 

For more information about the b2 command, please see the backblaze docs for the b2 command-line tool

Decrypt

OSE's backup data holds very sensitive content (ie; passwords, logs, etc), so they're encrypted before being uploaded to 3rd parties.

Use gpg and the 4K 'ose-backups-cron.key' keyfile (which lives in '/root/backups/' on the server and can also be found in the shared ose keepass) to decrypt this data as follows:

Note: Depending on the version of `gpg` installed, you may need to omit the '--batch' option.

[maltfield@hetzner2 backblaze]$ sudo su -
[sudo] password for maltfield: 
Last login: Sun Dec 23 12:09:45 UTC 2018 on pts/113
[root@hetzner2 ~]# cd /var/tmp/back
backblaze/                           backblazeRestore_20181223_135712/    backups_for_migration_from_hetzner1/
[root@hetzner2 ~]# cd /var/tmp/backblazeRestore_20181223_135712/
[root@hetzner2 backblazeRestore_20181223_135712]# du -sh *
17G     daily_hetzner2_20181223_111501.tar.gpg
[root@hetzner2 backblazeRestore_20181223_135712]# gpg --batch --passphrase-file /root/backups/ose-backups-cron.key --output daily_hetzner2_20181223_111501.tar daily_hetzner2_20181223_111501.tar.gpg
gpg: AES256 encrypted data
gpg: encrypted with 1 passphrase
[root@hetzner2 backblazeRestore_20181223_135712]# du -sh *
17G     daily_hetzner2_20181223_111501.tar
17G     daily_hetzner2_20181223_111501.tar.gpg
[root@hetzner2 backblazeRestore_20181223_135712]# 

There should now be a decrypted file. You can extract it to view the contents using `tar`.