Maltfield at 17:11, 21 May 2020

2020-05-21T17:11:08Z

← Older revision		Revision as of 17:11, 21 May 2020
Line 1:		Line 1:
			This guide describes how to update Discourse installed on our production [[OSE_Server]].

			For a guide on how to install Discourse on CentOS 7, see [[Discourse/Install]].

			To see our general Discourse page, see [[Discourse]].

	=Updating Discoruse=		=Updating Discoruse=

Line 243:		Line 249:

	Our backup cron also takes a tar of most of the '<code>/var/discourse</code>' directory, but this process doesn't stop the Discourse application, so it's an option for restore as a last-resort and not covered here.		Our backup cron also takes a tar of most of the '<code>/var/discourse</code>' directory, but this process doesn't stop the Discourse application, so it's an option for restore as a last-resort and not covered here.

			=References=
			{{Reflist}}

			=Links=
			# [[Discourse]]
			# [[Discourse/Install]]

			[[Category: IT Infrastructure]]
			[[Category: Software]]

Maltfield: Created page with "=Updating Discoruse= This section will describe how to update the Discourse software ==Discourse Versions== First, a note about Discourse releases: Discourse maintains a "s..."

2020-05-21T16:25:06Z

Created page with "=Updating Discoruse= This section will describe how to update the Discourse software ==Discourse Versions== First, a note about Discourse releases: Discourse maintains a "s..."

New page

=Updating Discoruse=

This section will describe how to update the Discourse software

==Discourse Versions==

First, a note about Discourse releases: Discourse maintains a "stable" release, but they don't actually backport bug patches to their stable releases like one would expect. There's no LTS (or STS!) for Discourse stable releases.

They only fix bugs in future beta releases (which will also include new commits that may break more things). Therefore, the default branch for production is "beta" releases, and they urge their customers not to use "stable." I honestly think this is a terrible idea, especially for a small org like OSE without any full-time ops staff to constantly update our prod apps.

* https://github.com/discourse/discourse/releases

IMHO, the takeaway to this is that updates should be done to Discourse very carefully and with through testing in staging before taking a backup and following the identical procedure on production.

==Check for Updates==

Discourse updates often, and it will tell you to update the app (requiring downtime) even if there's 1 (untested?) commit to their discourse github repository

First, determine your current version by logging-in and visiting the Admin section. You'll see the version listed as a version number (ex: 2.5.0.beta3) and a git commit hash (ex: 8e28ccb2ea). Be sure to write down the current version in your notes in-case something goes wrong during the upgrade.

* https://discourse.opensourceecology.org/admin

And you can determine the latest version by checking the Discourse projects' github releases section

* https://github.com/discourse/discourse/releases

==Step 0: Trigger Backup Scripts for System-Wide backup==

For good measure, trigger a backup of the entire system's database & files:

<pre>
sudo su -
time sudo /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log
exit
</pre>

When finished, list the backup files in our [[Backblaze]] B2 server backups bucket to verify that the whole system backup was successful before proceeding

<pre>
source /root/backups/backup.settings
$SUDO -u ${b2UserName} $B2 ls ${B2_BUCKET_NAME}
</pre>

==Step 1: Set variables==

Type these commands to set some variables, which will be used by the commands in the sections below. Replace 'www.opensourceecology.org' with the corresponding directory for the wp site you're updating.

<pre>
sudo su -
export vhostDir="/var/discourse/"
export stamp=`date +%Y%m%d_%T`
export tmpDir="/var/tmp/discourseUpgrade.${stamp}"

# verify
echo "${vhostDir}"
echo "${stamp}"
echo "${tmpDir}"
ls -lah "${vhostDir}"
</pre>

==Step 2: Make Vhost-specific backups==

The backups made in the previous step are huge. Because it's easier to work with vhost-specific backups, let's make a redundant copy available in /var/tmp/:

<pre>
mkdir "${tmpDir}"
chown root:root "${tmpDir}"
chmod 0700 "${tmpDir}"
pushd "${tmpDir}"

# discourse backup (db & uploaded files only)
nice rm -rf /var/discourse/shared/standalone/backups/default/*.tar.gz
time nice docker exec discourse_ose discourse backup
nice mv /var/discourse/shared/standalone/backups/default/*.tar.gz ${tmpDir}/

# files backup (all discourse files)
time nice tar --exclude "${vhostDir}/shared/standalone/postgres_data" --exclude "${vhostDir}/shared/standalone/postgres_data/uploads" --exclude "${vhostDir}/shared/standalone/backups" -czf ${tmpDir}/discourse_files.${stamp}.tar.gz /var/discourse/*

popd
</pre>

==Step 3: Update git==

First, try a `<code>git pull origin master</code>` from the '<code>/var/discourse</code>' directory. If it says that you're "Already up-to-date.", then skip this section. If you have [[#.60git_pull.60_fail|merge errors]], then you'll need to proceed with this section.

We've made some OSE-specific changes to the files in <code>/var/discourse</code> that conflict with the upstream git repo, so let's move those out of the way before updating. After the <code>git pull</code>, we'll update them again.

First execute the commands shown in the example below as root to get a list of all the files that have been modified locally.

<pre>
[root@osestaging1 /]# pushd /var/discourse
/var/discourse /
[root@osestaging1 discourse]# git status --short | grep -E '^\s*M'
M image/base/install-nginx
M launcher
[root@osestaging1 discourse]# popd
/
[root@osestaging1 /]#
</pre>

In the example above, we see that the '<code>/var/discourse/launcher</code>' script and '<code>/var/discourse/image/base/install-nginx</code>' script have both been modified in our custom OSE install. Now let's move those two files out of the way so we can do a clean `<code>git pull</code>` after.

Execute the following commands as root. If there were more than just the two files listed above that were modified, then you should update this documentation to reflect the necessary changes.

<pre>
export vhostDir="/var/discourse/"
stamp=`date "+%Y%m%d_%H%M%S"`

# launcher
mv "${vhostDir}/launcher" "${vhostDir}/launcher.${stamp}"
git checkout "${vhostDir}/launcher"

# install-nginx
mv "${vhostDir}/image/base/install-nginx" "${vhostDir}/image/base/install-nginx.${stamp}"
git checkout "${vhostDir}/image/base/install-nginx"
</pre>

Now sync to the upstream repo

<pre>
pushd "${vhostDir}"
git pull origin master
popd
</pre>

Before proceeding, let's get a diff of the changes so we can understand what may have changed upstream, which might break our commands in our OSE Discourse install guide

<pre>
diff "${vhostDir}/image/base/install-nginx.${stamp}" "${vhostDir}/image/base/install-nginx"
diff "${vhostDir}/launcher.${stamp}" "${vhostDir}/launcher"
</pre>

Write down the output from the above commands in your log; you may need it to debug later.

Next, to re-apply the changes to the '<code>install-nginx</code>' script and "<code>launcher</code>' script, see the following relevant section for a list of the necessary commands:

# [[#Nginx_mod_security]]

See also https://meta.discourse.org/t/how-do-i-manually-update-discourse-and-docker-image-to-latest/23325

==Step 4: Build Discourse docker image with OSE modifications==

Now we must rebuild the upstream Discourse Docker image with our OSE-specific modifications, such as giving nginx mod_security support.

<pre>
pushd "${vhostDir}/image/base"

# force a fresh build (no-cache) so the `git pull` lines will trigger
# note this will take a *ridiculously* long time; the Discourse team compiles many packages from source :(
time nice docker build --no-cache --network=host --tag 'discourse_ose' /var/discourse/image/base/

popd
</pre>

==Step 5: Rebuild the app==

<pre>
time nice ${vhostDir}/launcher rebuild discourse_ose
</pre>

You may also need to restart the "outer" nginx on the docker host too

<pre>
systemctl restart nginx
</pre>

=Creating a Backup=

To create a a backup of Discourse, see the "Updating Discourse" Steps 0-2 starting with [[#Step 0: Trigger Backup Scripts for System-Wide backup]]

=Restoring from Backup=

This section will describe how to restore Discourse from a previous backup.

==Discourse backup file==

The best way to restore Discourse from backup is to use the "proper" backup file that was safely generated by the Discourse app itself. After [[OSE_Server#Backups|retreiving the backup file]], copy it to the '<code>/var/discourse/shared/standalone/backups/default</code>' directory

<pre>
[root@osestaging1 discourse_ose]# [ -f /var/discourse/shared/standalone/backups/default/ ] || mkdir /var/discourse/shared/standalone/backups/default/
[root@osestaging1 discourse_ose]# cp discourse-2020-03-08-172140-v20191219112000.tar.gz /var/discourse/shared/standalone/backups/default/
[root@osestaging1 discourse_ose]# ls -lah /var/discourse/shared/standalone/backups/default/
total 56M
drwxr-xr-x. 2 root root 4.0K Mar 16 16:52 .
drwxr-xr-x. 3 tgriffing 33 4.0K Mar 16 16:52 ..
-rw-r--r--. 1 root root 56M Mar 16 16:52 discourse-2020-03-08-172140-v20191219112000.tar.gz
[root@osestaging1 discourse_ose]#
</pre>

Then execute the following commands as root to initiate the restore

<pre>
[root@osestaging1 discourse]# /var/discourse/launcher enter discourse_ose
root@osestaging1-discourse-ose:/var/www/discourse# discourse enable_restore
Restore are now permitted. Disable them with `disable_restore`
root@osestaging1-discourse-ose:/var/www/discourse# discourse restore discourse-2020-03-08-172140-v20191219112000.tar.gz
Starting restore: discourse-2020-03-08-172140-v20191219112000.tar.gz
[STARTED]
'system' has started the restore!
Marking restore as running...
Making sure /var/www/discourse/tmp/restores/default/2020-03-16-165545 exists...
Copying archive to tmp directory...
Unzipping archive, this may take a while...
Extracting dump file...
Validating metadata...
Current version: 20200311135425
Restored version: 20191219112000
Enabling readonly mode...
Pausing sidekiq...
Waiting up to 60 seconds for Sidekiq to finish running jobs...
Creating missing functions in the discourse_functions schema...
Restoring dump file... (this may take a while)
...
Cleaning stuff up...
Dropping functions from the discourse_functions schema...
Removing tmp '/var/www/discourse/tmp/restores/default/2020-03-16-165545' directory...
Unpausing sidekiq...
Marking restore as finished...
Notifying 'system' of the end of the restore...
Finished!
[SUCCESS]
Restore done.
root@osestaging1-discourse-ose:/var/www/discourse#
</pre>

That's it! The Discourse site should now be fully restored. After verification, be sure to disable the restore function again

<pre>
root@osestaging1-discourse-ose:/var/www/discourse# discourse disable_restore
Restore are now forbidden. Enable them with `enable_restore`
root@osestaging1-discourse-ose:/var/www/discourse#
</pre>

Note: You may need to restart the docker container to complete the restore:

<pre>
/var/discourse/launcher stop discourse_ose
/var/discourse/launcher start discourse_ose
</pre>

==File backup file==

Our backup cron also takes a tar of most of the '<code>/var/discourse</code>' directory, but this process doesn't stop the Discourse application, so it's an option for restore as a last-resort and not covered here.

Discourse/Updating - Revision history

Maltfield at 17:11, 21 May 2020

Maltfield: Created page with "=Updating Discoruse= This section will describe how to update the Discourse software ==Discourse Versions== First, a note about Discourse releases: Discourse maintains a "s..."