CHG-2018-01-03 migrate fef to hetzner2 - Revision history

Maltfield: Maltfield moved page CHG-2018-01-03 to CHG-2018-01-03 migrate fef to hetzner2

2020-04-27T12:21:15Z

Maltfield moved page CHG-2018-01-03 to CHG-2018-01-03 migrate fef to hetzner2

← Older revision	Revision as of 12:21, 27 April 2020
(No difference)

Maltfield at 19:58, 22 January 2018

2018-01-22T19:58:19Z

← Older revision		Revision as of 19:58, 22 January 2018
Line 1:		Line 1:
	=Status=		=Status=

			==2018-01-22 17:40 UTC==

			Catarina confirmed Filezilla for SFTP works. Marking CHG as Successful.

	==2018-01-22 17:19 UTC==		==2018-01-22 17:19 UTC==

Maltfield at 17:28, 22 January 2018

2018-01-22T17:28:04Z

← Older revision		Revision as of 17:28, 22 January 2018
Line 1:		Line 1:
	=Status=		=Status=

			==2018-01-22 17:19 UTC==

			Catarina confirmed that 3M image uploads work in the wordpress wui.

			She asked to make it possible for 64M uploads. Citing DOS concerns of increasing the query size limit, I highly discouraged this change. Instead, I asked her to test uploading the images via scp or sftp using Filezilla (as we did a training on many months ago). I'm still leaving this CHG open until that's been confirmed as working on her end.

	==2018-01-18 15:12 UTC==		==2018-01-18 15:12 UTC==

Maltfield at 15:37, 18 January 2018

2018-01-18T15:37:31Z

← Older revision		Revision as of 15:37, 18 January 2018
Line 14:		Line 14:
	## next, I got a 403 error from /wp-admin/async-upload.php		## next, I got a 403 error from /wp-admin/async-upload.php
	### /var/log/httpd/fef.opensourceecology.org/error_log shows a modsecurity issue:		### /var/log/httpd/fef.opensourceecology.org/error_log shows a modsecurity issue:
	~~{code}~~		<pre>
	==> /var/log/httpd/fef.opensourceecology.org/error_log <==		==> /var/log/httpd/fef.opensourceecology.org/error_log <==
	[Thu Jan 18 14:56:25.263164 2018] [:error] [pid 27682] [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_20_protocol_violations.conf"] [line "219"] [id "960915"] [rev "1"] [msg "Multipart parser detected a possible unmatched boundary."] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.9"] [maturity "8"] [accuracy "8"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_REQ"] [tag "CAPEC-272"] [hostname "fef.opensourceecology.org"] [uri "/wp-admin/async-upload.php"] [unique_id "WmC1mU7FUiUY6HdrzSWfWgAAAAA"]		[Thu Jan 18 14:56:25.263164 2018] [:error] [pid 27682] [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_20_protocol_violations.conf"] [line "219"] [id "960915"] [rev "1"] [msg "Multipart parser detected a possible unmatched boundary."] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.9"] [maturity "8"] [accuracy "8"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_REQ"] [tag "CAPEC-272"] [hostname "fef.opensourceecology.org"] [uri "/wp-admin/async-upload.php"] [unique_id "WmC1mU7FUiUY6HdrzSWfWgAAAAA"]
	~~{code}~~		</pre>
	### as above, whitelisted rule IDs:		### as above, whitelisted rule IDs:
	#### 960915, multipart_unmatched_boundary		#### 960915, multipart_unmatched_boundary

Maltfield at 15:36, 18 January 2018

2018-01-18T15:36:55Z

← Older revision		Revision as of 15:36, 18 January 2018
Line 1:		Line 1:
	=Status=		=Status=

			==2018-01-18 15:12 UTC==

			# attempted to fix the "Http error" reported by wordpress after attempting to upload a large image
			## using the browser debugger, I saw that it was nginx that returned a 413 error. I fixed this by increasing 'client_max_body_size' to '10M' in /etc/nginx/nginx.conf
			<pre>
			[root@hetzner2 dbChange.20180118_12:22:36]# grep -B 1 'client_max_body_size' /etc/nginx/nginx.conf
			# allow large posts for image uploads
			#client_max_body_size 1k;
			#client_max_body_size 900k;
			client_max_body_size 10M;
			</pre>
			## next, I got a 403 error from /wp-admin/async-upload.php
			### /var/log/httpd/fef.opensourceecology.org/error_log shows a modsecurity issue:
			{code}
			==> /var/log/httpd/fef.opensourceecology.org/error_log <==
			[Thu Jan 18 14:56:25.263164 2018] [:error] [pid 27682] [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_20_protocol_violations.conf"] [line "219"] [id "960915"] [rev "1"] [msg "Multipart parser detected a possible unmatched boundary."] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.9"] [maturity "8"] [accuracy "8"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_REQ"] [tag "CAPEC-272"] [hostname "fef.opensourceecology.org"] [uri "/wp-admin/async-upload.php"] [unique_id "WmC1mU7FUiUY6HdrzSWfWgAAAAA"]
			{code}
			### as above, whitelisted rule IDs:
			#### 960915, multipart_unmatched_boundary
			#### 200003, multipart_unmatched_boundary

			==2018-01-18 14:18 UTC==

			# confirmed that a db dump includes image tags with the domain-name hard-coded in the href. that was put in-place by Wordpress's "Add Media" wui button. That's not good; the links should be relative in the db!
			## did some research & found that wp core devs decided 7 years ago to keep absolute paths. This is especially bad for continuous integration or even a basic staging site
			* https://core.trac.wordpress.org/ticket/17048
			# there's no good, robust solution.
			* https://stackoverflow.com/questions/17187437/relative-urls-in-wordpress
			* https://wordpress.org/plugins/relative-url/
			* https://wordpress.org/plugins/root-relative-urls/
			* https://deluxeblogtips.com/relative-urls/
			* http://www.456bereastreet.com/archive/201010/how_to_make_wordpress_urls_root_relative/
			# take away is:
			## Let wordpress do its thing. Don't waste effort fighting wp when it auto-inserts an absolute path.
			## However, whenever you have to manually type a path in (ie: when configuring a widget, plugin nav bar, etc), please use a relative link.

			==2018-01-05 14:05 UTC==

			Catarina confirmed that the site is working, except for 2 minor issues:

			# WP inserts absolute-paths in images tags' src attributes when using the wui "Add Media" button
			# Uploads of large images fails with "Http error"

			==2018-01-04 22:07 UTC==

			I did a string string replacement of "http://opensourceecology.org/fef/" with "/" in the entire database.

			==2018-01-04 15:05 UTC==

			Catarina pointed out that many old posts still linked to 'http://www.opensourceecology.org/fef', but this was now all broken links.

			==2018-01-03 20:00 UTC==

			Fixed the banner icon by changing the hard-coded absolute link in the theme (Appearance -> Menus -> header_menu -> home) from 'http://opensourceecology.org/fef/' to '/'

			==2018-01-03 19:49 UTC==

			Marcin responded, indicating that the header icon linked to 'http://www.opensourceecology.org/fef', which linked to 'http://opensourceecology.org/fef-has-visitors/'

	==2018-01-03 17:52 UTC==		==2018-01-03 17:52 UTC==

Maltfield at 17:55, 3 January 2018

2018-01-03T17:55:31Z

← Older revision		Revision as of 17:55, 3 January 2018
Line 1:		Line 1:
	=Status=		=Status=

			==2018-01-03 17:52 UTC==

			Finished change & sent email to service owners for validation.

			==2018-01-03 17:00 UTC==

			Began Change

	==2017-12-21==		==2017-12-21==

Maltfield: /* Purpose */

2018-01-03T16:50:24Z

Purpose

← Older revision		Revision as of 16:50, 3 January 2018
Line 9:		Line 9:
	This change does the following for fef		This change does the following for fef

			# entirely migrate the 'fef' wordpress site from hetzner1 (shared hosting) to hetzner2 (dedicated hosting)
	# changes the domain from 'http://opensourceecology.org/fef/' to 'https://fef.opensourceecology.org'		# changes the domain from 'http://opensourceecology.org/fef/' to 'https://fef.opensourceecology.org'
	# update core wp from v4.2.18 to v4.9.1		# update core wp from v4.2.18 to v4.9.1

Maltfield at 16:47, 3 January 2018

2018-01-03T16:47:39Z

← Older revision		Revision as of 16:47, 3 January 2018
Line 55:		Line 55:

	# STEP 1: BACKUP DB		# STEP 1: BACKUP DB
			mkdir -p ${backupDir_hetzner1}/{current,old}
	pushd ${backupDir_hetzner1}/current/		pushd ${backupDir_hetzner1}/current/
	~~mkdir -p ${backupDir_hetzner1}/{current,old}~~
	mv ${backupDir_hetzner1}/current/* ${backupDir_hetzner1}/old/		mv ${backupDir_hetzner1}/current/* ${backupDir_hetzner1}/old/
	time nice mysqldump -u"${dbUser_hetzner1}" -p"${dbPass_hetzner1}" --all-databases \| bzip2 -c > ${backupDir_hetzner1}/current/${backupFileName_db_hetzner1}		time nice mysqldump -u"${dbUser_hetzner1}" -p"${dbPass_hetzner1}" --all-databases \| bzip2 -c > ${backupDir_hetzner1}/current/${backupFileName_db_hetzner1}

Maltfield: /* Status */

2018-01-03T15:25:45Z

Status

← Older revision		Revision as of 15:25, 3 January 2018
Line 1:		Line 1:
	=Status=		=Status=

	n/a		==2017-12-21==

			Catarina validated the ephemeral clone of the existing site running as staging on hetzner 2. She approved downtime for the migration on 2018-01-03 12:00 ET, followed by a 1 week content freeze.

	=Purpose=		=Purpose=

Maltfield at 15:23, 3 January 2018

2018-01-03T15:23:10Z

← Older revision		Revision as of 15:23, 3 January 2018
Line 5:		Line 5:
	=Purpose=		=Purpose=

	This change does the following for ~~openbuildinginstitute.org~~		This change does the following for fef

			# changes the domain from 'http://opensourceecology.org/fef/' to 'https://fef.opensourceecology.org'
	# update core wp from v4.2.18 to v4.9.1		# update core wp from v4.2.18 to v4.9.1
	# changes the docroot to a subversion repo of the wp codebase, making it easier to update in the future without sacrificing security (ie: permitting wordpress to update itself) for ease-of-use		# changes the docroot to a subversion repo of the wp codebase, making it easier to update in the future without sacrificing security (ie: permitting wordpress to update itself) for ease-of-use