Maltfield Log/2019 Q1

My work log from the year 2019 Quarter 1. I intentionally made this verbose to make future admin's work easier when troubleshooting. The more keywords, error messages, etc that are listed in this log, the more helpful it will be for the future OSE Sysadmin.

See Also

1. Maltfield_Log
2. User:Maltfield
3. Special:Contributions/Maltfield

Mon Feb 26, 2019

1. Duncan Cameron responded to my question about using the phplist cli, suggesting that it might be just a permissions problem with the session files https://discuss.phplist.org/t/phplist-cli-p-processqueue-does-nothing-fails-at-languages-php-session-start/5025/2?u=maltfield
   1. I verified that the permissions were OK, but pointed out that I do have a hardened php session config, which may be an issue if--for example--there's some poorly designed code somewhere that expects the session name to be the default = PHPSESSID"
2. I did some more digging. The lines that we're hitting this issue on is "@session_start()"
3. I found that the "@" at the start of the function call causes erros to be suppressed. Stupid but true. I removed it, and I got some output

[maltfield@hetzner2 ~]$ sudo su -s /bin/bash apache -c "/usr/bin/php /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php -c /var/www/html/phplist.opensourceecology.org/config.php -p processqueue"
[sudo] password for maltfield: 
PHP Fatal error:  session_start(): Failed to initialize storage module: files (path: ) in /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/languages.php on line 57
PHP Stack trace:
PHP   1. {main}() /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php:0
PHP   2. include_once() /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php:102
PHP   3. session_start() /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/languages.php:57
before languages
		test
[maltfield@hetzner2 ~]$ 

1. 1. note the "before lanugages" & "test" lines were echos that I added to determine where the issue lies
2. I did some googling for "session_start()" and "failed to initialize storage module"
3. I wonder if all the php.ini configs are applied when calling php via the cli. I checked it by trying to run a session_start() manually via the cli, and I got totally different results

[maltfield@hetzner2 ~]$ echo '<?php session_start();?>' | sudo su -s /bin/bash apache -c "/usr/bin/php"
PHP Fatal error:  Call to undefined function ecsession_start() in - on line 1
PHP Stack trace:
PHP   1. {main}() -:0
[maltfield@hetzner2 ~]$ echo '<?php session_start();?>' | sudo su -s /bin/bash apache -c "/usr/bin/php"
PHP Warning:  session_start(): open_basedir restriction in effect. File(/tmp) is not within the allowed path(s): (/home/wp/.wp-cli:/usr/share/pear:/var/lib/php/tmp_upload:/var/lib/php/session:/var/www/html/www.openbuildinginstitute.org:/var/www/html/staging.openbuildinginstitute.org/:/var/www/html/staging.opensourceecology.org/:/var/www/html/www.opensourceecology.org/:/var/www/html/fef.opensourceecology.org/:/var/www/html/seedhome.openbuildinginstitute.org:/var/www/html/oswh.opensourceecology.org/:/var/www/html/wiki.opensourceecology.org/:/var/www/html/cacti.opensourceecology.org/:/var/www/html/d3d.opensourceecology.org:/var/www/html/3dp.opensourceecology.org:/var/www/html/microfactory.opensourceecology.org:/var/www/html/phplist.opensourceecology.org:/var/www/html/microfactory.opensourceecology.org:/usr/share/cacti/:/etc/cacti/) in - on line 1
PHP Stack trace:
PHP   1. {main}() -:0
PHP   2. session_start() -:1
PHP Fatal error:  session_start(): Failed to initialize storage module: files (path: ) in - on line 1
PHP Stack trace:
PHP   1. {main}() -:0
PHP   2. session_start() -:1
[maltfield@hetzner2 ~]$ 

1. so, actually, it's trying to get to the session dir in "/tmp" that tells me it's veering off from the php.ini settings. But it still gets the open_basedir config from php.ini! Strange..
2. it looks like I actually don't set "session.save_path" in php.ini. Instead, I use session.cookie_secure. I think that this config prevents sessions from being used over http, requiring https. That makes sense
3. actually, that looks like a total config failure. The "session.cookie_secure" option should simply be a boolean http://php.net/manual/en/session.configuration.php#ini.session.cookie-secure
4. I changed "session.cookie_secure" to "1" and set "session.save_path" to "/var/lib/php/session"

session.save_path = "/var/lib/php/session"                                      
                                                                                
; Whether to use cookies.                                                       
; http://php.net/session.use-cookies                                            
session.use_cookies = 1                                                         
                                                                                
; http://php.net/session.cookie-secure                                          
;session.cookie_secure =                                                        
session.cookie_secure = 1      

1. now it works!

[maltfield@hetzner2 ~]$ sudo su -s /bin/bash apache -c "/usr/bin/php /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php -c /var/www/html/phplist.opensourceecology.org/config.php -p processqueue"
test1
phpList - phpList version 3.3.3 (c) 2000-2019 phpList Ltd, https://www.phplist.com                                                                              
phpList version 3.3.3 (c) 2000-2019 phpList Ltd, https://www.phplist.comphpList - Maximum time for queue processing: 99999 [0.0026350000] (107)                 
phpList - Recently sent : 0
phpList - Started [0.0042370000] (110)
phpList - Sending in batches of 1 emails [0.0002000000] (111)
phpList -  select id from phplist_message where status not in ("draft", "sent", "prepared", "suspended") and embargo  [0.0006080000] (112)                      
phpList - Processing has started, [0.0016170000] (115)
phpList - One campaign to process. [0.0015100000] (117)
phpList - sending of this campaign will stop, if it is still going in  3 days 22 hours 11 seconds [0.0035370000] (133)                                          
phpList - Processing campaign 22 [0.0096130000] (150)
phpList - Looking for subscribers [0.0059470000] (155)
phpList - User select query select distinct u.id from phplist_listuser as listuser                                                                              
		inner join phplist_user_user as u ON u.id = listuser.userid
		inner join phplist_listmessage as listmessage ON listuser.listid = listmessage.listid                                                                   
		left join phplist_usermessage as um ON (um.messageid = 22 and um.userid = listuser.userid)                                                              
		where
		listmessage.messageid = 22
		and listmessage.listid = listuser.listid
		and u.id = listuser.userid
		and um.userid IS NULL
		and u.confirmed and !u.blacklisted and !u.disabled
		  [0.0017760000] (158)
phpList - Found them: 0 to process [0.0027430000] (162)
phpList - Processed 0 out of 0 subscribers [0.0028240000] (165)
phpList - Hmmm, No subscribers found to send to [0.0014980000] (167)
phpList - It took very little time to send this message [0.0073380000] (183)
phpList - Script stage: 3 [0.0015600000] (186)
phpList - Finished, Nothing to do [0.0001340000] (187)
phpList - Finished, All done [0.0025960000] (192)
[maltfield@hetzner2 ~]$ 

1. I did some research about why editing the campaign & sending it again doesn't actually send anything. It's because phplist won't send a campaign (even after being edited & resent) to a subscirber >1 time. https://discuss.phplist.org/t/requeue-send-message-again-to-same-people-or/1259/3
   1. this is very helpful information; it means that I can try out sending the campain to our smallest list, then re-send it to the next largets list, and then finally to OSEmail without fear that I'll send the campaign more than once to a subscriber who is subscribed to more than one list!

Sat Feb 24, 2019

1. I traced the issue with the cli execution producing no output to the languates.php file's session_start() call https://github.com/phpList/phplist3/blob/849a53f8c93eb9940ae08180b960bd7a71957df2/public_html/lists/admin/languages.php#L56
2. The only other reference to this issue I could find was on the old phplist forms from 2012. Unfortunately, it got no replies https://discuss.phplist.org/t/phplist-cli-p-processqueue-does-nothing-fails-at-languages-php-session-start/5025
3. I posted a question about this to the phplist forums https://discuss.phplist.org/t/phplist-cli-p-processqueue-does-nothing-fails-at-languages-php-session-start/5025

Sat Feb 23, 2019

1. email
2. logging time

1. ...

1. I sent a test repermission campagin to the 'test' list. It worked.
2. One of my concerns about this real repermission campaign is sending mass mail for the first time. Will it work? Will it take hours? Days?
3. Our OSEmail list has 1,061 subscribers. True Fans is 957. Design Sprints is 349. Therefore, I think we should first do a test of the repermission campaign with the Design Sprints. Then True Fans. THen OSEmail.
4. I want to get cron queue processing complete before the real test, so that it will send the campaign to all our subscribers without me having to leave my web browser open.
5. This documentation talks about sending campaigns via the wui, cli, cron, or script https://resources.phplist.com/documentation/sendingcampaign
6. This documentation talks about cli options for batch processing https://resources.phplist.com/system/batch_processing
7. I found the relevant binary for manipulating phplist via the cli on our server

[root@hetzner2 phplist.opensourceecology.org]# pwd
/var/www/html/phplist.opensourceecology.org
[root@hetzner2 phplist.opensourceecology.org]# ls -lah bin/phplist
----r----- 1 not-apache apache 766 May 15  2018 bin/phplist
[root@hetzner2 phplist.opensourceecology.org]# 

1. It's important to note that this file is not executable. I fixed that here:

[root@hetzner2 phplist.opensourceecology.org]# chmod 0050 bin/phplist
[root@hetzner2 phplist.opensourceecology.org]# ls -lah bin/phplist
----r-x--- 1 not-apache apache 766 May 15  2018 bin/phplist
[root@hetzner2 phplist.opensourceecology.org]# pwd
/var/www/html/phplist.opensourceecology.org
[root@hetzner2 phplist.opensourceecology.org]# 

1. And we certainly don't want it to be executed as root for security reasons, so here's the command to execute it as the apache user. Interesting to note how it just spat a cronjob example at us when we use the '-pprocessqueue' argument per the documentation above

[maltfield@hetzner2 ~]$ sudo su -s /bin/sh apache -c "/usr/bin/php /var/www/html/phplist.opensourceecology.org/bin/phplist -pprocessqueue"                                                                                                                                 

# script to run phpList from commandline. You may need to edit this to make it work
# with your shell environment. The following should work for Bash on Linux
# but this may vary strongly in other situations. You will need to dig into the
# code to make sure it works for you.

# in commandline mode, access is restricted to users who are listed in the config file
# check README.commandline for more info
#
# when you set this file up, and place it in your PATH, you can do eg
#  $ phplist -pprocessqueue
# to run the queue

# run the phpList index file with all parameters passed to this script
# make sure the php binary is "cli"
# 
/usr/bin/php /home/website/public_html/lists/admin/index.php -c /home/website/public_html/lists/config/config.php $*
[maltfield@hetzner2 ~]$

1. Hmm...It looks like after I sent the campaign to the "test" list, it was removed from the list of campagin drafts in Campaigns -> Send a campaign -> Draft campaigns
2. I found that I could move an old campaign back into the "Draft campaigns" list by going to Campaigns -> List of campaigns -> View -> Edit campaign
3. I re-sent the campaign to the 'test' list again, but this time I didn't process the queue in the wui. I tried to run the above command to process the queue on the cli, but it just ouputed the cron thingy again. Reading it, I realized that I hadn't specified any users who can run the script. I checked the config_extended.php file and extracted some relevant options

// If you set up your system to send the message automatically (from commandline),                                                    
// you can set this value to 0, so "Process Queue" will disappear from the site                                                       
// this will also stop users from loading the page on the web frontend, so you will                                                   
// have to make sure that you run the queue from the commandline                                                                      
// check README.commandline how to do this                                                                                            
define('MANUALLY_PROCESS_QUEUE', 1);    
...
// set this to 0, if you set up a cron to download bounces regularly by using the                                                     
// commandline option. If this is 0, users cannot run the page from the web                                                           
// frontend. Read README.commandline to find out how to set it up on the                                                              
// commandline                                                                                                                        
define('MANUALLY_PROCESS_BOUNCES', 1);   
...
// if you use commandline, you will need to identify the users who are allowed to run                                                 
// the script. See README.commandline for more info                                                                                   
// $commandline_users = array("admin");                                                                                               
// or you can use the following to disable the check (take off the # in front of the line)                                            
$commandline_users = array();    

1. I added the following to our config file

// we set MANUALLY_PROCESS_QUEUE to 0 so that the "Process the Queue" button                                                          
// will disappear from the WUI as the queue is processed via cron                                                                     
define('MANUALLY_PROCESS_QUEUE', 0);                                                                                                  
                                                                                                                                      
// whitelist of users that are permitted to execute bin/phplist for cli control                                                       
$commandline_users = array( "apache" );

1. so the cron job actually tells me to use index.php. I tried it, but I'm still having the same problem..

[maltfield@hetzner2 ~]$ sudo su -s /bin/sh apache -c "/usr/bin/php /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php -c /var/www/html/phplist.opensourceecology.org/config.php -pprocessqueue"
[maltfield@hetzner2 ~]$ 

1. oh, the contents of that phplist script is simply that cron job!

[root@hetzner2 phplist.opensourceecology.org]# date
Sat Feb 23 18:53:58 UTC 2019
[root@hetzner2 phplist.opensourceecology.org]# pwd
/var/www/html/phplist.opensourceecology.org
[root@hetzner2 phplist.opensourceecology.org]# cat bin/phplist 
#!/bin/bash

# script to run phpList from commandline. You may need to edit this to make it work
# with your shell environment. The following should work for Bash on Linux
# but this may vary strongly in other situations. You will need to dig into the
# code to make sure it works for you.

# in commandline mode, access is restricted to users who are listed in the config file
# check README.commandline for more info
#
# when you set this file up, and place it in your PATH, you can do eg
#  $ phplist -pprocessqueue
# to run the queue

# run the phpList index file with all parameters passed to this script
# make sure the php binary is "cli"
# 
/usr/bin/php /home/website/public_html/lists/admin/index.php -c /home/website/public_html/lists/config/config.php $*
[root@hetzner2 phplist.opensourceecology.org]# 

1. executing the command does nothing, so I truncated it. At least the php bit is working..

[maltfield@hetzner2 ~]$ sudo su -s /bin/sh apache -c "/usr/bin/php -v"
PHP 5.6.33 (cli) (built: Jan 14 2018 08:07:11) 
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
	with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans
[maltfield@hetzner2 ~]$ 

1. slightly longer returns decent positive resultes too

[maltfield@hetzner2 ~]$ sudo su -s /bin/sh apache -c "/usr/bin/php /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php"
Cannot find config file
[maltfield@hetzner2 ~]$ 

1. with the config file it does nothing again

[maltfield@hetzner2 ~]$ sudo su -s /bin/sh apache -c "/usr/bin/php /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php -c /var/www/html/phplist.opensourceecology.org/config.php"
[maltfield@hetzner2 ~]$ 

1. I found better documentation on this cron, but it's not addressing my issue https://www.phplist.org/manual/ch036_setting-up-your-cron.xhtml
   1. Important to note: running this "-p processqueue" option over-and-over does not risk overloading the server; phplist will handle that
2. I'll have to open a forum post about this..

Sun Feb 17, 2019

1. I confirmed that our imported users into phplist don't have the "I agree to the OSE Privacy Policy" checkbox ticked
2. I confirmed, most unfortunately, that after clicking the big green button titled "I agree to the new Privacy Policy / Yes keep sending me the OSEmail newsletter!" button in the repermission campaign, the "I agree to the OSE Privacy Policy" attribute for the subscriber does not change (it'll still be unticked for imported users)
3. That said, the subscriber's History -> Subscription tab shows that they were added to the blacklist then removed from the blacklist "for manual confirmation of subscription". So we would probably have our ducks in a row here, since we could coorelate this specific event in the subscriber's history back to the campagin
4. I asked about this 3 months ago, but it's gotten no responses https://discuss.phplist.org/t/re-permission-campaign-requiring-explicity-tos-action/4710
5. In any case, new users will have the checkbox, and old users will either be blacklisted or necessarily have clicked our "I agree to the OSE Privacy Policy" button in order to be un-blacklisted

Fri Feb 15, 2019

1. Marcin still wants better formatting for the transaction message = "Content of message subscribers receive when they subscribe"
   1. I added a colon to the end of the line "In order to provide you with this service we'll need to"
   2. Marcin didn't want the newlines to be inserted, but it's not in our config. They occur as part of a wordwrap after 75-80-ish characters. This is normal, and I think the negatives (ie: a paragraph of 200 words all appearing on one line) would outweigh the positives if we were to change this behavoiur)

1. I finished my documentation Backblaze https://wiki.opensourceecology.org/wiki/Backblaze#Download_from_WUI
2. I logged into the Backblaze B2 account and saw our 'ose-server-backups' bucket had 16 files at a total of 193G. Our bill last month was $0.78, and the upcoming bill is estimated at $0.86

Tue Feb 12, 2019

1. Made some more changes to our phplist config per Marcin's requests
   1. I fixed a spelling mistake in the "State/Provence" attribute = 12. Changed to "State/Province"
   2. Deleted "Videography / Video Editing / Script Writing" option = 11 in attribte "Video Production / Video Editing / Script Writing / Explainer Videos " attribute = 16, as there was a redundant to the other option "Video Production / Video Editing / Script Writing / Explainer Videos" = 6.
      1. There are currently 33 skills is the current list of options for this attribute "Please identify which skills you possess"

Project Management
Community Management
Electrical Engineering
Computer Aided Design (CAD; CAM; CAE)
Computer Animation / Modeling
Video Production / Video Editing / Script Writing / Explainer Videos
Technical Writing / Documentation
Graphics / Design / Infographics / Computer Graphics
Fabrication / Digital Fabrication
Machine Design
Mechatronics
Power Electronics
Electrical Motor / Generator Design
Electronics
Hydraulic Motor Design
Industrial Laser Design
Industrial Robotics Design
Solar Engineering
Metallurgy
Hot Metal Processing
Tool & Die
Precision Machine Design
Wind Turbine Design
Mechnical Engineering
Agricultural Engineering
Automotive Engineering
Reliabilty Engineering
Industrial Engineering
Life Cycle Logistics
Computer Programming / Database / CMS / Wiki
Engineering
Hydraulics / Pneumatics
FreeCAD

1. 1. I changed attribute "City" = 11 to "City (so we can put you on a map)"
   2. Marcin wants the OSE new black/grey logo to appear at the top of the ose form here https://wiki.opensourceecology.org/wiki/File:OSE_Logo_-_Black.png
      1. he drew an example here https://docs.google.com/presentation/d/1gwaOiXfqc5jVx5-3VKVWDdEzjNqdahYPX5fkhy6TRYQ/edit#slide=id.g45ed589c64_0_0
      2. first, I downloaded the logo from the wiki link above and uploaded the logo to the wordpress site https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black.png
      3. I got the wordpress-generated smaller versions of this icon:
         1. https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-150x150.png
         2. https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-300x186.png
         3. https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-1024x635.png
      4. I checked the sizes of the above images. The thumbnail is unuseable as it's a square that cut off the left & right ends of the icon. The best option to use here is the 300x186 image.
      5. I made the logo appear at the top, shunk it to 215 pixels width, wrapped it in a 300 pixel div, and centered both the image & byline in the div.

	<div style="width:300px; text-align:center;">
		<img src="https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-300x186.png" alt="Open Source Ecology Logo" width="215" class="aligncenter size-medium wp-image-10298" style="margin: 0 auto; display:block;" align="center"/>
		<strong>Open Source Blueprints for Civilization</strong>
	</div>
	<br />

1. 1. Marcin also wanted to customize the transaction message = "Content of message subscribers receive when they subscribe" as shown in this example https://wiki.opensourceecology.org/wiki/OSEmail#Subscribe
      1. I updated the subject from "Request for confirmation" to "Open Source Ecology Newsletters - Confirmation Required"
      2. Unfortunately, it's non-trivial to change the astrick-bulleted list to a numbered list. Perhaps that could be obtainable via a code change & PR, but I don't think that juice is worth the squeeze..

 You have been subscribed to the following newsletters:

[LISTS]


Please click the following link to confirm it's really you:

[CONFIRMATIONURL]


In order to provide you with this service we'll need to

Transfer your contact information to phplist.opensourceecology.org
Store your contact information in your phplist.opensourceecology.org account
Send you emails from phplist.opensourceecology.org
Track your interactions with these emails for marketing purposes

If this is not correct, or you do not agree, simply take no action and delete this message.

1. 1. 1. I tried changing this to the following, but it didn't work as expected. It appears that I can't use html in this message.

<h1>Open Source Ecology Newsletters - Confirmation Required</h1>

Please confirm your email. You have been subscribed to the following newsletters:

[LISTS]


Please click the following link to confirm it's really you:

[CONFIRMATIONURL]


In order to provide you with this service we'll need to

 * Transfer your contact information to phplist.opensourceecology.org
 * Store your contact information in your phplist.opensourceecology.org account
 * Send you emails from phplist.opensourceecology.org
 * Track your interactions with these emails for marketing purposes

If this is not correct, or you do not agree, simply take no action and delete this message.

1. 1. 1. When I signed up, I got this back

Open Source Ecology Newsletters - Confirmation Required

Please confirm your email. You have been subscribed to the following
newsletters:

 * OSEmail

Please click the following link to confirm it's really you:

http://phplist.opensourceecology.org/lists/?p=confirm&uid=e018a9647ef71a498f4683adf5c10394

In order to provide you with this service we'll need to

* Transfer your contact information to phplist.opensourceecology.org
* Store your contact information in your phplist.opensourceecology.org
account
* Send you emails from phplist.opensourceecology.org
* Track your interactions with these emails for marketing purposes

If this is not correct, or you do not agree, simply take no action and
delete this message.

1. 1. 1. But I think that's sufficient. I'll send it to Marcin for approval.

Sat Feb 02, 2019

1. Made some more changes to our phplist config per Marcin's requests
   1. I fixed a spelling mistake in the "State/Provence" attribute = 12. Changed to "State/Province"
   2. Deleted "Videography / Video Editing / Script Writing" option = 11 in attribte "Video Production / Video Editing / Script Writing / Explainer Videos " attribute = 16, as there was a redundant to the other option "Video Production / Video Editing / Script Writing / Explainer Videos" = 6.
      1. There are currently 33 skills is the current list of options for this attribute "Please identify which skills you possess"

Project Management
Community Management
Electrical Engineering
Computer Aided Design (CAD; CAM; CAE)
Computer Animation / Modeling
Video Production / Video Editing / Script Writing / Explainer Videos
Technical Writing / Documentation
Graphics / Design / Infographics / Computer Graphics
Fabrication / Digital Fabrication
Machine Design
Mechatronics
Power Electronics
Electrical Motor / Generator Design
Electronics
Hydraulic Motor Design
Industrial Laser Design
Industrial Robotics Design
Solar Engineering
Metallurgy
Hot Metal Processing
Tool & Die
Precision Machine Design
Wind Turbine Design
Mechnical Engineering
Agricultural Engineering
Automotive Engineering
Reliabilty Engineering
Industrial Engineering
Life Cycle Logistics
Computer Programming / Database / CMS / Wiki
Engineering
Hydraulics / Pneumatics
FreeCAD

1. 1. I changed attribute "City" = 11 to "City (so we can put you on a map)"
   2. Marcin wants the OSE new black/grey logo to appear at the top of the ose form here https://wiki.opensourceecology.org/wiki/File:OSE_Logo_-_Black.png
      1. he drew an example here https://docs.google.com/presentation/d/1gwaOiXfqc5jVx5-3VKVWDdEzjNqdahYPX5fkhy6TRYQ/edit#slide=id.g45ed589c64_0_0
      2. first, I downloaded the logo from the wiki link above and uploaded the logo to the wordpress site https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black.png
      3. I got the wordpress-generated smaller versions of this icon:
         1. https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-150x150.png
         2. https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-300x186.png
         3. https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-1024x635.png
      4. I checked the sizes of the above images. The thumbnail is unuseable as it's a square that cut off the left & right ends of the icon. The best option to use here is the 300x186 image.
      5. I made the logo appear at the top, shunk it to 215 pixels width, wrapped it in a 300 pixel div, and centered both the image & byline in the div.

	<div style="width:300px; text-align:center;">
		<img src="https://www.opensourceecology.org/wp-content/uploads/2012/05/OSE_Logo_-_Black-300x186.png" alt="Open Source Ecology Logo" width="215" class="aligncenter size-medium wp-image-10298" style="margin: 0 auto; display:block;" align="center"/>
		<strong>Open Source Blueprints for Civilization</strong>
	</div>
	<br />

1. 1. Marcin also wanted to customize the transaction message = "Content of message subscribers receive when they subscribe" as shown in this example https://wiki.opensourceecology.org/wiki/OSEmail#Subscribe
      1. I updated the subject from "Request for confirmation" to "Open Source Ecology Newsletters - Confirmation Required"
      2. Unfortunately, it's non-trivial to change the astrick-bulleted list to a numbered list. Perhaps that could be obtainable via a code change & PR, but I don't think that juice is worth the squeeze..

 You have been subscribed to the following newsletters:

[LISTS]


Please click the following link to confirm it's really you:

[CONFIRMATIONURL]


In order to provide you with this service we'll need to

Transfer your contact information to phplist.opensourceecology.org
Store your contact information in your phplist.opensourceecology.org account
Send you emails from phplist.opensourceecology.org
Track your interactions with these emails for marketing purposes

If this is not correct, or you do not agree, simply take no action and delete this message.

1. 1. 1. I tried changing this to the following, but it didn't work as expected. It appears that I can't use html in this message.

<h1>Open Source Ecology Newsletters - Confirmation Required</h1>

Please confirm your email. You have been subscribed to the following newsletters:

[LISTS]


Please click the following link to confirm it's really you:

[CONFIRMATIONURL]


In order to provide you with this service we'll need to

 * Transfer your contact information to phplist.opensourceecology.org
 * Store your contact information in your phplist.opensourceecology.org account
 * Send you emails from phplist.opensourceecology.org
 * Track your interactions with these emails for marketing purposes

If this is not correct, or you do not agree, simply take no action and delete this message.

1. 1. 1. When I signed up, I got this back

Open Source Ecology Newsletters - Confirmation Required

Please confirm your email. You have been subscribed to the following
newsletters:

 * OSEmail

Please click the following link to confirm it's really you:

http://phplist.opensourceecology.org/lists/?p=confirm&uid=e018a9647ef71a498f4683adf5c10394

In order to provide you with this service we'll need to

* Transfer your contact information to phplist.opensourceecology.org
* Store your contact information in your phplist.opensourceecology.org
account
* Send you emails from phplist.opensourceecology.org
* Track your interactions with these emails for marketing purposes

If this is not correct, or you do not agree, simply take no action and
delete this message.

1. 1. 1. But I think that's sufficient. I'll send it to Marcin for approval.

Tue Feb 05, 2019

1. Marcin responded to my email about the updated wordpress post about phplist
   1. he asked if I could make the link to the privacy policy open it in a new window, which I totally agree with. I did this by changing the attribute #4 = "I agree to the OSE Privacy Policy." anchor link to include the attribute "target" with value "_blank" in the phplist wui. I also changed it in the html within the wp post

I agree to the OSE <a href='https://wiki.opensourceecology.org/wiki/Open_Source_Ecology:Privacy_policy' target='_blank'>Privacy Policy</a>.

1. 1. Marcin asked me to change the response after signup to include a period in the sentence "Thanks, you have been added to our newsletter."
      1. he also noted that the javascript includes a variable named "successMessage" with a distinct message. Indeed, the text that's displayed comes as a response to the ajax query to the phplist server, and this response is configurable from the phplist wui at Config -> Settings -> "Text to display when subscription with an AJAX request was successful". I changed it to:

<h3>Thanks, you have been added to our newsletter.</h3><p>You will receive an email to confirm your subscription. Please click the link in the email to confirm.</p>

1. 1. 1. that actually didn't work. it looks like that's a default setting, but there's also per-subscribe-list settings for this response. I changed this via Config -> Subscribe Pages -> Edit (for id=3, which matches the URL used in the html of the post) -> "Text to display when subscription with an AJAX request was successful". I changed this to:

<h3>Thanks, you have been added to our newsletter.</h3><p>You will receive an email to confirm your subscription. Please click the link in the email to confirm.</p>

1. 1. Marcin provided a description of the Design Sprints newsletter as well, which I added to the subscribe list's config in phplist
2. I updated the wiki documentation's example of an ajax form with the changes we made https://wiki.opensourceecology.org/wiki/Phplist#Example_Form_for_OSEmail

1. ...

1. The phplist team asked me to cherry pick a git commit for a PR I submitted, but I couldn't get it to work; I followed-up https://github.com/phpList/phplist3/pull/445#issuecomment-460923780

Sat Jan 26, 2019

1. I logged into backblaze b2 to ensure that my nightly backups from hetnzer2 are still coming in after I've deprecated the dreamhost backup script & replaced it with the backblaze one
2. I see that there's a file named "daily_hetzner2_20190126_072001.tar.gpg (started large file)" which I guess is what it looks like when a backup is _currently_ being uploaded.
3. I also see that there's a file from the 25th, yesterday--and the days leading up to it. My change was last week, so this looks good.
4. The whole 'ose-server-backups' bucket is 190.5 GB with 16 files
5. The billing section shows that we paid our $0.78 fee for the first month, and that it's estimating a $0.28 fee for the second month. pretty fucking good.
6. I also confirmed that the backup log file (/var/log/backups/backup.log) contains my new comments, which makes the `time` outputs much more useful. And the --noProgress argument has worked wonderfully--the log file is now showing a size of 4kb in du.

1. ...

1. Marcin drafted a new post to our main ose wordpress site about or transition to phplist. We want to publish this before issuing the repermission campaign.
2. I edited the post to add the ajax form in the post, so that new users just learning about the OSEmail newsletter could signup with phplist
   1. I hit some modsecurity false-positives
      1. 958052 xss
      2. 958407 xss
      3. 973305 xss
      4. 973307 xss
      5. 973307 xss
   2. even though I'm editing the post in the "text" tab (not "visual"), wordpress is still adding invisible "

      " tags into thet post. For example, it does it in the middle of a javascript function, which creates syntax errors and compeletly breaks the ajax form

      1. I went to my user profile & checked the box "DIsable the visual editorwhen editing"
      2. I removed my ajax form block. saved. added it again. saved. It's still injecting these paragraph tags!

...
</noscript></p>
<p><script type="text/javascript">
function checkform() {</p>
<p>	// first, clear the response div from the previous attempts results
	jQuery("#result").empty();</p>
...

1. 1. 1. I found some documentation on this behaviour here https://codex.wordpress.org/Function_Reference/wpautop#Disabling_the_filter
      2. We probably don't want to wholesale disable it for the entire blog, so the best option is probably to use this simple plugin https://wordpress.org/plugins/wpautop-control/
      3. I installed & activated this plugin with wp-cli for the osemain

[maltfield@hetzner2 htdocs]$ sudo -u wp -i wp --path=/var/www/html/www.opensourceecology.org/htdocs plugin install --activate wpautop-control
...

1. 1. 1. # To enable it, I had to go to edit the post and enable the "Custom Fields" checkbox under "Screen Options" at the top. Then I scrolled down to the "Custom Fields" -> "Add New Custom Field:" section, clicked "Enter new", and typed "wpautop" for the Name and "false" for the Value.
      2. When I reloaded the post's preview, the paragraphs were all running together. Success! I manually added the necessary

         tags around Marcin's paragraphs. I saved, refreshed the preview, and confirmed that it looks good

      3. I replaced the REGISTRATION LINK text with our ajax form code block, saved, refreshed the preview, and now the form appears!
   2. unfortunately, the form is broken. clicking the "SUBSCRIBE" button (which, interestingly, looks different as it's inhereted the css from our osemain website) produces an error that the checkform() function is not defined

ReferenceError: checkform is not defined[Learn More]

1. 1. 1. it appears to be unhappy about a syntax error in the middle of a comment. It looks like my multi-line comment tag is the html syntax, but I should really use the JS syntax. I changed it
   2. now it works better. I confirmed when I don't click the box "I agree to the OSE Privcacy Policy" that it pops-up with the error as expected. But when I do check the box, I get a pop-up "Sorry, we were unable to process your subscription." and the JS console shows that it's because of a Cross-Origin Request being blocked. Right, I need to add both www.opensourceecology.org _and_ microfactory.opensourceecology.org into the whitelist of phplist.opensourceecology.org's headers in the phplist config file

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://phplist.opensourceecology.org/lists/index.php?p=asubscribe&id=3. (Reason: CORS header ‘Access-Control-Allow-Origin’ does not match ‘https://microfactory.opensourceecology.org’).

1. 1. 1. well shit, it looks like the fucking Access-Control-Allow-Origin header only supports a single domain! The w3c recommends dynamically responding with the single approved domain using logic to determine who has requested the content and if they're in the whitelist. That sucks! https://www.w3.org/TR/cors/#resource-implementation
      2. fortunately, we can do this in nginx--which would occur before the varnish cache https://stackoverflow.com/questions/1653308/access-control-allow-origin-multiple-origin-domains/12414239#12414239
      3. I commented-out my line in the phplist config.php setting ACCESS_CONTROL_ALLOW_ORIGIN. This reset it back to "http://phplist.opensourceecology.org"
      4. I spent much time trying to get the nginx config to set the ACCESS_CONTROL_ALLOW_ORIGN header, but it never changed from "http://phplist.opensourceecology.org". Indeed, it appears that nginx won't set headers when using proxy_pass https://stackoverflow.com/questions/14501047/how-to-add-a-response-header-on-nginx-when-using-proxy-pass
      5. hmm, actually, for some reason the nginx config file that's applying for phplist is 'awstats.opensourceecology.org'. No wonder no changes I make are applying..
      6. doh! the filename was wong; it doesn't end in .conf!

root@hetzner2 conf.d]# date
Sat Jan 26 12:26:26 UTC 2019
[root@hetzner2 conf.d]# pwd
/etc/nginx/conf.d
[root@hetzner2 conf.d]# ls -1 phplist*
phplist.opensourceecology.org
phplist.opensourceecology.org.4443.disabled
[root@hetzner2 conf.d]# ls -1 awstats*
awstats.openbuildinginstitute.org.conf
awstats.opensourceecology.org.conf
[root@hetzner2 conf.d]# 

1. 1. 1. I fixed this with the following nginx config

[root@hetzner2 ~]# grep -A14 'location /' /etc/nginx/conf.d/phplist.opensourceecology.org.conf 

  location / {
	proxy_pass http://127.0.0.1:6081;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Forwarded-Proto https;
	proxy_set_header X-Forwarded-Port 443;
	proxy_set_header Host $host;

	# handle cors whitelist for ajax subscription to phplist
	proxy_hide_header Access-Control-Allow-Origin;
	if ( $http_origin ~ "^https://(www.opensourceecology.org|microfactory.opensourceecology.org)$" ) {
	  add_header Access-Control-Allow-Origin $http_origin;
	}

  }

1. 1. 1. I also updated the phplist config with a comment pointing out that this logic is now in nginx, not php

[root@hetzner2 ~]# grep -B8 "define('ACCESS_CONTROL_ALLOW_ORIGIN" /var/www/html/phplist.opensourceecology.org/config.php  

// allow AJAX queries to add subscribers to our db from other domains
// Note: The ACCESS_CONTROL_ALLOW_ORIGIN header does not support multiple
//       domains, so we instead have to maintain a whitelist logically and
//       dynamically return the relevant domain iff it's in the whitelist.
//       Therefore, we actually override this phplist ACCESS_CONTROL_ALLOW_ORIGIN
//       header in our nginx config. See the relevant nginx config file:
//         * /etc/nginx/conf.d/phplist.opensourceecology.org.conf
#define('ACCESS_CONTROL_ALLOW_ORIGIN', "https://www.opensourceecology.org" );
[root@hetzner2 ~]# 

1. It works! I made some further changes to it so that it looks good in the osemain wordpress post, and finally documented the final ajax form block on the wiki https://wiki.opensourceecology.org/wiki/Phplist#Ajax_Form
2. I also documented the CORS stuff above https://wiki.opensourceecology.org/wiki/Phplist#Cross-Origin_Resource_Sharing
3. I sent Marcin an email asking for review of the updated post & for a description of the Design Sprints list
4. I updated the description of the OSEmal list with what Marcin provided in the blog post

OSEmail is our main OSE Newsletter featuring news updates, workshop announcement, progress reports, and other noteworthy items. OSEmail comes out a few times per year at monthly or longer intervals. Anyone can sign up to receive our free newsletter. You can see more information at https://wiki.opensourceecology.org/wiki/OSEmail

1. 1. This is now reflected in our main subscription signup page on phplist's site (the one that we'll link to for people who want to subscribe to the Design Sprints list) https://phplist.opensourceecology.org/lists/?p=subscribe&id=1

Wed Jan 16, 2019

1. I logged into the server and found that the backup was currently uploading today's nightly backup to backblaze b2
2. The logs are being hella spammed with the upload progress bar (the backup log containing just today's backup was already 14M), so I updated the backup.sh script to use the --noProgress argument.

[root@hetzner2 ~]# du -sh /var/log/backups/*
6.2M    /var/log/backups/backup.log
48K     /var/log/backups/backup.log-20181206.gz
96K     /var/log/backups/backup.log-20181207.gz
48K     /var/log/backups/backup.log-20181209.gz
48K     /var/log/backups/backup.log-20181210.gz
96K     /var/log/backups/backup.log-20181211.gz
348K    /var/log/backups/backup.log-20181213.gz
1012K   /var/log/backups/backup.log-20181215.gz
340K    /var/log/backups/backup.log-20181216.gz
356K    /var/log/backups/backup.log-20181218.gz
748K    /var/log/backups/backup.log-20181219.gz
364K    /var/log/backups/backup.log-20181221.gz
372K    /var/log/backups/backup.log-20181222.gz
716K    /var/log/backups/backup.log-20181223.gz
712K    /var/log/backups/backup.log-20181225.gz
352K    /var/log/backups/backup.log-20181226.gz
392K    /var/log/backups/backup.log-20181227.gz
352K    /var/log/backups/backup.log-20181228.gz
352K    /var/log/backups/backup.log-20181230.gz
356K    /var/log/backups/backup.log-20181231.gz
360K    /var/log/backups/backup.log-20190101.gz
360K    /var/log/backups/backup.log-20190102.gz
700K    /var/log/backups/backup.log-20190103.gz
352K    /var/log/backups/backup.log-20190105.gz
352K    /var/log/backups/backup.log-20190106.gz
364K    /var/log/backups/backup.log-20190107.gz
388K    /var/log/backups/backup.log-20190108.gz
408K    /var/log/backups/backup.log-20190109.gz
360K    /var/log/backups/backup.log-20190110.gz
488K    /var/log/backups/backup.log-20190111.gz
352K    /var/log/backups/backup.log-20190113.gz
384K    /var/log/backups/backup.log-20190114.gz
14M     /var/log/backups/backup.log-20190115
[root@hetzner2 ~]# 
79M     .
[root@hetzner2 ~]# 

1. I also added many INFO echos to the script to make the logs more useful
2. I ssh'd into our dreamhost server. Our ose_marcin user's home directly is currently using 304G of disk space. Hetzner1 backups are at 13G & hetzner2 backups are at 288G

hancock% pwd
/home/marcin_ose
hancock% date
Wed Jan 16 02:04:22 PST 2019
hancock% du -sh .
304G	.
hancock% du -sh hetzner1
13G	hetzner1
hancock% du -sh hetzner2
288G	hetzner2
hancock% 

1. I confirmed that the backup from today (20190116) was not present on the dreamhost server. success!

hancock% du -sh hetzner1/*
12G	hetzner1/20180501-052002
259M	hetzner1/20180502-052001
464M	hetzner1/20180602-052001
464M	hetzner1/20180702-052001
hancock% du -sh hetzner2/*
15G	hetzner2/20180501-072001
15G	hetzner2/20180601_072001
15G	hetzner2/20180701_072001
16G	hetzner2/20180801_072001
17G	hetzner2/20180901_072001
17G	hetzner2/20181001_072001
17G	hetzner2/20181101_072001
17G	hetzner2/20181202_072001
33G	hetzner2/20190101_072001
33G	hetzner2/20190112_072001
33G	hetzner2/20190113_072001
33G	hetzner2/20190114_072001
33G	hetzner2/20190115_072001
hancock% 

Tue Jan 15, 2019

1. I renamed the /root/backups/backup.sh script to be 'backup.old.20180115.sh' and I renamed backup2.sh to be 'backup.sh'
2. I also renamed /etc/cron.d/backup_to_dreamhost to be 'backup_to_backblaze'
3. The above has the effect of disabling backups being sent to dreamhost. All backups will now only go to backblaze b2 going forward.

Thr Jan 10, 2019

1. checked backblaze wui
   1. total size of bucket is currently 157.7G
   2. I confirmed that th yearly backup file exists at 17.5G = yearly_hetzner2_20190101_111520.tar.gpg
   3. I checked our billing; in all of 2018 it was $0.02. In 2019, it's $0.63 so far in 2019. (which is actually the billing period between Dec 16 through Jan 10).
      1. I should do a test downloading the 2019-01-01 file and document the process (with screenshots) on our wiki after 2019-01-17--since we get a free quota of downloads once per month, and I used some of it for testing/documenting the download process via the cli in this billing period (which cost $0.61 to do)

Wed Jan 09, 2019

1. marked time
2. still waiting on Marcin to unblock me for phplist