Phplist

Introduction

Phplist is open source email list software. When OSE adopted it in 2018 for the OSE Newsletter, it was determined to be the most feature-rich FLOSS alternative to the gold-standard paid alternative, MailChimp.

Transition Announcement

Title: Moving to Libre and Open Source Email List Software - phpList

Hello Friends of OSE,

Due to the recent European privacy regulations - the GDPR (link to an explainer on GDPR) - to keep receiving updates from OSE - you will need to resubscribe to our lists if you are on them. Or to start receiving updates - you can subscribe for the first time:

REGISTRATION LINK

With respect to our email lists - we just took another small step on our path to creating the open source economy. We are pleased to announce that we have installed the free, libre open source (FLOSS) email list software on OSE servers - phpList - https://www.phplist.org/. We are now transitioning all of our email lists into phpList. When we adopted phpList in 2018 for the OSE Newsletter, it was determined to be the most feature-rich FLOSS alternative to the gold-standard paid alternative, MailChimp. And this is a good time to get into comliance with GDPR.

We have several email lists. OSEmail is our main OSE Newsletter featuring news updates, workshop announcement, progress reports, and other noteworthy items. OSEmail comes out a few times per year at monthly or longer intervals. Anyone can sign up to receive our free newsletter. You can see more information at https://wiki.opensourceecology.org/wiki/OSEmail

We have another list for Design Sprints. Design Sprints are online virtual collaboration events where we engage in design and documentation work. Design events last from one to a few hours - typically on Friday or weekends - where we collaborate in real-time as a team. We use online editable documents and the OSE wiki to coordinate development work. Anyone with technical skills can participate, and we host several design sprints per year as needed.

What kind of updates do we have in store? For one, I am taking a 'sabbatical' for one year to write a book. In 2008, we formulated the Global Village Construction Set (GVCS) and began blogging regularly. Then in 2011, I got on the world stage with my GVCS TED Talk (link). Since then, I have just not had the time to make proper updates. There are too many learnings, too much to tell. And I don't feel good about that, as the world needs to hear more about the learnings - which are unique. At about the decade mark - I decided to write a book to produce a concise manual on all of our learnings - and how to take the Global Village Construction Set to the next stage. Still, we are far from the kind of impact that Linux has done for software. Why? That is the central question I will attempt to answer. I believe that taking OSE to the next level requires a thorough analysis of all that happened to date. It's also important for me to summarize the main points of knowledge gained by civilization to date, because creating a new civilization from scratch requires such awareness. This helps put our work into perspective - as we are doing nothing new - just integrating and applying existing knowledge.

So if you would like to join our mailing lists to keep up with our progress - please do so by subscribing to the list below. For reasons of GDPR internet privacy regulations, we require that everyone on our lists resubscribe so that OSE is in compliance with the regulations - so if you would like to continue receiving emails from us - you need to resubscribe.

Usage Notes

https://phplist.opensourceecology.org/lists/ and admin at /lists/admin

1. Next - Ajax form for embedding email collection, privacy policy, import lists

1. GDPR - what we store (time stamp, check box, email), how long, includes privacy policy, we have to tell people what we store, why, do we share it, and accepts delete requests, and update) - ETA - mid-Nov new emails. Then repermission campaign.

1. Simple sending of email with links. Period. 2000 emails per month
2. Select multiple lists, and distinct attributes for each list
3. Alex's website - violates GDPR
4. Subscription for email list for website -
5. Automated processes for later
6. Email is collected and get a checkmark. Repermission campaign - click a link before they can receive the info.
7. Privacy policy of Mediawiki - not GDPR compliant
8. We store: email, timestamp, that they checked they saw our privacy policy
9. Workshop Email List - update, rectify, delete, etc. We need to be able to remove a user.

Installation

Installation for phplist is relatively trivial. Their documentation leaves much to be desired, but it's fairly straight-forward.

libsodium

Note that, while not strictly a requirement on their website, phplist does effectively require libsodium and the corresponding pecl libsodium php extension to be installed on our OSE Server.

Without it, the random_compat library will throw an exception (which, by default, phplist will suppress from being written to the error logs):

[Thu Aug 23 00:06:29.560157 2018] [:error] [pid 17617] [client 127.0.0.1:51262] PHP Fatal error:  Uncaught exception 'Exception' with message 'There is no suitable CSPRNG installed on your system' in /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/inc/random_compat/random.php:204\nStack trace:\n#0 /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/defaultconfig.php(3): random_bytes(10)\n#1 /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/index.php(103): require_once('/var/www/html/p...')\n#2 {main}\n  thrown in /var/www/html/phplist.opensourceecology.org/public_html/lists/admin/inc/random_compat/random.php on line 204

This, in part, is due to the use of 'open_basedir' in our /etc/php.ini file to whitelist the directories in which php is permitted to execute. The official response from the maintainers (see issue #99 of the random_compat repo on github ) is to simply add '/dev/urandom' to 'open_basedir' in '/etc/php.ini'. That didn’t seem like the wisest option, so--instead--we just install libsodium. Indeed, libsodium is the preferred source of entropy for random_compat, anyway .

yum install php-pecl-libsodium
httpd -t && service httpd restart

For more information about this issue and its fix, see

• https://tech.michaelaltfield.net/2018/08/25/fix-phplist-500-error-due-to-random_compat/

Proper File/Directory Ownership & Permissions

This section will describe how the file permissions should be set on an OSE phplist site.

For the purposes of this documentation, let's assume:

1. vhost dir = /var/www/html/phplist.opensourceecology.org

Then the ideal permissions are:

1. Files containing passwords (ie: config.php) should be located outside the docroot with not-apache:apache-admins 0040
2. Files in the public_html/uploadimages dir should be apache:apache 0660
3. All other files in the vhost dir should be not-apache:apache 0040
4. All other directories in the vhost dir should be not-apache:apache 0050

This is achievable with the following idempotent commands:

vhostDir="/var/www/html/phplist.opensourceecology.org"

chown -R not-apache:apache "${vhostDir}"
find "${vhostDir}" -type d -exec chmod 0050 {} \;
find "${vhostDir}" -type f -exec chmod 0040 {} \;

chown not-apache:apache-admins "${vhostDir}/config.php"
chmod 0040 "${vhostDir}/config.php"

[ -d "${vhostDir}/public_html/uploadimages" ] || mkdir "${vhostDir}/public_html/uploadimages"
chown -R apache:apache "${vhostDir}/public_html/uploadimages"
find "${vhostDir}/public_html/uploadimages" -exec chmod 0660 {} \;
chmod 0770 "${vhostDir}/public_html/uploadimages"

Such that:

1. the 'not-apache' user is a new user that doesn't run any software (ie: a daemon such as a web server) and whose shell is "/sbin/nologin" and home is "/dev/null".
2. the apache user is in the apache-admins group
3. the apache user is in the apache group
4. any human users that need read-only access to the phplist vhost files for debugging purposes and/or write access to the 'public_html/uploadimages/' directory (ie: to upload large files that are too large to be handled by the web servers chain), then that user should be added to the 'apache' group
5. any human users that need read-only access to the phplist vhost files, including config files containing passwords (ie: config.php), should be added to the 'apache-admins' group
6. for anyone to make changes to any files in the docroot (other than 'public_html/uploadimages/'), they must be the root user. I think this is fair if they don't have the skills necessary to become root, they probably shouldn't modify the phplist core files anyway.

Why?

The following explains why the above permissions are ideal:

1. All of the files & directories that don't need write permissions should not have write permissions. That's every file in a phplist docroot except the folder "public_html/uploadimages/" and its subfiles/dirs.
2. World permissions (not-user && not-group) for all files & directories inside the docroot (and including the docroot dir itself!) should be set to 0 for all files & all directories.
3. Excluding 'public_html/uploadimages/', these files should also not be owned by the user that runs a webserver (in cent, that's the 'apache' user). For even if the file is set to '0400', but it's owned by the 'apache' user, the 'apache' user can ignore the permissions & write to it anyway. We don't want the apache user (which runs the apache process) to be able to modify files. If it could, then a compromised webserver could modify a php file and effectively do a remote code execution.
4. Excluding 'public_html/uploadimages/', all directories in the docroot (including the docroot dir itself!) should be owned by a group that contains the user that runs our webserver (in cent, that's the apache user). The permissions for this group must be not include write access for files or directories. For even if a file is set to '0040', but the containing directory is '0060', any user in the group that owns the directory can delete the existing file and replace it with a new file, effectively ignoring the read-only permission set for the file.

Configuration

This section will describe how to properly configure phplist on our server

Cross-Origin Resource Sharing

In order to allow external domains from POSTing data to our phplist web server, our phplist server must whitelist the external domain using the Access-Control-Allow-Origin header.

Phplist has built-in functionality for setting this header via the ACCESS_CONTROL_ALLOW_ORIGIN constant, but this header does not support defining multiple domains. This is an issue, for example, if we want to have an ajax form for registering users to our phplist newsletter from multiple distinct domains/websites (ie: www.opensourceecology.org and microfactory.opensourceecology.org). The solution recommended by the w3c is to "generate the Access-Control-Allow-Origin header dynamically" . We implement this logic in nginx, as it's best to do it before the vanish cache.

Note the relevant comments in the phplist config file

[root@hetzner2 ~]# grep -B8 "define('ACCESS_CONTROL_ALLOW_ORIGIN" /var/www/html/phplist.opensourceecology.org/config.php  

// allow AJAX queries to add subscribers to our db from other domains
// Note: The ACCESS_CONTROL_ALLOW_ORIGIN header does not support multiple
//       domains, so we instead have to maintain a whitelist logically and
//       dynamically return the relevant domain iff it's in the whitelist.
//       Therefore, we actually override this phplist ACCESS_CONTROL_ALLOW_ORIGIN
//       header in our nginx config. See the relevant nginx config file:
//         * /etc/nginx/conf.d/phplist.opensourceecology.org.conf
#define('ACCESS_CONTROL_ALLOW_ORIGIN', "https://www.opensourceecology.org" );
[root@hetzner2 ~]# 

And the actual logic in the nginx config file

[root@hetzner2 ~]# grep -A14 'location /' /etc/nginx/conf.d/phplist.opensourceecology.org.conf 

  location / {
    proxy_pass http://127.0.0.1:6081;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-Port 443;
    proxy_set_header Host $host;

    # handle cors whitelist for ajax subscription to phplist
    proxy_hide_header Access-Control-Allow-Origin;
    if ( $http_origin ~ "^https://(www.opensourceecology.org|microfactory.opensourceecology.org)$" ) {
      add_header Access-Control-Allow-Origin $http_origin;
    }

  }

Troubleshooting

This section will provide hints for troubleshooting the phplist software. For general troubleshooting tips, also checkout the relevant documentation page on phplist.org's website here:

• https://www.phplist.org/manual/ch046_troubleshooting-techniques.xhtml

Enable error logging

By default, phplist will suppress errors from being written to log files. To fix this, find calls to error_reporting(0) and change them to error_reporting(1) in the phplist source code. For example, '/lists/admin/index.php' and '/lists/admin/init.php'

See Also

• Email List
• OSEmail

Formatting

Address -

Open Source Ecology Global Headquarters Maysville, MO 64469 USA

Do not include street address at this time.

edit

Welcome Email

Welcome to OSEmail - the Open Source Ecology Newsletter

Please keep this message for later reference.

Your email address has been added to the following newsletter(s):

*newsletter
*OSEmail

To update your details and preferences please go to http://phplist.opensourceecology.org/lists/?p=preferences&uid=d56f8157f023a2dc4217da85c43f9c71. If you do not want to receive any more messages, please go to http://phplist.opensourceecology.org/lists/?p=unsubscribe&uid=d56f8157f023a2dc4217da85c43f9c71.

Thank you

Repermissioning

• See OSEmail Repermissioning Campaign.

The True Fans List

• Deprecated - [1]
• Going forward, add this data to phplist's (private) list #4 = True Fans, which now has 957 list subscribers. [2]