Website mijn.makerspaceleiden.nl setup

On a demo/local laptop

git clone https://github.com/MakerSpaceLeiden/makerspaceleiden-crm.git

   cd makerspaceleiden-crm
   sh loaddemo.sh

then visit <a href="http://localhost:8000/">http://localhost:8000/</a>.

(edit two things in requirements.txt. First, the latest version of Django is 4.1.2 This version causes errors. So take an older version of Django. Second thing is import python-dateutil).

So ```requirements.txt``` starts with:

    Django==3.2.10
    python-dateutil
    pytz
    wheel
    django-import-export
    etc...

Note that going to version 4 is fairly trivial with ```django-upgrade``` - but requires a fix in one dependency (https://github.com/linevych/django-search-admin-autocomplete/issues/15). Perhaps we should remove that dependency - it is only used by the trustees really.

Prepare for restarts when editing code with:

source venv/bin/activate

and then do :

python3 manage.py runserver

as needed (most changes will be picked up automatically).

On the production server -- how to update

Check that you are in the 'crmadmin' group. Then:

 cd /usr/local/makerspaceleiden-crm
 git pull <right version and branch>

Activate the right vertual environment (rebuild with python3 -menv venv):

 source venv/bin/activate

If needed - make a backup of the database with either

 python3 manage.py dumpdata

or (passowrd in makerspaceleiden/my.cnf):

 mysqldump -u mslcrmuser  -p mslcrm

Check for any new static stuff & updates:

 pip3 install -r requirements.txt 

Next migrate the database if needed:

 python3 manage.py makemigrations
 python3 manage.py migrate

If the asset changes - then also run the collect static:

 python3 manage.py collectstatic --dry-run --noinput

examine the output and then either rerun the command or manually adjust.

And finally restart the webserver:

  sudo apachectl restart

NOTE: var/media contains the images.

So while you can actually wack & redo the entire setup - you will need to preserve var/media. (Perhaps we should move var/media to /var/media - fully outside the tree - as it is also the only thing that 'www-data' can write to.

In production - setup from zero

Note: pretty much everything below is stock/totally-standard django/python Standard Operating Procedure (except for the chmod/chgrp on the var/media upload).

Make sure the baseline tools are present:

  sudo apt-get install python3 libmysqlclient  default-libmysqlclient-dev msmtp-mta apache2 libapache2-mod-uwsgi

The default-libmysqlclient-dev module is needed as pip3 wants mysqlconfig - which is not in the baseline libmysqlclient.

Initial checkout of code:

   cd /usr/local
   git clone https://github.com/dirkx/makerspaceleiden-crm.git

Make evertyin group owned (crmadmin) and add that group to the accounts of those that need to maintain it.

Create random seed

   openssl rand 128 > /etc/crm_secret_key.txt
   chmod 640 /etc/crm_secret_key.txt
   chgrp www-data /etc/crm_secret_key.txt
   

Allow storing of uploads in media by the suid that the webserver runs as:

   mkdir -p var/media
   chown www-data var/media
   

Allow server to rotate/recreate logfiles on the fly

   mkdir /var/log/crm
   chown www-data:crmadmin /var/log/crm
   chmod 770 /var/log/crm

Prepare env for python and pull in the various dependencies.

   python3 -mvenv venv
   source ./venv/bin/activate
   pip3 install -r requirements
   

Set up the framework

   cd makerspaceleiden
   ln -s prod.py local.py
   cat > makerspaceledien/my.cnf <<EOM
   [client]
   database = mslcrm
   user = mslcrmuser
   password = XXXX-passowrd-XXX
   default-character-set = utf8
   EOM
   

Create database & user

   mysql (suply database admin arguments as and when needed)
   create database mslcrm;
   create user 'mslcrmuser'@'localhost' identified by 'XXXX-passowrd-XXX';
   grant all priveleges on mslcrm.* to 'mslcrmuser'@'localhost';
   flush priveleges;
   

Check for issues, init and build database & site

   python3 manage.py check --deploy
   python3 manage.py makemigrations
   python3 manage.py migrate
   python3 manage.py collectstatic
   

Check that email works:

  python3 manage.py sendtestemail your@email.address.com

Create temp super user so you can log into complete the setup

   python3 manage.py createsuperuser
   

Create apache config

   cat > /etc/apache2/sites-available/crm.conf <<EOM
   WSGIScriptAlias /crm /usr/local/makerspaceleiden-crm/makerspaceleiden/wsgi.py  process-group=crm
   
   WSGIDaemonProcess crm python-home=/usr/local/makerspaceleiden-crm/venv python-path=/usr/local/makerspaceleiden-crm/
   WSGIProcessGroup  crm 
   
   Alias /crm-static/ /usr/local/makerspaceleiden-crm/static/
   Alias /media/      /usr/local/makerspaceleiden-crm/var/media/
   
   <Directory /usr/local/makerspaceleiden-crm/>
     <Files wsgi.py>
        Require all granted
     </Files>
   </Directory>
   
   <Directory /usr/local/makerspaceleiden-crm/static>
           Options None
           order deny,allow
           allow from all
           Require all granted
   </Directory>
   
   <Directory /usr/local/makerspaceleiden-crm/var/media>
           Options None
           order deny,allow
           allow from all
           Require all granted
   </Directory>
   EOM

Activate this setup

   ln -s /etc/apache2/sites-available/crm.conf  /etc/apache2/sites-active/crm.conf 
   

Start server and keep an eye on the log:

   apacheclt configtest
   apachectl restart
   tail -F /var/log/apache2/error.log

Now go to https://makerspaceleiden.nl/crm and create the initial members/structure. Assign the trustees the super user permission; then delete the temporary admin you made.

Setup Signal Bridge

Requires openjdk, java-dbus-bin and dbussy. Binary package from Shttps://github.com/AsamK/signal-cli/; installed in /home/signal-cli (post install 600/700 locked down).

Post install - as user signal-cli - activation is required:

  $ signal-cli -u +317... register -voice
  $ signal-cli -u +317... verify 

which will create the right structure in ~/.local.

Source is required for the systemd files - installed with https://github.com/AsamK/signal-cli/wiki/DBus-service instructions.

Logfile retention

Configured in 'prod' settings - few MBs/days of logs is kept & then rotated out/deleted using the standard RotatingLogging handler.

See https://github.com/dirkx/makerspaceleiden-crm/commit/de5de35dd22a317bb93568dadc1b8737a819023a for details.

Cleanup and reminder crons

To be configured/written

Backup

This is done by the /etc/duplicity/run.sh setup (along with everything else). It does a dump of the MySQL database and captures the whole directory -- which includes var/media -- the uploads.

See the section backup on the general Server setup page.

Mailing list integration

Mailing lists are ran by a third party - we currently use a hack to manage these by faking browser access to their form API.

experimental update

Integration via their newly fangled "Sympa" web service.

Setup

Create a mysql user specific to sympa; $SYMPA_IP is the IP address of their gateway (currently just one) and $secret is a strong password:

    create user mlist@$SYMPA_IP identified by '$secret' REQUIRE SSL;

Then create a view with just the data needed:

    create view mailinglists_api as
       select distinct concat(first_name,' ',last_name) "name",name "list", email
               from members_user,mailinglists_subscription,mailinglists_mailinglist
       where   mailinglists_subscription.member_id = members_user.id and
               mailinglists_subscription.mailinglist_id = mailinglists_mailinglist.id and
               mailinglists_subscription.active =1
       order by email
       with read only;

And grant this integration user access to just that view:

    GRANT SELECT, SHOW VIEW ON mslcrm.mailinglists_api TO mlist@ $SYMPA_IP;

Then provide the mailinglist provider with the following file/data:

   db_type mysql
   db_port $PORT 
   db_host $OUR_IP
   db_user mlist
   db_passwd $secret
   db_name mslcrm
   sql_query select distinct name,email from mailinglists_api where list = 'spacelog';

Configure SSL on the right port

   [mysqld]
   ssl_ca=ca.pem
   ssl_cert=server-cert.pem
   ssl_key=server-key.pem
   require_secure_transport=ON
   port=$PORT

And allow acccess:

   iptables -A INPUT  -p tcp -s $SYMPA_IP --sport 1024:65535 -d $OUR_IP --dport $PORT \
          -m state --state NEW,ESTABLISHED -j ACCEPT
   iptables -A OUTPUT -p tcp -s 2$OUR_IP --sport $PORT -d $SYMPA_IP --dport 1024:65535 \
          -m state --state ESTABLISHED -j ACCEPT