Website mijn.makerspaceleiden.nl setup

Uit MakerSpace Leiden
Versie door MarkKlaauw (overleg | bijdragen) op 29 jul 2024 om 19:04 (MarkKlaauw heeft de pagina Mijn.makerspaceleiden.nl setup hernoemd naar Website mijn.makerspaceleiden.nl setup zonder een doorverwijzing achter te laten)
Naar navigatie springen Naar zoeken springen
De printervriendelijke versie wordt niet langer ondersteund en kan weergavefouten bevatten. Werk uw browserbladwijzers bij en gebruik de gewone afdrukfunctie van de browser.

On a demo/local laptop

git clone https://github.com/MakerSpaceLeiden/makerspaceleiden-crm.git

   cd makerspaceleiden-crm
   sh loaddemo.sh

then visit <a href="http://localhost:8000/">http://localhost:8000/</a>.


(edit two things in requirements.txt. First, the latest version of Django is 4.1.2 This version causes errors. So take an older version of Django. Second thing is import python-dateutil).

So ```requirements.txt``` starts with:

    Django==3.2.10
    python-dateutil
    pytz
    wheel
    django-import-export
    etc...

Note that going to version 4 is fairly trivial with ```django-upgrade``` - but requires a fix in one dependency (https://github.com/linevych/django-search-admin-autocomplete/issues/15). Perhaps we should remove that dependency - it is only used by the trustees really.

Prepare for restarts when editing code with:

source venv/bin/activate

and then do :

python3 manage.py runserver

as needed (most changes will be picked up automatically).

On the production server -- how to update

Check that you are in the 'crmadmin' group. Then:

 cd /usr/local/makerspaceleiden-crm
 git pull <right version and branch>

Activate the right vertual environment (rebuild with python3 -menv venv):

 source venv/bin/activate

If needed - make a backup of the database with either

 python3 manage.py dumpdata

or (passowrd in makerspaceleiden/my.cnf):

 mysqldump -u mslcrmuser  -p mslcrm

Update PIP if you get warnings first:

 python3.8 -m pip install --upgrade pip

Check for any new static stuff & updates:

 pip3 list --outdated
 pip3 install -r requirements.txt  --upgrade

Next migrate the database if needed:

 python3 manage.py makemigrations
 python3 manage.py migrate

If the asset changes - then also run the collect static:

 python3 manage.py collectstatic --dry-run --noinput

examine the output and then either rerun the command or manually adjust.

And finally restart the webserver:

  sudo apachectl restart

NOTE: var/media contains the images.

So while you can actually wack & redo the entire setup - you will need to preserve var/media. (Perhaps we should move var/media to /var/media - fully outside the tree - as it is also the only thing that 'www-data' can write to.

In production - setup from zero

Note: pretty much everything below is stock/totally-standard django/python Standard Operating Procedure (except for the chmod/chgrp on the var/media upload).

Make sure the baseline tools are present:

  sudo apt-get install python3 libmysqlclient  default-libmysqlclient-dev msmtp-mta apache2 libapache2-mod-uwsgi

The default-libmysqlclient-dev module is needed as pip3 wants mysqlconfig - which is not in the baseline libmysqlclient.

Initial checkout of code:

   cd /usr/local
   git clone https://github.com/dirkx/makerspaceleiden-crm.git

Make evertyin group owned (crmadmin) and add that group to the accounts of those that need to maintain it.

Create random seed

   openssl rand 128 > /etc/crm_secret_key.txt
   chmod 640 /etc/crm_secret_key.txt
   chgrp www-data /etc/crm_secret_key.txt
   

Allow storing of uploads in media by the suid that the webserver runs as:

   mkdir -p var/media
   chown www-data var/media
   

Allow server to rotate/recreate logfiles on the fly

   mkdir /var/log/crm
   chown www-data:crmadmin /var/log/crm
   chmod 770 /var/log/crm

Prepare env for python and pull in the various dependencies.

   python3 -mvenv venv
   source ./venv/bin/activate
   pip3 install -r requirements
   

Set up the framework

   cd makerspaceleiden
   ln -s prod.py local.py
   cat > makerspaceledien/my.cnf <<EOM
   [client]
   database = mslcrm
   user = mslcrmuser
   password = XXXX-passowrd-XXX
   default-character-set = utf8
   EOM
   

Create database & user

   mysql (suply database admin arguments as and when needed)
   create database mslcrm;
   create user 'mslcrmuser'@'localhost' identified by 'XXXX-passowrd-XXX';
   grant all priveleges on mslcrm.* to 'mslcrmuser'@'localhost';
   flush priveleges;
   

Check for issues, init and build database & site

   python3 manage.py check --deploy
   python3 manage.py makemigrations
   python3 manage.py migrate
   python3 manage.py collectstatic
   

Check that email works:

  python3 manage.py sendtestemail your@email.address.com

Create temp super user so you can log into complete the setup

   python3 manage.py createsuperuser
   

Create apache config

   cat > /etc/apache2/sites-available/crm.conf <<EOM
   WSGIScriptAlias /crm /usr/local/makerspaceleiden-crm/makerspaceleiden/wsgi.py  process-group=crm
   
   WSGIDaemonProcess crm python-home=/usr/local/makerspaceleiden-crm/venv python-path=/usr/local/makerspaceleiden-crm/
   WSGIProcessGroup  crm 
   
   Alias /crm-static/ /usr/local/makerspaceleiden-crm/static/
   Alias /media/      /usr/local/makerspaceleiden-crm/var/media/
   
   <Directory /usr/local/makerspaceleiden-crm/>
     <Files wsgi.py>
        Require all granted
     </Files>
   </Directory>
   
   <Directory /usr/local/makerspaceleiden-crm/static>
           Options None
           order deny,allow
           allow from all
           Require all granted
   </Directory>
   
   <Directory /usr/local/makerspaceleiden-crm/var/media>
           Options None
           order deny,allow
           allow from all
           Require all granted
   </Directory>
   EOM

Activate this setup

   ln -s /etc/apache2/sites-available/crm.conf  /etc/apache2/sites-active/crm.conf 
   

Start server and keep an eye on the log:

   apacheclt configtest
   apachectl restart
   tail -F /var/log/apache2/error.log

Now go to https://mijn.makerspaceleiden.nl/ and create the initial members/structure. Assign the trustees the super user permission; then delete the temporary admin you made.

Setup Signal Bridge

Requires openjdk, java-dbus-bin and dbussy. Binary package from Shttps://github.com/AsamK/signal-cli/; installed in /home/signal-cli (post install 600/700 locked down).

Post install - as user signal-cli - activation is required:

  $ signal-cli -u +317... register -voice
  $ signal-cli -u +317... verify code from call

which will create the right structure in ~/.local.

Source is required for the systemd files - installed with https://github.com/AsamK/signal-cli/wiki/DBus-service instructions.

Logfile retention

Configured in 'prod' settings - few MBs/days of logs is kept & then rotated out/deleted using the standard RotatingLogging handler.

See https://github.com/dirkx/makerspaceleiden-crm/commit/de5de35dd22a317bb93568dadc1b8737a819023a for details.

Cleanup and reminder crons

The following entries need to be added to the crontab (either to /etc/cron, to cron.daily/monthly or to the `users' cron):

    SHELL=/bin/sh
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    MAILTO=noc@makerspaceleiden.nl
    MAILFROM="MSL Server (cron)"
    NULLMAILUSER_NAME="MSL Server (cron)"
    ....
    # General maintenance mails of mijn.makerspaceleiden.nl.
    #
    # m h d m d user  cmd
    1 1 * * * USER test -x /usr/local/makerspaceleiden-crm/cron-midnight.sh && /usr/local/makerspaceleiden-crm/cron-midnight.sh
    1 2 1 * * USER test -x /usr/local/makerspaceleiden-crm/cron-monthly.sh && /usr/local/makerspaceleiden-crm/cron-monthly.sh

With 'USER' a user with the right rights.

Backup

This is done by the /etc/duplicity/run.sh setup (along with everything else). It does a dump of the MySQL database and captures the whole directory -- which includes var/media -- the uploads.

See the section backup on the general Server setup page.

Mailing list integration

Mailing lists are ran by a third party - sympalists.net. The integration via their newly fangled "Sympa" web service -- See Sympa mail list setup for details.

Query to find duplicates in the subscriptions:

       select * from mailinglists_subscription where id not in (select max(id) from mailinglists_subscription group by mailinglist_id,member_id);