Website mijn.makerspaceleiden.nl setup: verschil tussen versies
(→Setup Signal Bridge) |
(→experimental update) |
||
Regel 229: | Regel 229: | ||
Integration via their newly fangled "Sympa" web service. | Integration via their newly fangled "Sympa" web service. | ||
− | + | ==== Security/Privacy consideration ==== | |
+ | |||
+ | Our mailing list provider is a processed of our data; and distributes the email to our members. This is the basis for allowing this party access to the email addresses or our users. We can limit the information to exactly that. | ||
+ | |||
+ | Additional measures: | ||
+ | |||
+ | # Gather the /just/ needed data in a single purpose view | ||
+ | # Specific user that is restricted to just that view and has only select/view; no write. | ||
+ | # Strong password & secure transfer of the password to the third party | ||
+ | # Require the use of SSL/TLS for this user in mysql | ||
+ | # Lock down of this user to a specific IP in mysql | ||
+ | # Use of IP tables to lock down the connection to just the | ||
+ | # Move to a non-common port | ||
+ | |||
+ | Still open: | ||
+ | # Does the API still need the (full) name - as mailman currently does - for use in the From: ? Or can we further limit this. | ||
+ | # Can we use X.509 client certs for authentication. | ||
Create a mysql user specific to sympa; $SYMPA_IP is the IP address of their gateway (currently just one) and $secret is a strong password: | Create a mysql user specific to sympa; $SYMPA_IP is the IP address of their gateway (currently just one) and $secret is a strong password: |
Versie van 12 nov 2022 om 13:46
Inhoud
On a demo/local laptop
git clone https://github.com/MakerSpaceLeiden/makerspaceleiden-crm.git
cd makerspaceleiden-crm sh loaddemo.sh
then visit <a href="http://localhost:8000/">http://localhost:8000/</a>.
(edit two things in requirements.txt. First, the latest version of Django is 4.1.2 This version causes errors. So take an older version of Django. Second thing is import python-dateutil).
So ```requirements.txt``` starts with:
Django==3.2.10 python-dateutil pytz wheel django-import-export etc...
Note that going to version 4 is fairly trivial with ```django-upgrade``` - but requires a fix in one dependency (https://github.com/linevych/django-search-admin-autocomplete/issues/15). Perhaps we should remove that dependency - it is only used by the trustees really.
Prepare for restarts when editing code with:
source venv/bin/activate
and then do :
python3 manage.py runserver
as needed (most changes will be picked up automatically).
On the production server -- how to update
Check that you are in the 'crmadmin' group. Then:
cd /usr/local/makerspaceleiden-crm git pull <right version and branch>
Activate the right vertual environment (rebuild with python3 -menv venv):
source venv/bin/activate
If needed - make a backup of the database with either
python3 manage.py dumpdata
or (passowrd in makerspaceleiden/my.cnf):
mysqldump -u mslcrmuser -p mslcrm
Check for any new static stuff & updates:
pip3 install -r requirements.txt
Next migrate the database if needed:
python3 manage.py makemigrations python3 manage.py migrate
If the asset changes - then also run the collect static:
python3 manage.py collectstatic --dry-run --noinput
examine the output and then either rerun the command or manually adjust.
And finally restart the webserver:
sudo apachectl restart
NOTE: var/media contains the images.
So while you can actually wack & redo the entire setup - you will need to preserve var/media. (Perhaps we should move var/media to /var/media - fully outside the tree - as it is also the only thing that 'www-data' can write to.
In production - setup from zero
Note: pretty much everything below is stock/totally-standard django/python Standard Operating Procedure (except for the chmod/chgrp on the var/media upload).
Make sure the baseline tools are present:
sudo apt-get install python3 libmysqlclient default-libmysqlclient-dev msmtp-mta apache2 libapache2-mod-uwsgi
The default-libmysqlclient-dev module is needed as pip3 wants mysqlconfig - which is not in the baseline libmysqlclient.
Initial checkout of code:
cd /usr/local git clone https://github.com/dirkx/makerspaceleiden-crm.git
Make evertyin group owned (crmadmin) and add that group to the accounts of those that need to maintain it.
Create random seed
openssl rand 128 > /etc/crm_secret_key.txt chmod 640 /etc/crm_secret_key.txt chgrp www-data /etc/crm_secret_key.txt
Allow storing of uploads in media by the suid that the webserver runs as:
mkdir -p var/media chown www-data var/media
Allow server to rotate/recreate logfiles on the fly
mkdir /var/log/crm chown www-data:crmadmin /var/log/crm chmod 770 /var/log/crm
Prepare env for python and pull in the various dependencies.
python3 -mvenv venv source ./venv/bin/activate pip3 install -r requirements
Set up the framework
cd makerspaceleiden ln -s prod.py local.py cat > makerspaceledien/my.cnf <<EOM [client] database = mslcrm user = mslcrmuser password = XXXX-passowrd-XXX default-character-set = utf8 EOM
Create database & user
mysql (suply database admin arguments as and when needed) create database mslcrm; create user 'mslcrmuser'@'localhost' identified by 'XXXX-passowrd-XXX'; grant all priveleges on mslcrm.* to 'mslcrmuser'@'localhost'; flush priveleges;
Check for issues, init and build database & site
python3 manage.py check --deploy python3 manage.py makemigrations python3 manage.py migrate python3 manage.py collectstatic
Check that email works:
python3 manage.py sendtestemail your@email.address.com
Create temp super user so you can log into complete the setup
python3 manage.py createsuperuser
Create apache config
cat > /etc/apache2/sites-available/crm.conf <<EOM WSGIScriptAlias /crm /usr/local/makerspaceleiden-crm/makerspaceleiden/wsgi.py process-group=crm WSGIDaemonProcess crm python-home=/usr/local/makerspaceleiden-crm/venv python-path=/usr/local/makerspaceleiden-crm/ WSGIProcessGroup crm Alias /crm-static/ /usr/local/makerspaceleiden-crm/static/ Alias /media/ /usr/local/makerspaceleiden-crm/var/media/ <Directory /usr/local/makerspaceleiden-crm/> <Files wsgi.py> Require all granted </Files> </Directory> <Directory /usr/local/makerspaceleiden-crm/static> Options None order deny,allow allow from all Require all granted </Directory> <Directory /usr/local/makerspaceleiden-crm/var/media> Options None order deny,allow allow from all Require all granted </Directory> EOM
Activate this setup
ln -s /etc/apache2/sites-available/crm.conf /etc/apache2/sites-active/crm.conf
Start server and keep an eye on the log:
apacheclt configtest apachectl restart tail -F /var/log/apache2/error.log
Now go to https://makerspaceleiden.nl/crm and create the initial members/structure. Assign the trustees the super user permission; then delete the temporary admin you made.
Setup Signal Bridge
Requires openjdk, java-dbus-bin and dbussy. Binary package from Shttps://github.com/AsamK/signal-cli/; installed in /home/signal-cli (post install 600/700 locked down).
Post install - as user signal-cli - activation is required:
$ signal-cli -u +317... register -voice $ signal-cli -u +317... verify code from call
which will create the right structure in ~/.local.
Source is required for the systemd files - installed with https://github.com/AsamK/signal-cli/wiki/DBus-service instructions.
Logfile retention
Configured in 'prod' settings - few MBs/days of logs is kept & then rotated out/deleted using the standard RotatingLogging handler.
See https://github.com/dirkx/makerspaceleiden-crm/commit/de5de35dd22a317bb93568dadc1b8737a819023a for details.
Cleanup and reminder crons
To be configured/written
Backup
This is done by the /etc/duplicity/run.sh setup (along with everything else). It does a dump of the MySQL database and captures the whole directory -- which includes var/media -- the uploads.
See the section backup on the general Server setup page.
Mailing list integration
Mailing lists are ran by a third party - we currently use a hack to manage these by faking browser access to their form API.
experimental update
Integration via their newly fangled "Sympa" web service.
Security/Privacy consideration
Our mailing list provider is a processed of our data; and distributes the email to our members. This is the basis for allowing this party access to the email addresses or our users. We can limit the information to exactly that.
Additional measures:
- Gather the /just/ needed data in a single purpose view
- Specific user that is restricted to just that view and has only select/view; no write.
- Strong password & secure transfer of the password to the third party
- Require the use of SSL/TLS for this user in mysql
- Lock down of this user to a specific IP in mysql
- Use of IP tables to lock down the connection to just the
- Move to a non-common port
Still open:
- Does the API still need the (full) name - as mailman currently does - for use in the From: ? Or can we further limit this.
- Can we use X.509 client certs for authentication.
Create a mysql user specific to sympa; $SYMPA_IP is the IP address of their gateway (currently just one) and $secret is a strong password:
create user mlist@$SYMPA_IP identified by '$secret' REQUIRE SSL;
Then create a view with just the data needed:
create view mailinglists_api as select distinct concat(first_name,' ',last_name) "name",name "list", email from members_user,mailinglists_subscription,mailinglists_mailinglist where mailinglists_subscription.member_id = members_user.id and mailinglists_subscription.mailinglist_id = mailinglists_mailinglist.id and mailinglists_subscription.active =1 order by email with read only;
And grant this integration user access to just that view:
GRANT SELECT, SHOW VIEW ON mslcrm.mailinglists_api TO mlist@ $SYMPA_IP;
Then provide the mailinglist provider with the following file/data:
db_type mysql db_port $PORT db_host $OUR_IP db_user mlist db_passwd $secret db_name mslcrm sql_query select distinct name,email from mailinglists_api where list = 'spacelog';
Configure SSL on the right port
[mysqld] ssl_ca=ca.pem ssl_cert=server-cert.pem ssl_key=server-key.pem require_secure_transport=ON port=$PORT
And allow acccess:
iptables -A INPUT -p tcp -s $SYMPA_IP --sport 1024:65535 -d $OUR_IP --dport $PORT \ -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A OUTPUT -p tcp -s 2$OUR_IP --sport $PORT -d $SYMPA_IP --dport 1024:65535 \ -m state --state ESTABLISHED -j ACCEPT