Website mijn.makerspaceleiden.nl setup: verschil tussen versies
Geen bewerkingssamenvatting |
adds systemd documentation, resolving lingering todo |
||
(83 tussenliggende versies door 7 gebruikers niet weergegeven) | |||
Regel 1: | Regel 1: | ||
[[Category:Servers network & websites]] | |||
The code is hosted on GitHub at: | |||
https://github.com/MakerSpaceLeiden/makerspaceleiden-crm | |||
[https://github.com/MakerSpaceLeiden/makerspaceleiden-crm?tab=readme-ov-file#installation-options Instructions] for getting up and running locally are available in the Read Me. | |||
== Initial Setup == | |||
Note: '''pretty much everything below is stock/totally-standard django/python Standard Operating Procedure (except for the chmod/chgrp on the var/media upload).''' | |||
== | === Prerequisites === | ||
Make sure the baseline tools are present: | |||
== | <syntaxhighlight lang="bash"> | ||
sudo apt-get install msmtp-mta apache2 libapache2-mod-uwsgi | |||
sudo snap install astral-uv --classic | |||
</syntaxhighlight> | |||
=== Code Setup === | |||
Initial checkout of code: | |||
<syntaxhighlight lang="bash"> | |||
cd /usr/local | cd /usr/local | ||
git clone https://github.com/ | git clone https://github.com/MakerSpaceLeiden/makerspaceleiden-crm.git | ||
</syntaxhighlight> | |||
Make everything group owned (crmadmin) and add that group to the accounts | |||
of those that need to maintain it. | |||
=== Security Configuration === | |||
Create random seed: | |||
<syntaxhighlight lang="bash"> | |||
openssl rand 128 > /etc/crm_secret_key.txt | openssl rand 128 > /etc/crm_secret_key.txt | ||
chmod 640 /etc/crm_secret_key.txt | chmod 640 /etc/crm_secret_key.txt | ||
chgrp www-data /etc/crm_secret_key.txt | chgrp www-data /etc/crm_secret_key.txt | ||
</syntaxhighlight> | |||
Allow storing of uploads in media by the suid that the webserver runs as: | |||
<syntaxhighlight lang="bash"> | |||
mkdir -p var/media | mkdir -p var/media | ||
chown www-data var/media | chown www-data var/media | ||
</syntaxhighlight> | |||
Allow server to rotate/recreate logfiles on the fly: | |||
<syntaxhighlight lang="bash"> | |||
mkdir /var/log/crm | |||
chown www-data:crmadmin /var/log/crm | |||
chmod 770 /var/log/crm | |||
</syntaxhighlight> | |||
=== Python Environment === | |||
Prepare environment for python and pull in the various dependencies: | |||
<syntaxhighlight lang="bash"> | |||
uv venv | |||
source ./.venv/bin/activate | |||
uv sync | |||
</syntaxhighlight> | |||
Set up the framework: | |||
<syntaxhighlight lang="bash"> | |||
cd makerspaceleiden | cd makerspaceleiden | ||
ln -s prod.py local.py | ln -s prod.py local.py | ||
Regel 54: | Regel 78: | ||
default-character-set = utf8 | default-character-set = utf8 | ||
EOM | EOM | ||
</syntaxhighlight> | |||
=== Database Setup === | |||
Create database & user: | |||
<syntaxhighlight lang="sql"> | |||
-- mysql (suply database admin arguments as and when needed) | |||
CREATE DATABASE mslcrm; | |||
CREATE USER 'mslcrmuser'@'localhost' IDENTIFIED BY 'XXXX-passowrd-XXX'; | |||
GRANT ALL PRIVILEGES ON mslcrm.* TO 'mslcrmuser'@'localhost'; | |||
FLUSH PRIVILEGES; | |||
</syntaxhighlight> | |||
Check for issues, init and build database & site: | |||
<syntaxhighlight lang="bash"> | |||
uv run manage.py check --deploy | |||
uv run manage.py makemigrations | |||
uv run manage.py migrate | |||
uv run manage.py collectstatic | |||
</syntaxhighlight> | |||
Check that email works: | |||
<syntaxhighlight lang="bash"> | |||
uv run manage.py sendtestemail your@email.address.com | |||
</syntaxhighlight> | |||
Create temp super user so you can log into complete the setup: | |||
<syntaxhighlight lang="bash"> | |||
uv run manage.py createsuperuser | |||
</syntaxhighlight> | |||
=== Web Server Configuration === | |||
Create apache config: | |||
<syntaxhighlight lang="apache"> | |||
cat > /etc/apache2/sites-available/crm.conf <<EOM | cat > /etc/apache2/sites-available/crm.conf <<EOM | ||
WSGIScriptAlias /crm /usr/local/makerspaceleiden-crm/makerspaceleiden/wsgi.py process-group=crm | WSGIScriptAlias /crm /usr/local/makerspaceleiden-crm/makerspaceleiden/wsgi.py process-group=crm | ||
Regel 101: | Regel 147: | ||
</Directory> | </Directory> | ||
EOM | EOM | ||
</syntaxhighlight> | |||
Activate this setup: | |||
<syntaxhighlight lang="bash"> | |||
ln -s /etc/apache2/sites-available/crm.conf /etc/apache2/sites-active/crm.conf | ln -s /etc/apache2/sites-available/crm.conf /etc/apache2/sites-active/crm.conf | ||
</syntaxhighlight> | |||
Start server and keep an eye on the log: | |||
<syntaxhighlight lang="bash"> | |||
apachectl configtest | |||
apachectl restart | apachectl restart | ||
tail -F /var/log/apache2/error.log | tail -F /var/log/apache2/error.log | ||
</syntaxhighlight> | |||
=== Daphne Setup === | |||
Daphne is the ASGI server for the Django application. It is used to serve the application over HTTP. The following is a systemd service file `/etc/systemd/system/crm-daphne.service`: | |||
<syntaxhighlight lang="bash"> | |||
[Unit] | |||
Description=Daphne Webserver for CRM | |||
After=network.target | |||
[Service] | |||
Environment="AGGREGATOR_BASE_URL=http://127.0.0.1:**port**" | |||
Environment="AGGREGATOR_USERNAME=**username**" | |||
Environment="AGGREGATOR_PASSWORD==**password**" | |||
Environment="FORCE_SCRIPT_NAME=" | |||
Environment="LOGIN_URL=/login" | |||
Environment="LOGIN_REDIRECT_URL=/" | |||
Environment="LOGOUT_REDIRECT_URL=/" | |||
Environment="LOG_FILE_NAME=/var/log/crm/daphne-crm-debug.log" | |||
Environment="UV_CACHE_DIR=/var/cache/crm/uv" | |||
Environment="UV_PYTHON_INSTALL_DIR=/var/cache/crm/uv" | |||
Environment="STATIC_URL=/static" | |||
Type=simple | |||
User=www-data | |||
Group=crmadmin | |||
WorkingDirectory=/usr/local/makerspaceleiden-crm | |||
ExecStart=/usr/local/bin/uv run daphne --access-log /var/log/crm//daphne-access.log -b 127.0.0.1 -p 8000 -v 0 makerspaceleiden.asgi:application | |||
[Install] | |||
WantedBy=multi-user.target | |||
</syntaxhighlight> | |||
Enable and start the service: | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl enable crm-daphne | |||
sudo systemctl start crm-daphne | |||
</syntaxhighlight> | |||
Check the status: | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl status crm-daphne | |||
</syntaxhighlight> | |||
=== Initial Configuration === | |||
Now go to https://mijn.makerspaceleiden.nl/ and create the initial members/structure. Assign the trustees the super user permission; then delete the temporary admin you made. | |||
== Configuration & Integration == | |||
=== Signal Bridge Setup === | |||
Requires openjdk, java-dbus-bin and dbussy. Binary package from https://github.com/AsamK/signal-cli/; installed in /home/signal-cli (post install 600/700 locked down). | |||
Post install - as user signal-cli - activation is required: | |||
<syntaxhighlight lang="bash"> | |||
signal-cli -u +317... register -voice | |||
signal-cli -u +317... verify code from call | |||
</syntaxhighlight> | |||
which will create the right structure in ~/.local. | |||
Source is required for the systemd files - installed with https://github.com/AsamK/signal-cli/wiki/DBus-service instructions. | |||
=== Configurating External Services === | |||
Mailing lists are ran by a third party - sympalists.net. The integration via their newly fangled "Sympa" web service -- See [[Sympa mailing list setup]] for details. | |||
Query to find duplicates in the subscriptions: | |||
<syntaxhighlight lang="sql"> | |||
SELECT * | |||
FROM mailinglists_subscription | |||
WHERE id NOT IN ( | |||
SELECT MAX(id) | |||
FROM mailinglists_subscription | |||
GROUP BY mailinglist_id, member_id | |||
); | |||
</syntaxhighlight> | |||
== Maintenance & Monitoring == | |||
=== Deploying Code Changes === | |||
Check that you are in the 'crmadmin' group. | |||
<syntaxhighlight lang="bash"> | |||
cd /usr/local/makerspaceleiden-crm | |||
# Ensure you are on the correct branch | |||
git checkout prod | |||
# Pull the latest changes | |||
git pull | |||
# Run the deployment script | |||
./rollout-prod.sh | |||
</syntaxhighlight> | |||
=== Logfile Retention === | |||
Configured in 'prod' settings - few MBs/days of logs is kept & then rotated out/deleted using the standard RotatingLogging handler. | |||
See https://github.com/MakerSpaceLeiden/makerspaceleiden-crm/commit/de5de35dd22a317bb93568dadc1b8737a819023a for details. | |||
=== Automated Maintenance === | |||
The following entries need to be added to the crontab (either to /etc/cron, to cron.daily/monthly or to the `users' cron): | |||
<syntaxhighlight lang="bash"> | |||
SHELL=/bin/sh | |||
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin | |||
MAILTO=noc@makerspaceleiden.nl | |||
MAILFROM="MSL Server (cron)" | |||
NULLMAILUSER_NAME="MSL Server (cron)" | |||
.... | |||
# General maintenance mails of [https://mijn.makerspaceleiden.nl mijn.makerspaceleiden.nl]. | |||
# | |||
# m h d m d user cmd | |||
1 1 * * * USER test -x /usr/local/makerspaceleiden-crm/cron-midnight.sh && /usr/local/makerspaceleiden-crm/cron-midnight.sh | |||
1 2 1 * * USER test -x /usr/local/makerspaceleiden-crm/cron-monthly.sh && /usr/local/makerspaceleiden-crm/cron-monthly.sh | |||
</syntaxhighlight> | |||
With 'USER' a user with the right rights. | |||
=== Backup === | |||
This is done by the /etc/duplicity/run.sh setup (along with everything else). It does a dump of the MySQL database | |||
and captures the whole directory -- which includes var/media -- the uploads. | |||
See the section [[Server setup#Backup | backup]] on the general [[Server setup| Server setup page.]] | |||
# the |
Huidige versie van 15 jun 2025 20:30
The code is hosted on GitHub at: https://github.com/MakerSpaceLeiden/makerspaceleiden-crm
Instructions for getting up and running locally are available in the Read Me.
Initial Setup
Note: pretty much everything below is stock/totally-standard django/python Standard Operating Procedure (except for the chmod/chgrp on the var/media upload).
Prerequisites
Make sure the baseline tools are present:
sudo apt-get install msmtp-mta apache2 libapache2-mod-uwsgi
sudo snap install astral-uv --classic
Code Setup
Initial checkout of code:
cd /usr/local
git clone https://github.com/MakerSpaceLeiden/makerspaceleiden-crm.git
Make everything group owned (crmadmin) and add that group to the accounts of those that need to maintain it.
Security Configuration
Create random seed:
openssl rand 128 > /etc/crm_secret_key.txt
chmod 640 /etc/crm_secret_key.txt
chgrp www-data /etc/crm_secret_key.txt
Allow storing of uploads in media by the suid that the webserver runs as:
mkdir -p var/media
chown www-data var/media
Allow server to rotate/recreate logfiles on the fly:
mkdir /var/log/crm
chown www-data:crmadmin /var/log/crm
chmod 770 /var/log/crm
Python Environment
Prepare environment for python and pull in the various dependencies:
uv venv
source ./.venv/bin/activate
uv sync
Set up the framework:
cd makerspaceleiden
ln -s prod.py local.py
cat > makerspaceledien/my.cnf <<EOM
[client]
database = mslcrm
user = mslcrmuser
password = XXXX-passowrd-XXX
default-character-set = utf8
EOM
Database Setup
Create database & user:
-- mysql (suply database admin arguments as and when needed)
CREATE DATABASE mslcrm;
CREATE USER 'mslcrmuser'@'localhost' IDENTIFIED BY 'XXXX-passowrd-XXX';
GRANT ALL PRIVILEGES ON mslcrm.* TO 'mslcrmuser'@'localhost';
FLUSH PRIVILEGES;
Check for issues, init and build database & site:
uv run manage.py check --deploy
uv run manage.py makemigrations
uv run manage.py migrate
uv run manage.py collectstatic
Check that email works:
uv run manage.py sendtestemail your@email.address.com
Create temp super user so you can log into complete the setup:
uv run manage.py createsuperuser
Web Server Configuration
Create apache config:
cat > /etc/apache2/sites-available/crm.conf <<EOM
WSGIScriptAlias /crm /usr/local/makerspaceleiden-crm/makerspaceleiden/wsgi.py process-group=crm
WSGIDaemonProcess crm python-home=/usr/local/makerspaceleiden-crm/venv python-path=/usr/local/makerspaceleiden-crm/
WSGIProcessGroup crm
Alias /crm-static/ /usr/local/makerspaceleiden-crm/static/
Alias /media/ /usr/local/makerspaceleiden-crm/var/media/
<Directory /usr/local/makerspaceleiden-crm/>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
<Directory /usr/local/makerspaceleiden-crm/static>
Options None
order deny,allow
allow from all
Require all granted
</Directory>
<Directory /usr/local/makerspaceleiden-crm/var/media>
Options None
order deny,allow
allow from all
Require all granted
</Directory>
EOM
Activate this setup:
ln -s /etc/apache2/sites-available/crm.conf /etc/apache2/sites-active/crm.conf
Start server and keep an eye on the log:
apachectl configtest
apachectl restart
tail -F /var/log/apache2/error.log
Daphne Setup
Daphne is the ASGI server for the Django application. It is used to serve the application over HTTP. The following is a systemd service file `/etc/systemd/system/crm-daphne.service`:
[Unit]
Description=Daphne Webserver for CRM
After=network.target
[Service]
Environment="AGGREGATOR_BASE_URL=http://127.0.0.1:**port**"
Environment="AGGREGATOR_USERNAME=**username**"
Environment="AGGREGATOR_PASSWORD==**password**"
Environment="FORCE_SCRIPT_NAME="
Environment="LOGIN_URL=/login"
Environment="LOGIN_REDIRECT_URL=/"
Environment="LOGOUT_REDIRECT_URL=/"
Environment="LOG_FILE_NAME=/var/log/crm/daphne-crm-debug.log"
Environment="UV_CACHE_DIR=/var/cache/crm/uv"
Environment="UV_PYTHON_INSTALL_DIR=/var/cache/crm/uv"
Environment="STATIC_URL=/static"
Type=simple
User=www-data
Group=crmadmin
WorkingDirectory=/usr/local/makerspaceleiden-crm
ExecStart=/usr/local/bin/uv run daphne --access-log /var/log/crm//daphne-access.log -b 127.0.0.1 -p 8000 -v 0 makerspaceleiden.asgi:application
[Install]
WantedBy=multi-user.target
Enable and start the service:
sudo systemctl enable crm-daphne
sudo systemctl start crm-daphne
Check the status:
sudo systemctl status crm-daphne
Initial Configuration
Now go to https://mijn.makerspaceleiden.nl/ and create the initial members/structure. Assign the trustees the super user permission; then delete the temporary admin you made.
Configuration & Integration
Signal Bridge Setup
Requires openjdk, java-dbus-bin and dbussy. Binary package from https://github.com/AsamK/signal-cli/; installed in /home/signal-cli (post install 600/700 locked down).
Post install - as user signal-cli - activation is required:
signal-cli -u +317... register -voice
signal-cli -u +317... verify code from call
which will create the right structure in ~/.local.
Source is required for the systemd files - installed with https://github.com/AsamK/signal-cli/wiki/DBus-service instructions.
Configurating External Services
Mailing lists are ran by a third party - sympalists.net. The integration via their newly fangled "Sympa" web service -- See Sympa mailing list setup for details.
Query to find duplicates in the subscriptions:
SELECT *
FROM mailinglists_subscription
WHERE id NOT IN (
SELECT MAX(id)
FROM mailinglists_subscription
GROUP BY mailinglist_id, member_id
);
Maintenance & Monitoring
Deploying Code Changes
Check that you are in the 'crmadmin' group.
cd /usr/local/makerspaceleiden-crm
# Ensure you are on the correct branch
git checkout prod
# Pull the latest changes
git pull
# Run the deployment script
./rollout-prod.sh
Logfile Retention
Configured in 'prod' settings - few MBs/days of logs is kept & then rotated out/deleted using the standard RotatingLogging handler.
See https://github.com/MakerSpaceLeiden/makerspaceleiden-crm/commit/de5de35dd22a317bb93568dadc1b8737a819023a for details.
Automated Maintenance
The following entries need to be added to the crontab (either to /etc/cron, to cron.daily/monthly or to the `users' cron):
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=noc@makerspaceleiden.nl
MAILFROM="MSL Server (cron)"
NULLMAILUSER_NAME="MSL Server (cron)"
....
# General maintenance mails of [https://mijn.makerspaceleiden.nl mijn.makerspaceleiden.nl].
#
# m h d m d user cmd
1 1 * * * USER test -x /usr/local/makerspaceleiden-crm/cron-midnight.sh && /usr/local/makerspaceleiden-crm/cron-midnight.sh
1 2 1 * * USER test -x /usr/local/makerspaceleiden-crm/cron-monthly.sh && /usr/local/makerspaceleiden-crm/cron-monthly.sh
With 'USER' a user with the right rights.
Backup
This is done by the /etc/duplicity/run.sh setup (along with everything else). It does a dump of the MySQL database and captures the whole directory -- which includes var/media -- the uploads.
See the section backup on the general Server setup page.