Server setup: verschil tussen versies

Uit MakerSpace Leiden
Ga naar: navigatie, zoeken
(→‎Setup and rudimentary hardening)
(→‎Virus scan)
Regel 306: Regel 306:
 
     rm -rf tmp.$$
 
     rm -rf tmp.$$
 
     exit $?
 
     exit $?
 +
 +
 +
= Monthly Reporting of deelnemers count =
 +
 +
The provider has set up a crontab (through their support desk, manual action) that emails a uu-encoded gzipped lists of subscribers to all list owners.
 +
 +
* The user 'listreporter' has been created as an alias in '''/etc/aliases'''
 +
 +
  listreporter: |/etc/mailman/listreporter.sh
 +
 +
* and is activated with '''newaliases'''.
 +
 +
This is wired up to a swapping script (requiring '''apt install moretools''') and a script that is somewhat paranoid and parses/unpacks the email:
 +
 +
  #!/bin/sh
 +
  set -e
 +
  umask 077
 +
  test -x /etc/mailman/listreporter.pl || exit 0
 +
  TMPDIR=${TMPDIR:-/tmp}
 +
  TMPDIR=${TMPDIR}/mailmanlistreporter.$$
 +
  (
 +
  mkdir $TMPDIR
 +
  cd $TMPDIR
 +
  cat > in.msg
 +
  /usr/bin/perl /etc/mailman/listreporter.pl
 +
  ) 2>&1 | ifne mail -s "Fault with the list reporter." noc@webweaving.org
 +
  rm -rf "${TMPDIR}"
 +
  exit 0
 +
 +
This is then wired to the script sending out the actual email:
 +
 +
  #!/usr/bin/perl
 +
  # use strict;
 +
  use IO::CaptureOutput qw/capture_exec/;
 +
 
 +
  umask(077);
 +
 
 +
  open(STDIN,"in.msg") or die "Could not open msg file: $!\n";
 +
 
 +
  my $list;
 +
  my $subject;
 +
 
 +
  while(<STDIN>) {
 +
          last if m/^\s*$/;
 +
          $subject = $1 if m/^Subject\s*:\s*(.*)$/;
 +
  };
 +
  close(STDIN);
 +
 
 +
  open STDIN, '</dev/null'; # Prevent warnings deep down in IO capture.
 +
 
 +
  $list = $1 if $subject =~ m/List subscriber file:\s*(\S+)/;
 +
 
 +
  # exit 0 unless $list;
 +
 
 +
  die "Not the right subject <$subject>\n" unless $list;
 +
  die "Not a list I manage." unless $list eq 'deelnemers';
 +
 
 +
  my ($stdout, $stderr, $success, $exit_code) = capture_exec("munpack","-q","in.msg");
 +
 
 +
  die "Unpacking failed: $!\n" unless $exit_code == 0;
 +
 
 +
  my $d = '';
 +
  foreach my $f (<subscribers.$list.*.gz>) {
 +
          next unless $f =~ m/subscribers.$list.([0-9\-]+).txt.gz/;
 +
          $d = $1;
 +
 
 +
          system("gunzip", $f) == 0
 +
                  or die "Could not unpack $f\n";
 +
          $f =~ s/.gz//;
 +
          open(FH,$f) or die "Could not open subscriber list";
 +
          my @list = ();
 +
          while(<FH>) {
 +
                  chomp;
 +
                  s/\s*#.*//;
 +
                  s/\P{IsPrint}//g;
 +
                  next if m/^\s*$/;
 +
                  push @list, $_;
 +
          };
 +
          my $count= @list;
 +
 
 +
          open(FH,"| mail -s 'Makerspace Leiden, $count $list on $d' -aFrom:hetbestuur\@makerspaceleiden.nl  $list\@makerspaceleiden.nl")
 +
                  or die "Cannot open pipe.\n";
 +
 
 +
          print FH <<"EOM";
 +
  Hello All,
 +
 
 +
  We have currently $count makers united, listed below:
 +
 
 +
  Your can edit your own information here:
 +
 
 +
          https://mailman.makerspaceleiden.nl/mailman/options/deelnemers
 +
 
 +
  and this is also where you can switch to a digtest (just one big message/day)
 +
  or see the archive of historic messages.
 +
 
 +
  EOM
 +
          print FH join("\n", @list);
 +
          print FH "\n-- \nStiching Makerspace Leiden / hetbestuur@makerspaceleiden.nl\n\n";
 +
          close(FH);
 +
 
 +
          exit(0);
 +
  }
 +
  
  

Versie van 1 jul 2018 om 11:25

The front end server (mid 2018) runs on Linux; in a `cloud' hosted environment.

Below documents the initial setup of the base machine; followed by the setup for each of the modules.

The final section shows the monthly and annual maintenance cycles.


Setup and rudimentary hardening

  • Get the machine in a known state and install sudo (so we can disable root; and comply with 'named accounts' only policies):
apt update
apt upgrade
apt install sudo certbot certbot-apache moreutils
  • enable ufw and allow the usual ports in IPv4 and IPv6
 for port in 22 25 53 80 443
 do
   ufw allow  $port
 done
 # do this last so we're not kicked out.
 ufw enable
  • create named accounts for each of the admins (you need to get everyones their public SSH key):
 adduser \
  --system \
  --shell /bin/bash \
  --gecos 'Dirk-Willem van Gulik' \
  --group \
  --ingroup admin \
  --disabled-password \
  dirkx
  • Add an ssh key for each of these users
  • check that you can log in; and sudo with at least one of them.
  • Block root login and passwords in /etc/ssh/sshd.conf:
 PermitRootLogin no
 PasswordAuthentication no
 ChallengeResponseAuthentication no

Note: if you did not check the sudo/login of an admin user - they you are about to lock yourself out upon reboot.
  • Edit /etc/sysctl.conf to block spoofing, ICMP broadcast, source-packet routing, send redirect, SYN attacks, Martians and ICM redirects.
  • Prevent IP spoofing for DNS by replacing multi on to nospoof on in /etc/hosts.conf
  • Securing shared memory.
 echo tmpfs /run/shm tmpfs defaults,noexec,nosuid 0 0 >> /etc/fstab
  • Make sure we trap all crontab outputs systemctl edit cron.service:
  [Service]
  Environment="MAILTO=noc@makerspaceleiden.nl"
  • Reboot.

Setup of the basic website

We need to first set up a very basic website; in order to be able to fetch the required SSL certificates.

  • Install apache and certbot and the integration glue between the two:
sudo apt install apache2 certbot python-certbot-apache
  • Request the needed certs:
sudo certbot --apache -d makerspaceleiden.nl -d www.makerspaceleiden.nl -d wiki.makerspaceleiden.nl
  • Ensure they get renewed; and that the admins are emailed when this goes wrong:

Setup of the MTA

Log in as one of the admins.

  • Make sure /etc/mailname is set to makerspaceleiden.nl
  • Install postfix and basic mail stuff:
 sudo apt install postfix mailtools postgrey mavisd-new spamassassin clamav-daemon libnet-dns-perl libmail-spf-perl pyzor razor  arj bzip2 cabextract cpio file gzip  nomarch pax rar unrar unzip zip
  • Edit the /etc/aliases file to redirect the mail of 'root' and update:
 sudo vi /etc/aliases
 sudo newaliases
  • Edit the certs in /etc/postfix/main.cf
 smtpd_tls_cert_file=/etc/letsencrypt/live/makerspaceleiden.nl/fullchain.pem
 smtpd_tls_key_file=/etc/letsencrypt/live/makerspaceleiden.nl/privkey.pem
  • Give the virus/spam scanner mutual access:
 sudo adduser clamav amavis
 sudo adduser amavis clamav
  • As the amavis user; razor-admin -create and razor-admin -register if needed.
  • Edit /etc/amavis/conf.d/15-content_filter_mode to activate the filters and restart:
 sudo /etc/init.d/amavis restart
  • The whitelist for postgray contains a number of entries for the various deur logging systems:
... to do ...

Setup of WordPress

  • Install enough of the LAMP stack to get going:
 sudo apt install apache2 mysql-server php libapache2-mod-php php-mysql wordpress-theme-twentyseventeen wordpress
  • Secure your mysql install:
 sudo mysql_secure_installation
  • Configure apache:
  cat <<EOM > /etc/apache2/sites-available/wordpress.conf
 <VirtualHost *:80>
     ServerAdmin noc@makerspaceleiden.nl
  
     ServerName        makerspaceleiden.nl
     ServerAlias       www.makerspaceleiden.nl
  
     DocumentRoot /var/www/html
     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined
  
     RewriteEngine On
     RewriteCond %{HTTPS} off
     RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L]
 </VirtualHost>
 
 Listen *443
 <VirtualHost *:443>
     ServerAdmin noc@makerspaceleiden.nl
     ServerName        makerspaceleiden.nl
     ServerAlias       www.makerspaceleiden.nl
  
     SSLCertificateFile  /etc/letsencrypt/live/makerspaceleiden.nl/fullchain.pem
     SSLCertificateKeyFile /etc/letsencrypt/live/makerspaceleiden.nl/privkey.pem
 
     SSLProtocol all -SSLv2 -SSLv3
     SSLHonorCipherOrder On
     SSLCipherSuite EECDH+AES:EDH+AES:-SHA1:EECDH+RC4:EDH+RC4:RC4-SHA:EECDH+AES256:EDH+AES256:AES256-SHA:!aNULL:!eNULL:!EXP:!LOW:!MD5
 
     DocumentRoot /usr/share/wordpress
 
     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined
  
     <Directory /usr/share/wordpress>
           Options FollowSymLinks
           AllowOverride Limit Options FileInfo
           DirectoryIndex index.php
           Order allow,deny
           Allow from all
     </Directory>
 
     <Directory /usr/share/wordpress/wp-content>
           Options FollowSymLinks
           Order allow,deny
           Allow from all
       </Directory>
        
  </VirtualHost>

  EOM
  • Create a database and a config file (changing yourpasswordhere123):
 cat <<EOM |  sudo mysql --defaults-extra-file=/etc/mysql/debian.cnf
 CREATE DATABASE wordpress;
 GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER
 ON wordpress.*
 TO wordpress@localhost
 IDENTIFIED BY 'yourpasswordhere123';
 FLUSH PRIVILEGES;
 EOM

 cat <<EOM > /etc/wordpress/config-localhost.php
 <?php
 define('DB_NAME', 'wordpress');
 define('DB_USER', 'wordpress');
 define('DB_PASSWORD', 'yourpasswordhere123');
 define('DB_HOST', 'localhost');
 define('WP_CONTENT_DIR', '/usr/share/wordpress/wp-content');
 define('FS_METHOD', 'direct');
 ?>
  • Secure this file so that only the webserver can see this password.
    sudo chown root:www-data /etc/wordpress/config-localhost.php
    sudo chmod o-rwx,g-wx /etc/wordpress/config-localhost.php
  • Enable, kill default and restart:
  sudo a2ensite wordpress
  sudo a2dissite 000-default 
  sudo systemctl reload apache2
  • Check that it all works by visiting
  httpp://makerspaceleiden.nl/install.php
  • If you want to be able to update in place - be sure to have the wordpress content directory `www-data' owned.

Setup of Media Wiki

  • Install the base packages
 apt install mediawiki imagemagick php-apcu php-intl
  • Create the database:
cat <<EOM |  sudo mysql --defaults-extra-file=/etc/mysql/debian.cnf
CREATE DATABASE mediawiki;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX
ON mediawiki.*
TO mediawiki@localhost
IDENTIFIED BY 'yourpasswordhere12443';
FLUSH PRIVILEGES;
EOM
  • Restart apache.
  • Go to the wiki - and use the config values from above DB setup.,
  • Follow the instructions and copy the generated LocalSettings.php to the specified location.
  • Add the various users.

making a backup and importing it

  • Backup script:
 #!/bin/sh
 set -e
 set -x
 D=`date +%Y%m%d%H%M%S`
 mkdir media-wiki-backup.$D
 
 cd media-wiki-backup.$D
 
 mysqldump --user=wikiuser --password="XXXX" wikidb > file.sql
 mysqldump --user=wikiuser --password="XXXX" wikidb --xml > file.xml
 
 cp -r /usr/local/www/mediawiki/images .
 cp /usr/local/www/mediawiki/LocalSettings.php .
 
 cd ..
 tar zcf media-wiki-backup-$D.tgz media-wiki-backup.$D
 rm -rf  media-wiki-backup.$D
  • Dump the full wiki:
    jexec mls  ... script ..
  • Copy the file across.
  • Import it on the other machine
  #!/bin/sh
  set -ex
  
  if ! test -f latest.tgz; then
   		echo no last dump.
   		exit 1
   fi
   mkdir tmp.$$
   cd tmp.$$
   tar zxf ../latest.tgz
   (
   cd *
   
   rm -rf /var/lib/mediawiki/cache /var/lib/mediawiki/images
   
   mkdir import /var/lib/mediawiki/cache /var/lib/mediawiki/images
   cp -r images/? /var/lib/mediawiki/images
   chown -R www-data:www-data /var/lib/mediawiki/images  /var/lib/mediawiki/cache
   
   cat <<EOM |  sudo mysql --defaults-extra-file=/etc/mysql/debian.cnf
   DROP DATABASE mediawiki;
   CREATE DATABASE mediawiki;
   GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX
   ON mediawiki.*
   TO mediawiki@localhost
   IDENTIFIED BY 'XXXX';
   FLUSH PRIVILEGES;
   EOM
  
   cat file.sql | sudo mysql --defaults-extra-file=/etc/mysql/debian.cnf mediawiki
  
   sudo -u www-data /usr/share/mediawiki/maintenance/update.php
   sudo -u www-data php  /usr/share/mediawiki/maintenance/cleanupImages.php
   sudo -u www-data php /usr/share/mediawiki/maintenance/rebuildall.php 
   sudo -u www-data php /usr/share/mediawiki/maintenance/rebuildImages.php 
   
   ) || exit 1
   rm -rf tmp.$$
   exit $?


Monthly Reporting of deelnemers count

The provider has set up a crontab (through their support desk, manual action) that emails a uu-encoded gzipped lists of subscribers to all list owners.

  • The user 'listreporter' has been created as an alias in /etc/aliases
 listreporter: |/etc/mailman/listreporter.sh
  • and is activated with newaliases.

This is wired up to a swapping script (requiring apt install moretools) and a script that is somewhat paranoid and parses/unpacks the email: 

 #!/bin/sh
 set -e
 umask 077
 test -x /etc/mailman/listreporter.pl || exit 0
 TMPDIR=${TMPDIR:-/tmp}
 TMPDIR=${TMPDIR}/mailmanlistreporter.$$
 (
 	mkdir $TMPDIR
 	cd $TMPDIR
 	cat > in.msg
 	/usr/bin/perl /etc/mailman/listreporter.pl
  ) 2>&1 | ifne mail -s "Fault with the list reporter." noc@webweaving.org
  rm -rf "${TMPDIR}"
  exit 0

This is then wired to the script sending out the actual email: 

  #!/usr/bin/perl 
  # use strict;
  use IO::CaptureOutput qw/capture_exec/;
  
  umask(077);
  
  open(STDIN,"in.msg") or die "Could not open msg file: $!\n";
  
  my $list;
  my $subject;
  
  while(<STDIN>) {
          last if m/^\s*$/;
          $subject = $1 if m/^Subject\s*:\s*(.*)$/;
  };
  close(STDIN);
  
  open STDIN, '</dev/null'; # Prevent warnings deep down in IO capture.
  
  $list = $1 if $subject =~ m/List subscriber file:\s*(\S+)/;
  
  # exit 0 unless $list;
  
  die "Not the right subject <$subject>\n" unless $list;
  die "Not a list I manage." unless $list eq 'deelnemers';
  
  my ($stdout, $stderr, $success, $exit_code) = capture_exec("munpack","-q","in.msg");
  
  die "Unpacking failed: $!\n" unless $exit_code == 0;
  
  my $d = ;
  foreach my $f (<subscribers.$list.*.gz>) {
          next unless $f =~ m/subscribers.$list.([0-9\-]+).txt.gz/;
          $d = $1;
  
          system("gunzip", $f) == 0
                  or die "Could not unpack $f\n";
          $f =~ s/.gz//;
          open(FH,$f) or die "Could not open subscriber list";
          my @list = ();
          while(<FH>) {
                  chomp;
                  s/\s*#.*//;
                  s/\P{IsPrint}//g;
                  next if m/^\s*$/;
                  push @list, $_;
          };
          my $count= @list;
  
          open(FH,"| mail -s 'Makerspace Leiden, $count $list on $d' -aFrom:hetbestuur\@makerspaceleiden.nl  $list\@makerspaceleiden.nl")
                  or die "Cannot open pipe.\n";
  
          print FH <<"EOM";
  Hello All,
  
  We have currently $count makers united, listed below:
  
  Your can edit your own information here:
  
          https://mailman.makerspaceleiden.nl/mailman/options/deelnemers
  
  and this is also where you can switch to a digtest (just one big message/day)
  or see the archive of historic messages.
  
  EOM
          print FH join("\n", @list);
          print FH "\n-- \nStiching Makerspace Leiden / hetbestuur@makerspaceleiden.nl\n\n";
          close(FH);
  
          exit(0);
  }


Virus scan

There is a malware/virus scan set up across the uploads; with automatic refresh of the signature DBs: 

... todo
Overgenomen van "https://wiki.makerspaceleiden.nl/mediawiki/index.php?title=Server_setup&oldid=2475"