Difference between revisions of "Moving to new machine binti"

From Anarchaserver
Line 282: Line 282:
To renew them automatically, add in crontab as root :
To renew them automatically, add in crontab as root :
  crontab -e
  crontab -e
  certbot renew -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /root/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory
  */5 * * * * /usr/local/bin/certbot renew -q -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /root/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory
Don't forget to restart apache
Don't forget to restart apache
  systemctl restart apache2
  systemctl restart apache2
Also remove the Timer driven by systemd (installed by default with certbot threw the package mamager) to let the cron job do the renew process :
* List Existing Timers
systemctl list-timers
systemctl list-timers --all
* remove the certbot.timer and disable certbot.service
rm /etc/systemd/system/timers.target.wants/certbot.timer
systemctl disable certbot.service
* Check if the crontab works
crontab -e
cat /var/mail/root (a mail is sent each time the cronjob is done
tail -f /var/log/letsencrypt/letsencrypt.log
If certbot: error: unrecognized arguments: --certbot-plugin-gandi:dns-credentials so the crontab doesn't [https://github.com/obynio/certbot-plugin-gandi/issues/9 use the good certbot version], to check :
# certbot --version
-bash: /usr/local/bin/certbot: No such file or directory <- Former location of certbot 0.35 installed via pip
# which certbot
/usr/bin/certbot <- Location of certbot 0.28 installed with apt
# whereis certbot
certbot: /usr/bin/certbot /usr/share/man/man1/certbot.1.gz
# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
In anarchaserver case, the cron job was using the bad version of certbot (/usr/bin/certbot was throwing the error), let's remove them :
apt remove certbot python3-certbot
Indicate full path for the good certbotin the cronjob
*/5 * * * * /usr/local/bin/certbot renew -q -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /root/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory
Maybe we should add a hook to restart apache ?


== Netdata monitoring and alert system ==
== Netdata monitoring and alert system ==

Revision as of 18:14, 13 September 2020

Installation of the new machine "binti" in new server

Anoia > Anna (US) Moving to Binti > Ursula (SE)

Binti as the main character of a science fiction novella written by Nnedi Okorafor (2015). https://en.wikipedia.org/wiki/Binti_(novel)

We have setup a virtual server (KVM) on our host server for you: 

 Host server:    ursula.tachanka.org
 Virtual server: binti.tachanka.org
 Storage:        120G
 Memory:         4096M
 CPU:            2

Network configuration for your virtual server: IP: 198.167.222.149/32 (we did not put 32 cause was not accepted, only 198.167.222.149) Gateway: 198.167.222.1 Host name: binti DNS Server: 9.9.9.10 Domain name: anarchaserver.org 

 ssh binti@ursula.tachanka.org

The SSH host key fingerprints there are: 

 SHA256:rSBy7PUW9liNDBl/zjx52DG3nq+a3i4TsiiE5gAnfuE (ECDSA)
 SHA256:9XglwKf0gPHffnhKlgDRLWTB6EuMBAaplBKxhK86JPE (ED25519)
 SHA256:w7P41LnClVfHf9Te2y3fDkc8YhDO5nSmfdYLtPrIfFs (RSA)

Steps to do:

Install Debian Stable

Set up full disk encryption in your virtual machine during the installation, and keep encrypted backups of the passphrase as well as backups of any important data, as we do not keep backups of your data by default.

Add ssh keys inside binti

Next steps:

Install and configure some security things : ssh server iptable + ufw fail2ban things like chkrootkit rkhunter etckeeper ? configure an everyday mail report sent to sysadmins ? Install and configure some softwares on the host : apache2 LXC

Reinstall AS by testing current back up system > shutting donw anoia and passing it over to tachanka

Documentation: 

   Add new tech documentation
   Reframe current wiki page: https://alexandria.anarchaserver.org/index.php/Machine

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

b01 : yes, I could add the keys in/au /home/binti/.sshthorized_keys I can connect to the server with ssh binti@ursula.tachanka.org

Once connected with ssh binti@ursula.tachanka.org it is possible to reach the installation screen with : 

   screen -x

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Installation of debian stable 10 buster

Network configuration for your virtual server: IP: 198.167.222.149/32 (we did not put 32 cause was not accepted, only 198.167.222.149) Gateway: 198.167.222.1 Host name: binti DNS Server: 9.9.9.10 Domain name: anarchaserver.org

B01 creates a new password for root in binti > 40 characters and will share with sysadmin over GPG

A user Anarcha is created for non administrative tasks inside the server > b01 creates a new password > 40 characters and will share with sysadmin over GPG

Setting up full disk encryption in Binti during the installation - create a swap of 8 GB ? Pass phrase is created for encryption > b01 creates a new password > 40 characters and will share with sysadmin over GPG

We answer yes to be part of the debian package survey

Grub created and located in /dev/vda/

We could decrypt and log in binti with passwords created

Improve the security

ssh

basic sshd_config same as previous : 

PasswordAuthentication no
X11Forwarding no
PermitRootLogin no

Cammbiamos puerto al 2212, ahora nos logueamos con 

ssh -p 2212 user@binti.tachanka.org

Folder ssh keys of AS sysadmin added inside binti Process: Created users for all syadmin, add each at sudo group and added related SSH key (the one used for anoia) so you can access the server, from there you can decide to keep old key or to generate a new one for binti as described here https://alexandria.anarchaserver.org/index.php/Access_server

unattended upgrades

Activar unattended upgrades > se hace con el script que actualiza todas los containers mejor, hay que configurarlo, no? 

sudo apt install unattended-upgrades
sudo apt install apt-listchanges
Unattended-Upgrade::Mail "root";
Unattended-Upgrade::MailOnlyOnError "false";

Si queremos que nos llege un mail cada dia con las actualizaciones que se hicieron hay que configurar un servicio de correo (no lo hicimos)

En /etc/apt/apt.conf.d/20auto-upgrades: 

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

fail2ban

Activar fail2ban: 

apt-get install fail2ban
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

borramos las jails que no usamos

Activamos las jails: sshd , apache y apache badbots (dejamos para investigar como protejer los containers) acordarse de poner el puerto 2212 (hecho)

Firewall

ufw 
ufw default deny incoming
ufw default allow outgoing
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 2212/tcp -> 
ufw allow 8000/tcp

Security -References

Resources on security and devops/sysadmin:

About unattended upgrades

Fail2ban

UFW

Apache2 web server

Installation of apache2 server 

apt install apache2

Specific configuration : add some modules 

a2enmod ssl
a2enmod rewrite
a2enmod proxy_http
a2ensite > to activate the imported virtual hosts

Some changes has been made to increase the MaxWorkers numbers, see this article

Find which Multi-Processing Module (MPM) is currently in use: 

apache2ctl -V | grep MPM
Server MPM:     event

Edit the appropriated file : 

nano /etc/apache2/mods-enabled/mpm_event.conf
<IfModule mpm_event_module>
       StartServers            3
       MinSpareThreads         5
       MaxSpareThreads         10
       ServerLimit             250
       ThreadLimit             64
       ThreadsPerChild         25
       MaxRequestWorkers       1250
       MaxConnectionsPerChild   0
</IfModule>

Check the numbers of connexions (to check if the server is not under DDoS attack) : 

netstat -an | egrep ':80|:443' | grep ESTABLISHED | awk '{print $5}' | grep -o -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | sort -n | uniq -c | sort -nr

Install LXC containers

Installing the base of LXC following :

Restoration of the container from the previous machine with duplicity

Config of the containers with this type of config (at the end, the most basic one), there has been changes of the config file with debian 10 buster : 

nano /var/lib/lxc/livingdata/config
lxc.net.0.type = veth
lxc.net.0.hwaddr = 00:16:3e:ff:1f:df
lxc.net.0.link = virbr0
lxc.net.0.flags = up
lxc.apparmor.profile = generated
lxc.apparmor.allow_nesting = 1
lxc.rootfs.path = /var/lib/lxc/livingdata/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf
# Container specific configuration
lxc.tty.max = 4
lxc.uts.name = livingdata
lxc.arch = amd64
lxc.pty.max = 1024
lxc.start.auto = 1

is working and giving adresse 192.168.122.*

vhost apache proxy to the containers

Restoration of the apache2 vhost to proxy the traffic to the containers, exemple of transition subdomain / container : 

nano /etc/apache2/sites-enabled/transitional.conf

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName transitional.anarchaserver.org
        ErrorLog ${APACHE_LOG_DIR}/transitional-error.log
        CustomLog ${APACHE_LOG_DIR}/transitional-access.log combined
  ProxyPreserveHost       On
  ProxyRequests           Off
  ProxyPass / http://192.168.122.111/
  ProxyPassReverse http://192.168.122.111/ /
        <Proxy *>
                Order deny,allow
                Allow from all
        </Proxy>
RewriteEngine on
RewriteCond %{SERVER_NAME} =transitional.anarchaserver.org
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]
</VirtualHost>
nano /etc/apache2/sites-enabled/transitional-le-ssl.conf

<IfModule mod_ssl.c>
<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        ServerName transitional.anarchaserver.org
        ErrorLog ${APACHE_LOG_DIR}/transitional-error.log
        CustomLog ${APACHE_LOG_DIR}/transitional-access.log combined
        ProxyPreserveHost       On
        ProxyRequests           Off
        ProxyPass /  https://transitional.anarchaserver.org/
        ProxyPassReverse / https://transitional.anarchaserver.org/
        <Proxy *>
                Order deny,allow
                Allow from all
        </Proxy>
       SSLEngine on
       SSLProxyEngine On
       SSLProxyVerify none
       SSLProxyCheckPeerCN off
       SSLProxyCheckPeerName off
       SSLProxyCheckPeerExpire off

RewriteEngine on
SSLCertificateFile /etc/letsencrypt/live/anarchaserver.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/anarchaserver.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
</IfModule>

Add the ip of the container in /etc/hosts if the ssl proxy is a dns : 

nano /etc/hosts 
192.168.122.111 transitional.anarchaserver.org

SSL certificates

Utilisation of certbot with gandi plug-in (our registrar) to create and update the certificates for the main domain and all subdomains (wildcard) : One certificate for all the subdomains

Install Gandi plug-in on Debian 10 : 

apt remove python-cryptography
pip install cryptography
pip install certbot-plugin-gandi

Activate mod ssl for Apache : 

a2enmod ssl
systemctl restart apache2

Create and Install the certificates : 

certbot certonly -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /root/gandi.ini -d anarchaserver.org -d "*.anarchaserver.org" --server https://acme-v02.api.letsencrypt.org/directory

To renew them automatically, add in crontab as root : 

crontab -e
*/5 * * * * /usr/local/bin/certbot renew -q -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /root/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory

Don't forget to restart apache 

systemctl restart apache2

Also remove the Timer driven by systemd (installed by default with certbot threw the package mamager) to let the cron job do the renew process :

  • List Existing Timers
systemctl list-timers
systemctl list-timers --all
  • remove the certbot.timer and disable certbot.service
rm /etc/systemd/system/timers.target.wants/certbot.timer
systemctl disable certbot.service
  • Check if the crontab works
crontab -e
cat /var/mail/root (a mail is sent each time the cronjob is done
tail -f /var/log/letsencrypt/letsencrypt.log

If certbot: error: unrecognized arguments: --certbot-plugin-gandi:dns-credentials so the crontab doesn't use the good certbot version, to check : 

# certbot --version
-bash: /usr/local/bin/certbot: No such file or directory <- Former location of certbot 0.35 installed via pip
# which certbot
/usr/bin/certbot <- Location of certbot 0.28 installed with apt
# whereis certbot
certbot: /usr/bin/certbot /usr/share/man/man1/certbot.1.gz
# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

In anarchaserver case, the cron job was using the bad version of certbot (/usr/bin/certbot was throwing the error), let's remove them : 

apt remove certbot python3-certbot

Indicate full path for the good certbotin the cronjob 

*/5 * * * * /usr/local/bin/certbot renew -q -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /root/gandi.ini --server https://acme-v02.api.letsencrypt.org/directory

Maybe we should add a hook to restart apache ?

Netdata monitoring and alert system

Installation of Netdata to monitor the server activity and to send alert once something is not working, now available at https://netdata.anarchaserver.org 

 bash <(curl -Ss https://my-netdata.io/kickstart.sh) --stable-channel --disable-telemetry

 /etc/netdata/edit-config health_alarm_notify.conf

Netdata anarcha.png

Retrieved from "https://alexandria.anarchaserver.org/index.php?title=Moving_to_new_machine_binti&oldid=742"