Backup - Revision history

M4ra at 11:31, 1 October 2021

2021-10-01T11:31:42Z

← Older revision		Revision as of 11:31, 1 October 2021
Line 21:		Line 21:

	=== Process ===		=== Process ===
	* stop the containers (because AS containers were created and managed by root user, they need root privlege to stop them. We solved this by allowing the specific lxc stop/start commands for the backup user via the ~~sudoes~~.d configuration.)		* stop the containers (because AS containers were created and managed by root user, they need root privlege to stop them. We solved this by allowing the specific lxc stop/start commands for the backup user via the sudoers.d configuration.)
	* backup the containers		* backup the containers
	* restart the containers		* restart the containers

B01: /* Steps we follow to backup AS LXC containers */

2021-09-14T18:21:09Z

Steps we follow to backup AS LXC containers

← Older revision		Revision as of 18:21, 14 September 2021
Line 21:		Line 21:

	=== Process ===		=== Process ===
	* - stop the containers (because AS containers were created and managed by root user, they need root privlege to stop them. We solved this by allowing the specific lxc stop/start commands for the backup user via the sudoes.d configuration.)		* stop the containers (because AS containers were created and managed by root user, they need root privlege to stop them. We solved this by allowing the specific lxc stop/start commands for the backup user via the sudoes.d configuration.)
	* - backup the containers		* backup the containers
	* - restart the containers		* restart the containers

	=== System setup ===		=== System setup ===
	* - a backup 'user'		* a backup 'user'
	* - ssh keys to access remote backup repo		* ssh keys to access remote backup repo
	* - password for restic backup command		* password for restic backup command
	* - bash script with the necessary commands to run the process above		* bash script with the necessary commands to run the process above
	* - cron job added via the backup user to run the bash script on regular intervals		* cron job added via the backup user to run the bash script on regular intervals

	=== Configuration of the system setup ===		=== Configuration of the system setup ===
	==== 1. How to access the remote backup repo? ====		==== 1. How to access the remote backup repo? ====
	* - create new ssh keys for ssh access to the remote backup repo, and give directory flag for saving the keys under the desired directory.		* create new ssh keys for ssh access to the remote backup repo, and give directory flag for saving the keys under the desired directory.
	* - scp the new pub key in the remote backup repo, under the remote user's home .ssh/authorized_keys. Remote user is the one we use to ssh to the remote machine.		* scp the new pub key in the remote backup repo, under the remote user's home .ssh/authorized_keys. Remote user is the one we use to ssh to the remote machine.

	==== 2. How to run the backup command without root privelege ====		==== 2. How to run the backup command without root privelege ====
	when some of the filesystem to be backed-up is accessed only by root, aka execute a binary meant for root without being root? The idea is to execute the restic binary from backup user's home or from /usr/bin.		when some of the filesystem to be backed-up is accessed only by root, aka execute a binary meant for root without being root? The idea is to execute the restic binary from backup user's home or from /usr/bin.

	~~<code> # useradd backupuser </code>~~		creates the new user and a group with the same name
	# creates the new user and a group with the same name		useradd backupuser
	~~<code> # chown root:~~backupuser ~~/usr/bin/restic</code>~~		makes user root and group backupuser owners of the restic binary
	# makes user root and group backupuser owners of the restic binary		chown root:backupuser /usr/bin/restic
	~~<code> # chmod 750~~ /usr/bin/restic ~~</code>~~		user root has now read, write, execute permissions, and users in backupuser group can execute and read the restic binary
	# user root has now read, write, execute permissions, and users in backupuser group can execute and read the restic binary		chmod 750 /usr/bin/restic
	~~<code> #~~ setcap cap_dac_read_search=+ep ~backupuser/bin/restic </code>		assigns capabilities to backup the whole system
	~~# assigns capabilities to backup the whole system~~		setcap cap_dac_read_search=+ep ~backupuser/bin/restic </code>

	Ref: https://restic.readthedocs.io/en/stable/080_examples.html#backing-up-your-system-without-running-restic-as-root		Ref: https://restic.readthedocs.io/en/stable/080_examples.html#backing-up-your-system-without-running-restic-as-root
Line 58:		Line 58:

	But for now we will give the backupuser restricted root privelege for the specific lxc commands we need to run in the backup script.		But for now we will give the backupuser restricted root privelege for the specific lxc commands we need to run in the backup script.
	~~<code> #~~ usermod -aG sudo backupuser ~~</code>~~		usermod -aG sudo backupuser

	Give the backup user the acces for specific commands to be executed as 'root'.		Give the backup user the acces for specific commands to be executed as 'root'.
	Add these commands in a new file under /etc/sudoers.d/		Add these commands in a new file under /etc/sudoers.d/
	~~<code>~~ vi /etc/sudoers.d/00-backupuser ~~</code>~~		vi /etc/sudoers.d/00-backupuser
	bintibackup ALL=(ALL) NOPASSWD: /usr/bin/lxc-stop, /usr/bin/lxc-start, /usr/bin/lxc-info, /usr/bin/lxc-ls		bintibackup ALL=(ALL) NOPASSWD: /usr/bin/lxc-stop, /usr/bin/lxc-start, /usr/bin/lxc-info, /usr/bin/lxc-ls

	Ref: https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/		Ref: https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/

	++++		++++
	chown root:bintibackup /usr/bin/lxc-stop		chown root:bintibackup /usr/bin/lxc-stop
	> this is for changing the property to usr/bin/lxc-stop to root group and bintibackup		> this is for changing the property to usr/bin/lxc-stop to root group and bintibackup
	first, we are trying to make this happen with a bintibackup user with no root privilege - if it does no work, will switch to possibility to give the bintibackup user root provileges, but this will imply to have a vert strict security management policy for that userthis
			first, we are trying to make this happen with a bintibackup user with no root privilege - if it does no work, will switch to possibility to give the bintibackup user root provileges, but this will imply to have a vert strict security management policy for that userthis
	above scratched steps didn't work!		above scratched steps didn't work!
	++++		++++
Line 86:		Line 88:
	$user="backuphost"		$user="backuphost"
	So the command becomes:		So the command becomes:
	~~<code> #~~ restic -p ~/backup/pass -r sftp:backuphost@backups.org:/var/backups --verbose backup /var/foo ~~</code>~~		restic -p ~/backup/pass -r sftp:backuphost@backups.org:/var/backups --verbose backup /var/foo
	OR:		OR:
	~~<code> #~~ restic -p $PASSWORD -r sftp:$user@$host:$remote-path --verbose backup ~~</code>~~		restic -p $PASSWORD -r sftp:$user@$host:$remote-path --verbose backup

	Final bash script (the exact paths haave been changed to fictional ones)		Final bash script (the exact paths haave been changed to fictional ones)
	<~~code~~>		<syntaxhighlight lang="bash">
	#!/bin/bash		#!/bin/bash
	# a backup to our backup server of the LXC containers		# a backup to our backup server of the LXC containers
Line 137:		Line 139:
	sudo lxc-start $container		sudo lxc-start $container
	done		done
	</~~code~~>
			</syntaxhighlight>

	== Resources for backup scripts and helpful tips ==		== Resources for backup scripts and helpful tips ==
	restic for backups		restic for backups

B01: /* Current setup */

2021-09-14T18:16:06Z

Current setup

← Older revision		Revision as of 18:16, 14 September 2021
Line 148:		Line 148:
	* https://www.foxinfotech.in/2019/03/reading-a-password-from-a-file-in-linux.html		* https://www.foxinfotech.in/2019/03/reading-a-password-from-a-file-in-linux.html

	= ~~Current~~ setup =		= Old setup =

	We currently make a nightly backup of all containers from our virtual machine (called <code>anoia</code>) to the virtual machine of calafou.org (called <code>ebro</code>).		We currently make a nightly backup of all containers from our virtual machine (called <code>anoia</code>) to the virtual machine of calafou.org (called <code>ebro</code>).

B01: /* Current setup */

2021-09-14T18:15:04Z

Current setup

← Older revision		Revision as of 18:15, 14 September 2021
Line 10:		Line 10:

	= Current setup =		= Current setup =
			Now the backup of Binti is done with a script that relies on Restic tool

			We used to do backups with duplicity but we opted for restic as it is easier to install than duplicity. However duplicity has more options on how to fine tune backups.

			Simplest tool for backups is rsync (https://rsync.samba.org/) which copies files, which can be archived or not, and checks if a file has been modified, or is removed or created and updates the backup target repo accordingly. But it doesn't create snapshots and thus we cannot access the state of a system's files of a specific date in the past. Dedicated backup tools create snapshots and retain a history of the changes, so we can choose what version to restore. However we need to be aware of the backup's size storage and remove older snapshots.


			== Steps we follow to backup AS LXC containers ==
			Note: all system paths and parameters in this HOWTO are fictional.

			=== Process ===
			* - stop the containers (because AS containers were created and managed by root user, they need root privlege to stop them. We solved this by allowing the specific lxc stop/start commands for the backup user via the sudoes.d configuration.)
			* - backup the containers
			* - restart the containers

			=== System setup ===
			* - a backup 'user'
			* - ssh keys to access remote backup repo
			* - password for restic backup command
			* - bash script with the necessary commands to run the process above
			* - cron job added via the backup user to run the bash script on regular intervals

			=== Configuration of the system setup ===
			==== 1. How to access the remote backup repo? ====
			* - create new ssh keys for ssh access to the remote backup repo, and give directory flag for saving the keys under the desired directory.
			* - scp the new pub key in the remote backup repo, under the remote user's home .ssh/authorized_keys. Remote user is the one we use to ssh to the remote machine.

			==== 2. How to run the backup command without root privelege ====
			when some of the filesystem to be backed-up is accessed only by root, aka execute a binary meant for root without being root? The idea is to execute the restic binary from backup user's home or from /usr/bin.

			<code> # useradd backupuser </code>
			# creates the new user and a group with the same name
			<code> # chown root:backupuser /usr/bin/restic</code>
			# makes user root and group backupuser owners of the restic binary
			<code> # chmod 750 /usr/bin/restic </code>
			# user root has now read, write, execute permissions, and users in backupuser group can execute and read the restic binary
			<code> # setcap cap_dac_read_search=+ep ~backupuser/bin/restic </code>
			# assigns capabilities to backup the whole system

			Ref: https://restic.readthedocs.io/en/stable/080_examples.html#backing-up-your-system-without-running-restic-as-root

			==== 3. How to run specific root commands by a non root user? ====
			Note: we needed that for stopping, checking status and starting of the lxc containers
			We should run our LXC containers rootless. This requires to change configuration of containers, lxc config files (to be checked/implemented in the future, tasks to do for AS, to be added in tasks in gitlab; Mika can you create a git issue for this too?)

			Ref: https://www.cyberciti.biz/faq/how-to-create-unprivileged-linux-containers-on-ubuntu-linux/

			But for now we will give the backupuser restricted root privelege for the specific lxc commands we need to run in the backup script.
			<code> # usermod -aG sudo backupuser </code>

			Give the backup user the acces for specific commands to be executed as 'root'.
			Add these commands in a new file under /etc/sudoers.d/
			<code> vi /etc/sudoers.d/00-backupuser </code>
			bintibackup ALL=(ALL) NOPASSWD: /usr/bin/lxc-stop, /usr/bin/lxc-start, /usr/bin/lxc-info, /usr/bin/lxc-ls
			Ref: https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/

			++++
			chown root:bintibackup /usr/bin/lxc-stop
			> this is for changing the property to usr/bin/lxc-stop to root group and bintibackup
			first, we are trying to make this happen with a bintibackup user with no root privilege - if it does no work, will switch to possibility to give the bintibackup user root provileges, but this will imply to have a vert strict security management policy for that userthis
			above scratched steps didn't work!
			++++

			==== One line command to run backups from terminal ====
			Needs the following parameters:

			path to password file:
			PASSWORD="~backupuser/pass"
			absolute path to the remote backup repo:
			remote-repo ="/var/backups"
			abosulte path to the directory we want to backup:
			local-repo="/var/foo"
			remote host:
			$host="backups.org"
			$user="backuphost"
			So the command becomes:
			<code> # restic -p ~/backup/pass -r sftp:backuphost@backups.org:/var/backups --verbose backup /var/foo </code>
			OR:
			<code> # restic -p $PASSWORD -r sftp:$user@$host:$remote-path --verbose backup </code>

			Final bash script (the exact paths haave been changed to fictional ones)
			<code>
			#!/bin/bash
			# a backup to our backup server of the LXC containers

			set -e
			PASSWORD=/root/mypassword_enter
			# Destination
			DEST="sftp:binti-backups:/server/backups"
			STATIC_OPTIONS="--verbose backup"
			ROOT="/var/lib/lxc"
			# list containers
			CONTAINERS="$(sudo lxc-ls)"
			declare -a STOPPED_CONTAINERS

			# Check if containers are running and then stop the containers
			echo $containers
			for container in ${CONTAINERS[@]}; do
			echo $container
			status="$(sudo lxc-info $container \| grep 'State' \| xargs \| cut -d' ' -f2)"
			echo $status
			if [[ $status == "STOPPED" ]]; then
			STOPPED_CONTAINERS+=("$container")
			elif [[ $status == "RUNNING" ]]; then
			sudo lxc-stop $container
			# Update the list of stopped containers
			STOPPED_CONTAINERS+=("$container")
			fi
			done
			set +e
			restic -p $PASSWORD -r $DEST $STATIC_OPTIONS $ROOT

			BACKUP_SUCCESS=$?
			set -e

			if [[ $BACKUP_SUCCESS -eq 0 ]]; then
			echo "BACKUP succeeded"
			else
			# Send email to admins
			echo "BACKUP failed"
			sendemail admin@backups.org < ./mail.txt
			fi

			for container in ${STOPPED_CONTAINERS[@]}; do
			echo $container
			sudo lxc-start $container
			done
			</code>
			== Resources for backup scripts and helpful tips ==
			restic for backups
			* https://restic.readthedocs.io/en/stable/040_backup.html
			duplicity
			* https://blog.xmatthias.com/duplicity_backup_script/
			What to backup
			* https://www.debian.org/doc/manuals/debian-reference/ch10.en.html#_backup_and_recovery
			Read password from a file:
			* https://www.foxinfotech.in/2019/03/reading-a-password-from-a-file-in-linux.html

	= Current setup =		= Current setup =

Ooooo: /* Current setup */

2021-04-04T16:59:17Z

Current setup

← Older revision		Revision as of 16:59, 4 April 2021
Line 9:		Line 9:
	Cita el Domingo 04 de Abril - 3PM CET time aqui: https://bbb.futuretic.fr/b/spi-utv-eo9-lqe		Cita el Domingo 04 de Abril - 3PM CET time aqui: https://bbb.futuretic.fr/b/spi-utv-eo9-lqe

	= Current setup =		= Current setup =

	~~Initial stage of script:~~
	~~#!/bin/bash~~
	~~# a backup to 1984 of the LXC containers~~
	~~set -e~~
	~~PASSWORD=/home/bintibackup/1984_enter~~
	~~# list containers~~
	~~containers=$(lxc-ls)~~
	~~# TODO: check if containers are running and then stop the containers~~
	~~for container in $containers~~
	do
	~~lxc-stop $container~~
	~~done~~
	~~restic -p $PASSWORD -r sftp:binti-backups:/srv/backups --verbose backup /var/lib/lxc~~
	~~# TODO: check if containers are stopped and then start the containers~~
	~~for container in $containers~~
	do
	~~lxc-start $container~~
	~~done~~

	~~</nowiki>~~

	= Current setup =		= Current setup =

Ooooo at 16:39, 4 April 2021

2021-04-04T16:39:23Z

← Older revision		Revision as of 16:39, 4 April 2021
Line 11:		Line 11:
	= Current setup =		= Current setup =

	~~<nowiki>~~		Initial stage of script:
	Initial stage of script:
	#!/bin/bash		#!/bin/bash
	# a backup to 1984 of the LXC containers		# a backup to 1984 of the LXC containers

Ooooo: /* Current setup */

2021-04-04T16:32:56Z

Current setup

← Older revision		Revision as of 16:32, 4 April 2021
Line 11:		Line 11:
	= Current setup =		= Current setup =

	<~~file~~>		<nowiki>

	Initial stage of script:		Initial stage of script:
	#!/bin/bash		#!/bin/bash
Line 32:		Line 31:
	done		done

	</~~file~~>		</nowiki>

	= Current setup =		= Current setup =

Ooooo: /* Current setup */

2021-04-04T16:31:24Z

Current setup

← Older revision		Revision as of 16:31, 4 April 2021
Line 10:		Line 10:

	= Current setup =		= Current setup =

			<file>

	Initial stage of script:		Initial stage of script:
Line 30:		Line 32:
	done		done

			</file>

	= Current setup =		= Current setup =

Ooooo at 16:28, 4 April 2021

2021-04-04T16:28:06Z

← Older revision		Revision as of 16:28, 4 April 2021
Line 11:		Line 11:
	= Current setup =		= Current setup =

	~~'''~~Initial stage of script:		Initial stage of script:
	~~'''~~
	#!/bin/bash		#!/bin/bash
	# a backup to 1984 of the LXC containers		# a backup to 1984 of the LXC containers

Ooooo at 16:26, 4 April 2021

2021-04-04T16:26:12Z

← Older revision		Revision as of 16:26, 4 April 2021
Line 11:		Line 11:
	= Current setup =		= Current setup =

	~~<code>~~		'''Initial stage of script:
	Initial stage of script:		'''

	#!/bin/bash		#!/bin/bash
	# a backup to 1984 of the LXC containers		# a backup to 1984 of the LXC containers

	set -e		set -e

	PASSWORD=/home/bintibackup/1984_enter		PASSWORD=/home/bintibackup/1984_enter

	# list containers		# list containers
	containers=$(lxc-ls)		containers=$(lxc-ls)

	# TODO: check if containers are running and then stop the containers		# TODO: check if containers are running and then stop the containers

	for container in $containers		for container in $containers
	do		do
	lxc-stop $container		lxc-stop $container
	done		done

	restic -p $PASSWORD -r sftp:binti-backups:/srv/backups --verbose backup /var/lib/lxc		restic -p $PASSWORD -r sftp:binti-backups:/srv/backups --verbose backup /var/lib/lxc

	# TODO: check if containers are stopped and then start the containers		# TODO: check if containers are stopped and then start the containers

	for container in $containers		for container in $containers
	do		do
	lxc-start $container		lxc-start $container
	done		done
	~~</code>~~

	= Current setup =		= Current setup =