Hetzner Media Server
This guide aims to explain how to set up a secure media server on a hetzner media box.
LinuxTech

update dec 2018

I actually had another go at this. I spent forever putting together a complex docker-compose configuration for it. In the end I just couldn't get reliable bandwidth between the server, my home connection, or peers... it was a lot of work which was ultimately fruitless.

update feb 2017

I've since moved away from using a hetzner media server, so have no intention of maintaining / updating this guide.

front matter

synopsis

This guide aims to explain how to set up a secure media server on a hetzner media box.

features

Right now it explains how to install / setup

  • sabnzbd, rutorrent, sickrage, couchpotato, plex
  • firewall & fail2ban
  • openvpn
  • passwordless ssh
  • nginx reverse proxies
  • ssl for all services
  • user certificate authentication (no passwords for sab et al)

issues

Things that don't work / aren't included

  • openvpn config does work, but it should only be used for rtorrent's connections
  • rtorrent / rutorrent doesn't work at all. something to do with scgi
  • couchpotato browser plugin
  • plex apps (only browser will work)
  • nzb push for dognzb
  • encrypted partition for media library
  • couchpotato post processing on completed download.

fabric

In the course of writing this guide I played around with ways I might automate this setup. I really don't want to invest any time in it at the moment, but I did play around with a fab file momentarily. It's designed to be run on the remote server itself, not initiated remotely.

This is what I have so far (not much)

env.hosts = ['localhost']
env.port = 22022
env.use_sudo = True
env.user = 'reginald'
env.key_filename = '/home/reginald/.ssh/fab'
packages = [
    'ufw',
    'fail2ban',
    'etckeeper',
    'nginx',
    'git',
]

def setup():
    global packages
    upload_template(
        'templates/etc/apt/sources.list',
        '/etc/apt/sources.list',
        use_sudo=True,
        backup=True,
        chown=True,
        user='root',
    )
    update_index()
    upgrade()
    require.deb.packages(packages)

def nginx():
    require.deb.package('nginx')
    upload_template(
        'templates/etc/nginx/conf.d/gotr.conf'
        '/etc/nginx/conf.d/gotr.conf',
        use_sudo=True,
        backup=True,
        chown=True,
        user='www-data',
    )

nameology

This is a list of everything in the commands below which has been replaced. If you copy this guide, then you should be able to use an editor to replace every instance of these names, and then copypasta the commands.

hostname : chappy - as in the name of the machine personal user : reginald - the username you'll log in with utility user : chappy - the linux user which will run services fqdn : chappy.domain.com - the domain name you'll type in your browser to access your server box public IP : 55.55.55.55 your public IP : 66.66.66.66 box default gateway : 77.77.77.77

hetzner

choose a server, buy it, specify public key. Ignore all the moaning about rescue mode. Once you get the order confirmation you have to wait a few minutes for it to be provisioned before the server will show up in your list on hetzner. Hit the linux tab and choose your OS (I'm a debian guy) then you do need to reset (see reset tab) to initiate the clone.

hostname

/etc/hostname

chappy

/etc/hosts

127.0.0.1 localhost.localdomain localhost
55.55.55.55 chappy.domain.com chappy

Then you can hostname -f and you should see something like chappy.domain.com

users

root@chappy ~ # adduser reginald sudo
Adding user `reginald' ...
Adding new group `reginald' (1000) ...
Adding new user `reginald' (1000) with group `reginald' ...
Creating home directory `/home/reginald' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for reginald
Enter the new value, or press ENTER for the default
  Full Name []:
  Room Number []:
  Work Phone []:
  Home Phone []:
  Other []:
Is the information correct? [Y/n]

ssh key

Hetzner didn't seem to want to copy my key for me even though I uploaded it to their ui. So I just chose the password option and let them generate my login password for me.

Once users are set up you can log out, and on your local machine generate an ssh key as in ssh-keygen -t rsa and specify a path like /home/reginald/.ssh/chappy when asked. Then upload that key with ssh-copy-id -i ~/.ssh/chappy [email protected].

Thereafter you should be able to log in with ssh -i ~/.ssh/chappy [email protected]

hardening

Don't be tempted to "deal with security later", do it now.

ssh

You can find your ssh config in /etc/ssh/sshd_config. I'm not going to post the whole thing here, but set these:

  • Port 22022
  • PermitRootLogin no
  • PasswordAuthentication no

then systemctl restart ssh

firewall

There's loads of iptables helpers, I like ufw. I'm just going to deny everything, then allow anything from my current IP address, or to my custom ssh port 22022.

x apt-get install ufw
ufw default deny incoming
ufw allow from 66.66.66.66
ufw allow 22022
ufw enable

fail2ban

x apt-get install fail2ban nano /etc/fail2ban/jail.conf

scroll down to service definitions and change ssh port to your custom port number

openvpn

don't forget to configure DNS

x apt-get install openvpn

setting route

ip route add table 42 default via 77.77.77.77 ip rule add from 55.55.55.55/32 table 42

removing rules

ip rule del from 55.55.55.55 table 42 ip route flush table 42

testing connection

openvpn --config ./DE_Germany.ovpn --script-security 2 wget -qO- http://ipecho.net/plain ; echo

informational

ip route show ip rule list openvpn --show-gateway

ssl keys & user certs

may as well get this out of the way now, even though these keys are only used by nginx which we'll install later. SSL is the magic behind https which you've all seen and used, but user certs aren't very common so I'll explain the reasoning here. Anyone who want's to access this server will be issued a user certificate, they import the certificate into their browser, and it identifies that user to your server. There's a number of advantages to this:

  • no passwords to remember (except for plex)
  • all your services are secured by nginx, which is far more battle tested than sab & friends
  • less configuration in general

You should just be able to follow these commands, but if you're interested in what you're doing, there's a few terms to know:

KEY: Private Key CSR: Certificate Signing Request - contains your public key, and identity details CRT: Certificate - the signed CSR

CA: Certificate Authority - this isn't a single file, it's a key which is used to 'sign' csr files to create the crt. Thereafter a crt can be used to identify the bearer. A CA can determine whether it has signed a given crt.

So.. to business. This will create a certificate authority, as well as the ssl certs for nginx to use for https

mkdir /etc/ssl/nginx && cd /etc/ssl/nginx &&
openssl genrsa -out ca.key 4096 &&
openssl req -new -x509 -days 365 -key ca.key -out ca.crt -subj "/O=Chappy/CN=Chappy CA" &&
openssl genrsa -out server.key 1024 &&
openssl req -new -key server.key -out server.csr -subj "/O=Chappy/CN=*.chappy.domain.com" &&
openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -CAserial ca.srl -out server.crt

We're also going to create user certificates, this is basically an authentication layer which runs on top of that provided by sabnzbd, couchpotato, plex, or whatever other services we're providing. Users need to import their certificate file into their browser before they can access your services.

Just so you don't need to remember the command to create a new certificate, we can create a bash script to create new certs

/etc/ssl/nginx/new-cert.sh

if [ -z "$1" ];
then echo "usage /etc/ssl/nginx/new-cert.sh <username>";
else
  openssl genrsa -out "$1".key 4096
  openssl req -new -key "$1".key -out "$1".csr -subj "/O=Chappy/CN=$1"
  openssl x509 -req -days 365 -in "$1".csr -CA ca.crt -CAkey ca.key -CAcreateserial -CAserial ca.srl -out "$1".crt
  openssl pkcs12 -export -clcerts -in "$1".crt -inkey "$1".key -out "$1".p12
fi

make executable

chmod 700 /etc/ssl/nginx/new-cert.sh

then create a cert for reginald, repeat for whatever users you like. You'll be asked for an export password here. I suspect that firefox wont allow imported certs with no password, but I need to confirm this.

./new-cert.sh reginald

Now, anyone wanting to access services needs a copy of the ca.crt and username.p12 files you just created. In their prowser preferences (under advanced for both firefox and chrome) there's a 'view certificates' button. You import the ca.crt in the authorities section, the browser will ask whether you want to trust certs issued by that authority, obviously we say yes. Then in the "your certificates" section import the p12 you created, input export password used during creation.

sabnzbd

apt-get install python2.7 python-cheetah python-support python-cryptography python-yenc python-dbus unzip p7zip p7zip-rar unrar
apt-get install devscripts build-essential git debhelper dh-autoreconf libtbb-dev

do some multithreaded par2 magic.

DIR="$(mktemp -d)"
cd "$DIR"
git clone https://github.com/jcfp/debpkg-par2tbb.git
cd debpkg-par2tbb
uscan --force-download
dpkg-buildpackage -S -us -uc -d
dpkg-source -x ../par2cmdline-tbb_*.dsc
cd par2cmdline-tbb-*
dpkg-buildpackage -b -us -uc
echo; echo "to install, run: sudo dpkg -i $(readlink -f ../par2-tbb_*.deb)"

Notice that the last line here is just going to echo the path to the deb file you just built, so you'll need to dpkg -i /tmp/tmp.cD16VxMJv3/debpkg-par2tbb/par2-tbb_0.4+20150503-1_amd64.deb or so.

git clone https://github.com/sabnzbd/sabnzbd
cd sabnzbd
git checkout tags/1.1.0
groupadd chappy
useradd chappy -g chappy -d /opt

This is my system ctl unit definition, based on linux/[email protected] in the repo

[Unit]
Description=SABnzbd binary newsreader
Documentation=https://sabnzbd.org/wiki/
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/opt/sabnzbd/SABnzbd.py --logging 1 --browser 0
User=chappy
Type=simple
Restart=on-failure

[Install]
WantedBy=multi-user.target

ln /opt/.sabnzbd/sabnzbd.service /lib/systemd/system/sabnzbd.service &&
systemctl enable sabnzbd &&
systemctl start sabnzbd &&
systemctl status sabnzbd

sickrage

apt-get install unrar-free git-core openssl libssl-dev python2.7
git clone https://github.com/SickRage/SickRage.git /opt/sickrage
cp ./sickrage/runscripts/init.systemd ./.sickrage/sickrage.service
nano ./.sickrage/sickrage.service
ln /opt/.sickrage/sickrage.service /lib/systemd/system/sickrage.service
chown -R chappy:chappy /opt/*

when editing sickrage.service just change user & group from sickrage to chappy

couchpotato

apt-get install python-lxml pip &&
apt-get install build-essential libssl-dev libffi-dev python-dev &&
pip install --upgrade cffi &&
pip install cryptography pyopenssl &&
git clone &&
mkdir /opt/.couchpotato &&
cp /opt/CouchPotatoServer/init/couchpotato.service /opt/.couchpotato/couchpotato.service &&
ln /opt/.couchpotato/couchpotato.service /lib/systemd/system/couchpotato.service

[Unit]
Description=CouchPotato application instance
After=network.target

[Service]
ExecStart=/opt/CouchPotatoServer/CouchPotato.py
Type=simple
User=chappy
Group=chappy

[Install]
WantedBy=multi-user.target

plex

wget https://downloads.plex.tv/plex-media-server/0.9.12.4.1192-9a47d21/plexmediaserver_0.9.12.4.1192-9a47d21_amd64.deb
dpkg -i plexmediaserver_0.9.12.4.1192-9a47d21_amd64.deb
service plexmediaserver stop
nano /etc/default/plexmediaserver

change user to chappy

service plexmediaserver start

rtorrent & rutorrent

apt-get install gcc pkg-config libssl-dev g++ make libncurses5-dev libsigc++-2.0-dev libcurl4-openssl-dev screen nano php5-fpm &&
DIR="$(mktemp -d)" && git clone https://github.com/mirror/xmlrpc-c.git "$DIR" && cd "$DIR"/stable &&
./configure &&
make &&
make install &&
DIR="$(mktemp -d)" && git clone https://github.com/rakshasa/libtorrent.git "$DIR" && cd "$DIR" &&
autoreconf --install &&
autoconf &&
./configure &&
make &&
install &&
DIR="$(mktemp -d)" && git clone https://github.com/rakshasa/rtorrent.git "$DIR" && cd "$DIR" &&
./autogen.sh &&
./configure --with-xmlrpc-c &&
make &&
make install &&
DIR="/var/www/html/rutorrent" && git clone https://github.com/Novik/ruTorrent.git "$DIR" &&
chmod -R 775 "$DIR"

usermod -a -G chappy www-data
mkdir rtorrent

/opt/rtorrent/rtorrent.service

[Unit]
Description=rTorrent
Requires=network.target local-fs.target

[Service]
Type=oneshot
RemainAfterExit=yes
KillMode=none
User=chappy
ExecStart=/usr/bin/tmux new-session -s rt -n rtorrent -d rtorrent
ExecStop=/usr/bin/tmux send-keys -t rt:rtorrent C-q
WorkingDirectory=/opt/rtorrent

[Install]
WantedBy=multi-user.target

/opt/.rtorrent.rc

scgi_port = 5000
execute.nothrow = rm,~/.rtorrent.sock
network.scgi.open_local = /opt/rtorrent/.rtorrent.sock
schedule = socket_chmod,0,0,"execute=chmod,0660,/opt/rtorrent/.rtorrent.sock"
schedule = socket_chgrp,0,0,"execute=chgrp,chappy,/opt/rtorrent/.rtorrent.sock"

/etc/php5/fpm/pool.d/rutorrent.conf

[rutorrent]
user = chappy
group = chappy
listen = /var/run/php-fpm-rutorrent.sock
listen.owner = chappy
listen.group = chappy
listen.mode = 0660
pm = static
pm.max_children = 2
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
chdir = /

sed -i.backup -e 's/;cgi\.fix_pathinfo=1/cgi\.fix_pathinfo=1/' /etc/php5/fpm/php.ini
ln /opt/rtorrent/rtorrent.service /lib/systemd/system/rtorrent.service

nginx

apt-get install nginx &&
cd /etc/nginx

/etc/nginx/conf.d/server.conf

# ======================================================================= ssl ==
ssl_certificate      /etc/ssl/nginx/server.crt;
ssl_certificate_key  /etc/ssl/nginx/server.key;
ssl_client_certificate /etc/ssl/nginx/ca.crt;
ssl_verify_client on;
ssl_session_timeout  5m;
ssl_ciphers  HIGH:!aNULL:!MD5;

# ============================================================ something else ==
client_max_body_size 10m;
client_body_buffer_size 128k;

# ======================================= =Timeout if the real server is dead ==
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;

# Advanced Proxy Config
send_timeout 5m;
proxy_read_timeout 240;
proxy_send_timeout 240;
proxy_connect_timeout 240;

# ======================================================== Basic Proxy Config ==
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect  http://  $scheme://;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_cache_bypass $cookie_session;
proxy_no_cache $cookie_session;
proxy_buffers 32 4k;

/etc/nginx/sites-available/server

server {
  listen 80;
  listen 443 ssl;
  server_name sabnzbd.chappy.domain.com;
  if ($scheme = http) {
    return 301 https://$server_name$request_uri;
  }
  location / {
    proxy_pass http://localhost:8080;
  }
}
server {
  listen 80;
  listen 443 ssl;
  server_name sickrage.chappy.domain.com;
  if ($scheme = http) {
    return 301 https://$server_name$request_uri;
  }
  location / {
    proxy_pass http://localhost:8081;
  }
}
server {
  listen 80;
  listen 443 ssl;
  server_name couchpotato.chappy.domain.com;
  if ($scheme = http) {
    return 301 https://$server_name$request_uri;
  }
  location / {
    proxy_pass http://localhost:5050;
  }
}
server {
  listen 80;
  listen 443 ssl;
  server_name plex.chappy.domain.com;
  if ($http_x_plex_device_name = '') {
    rewrite ^/$ http://$http_host/web/index.html;
  }
  proxy_set_header Host $http_host;
  proxy_redirect off;
  if ($scheme = http) {
    return 301 https://$server_name$request_uri;
  }
  location / {
    proxy_pass http://localhost:32400;
  }
}

server {
  listen 80;
  listen 443 ssl;
  server_name rutorrent.chappy.domain.com;
  if ($scheme = http) {
    return 301 https://$server_name$request_uri;
  }
  root /opt/ruTorrent;
  index index.html index.htm index.php;

  location / {
    try_files $uri $uri/ =404;
  }

  location ~ \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_index index.php;
    include fastcgi_params;
  }

  location /RPC2 {
    include /etc/nginx/scgi_params;
    scgi_pass 127.0.0.1:5000;
  }
}

ln -s /etc/nginx/sites-available/server /etc/nginx/sites-enabled/server
systemctl nginx reload

file structure

cd /srv && mkdir movies mkdir tv mkdir blackhole mkdir downloads cd blackhole mkdir tv mkdir movies chown -R chappy:chappy /srv/*

sabnzbd config

head to sabnzbd.chappy.domain.com and fire up the wizard you just choose a language and input details of a primary usenet server, then sab restarts. The link resented probably won't be correct, so you'll have to go back to sabnzbd.chappy.domain.com. You'll see some errors, don't worry about that.

general

  • SABnzbd host : leave this as localhost, or it will mess up your reverse proxy
  • maybe copy API key and NZB key to save you coming back later 1c8f4b1fa32d62aada5a9899ccd8b206 d4177e7d23d39161c4a989f3549451e3
  • enable https : no, nginx is doing that for us

folders

  • Temporary Download Folder: /srv/downloads/incomplete
  • Minimum Free Space: 10G
  • Completed Download Folder: /srv/downloads
  • Permissions : blank
  • Watched Folder: /srv/blackhole

servers

you know what to do here

categories

  • in the last row set category, folder/path, and groups to 'movies', then add.
  • repeat for 'tv'

switches

  • Check before download: yes

others

as required

sickrage config

general > misc

launch browser: no show root directories: new > '/srv/tv'

general > interface

no required changes here. No need for https or user settings as nginx is taking care of that

search settings this section is a bit weird, there's three tabs, each with a "save settings" button at the bottom, but then there's like a master "save settings" below that. As you complete each tab, use that tab's button. Then once you're done with the section, hit the master button down the bottom or else your settings won't stick.

search settings > NZB search

search nzbs: enable send .nzb files to: black hole black hole folder: /srv/blackhole/tv

search settings > torrent search

send .torrent files to: black hole black hole foldeR: /srv/blackhole/tv

search providers

you know what to do.

post processing

post processing dir: /srv/downloads/tv processing method: move

plex config

I had some drama with this. The instance on this machine will not allow you to connect until you first sign in from a local connection.

I'm not sure whether this can be achieved using the nginx proxy

In my own case I fired up ssh on my local machine with:

ssh -p 22022 -L 32400:localhost:32400 -N [email protected]

then pointed my browser at localhost:32400, logged in with my plex account, in Settings > Server > General and thereafter I could log in fine using plex.chappy.domain.com