A (kind of) how-to: MISP without Docker

I absolutely understand the appeal of containers. I use them regularly. But when it comes to hosting services I don't particularly like them - partially because my knowledge of working with them is far too small for me to be comfortable with exposing containers to the wider Internet, partially because I (personally, professional systems administrators will - likely rightfully so - disagree with me) find them cumbersome when it comes to debugging issues.

Knowing this about me, you can probably imagine my disappointment when I decided to run my own instance of MISP on a server outside my small homelab, only to find out that running it on "bare metal" wasn't actively discouraged, but not exactly encouraged either.

The official documentation for installing it without Docker lists Ubuntu 22.04 as the most recent operating system. In this case, 'most recent' can be read as 'is going to be unsupported in around six months'. There is an installation script for the current version of Debian Stable available in the repository on GitHub, but it makes some design decisions that I question on grounds of technical necessity (the version of composer in the Debian-repositories is reasonably recent) and personal preference (I haven't touched Apache in literal years, and I'd rather not do so unless absolutely necessary).

Because there was no comfortable, at least semi-automated way of installing MISP to my liking I went ahead and did it manually. I've documented the steps I have gone through in this post. In theory, the results should be reproducible on any system running Debian Trixie (and I might, at some point, turn everything into an Ansible-playbook), but this is not intended to be a step-by-step guide. See it more as inspiration, not instruction.

Prerequisites

MISP needs a number of packages in order to be able to run properly. Those need to be installed:

sudo apt-get install git curl python3 python3-pip python3-virtualenv gcc sudo binutils zip openssl supervisor libfuzzy-dev mariadb-server mariadb-client redis-server php8.4 php8.4-fpm php8.4-cli php8.4-dev php8.4-xml php8.4-mysql php8.4-opcache php8.4-readline php8.4-mbstring php8.4-zip php8.4-intl php8.4-bcmath php8.4-gd php8.4-redis php8.4-gnupg php8.4-apcu php8.4-curl composer

Database

As with most applications, you'll need a database and a user that may access it:

sudo mysql -e "CREATE DATABASE misp CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
sudo mysql -e "CREATE USER 'misp'@'localhost' IDENTIFIED BY '$password';"
sudo mysql -e "GRANT USAGE ON *.* TO 'misp'@'localhost';"
sudo mysql -e "GRANT ALL PRIVILEGES ON misp.* TO 'misp'@'localhost';"
sudo mysql -e "FLUSH PRIVILEGES;"

It might be my paranoia, but I've been bitten by this before - check that everything actually worked:

mysql -u "misp" -p -e "USE misp; SELECT 1;"

Webserver

PHP configuration

Strictly speaking, neither of those settings are necessary. But I would increase some limits in /etc/php/8.4/fpm/php.ini just to avoid unnecessary headaches later on:

upload_max_filesize=50M
post_max_filesize=50M
max_execution_time=300
max_input_time=300
memory_limit=2048M

Additionally, configure php-fpm to use redis for session handling:

session.save_handler = redis
session.save_path = 'tcp://localhost:6379'

Don't forget to restart php-fpm after the changes:

sudo systemctl restart php8.4-fpm

Virtual host

Please note: I'm assuming that you already have a TLS-certificate at hand. This post is not covering acquiring and renewing those.

Configure a virtual host at /etc/nginx/sites-enabled/misp.conf:

server {
    listen 80;
    listen [::]:80;
    server_name $mispdomain;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name $mispdomain;

    root /var/www/misp/app/webroot;
    index index.php;

    ssl_certificate SSL_CERT_PATH_PLACEHOLDER;
    ssl_certificate_key SSL_KEY_PATH_PLACEHOLDER;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    add_header Strict-Transport-Security "max-age=15768000; includeSubdomains" always;
    add_header X-Frame-Options DENY always;
    add_header X-Content-Type-Options nosniff always;

    client_max_body_size 50M;

    access_log /var/log/nginx/misp_access.log;
    error_log  /var/log/nginx/misp_error.log;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/run/php/php8.4-fpm.sock;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param HTTPS on;
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }
}

Installing MISP

Theoretically you can install MISP wherever you want, but I tend to stick to the old, trustworthy /var/www:

sudo git clone https://github.com/MISP/MISP.git /var/www/misp
cd /var/www/misp
sudo git fetch origin 2.5
sudo git config --global --add safe.directory /var/www/misp
sudo git -C /var/www/misp submodule update --init --recursive
sudo git -C /var/www/misp submodule foreach --recursive git config core.filemode false
sudo chown -R www-data:www-data /var/www/misp /var/www/misp/.git
cd /var/www/misp/app
sudo -u www-data composer install --no-dev --no-interaction --prefer-dist

Configuring MISP

Importing the database schemata

mysql -u misp -p misp < /var/www/misp/INSTALL/MYSQL.sql

Configuring PHP-settings

"Un-default" the configuration files:

cd /var/www/misp/app/Config
sudo -u www-data cp -a bootstrap.default.php bootstrap.php
sudo -u www-data cp -a database.default.php database.php
sudo -u www-data cp -a core.default.php core.php
sudo -u www-data cp -a config.default.php config.php

Then add the database credentials to database.php, set the baseurl in config.php. There are other settings that might be relevant for your setup as well, such as configuring a PGP-key, but I'm not going to cover this here.

Configure background workers (with supervisor)

MISP offloads long-running jobs (such as pulling feeds, cache regeneration, scheduled tasks) to background workers, managed by supervisor. For that, MISP needs to query supervisor over HTTP - which is disabled by default and has to be enabled in etc/supervisor/supervisord.conf:

[inet_http_server]
port=127.0.0.1:9001
username=misp
password=$password

Configure workers in /etc/supervisor/conf.d/misp-workers.conf:

[group:misp-workers]
programs=default,prio,email,update,cache,scheduler

[program:default]
directory=/var/www/misp
command=/var/www/misp/app/Console/cake start_worker default
process_name=%(program_name)s_%(process_num)02d
numprocs=5
autostart=true
autorestart=true
redirect_stderr=false
stderr_logfile=/var/www/misp/app/tmp/logs/misp-workers-errors.log
stdout_logfile=/var/www/misp/app/tmp/logs/misp-workers.log
user=www-data

[program:prio]
directory=/var/www/misp
command=/var/www/misp/app/Console/cake start_worker prio
process_name=%(program_name)s_%(process_num)02d
numprocs=5
autostart=true
autorestart=true
redirect_stderr=false
stderr_logfile=/var/www/misp/app/tmp/logs/misp-workers-errors.log
stdout_logfile=/var/www/misp/app/tmp/logs/misp-workers.log
user=www-data

[program:email]
directory=/var/www/misp
command=/var/www/misp/app/Console/cake start_worker email
process_name=%(program_name)s_%(process_num)02d
numprocs=5
autostart=true
autorestart=true
redirect_stderr=false
stderr_logfile=/var/www/misp/app/tmp/logs/misp-workers-errors.log
stdout_logfile=/var/www/misp/app/tmp/logs/misp-workers.log
user=www-data

[program:update]
directory=/var/www/misp
command=/var/www/misp/app/Console/cake start_worker update
process_name=%(program_name)s_%(process_num)02d
numprocs=1
autostart=true
autorestart=true
redirect_stderr=false
stderr_logfile=/var/www/misp/app/tmp/logs/misp-workers-errors.log
stdout_logfile=/var/www/misp/app/tmp/logs/misp-workers.log
user=www-data

[program:cache]
directory=/var/www/misp
command=/var/www/misp/app/Console/cake start_worker cache
process_name=%(program_name)s_%(process_num)02d
numprocs=5
autostart=true
autorestart=true
redirect_stderr=false
stderr_logfile=/var/www/misp/app/tmp/logs/misp-workers-errors.log
stdout_logfile=/var/www/misp/app/tmp/logs/misp-workers.log
user=www-data

[program:scheduler]
directory=/var/www/misp
command=/var/www/misp/app/Console/cake start_worker scheduler
process_name=%(program_name)s_%(process_num)02d
numprocs=1
autostart=true
autorestart=true
redirect_stderr=false
stderr_logfile=/var/www/misp/app/tmp/logs/misp-workers-errors.log
stdout_logfile=/var/www/misp/app/tmp/logs/misp-workers.log
user=www-data

Before running sudo systemctl restart supervisor, make sure that the directory supervisor uses for logging exists:

sudo -u www-data mkdir -p /var/www/misp/app/tmp/logs

Then point MISP's settings at supervisor and redis:

sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.enabled" 1
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.redis_host" "127.0.0.1"
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.redis_port" 6379
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.redis_database" 13
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.redis_namespace" "background_jobs"
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.supervisor_host" "localhost"
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.supervisor_port" 9001
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.supervisor_user" "misp"
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.supervisor_password" "$password"
sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "SimpleBackgroundJobs.redis_serializer" "JSON"

Configure virtual environment

Now it's time to set up the virtual environment and install the required Python-packages:

sudo -u www-data virtualenv -p python3 /var/www/misp/venv
cd /var/www/misp
sudo -u www-data /var/www/misp/venv/bin/pip install -r /var/www/misp/requirements.txt
sudo chown -R www-data:www-data /var/www/misp/venv

sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "MISP.python_bin" /var/www/misp/venv/bin/python

Create initial user

sudo -u www-data /var/www/misp/app/Console/cake Admin setSetting "MISP.osuser" www-data
sudo -u www-data /var/www/misp/app/Console/cake Admin runUpdates
sudo -u www-data /var/www/misp/app/Console/cake User init

The last command prints the default admin's API-key, you should note that down. I'd also strongly suggest immediately changing the password of the default account:

sudo -u www-data /var/www/misp/app/Console/cake User change_pw admin@admin.test $password

Note: Unless you avoid it with "tricks" like leaving a space before entering the command, this will leave the password in your shell history. Ye be warned.

Final cleanup

sudo chmod -R 750 /var/www/misp
sudo chmod -R g+ws /var/www/misp/app/tmp
sudo chmod -R g+ws /var/www/misp/app/files
sudo chmod -R g+ws /var/www/misp/app/files/scripts/tmp
sudo systemctl restart php8.4-fpm

While restarting php-fpm isn't strictly necessary, I've run into some weird caching errors when not doing it .. I admit that I am too lazy to debug that further, I'll just roll with the restart.

After all is said and done you have a basic install of MISP running without any containers involved. I highly recommend consulting the official documentation for further configuration steps.