Installing and Configuring CrowdSec in an LXC Container Alongside Traefik

CrowdSec is a modern cybersecurity solution capable of protecting your server against attacks, scanners, and brute-forcing. In this guide we’ll cover installing CrowdSec inside an LXC container in a Proxmox VE environment.

If you enjoyed this article, you can support the author by becoming a sponsor on Boosty (link in the contacts section).

You might ask, hey stilicho2011, didn’t you already write an article about this? True, but there’s a nuance, as they say. The article you can find at that link describes the situation where we install everything in Docker.

This time we’ll talk about the situation where our reverse proxy is installed in an LXC container, which I covered here and here.

But since we’ve started doing these strange things, it would be wrong to stop halfway. So if we’ve installed Traefik into an LXC container, we also need to take care of it with the modern protection that Crowdsec gives us.

Of course, the general operating principle of Crowdsec stays the same regardless of the installation method, so to keep this article shorter, I’ll only cover the things specific to installing Crowdsec in an LXC container and hooking it up to the Traefik reverse proxy.

What is CrowdSec?

#

CrowdSec, as mentioned above, is a modern open-source security system built on principles similar to a “crowdsourced firewall.” It analyzes logs, detects suspicious activity (such as brute-force attempts, port scanning, DDoS), and automatically applies decisions to block or limit access for malicious IP addresses.

CrowdSec’s main feature is sharing data about malicious IP addresses among all users of the system, which builds a collective knowledge base formed from data contributed by all participants. So if one server detects an attack, other participants can block the attacker preemptively.

Since there’s already a much more detailed article about the app’s functionality on the site, I won’t dwell on every feature of Crowdsec here, with your permission.

But by the laws of the genre, I’m obligated to cover the key architectural features of Crowdsec.

CrowdSec architecture

#

CrowdSec is built on a modular architecture and is globally split into two layers:

1. Agent (analysis engine) - analyzes log journals (Nginx, Traefik, SSH, Postfix, system logs, etc.).

2. Bouncers (blocking modules) - apply blocking decisions (ban, captcha, throttle) to protect or allow access (for example, iptables, Nginx middleware, Traefik bouncer).

The Agent only handles detecting bad IPs, while blocking is carried out by the bouncers.

This means CrowdSec can be easily integrated into any infrastructure without directly touching network rules.

We already have a working reverse proxy, Traefik, installed and configured in an LXC container. We’re going to install Crowdsec into that same Debian-based container. Once again, there’s a detailed article on the site about configuring Crowdsec alongside Traefik in Docker.

Installing Crowdsec on Linux

#

The first thing we need to do is install the Crowdsec repository on the system. This can be done manually, but it’s long and tedious. So we’ll use the official script prepared by the Crowdsec developers. This repository contains the latest stable version of Crowdsec, and this is the installation method recommended by the developer.

curl -s https://install.crowdsec.net | sudo sh

Let’s verify that the repository was set up and the system can see Crowdsec versions.

apt list crowdsec

Let’s install Crowdsec with the unsurprising command

apt install crowdsec

Just a reminder: the so-called security engine on its own can only detect suspicious/malicious activity, but not block the IPs behind it. That’s why we need to install the corresponding bouncers.

There’s one important point here. If you’re installing everything in a privileged LXC container, then before configuring our setup and testing whether it works, we need to first, without waiting for peritonitis, install a standalone bouncer for iptables. This bouncer will directly protect our LXC container from brute-force attacks. Let’s do this with the command

apt install crowdsec-firewall-bouncer-iptables

This needs to be done if, for some inexplicable reason, you feel like taking the risk of installing things inside a privileged LXC container. Otherwise, there’s no need to install the iptables bouncer (but no promises). And now let’s move on to the final configuration.

Once we’ve installed Crowdsec

Let’s check CrowdSec’s status.

systemctl status crowdsec

Let’s create a whitelist of addresses that Crowdsec shouldn’t check. Typically these are private networks, but if you need finer-grained control, you can add specific trusted IP addresses.

Create the list with the command

nano /etc/crowdsec/parsers/s02-enrich/01-my-whitelist.yaml

and add the corresponding values

name: crowdsecurity/my-whitelists
description: "Whitelist events from my ipv4 addresses"
#it's a normal parser, so we can restrict its scope with filter
filter: "1 == 1"
whitelist:
  reason: "my ipv4 ranges"
  ip: 
    - "127.0.0.1"
  cidr:
    - "192.168.0.0/16"
    - "10.0.0.0/8"
    - "172.16.0.0/12"

This way all requests coming from private IP addresses won’t be checked by crowdsec. This removes unnecessary load from the application, and rules out cases where crowdsec might ban our private IPs because frequent requests could be interpreted as a brute-force attack.

Restart the service so the changes take effect

systemctl restart crowdsec

The next step is to connect to the Crowdsec console on their website.

It’s a simple procedure, everything there is logically laid out.

Step-by-step guide from Crowdsec on their site

This is needed so we can connect to the global network for sharing malicious IPs. One of the undeniable perks of Crowdsec.

Yes, the developers will get information about whether you’ve been attacked. But this isn’t the most sensitive information out there, and given how Crowdsec works, I think connecting to the console is worth it.

We did everything per the instructions: registered; chose the OS we’re running (Linux, in our case); got the enrollment key and the command that needed to be typed into our container’s command line; restarted the service; went back to the Crowdsec website and saw that we could pick 3 free blocklists.

The most interesting part, of course, is the information about the crowdsec hub and the list of collections depending on the service you want to protect.

Let’s check which collections are currently present in the system, and check the values in the acquis.yaml file, to see whether our crowdsec instance is ready to parse logs.

Checking the list of collections

cscli collections list

Checking the list of logs that Crowdsec is parsing

cscli metrics show acquisition

Definitely none of the collections we need are there. Let’s go ahead and install them.

In this guide we’ll use the following collections:

crowdsecurity/traefik traefik support: parser and generic http scenarios

crowdsecurity/http-cve Detect CVE exploitation in http logs

crowdsecurity/base-http-scenarios http common : scanners detection

crowdsecurity/sshd support : parser and brute-force detection

crowdsecurity/linux core linux support : syslog+geoip+ssh

crowdsecurity/appsec-generic-rules A collection of generic attack vectors for additional protection

crowdsecurity/appsec-virtual-patching a generic virtual patching collection, suitable for most web servers.

crowdsecurity/appsec-crs Appsec: Modsecurity core rule set rules

Let’s install the collections with the command

cscli collections install crowdsecurity/traefik crowdsecurity/http-cve crowdsecurity/base-http-scenarios crowdsecurity/sshd crowdsecurity/linux crowdsecurity/appsec-generic-rules crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-crs

Now we need to tell Crowdsec how and where to parse the local logs of our Linux machine and the logs of the Traefik reverse proxy. To do this we need to edit the acquis.yaml configuration file at the corresponding path, sudo nano /etc/crowdsec/acquis.yaml. We’re also specifying the port our WAF will run on.

Let’s create a sample file that looks like this:

filenames:
    - /var/log/traefik/*.log
labels:
    type: traefik
---
filenames:
    - /var/log/syslog
    - /var/log/auth.log
labels:
    type: syslog
---
listen_addr: 0.0.0.0:7422
appsec_config: crowdsecurity/appsec-default
name: myAppSecComponent
source: appsec
labels:
    type: appsec
#
#filenames:
# - /var/log/authentik.log
#labels:
#  type: authentik
#---
#filenames:
# - /var/www/nextcloud/data/nextcloud.log
#labels:
#  type: Nextcloud

For the sake of a clean experiment, let’s restart the service.

systemctl restart crowdsec

Checking the list of logs that Crowdsec is parsing

cscli metrics show acquisition

Let’s install the bouncer for our Traefik reverse proxy.

If you followed my guide on installing Traefik in an LXC container, the corresponding bouncer, as a plugin, is already set up in the static config of your reverse proxy, downloaded, and installed. It just doesn’t yet know what to do.

First we’ll get the corresponding API key, so the bouncer can talk to Crowdsec.

Let’s do this with the command

cscli bouncers add traefik-bouncer

Copy the resulting value into a notepad. You’ll only see the key value once, don’t forget that.

Now let’s go into the dynamic configuration of our reverse proxy, at the location

nano /etc/traefik/dynamic/config.yaml

If you followed my article on configuring Traefik in an LXC container, your dynamic configuration file looks roughly like this.

http:
# https://doc.traefik.io/traefik/routing/routers/
  routers:
   # harden dashboard access: can only be accessed with a username/password
    dashboard:
      entryPoints:
        - websecure 
      rule: "Host(`traefik-dashboard.domain.ru`)"
      service: api@internal
      middlewares:
        - auth
      tls:
        certResolver: cloudflare  

    radarr:
      entryPoints:
        - "websecure"
      rule: "Host(`radarr.domain.ru`)"        
      middlewares:
        - default-headers
        - https-redirect
      tls:
        certResolver: cloudflare
      service: radarr

# https://doc.traefik.io/traefik/routing/services/
  services: 
    radarr:
      loadBalancer:
        servers:
          - url: "http://192.168.x.x:7878" # Change IP Address to your radarr instance
        passHostHeader: true

# https://doc.traefik.io/traefik/middlewares/http/overview/
  middlewares:
# Middleware for Redirection
# This can be used instead of global redirection
#
    auth:
      basicAuth:
        users:    # users and their MD5 hashed passwords, granted access to the traefik dashboard 
          - "admin:hashedpassword" # openssl passwd -1 "hashedpassword"
#
    https-redirect:
      redirectScheme:
        scheme: https
        permanent: true
#
    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https
#
    crowdsec:
      plugin:
        bouncer:
          enabled: true
          logLevel: INFO
          updateIntervalSeconds: 15
          updateMaxFailure: 0
          defaultDecisionSeconds: 15
          httpTimeoutSeconds: 10
          crowdsecMode: stream
          crowdsecAppsecEnabled: true
          crowdsecAppsecHost: localhost:7422
          crowdsecAppsecFailureBlock: true
          crowdsecAppsecUnreachableBlock: true
          crowdsecLapiKey: yourlapikey # Replace CrowdSec API key (docker exec crowdsec cscli bouncers add crowdsecBouncer)
          crowdsecLapiHost: localhost:8080
          crowdsecLapiScheme: http
          forwardedHeadersTrustedIPs:
            - 10.0.0.0/8
            - 172.16.0.0/12
            - 192.168.0.0/16                                                        
          clientTrustedIPs:
            - 10.0.0.0/8
            - 172.16.0.0/12
            - 192.168.0.0/16

# https://doc.traefik.io/traefik/https/tls/
tls:
 options:
   default:
     minVersion: VersionTLS12    # change to a lower version if you expect to service Internet traffic from around the world
     curvePreferences:   # below priority sequence can be changed
       - X25519     # the most commonly used 128-bit
       - CurveP256  # the next most commonly used 128-bit
       - CurveP384  # 192-bit
       - CurveP521  # 256-bit
     sniStrict: true     # true if our own certificates should be enforced
     cipherSuites:
       - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
       - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
       - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
       - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
       - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
#  certificates:
#    - certFile: /etc/traefik/domain.cert
#      keyFile: /etc/traefik/domain.key
#    - certFile: /etc/traefik/certificate.pem
#      keyFile: /etc/traefik/private_key.pem
#### Traefik uses its own default certificate for connections without SNI, or without a matching domain.
#### However, we can provide our own default certificate, instead of using the Traefik default.
#  stores:
#    default:
#      defaultCertificate:
#        certFile: /etc/traefik/cert.crt
#        keyFile: /etc/traefik/cert.key
#### Alternatively, we can use an ACME generated default certificate.
stores:
  default:
    defaultGeneratedCert:
      resolver: cloudflare
      domain:
        main: domain.ru
        sans:
          - "*.domain.ru"

As you can see, we’ve set certain parameters in the middleware. There are, as I mentioned above, many, many more configuration options available. You can read more about it in the official documentation.

Make sure to fill in the corresponding field with the value of your API key that we created earlier.

Also note that we set values for forwardedHeadersTrustedIPs and clientTrustedIPs. This defines the private IPv4 subnets as trusted IP addresses.

This is necessary because we want to trust the HTTP headers of our Traefik reverse proxy, such as X-Forwarded-For and X-Real-IP. These headers typically carry the real IP addresses of visitors to our site - and, of course, attackers too - which CrowdSec uses to make its blocking decisions.

You can fine-tune this further and add Traefik’s exact IP address in a /32 range. However, whitelisting all the private-class ranges is the more convenient approach. This especially applies to the dynamic IP addresses of Docker subnets, which can change on container restart.

With the clientTrustedIPs variable, we specify the trusted IP addresses or ranges we consider trustworthy. These IP addresses or network ranges won’t be blocked or denied by CrowdSec. In effect, this is a whitelist-based option, one we’ve already set up. To some extent this duplicates what we’ve already done, but I’m obligated to show all the options.

In my case I trust the local LAN, which is reflected by adding all the private-class ranges to the whitelist. You can restrict them to your current LAN subnet range (CIDR) if needed.

Since these are Traefik’s dynamic settings, there’s no need to restart it. It’ll pick them up automatically.

Now we can protect our services with Crowdsec selectively, by specifying the - crowdsec@file middleware in the routers section of our reverse proxy, or we can do it globally. That is, all incoming requests to our reverse proxy will be processed by Crowdsec. I think this is the more correct approach.

For example, the static configuration file for Traefik might look like this:

# Traefik entrypoints (network ports) configuration
entryPoints:
  web:
    address: :80
    forwardedHeaders:
      trustedIPs: &trustedIps
        # Start of Cloudlare's public IP list
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 104.16.0.0/13
        - 104.24.0.0/14
        - 108.162.192.0/18
        - 131.0.72.0/22
        - 141.101.64.0/18
        - 162.158.0.0/15
        - 172.64.0.0/13
        - 173.245.48.0/20
        - 188.114.96.0/20
        - 190.93.240.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 2400:cb00::/32
        - 2606:4700::/32
        - 2803:f800::/32
        - 2405:b500::/32
        - 2405:8100::/32
        - 2a06:98c0::/29
        - 2c0f:f248::/32
        # End of Cloudlare public IP list
    http:
      redirections:
        entryPoint:
          to: web
          scheme: https

  # HTTPS endpoint, with domain wildcard
  websecure:
    address: :443
    forwardedHeaders:
      # Reuse the list of Cloudflare's public IPs from above
      trustedIPs: *trustedIps
    http:
      middlewares:
        - crowdsec@file # reference to a dynamic middleware for enabling crowdsec bouncer

You can read more about this in the article dedicated to setting up Crowdsec in Docker, since I want to avoid duplicating information that’s already on the site.

Now there’s just the question of verifying that our setup works.

The command will show us CrowdSec’s metrics. The important thing is that CrowdSec is parsing the reverse proxy’s logs. If it isn’t, restart Traefik and check again.

cscli metrics

Let’s ban our address from the private network.

cscli decisions add --ip 192.168.x.x

Let’s check whether we’ve been added to the ban list.

cscli decisions list

Our IP should show up in the list, which we manually added to the ban.

Note that in practice it might not actually be banned, because this IP is also in our whitelist.

Now let’s unban it without waiting the 4 hours.

cscli decisions delete --ip 192.168.x.x

Let’s check the result.

cscli decisions list

Our IP should no longer be in the list of banned addresses.

That’s it - CrowdSec is configured. But either way, I recommend reading my article on Crowdsec in Docker, where some commands are covered in more detail. As I mentioned above, to avoid duplicating information, I don’t want to copy the same content from one article to another.