Configuring Logstash to consume pfSense logs

Elasticsearch, Logstash and Kibana for collecting and visualizing firewalling data

Posted by Emre Bastuz on March 28, 2015

Introduction

There are already great tutorials out there how a combination of

  • Elasticsearch
  • Logstash and
  • Kibana

(a.k.a. the "ELK stack") can be configured to collect and visualize log data from a pfSense firewall.

One one of those tutorials is written by Elijah Paul and is available at http://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/

Another information source for me was a post by ljoergensen at https://forum.pfsense.org/index.php?topic=87846.0

Trying to implement the configuration that is being described by Elijah and ljoergensen I had to make sme changes:

  • As the logformat of pfSense has changed for version 2.2 so the Logstash filter configuration needs to be adapted
  • The Kibana configuration needs to be adapted to the new log format as well

In the following section I will show how the config of my setup looks to consume and visualize pfSense logs.

The changes are mainly regarding the field names and some minor modification to the regular expressions.

Further more I had to change the type for the fields src_port and dst_port from WORD to INT and also cast it to an INT (weird).

Screenshots

This is how the data looks like when shown via Kibana:

Attackers on the map

Attackers on a map

Events by time

Events by time

Blocked IPs

Blocked IPs

Log entries

Log entries

Logstash configuration

400: Invalid request

Kibana dashboard configuration

400: Invalid request