Fun with pfSense and Splunk

Viewing 6 reply threads
  • Author
    • #8142

      So I was able to take some time during a company shutdown week and do some network redesign in the home.  I took an Atom based “Shoebox” system and finally built myself a pfSense firewall.  I added Snort as well (built-in version to pfSense).  More on my adventure here.  I then spent a couple days just staring at the new found wonders of real logs!!!  I used some of the built-in look-ups and seeing who was scanning my IP.  I also threw Snort on but due to some frustrations, I placed it in detection mode. I will eventually go back to and turn up the prevention setting. 

      So I had all this new logging capability, but really didn’t have anything in place to collect it for analysis.  That was where Splunk came, pretty cool product if you have never used it.  It is primarily a log collection/reporting tool with a number of 3rd party applications that can be loaded in.  One in particular that I found useful, was the Google Maps App.  I then configured pfSense to Syslog the Firewall logs and configure Snort to send it’s alerts to the System Log.  Then I setup the UDP listener in Splunk to pull the pfSense logs in.  Now for the fun part, Google Maps setups up Geo_IP plotting.  The simple search rule was basically showing all pfsense-firewall block activity.  I still haven’t written this up, but I will eventually. 

      This information is nifty but I am much more interested in the Snort data.  Unfortunately the Snort data was not being completely parsed in Splunk like the Firewall data.  I have been trying to find information on how to do this, but I have not had any real luck.  Splunk does have an app for Snort, but it seems it may be designed for the stand-alone.  I was thinking there may be much more I can configure for Snort behind pfSense but haven’t gone down that route. If any one has any ideas on parsing out the Snort data, I’d be interested in hearing them.

    • #51510

      How do you have Snort logging at the moment? It can log to syslog. I assume Splunk can work with syslog data…

    • #51511

      Currently I have the logs being sent to the pfsense system log.  I haven’t been able to find any splunk docs properly parsing out the snort data.  Before I configured splunk to sort out the firewall data, I had to edit 2 files, which are (I imagine) instructions on how to parse the data properly:


      TRANSFORMS-pfsense-firewall = pfsense-firewall
      TRUNCATE = 0
      MUST_NOT_BREAK_AFTER = pf: .* rule ([-d]+/d+)(.*?):
      MUST_BREAK_AFTER = pf: .* () +(d+.d+.d+.d+).?(d*):
      REPORT-pfsense-firewall = pfsense-firewall


      REGEX = .* (?pass|block) .* (?TCP|UDP|IGMP|ICMP) .* (?(d+.d+.d+.d+)).?(?(d*)) [] (?(d+.d+.d+.d+)).?(?(d*)): (.*)
      CLEAN_KEYS = 1
      MV_ADD = 0

      This then formats the reports in Splunk.  I imagine I can do something similar with the snort rules.  The snort logs come in looking like this:

      Jan 17 20:54:15 Jan 17 20:54:14 snort[61858]: [1:2500006:2752] ET COMPROMISED Known Compromised or Hostile Host Traffic TCP (4) [Classification: Misc Attack] [Priority: 2] {TCP} SRCIP:PORT -> DESTIP:PORT

      I may just have to read up on the use of the transforms and props files. 

    • #51512

      Sorry to Necro this thread, but did you ever get this working for you?

      I use Splunk heavily so if you get stuck, give me a shout and I might be able to help.

    • #51513

      Hey no problem, no I never did get this piece working.  Haven’t had a chance to revisit it but if you have some thoughts, I’d be interested in hearing them.

    • #51514

      I was going to do some extractions myself based on your sample, but there appears to be a Splunk app for this already:

      The app seems to suggest that you’ve got just got to change the source types

      Snort expects full alert logs to have a sourcetype of "snort_alert_full" and fast alert logs to have a sourcetype of "snort_alert_fast"

      So assuming that you’ve only got one of the two input methods coming in, and assuming that only Snort is being pumped in via syslog on port 514, you could simply add one of these lines to the relevant inputs.conf

      sourcetype = snort_alert_full
      sourcetype = snort_alert_fast

      In theory the app should take care of the rest.

    • #51515

      Hmm, I do have the app.  I think it was designed for a designated Splunk box.  pfsense is currently dumping all logs to syslog and the current Firewall setting is parsing that information.  I can probably change the settings to see if it can pull those out.  If so maybe I can do something with pfsense to send the snort logs another way.  This helps though.  I may muck around with it over the weekend.


Viewing 6 reply threads
  • You must be logged in to reply to this topic.

Copyright ©2020 Caendra, Inc.

Contact Us

Thoughts, suggestions, issues? Send us an email, and we'll get back to you.


Sign in with Caendra

Forgot password?Sign up

Forgot your details?