.

Fun with pfSense and Splunk

<<

Triban

User avatar

Hero Member
Hero Member

Posts: 620

Joined: Fri Feb 19, 2010 4:17 pm

Post Sun Jan 13, 2013 11:12 pm

Fun with pfSense and Splunk

So I was able to take some time during a company shutdown week and do some network redesign in the home.  I took an Atom based "Shoebox" system and finally built myself a pfSense firewall.  I added Snort as well (built-in version to pfSense).  More on my adventure here.  I then spent a couple days just staring at the new found wonders of real logs!!!  I used some of the built-in look-ups and seeing who was scanning my IP.  I also threw Snort on but due to some frustrations, I placed it in detection mode. I will eventually go back to and turn up the prevention setting. 

So I had all this new logging capability, but really didn't have anything in place to collect it for analysis.  That was where Splunk came, pretty cool product if you have never used it.  It is primarily a log collection/reporting tool with a number of 3rd party applications that can be loaded in.  One in particular that I found useful, was the Google Maps App.  I then configured pfSense to Syslog the Firewall logs and configure Snort to send it's alerts to the System Log.  Then I setup the UDP listener in Splunk to pull the pfSense logs in.  Now for the fun part, Google Maps setups up Geo_IP plotting.  The simple search rule was basically showing all pfsense-firewall block activity.  I still haven't written this up, but I will eventually. 

This information is nifty but I am much more interested in the Snort data.  Unfortunately the Snort data was not being completely parsed in Splunk like the Firewall data.  I have been trying to find information on how to do this, but I have not had any real luck.  Splunk does have an app for Snort, but it seems it may be designed for the stand-alone.  I was thinking there may be much more I can configure for Snort behind pfSense but haven't gone down that route. If any one has any ideas on parsing out the Snort data, I'd be interested in hearing them.
Certs: GCWN
(@)Dewser
<<

dynamik

Recruiters
Recruiters

Posts: 1119

Joined: Sun Nov 09, 2008 11:00 am

Location: Mile High City

Post Tue Jan 15, 2013 10:51 pm

Re: Fun with pfSense and Splunk

How do you have Snort logging at the moment? It can log to syslog. I assume Splunk can work with syslog data...
The day you stop learning is the day you start becoming obsolete.
<<

Triban

User avatar

Hero Member
Hero Member

Posts: 620

Joined: Fri Feb 19, 2010 4:17 pm

Post Thu Jan 17, 2013 10:11 pm

Re: Fun with pfSense and Splunk

Currently I have the logs being sent to the pfsense system log.  I haven't been able to find any splunk docs properly parsing out the snort data.  Before I configured splunk to sort out the firewall data, I had to edit 2 files, which are (I imagine) instructions on how to parse the data properly:

props.conf
  Code:
[source::udp:514]
TRANSFORMS-pfsense-firewall = pfsense-firewall
SHOULD_LINEMERGE = true
TRUNCATE = 0
MUST_NOT_BREAK_AFTER = pf: .* rule ([-\d]+\/\d+)\(.*?\):
MUST_BREAK_AFTER = pf: .* (<|>) +(\d+\.\d+\.\d+\.\d+)\.?(\d*)\:
REPORT-pfsense-firewall = pfsense-firewall


transforms.conf
  Code:
[pfsense-firewall]
REGEX = .* (?<action>pass|block) .* (?<protocol>TCP|UDP|IGMP|ICMP) .* (?<src_ip>(\d+\.\d+\.\d+\.\d+))\.?(?<src_port>(\d*)) [<|>] (?<dest_ip>(\d+\.\d+\.\d+\.\d+))\.?(?<dest_port>(\d*)): (.*)
CLEAN_KEYS = 1
MV_ADD = 0


This then formats the reports in Splunk.  I imagine I can do something similar with the snort rules.  The snort logs come in looking like this:

  Code:
Jan 17 20:54:15 192.168.0.254 Jan 17 20:54:14 snort[61858]: [1:2500006:2752] ET COMPROMISED Known Compromised or Hostile Host Traffic TCP (4) [Classification: Misc Attack] [Priority: 2] {TCP} SRCIP:PORT -> DESTIP:PORT


I may just have to read up on the use of the transforms and props files. 
Certs: GCWN
(@)Dewser
<<

UKSecurityGuy

User avatar

Jr. Member
Jr. Member

Posts: 88

Joined: Wed Mar 27, 2013 10:51 am

Post Wed Apr 17, 2013 5:05 am

Re: Fun with pfSense and Splunk

Sorry to Necro this thread, but did you ever get this working for you?

I use Splunk heavily so if you get stuck, give me a shout and I might be able to help.
<<

Triban

User avatar

Hero Member
Hero Member

Posts: 620

Joined: Fri Feb 19, 2010 4:17 pm

Post Wed Apr 17, 2013 7:21 am

Re: Fun with pfSense and Splunk

Hey no problem, no I never did get this piece working.  Haven't had a chance to revisit it but if you have some thoughts, I'd be interested in hearing them.
Certs: GCWN
(@)Dewser
<<

UKSecurityGuy

User avatar

Jr. Member
Jr. Member

Posts: 88

Joined: Wed Mar 27, 2013 10:51 am

Post Wed Apr 17, 2013 8:08 am

Re: Fun with pfSense and Splunk

I was going to do some extractions myself based on your sample, but there appears to be a Splunk app for this already:

http://splunk-base.splunk.com/apps/22369/splunk-for-snort-splunk-4x

The app seems to suggest that you've got just got to change the source types

  Code:
Snort expects full alert logs to have a sourcetype of "snort_alert_full" and fast alert logs to have a sourcetype of "snort_alert_fast"


So assuming that you've only got one of the two input methods coming in, and assuming that only Snort is being pumped in via syslog on port 514, you could simply add one of these lines to the relevant inputs.conf

sourcetype = snort_alert_full
sourcetype = snort_alert_fast


In theory the app should take care of the rest.
<<

Triban

User avatar

Hero Member
Hero Member

Posts: 620

Joined: Fri Feb 19, 2010 4:17 pm

Post Thu Apr 18, 2013 6:17 pm

Re: Fun with pfSense and Splunk

Hmm, I do have the app.  I think it was designed for a designated Splunk box.  pfsense is currently dumping all logs to syslog and the current Firewall setting is parsing that information.  I can probably change the settings to see if it can pull those out.  If so maybe I can do something with pfsense to send the snort logs another way.  This helps though.  I may muck around with it over the weekend.

Thanks!
Certs: GCWN
(@)Dewser

Return to Other

Who is online

Users browsing this forum: No registered users and 0 guests

cron
.
Powered by phpBB® Forum Software © phpBB Group.
Designed by ST Software