Keeping an eye on your logs is critical (really, its number 14 on the SANS critical list of controls: https://www.sans.org/critical-security-controls/control/14 .) Earlier Rob VandenBrink shared some techniques to find nuggets hiding in your logs (https://isc.sans.edu/forums/diary/Syslog+Skeet+Shooting+Targetting+Real+Problems+in+Event+Logs/19449/ .) Today Im going to share some tricks to squeeze every last bit out of your logs through repurposing logs. I mean repurposing log files, not this: https://www.pinterest.com/dawnreneedavis/repurposed-logs/ .
Logs are given their original purpose when programs determine when and how theyre going to record a log entry. Today I want to discuss unintended value, or how to get more out of your logs than the programmers intended, or how to recover value that is easily overlooked.
Lets start with an example. Suppose you work in a large siloed environment and you dont have access to the logs from every group. Youre in a security or investigative function, and have access to the AV logs. The obvious use of the logs is to record the alerts generated by the endpoints, or find machines that arent updating signatures properly or are have detection engines that are out of date. A bit that you might be overlooking is the value of the checkin message itself. Ive found it very useful to keep the checkins for a long period of time, which gives you a history of what IP and what user was logged into a machine when it regularly checks in. It doesnt have the resolution and accuracy that you would get from you AD authentication logs, or your DHCP logs, but you might not have easy access to those. This small investment in disk space, or simple database can give you quick snapshot views of machine and user mobility. You can easily see if this desktop consistently has this IP, or if this laptop moves around through your campus. You can get the same feel out of your user accounts too, without having to invasively dig through badge access logs.
This is the first technique that I want to share: extract a daily event out of your logs and store it over time. This creates an additional product that keeping a rolling history of logs cant provide.
Now consider what hidden and unexpected information might be hiding in your web proxy logs. Take a look at the W3C standard fields. If you reduce the displayed fields down to just timestamp, c-ip, r-host, and r-ip, youve got yourself a quick passive-DNS feed. Granted its just looking at web traffic, but a good chunk of your network mischief is traveling through that channel at least once.
Trick number two: look for unexpectedly-useful combinations of columns in your log entries.
On to number three: data reduction and indexing. Logs are big, and logs are noisy. While I recommend that you keep the raw logs for as long as you can, I understand that isnt possible and that you have to make tough choices on what you store and for how long. One way to squeeze out more time from your logs is to reduce the number of columns that you keep for your archives. Using the web proxy logs as an example, you might not be able to keep every log entry for 24 months, but keeping just the c-ip,r-host,r-ip columns can be very helpful when youre looking back through an old undiscovered compromise or are dealing with an information request like has any system on your network interacted with one of these IPs?
Years ago I would recommend further daily reduction and indexing of these files, but these days you probably have a splunk instance or an ELK stack (https://digital-forensics.sans.org/summit-archives/dfirprague14/Finding_the_Needle_in_the_Haystack_with_FLK_Christophe_Vandeplas.pdf) and you just dump logs in there and hope that magic happens. Theres value in examining and repurposing logs in these days of map reduce. The reduced files that you create from the logs are easy to drop into your hadoop cluster and build a hive table out of.
So, lets tie this all together. Youve received your list of IPs from your intelligence vendor and youre tasked with finding any activity on your network over the past 2 years. In your web proxy index you see that you had a hit 8 months ago. Now youve got an IP and and date, what machine had that IP then? Now you search through your AV checkin data and get machine name. But the AV checkin logs are daily, not logged by minute, so you search around for the IP history of that machine in the AV logs and hopefully you see it consistently checking in from that IP and not moving around a lot. If youre not so lucky, well, its time to open up request tickets to hopefully get at the DHCP logs from back then.
One last parting thought: do you have waste/useless logs? If you apply one or more of these techniques to it, can you find a way to process them into something useful?
-KL
(c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.