When you collectively look at a lot of the episodes in this series, one of the things you notice is that we are always grabbing information from somewhere in our network. When we talk about SNMP. We're talking about querying different types of boxes on our network. When we're talking about intrusion detection, we're worried about different devices that are looking for things that we want, we don't want to see. The bottom line is that as a network administrator, particularly one with a security bent, I am constantly monitoring the network for all kinds of different stuff. Now over the years, this has become more and more complicated.
And it's actually generated a whole new universe of technology known collectively as security information and event management, better known as sim sim really takes all of these different disparate types of monitors and puts it together into a single package. So when we talk about Sims, there's two things for us that are most important. Number one is aggregation. Aggregation simply means that We are grabbing data from different places we're collecting this data, and then we're storing it. Secondly, it means correlation. Correlation means now that we got all this data, let's go ahead and analyze that data.
And most importantly, report it in such a way that these crazy human beings can actually look at the data, understand the analysis and potentially do something about it. So let's break these two down. First of all, let's talk about aggregation. Now, aggregation has a lot of different pieces to it. Now, of course, you're going to have sensory devices laying around that are going to be actually grabbing the data. But more important than that is that just putting all this data into one pile is a terrible, terrible idea.
Because you need data that you can actually deal with certain things become very important. For example, time synchronization. If you want to know something's going through a firewall, about the same time that somebody is attacking one of your servers behind the firewall, those two different types of collection devices have to be on The same time, so time synchronization is big. The other big issue is event duplication. A lot of times you'll have a whole bunch of devices that all notice the same problem. And they start pouring into the log files, basically the same information.
So sim actually works very hard to deal with these types of issues. The last thing I want to talk about is normalization. normalization is kind of interesting. So take a look at this example database that I have here. Now you see, it's only got a little bit in there. But what it's doing is you're seeing information in such a way that if I lost any of these records, I'd be in trouble.
Normalization is a pretty straightforward idea. normalization will actually create multiple tables, or whatever you might need. So the data can become more efficient and allows our analysis reporting tools to work a little bit better. Now, keep in mind, the other big thing about aggregation is logs. So you're always Pouring all of this stuff into one type of logger or another. Now without Sim, these logs can be very disparate and all over the place.
So a big part of this is to be able to put these logs together. Now there is one phrase on the security plus that actually amuses me. But for completeness, I'm going to mention it. And that is the word worm. Right? once read many, the concept being is that log files are precious.
And a lot of times you might want to look at them in an archival way, so that we can use optical media like worm drives to store them. So a little bit of a dated thing, don't worry, I'm going to call company to see if we can take care of that particular issue. Today, really, with the price of mass storage logs are stored for the most part on hard drives, lots and lots of big high capacity hard drives. Okay, so that's our aggregation part. Now, the next part all this is correlation. So the big thing about correlation is number one, we have to have some form of alerting and triggering.
If something's going bad at the IDS notices something If one of our switches, suddenly one of the ports starts going nuts, if an individual host detects a piece of malware, we need some kind of alerting system. And sim is designed to do this from the ground up. Secondly, we're going to have to have triggering. Now when we talk about triggering, really what we're saying is what sets an alert off. So usually using tools like SNMP, we can go into devices and into software and say, Look, if this certain bad thing hits the certain threshold, that's a trigger. And we want to get some information not only in a log, but also in some kind of alarm or something that can show up on my screen right now.
So I know that these things are taking place. Okay, so that's the basic concept of how all of this goes together. What I want to do right now is take a moment and go through some examples of some popular sim software. Now keep in mind, I'm not listing them all. There's probably thousands of these, but these are a few of the big names that I want to mention. First one I'm probably going to mention is Splunk spokes been around for a while this is a spendy.
But powerful piece of software that is well represented all over the world. Here's a couple of screens just to give you an idea of what Splunk looks like. notice a big thing that we're looking at when you see screens like this is that we're generating interfaces, we're generating graphs and things like that, that people can use in a real time basis to see if there's any problems. Second, I'm going to do arcsight arcsight. It's also a very popular one. And while I'm not quite a big fan of the screens that you can get with arcsight.
Here's a couple of examples. Just to give you an idea of what the interface looks like, when we're actually looking at what this thing is reporting to us. Probably my personal favorite, though, is the only freeware one that I'm mentioning, and that is elk. elk stands for three very different programs, Elasticsearch, Log Stash and cabana now Gabon is really the reporting part of this. So let me throw up a couple of screens. I don't know about I just I think Obama is pretty.
And it's also very popular. And it's also completely open source. Now, you got to be careful about people going, Oh, why would I spend a gazillion dollars on something proprietary, when I've got this out thing which is totally free? Well, the reality is, is when you get into larger and larger enterprises, free ain't necessarily so in a lot of situations, the amount of support you get with proprietary tools, the amount of systems that work with these more proprietary tools makes it a lot easier. So when you're thinking about what you need, there's going to be a lot of research but keep in mind, if you're anything that's the smallest network, you're going to need some sim