Windows Servers Event Logs Consolidator

[err]

We have about 90-100 Windows servers in all flavor NT4, Win2K Server & Win2K3 Server.

There is a new requirement from our management to actively monitor those logs in our technical services team.

It would be definitely nuts to look at all the event logs everyday on 90 different servers. My 8 hour day would be gone in a snap.

Any tool out there that would do event log consolidation for Windows servers, preferrably with the following features?

- Cheap
- Can consolidate all event logs in one screen. Easily sortable, searchable and manageable.
- Ability to place note or knowledge base in the software
- Would be nice if the software would tell us what kind of solution is available for that particular event.

We do have a nice network-intelligence box for consolidating firewall logs. However it is a $50K solution and this wouldn't fly through the management.

Ideas?

Thx

 

[alent1234]

I don't know about cheap. where i work it's getting to this point where we have too many servers to monitor.

Check out Microsoft Operations Manager, Systems Management Server and products from NetIQ. The original version of MOM was a lite version of NetIQ, but I'm not sure about the recent versions.

You can also check out scripts on Technet and set up a system where you write certain events to a database.

 

[netsysadmin]

Check out the product below. We are porting the logs to MySql and it is working good so far. Price is also good too. We are using this product to meet HIPAA requirements regarding log analysys.

EventSentry

John

 

[err]

eventsentry seems like a cool recommendation. Pricing is also reasonable. I will check this out more and hopefully able to sell this to the management.

MOM, NETIQ are definitely out of the picture. For that much of a price, I would rather get a nice network-intelligence box.

Thanks for the recommendation, keep it coming !

ed

 

[spidey07]

gfi makes a fabulous tool for just what you require.

sarbanes-oxley is a bitch ain't it?

 

[err]

Its actually HIPPA. We've made so much security improvement which is nice for the network. However boring for us system admins (Yawn)

I've looked at GFI as well. Seems good, but also on a more expensive side.

Well, I am going to be comparing whatever product againts $50K Network-intelligence box, so I think I am going to have some leverage I hope

 

[spidey07]

Originally posted by: err
Its actually HIPPA. We've made so much security improvement which is nice for the network. However boring for us system admins (Yawn)

I've looked at GFI as well. Seems good, but also on a more expensive side.

Well, I am going to be comparing whatever product againts $50K Network-intelligence box, so I think I am going to have some leverage I hope

good luck with HIPPA.

Just present "this is what you have to do to be compliant. Please sign this affidavid stating that I've provided a solution"

 

[Rogue]

How do these work? For instance, we just got a 16TB SAN and I'm trying to convince our server folks to create a location on the SAN to store all of these log files in a central location, perhaps with a folder for each server (by server name). Would EventSentry then scan those directories and assemble the logs into a comprehensive report?

 

[netsysadmin]

We had another group here at the university I work out monitor the network traffic of GFI SELM. Since it is a pull type setup from ther servers to the logging server it generated more traffic. EventSentry pushes agents out to you machines and those agents monitor the logs on the server. We also use the ODBC funtionality to pump all the logs out to a MySQL server that stores all our logs for forensic purposes later if needed. So far it is working good. To satisfy the reporting that you may need for HIPAA you can either query the site that comes with EventSentry or use and ODBC connector and link to the tables on the MySQL server with Access and write some queries that will extract your data that you need. We are doing the Access route right now. There are some other software packages out there right now that come with reports built in, but they are all much more $$$ than EvenSentry and still dont meet all the reporting funtions we need such as trending analysis. We couldnt get management to spend the extra money so we have a little developing here to do. Also to add to the features EventSentry also has the ability to monitor a lot of other things like services and disk space. Its a nice program for the money.

John

 

[Woodie]

Interesting thread. I'm in the process of completing version 2 of our event-log capture software. (in-house use only!)

It's kind of a pain, and the volume of events is absolutely horrendous. I'm seeing >50,000 security events per day, with only one DC reporting in, from a low-volume development domain. I don't know how it's going to handle production, w/ >150 DCs reporting in!

eventSentry looks very interesting, but doesn't seem like a match for me. I'm looking for archival of user activity, not just error or other unusual activity. I just need to capture the data, and store it for an extended period of time.

 

[netsysadmin]

Rogue....They offer a free full featured trial download on there site so you can check out the features. Essentially each server will have an agent on the machine that will scour the event logs looking for events that you have specified as triggers. You can have the machine email, page, print from a printer, etc when it finds and particular event. It also had the ability to port all the logs to an outside database sever such as MySQL, SQL, etc. This allows us to archive all the security logs our servers generate. These archives are required by the university that I work for for possible forensic analysys at a later date in case of a breach of security. We also do out trend analysys reports from these logs in our MySQL database. The software also has a lot of other features to check out that I have not mentioned anything about. It is quite nice for the price. I suggest loading up a test version so you can kick the tires of it.

EventSentry Screenshots

John

 

[netsysadmin]

Woodie...what kind of user activity are you trying to track? We track log on/off and access to ePHI for all the users in our domain. We also monitor administator usage also such as account creation...etc. We also archive all the security logs from all our servers to a MySQL database.

John

 

[Woodie]

Audit requirement...looking to track pretty much all user activity:
Logons/Logoffs
All account changes (pwords, groups, etc...)
Security Policy changes

Luckily, since we don't use/permit local accounts, we only have to handle Domain Controllers. Still, 150+ DCs, >30,000 user accounts...drives a LOT of volume. Since we upgraded the DCs to 2K3, we've at least doubled the volume of events, with only a 20% increase in number of users.

I've thought about SQL, but it's easier to just zip the dumped .txt files and store them. Retrieval /reporting is much harder...but there doesn't seem to be much demand for that.

Volume-wise, I'm seeing a single PDCE, for a single day, in a DEV domain:
142,109,250 bytes in a .txt file
444,812 events (lines)

Equates out to about 66,721,800 records per day. I suspect that I'd have to buy an awfully big box to run SQL w/ that many daily inserts and purges. Then multiply by the number of days...

 

[alent1234]

How much space do the logs take up? I've been looking for something to take certain events and put them into SQL.

Where I work me and one other person do all the account related stuff with a written request. We have been thinking of letting the HD do it, but we want to audit it. They have too many friends with the people they support and may make changes without the appropriate request and permission from management.

 

[err]

woodie,

if you company can afford the dough, I would highly suggest going with network-intelligence.

Excellent box with their own built-in database engine that compress records to 1/28. Pretty amazing.

Very easy to setup and deploy within the domain as well and as the sales person said it "we don't need no stinking agents". Yes it is an agentless log consolidator box.

We currently have this in house to consolidate syslog events from firewall. We are a small shop of 500 users and I am seeing about 200,000 events from the firewalls alone / day.

ed

 

[err]

netsysadmin,

eventsentry seems promising after I've looked into the software a little deeper. Seems to be a perfect solution for me rather than manually analyzing the logs.

I'll just have to lab it out and convince the management

ed

 

[Woodie]

err The company "can" afford it..just a question of are they "willing" to. Looks interesting...however we don't have a current requirement for event correlation. (at least not in the mind of the management). They're just looking to capture/store the data. They're also greatly concerned about the network overhead on the WAN links. Some of those field-office pipes are pretty small. (down to 256)

Still, it looks like a really nice tool. <drool>

 

[Nothinman]

Personally we're just going to use a syslog server for everything, at least that's the current plan. Everything but Windows does syslog out of the box and there are small, free tools to do Event Log->Syslog. That along with a little time, shell and perl scripts and you can have all of your logs centrally stored, rotated and analyzed.

 

[err]

Yes there are tools to convert event log to syslog. I am using Snare myself for windows events