Over the past few weeks SIEM vendor Intellitactics has attempted to answers some of the most commonly received questions about their product. Here are the problems I have with their summary of their responses:

Question 1) Will Intellitactics’ PCI DSS Compliance reports satisfy a Qualified Security Asseessor?

Intellitactics Answer: YES

REALITY: Unfortunately not all auditors are created equal and each one has their own interpretation of what will satisfy the PCI DSS.

Question 2) Can you write your own reports?

Intellitactics Answer: OF COURSE but you won’t need to.

REALITY: I highly doubt that Intellitactics has thought of every possible reporting scenario and I have spent a fair amount of time creating my own reports based off of vendor canned reports. Never say never.

Question 3) When you’re using Intellitactics SIEM Solutions will you be able to collect ALL LOGS from any device or data source?

Intellitactics Answer: YES YES YES!!

REALITY: From my C64? Really? How about my mainframe that logs to a screen, has no unique identifiers, and no native method to ship those logs off to your product? This smacks of marketing and don’t be fooled. Always verify that your products can log to the SIEM you’re looking at and if the vendor says your custom/obscure/dated application will log to their system – ask for a proof of concept with YOUR system.

Question 4) Can you do root cause analysis?

Intellitactics Answer: YES and you can do it graphically – in the case of Intellitactics SIEM solution – a picture really is worth a thousand lines of events.

REALITY: Pictures are great for 10,000 foot views but the answer is in the data. Use flashy graphics as a starting point but don’t believe that your SIEM is smarter than a trained analyst.

Question 5) How fast is Intellitactics SIEM solutions?

Intellitactics Answer: FAST ENOUGH – Consistently for effective and efficient log and event management.

REALITY: It may be “fast enough” for the vendors benchmark tests but that doesn’t mean that it’ll be fast enough for your needs. Always challenge your vendor on their figures.

Question 6) How many devices does Intellitactics support

Intellitactics Answer: ALL the ones that are important to you and then some.

REALITY: How could a vendor know which devices, applications, logs are important to me. What’s important now won’t necessarily be what’s important in 6 months and you mean to tell me that you’ll anticipate this requirement?

BOTTOM LINE: Always challenge your vendors and get what’s right for you folks. Don’t read into the marketing.

I sat down this afternoon and passed my GIAC Advanced Filesystem Recovery and Memory Forensics Skills Test and Report (STAR) test. I took the SANS Security 526:Advanced Filesystem Recovery and Memory Forensics course while at SANS Network Security 2009 in San Diego at the tail end of my week long SANS Computer Forensics, Investigation, and Response class (and boy was I tired).

I can’t say enough about how great both of these courses are and I hope that I fare as well on the GIAC Certified Forensics Analyst (GCFA) exam when I sit for it in the coming months. If you ever get an opportunity I strongly recommend you take both of these courses.