On Precision and Big Data

Most true-positive threat detection is rule based. We use our powers of perception and analysis to find patterns in the data. This is effective because threat behavior is highly repetitive. One can’t say this is data intelligence in the strictest definition, but more of an expert pattern. Albeit behavior, I argue this still resides on the edge of the signature playpen. This is fine as long as it continues to work for the security marketplace (and so far, does). Regarding Big Data; In lieu of ingesting huge quantities of data in the hopes that some needle will become self-evident, I suggest continued development of rigorous expert patterns. Of particular value are patterns that can match against host-endpoint behavior (in conjunction with netflows at the perimeter). I believe this can produce highly effective, non-specific (i.e., resilient) extraction of high-fidelity threat events. With data overload being a huge issue, the role of precision becomes ever important.

-Greg Hoglund
www.veraxes.com