Fast Horizon

Detecting APT Attackers in Memory with Digital DNA™

2011-11-02T11:26:00.000-07:00

HBGary’s Digital DNA™ system is an alternative to traditional signature-based approaches to detecting malicious backdoors. While the “APT is not Malware” mantra is common, APT commonly use malware. To be precise, APT is just a hacker in the network. Remote access to the network is guaranteed only through stolen VPN credentials, or through the placement of a remote access tool (RAT) – in other words, malware. So, enter DDNA.

DDNA is designed around generic detection of subversive code. To do this, HBGary disassembles everything on-the-fly and pushes it through a sieve of regular expressions that match against control flow and data flow features. I thought it would be fun to delve into some specific examples.

As Martin recently pointed out in his blogpost, APT has started to use in-memory injections as a means to hide code. We have noticed remote-access functions injected and split over a range of memory allocations.

In the screenshot, you can see a dozen 4K (0x1000) allocations injected into explorer.exe. (Note: this type of activity can be detected using the free Responder CE.) Each page of memory only contains a tiny portion of the overall malware – something that would frustrate most AV scanners. However, the allocations themselves are suspicious to Digital DNA™, and in particular the last page has a suspicious code fragment that scores quite heavily in Digital DNA™. This illustrates why a filesystem-only view is not sufficient to detect APT tools. Many advanced techniques involve modifications to the running system and can only be detected in memory.

In this example, the hacker hasn’t hooked anything. Instead, he starts some additional threads to service the malware code. Even though the malware has been split over a dozen pages, the hacker has only started two threads. In this example, allocations #8 and #11 each host a thread subroutine. The other memory pages each hold specific subroutines. For example, one of the memory pages has a function for installation into the registry, while another has a function for hiding a copy of the malware in an alternate data stream. It’s these suspicious behaviors that Digital DNA™ is focused on detecting. Furthermore, it’s the behaviors being used together that will really light up color-coded DDNA alerts.

One suspicious feature is when code exists outside the bounds of a known module. This will occur if the hacker allocated additional space for storing an injected routine. This is commonly done using VirtualAllocEx(), but can also be achieved using the stack of an injected thread. In the latter case, CreateRemoteThread() is used with a stack size argument large enough to store an injected routine. In either case, executable code is detected outside of a defined module, and this will score as suspicious by default even without further analysis.

Moving further, however, injected code is typically handwritten assembly. In most cases, the operational code will not resemble known compiler patterns (such as code compiled by Visual C++ or Borland). In particular, the code may contain position-independent operations – function calls and data references that are designed to work independent of the address where the code lives in memory. These are further indicators of suspicion. In my experience, the only time this kind of code appears in a legitimate binary is when DRM is being used (DRM looks and smells like malware anyway).

To look back at our example, it had some interesting techniques for embedding data inline with code:

In the example, you see the “w32_32” string in use, but what makes this interesting is how the string is embedded inline to the code. Right before the string we see a short call that jumps over the string, and code execution continues on the other side. Again, this idiom is suspicious and can be detected generically, as opposed to reliance on a specific string or byte pattern.

In the case of Digital DNA™, code 16 30 detects short calls and jumps over inlined networking related strings. How did we get here? HBGary detected that some APT groups were producing this code pattern as a result of some code-level anti-forensics tools. This is exactly the kind of pattern that produces big wins on the detection side as the code is often cut-and-paste or the obfuscation is applied in batch to otherwise custom-compiled malware. (Of course, now that I’ve blogged about it they will switch off to another trick – it’s OK, we have thousands of traits to detect suspicious behaviors).

Another example of handwritten code is the CRC function used by the hacker to load his table of function pointers. This CRC-based technique has been around in shellcode for a long, long time (digression: I think I released the first public CRC loader in shellcode in the early 2000’s – it was 32-bit CRC. Thinking back, Halvar Flake publicly released a better and smaller 16-bit CRC loader in shellcode shortly afterward. The technique has been written about many times since).

The routine that actually calculates the CRC is usually hand-made – so it too can become a form of attribution. But even if it’s not hand-made, the proximity of CRC to a GetProcAddress() call would be indicative of this pattern. In our APT example, the author has created a CRC for loading a function table:

The CRC calculation is referenced from a routine that is rolling through KERNEL32.DLL and calling GetProcAddress(). This pattern screams for attention “Hey! I’m malicious!”

So again, Digital DNA™ for the win. The CRC can be detected using a generic method, and when detected in control flow in proximity to GetProcAddress() loop, it scores hot with trait C3 F7.

These are just some examples of how Digital DNA™ focuses on analyzing the code itself, as opposed to blacklisted MD5’s or ASCII strings. It is not possible to specify these behavioral patterns with simple languages like OpenIOC or even ADXML (Active Defense’s XML for scan policies) – they can only be detected programmatically. That is why our product Active Defense doesn’t depend on IOC’s alone to do the job – in fact, Active Defense starts with full physical memory analysis and Digital DNA™ sequencing. IOC’s come second and only if the user wants to extend the default detection capability with custom threat intelligence. The two methods work well together, Digital DNA™ to detect new and unknown threats, and IOC’s as a follow-up sweep for known APT behaviors.

Using IOC’s effectively

One of the reasons we invented Digital DNA™ is because IOC’s alone aren’t good enough. A problem arises when IOC’s are only used to detect known threats. Think about this – if your IOC’s are just a blacklist of recently discovered malware MD5’s and unique strings then its equivalent to a small AV dat file. Even though IOC’s can be used to detect TTP’s (i.e., scanning the enterprise for split RAR archives or recent use of ‘net.exe’) we generally see them employed to detect specific malware files. If your organization has a database of IOC’s then look for yourself. How many entries have MD5 checksums? How many are specific to a malware sample, a specific registry key used to survive reboot, etc? If you see an overabundance of these signatures then beware – this is the same old blacklist-driven security model that has been failing us for over 10 years now. On the other hand, if you are using IOC’s to scan for more generalized things, such as command-line usage, access times on common utilities, executables in the recycle bin, etc., then you are on a far better trajectory. I support open intelligence sharing, but I caution you against falling into the “magical strings” bucket. Too often our industry shares threat intelligence in the form of blacklisted MD5’s or IP addresses – this kind of threat intelligence is nearly useless.

HBGary’s managed services team generates many IOC’s in the course of their work, and I am happy to say that we share all of them with our Active Defense customers – we don’t keep them secret. They are provided automatically in the form of a library that is auto-updated. Customers can pick and choose from many search definitions and use these as a basis to create their own custom searches. Our team tries to steer away from malware-specific indicators, and instead focuses on the generic attack patterns that can be detected at the host. We give these to our customers because we want them to get the most from our software. We enable people to be self-reliant.

When you use Digital DNA™ and IOC’s together, you aren’t relying on a “magical bag of strings” that go stale every two months. Instead, you are detecting new threats and then using IOC’s to apply attrition against the attacker’s persistence. This is a strong defensive position. This is why our proven behavior-based solution approach is increasingly winning us new customers – even unseating our competition in many accounts.

-Greg

APT - The Plain Hard Truth

2011-09-22T11:54:00.000-07:00

The survivors from the front line have reported in. We stand on the ridge, a tangled mess of bodies behind us. We are the ones who have chased the demon, descending into the binary pit the users call the “enterprise”, and climbed up the other side. What we have seen is not pretty. The collective corporate filesystem is a parking lot for castaway software barely able to run on modern operating systems, squeezing the last bit of life out of burned out win32 DLL’s. There are big piles of unwashed garbage downloaded by employees that were passing by, never deleted, never clean. The strangest mutated crap has been swept tightly into temporary directory corners that have since calcified and become permanent.

More than a single digit percentage of these software programs are a biohazard. Some are just plain broken, wheezing out juice from a hooked windows message chain just long enough to cough up and die, only to be resurrected by the swift kick of a boot-time registry key the next time the machine reboots. Some have pretty little labels of well-known companies – clearly so you won’t look twice at them and notice how they are exfiltrating personal browsing statistics and other data to some cloud server – really like malware but allowed by the EULA that you didn’t read. Some of these things don’t seem to have any purpose but to act as a low-fidelity binary listening device.

Everything looks bad. So, it’s no wonder that hackers can just plug something new in and nobody notices. As long as it doesn’t infect five million residential banking customers then nobody is going have a description of the suspect. That is the reality of hacking today, and it has nothing to do with advanced persistent threat. It has to do with the enterprise and the complete LACK of control you have over the endpoint. When security is limited to the network perimeter, you are not in control. Oh, and what a breath of fresh air the mobile device is. A new pile of software, mostly social media, that is directly connected to thousands of strangers that are not your employees, communicating in real-time with processes running within your defensive wall. In effect, you now have thousands of potential multi-homed routers to 3G-space* from your network that don’t belong to you.

*4G if your lucky

Here are some basic security facts:

Today, malware is a tool for persistent adversaries
Adversaries are financially or politically motivated
Intrusions involve a real human being or hacking group that targets your organization directly (*)
Attackers are motivated to steal something from your network

*Somehow in the mid-2000’s it seems like the security industry lost its way and forget about the basic tenants of Hacking Exposed – unfortunately you cannot condense a set of MD5 checksums out of the hacker problem.

Recently during presentations I have outlined three primary threat groups we face today. I have illustrated the evolution of these in the following diagram.

A. Criminal Enterprise – these are the guys who make more money than drug cartels and the reason a malware economy emerged over the last few years. This is what mere mortals mean when they talk about malware, and the reason people get malware and hackers mixed up all the time.

B. Rogues – these are the hacking groups that you can enumerate on any given day. There are hundreds, if not thousands worldwide. These guys are all capable. The graph expands much slower than criminal enterprise because they aren’t fueled by cash. As early as 2000 these guys were already defacing, DDOSing, and partaking in ‘mostly harmless’ hackery. Yet, a small subset have always been deeply malicious and get pleasure out of destroying things. Others pick up a cause and act like cyber terrorists. And still others really are cyber terrorists.

C. Rogues meet cash - these hired mercenaries are the ones who write malware, sell zero day, and get sucked into the vortex of organized crime. These guys are very, very dangerous.

D. The problem today - all the membranes have been breached - the threat is blended. We live in a time where a state interest can simply buy access to adversary networks from criminals who are selling their botnets. Where state sponsored attacks can be vectored through private hacking groups. Where private hacking groups can fund their operations from cybercrime, while targeting corporations and governments with methodology indistinguishable from APT. There is no tidy bucket to place the threat, all the wires are now crossed. The only thing that is consistent here is that hacking is hacking, and it always looks and smells the same when you see it. This is why the term ‘APT’ is so tired.

E. Private hackers working for the man - when you catch a Chinese malware in a DoD contractor network, it almost always looks like it was written by a “kid”. This “kids” malware is then used to steal the plans for a weapons program that can only have value to the PLA. All the security vendors looking at APT come up with corny little codenames for all the hacking groups (HBGary included), but at the end of the day it’s all the same thing.

F. Thank God for APT - a board room level term that we can all use to cover our you-know-what when we tell the man our millions of dollars in security spending has done nothing for us.

If you want a no-holds-barred, no excuses, and no-snakeoil analysis of APT and the reality of countering it, you should check out HBGary’s new whitepaper The New Battlefield.

-Greg

Social Terrorism

2011-09-07T17:29:00.000-07:00

Social networking does something to people, intoxicating them with near-zero accountability for impulsive behavior protected under a banner of free speech. Fierce defenders of the social media revolution think that because this technology is novel, somehow it should be afforded a special layer of protection. Social media empowers people, but it shouldn't make free speech apply to all forms of the 'fire in a crowded theatre'. Thankfully there are policy makers and courts who still feel that inciting violence, organizing illegal activities, causing riots, partaking in slander and libel, or harassment and abuse is wrong and/or criminal in nature regardless of the medium of communication.

New forms of 'fast and wide' communication technology have effectively armed common citizens with an information warfare tool. This is fine, but handle with care. Like any real tool of value, it can cut you. This is not a free speech issue, it's one of safety. When BART wants to shutdown communications due to threat of riot and crime, it's their right to do so. When Philadelphia wants to put a curfew in place to stop flash mobs, they are protecting the citizen. When authorities in London want to curb-stomp looting they should be able to do things like shut down riot tweeters. When the NYPD runs an intelligence group to hunt down terrorists and criminals on Facebook and Twitter, it's their right to do so - in fact, it's THEIR JOB to do so. If you are dumb enough to put your personal information on the 'net and then commit crimes, fair play (as Lulzsec has learned). Social media companies have a responsibility to work with government, law enforcement, and private authorities to ensure that they aren't enabling damage. Terrorists using Twitter are still terrorists.

When someone falsely claims a bomb threat, they are committing a crime. When they do it on Twitter, they are still committing a crime. As two people just learned in Mexico, putting it on Twitter doesn't make it legal. And, several men were jailed in the UK for using Facebook to incite violence during the riots. And today it's common for cases to be won against cyber bullying. Yes, embrace social media, but don't think that entitles people to be assholes.

-Greg

Inside an APT Covert Communications Channel

2011-08-16T09:07:00.000-07:00

Note: I shortened the title of the post from "Inside an APT “Comment Crew” Covert Communications Channel" to "Inside an APT Covert Communications Channel". To be clear, multiple threat groups are using HTML comments as a means of COVCOM. Thus, this should be considered a general technique as opposed to attribution on a specific group. Both Shady RAT and "Comment Crew", as well as others with additional codenames, have been associated with the use of HTML comments as a means of COVCOM.

For many years, hackers operating out of China have been attacking a myriad of commercial and government systems here in the US and abroad. The term “APT” or Advanced Persistent Threat has often been used to describe these attackers. While HBGary is primarily a product company selling an enterprise incident response product, the team has been deep into APT analysis for over five years. Most of the analysis work is in direct support of Digital DNA – an automated system for detection of unknown malware and APT intrusions. I presented a technical description of how this attribution works, what is solves and what it doesn’t, at the BlackHat Conference last year. The work is about tracking threat groups – that is, tracking the humans and the human factors behind the digital artifacts we see. There are many hacking groups involved in these intrusions. One such group has often been called “Comment Crew” for their use of HTML comments as a means of command and control. This group has been associated with the recent “Shady RAT” intrusion revealed by McAfee. For this article I am going to give you a technical in-depth tour of how such a group operates.

For starters, the attackers will gain access to the network via spear-phishing. In almost all cases we have investigated, spear-phishing was the initial point of infection. These phishing emails are full of very specific project names, names of associates, official sounding documents, etc. It is very clear that the hacking group is using stolen email to learn about their targets before crafting a very convincing email. This underscores why the recent spate of SQLi attacks over the last few months pose a far greater threat than most people realize.

Exploit and Dropper

Once access is gained into the network, the hacking group places remote access tools into the environment. These are backdoor programs that are downloaded automatically by the exploit email – we called these “droppers”. In the diagram, point A shows the exploit email ‘detonating’ after being viewed by the victim, point ‘B’ is a server where a ‘dropper’ is stored, and point ‘C’ is the dropper backdoor being placed onto the compromised computer.

Once the dropper has established a beachhead into the network, a hacker will access the host and uninstall the original backdoor, replacing it with a new and more powerful backdoor. These backdoors, especially the secondary and more powerful one, are called “RAT”s – for Remote Access Tool. Many of these RATs are custom written and that can be the basis for a great deal of attribution, allowing us to detect the malware in physical memory.

Interaction with the Host

Remember that most networks are firewalled. This means the attacker can’t just make a TCP connection into the RAT program. The RAT program is within the internal network so it must first make an outbound connection to the attacker. The RAT is designed to connect outbound over port 80 or 443, a port that is allowed outbound by almost all firewall policies. Once the outbound connection is made, the attacker can use the established TCP session to interact with the host, download tools, run command line programs, and laterally move about the network. In the diagram, point A is where the RAT makes an outbound connection to a server on the Internet, point B is a server under the hacker’s control, and point C is where the hacker uses the established TCP connection to interact with the RAT program and subsequently the host environment, potentially exploiting additional machines nearby in the network.

One of the greatest challenges for an incident response team is discerning the difference between ‘normal’ malware and an APT attack. As we can see in this example, an APT attack involves a real human at the other end of the keyboard performing actions on the host. We call this ‘interaction with the host’ and we recommend that an IR team pull a timeline of last-access times from the MFT (master file table), browsing history from index.DAT, event log, and other sources to determine if such interaction is occurring. This is a fast and easy way to discern the difference between a non-targeted external threat (which over 80% of all adverse events will fall into this category) and external targeted attacks (of which APT is included, probably less than 2% of all adverse events).

The RAT program doesn’t contain any fancy stealth or anti-forensics measures. In fact, we rarely even see packers in use (a packer is a method of obfuscating a program after compilation and is a low-cost way for a hacker to add anti-forensics to his malware). It seems the most of the covert methods are applied to the way to RAT communicates with the hacker. This makes sense. Consider that most of the intrusion detection capability lies at the perimeter of the network, and this is what the hacker is trying to defeat. Thus, the HTML comment method of configuring and controlling the RAT programs.

Hidden Comments for Covert Communication (COVCOM)

Instead of letting the RAT connect directly to his personal server, the hacker will first exploit a webserver somewhere on the Internet. This exploited webserver will then be used as the ‘middleman’ to communicate with the RAT. The hacker will place a hidden comment on an otherwise normal webpage and have the RAT connect outbound to this page. Using the hidden comment, the hacker will be able to give commands to the RAT. The RAT will make periodic outbound connections, sometimes waiting days before checking the page. The hidden comment will contain an encoded message that the RAT knows how to decipher. In this case example, the hidden data is base64 encoded. In this diagram, point A is the RAT program making a periodic outbound connection, point B is a compromised webserver somewhere on the Internet, point C is the hidden comment on the webpage, and point D is where said comment is decoded into actual instructions for the RAT. An example of such a comment is shown in the next image. It is interesting to note that the hacker has attempted to make the page look like a 404 HTML error page if viewed in a normal web browser.

Example of BASE64 Encoded Hidden Comment

Once the RAT decodes the message, the data becomes a configuration file for the malware. The file has many features, such as the ability to specify which server addresses to use on the Internet, including backup servers, configuration of the check-in times, and even has the ability to completely update the RAT binary in the field (shown in the diagram as a .bmp file – this is actually a normal PE header executable).

The Decoded Configuration File

All of the above technical information can be detected on a host after intrusion. The RAT program itself is near trivial to detect once you know what you are looking for. But beyond that, because the RAT program has certain outbound connection characteristics, sleep timers, and built-in “host interaction” capabilities, HBGary’s Digital DNA lights it up like a Christmas Tree (example shown in image).

Digital DNA Detects Unknown Malware

Even if you had no prior knowledge about this specific RAT, you would have detected it with HBGary. Beyond that, the decoded configuration file can also be found in physical memory – the primary search method used by Active Defense. Regardless of the configuration values, the option headers shown in the example above have a specific pattern that can be detected quite easily, even if fragmented over multiple buffers. This is exactly the kind of information I am referring to when I talk about “actionable threat intelligence”. Once you know about the attackers TTP’s (tactics, techniques, and procedures) you can encode this into an enterprise-wide scan. We call it ‘continuous protection’ when you adopt continual scanning while also updating the threat intelligence as you learn more about the attacker. In essence, you are applying attrition against the attacker’s presence in your network. For example, if you know how to detect the above configuration file, then the attacker has to change the way that configuration file looks to defeat you – something that also requires them to recode their parser in the malware. Hence, you cost the attacker time and money. That is a Good Thing.

I hope this gave you a somewhat concrete tour of how a real APT covert communication (COVCOM) channel works. Also, I hope it has illustrated some of the threat intelligence that you access on the host. Using enterprise-wide scans, your IR or security team can put a severe dent in the APT presence in your network. As far as product solutions to enable you, obviously we build HBGary’s Active Defense. If you are interested in continuous protection and threat intelligence, we offer 50-node evaluations of Active Defense that can be installed on a laptop. We also offer a deploy-on-demand license for incident response teams (our 500-node pack has been quite popular), as well as the perpetual node model for full enterprise proactive deployments.

-Greg

Shady RAT is Serious Business

2011-08-15T11:28:00.000-07:00

Ira Winkler makes some interesting points in his CIO article on Shady RAT. I tend to agree with his observation that security vendors spend too much energy infighting when we all should be facing a common enemy. It is true that Shady RAT is just one of many other, similar attacks. There is no harm in trying to draw attention to the elephant in the room - APT is a grave and serious threat to U.S. companies as well as national security. Shady RAT may appear to be 'sloppy' but it can still be APT. Within infosec the term APT has been debated - but we at HBGary have a very simple definition: if there is interaction with the host, we call it APT. Now, most of the attacks we deal with are targeting intellectual property and appear to have state sponsored underpinnings. The attackers usually leave tools behind, additional backdoors, etc., but none of these are very complex. The malware and techniques are mostly unsophisticated and sloppy, but yet they succeed and remain persistent. Our assumption on this - APT does the minimum necessary to get the job done. If they don't need hard core boot sector viruses and kernel rootkits, they aren't going to use them. We as an industry have a responsibility to protect our customers from a very serious and evolving threat. Downplaying the seriousness of this threat undermines the reason we are here.

-Greg

Command Line Programming with Responder PRO

2011-08-09T13:52:00.000-07:00

One little known feature of HBGary’s Responder product is that it ships with the full source code to a command-line version. This command-line version of the product can be customized for automated tools, batch processing, and statistical utilities. HBGary is still working to produce an 'official' documentation on the SDK, but in the meantime I figured I would walk the more adventurous of you through some code.

First you need Microsoft Visual Studio. I use VS2008 Pro Edition with version 3.5 SP1 of .NET. In the SDK subdirectory of your Responder installation, you should find the ITHC directory. Just a backstory, but ITHC means Inspector Test Harness Client – it was originally a test harness used by our QA team that eventually proved so useful for batch processing that we included it for customers. The code is written in C#.

When I first opened the .sln file on my Responder install, I found that the project file needed some tweaking. Your mileage may vary, but here are some steps I had to take. First, the references to all the Responder DLL’s were broken. By editing the .csproj file I was able to fix this. The trick is to use a HintPath variable with a relative path to the main install directory, which is two folders above the ITHC directory (see image). I’m not sure why it shipped this way, but alas I was able to fix it.

Fixing the references

Now, in most cases, I like programming in Debug mode so I can single step, use breakpoints, inspect variables, etc. I ran into a snag with my debug build and had to get one of the HBGary engineers to take a look. Again, it was a configuration thing. When you make build settings, the platform will probably be set to AnyCPU. You will need to set the platform target to x86 (see image). This has something to do with mixed mode code and if you don’t set this to x86 you will get a binding error when you attempt to run the ITHC exe. Lastly, I set my output path so the ITHC.exe ended up in the main Responder install directory (see image).

Setting the platform target

Setting the output path

Running the tool requires some precise command line arguments (see image). The project path needs to be as shown path/projectname/projectname.proj and the path to the memory image needs to be fully qualified. If you want to change any of that, you can edit the code in NewProject() and OpenProject() to parse the path differently. At this point I had a fully functional ITHC.exe that would analyze Windows physical memory snapshots.

Command line parameters to the tool

Most of the analysis magic happens in THCAnalyzeFile(). The project file ends with the .proj extension and this will be created or opened if it already exists. There is also a .tmp file that contains cached lookup data for Responder which only exists after an analysis. THCAnalyzeFile() will handle all of this.

At this point I need to explain packages and classes. In Responder, a package is any binary object. For example, the physical memory snapshot is a package. Every extracted livebin is also a package. If you import a file for static analysis, that file is considered a package.

Both packages and classes can have parent/child relationships. The difference is that a class is simply a container without any associated binary data. Think of it as just a folder. In fact, in the Responder GUI, classes are shown as folder icons. Just remember that packages can have child classes, classes can contain other classes, classes can contain packages – there is no restriction on the way you nest these objects.

Around line 249 in the ITHC example you will see the creation of the root package (see image). Every project has a single root package that everything else will reside under. Usually this package has no associated binary object and is simply a placeholder. We usually set this to the name of the forensic case – such as “Case 04321”. In Responder’s GUI, the root package is always shown with a safe icon. Depending on the project type, a class will be created directly under this root package. The name of this class is very important and affects the kinds of things Responder will let you do. So, for a physical memory analysis you need to name this first class "Physical Memory Snapshot". You will see this created around line 266.

root package, bulk update, named attributes

Now just a word on event management. Responder has a robust event alerting system that will post an event to your code whenever an object is modified. You could subscribe to these events and be notified if the user changed a property of an object anywhere in the GUI, for example. But, there is a flipside – if you make a large number of changes all at once you will flood the system with these messages. Most of the time if you are going to change a bunch of objects all at once, you want to disable events for a short time. To do this, you use the BeginBulkUpdate() and EndBulkUpdate() methods. You will see these in use around line 249 (see image).

Around this same section of code you will also see named attributes being set on the case. These attributes are being applied to the root package, the one that shows up as a safe icon when you view it in Responder’s GUI. Any object, including packages and classes, can have named attributes set. The attribute system is typed and the first letter of the name indicates the type. See my previous post on plugin development for a description of these.

Around line 293 you will see the creation of a second package. This package is the one associated with the physical memory snapshot. It is placed under the root node and folder. You will also see the creation of something called a snapshot that is then linked with the package. This is how you link a binary to the package – via the snapshot object. The snapshot is just a small header of metadata that is associated with the binary file – including the path to the file – and this is set as the “.InitialSnapshot” property of the package. After this step, the package and the binary are linked.

package and snapshot for the physical memory image

The most important function is then called – the AnalyzeMemory function (around line 329). This function performs the bulk of the memory analysis. It returns true or false depending on whether it understood the memory snapshot. Just a note; it will return false if you don’t have a valid license. If you have the free version of Responder CE, you still have a license file that must be present or this call will bail out on you.

After analysis is complete, the analysis history is updated to include “WPMA”. This tells Responder that “WPMA” analysis has already completed, so it won’t attempt a second analysis later. Note: WPMA means Windows Physical Memory Analysis. Responder has other analysis types that can be added to this history. You can also add your own for reference later.

Now that analysis is complete you can parse the datastore, query all the found windows objects, processes, modules, etc. You can also query the DDNA results if you are using the Pro version. Some object types, such as control flow, disassembly, dataflow, graph objects, and recon traces are only available in the Pro version. However, the results of the windows memory analysis are fully available in all versions, including the free CE version. See the THCDumpProject() function for more information on parsing the project’s object tree.

Package: ws2_32.dll Parent Package: svchost.exe Length: 0 bytes. Class: Symbols Class: Strings Class: Report Items Class: Global Strings: Package: vmwaretray.exe Parent Package: VMwareTray.exe Length: 0 bytes. Class: Strings Class: Global Class: Report Items Class: Symbols Strings: Package: msctf.dll Parent Package: IEXPLORE.EXE Length: 0 bytes. Class: Strings Class: Symbols Class: Global Class: Report Items

a short snippit of output from the THCDumpProject() function

For those of you using the Pro version, ITHC includes examples of not just physical memory analysis, but also extraction of livebins and code-level analysis of extracted livebins. If you made it this far, then take a look at AnalyzePackage(), AnalyzeExtractedPackage(), and ExtractPEImageFromMemory() to get more familier with the code level analysis features. I hope that I can write some more specific posts about these features in the near future.

ITHC.exe analyzing a memory snapshot

Because the ITHC utility is written in C# it’s very easy to interface to other systems. Microsoft has done a good job building a robust set of API’s that can be used for SQL database access, serializing files, communicating over the web or TCP/IP, regular expressions, etc. All of this is at your fingertips and can be interfaced with the results of physical memory assessments. I am partial to building bulk analysis tools for large directories of memory snapshots. You are only limited by your imagination.

The SDK directory should be in your Responder install directory. If you are using the free Community Edition you may not have the SDK directory. In this case you can download the SDK as a small but separate download from the free tools section on HBGary's support site. Visit www.hbgary.com for more information.

Asymmetric Warfare and Cyber Terrorism

2011-07-26T10:04:00.000-07:00

In the newly released document, “DoD Strategy for Operating in Cyberspace", the Pentagon states that “while the threat to intellectual property is often less visible than the threat to critical infrastructure, it may be the most pervasive cyber threat today.” Pervasive, yes – but not necessarily the most dangerous.

In 2003, I founded my company, with the help of the federal government’s Small Business Initiative Research (SBIR) program, to develop products to counter these advanced unknown, stealth cyberthreats today often referred to today within the security community as Advanced Persistent Threats (APT).

While the APT threat is significant, the attacker can take months or even sometimes years to steal the information. However, the recent attacks made by small hacking groups illustrate a highly more tangible, immediate, and potentially more severe form of economic damage. It is appropriate to classify these acts as asymmetric warfare, and possibly as a type of cyberterrorism.

In contrast to APT threat actors and other traditional cyber criminals, cyberterrorists are not motivated by monetary gain. Instead, the cyberterrorist wants to cause grave harm or economic damage as quickly as possible, and to get attention for it. Attacks may be economic, political, or even shutting down the power in the dead of winter. The technical aspects of the attack may be similar to APT, but the intent and goal is wholly different.

Cyberterrorism first was a buzzword in the late 90’s associated with power outages and explosions orchestrated over computer networks. These types of attacks seemed like the digital equivalent of IED’s. While traditional terrorists clearly use the Internet to recruit and communicate, we operate under the assumption that the ‘ground of action’ is still the physical world – think suicide bombers. But, recent events have shown that attacks don’t have to be kinetic to cause damage. The ground of action can be entirely in cyberspace and damages can be measured in billions of dollars of stock value and the threats to persons are very real.

Edit: There are different views on the definition of cyberterrorism. In 'Computer Attack and Cyberterrorism: Vulnerabilities and Policy Issues for Congress', Clay Wilson defines two forms of cyberterrorism:

Effects-based: Cyberterrorism exists when computer attacks result in effects that are disruptive enough to generate fear comparable to a traditional act of terrorism, even if done by criminals.

Intent-based: Cyberterrorism exists when unlawful or politically motivated computer attacks are done to intimidate or coerce a government or people to further a political objective, or to cause grave harm or severe economic damage.

Since the early 2000’s, ‘electronic jihadists’ (i.e., Younes Tsouli, Mohammad Peerbhoy, etc) and other hacking groups (many can be researched on www.zone-h.org) have been content with web defacement and the occasional DDOS. But, these actions never gained the media attention like the recent spree of hacks in 2011. This is, in part, due to the advent of social networking. Former British Prime Minister Margaret Thatcher once stated “Publicity is the oxygen of terrorism”. Anyone studied in matters of terrorism knows that the primary goal of terrorism is media attention. The act is secondary to the message.

Younes Tsouli and Mohammad Peerbhoy, both criminal hackers working with Islamic extremist groups (photos via Associated Press)

A small sampling of criminal hacking groups operating in the Middle East. All of these groups are at least as-skilled as the current Lulzsec/Anonymous hackers, as evidenced by similar techniques, use of SQL injection, etc. The myth that traditional terrorist groups don't have access to hacking skill is simply outdated. (groups via zone-h.org)

In the words of William Gibson, “Terrorism is ultimately about branding”. Every press release, tweet, and claim is part of that brand to raise awareness for their cause or message. And, the media can function as an extension of the group’s propaganda machine. As TechCrunch columnist Paul Carr recently pointed out in his piece on the media coverage of the now defunct LulzSec group, most journalists were all too happy to hop aboard the ‘Lulz Boat’ and parrot propaganda verbatim without a hint of criticism and provide ‘celebrity fluff’ reporting. Paul especially calls out online journalists and bloggers as “downright shameful” for showing support for these criminal hackers. Gene Spafford, the professor and director at Purdue University and a leading security expert, has also objected to how reporters romanticize criminal hackers, drawing a parallel to computer virus authors in the early 90’s portrayed as “swashbuckling, electronic pirates” (pointing out that their legacy is now costing billions in damages).

Even in recent days, reporters have used lofty, inconsistent terms such as “masked crusaders,” a “loose hacker movement” and an “online activist group” to describe Anonymous. The fear of retribution by the criminal hackers within this group is real. No one wants to become a target. News organizations need to take a step back and take a close look at how they are covering these incidents and make sure they aren't enabling these groups’ propaganda machine.

Edit: as a case in point, notice the significant lack of the word 'criminal' when media reports on Anonymous/Lulzsec. To illustrate, here is how reporters/bloggers described Anonymous in the 24 hours following the Monsanto/Booz Allen Hamilton attacks:

"Online activist collective" - CNET
"hacker group" -- IT Business Edge
"Hactivist collective" -- The Inquirer
"Hacking Group" -- MSNBC
"Hacktivist Group" -- SC Magazine
"Hacker Group" -- WSJ
"Hacker Group" -- Network World/IDG
"Notorius Hactivist Collective" -- The Register
"Group of hactivist computer-savvy hackers" -- Economist
"Loose-hacker movement" -- Forbes
"Masked crusaders" -- Time
"Cyber-activist group" -- Financial Times
"Hacker Group" -- Dark Reading
"Online Activist Group" -- Associated Press
"Hacker Group" -- BBC News
"Hacking collective" -- NY Times
"Hacker Group" -- Washington Post

While the threat landscape is always changing, we must continue to highlight that a real criminal is at the other end of the keyboard, and that he is persistent and will keep coming back. While the DoD outlines some important initiatives for a more secure cyberspace, we, as citizens, also have a role. Just as we all participate in our local neighborhood watch to keep our physical community safe, we, as Internet users, need to be vigilant and work together to ensure our cyberspace remains safe.

-Greg Hoglund

Scripting with Responder™ Community Edition

2011-06-23T11:59:00.000-07:00

One of the most powerful features of Responder (all three versions, including the free Community Edition) is the ability to write custom plugins. The entire application is basically a GUI over an API. You have the ability to access this same API and extend the application in any way. HBGary hasn’t produced an official SDK document yet, so it’s best to learn by example. For this exercise, I am going to illustrate a plugin that ties information from Responder into Google maps.

First, you should become familiar with the object tree. The object tree (shown in the graphic below, point A) illustrates how the data is organized within Responder after a physical memory snapshot has been reconstructed. You can query any of this data directly using the Responder API’s. For example, you could query low-level details about running processes (point B).

For this example, we are going to query the open network sockets. These are reconstructed from internal undocumented structures within the kernel (the same ones used by tcpip.sys and afd.sys). Even if a rootkit is hooking netstat, the data would still be revealed in Responder. In our example, we have some outbound connections to China. Using our plug-in, we are going to read the connection data and plot the location of the registering entity using Google Maps.

To load the script, first go to the script TAB and select OPEN. Once open, the script will be visible in a code-editing window. Press the PLAY button to load the script.

As you can see, the script is written in C#. Almost all of the GUI components in Responder are written using C# and, for those who haven’t tried it, you will find it to be very similar to Java. The language is very easy to learn and use.
After we load the plugin, the list of network connections are obtained along with registration data. The address of the registration is then plotted on Google Maps.

When a plugin is loaded, the OnLoad function will be called with a list of all open “Documents”. In Responder, a “Document” is a container for data. The architecture requires that the user-interface be decoupled from the data. For those of you with programming experience, you may recognize the “Document/View” pattern here. At any rate, the list of open documents is passed into the OnLoad function and we need to locate the “NetworkBrowserDocument”. The network browser document has the list of all open sockets.


public bool OnLoad(ArrayList OpenDocuments)
        {
            try
            {
// get the frame document, this allows us to add menu items and menu bars
                _frame = FindMainWindow(OpenDocuments);

// see the Launch() subroutine to learn how to launch your own popup window
                Launch();

// init the whois class for later use
                _whois.ResponderForm = (Form)_frame.MainWindowInstance;
                _whois.Inspector = FindInspector(OpenDocuments);
// the network browser document gives access to open sockets
                _whois.Net = FindNetworkBrowserDocument(OpenDocuments);

For those who want to explore other documents, there are several example plugins that ship with Responder. For example, "StringsBrowserDocument" is responsible for showing lists of strings associated with a livebin. "SymbolsBrowserDocument" is responsible for symbols when a livebin has been disassembled (Responder PRO only). The "DriversBrowserDocument" has the list of detected device drivers.

In this plugin example, we have a helper function defined to locate the network browser document. Notice we use GetType() to locate the actual type of each document in the list. As stated, there are many different document types in Responder, usually one type for every visible window or panel in the application.


Logic.NetworkBrowserDocument FindNetworkBrowserDocument(ArrayList documents)
        {
// note the use of IDocument interface class here, 
// use GetType() to compare instanced type against Logic.XXXX where
// XXXX is the document type you are after. Use reflection to see the
// whole list...
            foreach (IDocument doc in documents)
                if (doc.GetType() == typeof(Logic.NetworkBrowserDocument))
                    return (Logic.NetworkBrowserDocument)doc;

            return null;
        }

After finding the network document we can use it to query the list of sockets. Documents will have custom methods and utility functions for dealing with specific data (these are all different depending on document type). You can also access the raw data directly, usually in the form of name/value pairs (my preferred way to do it). This is shown below. Each attribute has a specific name and type as shown.


ArrayList socks = _net.Sockets();

// all objects are referenced by GUID
foreach (Guid socketEntryID in socks)
{
  // src and dest ip are stored as string
  string source = _net.ObjectName(socketEntryID, "sSource") as string;
  string target = _net.ObjectName(socketEntryID, "sDestination") as string;
  
  // remember that 'i' is UNSIGNED 
  UInt32 sourcePort = (UInt32)_net.ObjectName(socketEntryID, "iSourcePort");
  UInt32 targetPort = (UInt32)_net.ObjectName(socketEntryID, "iDestinationPort");

  // the src and dest DNS names, obviously string as well
  string sourcename = _net.ObjectName(socketEntryID, "sSourceName") as string;
  string destname = _net.ObjectName(socketEntryID, "sDestinationName") as string;

  // a bool stores whether the session is TCP or UDP 
  bool bTcp = (bool)_net.ObjectName(socketEntryID, "bIsTCP");

  string sockType = ((bool)(_net.ObjectName(socketEntryID, "bIsTCP"))) ? "TCP" : "UDP";

The socket list is stored as a list of object ID’s. Responder uses a GUID to identify every object in the project database. Every object that is found in the physical memory snapshot is assigned a GUID and can subsequently be looked up. In this example, we have a list of objects which represent sockets. The object ID can then be used to query additional attributes. In this example we query “sSource” “sDestination” “iSourcePort” etc. This is the generic attribute naming system used by Responder. The prefix is a type. ‘s’ means string, ‘i’ means integer, 'b' means bool. There are hundreds of these named attributes across the application - something I hope HBGary writes an SDK document for soon.

After obtaining the source and destination IP’s, our example plugin has a Whois class that is used to lookup the name and address of the registrar. This data is then passed to a browser control along with the URL for Google Maps so the location will be mapped on the right.

This plugin could be extended in many ways. For example, a geoip database or service like ip2location could be used to locate the missile-coordinates for a specific IP address, as opposed to the registration data. The plugin could also be extended to extract IP addresses from artifacts in memory, as opposed to active connections in the socket list. For example, IP address fragments stored in tagged page pool memory.

The plugin is open source and can be downloaded from HBGary’s support site.

Cheers,
-Greg

Ps. Thanks to Dean, the HBGary engineer who wrote this plugin

Changing APT Tactics: Remote-Access Tools vs. Stolen Credentials

2011-06-15T05:20:00.000-07:00

Advanced Persistent Threats (APT) are adaptive, their tactics will cycle after an intrusion takes place. For example, an APT group may start to lean away from RATs (remote-access tools) and rely more on stolen credentials. Let me explain.

An APT initially will enter the network via malware, typically through spear-phishing. Once on the compromised host, the threat actor will place one or more RATs into the environment. If we pick up RATs with our Digital DNA solution or another indicator, we start hunting them down. After targeting and removing these RATs in the customer environment, we have found that specific malware will last about a week, maybe two, before the APT drops it altogether and switches tactics to remain in the network. We commonly see APT shift to using stolen credentials and no malware at all.

Stolen credentials are the very currency of APT. As it turns out, it’s much harder to detect malicious users than to detect RATs. In fact, the APT will use these accounts the same way a legitimate admin would – making it very hard to tell the difference. They create file shares, use the ADMIN$ share, and defrag the hard drive. APTs will even update the AV and patch the machine. Of course, the defrag is actually a way to cover up forensic evidence on the drive, and the ADMIN$ is a way to laterally move malware and tools between machines. One would think that upgrading the AV would be counter to an APT’s self preservation. Actually, the APT updates it purely for self preservation – to appear “normal” as a legitimate admin.

At this point in the investigation, in terms of malware, we are still picking up a great deal of material – but not RATs. When the APT shifts to credentials, we start to pick up password sniffers and keyloggers that have no outside network capability. The malware in this case is entirely focused on obtaining more credentials. Finally, once the customer updates all the passwords, one or more RATs pop out of the woodwork and the cycle repeats itself.

A Brief History of Physical Memory Forensics

2011-05-25T08:07:00.000-07:00

Lately, we have been doing a lot of work around physical memory forensics. Recently, we released the free, community edition of our Responder™ product and plan to release the fourth generation of our memory analysis engine later this year. During this work, I have been reflecting on the origins and advancements in the field of physical memory forensics over the last 10 years.

In the early 2000’s, two headline-making malware infections, Code Red and SQL Slammer, demonstrated the possibility that malware could reside only in memory and never leave a file on disk. In the world of incident response, the evidence challenged the traditional notion of dead-box forensics. It meant that critical data would not be obtained by the traditional forensic methodology. It also set the stage for future malware that would subvert API calls, forcing live response scripts to rely on the OS as little as possible.

Physical memory analysis started as crash dump analysis for debugging, but it soon became apparent that volatile data in memory could contain encryption keys, passwords, and other critical information about recent user activity. From a tools perspective, the well-known dd utility has been able to acquire memory from the start, simply by reading /dev/mem or /device/physicalmemory. Other memory tools also emerged. In 2002, Eoghan Casey documented how Arne Vidstrom’s PMDump tool could be used to dump virtual memory and defeat PGPTray.

Rootkits helped drive development of memory forensics –more for malware detection than evidence collection. In 2003, Jamie Butler demonstrated the DKOM (Direct Kernel Object Manipulation) method for hiding processes by removing items from a linked list directly in memory. This was a data-only attack and didn’t involve any kernel hooking. It would be a few years before researchers like Andreas Schuster and Chris Betz developed memory-forensics methods for finding hidden processes that countered Butler’s DKOM . Things took another significant step forward in 2005 when Sherri Sparks released Shadow Walker, a rootkit that was able to hide sections of virtual memory from scanning tools. This lead to the notion of physical memory acquisition – using a raw dump of RAM instead of using OS- supplied virtual memory reads – as a means for rootkit detection.

Attempts at OS reconstructions didn’t really start until the DFRWS memory analysis challenge in 2005, where George Garner [kntlist] and Chris Betz [memparser] developed process and thread reconstruction for Windows®. Everything changed after this – instead of searching for binary patterns and strings, the memory image was seen as a complex snapshot of interrelated structures and data arrays. A keystone development was the ability to discover the page tables in physical RAM and thus translate virtual addresses to their physical offset. In February 2006, I wrote the first version of this technology for HBGary using the self-referencing physical address pointer trick (AFAIK first publically documented by Joe Stewart w/ the TRUMAN project), and we soon added PAE support. Physical memory forensics had become a hot new area of research. Later that year Mariusz Burdach presented on physical memory forensics at the Blackhat conference. Jamie continued his research as well and presented numerous advances in physical memory analysis to detect rootkits at the Blackhat 2007 conference. Shortly after Jamie’s talk, AAron Walters released Volatility. It were these initial advances with page table translation and OS reconstruction that lead to ”modern” physical memory analysis.

By this time, Brian Carrier and Joe Grand had already released Tribble, a PCI card that could monitor and analyze physical memory. It was later that several commercial attempts were made to build a rootkit protection solution in the form of a PCI card. Via a DHS grant, HBGary was subcontracted to work on a similar project and this lead to a prototype PCI card that could analyze Windows XP and detect kernel hooks. Jamie Butler joined Komoku, which had already built a similar device, around that time. Joanna Rutkowska was quick to respond to all of this and developed an extremely low level software-only rootkit for Windows that could defeat even a PCI-based physical memory read – by reprogramming microchips that are part of the bus controller and I/O chipset. In the end, a hardware solution for rootkit detection was not economically feasible and these projects were never successfully commercialized.

HBGary’s work on the hardware PCI card was the genesis for more R&D memory forensics work to come. We abandoned the hardware approach and developed a software library called WPMA (Windows Physical Memory Assessment) - written in C++ and core to Responder’s memory parser. We later developed a second-generation parser and started reverse engineering all the different memory footprints left by every conceivable version of Windows and service pack (we didn’t analyze NT 4.0 – only Win2K and newer). It took about two years to get the Windows platform complete. This work led to the development of our flagship product, Responder™, and the library that performs the physical memory parsing is integrated into our enterprise product’s Active Defense™ agent as well.

I’ve highlighted only a few of the researchers in this important field of physical memory forensics – there are many others who have also made significant contributions. At HBGary, as I mentioned, we will soon release a completely rewritten version of our physical memory analysis engine marking the fourth generation of the technology. Recently, I was watching the performance testing in the lab and I have yet to see it cap 150 MB memory usage while analyzing a 10-gig snapshot, and it is about 30% faster than our current generation. I will post more details on this work as we progress, as the new engine has many additional features that extend our Digital DNA™ technology.

-Greg Hoglund

Stop PDF Exploits Cold

2011-05-12T12:28:00.000-07:00

I’m happy to announce that HBGary has released another free tool, similar to the Aurora scanner and the Chinese RAT catcher tools we released in past months. This one isn’t looking for malware, however. Acroscrub is an agentless scan of the enterprise that will find out-of-date versions of Acrobat Reader. Adobe is pretty good about patching vulnerabilities, but many machines in the enterprise won’t have the latest version of Acrobat Reader. PDF exploits are a common method used with spearphising attacks and APT intrusions so it’s imperative that organizations keep this software up to date. HBGary has released many popular free tools over the years and Acroscrub is another cool addition to the toolbox.

All of the existing free tools are available to users on the HBGary support site. We have upgraded the security on the community support site and now require two factor authentication for all access, both for commercial customers and for free tools, so that means no more direct downloads. I support this upgrade to authentication and believe it acceptable for legitimate practitioners in the security industry.

http://www.hbgary.com/hbgary-releases-acroscrub

Is APT really about the person and not the malware?

2011-04-19T07:24:00.000-07:00

Maybe the “APT is person not malware” pendulum is swinging to the extreme. Understandably it’s a response to commercial enterprises being obsessed with pure-play malware detection. But what is the alternative? Spend tons of money on consulting and RE/forensic services for years on end? Customers are tired of paying for that. They must build a security methodology that accounts for persistent attackers – something that can be managed internally and that leverages automated detection as much as possible. To that end, detecting APT must include the malware, tools, and codified threat intelligence.

As tired as it is, the ‘hacking exposed’ story hasn’t changed. We must continue to highlight that a real criminal is at the other end of the keyboard, and that he is persistent and will keep coming back. We know that he will use more than one tool, more than one method of entry, and he won’t go away no matter what kind of malware detection you have. But the idea that it’s all about the human and not malware or TTP’s is simply untrue. Malware and TTP’s have a critical role to play in combating APT.

To date this year, HBGary has identified and tracked multiple human threat actors using the science of attribution, many of them operating overseas. Our attribution begins with profiling the CnC, the developer toolmarks, and forensic artifacts left behind after an intrusion. While some RAT’s are “easy to detect - difficult to attribute” (i.e., poison ivy) we have also found modified and custom tools that contain unique indicators. This information can be used along with open source intelligence and link analysis (we heart Maltego) to locate online identities, forums, and social spaces. This can lead to the discovery of real identities – the attacker’s real name, address, and even photographs.

It makes no sense to separate the human from the malware and TTP’s. They are two ends of the same spectrum. This is not a black and white science; it works because humans aren’t perfect. It works because humans are creatures of habit and tend to use what they know. They use the same tools every day and don’t rewrite their malware every morning. They don’t have perfect OPSEC. They put their digital footprints out on the Internet long ago – and it’s usually just a few clicks away from discovery. There is a reflection of the threat actor behind every intrusion. To discount this is to discount forensic science.

Digital attribution is important because it scales. An army of consultants watching your network does not scale, they don’t share their threat data, and they’re expensive. Couple that with out of date methods for determining a breach (imaging a 500GB hard drive to find 200 bytes of actionable data) and you can see why customers want/need a better solution to empower their own teams. This is why researching automated methods for threat detection is so important. Threat detection leads to threat intelligence, actionable data you can feed back into your process to make it more difficult for the attacker to succeed in your network. For example, the endpoint physical memory can reveal decrypted CnC addresses that can plug directly into the perimeter IDS – making your existing investment smarter.

For me, the concept is clear – reverse engineer the endpoint hosts down to the rawest dataset. From this, automatically piece together the parts that appear to relate to suspicious activity. Map this against a database of known malicious behaviors – software, host, timeline, forensic, all of it. Do this automatically and alert on the outliers. HBGary’s Digital DNA does this by using a weighted fuzzy hash of the behaviors. Fuzzy hash because hashes are understood in the enterprise, and weighted because security is a risk management problem that begs for red/yellow/green. The result is huge scalability and effectiveness for a problem that is traditionally expensive and understaffed.

Two new threat intelligence papers CSO's will want to read

2011-04-12T10:13:00.000-07:00

Industrial Espionage in the Global Energy Market

Since 2005, HBGary has been tracking variants of malware created and originated in China that indicate a complex cyber espionage operation targeting multiple industries, including the energy sector. In this new whitepaper, "Industrial Espionage in the Global Energy Market," HBGary provides technical details about these cyberattacks as well as the type of critical data targeted and successfully obtained and sent back to China. This report is restricted release to qualified executives, government, and law enforcement only. Available from hbgary.com

Threats in the Age of WikiLeaks

HBGary has released its threat report ‘Threats in the age of WikiLeaks’ – CSO's will want to read this report. Cyber-threats are evolving fast but we must stay ahead if we are to secure our information systems and our brands. With leak platforms (WikiLeaks, AnonLeaks, CrowdLeaks, InfoLeaks, People’s Liberation Front) comes the increased risk of insider threats and acts of information terrorism. Unlike traditional APT which damages over years, leak platforms represent immediate damage to stock value, profitability, and brand. Acts of cyber terrorism can disrupt systems and business continuity. To date, the severity of this threat has been underplayed in the press – this report exposes the true and dangerous nature of the threat. The report provides immediate and actionable data to help you detect potential insider threats and attacks. This report is restricted release to qualified executives, government, and law enforcement only. Available from hbgary.com

Rootkit Evolution

2011-04-08T20:47:00.000-07:00

Over the last few years HBGary has researched significant advancements in rootkit technology. We are pushing the envelope of what’s possible in the windows kernel. I’m glad to say that we haven’t seen anything in the wild that is remotely close to what we have developed in our labs. So, we are still ahead of the threat. This keeps our Digital DNA ‘frosty’ so-to-speak, but probably further ahead of the threat curve than it needs to be. That’s not a bad thing for people protecting against APT – we want to stay one step ahead of the bad guys. For those who have followed my work in rootkits over the years you probably noticed I stopped releasing public material on the subject years ago. This is because I didn't want to educate the bad guys on how to develop this stuff. But, that doesn’t mean the research has stopped – just that some things should only be briefed behind closed doors.

Cyber Conflict and State Power

2011-03-14T14:14:00.000-07:00

There has been a rapid change in the global security paradigm. Cyberspace has fundamentally changed the stability between state and society. New conflict groups are not tied to any one state. There is a boom in conflict. Dangers come from many sources, not just military. The distinction between civilian, domestic, guerilla, terrorist, and criminal is blurred – small numbers of individuals can inflict great harm upon the establishment – perhaps more-so than any army. Recent activities have been directed at states themselves (Egypt/Iran/US/Estonia/Georgia). International bodies have been notably absent in their duties to protect its members (UN/NATO).

The security environment is defined by the state’s weakness in cyberspace. The borders are permeable because the information flow is weakly controlled – there is no better example than Wiki-Leaks. The threat today is not from the projection of power, but instead from the projection of instability. Power projection defines a state's ability to influence and enforce their policy globally, which can be seriously harmed by not applying equal effort in cyberspace (Georgian conflict). You need a passport to travel to a foreign land but can reach that country's marketplace in milliseconds via cyberspace, without ever crossing a checkpoint. Any group can influence a state's population using social media outlets, including but not limited to instigating riots or uprisings (Egypt/Iran), as well as spreading disinformation.

The U.S. war on terrorism is an example of this fight. The shadowy cell-based terrorist network cannot be linked to any one state. We live in an increasingly borderless world system. Groups are recruited and mustered entirely on the international stage of cyberspace, and include members from many countries. New conflict actors are flocking to cyberspace for communication, organization, and as a medium of attack – both directly through criminal assault and through influence campaigns and control of media. Threat actors include transnational criminals, warlords for profit, economic insurgents, state intelligence, and agents of industrial espionage.

Cyber is a zone of lawlessness and conflict. While not armed in the traditional sense of explosives, the landscape is ripe for soft munitions that can alter industrial operations with a few lines of code (Stuxnet). The traditional means of peaceful activists have migrated to acts of criminal nature, favoring methods such as denial of service, intimidation, theft, harassment, defamation, disinformation, hacking, and cyber-thuggery. Peaceful protests such as sit-ins or boycotts have been replaced by violations of Federal statutes without fear of prosecution, and states are increasingly challenged to bring charges against the perpetrators due to the ability to exploit the world stage of cyberspace.

When the citizens of one nation wage cyberwar against the government of another, the international treaties that trigger the right to wage war (jus ad bellum) are absent, and the conduct of protecting a nation under these acts are not governed (jus ad bello).

The implications of all nations not cooperating to develop and enforce regulations, treaties, extradition, and establishing cyber checkpoints will continue to occur with increasing severity.

-Greg Hoglund

Malware Persistence in the Cloud

2010-12-13T08:43:00.001-08:00

The cloud is certainly going to change some things about malware infection. When a desktop is reset to clean state every time an employee logs in, you now have to wonder how malicious attackers are going to maintain persistent access to the Enterprise. This is similar to what happens when an infected computer is re-imaged only to end-up infected all over again.

There are several ways to maintain persistent access without having an executable-in-waiting on the filesystem. Memory-only based injection is an old concept. It has the advantage of defeating disk-based security. One common observation is that such malware doesn't survive reboot. That is true in the sense that the malware is not a service or a driver - but this doesn't mean the malware will go away. Stated differently, the malware can still be persistent even without a registry key to survive reboot. This applies to the problem of re-infection after re-imaging (a serious and expensive problem today in the Enterprise) and it also applies to the future of cloud computing (where desktop reset is considered a way to combat malware persistence).

The most common method for persistence without reboot is re-infecting the system from a neighboring, already infected system. It has sometimes been called the "Hack Finn" model - two or more malware programs that know about each other. Unless you kill both of them simultaneously the one will re-create the other. In today's world, the neighbor doesn't need to be physically nearby - it can be anything that has some access path to the other machine. This neighbor could be a social networking peer, a shared desktop (think exploited .ini), or a machine with lateral domain credentials.

Another way to maintain access is to store crafted (exploit) data in a commonly used document - think PDF exploit but for google docs. User's in a cloud based environment are going to have persistent data storage, whether this is up in the cloud or down on a USB stick. When the execution environment is constantly reset, as it might in a desktop cloud, the attacker can move method of persistence to the data itself. The malicious code must obtain execution cycles - think of the cloud based desktop simply as an execution space. The user opens said boobytrapped document every day as part of their work, and the malicious code activates. Or it can be delivered via a system used on a daily basis, such as an exploited image on an ad-banner, or the little calendar program in the corner of your timecard system.

For the window of time the user is interacting with the desktop, the code has execution cycles. This is when data is most at risk - this is when other documents are open, other social network contacts are online, and the user's access token is live and can be used to access other resources.

Remember, the attackers always adapt to new environments. The cloud just provides new ways for our adversaries to attack us.

Intel Validates the End Node

2010-08-19T20:44:00.000-07:00

In the security industry, there is an obsession with the perimeter, which is why this Intel-McAfee announcement, at first glance, seems so surprising. McAfee represents visibility and control at the end node - the environment where the bad guys actually live. Intel's investment into end-node security is significant because it represents a fundamental shift in security . Adoption of mobile technology is faster than ever before and the chip-war cannot continue to be based solely on horsepower or power consumption - there needs to be a platform and security has to play a big part. An integration between hardware and software is natural and can create a platform for the future mobile user. To understand this acquisition, you have to think ahead. Intel knows the future is in small, always-on access-to-the-cloud -- and they can obviously deliver the horsepower required for immersive environments such as High-Def, 3D that will become the norm. But the future is not without risk. To create a secure online experience for the user, Intel understands that the security features in the chip need to be united with application layer surfaces. While the benefits for Intel are clear, this deal also might be the best thing that ever happened to McAfee. They have the domain knowledge about hooking into software flows and scanning for patterns, but the end-node for McAfee is the PC -- not the future devices. Intel's business has always been about massive scale and this might push McAfee into the future. This event may signal an emergence period of security integration into the end node. The perimeter is vanishing daily - this event just validates that the end-point is where the action is.

2010-07-18T13:16:00.001-07:00

Blackhat is almost here again! This has always been one of the coolest security shows with great research presentation. The show is a great place to catch up with friends and colleagues (most of whom I only see once a year). In many ways, Blackhat is the reason I decided to get into the security industry back in the late 90's - I lived very near Jeff (the founder of Blackhat and Defcon) and he inspired me to reach for something beyond just a network admin job. I am glad to see that both Jeff and Blackhat have survived. Hope to see you all at the show!

A dose of clarity for the "APT"

2010-05-14T13:03:00.000-07:00

Finally a dose of clarity for the "APT". It is an overused word, one used to sell security products, even if these are the 'same' security products you have been using for the past 10 years. In his recent Spotlight report, Josh Corman of The 451 Group really laid out the term, where it came from, what it means, and more importantly WHAT IT DOES NOT mean. I posted a similar blog on the topic a while ago and got comments like “it’s the person, not the malware”. I know that. I’ve been saying that for years, but how the term APT is used today by most people make it sound like it's ONLY malware. In fact, it’s not only malware - it’s the actors and their intent. Josh gets this -- more importantly he felt the need to speak up about it. I agree, it’s about the ADVERSARY. Malware is just a tool, one of MANY that these adversaries use. Focusing on one aspect of security is not going to make you secure; rather, it’s understanding what they are trying to get. I would argue a 'slightly' different take in that I don’t necessarily believe it’s only scarce resources these adversaries are after. They want actually anything that gets them 'closer' to the info they are seeking. This could be money, IP, marketing plans, hiring plans, IT resources, or personally identifying information. Because while APT were at one time ONLY focused on military, they’ve expanded.

I also applaud Josh’s note that APT uses existing tools. Other experts seems to think this is not the case, or that they don’t use packed malware, or that APT don't use botnets. Why wouldn’t they? It seems the more that someone tells me what APT isn't, the more it becomes clear they have no idea what APT really is. If APT use existing malware, which I’ve always maintained, then packing is par for the course, because it's a cheap way to defeat signature based detection definitions at the gateway and host alike. Perhaps the APT did some recon into the network and learned that using XYZ packer would defeat the AV solution at the desktop. The US government created the term APT to characterize a class of threats originating from Chinese and Russian state sponsored and criminal elements, not to classify a type of malware.

Since the government coined the term "APT" it has always been about Russian and Chinese attackers, BOTH criminal and state sponsored. For the government, it's very difficult to draw a line between the two. If you understand information operations, then you know that APT will use any and all means at their disposal to achieve the mission objective. If this means use of packers, so be it. The same applies to _any_ rule or definition someone puts in my face telling me what APT is and is not. An IO campaign will include a full spectrum of capabilities. In the context of cyber, each attack on a government facility, contractor, or commercial entity could be a single operation that is part of a larger campaign. Operations could be designed to assume false personas such as impersonating college students in a dorm room, or even a false-flag - impersonating the intelligence service of another foreign country. If you truly know what APT is about, you know that you can't start boxing it up and packaging it.

As Josh pointed out, the adversaries are constantly evolving and adapting. The old models are being defeated. The government has known about APT for a while and that new approaches to enterprise security had to emerge. This is one of the reasons that the USAF and DHS both funded HBGary to address this evolving threat. Our company and our technology are pushing the envelope forward.

Cyber Threat

2010-04-30T04:35:00.000-07:00

I imagine cyberspace like in the movies, as points of light. It's like looking down from an airplane at night - mostly vast darkness but interspersed with brightly lit areas of activity. There is a lot going on in these brightly lit areas: data being moved, software executing. In real life, there are people down there doing people stuff. But in cyberspace, the people are represented by software entities. Software is created by people, and thus is an extension of human intent.

When we talk about cyber threats, we need to conceptualize the relationship between the person and the software entity. Cyber threats are not just about malware, nor just about a human actor. You can't separate the two. It doesn't make sense to talk about human threats that don't manifest in cyberspace. And, malware that isn't backed by human intent would cease to be a threat. Our job is to explore this relationship between a human and his manifestation in cyberspace. The very word 'cyber threat' implies this relationship. It's direct and basic.

Active Reversing

2010-04-05T08:45:00.000-07:00

I first presented active reversing in 2007 at the Blackhat show in Vegas. At that time, the concepts were mostly theory, backed by a few prototypes I demo'd during the talk. It's taken us three years but I am very proud to say we have stayed the course and delivered something that is a true game changer for reverse engineers. It works so well that it's cheating.

Active reversing is when you obtain program understanding via runtime instrumentation, data collection, and statistics. Active reversing puts the focus on volatile runtime behavior as opposed to static disassembly. This transforms the tradecraft of reversing in many ways. For one thing, it promotes reversing to a larger professional audience. People who are already comfortable reading packet sniffer logs can now reverse engineer software. The data collected from a point in memory resembles the kind of data collected by a packet sniffer, the only difference being the data structure of the packet is actually an internal structure within a software program. Observing just a string can reveal what a function is responsible for.

Active reversing empowers you to reverse engineer by exercising a software program's capabilities and features. Consider that runtime code coverage reveals which functions are executing and when. This allows you to simply observe which functions execute in response to an action you have taken with the software. For example, if you want to find the password handling function, just review which functions executed after you typed in the login information. Filtering can be used to remove code that has executed more than once, or code that has already executed in response to another action. This so-called background noise may represent utility functions and general purpose packet handling. Once filtered, you are left with only the newly executed password handler functions. This approach can identify functions when searching for data might be difficult - for example if the data is numeric and not easily predicted. More than anything, this approach is fast. In just a few minutes you can have most of the major features of a program mapped to code.

HBGary just published a very technical whitepaper on software exploitation using REcon, with lots of screenshots and step-by-step how-to. Check it out: Software Exploitation with HBGary's REcon.

The Nature of Funded Threats

2010-03-12T08:57:00.000-08:00

Most incidents of espionage are never reported - even though tens of investigations are currently underway as you read this. Without consolidated statistics it will be hard to really understand the damages caused, even direct measurable damages. Espionage is not new, and the primary threat to your intellectual property has always been your front door. The only thing new about e-espionage is the 'e'. A few years back I gave a keynote at DFRWS titled "Funded Threats", in preparation for which I had to dig up some statistics. Although this was back in 2007, I would posit that not much has changed between then and now. I found some FBI statistics earlier that year stating that industrial espionage and IP theft was costing US companies more than $100 Billion USD per year. That was only a national figure. Measured worldwide the damages would probably be in the trillions. Those include indirect costs. Also of interest is that over 70% of a corporations intellectual property is stored online, digitally. It's easy to visualize the changing risk landscape - malware and information living on the same systems. ZDNet had just reported that in 2006, 80% of all malware samples were being missed by the top three AV vendors. Meanwhile, HBGary had just completed an extensive DARPA study into the nature of current rootkit threats (back then everyone liked to use the term 'rootkit' to describe 'advanced' attack tools). The report went dark, but I can sum up some salient points: as early as 2006 there was a global "cyber-arms bazaar" where cyber weapons could be bought and sold. We had samples that would bypass everything (by everything, I mean all the commercial and freeware anti-malware solutions). Most of the participants in this growing black market were foreign to the US. The stuff was well tested and well written, backed by a real software development lifecycle. One of the most specific things we learned was that every major AV and IDS product was installed and tested against during the QA process. When we presented this material to the DoD we joked around saying "the bad guys malware is higher quality than the commercial stuff they are exploiting". It was also becoming clear that information monetization was occurring - mostly identity theft. However, certain activities such as source code theft established themselves well before the mob figured out how to bank online. If you rewind back five years or more, you are going to find lots of interesting indicators that foretell our current situation. The people who operate malware today are the same people who operated malware back then, it's a career choice.

Responder 2.0 Released!

2010-02-08T15:33:00.001-08:00

We have been slaving away long hours since early December last year, but we made it. The 2.0 release of Responder is released. It's amazing. Hard to believe, but it's been two years since we announced Responder at the CEIC show in Vegas. After so many years of coding, I have come to understand there is a big difference between building commercial products, and just building hacker tools. There is so much work you wouldn't expect going in. I would like to publicaly thank my engineering team for all their hard work.

For fun, here are some sneak peeks of Responder 2.0

Above is the REcon recorded execution timeline from a particularly nasty APT malware that plagues the DoD by constantly hopping from NIPRNet to SIPRNet and vice versa, primarily through the dreaded USB stick.

The above diagram is a second malware. This malware puts the capital 'P' in APT. We were first introduced to this one back in 2005 during a DoD infection. The above sample we obtained just a few weeks ago, and clearly it's a variant of the original - that's almost 5 years and still going strong! The diagram shows the code and dataflow for a function that enumerates machine-specific data after an infection drops, this is then sent via the C&C channel when the malware reports in. This kind of behavior is very common in persistent malware, as the bad guys need to keep track of their machine-infection inventory. A little story goes like this: location 1) this queries the uptime of the machine.. 2) checks whether it's a laptop or desktop machine... 3) enumerates all the drives attached to the system, including USB and network... 4) gets the windows username and computername... 5) gets the CPU info... and finally, 6) the version and build number of windows. Quite thorough eh?

In a nutshell, Responder 2.0 cuts through APT like butter. More Info.

See You in Sac-Town at ISSA

2010-02-04T23:09:00.000-08:00

I will be giving a talk on APT at the Sacramento Valley Chapter of the ISSA, Feb 19th. This should be fun, since Sac-Town is my home turf. When I first thought of moving to Sacramento, I had horrid fears, but these were unfounded. Sac is a really cool place in California. Not just because it's the Capital, but it has great and diverse ethnic restaurants, more trees than you can count, and EPIC FISHING within an hour drive in any direction. It's also a great place for a tech company right now. Way better than the 'valley. Housing is much more affordable, and there is a great hiring pool with HP, Intel, and two major universities in the area. There are numerous initiatives to support small business growth, including tax incentives to build out on either Mather or McClellan AFB (I drive past Mather on the way into HBGary in the morning). Sac also has hacker history too, callbacks to the old days. HBGary already reached out to the locals last year when Penny outfitted the Sac PD with free copies of Responder Field Edition. In the local area we have given memory forensics training at International High Technology Crime Investigation Association (HTCIA) and Internet Crimes Against Children Task Force (ICAC). There is a lot going on around here and I glad to be a part of it.

Is the term ‘malware’ eclipsed by ‘APT’?

2010-02-02T22:37:00.000-08:00

I am wondering why we need to change the term 'malware' to 'APT'. APT stands for 'Advanced Persistent Threat' and the term was cooked up by the Department of Defense to describe malware that worked, as opposed to malware that got caught. The term APT has been in the press since the very public Google hacking incident. I like the term APT, but I still wonder why we need it. To its credit, APT is a great term because it accurately describes the problem. On the other hand, it also confuses people. It makes APT sound different and new, when in fact there nothing is new.

Many of us have been analyzing malware for years and functionally the malware today is just the same as it was three years ago, but back then we didn't call it APT. In general, this got me wondering why people change the terms used to describe something (for anything in life)? For this malware and APT thing the best I could come up with is that many existing security companies adopted the term 'malware' many years ago, and thus their product offerings become tightly associated with the word 'malware'. Over the last few years or so, as people within the DoD and elsewhere realized that existing security investment wasn't adequate to protect their network, instead of calling the investment a failure for missing the malware, they instead decided to call the missed malware by a new name. Think: "This isn't malware; it's something new, so of course the millions we've spent doesn't address it!". A stretch?

The fact is this - malware has always had the ability to be updated in the field, it has always been able to be remote controlled, and it has always had the ability to spawn a remote shell to a live attacker. And, it has always had the ability to scan the file-system for files like source-code and CAD drawings, and it has always had the ability to exfiltrate those files. At all times and without exception, these malware programs have been operated by real and persistent humans at the other end. The malware doesn't operate itself, it's not an automaton. For the last 365 days, I just called that malware.

Post Execution - A New Paradigm for Debugging Malware

2010-01-28T12:55:00.000-08:00

With the upcoming 2.0 Release of Responder, REcon plays a much more integrated role in the analysis of malware. The report automatically details all the important runtime behavior from a malware sample, including network activity, file activity, registry activity, and suspicious runtime behavior such as process and DLL injection activity. All activity is logged down to the individual disassembled instructions behind the behavior, nothing is omitted. Code coverage is illustrated in the disassembly view and data samples are shown at every location. This is like having a post-execution debugger, with registers, stack, and sampled data for every time that location was visited.

Post-execution debugging is a paradigm shift from traditional interactive live debugging. Traditional debugging is cumbersome and requires micromanagement to collect data. The traditional debugging environment is designed for CONTROL of the execution, as opposed to OBSERVATION ONLY. Typically, the malware analyst does not need to control the execution of a binary at this level, and instead only needs observe the behavior. HBGary's REcon approach to debugging is far superior because the analyst can see and query so much more relevant data at one time without having to get into the bits and bytes of single-stepping instructions and using breakpoints. It's like having a breakpoint on every basic block 100% of the time, without having to micromanage breakpoints.

HBGary and Palantir

2010-01-26T11:44:00.000-08:00

I am very excited about our new partnership with Palantir. The Palantir link analysis capability is outstanding. Our team here in Sacramento is currently processing over 1.5 gigs of malware drops per day for Digital DNA, the new link analysis capabilities will allow us to move from the malware developer to the actual individuals who are operating the malware. We are already able to use forensic toolmarks to identify the individual malware developers, but this next step of analysis is to actually track those that have purchased or funded the development of the malware weaponry, and subsequently operate the malware in live operations. While tracking the developers themselves is easy due to the nearly impossible-to-avoid toolmarking caused by code and compiler, tracking the operators is much more difficult. But, HBGary is going to make a dent in this problem. We are approaching malware the same way an intelligence agency would.

Using Handle Tables in Physical Memory

2010-01-25T17:18:00.000-08:00

One of the challenges we face at HBGary while developing Responder is the sheer volume of information available in physical memory. We have to reverse engineer a large volume of underlying data structures within the Windows operating system, not just for one operating system but for every single version of Windows (that includes service packs). In other words, a lot of time spent in windbg. One of the more interesting information sources available from the kernel is the handle table. Handle tables are extremely powerful and allow you to determine, for example, what files and registry keys are open by a given application.

Consider the way malware drops and installs itself. Many times, malware will inject itself into other processes as part of its "installation and deployment". At HBGary, we divide all the various behaviors of malware into categories we call "malware development factors". One of those malware development factors is called "installation and deployment" and it's all the stuff that malware does to survive reboot. In our Digital DNA(tm) we have well over 400 rules to detect just this kind of behavior. For example, malware may install itself as an extension to Internet Explorer and sniff online banking credentials. There are many ways to detect the presence of these capabilities. File handles can be used to detect when malware has been injected into secondary processes.

Malware injected into Internet Explorer opens a suspicious log file. This data is obtained from the kernel handle table.

Another anomaly that you can look for is a strange file path or executable name. Simply compare all the paths that are available in all the modules. In Responder, you can double click the drivers folder,m or the 'All Modules' folder, to view all drivers or modules along with their paths, respectively. Most modules are in consistent locations. When you examine all the module paths together in a single view, the anomalies will stand out. You don't really notice this until you see it in reference to all the other paths on the system. Seeing everything at once helps you detect an outlier quite quickly. This is just one of the types of things you can do when you have all the information at your fingertips.

Executables that are located in a suspicious temp directory. Also, one of executables has a non-standard file extension (.tmp).

A very suspicious kernel-mode device driver. This driver has no path, just an odd name with no file extension. This is a rootkit that was later identified as a variant of the "Black Energy" rootkit family. Notice that Digital DNA has automatically identified this driver as highly suspicious.

Executables that are running in a process that don't have a corresponding module name or path. These are very suspicious and we determined these were injected DLL's that were unlinked from the module list. Notice that Digital DNA has automatically identified these as highly suspicious.

One thing I really like about Responder is that you can sort the information in columns alphabetically and bring the outliers right to the top. Another thing that I like about Responder is that you can also write plug-ins that extend the functionality of the user interface at any point. For example, I could write a regular expression that would search all the file handles for certain patterns and that could include executables used in multi-stage installation, such as CMD.EXE or RUNDLL32.EXE, or files that appear to be in suspicious paths.


Regex r = new Regex(".*\\\\temp\\\\.*\\.exe$", RegexOptions.IgnoreCase);
Match m = r.Match(stringData.Name);
if (true == m.Success)
{
         // add an item to the report, associated with the DLL
 IReportObject wo = theDLLPackage.CreateReportObject(
             "suspicious exe in temp path",
             "This path looks suspicious, examine further");
}

Responder scripts are written in C#.

I could even automatically add those to my report. The upcoming 2.0 release of Responder has an interactive report where you basically just drag and drop any specific data item to your report and drop it where you want the report item to appear. The dropped data actually just appears right there in the report at that location, including a description of what it is. Of course, this is editable by you, and you can expand upon it, but it makes it very easy for you to assemble a collection of those things that you find important in the memory snapshot. In summary, the report is a way to export and print the data I care about. I guess it also gives me a way to come back and reference those report items later (if you find yourself re-exploring old memory images). This is a short summary of some things you can do using the handles and the paths that are available to you in a physical memory snapshot.

Puffer Machines, El Al, and Defense in Depth

2009-12-29T11:41:00.001-08:00

Airline security is a great case study in large systems security, and specifically the challenges of defense in depth implementation. While the U.S. will rapidly get back to business-as-usual, forgetting the near disaster of a Christmas day airline bombing, other countries have the threat of attack put in their face daily. This isn't something a person wishes the U.S. to experience, but you cannot ignore the ongoing annealing effect this has on the security posture of these foreign communities. It translates into large expenditures of money being applied to security because the threat is real. Contrast the TSA here in the States with El Al in Israel. El Al catches terrorists using multiple layers of security. The first defense is a knowledge of the world they live, the status of their enemies and who is likely to target them at any given time. This means putting intelligence to work and informing different organizations so they can work together. The second layer is a skilled person interrogating passengers. El Al realizes it is not a right to fly, it is a privilege. The screener focuses on the person. El Al respects the power of human threat detection by incorporating the interview into the screening process. The traveler may be asked to produce receipts for the places he reports to have stayed. If the person makes the screener nervous, that person gets set aside for more in depth screening. Simple. This increases the time it takes to check in, but this process has been proven effective. Humans are the best threat detectors in our known universe (seconded by our trusted animal companions). But, here in the States, we are so afraid of being accused of racial profiling and discrimination that TSA is forced to ignore human solutions, and instead relies on mechanical procedures and a compulsive focus on carry-on luggage. The next layer of security is technology based - if a traveler fails to pass the human screener, he or she may be asked to submit to a full body scan, a technology that raises hackles here in the States because of 'privacy' - never mind that it will actually detect plastic explosives taped to the body. The extra cost of sending would-be passengers through a puffer machine is easily shouldered by El Al, because they know it works at detecting explosives. The defense-in-depth goes even further: El Al has special reinforcements in the aircraft fuselage to protect the weakest point against an explosive blast. This is the kind of security that would make me feel safe to fly. All of this is expensive, time consuming and necessary, because it saves lives. The cost of security can't always be measured by money. Some things are more valuable like reputation, goodwill, and peace of mind. The U.S. should take some lessons from this, and start spending smart money on a few key defense-in-depth strategies that work, not only in our airline screening process but in our networks and infrastructure as well. Risk Intelligence is a lot cheaper to implement than we think if we consider the consequences.

Not Kind, Not Gentle. The turn of the decade in security.

2009-11-22T19:56:00.000-08:00

The decade in review: The most painful thing we learned is that computer security hasn’t worked. We are, at this very moment, MORE insecure than we were in the year 2000. Billions of dollars were wasted on security technology that isn't working. In the last ten years, true cybercrime was born. Maybe we were just naïve about the coming storm. At the turn of the century, it was hard to get past the romantic idea of a university student hacker who prowled systems harmlessly for fun. Blocking ports and preventing network based buffer overflow attacks seemed so important. None of this technology prevented true criminals from pulling off the biggest heist in computer history – the massive theft of identity and subsequent banking fraud of the last few years. The traditional hacker is dead. Hackers are now called terrorists. The Russian mafia pays developers six figure salaries to write rootkits and malware. Independent researchers can and will sell a reliable working exploit of Internet Explorer for more than $50,000 USD. It began to hurt so bad that even Microsoft had to jump on the secure coding bandwagon, declaring a massive effort to make their code more secure. But this isn’t working either. You see, we are adopting technology at a rate far faster than we can secure it. By the time we have secured something, the landscape has changed and the attackers have moved on. In fact, that is why desktop exploitation has become the dominant attack vector. Over the last few years, malicious documents and media, especially “rich content” that contains embedded logic, parse-able metacode or script, and other logical constructs that can be malformed, emerged as the dominant method of exploitation. The API’s, COM objects, and other hoo-hah piled sky high on your windows workstation is a garden of carnal delights to a skilled attacker. Exploits of this nature have been mostly delivered via Internet Explorer and email. In fact, Internet Explorer is quite possibly the largest software disaster ever. As a software program, it has probably caused over a hundred billion dollars in damages since its release. This isn't about blame - if IE wasn't there, someone else's browser would have been the target. The browser is the portal into the Enterprise, so it's going to be where the bad guys focus. Finally, even before all this was going on, every nation state on the planet was standing in the shadows scared out of their britches. Smart people in high (low?) places could see the writing on the wall. It is TRULY AMAZING that a terrorist hasn’t hacked into the SCADA systems of a municipal power utility, started a cascade failure, and shut down half a state in the dead of winter. It’s because of this that I think [most of] those so-called terrorists aren’t very bright. As we close out the first decade, we must realize we have just entered one of the biggest arms races in the history of warfare. In fact, one can easily say that true cyber warfare was birthed in the last ten years.

So, now my predictions for the next ten years: Very early in the next decade, online identity theft and banking fraud will replace drug trafficking as the dominant criminal problem worldwide. Cyber cartels will make more money annually than drug cartels. Exploitation will continue to be focused on content-based delivery – that is, malicious documents & media. This will be coupled with a massive growth in online social networking. Trust, as a human concept, will be exploited as a means to spread malware throughout social networks via your online digital identity. Again, we will adopt new technology at a rate faster than we can secure it. The largest domain of attack will be software running on cellular phones. The phone will truly evolve into a network terminal – a slightly thicker thin client, loaded with more software in the palm of your hand than you could cram into a Windows 95 box in the year 2000. Yep, you guessed it, another garden of carnal delights – these new platforms will arrive unsecured – the development tools to make software will be insecure, and the people writing the code aren’t going to give a bug’s butt about secure coding practices. So, cyber crime is going to get a lot worse. Meanwhile, we are going to see at least one major SCADA based terrorist attack. We may have no idea that a terrorist did it, because the authorities will never admit it if they can plausibly lie, but it will happen. In fact, it may have already happened. Security spending will shift as well. Starting now, and reaching a heyday in about 6 years, security spending will shift towards host based security solutions. First the government, and then commercial enterprises, will realize that netflows and gateway solutions are not going to stop malware – it’s just too hard to predict what software will do without actually running it. And, online social relationships will be an extension of our professional identity - in other words, when an employee sits down at his workstation, his entire social network sits down with him. Network based security cannot hope to analyze complex documents and media, much less who to trust and when. Because everything will be hosted online, blocking content will effectively break the Internet, and looking inside the content will never happen at the network gateway (don’t invest in companies that think they can solve that problem). Concepts like malware-tolerance will become a hard reality, people will realize you can't keep the bad guys out. While the majority of online crime will continue to be in banking fraud, we are going to see industrial espionage and state-sponsored attacks in the press more than once. And, while banking fraud hurts the individual, the scope and damage of espionage is far far greater. Whether its classified state secrets or the recipe for Coke makes no difference, when the criminals out there figure out the value of information, they WILL steal it. The next ten years are not going to be kind or gentle to the security space. The hardest hit are going to be the biggest in the space – AV vendors are going to take the hardest fall. Their signature based solutions don’t work today, but not everyone knows that yet. But over time, that truth will seep farther into the IT space. So, perhaps my biggest prediction is this – AV will lose their place as the #1 security expenditure in the Enterprise. I’m not sure what will replace it exactly, but I do know that people are going to stop throwing good money after bad.

Blackhat Training is almost here!

2009-07-22T15:49:00.000-07:00

I am gearing up for the Blackhat Training session on Monday-Tuesday of next week. We have made room for 30 students. We spent almost four weeks working on materials, remastering the demo and recap videos, and collecting malware samples that illustrated each of the subjects we are presenting. The task was alot harder than I originally expected, especially the collection of malware. I discovered a great trick using our feed processor, which is the clever use of search terms against strings to locate malware that were using specific techniques, keylogging methods, hooking styles, even specific languages. We have a solid methodology we teach behind our Responder product, so I had to find malware that illustrated specific concepts, as opposed to tailoring a training around whatever malware happened to be available. I will try to keep the training as high level as I can, and stay out of disassembly code as much as possible, but as expected there are some key reversing skills that can never be avoided, such as the reconstruction of parameters passed to a call. But, as for arithmetic and hard logic reconstruction, the only exercise where we get into that level of detail will be the one on crypto and stego. We have one coding exercise using the new built-in scripting interface, so thats a short bit of hardcore fun as well. But most of the material is about getting reverse engineering done rapidly, getting what you need, and not bogging down - which is the name of the game.

Reverse engineering process-injecting malware

2009-07-13T20:06:00.000-07:00

I posted a video demonstrating some RE work with Responder:

Process Injecting Malware

The RE process starts by searching a livebin's symbols for "remote". A livebin is the in-memory version of an EXE as extracted by Responder. It’s not an executable format, but instead represents the exact layout of the PE formatted file once it loads into virtual memory. Section information does not need to be interpreted to remap the binary in this case, as the OS loader has already done that, including remapping and any other modifications that are made to the layout of code and data at runtime. Many malware programs will be packed on disk, but the livebin will contain large unpacked sections that can be analyzed without the RE having to know anything about the packing methods used, as in effect they are already unpacked for you.

The symbol "CreateRemoteThread" is of interest. For process injection malware you will find this API call almost 100% of the time. There are a few other API calls that are used in conjunction. We drag the symbol to the canvas and examine the region around it. Specifically, we see WriteProcessMemory and VirtualAllocEx - this is a dead giveaway that process injection is in use. Usually a malware will inject a thread that points to the function "LoadLibrary" with the first argument being a path to a DLL that was decompressed to disk - typically in a temporary directory. This is part of the malware's installation system.

In the example, we find that a PID is used to locate a process to inject into. We follow this PID and find the argument is used with LEA. In this case, the LEA is like using the address-of operator in 'c' code.
Imagine the following code: void *myFunctionPointer; some_function_call( &myFunctionPointer );

The "some_function_call" is getting a reference to myFunctionPointer, and this means it has the ability to initialize or assign a value into myFunctionPointer. The LEA instruction you see in the video is the assembly version of this same operation. We see this, and follow the function to find a loop where ToolHelpSnapshot32 is used. The toolhelp API set is another very suspicious behavior - if you see this in a potential malware you are very likely dealing with something that enumerates other processes on the system. This is usually a step prior to injection (or an attempt to find a virus scanner or firewall exe and kill it).

There is a string comparison in the process hunting loop - so the malware author is attempting to find a process by name. We follow the arguments back up and see that it's searching for "explorer.exe". The steps shown in the video require moderate-level RE skills, but are not daunting. With a little practice you can follow arguments in and out of function calls without losing your place. The trick is simply to remember that arguments are usually a positive base off of EBP, and local variables are a negative offset. "Parameters are Positive" - use that rule to remember.

The is one exception that is likely to drive you crazy - malware written in Delphi (and there is ALOT of that) usually passes parameters in registers. This can be harder to follow, but again if you label the arguments going into the function you can see these labels at the function boundaries so you don't lose your place.

There are no isolated networks anymore

2009-04-27T17:40:00.000-07:00

Highly specialized networks, such as those that control power grids, or esoteric equipment, such as MRI scanners, are not typically considered at risk from Internet attacks. Yet, the recent conficker worm was able to infect these things. It is important to understand that just because hardware seems specialized and distant, it can still be connected to a TCP/IP network. Even if the equipment doesn't offer a convenient web-addressable interface to hack, it can still have a protocol and perform I/O.

Almost all modern but specialized equipment has embedded TCP/IP capabilities and the associated ethernet jack. Web and TCP/IP based technology is a good choice for machine interfacing and configuration. Browsers eliminate the need for specialized client software. Non-specialized programmers can write code that works with a HTTP or HTTPS interface to provide remote configuration capability - this equals lower software development costs.

Specialized equipment often contains a remote data terminal (RDT) which is like an embedded board that contains a mini-OS, likely based on a linux variant or even something like VXWorks. Newly emerging technology, like System on a Chip (SoC) is both inexpensive, and easy to interface to. Even when an RDT type function is not available, these devices may stream large volumes of data outbound over TCP/IP, with the port intended to be used in a specialized LAN configuration for image capturing or other functions (think medical equipment like MRI scanners or X-Ray machines that are interfacing to the PACS network).

The overall point is that these machines are connected to a network that talks TCP/IP. And, following the very nature of TCP/IP, it's easy to make connections that are unintended. So, even though the MRI scanner is not supposed to be connected to the Internet, the imaging workstation will need to talk with the database in Radiology which is then connected to the Hospital Information System (HIS), which is connected to the Internet. You now have an MRI scanner that is attached to machines that can browse the Internet. This is how Conficker got into Heart Monitors running an old unpatched Win2K systems.

Even old equipment falls prey to these unintended exploit paths. Especially for older SCADA equipment, there are tons of devices that will interface good old serial ports to ethernet and TCP/IP pathways. To lower costs, SCADA networks have been refitted with remote access that is routable over ethernet and TCP/IP. The protocols are old and weird, but anyone who does their research can attack them. Even when not directly connected to the Internet (and yes, sometimes they are), devices like power relays are just a few hops away from the Internet-facing gateway. These devices really do control power for small northeastern towns in the dead of winter.

A large amount of the risk here is simply that specialized networks are connected to the Internet via unintended means. These unintended connections between the so-called “protected” networks, and the totally unpatched open equipment is something like a void. It’s not well audited. In some cases, the IT staff may even be discouraged from auditing. In one factory a few years back, the IT staff were forbidden from even running port scans to inventory the network. Apparently doing so once crashed a SCADA controlled machine on the factory floor, so management had forbidden the practice hence. To make things worse, it's incredibly easy to bridge networks without thinking about the security implications. An end user can co-fuse two networks just by plugging in a cable incorrectly. A network admin may not have an extra switch so they use the existing one out of convenience. There are countless scenarios where it's easier to think of specialized systems as non-internet devices, thus not a problem for security.

When dealing with network security, you should always think of every networked device as containing an operating system. It would not harm your security to even think of them as embedded windows operating systems that are vulnerable to conficker worms. You should never think of them as non-internet devices.

Ongoing SCADA Attacks and Network Probes

2009-04-08T05:39:00.001-07:00

Consistent and ongoing recon-probes continue to be launched into the US Infrastructure, including government and municipal systems. Boldly stated, all large Enterprises (government and corporate alike) are compromised by some form of malware that is CURRENTLY under C&C from a remote attacker. Malware infections are the tip of the spear - at the other end of an active malware C&C network is a human being or organization with intent and funding.

Recon-probes are malware implants that only scan ports and inventory the resources in the network, then phone home with the data. In many cases, probes are not targeted (not aimed only at your network, but rather like a shotgun approach) - there is an ongoing effort to simply map everything that sits behind the public gateways. In particular, probes have been launched into the US Infrastructure SCADA networks - think power grids and water plants.

Probes will be less complex than a full-blown botnet agent. One component to be on the lookout for is a TCP/IP-only capability - not something that injects into IE or Explorer, but rather a cleaner implant with a port scanning and sniffing capability. These probes will have a C&C backchannel of some kind, but are likely to store their information on disk for a while, as they don't phone home very often. These are forward probes designed to map your network. They may even query the hard-drive serial number via IOCTL's to the NTFS driver, this is for node identification decoupled from the IP address of the host. There will also be a query to see if the box has multiple interfaces.

If you find a probe operation, immediately assume that secondary attack tools have been brought into the network, perhaps in select subnets or on critical gateway machines. Be especially attentive to any sniffing capability on a collision domain near a gateway, or even on the gateway itself. In some cases, secondary capabilities have been dropped that have the ability to shutdown and destroy the computer. If you have captured a probe, immediately check all embedded registry keys and file paths for potential storage locations for secondary equipment.

Rich and Greg in Va. – Ghillie Suits, AR-15’s, Russian Ammunition and Chinese Malware

2009-04-04T20:17:00.000-07:00

The morning was spent discussing how lame Conficker.C turned out to be and how it was most likely just barrage jam… meaning a smoke screen diversion to throw off the scent for the “real” slow and low pdf attacks that were slipping into financial institutions in droves. Then on Friday morning HBGary was lucky enough to receive a nice excel spear phishing attack. Unlike most companies we love this stuff. This gives us something to do over the weekend. Greg and I also discussed our new global services offerings which will soon appear on our web site.

After breakfast Greg and I went to an undisclosed location in the Northern Virginia area, got suited up with Ghillie-Suits, AR-15’s, and a 1000 rounds of Wolf Performance Ammunition from Russia. Our mission was to get from point A to point B without getting caught by numerous “individuals”. If we weren’t caught and we made it to point B, we were then to shoot the 500 rounds each at targets from 25 yards up to 100 yards. We had up to 4 hours to cover the terrain, get to Point B, shoot our rounds and get back to point A. without getting caught; again the main point was not getting caught. It was a great! We covered a crazy amount of territory in a short period of time climbing through all kinds of thick brush and most of it was straight up hill to reach point B. At one point we we’re within 50 yards of some of the “individuals” but remained completely still and since we were in our Ghillie-Suits we remained completely still and remained undetected, just like a good rootkit. ;) We ultimately made it to Point B., where we celebrated by drinking some water and dropping our packs to load our rifles. We target practice with Russian ammunition because it’s cheap, pretty reliable and readily available.

As the sun was setting, we had already infected a VM with one of the recent boobytrapped PDF documents. Using a snapshot and Flypaper, we extracted several binaries with Responder and discovered a running botnet out of Russia. The PDF document immediately grabs a malware loader executable from a hacked chinese website, including a flash module. Once the loader executes, the main loader contacts a bot controller located in the ukraine, and the subsequent payload that is downloaded loads a kernel mode rootkit and a usermode module that communicates with a single drop point - a single commercial hacked website to store a drop point, and from this scripted location, data being emailed to a completely different and single hacked email account. The bot control software is something called "JRoger BManager v1.5" and in this case, was operated from a Russian language asset. We made heavy use of NetWitness Informer to capture C&C traffic and compressed downloads of infection modules. We are now tracking this threat to learn more.

Here are some pics:

The Bot Controller

Responder graph of the usermode portion of the malware

NetWitness really boils off the fat. You can slice and dice the data from a packet capture in so many ways. Here are shots:

Overall a good day.

The Sky is Falling, When it Rains

2009-04-03T16:12:00.000-07:00

We have come to distrust any doomsaying in the security industry. We can't identify an authoritative and impartial entity that can stand back and really make an assessment of risk. Claims about the cyber threat level resemble the Orange Threat Level at the airport - a distant flag of color, washed out behind the gate call and the long line at Starbucks. To an outsider, the latest threat reports published by security companies seem to be coat tailing on Conficker - a recycling furnace of self-fulfilling prophecy, the press thermometer following along, ticking up to the final doomsday hour when conficker went... fizzle pop. Conficker a bust. Move on, this is not the threat you're looking for.

From y2k to Al Qaeda threats on the Capital, the lack of materialization can lead us beyond healthy skepticism to a place where we conceptually disenfranchise threat intelligence as a whole. This is where we have to be careful and step softly in those dark woods beyond the campfire. Just because conficker didn't blow up the Internet does not mean it couldn't. If anything, conficker brought a lot of press attention to the problem of malware, and that is a Good Thing. When tens of millions of computers remained infected with a variant of conficker on April 1st and still today, we all need to understand that someone somewhere could have lit the flash powder. Conficker is old news. New variants of malware are released daily. In one discussion I heard upwards of fifty thousand new variants per 24 hour period (think autopacking on deployment). If conficker is truly controlled by the Russian Mafia, then blowing up the Internet serves no purpose for the their bottom line. Silent ongoing presence is what steals intellectual property and banking credentials; not DDOS, not software vulnerabilities that amount to sexed up access violations. Real attacks are about reliable access to money and information. The security industry can sometimes get caught up in stuff that really doesn't matter that much, while ignoring the threat that is right there, in front of your face, in your computer right now.

Malware commonly hunts down and kills anti-virus programs

2009-03-30T17:22:00.000-07:00

Much of the malware we are processing has the ability to locate and kill anti-virus programs and desktop firewalls. The following malware example illustrates the behavior clearly. There are long strands of code that query through a list of known security software process-names and subsequently sabotage them.

Click for larger image

The means by which the malware detects the security software is by process name. There are long lists of process names that appear in sequence, these nodes are shown on the graphic as label 'A'. Almost all variants of this behavior are similar in structure, even though they are employed across many different and unrelated malware strains.

update:
I took the time to zoom in on one single operation (marked as B. in the following image) from the strand of control flow shown above. The strand shown above contains hundreds of these.

And, here is the disassembly for one operation:

10001A98 BB 5C 65 00 10 mov ebx,0x1000655C // webtrap.exe
10001A9D 53 push ebx
10001A9E E8 C3 25 00 00 call 0x10004066▼ // __imp_MSVCRT.dll!strlen
10001AA0 ASCII: %
10001AA0 : 25 00 %.
10001AA0 : 25 00 00 %..
10001AA3 loc_10001AA3:
10001AA3 59 pop ecx
10001AA4 50 push eax
10001AA5 53 push ebx
10001AA6 8D 4D F0 lea ecx,[ebp-0x10]
//__imp_MSVCP60.dll!?assign@?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@QAEAAV12
10001AA9 FF 15 8C 50 00 10 call dword ptr [0x1000508C]
10001AAF loc_10001AAF:
10001AAF 8D 45 F0 lea eax,[ebp-0x10]
10001AB2 8D 4D DC lea ecx,[ebp-0x24]
10001AB5 50 push eax
10001AB6 FF 75 E4 push dword ptr [ebp-0x1C]
10001AB9 E8 BB 22 00 00 call 0x10003D79▼ // sub_10003D79
10001ABE loc_10001ABE:
10001ABE 57 push edi
10001ABF 8D 4D F0 lea ecx,[ebp-0x10]
// __imp_MSVCP60.dll!?_Tidy@?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AAEX_N@Z
10001AC2 FF 15 90 50 00 10 call dword ptr [0x10005090]
10001AC8 loc_10001AC8:
10001AC8 8A 45 0B mov al,byte ptr [ebp+0xB]
10001ACB 56 push esi
10001ACC 8D 4D F0 lea ecx,[ebp-0x10]
10001ACF 88 45 F0 mov byte ptr [ebp-0x10],al
// __imp_MSVCP60.dll!?_Tidy@?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AAEX_N@Z
10001AD2 FF 15 90 50 00 10 call dword ptr [0x10005090]

What is interesting is that the list of malware to kill is commonly cut and paste around, but the methods used to parse it can vary based on which malware author is developing it. In this instance, the malware author uses the standard template library. This is a good fingerprint that can be used to find other variants of this particular author's work. Data like this can be used as an attribution factor.

Nabbing Conficker with Digital DNA

2009-03-28T15:50:00.000-07:00

What follows is a step by step analysis of Conficker using HBGary Responder. The conficker worm represents a significant and current threat. The following analysis was performed against a sample at HBGary's lab. The first step was to obtain a dropper for one of the conficker variants and subsequently infect a 'sacrificial lamb' machine. The sacrifice machines have no network card and the USB ports are blocked as a precaution. There is a secure one-way method to take a physical memory snapshot and pull it from the machine. We resort to such measures when the malware won't execute in a VM. Once the infection was deployed and the snapshot obtained, we simply import the memory snapshot into Responder. The Digital DNA (available in Pro edition and also for the Enterprise via the McAfee ePO integration) detects and weights digital objects based upon a numerical string that is generated for every identified object. In this case, the Digital DNA calculated for one of the VAD tree memory ranges indicates high suspicion.

Click for a larger image

This memory range is subsequently extracted and disassembled / decompiled. Code, data, symbols, and strings are all recovered from the dynamic snapshot. This is an interesting fusion between static and dynamic analysis, given that its a snapshot in time of an actual running instance of the worm. Buffers contain fixed up address data, decoded data, full call stacks, etc. Many arguments can be reconstructed that would not be available in a traditional static file-based analysis. By using memory, for example, we didn't even have to worry about the packer. In this case, the packer has already ran and the malware is sitting in memory fully unpacked. We start our analysis by dragging all the strings to the graphing canvas and sorting them into colored layers.

Click for a larger image

Further analysis is largely graph-driven. Each layer represents a different property or 'factor' of the malware. For example, all of the installation capabilities are put on their own set of layers, while the communications are isolated to a different layer. The sorting to layers takes about 10-15 minutes by hand.

Click for a larger image

Once sorted, I pick off an individual layer - in this case the 'installation and deployment' layer. I toggle off the visibility of all the other layers and just focus on this single layer.

Click for a larger image

I pick through the layer sorting each small island of nodes into a more refined set of layers - moving some to a layer regarding the DLL injection capability, another describing the patch conficker makes to the tcpip.sys driver, etc. This process continues for about an hour as I drill down on connect many nodes on the graph. As I go, I bring up the code view and label functions with bookmarks that will end up in my report. The bookmarking lets me make annotations to the disassembly and the graph that are preserved for reporting purposes.

Click for a larger image

Whenever I dive into a function, I use the built-in dataflow analysis and graph-based decompilation. You can see in the screenshot how all the graph nodes are annotated with the logical conditions required to follow the branch. For example, in the screenshot a loop is detected and the loop control conditions are shown. This is a low level feature.

Click for a larger image

Finally, after about an hour or so, I have built graphs into over a dozen layers describing portions of the conficker worm's code and capability set.

Click for a larger image

When I am finished I auto-generate a report in word format, clean up the edges a bit, and print it to a PDF file. The following link shows a partial report detailing some select areas of the conficker worm.

Responder Report for Conficker (PDF)

Responder is better than IDA Pro for analyzing malware.

2009-03-27T19:24:00.000-07:00

Greg's Note: This blog post was made by Martin Pillion, the most senior reverse engineer at HBGary and one of the most skilled RE's I've met. I have crossposted it to Fast Horizon so it would be picked up on the RSS feeds. Martin's assessment of IDA vs Responder is timely and highlights the amount of experience required to look at assembly vs. graphs. Features like proximity browsing lower the bar significantly so that more practioners can help solve hard RE problems.

Responder is better than IDA Pro for analyzing malware. I do not make this statement lightly. I have been reverse engineering code (on and off) since the late 1980s, I am a long time user of IDA Pro (since ~2000), and I have written a fair number of IDA Pro scripts. IDA Pro has been the definitive disassembly tool for nearly 10 years.

Let me also point out that I am an HBGary employee and I certainly have a bias toward the Responder product. I have helped design, architect, and develop the Responder product for many years. During that same time, I often worked with the HBGary Services division to assist customers with reverse engineering malware. I used Responder as much as possible, but often found myself loading up IDA Pro and running both products at the same time. However, in the past several months, Responder has improved to the point that I no longer use IDA Pro at all.

Responder utilizes a different approach to reverse engineering than IDA Pro. Where IDA Pro relies on inspecting large amounts of assembly, Responder relies on a more visual, graph based approach.

Responder displays information visually and is navigated based on relationships. Responder has a feature called 'Proximity Browsing' that allows you to expand a graph based on the cross references to or from the currently selected node. This makes it easy to quickly locate related code and visually examine those relationships. IDA has a popup dialog to list xrefs, but you must click each xref individually to examine it (time consuming).

Responder uses the color of xref lines to indicate what kind of xref it is. Grey for data xrefs, black for block xrefs, and red for call xrefs. Node shapes can indicate function starts, ends, regular blocks, data, comments, or bookmarks. Node color is used to organize graphs and is determined by layer. Layers are similar to the layer concept in Adobe Photoshop.

IDA Pro relies on WinGraph32 to perform graphing and it is clearly a secondary feature in the tool. There is support for a limited set of graph based analysis features, mainly built around flow chart and call flow graphing. The UI also leaves a lot to be desired... it is not possible to modify an existing graph, instead you must go back the text based UI and regenerate an entirely new graph.

Responder lets you "Collapse" a graph node. A Collapsed node is an entire function, instead of a single block of disassembly. You can Proximity Browse from a collapsed node and you will expand only call xrefs, with additional nodes also being collapsed functions. Using this feature you can quickly identify the purpose of main functions and label them appropriately.

Responder can also perform "Data Flow" tracing. Data Flow tracing allows Responder to follow the movement of data through a function, even if it is moved to a memory address (like the stack) and later moved into a register. This means that Responder can follow stack variables on functions with frame pointer omission.

Data Flow tracing is powerful and Responder utilizes it every time you rename an instruction operand. This means that your custom labels may show up later in the function and be used in a way that you did not realize. Data Flow tracing will track multiple levels of dereferences and indirections, memory addresses, registers, and even logical manipulations.

These are just a few of the features make Responder better than IDA Pro for malware analysis. I discuss others and also expand on the technologies behind each feature in future blogs. Ultimately, the primary work flow of a reverse engineering is one of organizing and understanding data. Responder enables me to do this with malware faster than I would be able to using IDA Pro.

- Martin

Server-class Analysis Now Possible with Responder

2009-03-24T12:05:00.000-07:00

HBGary has been lifting some heavy iron, testing a variety of large memory configurations over the last few weeks. The latest version of HBGary Responder now sets the milestone: 64 gigabytes physical memory analysis - a sizeable snapshot indeed. This makes Responder a server-class product. This is an important step forward for HBGary, as the Digital DNA and malware analysis capabilities can now be applied against critical servers in the Enterprise. Large memory footprints can be found on server class machines running Windows Vista, 2003, and 2008. Ensuring servers remain free of rootkits and malware is crucial for regulatory compliance. A case in point, Visa recently announced that PCI compliance was being revoked for both RBS WorldPay and Heartland, due to malware intrusions and subsequent breach of security. Early detection of an intrusion can prevent data theft, as malware typically infects a system and remains there for quite some time. A recent data-breach study by Verizon (spanning over 4 years and 500 intrusions) reports that over 70% of victim companies had been compromised for over a year before the intrusion was detected. FISMA, PCI-DSS, and HIPPA all mandate various forms of intrusion detection to help limit the scope of damage caused by an intrusion. Sound defense in depth strategy advocates that Enterprises monitor server memory for zero-day malware and rootkits.

SMM Rootkit: Old, Obscure, and Unnecessary

2009-03-20T11:25:00.000-07:00

Be mindful that you don't overreact to the 'new' SMM exploit (properly, reported by Loic Duflot, a very accomplished low level hardware researcher, at the recent cansecwest conference). The exploit itself is really a documented 'feature' of the Intel 5100 Memory Controller chipset, and has been a known issue with SMM for quite some time. See the 5100 data sheet:

In order to make cacheable SMM possible, the chipset must accept EWB’s and must absorb IWB data regardless of the condition of the SMMEM# pin. The Intel® 5100 MCH Chipset will not set the error bit EXSMRAMC.E_SMERR in this case. Because of this, care must be used when attempting to cache SMM space. The chipset/platform cannot protect against processors who attempt to illegally access SMM space that is modified in another processor’s cache. Any software that creates such a condition (for example, by corrupting the page table) will jeopardize the protective properties of SMM.

You might ask why it hasn't received more attention until now? Stated simply, such low level tactics are simply unnecessary for a real rootkit to be effective. Remember that you need to be in ring-0 (kernel) before you can even attempt installing into 'ring -1', and being at ring 0 is plenty of privilege for even the most stealthy of rootkits.

There are other reasons that an SMM rootkit is best left to the science fair: to make one that is effective across more than a select hardware platform, you would need to invest boat loads of development dollars in testing. At worst, someone might build an SMM rootkit that works on a well-known and distributed model of laptop and post that for publicity - but the real criminals don't build stuff like this, nor do they have to.

The majority of malware threats today are, in fact, usermode. The bad guys simply don't need to go any lower to get their work done. Remember, the lower you go, the less re-usable code you can leverage. That is, lower means no libraries, no API's. Lower means you write all the device, memory, and hardware logic yourself. It approaches the complexity of device driver development and operating system design. This all translates into expensive and non-ubiquitous. Malware avoids this development cost by simply installing itself like any other program, re-using the existing API's and libraries under windows that already provide network access, memory management, file access, and the like.

The modus operandi of real malware authors is: Write once, use many times. An SMM rootkit is a really neat science experiment and excites technical curiosity, but such an approach is not useful in practice. Let's stay focused on our Day Job, tackling real threats.

Digital DNA - Numerical Expressions to Describe Malware Behaviors

2009-03-09T19:58:00.000-07:00

HBGary unveiled Digital DNA today at the Infosec Conference in Orlando. (I wasn't able to make it down to the show, although I had planned to be there. Last minute stuff and I had to jet back to the West Coast.) The engineering team has been working on Digital DNA for months. In a nutshell, we have automated the reverse engineering of loaded modules in the physical memory snapshot and generate Digital DNA (DDNA) based on the collected data (millions of data points). All of these data points are codified in way that allows them to be matched against rules. The Digital DNA system will "sequence" a software program or document and generate trait-codes based on the behaviors and schematic artifacts found in the software or document. Each trait has a complex rule (think regular expression with boolean logic) associated with it, and if the rule matches the trait is considered "expressed". Expressed traits are concatenated together to make a "sequence". We chose to do it this way because the final DDNA sequence looks and smells like a hash, even though it's not actually a hash at all. But, customers are used to managing hashes, thinking about hashes, and cut-n-pasting hashes - so a hash it would be.

Digital DNA is based on the reverse engineered behaviors, not the specific compilation or packer used with the malware. You can pack the same malware with three different packers and it will still produce the same Digital DNA. Two similar programs will produce similar DDNA. Here is an example of two versions of Rustock.B.

Interestingly, the technology can identify digital objects. Here is an example of tracking Intellectual Property with it.

Digital DNA is a Big Idea. For now, HBGary is going to focus it on detection of zero-day malware threats. We have over 2,000 traits in the DDNA genome currently, and will probably have many more soon. We sort all the traits into Factors, Groups, and Subgroups, defining a "genome" of behaviors that are common to malware. This part plays into a weighting system. I will blog more about this over the coming weeks - dinner is calling.

Your online payments are being sniffed; accept it, live with it

2009-02-24T09:53:00.000-08:00

PCI compliance is clearly not enough to protect credit card numbers or account information. It’s about time everyone who uses an account for online payment simply accept the facts: your credit card numbers have been stolen. Check your statements monthly. Why? This isn’t about Heartland or the breach-of-the-week; this is about a constant effort well funded by a criminal underground. The primary tool in the cyber criminal hand, the malware program, keeps getting better. Malware authors are intelligent and focused developers who are well paid for their work. They have developed toolkits so they can generate new malware with little development overhead. They can generate new attack bits in a matter of hours that, to a virus scanner, may as well be a zero day – no signature means no detection, and no protection. Most of this malware decrypts live to memory and never touches the disk. The computing infrastructure is easy prey. It has never been secure, and won’t be secure anytime in the next ten years. Computer security is a constant effort that will never fully work. It’s partial risk reduction, not resolution. The billions of dollars spent since the turn of this century on IDS, firewalls, and virus scanning hasn’t made a more secure Internet. The growth of online technology has far outpaced our ability to secure it. Millions of credit card numbers are being stolen THIS MORNING. They were being stolen yesterday. They are going to continue to be stolen tomorrow.

Melissa Hathaway, on track to make a difference?

2009-02-11T11:00:00.000-08:00

Unlike previous cybersecurity czars, Ms. Hathaway has experience. She understands how hard national security can be. Notably, Ms. Hathaway has been working on the Dark Side (think classified) of the government, which means she knows the reality of cyber threats - how effective cyber espionage really is, what is being stolen, and who is stealing it. It also means she knows the definition of a "Funded Threat." And, to combat these funded threats, she understands that it's not just defense, but also offense (think geolocation, trace back to the human, and the money). During his campaign, President Obama stated that he would take cyber attacks as seriously as nuclear or biological. A strong statement like this ultimately translates to budget.

Obama seems to want to dip his toe in the water first. Ms. Hathaway will not have the White House power position, at least not yet - there will be some bureaucracy between her and the president. We will have to see what happens in the next 60 days. But, bureaucracy will be one of Ms. Hathaway's greatest challenges. To her credit, she comes from the right community. She has the relationships in place that can help her succeed.

One of the things I like about Ms. Hathaway is her understanding that cooperation between agencies is required for success. The government is a big place, and the computer networks within it are like little fiefdoms. Coordination is difficult -- not because people lack the will to work together (although that adds difficulty), but because searching through ALL the information is required to find out what's important or critical. Most people want security to be someone else's problem. Those responsible for security want it to be easy. But that is core of the problem. Security is NOT easy. There is no shiny button.

Real security takes work. Ms. Hathaway supports building new technology to address new types of threats that go beyond what yesteryear had. We need to realize that people are out to get us, we are being attacked, and if smart people in the Enterprise say it's an "arms race" you better believe the government knows it is. She needs to be frank with everyone that there is no magic pill. She must require people to step up and do more and not rely on outdated security technology but to supplement with newer technologies.

The 60-day security review may bring back bad news - that things are terrible out there and the Nation's security is worse than it has ever been. We are in tough times, and some tough decisions will likely be made. Ms. Hathaway appears to have the big picture -- finally someone who might actually be able to change security for the better. Hopefully Obama will give her the authority to do so.

Its going to get a lot worse before it gets better

2008-12-17T10:38:00.001-08:00

There is an interesting mix of problems going on right now that, when combined, create a sort of "cybercrime" perfect storm. Historically, there is an obvious correlation between economic downturns and the rise in crime. What makes the modern downturn interesting is the ease with which cybercrime can be perpetrated. First, there is the growing and fluid blackmarket for financial data. One doesn't have to browse far to find reports of a rise in phising, drive-by web infections, and advances in bot-net technology. Insiders with access to financial information will find easy money. Large financial institutions are already experiencing a rise of internal investigations. Layoffs in the high technology sector are closely related to intellectual property theft - employees are very likely to download intellectual property that may help them secure a new job - its a simple backup plan that is easy for the human mind to justify. This isn't even that high-tech - it's as simple as USB thumbdrive and an unprotected port. Internationally, high tech workers are losing their jobs, and programmers out of work are willing to take malware development jobs for low pay. IT professionals out of work in Eastern Europe and Asia are already getting roped into the identity theft blackmarket, using toolkits to develop and deploy phising attacks. The endpoint systems within enterprises are frail and easy to attack with malware, they are already infected to a large degree. The virus scanning technology that is the leaning post of enterprise security just doesn't work. The massive investment in security solutions over the last decade hasn't helped at all - enterprises are just as vulnerable and exploited today as they were in the late 90s. I think its an obvious conclusion to be drawn, the malware problem is going to experience a surge over the next 24 to 36 months. Investigators are just now starting to understand that there IS a problem, much less combat it.

64 Bit Analysis - the future is here

2008-11-24T12:10:00.000-08:00

64 bit systems are more than 10 years old, but only until recently did they emerge in the market. In the past, 64 bit was found doing the “hard stuff” - complex fluid mechanics, computational modeling, and ultra large databases. Today they are mass market - it’s hard to find a 32 bit machine, and you pretty much have to custom order to get XP. More complex applications are going to demand upgrade - virtual environment collaboration for example - but even more important is just the fact we are dealing with large data - think > 4GB files. If you need an example, think of a DVD movie. Multimedia alone is driving upgrades in the consumer market.

64 bit architecture is just plain faster - it's not so much about the CPU speed - it's more about how much data can be moved. Data movement is more important, IMHO, than Data calculation for most users. With the possible exception of codec's and compression, people just don't USE their CPU's (Video cards with cooling manifolds, GPU's, and "Left 4 Dead" addicts aside :-).

This means “big” things for memory forensics – and also malware analysis in terms of the 64 bit operating system. While I think malware will continue to be coded in 32 bit (for maximum compatibility if nothing else) the stations they infect are migrating to 64 bit windows.

Simply stated, workstations are the point of penetration into the Enterprise. It’s the place we care most about analyzing. To understand malware from our (that is, HBGary’s) point of view, we need to understand the operating system – and by this I mean Windows Vista 64bit, Windows XP 64bit, Windows 2003 64bit, and Windows 2008 64bit. We follow the OS, we find the malware. Pretty much any pre-2008 rootkit technology just raises its hand “here I AM” in an offline memory analysis. From a rootkit developer’s point of view, things are happening live-action, hooking a function pointer makes sense because the HIDS is going to be totally subverted. Offline, everything changes. Said hook is a big red flag waving “over here – find me over here!” But, of course, this means being able to analyze the OS data structures in the first place. Hence, the importance of 64 bit.

HBGary released the 1.3 version of Responder a few days ago. This is the 64 bit platform upgrade. It includes analysis of 64 bit Windows platforms, and the FDPro dumping utility that can dump physical memory images from 64 bit systems (including those that require signed drivers). This was the longest development iteration for our team so far this year. The 64 bit upgrade was a lot harder and more work that I originally expected – there were upgrades and point-fixes in every part of the product from the GUI controls down to the memory-acquisition routines. Over 1,000 points of code had to be fixed just for 64 bit address and offset support. Yeah, big job. And the testing, wow… So here it is. I think HBGary might be the first to market with 64 bit support (that means analysis AND acquisition, and full shipping non-alpha). The new version of FDPro is pretty nice too, supporting compression and probing, speed upgrades, and nearly 100% reliable memory-page queries even for systems with more than 4GB of RAM.

I am glad this release went out the door and hopefully I won’t spend the rest of the year in a troll-cave. I’m going to be in New York City next week and also a few days in D.C. showing off the new bits. If you’re a current customer you should click the upgrade button in the about box :-) Contact sales@hbgary.com for an eval and the website is www.hbgary.com.

p.s. Thanks to everyone who uploaded me 64 bit memory dumps for QA – you’re the best!

Been working on 64-bit

2008-10-16T19:18:00.000-07:00

I'm sorry it has been so long since I have posted. Stated bluntly, I've been scrambling with product releases. Although I hope to rise into a management position deserving of my title at HBGary, -- I am in fact a developer. I code every day, new feature or bug fixes. We made the decision a while back to adopt agile development - a buzz word I know, but one actual hard side effect is the short development iteration. We try to patch our Responder product about once every two weeks. We develop on very short cycles. Even a few bug fixes are good enough to warrant a patch. Even more interesting, we adopted a commerical patching solution used by MMO's (massive multiplayer online games) - we figured the patch-every-2-weeks system was very close to the game industry requirement. I think that model is pretty advanced for a software company. Our upcoming releases are very exciting, least of which is the 64-bit upgrade to our platform. That means analysis of 64 bit windows, running on 64 bit windows, and also imaging 64 bit systems. The acquisition of 64-bit data is much harder than it sounds, it turns out. Not only do we have to analyze physical memory snapshots from vista and 2003 64 bit images, we also have to be able to acquire them. To do this requires a device driver. And, since we are adding pagefile support, we also have to build a parser for raw NTFS filesystems - THAT is a major effort and is non-trivial. Our next patch will add pagefile support for 32 bit systems, and the follow on will add 64 bit. There is a huge amount of engineering going on internally right now. I haven't had time to write about world events.

Crossing the Streams – Blizzard vs MDY

2008-07-16T23:25:00.000-07:00

In the case against Michael Donnelly, Blizzard has once again conjured interpretations of the DMCA that border on magical. Without dissecting the whole thing, the claim is basically this:

1) If you copy an executable (.exe) from disk into RAM, this is copyright infringement

If the argument stopped there, this means that anyone that executes the software is, by claim, infringing upon copyright and violating the DMCA. Obviously there has to be more. So, it goes like this:

2) If you agree to and abide by the EULA, the EULA grants you a limited license to copy the program into RAM

This means that legitimate users who agree to ALL the terms of the EULA have a license to copy the software into RAM (aka, execute it). For most regular users, this is where it stops. For anyone else who may have violated the EULA, this doesn’t stop. Here it what it means:

3) If you violate the EULA, you have violated your license to copy the program into RAM

4) Since the copy of the program in RAM is no longer licensed, this is copyright infringement

Stop there. If you connect the dots to this point, we have just drawn the following:

5) If you violate the EULA, you violate copyright

It is exactly here where the streams are crossed and the dark gate opens.

What started as a breach of contract has been polymorphed into a copyright violation. In the real world, copyright law and contract law are two entirely different sets of laws. Damages are even paid in different ways. They don’t belong in the same lawsuit.

In this case, there is a well established body of contract law that protects Blizzard against what Donnelley is doing that has nothing to do with copyright law. Yes, wowglider breaks the EULA, but NO Wowglider does not make any illegal copies of the video game, never attempted to in the first place, and has no intention of doing so. It’s a program for botting, not making copies of the game. The DMCA has no place in this case.

So, what I am trying to understand is - why Blizzard is bothering to use copyright claims at all? By selling a bot that knowingly causes the user to break the EULA, Donnelley may held accountable for damages that result from this widespread breach of contract under something known as “tortious interference with contracts”. Why are they trying to grossly over-extend the DMCA to make it apply to EULA breaches via the “RAM connection”?

I am left with the feeling that something sneaky is at play. This isn’t just because of bnetd case history or even that the attorneys at Sonnenschein have had prior success with DMCA arguments. It may only be that DMCA is new, so it’s easier to “get away with things”. It may simply be that Hollywood-types like to make everything about copyrights. But, even these straightforward answers don’t add up for me. I think there is something much greater at stake here. The future of persistent virtual worlds is huge – billions of dollars. I think that by polymorphing EULA’s into copyright infringement, Blizzard is setting the stage not for WoW, but for their next big game. This case law will be used to protect their future games from competition.

As we all know, EULA’s typically contain clauses about the use of 3rd party programs, publication of performance statistics, and forbid reverse engineering. Taken to the extreme, I could become a copyright infringer because I publish in my blog the percentage of CPU usage used by WoW as reported by Microsoft’s taskman utility. BTW, the usage averages about 49-51% on my Vista machine. So, I have violated the EULA, therefore my copy of WoW in RAM is no longer valid, and I am now a copyright infringer that may have to pay six figures in willful damages.

Blizzard has pulled out the DMCA billy-club again and again - and they may have reached too far this time. The argument that RAM is the magic connection between a EULA agreement and a copyright infringement is creative – yes they get lawyer credits for coming up with that. However, the claim that making a copy of a copyrighted executable into RAM is copyright infringement is simply wrong. It’s wrong because it’s against the law. Unfortunately, in this ruling, the Judge didn’t see it that way.

Copyright law specifically addresses the RAM issue clearly in 17 U.S.C. §117. In order to execute a copyrighted software program, a copy must be made into RAM. That a copy has to be made into RAM is a simple requirement to get the software to execute. This section of copyright law makes it clear:

(..) it is not an infringement for the owner of a copy of a computer program to make or authorize the making of another copy or adaptation of that computer program provided:

(1) that such a new copy or adaptation is created as an essential step in the utilization of the computer program (...)

Could it be more plainly stated? The copy into RAM is not something that infringes copyright, per the law. However, the court’s opinion was that “users of WoW and Glider are not entitled to a section 117 defense, and Glider users therefore infringe Blizzard's copyright”. This is a travesty.

Nobody here is saying that cheating should go “unpunished” – it’s clearly a violation of the EULA. It may be a stretch to prove damages that result from wowgliding, but Blizzard is welcome to try. It is reasonable for Blizzard to want to stop botters. There are established laws and methods by which Blizzard can do this (again, that have nothing to do with the DMCA). But stop there. Bringing the DMCA into play is reckless and short-sighted. If handled improperly, it stands to set legal precedents that take away fundamental rights granted to software users. EULA’s can contain any restriction imaginable - they could turn the most absurd actions into DMCA violations. Let me just ask you this - does a company that pulls in $1.5 Billion in annual revenue need to employ such brazen tactics to be and stay successful? One has to ask, how much is enough? Frankly, I think it borders on rude.

Whitelisting is the next snake oil

2008-06-30T10:07:00.000-07:00

Most people experienced in computer security know that ‘signatures’ are the dominant technology used to combat malware. Signatures – short descriptions of otherwise large binaries, are extremely effective at detecting specific, known programs and documents. They are perfect for scanning the enterprise for known malware, known insecure software, known intellectual property. They are the cash cow of the anti-virus companies.

There are two approaches to signatures – blacklisting and whitelisting. The idea is simple – signatures of known bad stuff is a blacklist, signatures of known good stuff is a whitelist. Blacklisting has been the preferred method for AV over the last decade. Blacklisting has the benefit of near-zero false positives – something customers expect. Blacklisting also keeps the customers coming back – new malware means new signatures – perfect for recurring revenue models for vendor’s balance sheet.

Blacklisting sounds ideal, but it doesn’t work. New malware emerges daily that has no corresponding blacklist signature. The malware must first be detected, and then processed. There is always a time window where Enterprises have no defense. Recent figures suggest that the AV vendors are falling so far behind this curve that they will never catch up with the deluge of new malware arriving daily. It can take weeks for a signature to become available.
This deluge of new malware is due to several factors. First, there is more money behind malware development than ever before. Second, we weren’t really that good at capturing malware in the past. Today, new malware can be automatically collected, without human intervention. The slow trickle of malware turned into a flood as honeypot technology emerged. Sensor grids can obtain new malware samples with efficiency - they automatically ‘drive by’ (aka spidering) malicious websites to get infections and leave open ports on the ‘Net so automated scanners will exploit them. In parallel to the automated collection efforts, cybercrime has risen to epic levels. Finally, the barrier to entry has dropped for the cyber criminal. Cyber weapon toolkits have become commonly available. Anti-detection technology is standard fare. New variants of a malware program can be auto-generated. A safe bet is to expect thousands of new malware to hit the Internet per day.

The flaw in blacklisting has been exposed – it cannot address new and unknown malware threats. Figures range, but a safe claim is that 80% of all new malware goes undetected. This isn’t just a minor flaw; it’s a gross misstep in technology. Blacklisting is, and always has been, snake oil.

Enter the whitelist. The whitelist seems like a natural response to the “new and unknown malware” problem. Anything that is not known to be good will be considered suspicious, or possibly bad. Sound familiar? Whitelisting is not new, of course. Programs like “Tripwire” were in the market in the 90’s – and proven not to work. I founded rootkit.com originally to disprove the entire concept of OS-based whitelisting.

I agree with the idea that “suspicious” is good enough to warrant a look. This is smart thinking. Whitelisting is the solution.

There is a lot more “not-known-good” in the Enterprise than actual malware. Obviously the Enterprise cannot afford the additional workload caused by “false positives”. So, racing to catch up are the whitelist vendors – to remove all the “noise” so the staff can focus on the signal. Millions of dollars are already being invested into whitelisting files – and there are solid technical reasons this doesn’t work.

Whitelists are based upon files on disk. A whitelist, in current industry terms, means a list of the MD5 sums for files ON DISK. Please understand that files on disk are not the same as files in memory. And all that matters is memory. When a file is LOADED into memory, it CHANGES. This means on-disk MD5 sums do not map to memory. There are several reasons memory is different:

1) Memory contains much more data than the on disk file
2) Memory contains thread stacks
3) Memory contains allocated heaps
4) Memory contains data downloaded from the Internet
5) Memory contains secondary or tertiary files that were opened and read
6) Memory contains data that is calculated at runtime
7) Memory contains data that is entered by a user

All of the above are not represented by the file on disk. So, none of the above are represented by the whitelist MD5 sum. Yet, when the file hash on disk passes for white-listed, the running in-memory file is considered whitelisted by proxy. This is where the whole model breaks down. In memory, there are millions of bytes of information that are calculated at runtime – they are different every time the program is executed, the DLL is loaded, or the EXE is launched. These bytes are part of the process, but unlike the file on disk they change every time the program is executed. Therefore, they cannot be whitelisted or checksummed. This data can change every minute, every second of the program’s lifetime. None of this dynamic data can be hashed with MD5. None of this dynamic data is represented by the bytes on disk. So, none of it can be whitelisted.

While an executable file on disk can be whitelisted, well over 75% of that program cannot be whitelisted once it’s actually running in memory. This missing 75% can easily contain malicious code or data. It can contain injected code. It can contain booby-traps in the form of malicious data. It can represent an injected thread. The assumption that an on-disk whitelist match means that this dynamic data is ‘trusted by proxy’ is absurd. Yet, this is what the whitelisters want us to believe.

For malware authors, the whitelist is a boon. It means that a malware author only needs to inject subversive code into another process that is whitelisted. Since the whitelist doesn’t and cannot account for dynamic runtime data, the malware author knows his injected code is invisible to the whitelist. And, since the process is whitelisted on disk, he can be assured his malware code will also be whitelisted by proxy. So, in effect, whitelisting is actually WORSE than blacklisting. In the extreme, the malware may actually inject into the desktop firewall or resident virus scanner directly as a means of obtaining this blanket of trust.

The mindset that “suspicious is good enough to warrant a look” is a step in the right direction. But, whitelisting is not the correct approach. The only way to combat modern malware is to analyze the physical running memory of a system. In memory will be found the indicators of suspicion, and this is much more like a blacklist than a whitelist – except that it’s generic and based on the traits and behaviors of software, not hard signatures. For example, there are only so many ways you can build a keylogger. Once you can detect these traits in runtime memory, you are going to detect the keylogger regardless of who wrote it, what it was compiled with, what attack toolkit was used, or what it was packed with. As a security industry we need to stop climbing uphill with traditional approaches proven not to work. We need to change the fundamental way we do things if we are going to win.

Flypaper 1.0 Released

2008-06-24T13:10:00.000-07:00

I'm happy to announce the release of a free tool from HBGary. It's something I put together to save me time when doing malware analysis for customers.

Most malware is designed into two or three stage deployment. First, a dropper program will launch a second program, and then delete itself. The second program may take additional steps, such as injecting DLL's into other processes, loading a rootkit, etc. These steps are taken quickly, and it can be difficult for an analyst to capture all of the binaries used in the deployment. HBGary Flypaper solves this problem for the analyst.

HBGary Flypaper loads as a device driver and blocks all attempts to exit a process, end a thread, or delete memory. All components used by the malware will remain resident in the process list, and will remain present in physical memory. The entire execution chain is reported so you can follow each step. Then, once you dump physical memory for analysis, you have all the components 'frozen' in memory - nothing gets unloaded. All of the evidence is there for you.

HBGary Flypaper is designed to be used with a virtual machine. Once activated, Flypaper will also block network traffic to and from the machine. If you are using HBGary Responder with the virtual machine, only the traffic to and from Responder is allowed, effectively quarantining the malware for analysis. (Note, this blocking operation would not block NDIS level rootkit material, only malware that uses the existing TCP/IP stack.)

You can get it from the HBGary website. (www.hbgary.com)

Microsoft wipes out 700,000 - too late to the game

2008-06-20T16:46:00.000-07:00

A very interesting post came out on the MMPC blog today – Microsoft added some sigs to capture Taterf and Frethog malware variants and captured waaaay more than they expected (http://blogs.technet.com/mmpc/). On the first day alone they detected 700,000 Taterf variants, millions in the first week. What is interesting is the sheer volume of malware designed to steal online gaming credentials. This is equivalent to the threat faced by financial institutions every day in the form of keyloggers that steal financial credentials. Except, in this case, the money is stored in game servers. But, like all money – money is just a digit in a computer somewhere. This is not different. The target smells the same if you step back. Just like stolen banking accounts, these accounts are stored in a bad-guy SQL server somewhere and sold for cash based on whatever inventory the character happens to have. The Asia-Pac region is already full of companies that farm gold (aka ‘real cash economy’) – they already have existing relationships with real purchasers in the real-cash economy with set quotas. So, it’s not a stretch to imagine they can clear out and launder 50 million wow gold in 90 days. At the scale of the malware infection described in Microsoft’s blog, this was a huge operation (with the sheer volume of flash and quicktime exploits over Q1 this doesn’t surprise me either). And, by the time these infections were cleaned by Microsoft, it was too late. The game was already over.

Welcome to Greg Hoglund's new Blog

2008-06-16T14:31:00.000-07:00

Welcome to my new blog, Fast Horizon. I have retired my old blog on rootkit.com and opened up shop here at blogger. I am the CEO of HBGary, Inc. (http://www.hbgary.com/) – a new company in the computer security industry. We released our first product this year (Responder, www.hbgary.com/responder_pro.html). HBGary is actually about five years old, but until now we have been a services company working primarily for the U.S. Dept. of Defense and Intelligence Community. I am excited to be part of the shift toward product development. This is my third startup. I am the author of three books and have been educating people about security threats – especially rootkits – for almost 10 years. I have a great foresight for trends – thinking of ideas about 5 years too soon for the market - and an almost cynical edge to my observations. Most people know me as a hacker, but in truth I probably know more about business and product development than hacking at this point. All of my startups have been in software development. I have probably experienced every management nightmare that can be listed, and dealt with it. I like to take big bites - so HBGary is tackling the biggest threat in computer security today – malware. Unlike most companies however, we aren’t selling snake-oil. Instead, our philosophy is that it’s IMPOSSIBLE to keep the bad guys out. The billions of dollars spent on security since the millennium has been a complete waste. Instead, we assume the bad guys will succeed – and it’s our job to catch them once they get in.

I could describe our solution as a platform for analyzing physical memory. You see, if there truly is a cyberspace in the Enterprise, it’s represented by the ones and zeroes in physical RAM.

There are only three kinds of data in the enterprise:
- Data at rest, on hard drives
- Data in motion, over the network
- Data in execution, in RAM

For any data to be used, it has to exist in RAM. Everything that matters must exist in RAM. By being in RAM, you are the center of the universe. Yet for all its power, until now nobody has a platform to analyze RAM. There are host-based IDS products, and AV, but all of these depend on the OS to query things about the OS – age old rootkit problem. The system is subverted and it’s game over. Our solution steps aside the OS and analyzes the physical RAM snapshot –offline-, thus avoiding any malware trickery.

There is a high barrier to entry to this work. We open the RAM, look inside, and extract objects. We reverse engineered every version and service pack of Microsoft Windows to be able to do this. We can find every process, every driver, and every line of assembly code of every software component. And, we do it without using the operating system – we do it without executing the environment we analyze.

In my grand vision we will build a picture of the true enterprise cyberspace. We have radical new technologies, like Digital DNA, that can be used to identify fragments of documents, strains of malware, intellectual property, fingerprints of email attachments, etc. Although we are tackling malware, our platform is generic and could be used for many other markets (IP asset tracking, E-Discovery, etc). As a company, we couldn’t ask to be in a better place in a market. We are set to explode.