honeyblog - CWSandbox

Automatic Analysis of Malware Behavior using Machine Learning

nospam@example.com (Thorsten Holz) — Mon, 28 Dec 2009 12:15:00 +0100

In the last couple of years, several honeypot solutions to automatically "collect" malware samples were developed. With these tools, it is possible to obtain copies of malware samples without any human interaction. As a result, we are able to collect quite a few malware samples per day, which then also need to be analyzed. Thus, several sandbox solutions were developed that automate the analysis step by performing dynamic, behavior-based analysis. The result of the dynamic analysis is typically a report that summarizes the observed behavior. The next logical step is to use that information to perform malware classification and malware clustering: at the end of that process, we can then obtain information about which samples perform basically the same kind of activity. We can then automatically find variants of well-known threats, identify new malware families, and reduce the manual effort needed to analyze the large number of incoming malware samples.

In the last couple of months, we worked on malware classification and malware clustering. The results are summarized in a technical report. In the article, we introduce a learning-based framework for automatic analysis of malware behavior. To apply this framework in practice, it suffices to collect a large number of malware samples and monitor their behavior using a sandbox environment. By embedding the observed behavior in a vector space, reflecting behavioral patterns in its dimensions, we are able to apply learning algorithms, such as clustering and classification, for analysis of malware behavior. Both techniques are important for an automated processing of malware samples and we show in several experiments that our techniques significantly improve previous work in this area. For example, the concept of prototypes allows for efficient clustering and classification, while also enabling a security researcher to focus manual analysis on prototypes instead of all malware samples. Moreover, we introduce a technique to perform behavior-based analysis in an incremental way that avoids run-time and memory overhead inherent to previous approaches.

Abstract
Malicious software — so called malware — poses a major threat to the security of computer systems. The amount and diversity of its variants render classic security defenses ineffective, such that millions of hosts in the Internet are infected with malware in form of computer viruses, Internet worms and Trojan horses. While obfuscation and polymorphism employed by malware largely impede detection at file level, the dynamic analysis of malware binaries during run-time provides an instrument for characterizing and defending against the threat of malicious software.
In this article, we propose a framework for automatic analysis of malware behavior using machine learning. The framework allows for automatically identifying novel classes of malware with similar behavior (clustering) and assigning unknown malware to these discovered classes (classification). Based on both, clustering and classification, we propose an incremental approach for behavior-based analysis, capable to process the behavior of thousands of malware binaries on a daily basis. The incremental analysis significantly reduces the run-time overhead of current analysis methods, while providing an accurate discovery and discrimination of novel malware variants.

The full technical report is available at http://honeyblog.org/junkyard/paper/malheur-TR-2009.pd. It was joint work with Konrad Rieck, Philipp Trinius, and Carsten Willems. And the word cloud was generated using http://www.wordle.net/.

AV Tracker

nospam@example.com (Thorsten Holz) — Thu, 22 Oct 2009 12:49:00 +0200

A couple of days ago, the website "AV Tracker" went online, which publishes information about various automated analysis systems. The idea is that the attacker uploads a binary to an analysis system, waits for the sample to be executed, and then the binary phones home some information to a server under the control of the attacker. The collected information is then published at "AV Tracker", exposing information about the analysis systems. Besides some well-known AV companies, also CWSandbox and Anubis were affected.

We analyzed the binary and found that it sends a simply HTTP request, in which all extracted information is encoded. An example for an analysis report generated by one of the samples is http://anubis.iseclab.org/?action=result&task_id=361b5a8ee7235954252b02d33b3a7d24. This can be defeated by blocking access to the reporting server or by regularly changing the IP address of the analysis systems, but at the end this will be some kind of arms race again.

Some other interesting information is also embedded in the binary. When extracting the strings from the sample, the following text becomes visible (some information is hidden by dots):

This is Peter Kl....... fuck ...... fuck the world fuck you all!
I was once working with ...... and was a white hat, now I am the worst mean motherfucker black hat and I am selling the source code of ...... .. :D
I am with the ~~Sinowal~~Whistler developers, funny days, aren't ;) and fuck ..... they don't have no idea :D bitches

A related article was also published today at http://www.viruslist.com/en/weblog under the title "A black hat loses control".

Thread Graphs for Visualizing Malware Behavior

nospam@example.com (Thorsten Holz) — Tue, 25 Aug 2009 23:33:00 +0200

The last blog post dealt with our recent research on visualizing malware behavior. Now a quick update on the thread graphs we generate for visualizing malware behavior: since tree maps display nothing about the sequence of operations, we use another presentation format to visualize the temporal behavior of the individual threads of a sample. A thread graph can be regarded as a behavioral ﬁngerprint of the sample that represents the temporal order of executed system commands and the different threads spawned by a binary. The x-axis represents the time (sequence of performed actions), while the y-axis indicates the operation/section of the performed action. An analyst can then study this behavior graph to quickly learn more about the actions of each individual thread.

The following two pictures show examples of this kind of visualization:

On the left hand picture, we can see that one thread is responsible for the majority of operations for the sample. This thread performs many registry operations and initially performs many network- and system-related operations (operations 90-140). Additionally, two more threads are spawned, but they perform only a limited amount of operations during the analysis phase. The thread graph for the malware sample on the right side is completely different and an analyst can get a quick overview of what actions a given samples performs.

"Visual Analysis of Malware Behavior Using Treemaps and Thread Graphs"

nospam@example.com (Thorsten Holz) — Fri, 21 Aug 2009 15:45:00 +0200

I continue the series of recently or upcoming papers with a paper we will publish at VizSec'09 entitled "Visual Analysis of Malware Behavior Using Treemaps and Thread Graphs". In the recent years, we saw a lot of progress in the area of automated malware analysis. Nowadays tools such as CWSandbox, Anubis, ThreatExpert, or Norman Sandbox are available. These tools analyze a given binary and generate a report which contains a summary of the observed behavior while executing the sample. Such reports are often quite long, it is for example not uncommon for a CWSandbox report to be longer than 100 lines. An analyst thus has to read the report in order to get an understanding of what a given sample is doing. In this paper we present an approach to visualize the behavior report with treemaps and behavior graphs (i.e., visualizing the behavior of the individual threads over time). This helps to get a quick overview of what a given sample does and also samples from one malware family have a similar looking treemap/behavior graph.

As an example, consider the following three pictures which each show the treemap generated for three distinct samples of the Bagle worm:

Each picture shows a treemap of the behavior: the x-axis depicts the type of action performed, e.g., whether the sample performed actions related to the filesystem, the registry, or the network. The y-axis devides the actions into operations, i.e., whether it was a read or write access to the registry. As you can see, the behavior of the Bagle sample is (more or less) consistent across different samples from the same family. Below you can find the visualization of two Swizzor samples and one Allaple sample.

Samples from the same family have a similar visualization, while samples from different families look different. This could help an analyst to quickly identify if the sample is interesting or just another small variant of a well-known family. This research will be integrated in the frontend of http://cwsandbox.org.

Abstract: We study techniques to visualize the behavior of malicious software (malware). Our aim is to help human analysts to quickly assess and classify the nature of a new malware sample. Our techniques are based on a parametrized abstraction of detailed behavioral reports automatically generated by sandbox environments. We then explore two visualization techniques: treemaps and thread graphs. We argue that both techniques can effectively support a human analyst (a) in detecting maliciousness of software, and (b) in classifying malicious behavior.

Malicious PDFs Analysis Continued

nospam@example.com (Thorsten Holz) — Mon, 12 Jan 2009 13:18:00 +0100

After my initial posting about the possibility to analyze PDF files with CWSandbox we received a few more such samples. In all cases the PDF file exploits a vulnerability in Acrobat Reader once the file is opened. With the help of CWSandbox it is possible to observe this exploit and also the actions of the malware after the compromise (e.g., downloading of additional malware from another server). Please find below three additional examples of such reports:

https://cwsandbox.org/?page=report&analysisid=879663&password=vqtgp

https://cwsandbox.org/?page=report&analysisid=878305&password=utxuc

https://cwsandbox.org/?page=report&analysisid=878393&password=pmviw

If you happen to have more malicious PDFs, please submit them at cwsandbox.org :-)

Analyzing Malicious PDF Files

nospam@example.com (Thorsten Holz) — Mon, 22 Dec 2008 13:50:00 +0100

Recently we added a new feature to cwsandbox.org: It is now also possible to upload suspicious PDF files that are then analyzed with the help of CWSandbox. Basically we open the submitted file with Acrobat Reader 8.1.1 since that version has several vulnerabilities. During runtime, we then observe the behavior of Acrobat and can detect suspicious changes such as new files on the hard disk or modified registry keys. Based on the generated report, it is then possible to detect malicious PDF files.

An example of such an analysis is available at https://cwsandbox.org/?page=details&id=520505&password=sfgpk. The PDF file 0416.pdf is malicious and has a rather good detection by AV vendors (21/38 - full details). In the CWSandbox report, we can see that the PDF file is opened with Acrobat Reader and then it drops a new file called wuweb.exe which is also executed. Afterwards, several other files are dropped and a server located in Singapore is contacted. Unfortunately this server is now offline, but presumably the server was used to download additional malware from the system

Banking Trojans

nospam@example.com (Thorsten Holz) — Thu, 18 Dec 2008 09:43:00 +0100

My previous post already contains some information on our recent work, but I think it makes sense to include some more details. We wanted to study an attack class we call impersonation attacks, i.e., all attacks in which an attacker wants to steal a credential from a victim in order to impersonate as the victim at a provider:

This kind of attacks is quite common, for example also phishing attacks fall under this class: In such an attack, the attacker uses phishing e-mails as an attack channel and lures the victim into revealing his credentials at a bogus site. These credentials are then sent to the attacker using the harvesting channel, which can for example be e-mail. The attacker can then use the stolen credentials to impersonate as the victim, for example at an online bank.

We studied a specified type of impersonation attacks, namely the attacks in which keyloggers and banking trojans are used by the attacker. Example of such malware include ZeuS/Wsnpoem and Limbo/Nethell, which we studied in detail. Based on the information collected during dynamic analysis, we found many dropzones and got access to many logfiles. We performed a statistical analysis of this data and here are some highlights:

We found a total of 175 different countries in which the 170,000 victims are located and almost one third of the infected machines are located in either Russia or the United States.

We also found that the dropzones are located in many different Autonomous Systems (68 different AS in total), but several AS host a larger percentage of ZeuS dropzones: The three most common AS host 49% of all dropzones, indicating that there are some providers preferred by the attackers. Presumably those providers offer bullet-proof hosting, i.e., takedown requests are not handled properly by these providers.

In total, we found 10,775 unique bank account credentials in all logﬁles. This includes passwords and all bank account details as entered by a victim during a normal transaction. Furthermore, we found more than 5,600 full credit card details and tens of thousands of passwords for different sites.

The distribution of victim IP addresses is highly non-uniform: The majority of victims are located in the IP address ranges between 58.* – 92.* and 189.* – 220.*.

The results of analyzing the potential income of an attacker indicate that an attacker can earn several hundred dollars per day based on impersonation attacks with keyloggers – a seemingly lucrative business.

Full details are available in the technical report. Note that the data we collected during this study is very sensitive. We thus handed over this data to AusCERT, the national Computer Emergency Response Team (CERT) for Australia, since they are in a position to notify the victims.

Update: I received a few comments regarding how to protect against this threat. Best way for protection is patching and not clicking all links and attachments. Furthermore, you can protect yourself against keyloggers by using two-factor authentification when doing bank transactions. German banks offer services such as mobile TAN/SMS-TAN in which a transaction number is sent to the mobile phone to authorize a transaction. A weaker system is iTAN (indexed TAN). The Postbank also published some guidance on how to protect yourself. If you follow these guidelines, you should be relatively secure and not affected by banking trojans.

Technical Report: "Learning More About the Underground Economy: A Case-Study of Keyloggers and Dropzones"

nospam@example.com (Thorsten Holz) — Thu, 18 Dec 2008 09:27:00 +0100

In the last few months, we analyzed quite a few malware samples that are related to stealing of banking credentials. These keyloggers are used by attackers to harvest sensitive information like credit cards numbers, username/password combinations and similar data from an infected machine. We developed some techniques to automatically find the dropzones, i.e., the server that is used by the bad guys to send the stolen information to. The following picture illustrates the attack process:

The basic idea of our approach is to use honeypots to automatically collect malware samples, perform dynamic analysis with the help of CWSandbox and a user simulation, and use the observed data to find the dropzone in an automated way. Using these techniques, we were able to find more than 300 dropzones and we were also able to fully access more than 70 dropzones. We found stolen information from more than 170,000 victims (33 GB of data) and also analyzed this data: Within the dropzone data, we found more than 10,000 bank accounts with full information, more than 140,000 e-mail passwords for large portals and some other interesting infos.

Today we published a technical report that summarizes our findings.

Abstract: We study an active underground economy that trades stolen digital credentials.We present a method with which it is possible to directly analyze the amount of data harvested through these types of attacks in a highly automated fashion. We exemplify this method by applying it to keylogger-based stealing of credentials via dropzones, anonymous collection points of illicitly collected data. Based on the collected data from more than 70 dropzones, we present the ﬁrst empirical study of this phenomenon, giving many ﬁrst-hand details about the attacks that were observed during a seven-month period between April and October 2008. This helps us better understand the nature and size of these quickly emerging underground marketplaces.

Facebook friend spam / Koobface

nospam@example.com (Thorsten Holz) — Thu, 04 Dec 2008 13:27:00 +0100

Since a few days, a new round of malicious friend messages is going around at Facebook. The messages all look similar, an example is

"Oh noooooo
hxxp://www.facebook.com/l.php?u=hxxp://geocities.com%2Fmaxmonroe79%2Findex.htm..."

To reply to this message, follow the link below:
http://www.facebook.com/n/?inbox/readmessage.php&t=10085171....

Once a victim clicks on the link, he also needs to confirm the redirect on the Facebook site. Afterwards, the attackers use social engineering to trick the victim into installing the malware sample named flash_update.exe. I have also uploaded a movie to illustrate the infection process and to test the new media options I added to this blog: http://honeyblog.org/pages/20081204-koobface.html

Fortinet has some more information on a related incident: http://www.fortiguardcenter.com/advisory/FGA-2008-26.html