Security Data analysis: Web Proxy Log

You are most welcome to this post.

Really thanks to you for your interest in this topics. :)

Malware is often spread using vulnerabilities in web browsers and malware may also use HTTP/HTTPS for the CnCcommunication.

Many web proxies, such as the Cisco Web Security Appliance (Cisco WSA), can also decrypt SSL/TLS traffic so that it can be inspected. Today, much of the web traffic of Facebook, Google, and so on, uses HTTPS instead of HTTP to provide an encrypted web connection.

Most web proxy server logging is enabled by default to collect HTTP/HTTPS related log data.

Web proxy logs allow the security analysts to see what websites are accessed by the networked hosts. Web proxy provides precise logging of browsing sessions and can be used to help investigate web-based attacks. For example, if there are many POST commands going to an unknown site with a bad reputation, then data exfiltration may be assumed. POST commands, however, are also normal because queries to the web applications can be submitted through POST commands. GET commands to a suspicious website with a bad reputation containing .exe files could be a sign of droppers. A dropper is a program that has been designed to install some sort of malware on the victim’s machine.

! Squid Web Proxy Access Log Example

1265529266.756 261 192.168.1.45 TCP_TUNNEL/200 3511 CONNECT services.example.com:443 - HIER_DIRECT/209.165.200.235
1265529273.873 8 192.168.1.254 TCP_HIT/200 1563 GET http://www.example.com/en-us/default/layout/previous.gif - NONE/- image/gif
1265529273.881 12 192.168.1.254 TCP_HIT/200 2961 GET http://www.example.com/gif/lightning-100x60.jpg - NONE/- image/jpeg
1364222662.246 90 192.168.0.6 TCP_MISS/200 619 GET http://www.example.com/pagead/conversion/? - DIRECT/172.16.0.152 image/gif
1364222662.257 85 192.168.0.6 TCP_MISS/200 2347 GET http://www.example.com/pagead/conversion/? - DIRECT/172.16.0.152 image/gif
1364222675.456 15 192.168.2.1 TCP_MISS/200 141159 GET http://spotlight2.com/system/logs/k1.exe 
1364222689.532 77 192.168.2.1 TCP_MISS/200 36098 GET http://spotlight2.com/module/4c06c7a4c2bd4567139df133455

In the example above, the output of some of the Squid Web Proxy access log entries are shown. Squid is a common open source web proxy server. The fields in the entry describe the HTTP request or response. The first field that is displayed is the time stamp. The format that is used in this time stamp may be unfamiliar, but it is commonly used in Unix and Linux environments. This format is called “epoch time,” which is the number of seconds since January 1, 1970, at midnight GMT.

. Many online epoch converters can convert the epoch time to a more familiar format.

Fields in these entries show the IP address of the requester, along with the HTTP method and the URL that is contained within the request. Other fields focus on the status of the response, including the HTTP status code (200 = OK for example) and whether the response was cached or not. A TCP_HIT log means that the HTTP response was cached in the web proxy cache. A TCP_MEM_HIT log means that a valid copy of the requested object was in the RAM memory, thus avoiding disk accesses. A TCP_TUNNEL log means that the traffic was tunneled as raw binary to the web server for which Squid is not able to act as a proxy. In this case, the Squid proxy does not understand or interpret the tunneled traffic, for example, if the tunneled traffic is encrypted HTTPS traffic.

Looking at the last two log entries, an analyst may be suspicious of the two GET requests for downloading an .exe followed by a suspicious filename. By using Virustotal, http://spotlight2.com maybe a malicious site since 9 out of 68 URL scanners found that site to be malicious.

Further investigate by Googling system.log/k1.exe. The k1.exe file is related to the Vawtrak malware.

When analyzing HTTP events, the analyst should be able to recognize some of the common HTTP request methods and status code.

For example, URL redirection is used to cause the browser to be redirected from one URL to another URL. A URL redirection attack occurs when the victim’s browser going to a legitimate web page is redirected to the attacker’s web page with exploit code. Un-validated URL redirects are possible when a web application accepts un-trusted input that could cause the web application to redirect the request to a URL contained within un-trusted input.

An attacker could send the victim a link such as http://www.example.com/login?toURL=http://attacker.com/login. Since the malicious link is identical to the original site (http://www.example.com/login), the user will think it is the legitimate site, and enter valid credentials, then be redirected to http://attacker.com/login. Then it could ask for the credentials again by claiming they are not valid, which leads to a simple and efficient phishing attack.

A URL redirect is a response with a status code beginning with 3 that causes a browser to display a different web page. Different status codes are used by the browser to understand the purpose of the URL redirect, how to handle caching, and which request method to use for the subsequent request.

The following tables show some of the common HTTP status codes and HTTP request methods.

Common HTTP Status Codes, from RFC 2616

Transaction/Error	Status	Description
Successful Transactions	200	OK
	201	Created
	202	Accepted
Redirected Transactions	301	Moved permanently
	302	Moved temporarily
	304	Not modified
Client-Side Errors	400	Bad request
	401	Unauthorized
	403	Forbidden
	404	Not found
Server-Side Errors	500	Internal server error
	501	Not implemented
	502	Bad gateway
	503	Service unavailable

Common HTTP Request Methods, from RFC 2616

Request	Method	Definition
Common, Legitimate Requests	GET	Retrieval and simple searches
	POST	Submit data-query
	PUT	Upload data-files
Uncommon, Potentially Malicious Requests	HEAD	Metadata retrieval
	DELETE	Remove resource
	TRACE	Application layer trace of route
	OPTIONS	Request available methods
	CONNECT	Tunnel SSL connection
	PROPFIND	Retrieve properties of an object

Collected from cisco.com