Google Safe Browsing

Monday, November 13, 2006

While Google's Safe Browsing service has been available for some time via the Google Toolbar, this feature will see wide deployment with the release of Mozilla Corp's Firefox 2. In this analysis, we will delve into the workings of Google's Safe Browsing service and its integration with Firefox.

Safe Browsing

Google Safe Browsing (SB) is an initiative to curb malicious phishing activity by warning users when they navigate to a potential phishing site. Obviously, this requires some coordination with the user's web browser. Currently, Firefox 1.x users can download the Google Toolbar and Firefox 2.x has native SB support. MSIE users cannot currently take advantage of Google's SB service, but IE 7 has native phishing protection provided by Microsoft.

Methodology

In Firefox 2.x, there are two methods to utilize SB:

1) Local database comparison

In this mode, each URL visited is compared against a local database of suspected phishing sites. The local database is updated periodically while Firefox is running by querying Google.

2) Remote querying

Using this method, visited URLs will be sent to Google over HTTP to be verified against Google's latest list. While the URLs are sent over HTTP, they are encrypted using a shared secret that was previously bootstrapped via HTTPS. This method provides enhanced protection as there's no chance of verifying against a potentially stale local DB.

As the local database method is enabled by default in Firefox 2.x, we will explore it in further detail.

Storage Format

The local database is stored in a sqlite DB at the following location:

~/.mozilla/firefox/<profile>/urlclassifier2.sqlite

Details of the urlclassifier2.sqlite schema:

sqlite> .tables
goog_black_enchash  goog_black_url      goog_white_domain   goog_white_url
sqlite> .schema
CREATE TABLE 'goog_black_enchash' (key TEXT PRIMARY KEY, value TEXT);
CREATE TABLE 'goog_black_url' (key TEXT PRIMARY KEY, value TEXT);
CREATE TABLE 'goog_white_domain' (key TEXT PRIMARY KEY, value TEXT);
CREATE TABLE 'goog_white_url' (key TEXT PRIMARY KEY, value TEXT);

goog_black_enchash - details on encrypted hash format can be found here
goog_black_url - contains blacklisted URLs of suspected phishing sites
goog_white_domain - contains whitelisted hostnames as determined by Google
goog_white_url - not currently used

Sample contents of the goog_black_url table:

sqlite> SELECT key FROM goog_black_url LIMIT 3;
uggc://0168864.argfbyubfg.pbz/jjj.shyygvygcbxre.pbz/gbc5/
uggc://0168864.argfbyubfg.pbz/jjj.shyygvygcbxre.pbz/gbc5/qravrqargryyre.ugzy
uggc://023417.pbz/rhebonax.te/rhebcbegny/ybtva.ugzy

Sample ROT13'ed contents of the goog_black_url table:

sqlite> SELECT ROT13(key) FROM goog_black_url LIMIT 3;
http://0168864.netsolhost.com/www.fulltiltpoker.com/top5/
http://0168864.netsolhost.com/www.fulltiltpoker.com/top5/deniedneteller.html
http://023417.com/eurobank.gr/europortal/login.html

As you can see from the above entries, the contents of the sqlite database are ROT13'ed. The reason for this obfuscation is to avoid falsely tripping anti-virus/malware programs which may come across these URLs in their scanning activities. This is apparent in the Firefox source:

mozilla/toolkit/components/url-classifier/src/nsUrlClassifierDBService.cpp:631

// We use ROT13 versions of keys to avoid antivirus utilities from
// flagging us as a virus.
nsCString keyROT13(key);
Rot13Line(keyROT13);

Update Mechanism

The local sqlite database is updated by Firefox via a HTTP GET request to Google at the following URL:

http://sb.google.com/safebrowsing/update

That URL will list the latest revision number of each table. By specifying the current revision of the local database, the Firefox client can request incremental updates of each table. For example, a Firefox client with revision 1.7822 of goog_black_url could request:

http://sb.google.com/safebrowsing/update?version=goog-black-url:1:7822

A sample response from Google would be:

[goog-black-url 1.7824 update]
-http://12.148.105.125/12.30.161/.www.paypal.com.webscr.phpcmd=login/
-http://200.105.34.163/~admin/www.paypal.com/index.php
-http://200.32.3.213/~mysql/www.paypal.com/index.php
...
-http://www.paypal-update.com/details.php
+http://195.128.35.60/www.paypal.com/cgi-bin/webscr_cmd=_login-run3194/ c

URLs prefixed with a "-" are pruned from the sqlite database while URLs prefixed with a "+" are inserted in the db.

Update Interval

When Firefox is first started, an update request will be sent randomly between 0 to 5 minutes. The next update request will be sent 15-45 minutes later. All subsequent update requests will occur every 30 minutes. If errors or timeouts occur, the client backs off on the update requests exponentially.

More information on the update interval can be found on Mozilla's wiki.

User Response

When a user visits a suspected phishing site and is warned by Firefox, they are given the option to navigate away from the site or ignore the warning and continue on. The user's response to this prompt is then transmitted back to Google through the following URL:

http://sb.google.com/safebrowsing/report?evts=phish<response>&evtd=<url>

where <response> is either "accept" or "decline" and <url> is the URL of the suspected phishing site. Users are also given the option to report potential false positives that have been flagged as a phishing site.

Pcap Traces

Attached are a few pcap traces of some of the SB features discussed:

Copyright © 2021 - Jon Oberheide