Andrew Hay

the man, the myth, the blog

July 21, 2014
by Andrew Hay

New Tool: web2intel


Script to fetch malicious domain and URL lists from sites that publish RSS feeds or raw HTML pages.


To obtain the tool, please visit and download the associated files or simply run the following command at your command prompt:


Supported Lists


./web2intel.rb <option> <extras> 

For command syntax, please visit the GitHub repository.

Example 1 – Domains only

$ ./web2intel.rb --sucuri_iframe
#Title: Sucuri Research Labs Hidden iframes list
#2014-07-20 15:08:14 -0700
....list of domains....

Example 2 – Full URLs

$ ./web2intel.rb --sucuri_iframe --urls
#Title: Sucuri Research Labs Hidden iframes list
#2014-07-20 15:08:42 -0700
....list of URLs....


For any questions, bugs, or concerns, please use the GitHub issue submission system and/or reach out to @andrewsmhay on Twitter.

© Andrew Hay, 2014

May 1, 2014
by Andrew Hay

Research: Xerox Printers Beaconing Out To The Internet

4896710690_8cf5781412_b-241x300While conducting some research, I happened to notice a rather odd domain name that a particular server was beaconing out to. The domain in question was Initially, I figured that the domain could be malware or phishing related as the likelihood of Xerox Corporation using such a long domain was relatively low. Upon further investigation, the xeroxdiscoverysupernode3 domain wasn’t even registered. Could a piece of malware have been constructed to call out to this specific domain to download additional files? Why wouldn’t the malware author register the domain ahead of time if that was the plan?

As this domain ended in the number 3, I pondered the idea of there being a ’1′, ’2′, or maybe even a ’4′ numbered domain that followed this same pattern. It turned out that xeroxdiscoverysupernode1, xeroxdiscoverysupernode2, and xeroxdiscoverysupernode3 were actively being queried for within the OpenDNS global infrastructure. Not only were the domains being queried, but each was receiving roughly 2,000 queries per hour (as seen below).


The plot thickens…

The full post can be read here:

Photo Credit: via Compfight cc

April 1, 2014
by Andrew Hay

Quick fix for Ruby after Xcode 5.1 update

If you’ve recently upgraded XCode to 5.1 on your OS X workstation/laptop you may have run into the following error when trying to install or update a gem:

root# gem install shodan
Fetching: json-1.8.1.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing shodan:
ERROR: Failed to build gem native extension.

/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb
creating Makefile

make "DESTDIR="
compiling generator.c
linking shared-object json/ext/generator.bundle
clang: error: unknown argument: '-multiply_definedsuppress' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
make: *** [generator.bundle] Error 1

Gem files will remain installed in /Library/Ruby/Gems/2.0.0/gems/json-1.8.1 for inspection.
Results logged to /Library/Ruby/Gems/2.0.0/gems/json-1.8.1/ext/json/ext/generator/gem_make.out

As discussed here the reason for this sudden error is found in the Xcode Release Notes:

The Apple LLVM compiler in Xcode 5.1 treats unrecognized command-line options as errors. This issue has been seen when building both Python native extensions and Ruby Gems, where some invalid compiler options are currently specified.

According to the blog post it seems that the newer version of the llvm compiler shipping with Xcode 5.1 is a little more restrictive when it comes to warnings. Furthermore it says that:

Projects using invalid compiler options will need to be changed to remove those options.
That is, developers should not expect this change to be reverted in the future.

It turns out that the temporary fix, until everyone updates their gems, is to run the following command instead of the usual gem install command:

root# ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future gem install gem_name

For example:

root# ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future gem install shodan
Building native extensions. This could take a while...
Successfully installed json-1.8.1
Fetching: shodan-1.0.0.gem (100%)
Successfully installed shodan-1.0.0
Parsing documentation for json-1.8.1
Installing ri documentation for json-1.8.1
Parsing documentation for shodan-1.0.0
Installing ri documentation for shodan-1.0.0
2 gems installed

There you go. Hope it helps.