One Packet OS Fingerprinting And API Access Unveiled

The latest version of SinFP3 (v1.20, as of this writing) introduces two new cool features: the ability to perform a SYN scan and doing OS fingerprinting at the same time. The idea is to use SYN|ACK answers to the SYN scanning process to accurately identify the remote operating system nature. The second new feature is a server mode allowing third-party applications to access the SinFP3 fingerprinting engine. We also created a new output plugin to display results in a simpler manner than in previous versions of SinFP3.

One packet OS fingerprinting

If you don’t have installed SinFP3 yet, you can follow these guides:

Then, try to identify your target’s classic TOP10 open ports as follows. The -synscan-fingerprint is the new argument giving access to the feature, and the new default output plugin is the simple one:

# sinfp3.pl -synscan-fingerprint -target openbsd.org -port top10 -best-score
[+] [J:0] Loaded Input:  Net::SinFP3::Input::SynScan
[+] [J:0] Loaded DB:     Net::SinFP3::DB::SinFP3
[+] [J:0] Loaded Mode:   Net::SinFP3::Mode::Active
[+] [J:0] Loaded Search: Net::SinFP3::Search::Active
[+] [J:0] Loaded Output: Net::SinFP3::Output::Simple
[+] [J:0] Starting of Input [Net::SinFP3::Input::SynScan]
[+] [J:1] Starting of job with Next [199.185.137.3]:25 flags: 0x12
[+] [J:2] Starting of job with Next [199.185.137.3]:80 flags: 0x12
[199.185.137.3  ]:80     reverse: unknown  [ 94%: OpenBSD 4.x]
[199.185.137.3  ]:80     reverse: unknown  [ 94%: OpenBSD 3.x]
[199.185.137.3  ]:25     reverse: unknown  [100%: OpenBSD 4.x]
[199.185.137.3  ]:25     reverse: unknown  [100%: OpenBSD 3.x]

 

You can see that two open ports have been found, and the detected operating system (on a per open port basis) appears to be OpenBSD 3.x or 4.x.

You can still access the more verbose output by launching the fingerprinting process in full mode (2 packets for Internet fingerprinting with -active-2 parameter) and displaying results with the console output plugin with -output-console parameter:

# sinfp3.pl -active-2 -target openbsd.org -port top10 -best-score -output-console 
[+] [J:0] Loaded Input:  Net::SinFP3::Input::SynScan
[+] [J:0] Loaded DB:     Net::SinFP3::DB::SinFP3
[+] [J:0] Loaded Mode:   Net::SinFP3::Mode::Active
[+] [J:0] Loaded Search: Net::SinFP3::Search::Active
[+] [J:0] Loaded Output: Net::SinFP3::Output::Console
[+] [J:0] Starting of Input [Net::SinFP3::Input::SynScan]
[+] [J:1] Starting of job with Next [199.185.137.3]:25 hostname[openbsd.org] \ 
   reverse[unknown] mac[00:24:d4:ae:f4:3c]
[+] [J:2] Starting of job with Next [199.185.137.3]:80 hostname[openbsd.org] \ 
   reverse[unknown] mac[00:24:d4:ae:f4:3c]
Result for target [199.185.137.3]:80:
S1: B11113 F0x12 W16384 O0204ffff M1460 S0 L4
S2: B11113 F0x12 W16384 O0204ffff01010402010303ff0101080affffffff44454144 M1460 S3 L24
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.3
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.5
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 3.8
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 3.6
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.0
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.6
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.8
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 3.5
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.2
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.7
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 3.9
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 3.7
IPv4: [score:94]: BH0FH0WH0OH0MH0SH1LH0/S1S2: BSD: OSS: OpenBSD: 4.4
Result for target [199.185.137.3]:25:
S1: B11113 F0x12 W16384 O0204ffff M1460 S0 L4
S2: B11113 F0x12 W16384 O0204ffff01010402010303ff0101080affffffff44454144 M1460 S0 L24
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.3
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.5
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 3.8
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 3.6
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.0
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.6
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.8
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 3.5
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.2
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.7
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 3.9
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 3.7
IPv4: [score:100]: BH0FH0WH0OH0MH0SH0LH0/S1S2: BSD: OSS: OpenBSD: 4.4

 

SinFP3 API access with the SinFP3 network protocol

The second feature allows access to the SinFP3 server. The idea is to launch SinFP3 in daemon mode, and have a client/server communication protocol to “ask” the fingerprinting engine to perform actions. This protocol is described within an RFC, available here.

You can implement this protocol in any language you want, and you will be able to use the fingerprinting engine in whatever language you prefer. For now, a Perl version of the communication protocol exists: Net::Frame::Layer::SinFP3.

Here is how to launch the SinFP3 server:

% sinfp3.pl -input-server -output-client -passive -best-score
[+] [J:0] Loaded Input:  Net::SinFP3::Input::Server
[+] [J:0] Loaded DB:     Net::SinFP3::DB::SinFP3
[+] [J:0] Loaded Mode:   Net::SinFP3::Mode::Passive
[+] [J:0] Loaded Search: Net::SinFP3::Search::Passive
[+] [J:0] Loaded Output: Net::SinFP3::Output::Client
[+] [J:0] Starting of Input [Net::SinFP3::Input::Server]

 

The server is now listening on localhost with port 32000/TCP. An example client application is available in the examples/ directory of Net::SinFP3 distribution. For now, only the passive mode is implemented within the server. If there is some interest from the community, we will enhance it to give access to the active mode.

The example below is crafting a client request embedding a TCP SYN packet in raw format (sent by the sinfp3-request-passive-netcat.pl script). It is sent to the SinFP3 server which is configured in passive mode. Then, the reply is analyzed by sinfp3-decode-netcat.pl script and printed on standard output.

% perl examples/sinfp3-request-passive-netcat.pl | nc localhost 32000 | \ 
   perl examples/sinfp3-decode-netcat.pl
SinFP3: version:1  type:0x04  flags:0x0270
SinFP3: code:0x01  tlvCount:4  length:88
SinFP3::Tlv: type:0x24  length:7  value:46726565425344 [FreeBSD]
SinFP3::Tlv: type:0x25  length:3  value:372e34 [7.4]
SinFP3::Tlv: type:0x26  length:3  value:372e78 [7.x]
SinFP3::Tlv: type:0x29  length:1  value:64 [100%]
SinFP3::Tlv: type:0x24  length:7  value:46726565425344 [FreeBSD]
SinFP3::Tlv: type:0x25  length:3  value:372e33 [7.3]
SinFP3::Tlv: type:0x26  length:3  value:372e78 [7.x]
SinFP3::Tlv: type:0x29  length:1  value:64 [100%]
SinFP3::Tlv: type:0x24  length:7  value:46726565425344 [FreeBSD]
SinFP3::Tlv: type:0x25  length:3  value:382e33 [8.3]
SinFP3::Tlv: type:0x26  length:3  value:382e78 [8.x]
SinFP3::Tlv: type:0x29  length:1  value:64 [100%]
SinFP3::Tlv: type:0x24  length:7  value:46726565425344 [FreeBSD]
SinFP3::Tlv: type:0x25  length:3  value:382e32 [8.2]
SinFP3::Tlv: type:0x26  length:3  value:382e78 [8.x]
SinFP3::Tlv: type:0x29  length:1  value:64 [100%]

 

On the server, a new job has been processed:

[+] [J:1] Starting of job with Next [127.0.0.1]:37125

 

That’s all for today, folks. Feel free to follow @metabrik for updates, or subscribe to SinFP mailing list:

Full list of changes for v1.20

1.20 Sun Nov 25 14:44:37 CET 2012
   - NEW: Input::Server: runs as a daemon to answer requests coming from
          clients. They must speak the SinFP3 protocol.
   - NEW: Output::Simple: now default mode instead of Output::Console
   - NEW: Input::SynScan: can be used to fingerprint target by just using the
          SYN|ACK response to our SYN (one packet fingerprinting \o/)
   - UPDATE: verbose mode 1 is now default. Many updates on log messages
             levels.
   - NEW: sinfp3.pl: -synscan-fingerprint argument
   - new: sinfp3.pl: -version prints Perl modules version
   - new: sinfp3.pl: -quiet to set verbose level 0
   - new: sinfp3.pl: -passive to set to Mode::Passive and Search::Passive
                     plugins
   - new: integration of p0f-3.06b passive signatures into sinfp3.db
   - update: Input::Sniff: must use Net::Frame::Dump 1.12 now
   - bugfix: on -dns-reverse, moved to Search modules, when generating Results
   - bugfix: Search::Passive: sets ip and port attributes for Results
   - bugfix: Global: when -port argument has an invalid format
   - bugfix: sinfp3.pl: usage help

Installation Guide For SinFP3 Under Debian/Ubuntu

Installation of SinFP3 under Debian/Ubuntu may be a bit tricky. They decided to rename one of the best low-level-hiding-thingy library initially called libdnet. Under these Linux distributions, the new name is libdumbnet. We will explain in this guide how to manually install SinFP3 on a fresh Debian/Ubuntu installation.

Note: Net::Libdnet is now part of Ubuntu. An easier guide to install SinFP3 is available.

Installing required C libraries and development headers

First things first, you have to install libpcap and libdnet libraries and headers. Other libraries and headers are also required, like libexpat.

# aptitude install libdumbnet1 libdumbnet-dev libpcap-dev libexpat-dev

Installing Net::Pcap and Net::Libdnet

Net::Pcap should be straightforward to install. For later use, you must also install Class::Gomor:

# aptitude install libnet-pcap-perl libclass-gomor-perl cpanminus

For Net::Libdnet, you have to do it manually:

# cd /tmp
# wget http://search.cpan.org/CPAN/authors/id/G/GO/GOMOR/Net-Libdnet-0.97.tar.gz
# tar zxvf Net-Libdnet-0.97.tar.gz
# cd Net-Libdnet-0.97
# patch -p0 < ubuntu-new.patch
# perl Makefile.PL && make && make test && make install
[..]
Installing /usr/local/bin/dnet.pl
Appending installation info to /usr/local/lib/perl/5.14.2/perllocal.pod

You should have a working installation of Net::Pcap and Net::Libdnet, the base tools on which SinFP3 relies on.

Finalizing the installation

You did the hard part, to finish it is just a matter of launching a single `cpan’ command. You can answer all prompts with the default value.

# cpanm Net::SinFP3
--> Working on Net::SinFP3
Fetching http://www.cpan.org/authors/id/G/GO/GOMOR/Net-SinFP3-1.20.tar.gz ... OK
Configuring Net-SinFP3-1.20 ... OK
Building and testing Net-SinFP3-1.20 ... OK
Successfully installed Net-SinFP3-1.20
1 distribution installed

You can now fire SinFP3 by using `sinfp3.pl’ command. Happy fingerprinting to you. Ah, don’t forget to update the database:

# sinfp3.pl -db-update -verbose 1

Scanning The Internet: How To Not Get Caught

Scanning the Internet is a task undertaken for different purposes. Good guys are doing it for research purposes, like obtaining statistics, and bad guys to gather easy targets to exploit. A key point for both is to not be detected, and not raise abuse complaints. We propose a method to scan the full Internet IPv4 address space with the goal to avoid easy detection.

Counting the number of possible targets

A first step is to list the number of “routable” IPv4 addresses. An IPv4 address is made of 32-bits, thus the total number is:

% echo "2 ^ 32" |bc
4294967296

It’s around 4.3 billion addresses. But from this full range, we can remove subnets that should not be used on the Internet. Some of these subnets are listed on RFC3330 standard. For our methodoly, we decided to not remove those reserved for future allocation. Of course, we have run out of IPv4 addresses, so these subnets will probably be attributed. We end up with the following list of IPv4 addresses to remove for our scanning purposes:

0.0.0.0/8      : 16777216 addresses
10.0.0.0/8     : 16777216 addresses
127.0.0.0/8    : 16777216 addresses
169.254.0.0/16 : 65536 addresses
172.16.0.0/12  : 1048576 addresses
192.0.2.0/24   : 256 addresses
192.168.0.0/16 : 65536 addresses
224.0.0.0/4    : 268435456 addresses
240.0.0.0/4    : 268435456 addresses
Full space     : 4294967296 addresses
Reduced space  : 3706584832 addresses

By removing these subnets, we have a final number of targets around 3.7 billion addresses. We could also have removed network and broadcast adresses, but given the dynamic nature of the Internet, we preferred to not follow that path. Furthermore, you would have to gather all registered subnets to know these special addresses, and it would make the generation of addresses a little bit harder (and longer).

Divide and conquer

Our methodoly for scanning the full Internet and not get caught fulfills these goals:

  • allows distributed scanning without overlaps from different scanners
  • stealthiness to avoid abuse complaints from target administrators
  • gentle against target subnets by not sending too much trafic to a single subnet

To cope with these needs, we decided to generate randomly all the possible 3.7 billion addresses, one time. We split the IPv4 address space into 1000 chunks, to allow distributed scanning without overlaps. 1/1000th of the Internet is around 3.7 million addresses. We used the following Perl script to generate 1000 files, each of them containing one address per line. To have a complete level of randomization, after the script has ended generating the 1000 chunks, we used the `shuffle’ Linux command against each file.

Here is a sample of the 10 first addresses found in one chunk:

% head -n 10 ip4-space-100.txt
207.100.116.47
181.197.55.55
46.29.180.153
130.24.211.155
114.202.145.13
45.144.69.24
177.189.197.238
152.14.200.56
7.12.20.49
177.66.110.168

Rainbow tables of Internet scanning

Yes, you can think of these files as the rainbow tables of Internet scanning. We decided to publish 10 of them (to limit the bandwidth taken from our server when you download them). Each file takes around 20 MB, gzipped. This allows you to use these rainbow tables to perform scanning against 1% of the Internet IPv4 address space.

To download them, you have to run the following script:

for i in 100 101 102 103 104 105 106 107 108 109; do
   wget http://www.metabrik.org/wp-content/uploads/2015/02/ip4-space-$i.txt.gz
done

Happy scanning.