Metabrik Core And Repository 1.10 Released

Following our lightning talk from 2015 conference, we are proud to release the version 1.10 of Metabrik Core and Repository. Update using Mercurial or follow the installation procedure.

You can find the few slides which were presented at the following link.

Lots of new awesome Briks

We added many Briks for this new release, here is the description for them:

  • api::bluecoat: play with Bluecoat REST API
  • api::splunk: play with the Splunk REST API
  • api::virustotal: play with Virustotal REST API
  • client::udp: a UDP socket client (UDP netcat)
  • client::ssl: check various stuff about a SSL/TLS connection
  • client::rest: the base REST client for use with Briks from API Category
  • client::rsync: a wrapper around rsync program
  • client::twitter: a Twitter client
  • database::mysql: interract with MySQL databases
  • file::dump: read and write dump files
  • file::hash: genrated various digests from files
  • file::ole: play with Microsoft files that embed OLE components
  • lookup::iplocation: geolocation for IP addresses
  • string::ascii: convert ASCII characters
  • string::csv: encode/decode CSV strings
  • string::hostname: parse a FQDN
  • string::regex: experiment with regexes
  • system::freebsd::pf: control Packet Filter
  • system::freebsd::jail: control jails

Just type help <Brik> to know more:

Meta:~> use string::regex 
[*] core::shell: use: Brik [string::regex] success
Meta:~> help string::regex 
[+] run string::regex encode <$regex|$regex_list>

Complete list of changes


1.10 Tue Oct 27 20:13:36 CET 2015
   - FEATURE: core::context: allows to pass complex structs arguments to run and set Commands
     Example: run network::arp scan $info->{subnet}
   - FEATURE: core::context: allows also to execute Perl code within an Argument of a
     run Command
     Example: run client::dns ptr_lookup "[ map { @$_ } values %$RUN ]"
   - FEATURE: core::shell: allows to complete aliases (can be disabled via
     aliases_completion Attribute
   - FEATURE: shell::command: use_sudo Attribute to launch sudo on executing external command
   - FEATURE: shell::command: file globbing enabled with capture Command
   - UPDATE: moved attributes_default() from brik_use_properties to brik_properties when
     there is no need to use $self. It allows instanciated Attributes inheritage to work.
   - UPDATE: shell::command: do not print STDERR when using capture Command when there is no
     STDERR string captured.
   - new: shell::command: execute Command to use capture_mode Attribute to launch either
     capture or system Command
   - bugfix: core::context: save_state to use Metabrik brik_attributes Command to correctly
     retrieve all Brik Attributes even those inherited
   - bugfix: core::shell: display error on chdir() failure
   - bugfix: core::shell: escapes " character when executing a multiline Perl/Metabrik Code
             my $test = 'root'
             for (0..1) {
                'run shell::command system "ls /$test"'
   - bugfix: Metabrik: error checking within new_from_brik_init Command
   - bugfix: Metabrik: logging correctly on class calls to _log_*()


- bugfixes and new Briks

   AFFECT: network::arp

   - network::arp scan Command now returns a hashref with results sorted
     with keys named by_mac, by_ipv4 and by_ipv6

   AFFECT: network::rsync

   - network::rsync renamed to client::rsync

   AFFECT: crypto::x509

   - Argument order changed for ca_sign_csr and cert_verify Commands

   AFFECT: file::csv

   - removed get_col_by_name and get_col_by_number obsolete Commands

IP location with Metabrik: different ways of doing it

We just released a new Brik: lookup::iplocation. Thanks to this Brik, you will be able to easily geo-locate any IP address. Well, it has to be in MaxMind’s database for that to work, of course. In this post, we propose to explore different ways of using a Metabrik Brik with this lookup::iplocation example.

Using The Metabrik Shell

Of course, the simplest way of doing is using The Metabrik Shell (which is just a simple Brik called core::shell). A screenshot is what best demonstrates the power of Metabrik.

The first call to ‘use lookup::location‘ fails because we currently lack one Perl module: Geo::IP. Metabrik tells you so and you can install it by using the perl::module Brik.

Then, you want to update the Geo::IP database. If you don’t, you will not have the required data to perform the lookup. You have to call ‘run lookup::location update‘.

Finally, you can geo-locate the IP address by calling ‘run lookup::location from_ip <IP>‘.


Using a Metabrik script

Before being able to execute Metabrik scripts (.meta files), you have to initiate the .metabrik_script_rc file. Just edit it using vi, for instance, and put in the lines below:

vi ~/.metabrik_script_rc
set core::shell echo 0
my $home = $ENV{HOME}
my $repository = "$HOME/metabrik/repository/lib"
push @INC, $repository
run core::context update_available

Now you are ready to write your very own Metabrik script. Create one using vi and put in this content:

vi iplocation-script.meta
use lookup::iplocation
if ($ERR) { exit }

run lookup::iplocation update
if ($ERR) { exit }

run lookup::iplocation from_ip
if ($ERR) { exit }

use Data::Dumper
print Dumper($RUN)

You can mix Metabrik Commands with Perl code. Wonderful, isn’t it? Ah, maybe you want to execute it:

metabrik --script iplocation-script.meta

The third way: use a Perl script

And finally, this is Perl. You can use a Brik as a plain Perl module if you want to. Try that:

use strict;
use warnings;

use Data::Dumper;
use Metabrik::Core::Context;
use Metabrik::Lookup::Iplocation;

my $context = Metabrik::Core::Context->new;
$context->brik_init or die("[FATAL] context init failed\n");

my $li = Metabrik::Lookup::Iplocation->new_from_brik_init($context)
   or exit(1);

   or exit(2);

my $info = $li->from_ip("")
   or exit(3);

print Dumper($info);


You see you can play with Metabrik in different ways. Even though Metabrik has been thought as a shell from the beginning, you are not bound to it in any way. And you, what is your favorite way of using Briks?

Metabrik Core And Repository 1.08 Released

A new version of Metabrik Core and Repository is available. Update using Mercurial or follow the installation procedure.



1.08 Thu Mar 19 06:48:34 CET 2015
 - FEATURE: core::shell: run executable commands found in PATH through system Command
 - UPDATE: shell::command: now use IPC::Run3 to capture shell commands output
 - update: shell::command: system Command now returns $? on success
 - update: Metabrik: display every missing items from brik_require_*_check() before erroring
 - update: new dependance on IPC::Run3

1.07 Sun Mar 8 17:52:37 CET 2015
 - bugfix: shell::history: correctly writes $* variables when calling history Commands
 - new: shell::rc: create an alias to make it easy to switch to root from Shell
 - new: brik::search and perl::module Briks integrated in main distribution
 - remove: no more metabrik-cpanm, enforce use of standard cpanm


- bugfixes and new Briks

 AFFECT: http::proxy, iana::countrycode, network::arpdiscover

 - http::proxy Brik renamed to proxy::http
 - iana::countrycode Brik renamed lookup::countrycode
 - network::arpdiscover removed: merged with network::arp

 AFFECT: client::www

 - post Command returns an HASHREF i/o of WWW::Mechanize object

 AFFECT: network::portscan

 - synscan Command renamed to tcp_syn


Exploiting ElasticSearch RCE For CVE-2015-1427

I told you so: it is a work for Metabrik. While the main target for Metabrik is not to write exploits (you have Metasploit for that), you can still write Briks within the Audit Category. Based on the exploit provided XiphosResearch, we wrote a Command to verify if an ElasticSearch target is vulnerable, and another Command to exploit the issue to execute commands on the target.

Loading the Brik

As always, if a Brik is not loaded yet, you have to do it. Then, the first thing to do is to ask for help, or how to use the Brik, which Attributes can be set and which Commands can be run.


Using a check or an exploit Command

To test a target, you can either use the check_cve_2015_1427_rce  Command or use the exploit_cve_2015_1427_rce Command with an innocuous command. We recommand to use the check one, which is actually made to be innocuous.

To use the check or exploit one, you just have to use the run Metabrik Command with the name of a Command, and add Arguments to it. Some Arguments can be set globally for the Brik: here you may use the set Metabrik Command to set uri Attribute.

Note: And don’t forget to use the <Tab> key to perform completion on every Command, and use the <Up> key to recall previous ones.



Our target is vulnerable. Too bad, but it is an exploit lab, it exists for that.

Exploiting the issue to execute commands

But well, if you are here, it is probably because you want to exploit a Remote Command Execution within ElasticSearch. With Metabrik, it is as easy as:


We even added some “post intrusion” work, like downloading a file.

A key feature of Metabrik

Yes, one of the key feature of Metabrik is to assemble Briks together to execute a complete scenario. We have shown how to use Briks, especially the one on exploiting a vulnerability within a product and, after this exploitation process, we have shown that we can use a special $RUN variable with another Brik.


  • run Command sets the global variable $RUN
  • set Command sets the global variable $SET
  • get Command sets the global variable $GET

Thanks to the $RUN variable, you can chain the use of Briks. In fact, Briks within Metabrik may also be written with other Briks. For instance, audit::elasticsearch Brik relies on:

Here, we have chained a few Briks, and one of them allowed to save a remote file to a local file (file::text). You have more global variables, but that’s enough for today. Follow us on twitter @Metabrik.



Why Writing A TCP SYN Scanner In Perl Can Be Efficient

Everyone wants to write (or already did) its own TCP SYN scanner. Why? Because it is a fun exercise which will teach you a lot of things, like what raw sockets are and, more importantly, how to build packets.

Perl for the task

Some may argue that using Perl would be inefficient for such a task. Plain false. You can use Perl/XS to write part(s) of a program that require performance. That’s what we did with Net::Write::Fast: a module able to send packets over the network at speed of wires.

Design goals behind Net::Write::Fast

A few design goals were followed when writing this module:

  • TCP SYN scanner working over IPv4 and IPv6
  • Portable on at least Linux, FreeBSD and MacOS X
  • Perform a scan in a predictable way: we know when it will finish
  • Send packets over a transport layer socket

The transport layer socket

Usually, people tend to write packets using raw sockets directly from the network layer, requiring them to also craft the IP layer. And that’s a total waste of code and time. You can send your TCP or UDP packets without having to craft the IP header; no need to use the include IP header socket option.

To make it short, opening a raw socket within Net::Write::Fast was as simple as:

int fd;
fd = socket(v6 ? AF_INET6 : AF_INET, SOCK_RAW, IPPROTO_TCP);

The TCP header

So you did open a raw socket at the transport layer, now you just need to write the TCP layer. You don’t have to care about IP layer at all, and the differences of endianess from OS to OS for some of its fields (header length, for instance). Well, you still have to write the pseudo header so you can compute TCP checksum, but that’s another story.

Glueing C code to Perl

Full Perl/XS code can be found here. It is exactly like standard C code. But for the C function to be callable from Perl, we need to add a binding. It is a little bit tricky when you want to do it right. The prototype of the function is:

l4_send_tcp_syn_multi(src, dst, ports, pps, n, v6, warnings = NO_INIT)
      char *src
      SV   *dst
      SV   *ports
      int   pps
      int   n
      int   v6
      int   warnings

A forked sender process and a listener

Now everything is in place to send packets at the speed of wires. We still have to think about listening for answers and parsing them. Here, no need for Perl/XS for the listening task: you have Net::Frame::Dump module which is a pure Perl one.

Now, we need proof

Here is a screenshot from a TCP SYN scan of the TOP 1000 most common ports on the local network (256 hosts). We configure the scanner to send 2 times a probe, at a rate of 100 000 packets per second. If you do your math, you will see it should take 5 seconds and consume around 8 MB of bandwidth per second. One packet takes 20 bytes for IP header followed by 60 bytes for the TCP header plus its full options.

Of course, you can send at a greater speed, or change the number of tries to tweak your scanning process. Screenshot:


So, it took a little bit more than the 5 seconds estimation, but you have 2 or 3 seconds needed by the program to load and compile, and to wait at the end of sending process to gather remaining replies.

But what about IP source address spoofing?

Some may argue that we cannot spoof source IP address using transport layer sending. And that’s just wrong. You can do it by just adding an alias to your network interface, and using that as the source address for sending packets. It will be used right away by your transport layer raw socket.

Tweaking your TCP/IP stack

Under GNU/Linux, don’t forget to increase your send buffers. Or you will have the following error:

WARNING: Net::Write::Fast: sendto: ENOBUFS, sleeping for 1 second

# Increase send buffer to 100 MB
sysctl -w net.core.wmem_max=109051904
sysctl -w net.core.wmem_default=109051904


Writing a TCP SYN scanner in Perl is reliable, fast, and predictable. And of course, there is now a Brik for that within Metabrik, it is named network::portscan. Happy scanning, and don’t forget to follow us on Twitter: @metabrik.

Metabrik Core And Repository 1.06 Released

A new version of Metabrik Core and Repository is available. Update using Mercurial or follow the installation procedure.



1.06 Fri Feb 27 07:17:59 CET 2015
 - bugfix: shell::command: go trough PATH to find a cmd to run (like less PAGER)
 - bugfix: core::shell: on SIGINT handling, now allows to break multiline and run Commands
 - bugfix: core::shell: allow user to get out of multiline mode by hitting Ctrl+C
 - bugfix: core::shell: better management of Metabrik Commands in multiline mode
 => you can put Metabrik Commands within single quotes anywhere
 - update: shell::script: load Command can take an optional file parameter
 - update: metabrik, shell::rc/script, core::shell: easier handling of rc file loading
 - new: metabrik: --script-rc argument to load a specific rc file for scripts


- bugfixes and new Briks
- see UPDATING file for changes since your last update


Playing With Ten Million Passwords And Logins

A researcher has released a database of ten million logins and passwords. We will show how we can play with it with Metabrik.

Fetching the data

You have to find the data by yourself, go check it from Mark Burnett post: Today I Am Releasing Ten Million Passwords.

Loading Metabrik Briks

Note: You will need Metabrik 1.06+ to run this example. Check it out via the Mercurial repository.

Note: When you are using the Shell, do not forget that you can use key to auto-complete Commands and Variables.

The file is a zip archive, so you will use the file::compress Brik. The uncompressed file being a text file, you will need to parse it in a low-level way: file::read Brik will be used.

Meta:~> use file::compress
[*] core::shell: use: Brik [file::compress] success
Meta:~> use file::read
[*] core::shell: use: Brik [file::read] success


Let’s read the data and do some statistics

Uncompress the archive

So, we have first to uncompress the archive. By default, uncompressed data will be put in the Brik home directory: ~/metabrik/file-compress.

Meta:~> run file::compress unzip ~/Downloads/
Meta:~> l /home/gomor/metabrik/file-compress/10-million-combos.txt
  "-rw-r--r-- 1 gomor gomor 194130539 Feb  9 12:59 /home/gomor/metabrik/file-compress/10-million-combos.txt",


Read and parse

# No need to print anything by default during processing
set core::shell echo 0

# Configure the file::read Brik
set file::read input ~/metabrik/file-compress/10-million-combos.txt
set file::read encoding ascii
set file::read strip_crlf 1

# Start using it
run file::read open

Now, file is open, and we can start reading and parsing it. What we want here is to perform some statistics, like getting the top used passwords. To that end, we will mix Metabrik Commands with Perl code. The Shell allows you to do that in a simplified way that going to write a Perl script or program.

my $stats = {}   # Declare a variable to store results

# Get access to Perl object, this is a performance hack for power users.
run file::read brik_self
my $read = $RUN

my $count = 0
# Alternatively (but slower), we could have replaced the following line by:
# while ('run file::read read_line') {
while (my $line = $read->read_line) {
   my ($l, $p) = split(/\t/, $line);
   if (! ($count % 10_000)) {  # Print count every 10_000 lines
      print "$count\n";
   last if $read->eof;

# Get top 20 passwords
my $top = 20;
for my $k (reverse sort { $stats->{$a} <=> $stats->{$b} } keys %$stats) {
   print "$k => ".$stats->{$k}."\n";
   last if $top == 0;


And top 20 is…

  • 123456 => 55893
  • password => 19580
  • 12345678 => 13582
  • qwerty => 13137
  • 123456789 => 11696
  • 12345 => 10938
  • 1234 => 6432
  • 111111 => 5682
  • 1234567 => 4796
  • dragon => 3927
  • 123123 => 3845
  • baseball => 3565
  • abc123 => 3511
  • football => 3494
  • monkey => 3246
  • letmein => 3118
  • 696969 => 3050
  • shadow => 2956
  • master => 2931
  • 666666 => 2905

And what about a Script?

Yes, you can write a Metabrik Script to perform this task. Just copy and past this data into a file named top20-password.brik for instance:

use file::read
set file::read input ./10-million-combos.txt
set file::read encoding ascii
set file::read strip_crlf 1
run file::read open

set core::log level 0

run file::read brik_self
my $read = $RUN

my $stats = {}
my $count = 0
while (my $line = $read->read_line) {
   my ($l, $p) = split(/\t/, $line);
   if (! ($count % 10_000)) {
      print "$count\n";
   last if $read->eof;

my $top = 20;
for my $k (reverse sort { $stats->{$a} <=> $stats->{$b} } keys %$stats) {
   print "$k => ".$stats->{$k}."\n";
   last if $top == 0;


And run the script:

$ metabrik --script top20-password.brik


That’s all for today. Follow @metabrik on twitter.

Metabrik Example: Searching CVE Database

In this Metabrik Example, we will show you how to search entries in the CVE database and how to save it to a CSV file.

Loading required data

First thing to do is to load the required Briks, and ask for help on their usage. Don’t forget you can use so you have completion and you don’t have to type every single caracter.

Meta:~/metabrik> use database::nvd
Meta:~/metabrik> help database::nvd


Meta:~/metabrik> run database::nvd update recent
Meta:~/metabrik> run database::nvd load recent


Now, you have all XML data from the recent CVE feed loaded into memory. You can start your searches. As another effect, the loading result is saved as XML data into $RUN variable. You can verify that by just typing $RUN at the prompt.

Let’s search the database

Meta:~/metabrik> run database::nvd search_all
Meta:~/metabrik> my $all = $RUN
Meta:~/metabrik> my $a = []
Meta:~/metabrik> for my $e (@$all) {
   push @$a, { cve => $e->{cve_id},
      published => $e->{published_datetime}
Meta:~/metabrik> $a
    cve => "CVE-2014-9365",
    published => "2014-12-12T06:59:07.063-05:00",
    cve => "CVE-2014-9385",
    published => "2014-12-15T13:59:27.723-05:00",
    cve => "CVE-2014-7268",
    published => "2014-12-19T06:59:04.447-05:00",


How to save the result in CSV format

We have found what we were searching for: the list of recent CVE IDs and their published date and time. Now, let’s save that to a CSV file. But how? Let’s call for help. You can have help by just typing help followed by the name of a Brik. Here, the one we want is file::csv.

Meta:~/metabrik> use file::csv
Meta:~/metabrik> help file::csv


Don’t fear the output, default values are usually ok for standard usage. set Commands are used to modify the default behaviour, we won’t touch anything in this example. Try the write Command as follows, and verify file was correctly written:

Meta:~/metabrik> run file::csv write $a /tmp/cve.csv
Meta:~/metabrik> run shell::command capture cat /tmp/cve.csv


You can download Metabrik from there: Downloads.

The source code for database::nvd Brik can be found here: Database/

That’s all for today. Follow @metabrik on twitter.

Getting Metabrik Up And Running

Metabrik is available via a mercurial repository or from a tarball. Follow this installation guide to get Metabrik running. The following of this post will guide you through your first steps with the platform.

Ready to serve

Everything should be fine from now on, you can run The Metabrik Shell. A default .rc file will be created for you:

A first run example

Say you want to know which wifi AP you have in your neighborhood, what you need is a wifi Brik allowing you to scan it. One of the first Brik you will need is the one to search for a suitable Brik:

Now, search for a wifi Brik:

Search result is displayed, along with the number of matched Briks (2 in our example). Every Brik has a set of tags that allows you to search for them.

Ready to scan:

This run Command returned some results, that’s great. We did different things. First, we asked to load and use a Brik with the use Command, we then set some Attribute with the set Command, and finally called the scan Command with the run one.

At the end of the run, result is returned, but the most interesting thing is unnoticed (yet), a special Perl variable has been set: $RUN. Try typing $RUN at the prompt, and you will understand the power of Metabrik.

That’s all for today. Follow @metabrik on twitter.

How Long Does It Take To Scan The Internet

At security researchers, we sometimes do massive Internet scanning. We do it like many others are doing it: SHODAN, ERIPP, governments?, to name a few. To correctly plan a scanning session, we need to know how long it will take regarding the task we want to accomplish. We propose to do the math in this post.

Our first post regarding Internet scanning can be found on our previous blog article on how to not get caught while doing that.

Calculating packet size

So, you want to scan a portion of the Internet, one of your A-class (or /8 in CIDR notation), or maybe the full Internet? You have to decide what information you want to extract. Usually it boils down to just identifying open TCP ports. For that, you perform a classic half-open TCP SYN scan.

The size of a TCP SYN varies according to the number of TCP options you add. Standard size of a TCP header is 20 bytes. The minimum TCP option you should put is the MSS one, which takes 4 bytes. Standard IPv4 header is 20 bytes. Depending on your physical network connection, you have to add the layer 2 (usually Ethernet, which takes 14 bytes). Now, for the math:

% echo "14 + 20 + 24" |bc

58 bytes for a classic SYN. By default, we prefere to count 20 bytes (instead of only 4) of TCP options (Window scale, selective acknowledgment, timestamps, nop, and some others). Thus, our TCP header size is 16 bytes more:

% echo "58 + 16" |bc

Our scanning task will be to send 1 TCP SYN of 58 bytes to 1/1000 of the Internet (around 3.7 million IPv4 addresses) without retry.

Is it possible to do this from an ADSL connection?

You may want to do that from your ADSL connection (bad idea). The good news is that’s feasible. By doing the math, with the following variables:

  • Bandwidth: 100k (upload)
  • Target count: 3.7 million
  • Try: 1 (no resend)
  • Port count: 1 port
  • Packet size: 58 bytes

By using this calculation method, you see that scanning 1/1000 of the Internet from an ADSL connection takes approximately 35 minutes, at a rate of 1724 packets per second. Not too bad. Now, to scan the full Internet for one port: around 25 days. Less than a month. Everyone can do it (and receive abuse complaints).

Happy scanning.