A researcher has released a database of ten million logins and passwords. We will show how we can play with it with Metabrik.
Fetching the data
You have to find the data by yourself, go check it from Mark Burnett post: Today I Am Releasing Ten Million Passwords.
Loading Metabrik Briks
Note: You will need Metabrik 1.06+ to run this example. Check it out via the Mercurial repository.
Note: When you are using the Shell, do not forget that you can use key to auto-complete Commands and Variables.
The file is a zip archive, so you will use the file::compress Brik. The uncompressed file being a text file, you will need to parse it in a low-level way: file::read Brik will be used.
Meta:~> use file::compress
[*] core::shell: use: Brik [file::compress] success
Meta:~> use file::read
[*] core::shell: use: Brik [file::read] success
Let’s read the data and do some statistics
Uncompress the archive
So, we have first to uncompress the archive. By default, uncompressed data will be put in the Brik home directory: ~/metabrik/file-compress.
Meta:~> run file::compress unzip ~/Downloads/10-million-combos.zip
"/home/gomor/metabrik/file-compress"
Meta:~> l /home/gomor/metabrik/file-compress/10-million-combos.txt
[
"-rw-r--r-- 1 gomor gomor 194130539 Feb 9 12:59 /home/gomor/metabrik/file-compress/10-million-combos.txt",
]
Read and parse
# No need to print anything by default during processing
set core::shell echo 0
# Configure the file::read Brik
set file::read input ~/metabrik/file-compress/10-million-combos.txt
set file::read encoding ascii
set file::read strip_crlf 1
# Start using it
run file::read open
Now, file is open, and we can start reading and parsing it. What we want here is to perform some statistics, like getting the top used passwords. To that end, we will mix Metabrik Commands with Perl code. The Shell allows you to do that in a simplified way that going to write a Perl script or program.
my $stats = {} # Declare a variable to store results
# Get access to Perl object, this is a performance hack for power users.
run file::read brik_self
my $read = $RUN
my $count = 0
# Alternatively (but slower), we could have replaced the following line by:
# while ('run file::read read_line') {
while (my $line = $read->read_line) {
my ($l, $p) = split(/\t/, $line);
$stats->{$p}++;
$count++;
if (! ($count % 10_000)) { # Print count every 10_000 lines
print "$count\n";
}
last if $read->eof;
}
# Get top 20 passwords
my $top = 20;
for my $k (reverse sort { $stats->{$a} <=> $stats->{$b} } keys %$stats) {
print "$k => ".$stats->{$k}."\n";
$top--;
last if $top == 0;
}
And top 20 is…
- 123456 => 55893
- password => 19580
- 12345678 => 13582
- qwerty => 13137
- 123456789 => 11696
- 12345 => 10938
- 1234 => 6432
- 111111 => 5682
- 1234567 => 4796
- dragon => 3927
- 123123 => 3845
- baseball => 3565
- abc123 => 3511
- football => 3494
- monkey => 3246
- letmein => 3118
- 696969 => 3050
- shadow => 2956
- master => 2931
- 666666 => 2905
And what about a Script?
Yes, you can write a Metabrik Script to perform this task. Just copy and past this data into a file named top20-password.brik for instance:
use file::read
set file::read input ./10-million-combos.txt
set file::read encoding ascii
set file::read strip_crlf 1
run file::read open
set core::log level 0
run file::read brik_self
my $read = $RUN
my $stats = {}
my $count = 0
while (my $line = $read->read_line) {
my ($l, $p) = split(/\t/, $line);
$stats->{$p}++;
$count++;
if (! ($count % 10_000)) {
print "$count\n";
}
last if $read->eof;
}
my $top = 20;
for my $k (reverse sort { $stats->{$a} <=> $stats->{$b} } keys %$stats) {
print "$k => ".$stats->{$k}."\n";
$top--;
last if $top == 0;
}
And run the script:
$ metabrik --script top20-password.brik
That’s all for today. Follow @metabrik on twitter.