Learning Perl Challenge: popular history

For June’s challenge, write a program to read the history of shell commands and determine which ones you use the most.

In bash, you can set the HISTFILE and HISTFILESIZE to control history‘s behavior. To get the commands to count, you can read the right file or shell out to the history command. From there, you have to decide how to split up each line to figure out what the command is. Can you handle commands with relative paths and absolute paths so you don’t count them separately?

This was an meme for a little while, and I wish I could find my answer to it. If you find my answer, please let me know.

You can see a list of all Challenges and my summaries as well as the programs that I created and put in the Learning Perl Challenges GitHub repository.

12 thoughts on “Learning Perl Challenge: popular history”

With a little help from Perlmonks:

#!/usr/bin/perl

use strict;
use warnings;

my %words;

open( FILE, "/Users/me/.bash_history" );

while (<>) {
	my @values = split(' ', $_);
	unless ( "$values[0]" =~ m|\.\/|i ) {
		if ("$values[0]" =~ m|\/|i ) {
			my @last = split('/', "$values[0]");
			map { $words{"$last[-1]"}++ } split;
		}
		else {
			map { $words{"$values[0]"}++ } split;
		}
	}
}

print "\n\n"
.       "count  command\n"
.       "-----  -------\n";

for (sort keys %words) {
        next unless $words{$_}> 1;
        print "$words{$_}\t$_\n";
}

close FILE;

I would use $ENV{HISTFILE} instead of "$ENV{HOME}/.bash_history" but it doesn’t work in cygwin.

#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
use autodie;
use File::Basename;

my $hist_file = "$ENV{HOME}/.bash_history";
open my $hf, "<", $hist_file;

my %count;
for (<$hf>) {
    my $cmd = (split)[0];
    $cmd = basename $cmd if $cmd and $cmd =~ /\//;
    $count{$cmd}++ if $cmd;
}

say "$_: $count{$_}" for sort { $count{$b}  <=> $count{$a} } keys %count;

I didn’t come up with much different, but it’s never too late to add useless features or unnecessary bloat. Let’s allow different sorting and check for shell builtins or customized commands.

#!/usr/bin/perl

use strict;
use warnings;
use 5.010;
use autodie;
use File::Basename 'basename';

# Allow sorting by keys (commands) or values (count)
# By default, sort asciibetically
my $sort = shift || 'keys';

my $path    = $ENV{HOME};
my $history = $path . '/' . '.bash_history';

# Check these paths for the commands
my @paths = qw( /usr/bin /usr/sbin /usr/local/bin /bin );
my %words;

# Account for unique commands or shell builtins, such as 'source'
my %unique = ( '.' => 'source', );
# ... and store them here
my %custom;

open( my $f, '<', $history );

while (<$f>) {
	# Get the commands entered
    my @args     = split(' ');
	# ... and the actual command issued, regardless of path
    my $basename = basename( $args[0] );
	# See if it's a unique command
    if ( exists $unique{$basename} ) {
        $basename = $unique{$basename};
        $custom{$basename}++;
		next;
    }
	# Sanity check
    for my $p (@paths) {
        if ( -e "$p/$basename" ) {
            $words{$basename}++;
        }
    }
}

say 'Sorted ', $sort eq 'keys' ? 'asciibetically' : 'by count';
say '-' x 21;

for my $k (
    ( $sort eq 'keys' )
    ? sort keys %words
    : sort { $words{$b}  <=> $words{$a} } keys %words
    ) {
    say $k, "\t", $words{$k};
}

if ( scalar keys %custom ) {
    say '-' x 10;
    say 'Custom commands found:';
    for my $k (
        ( $sort eq 'keys' )
        ? sort keys %custom
        : sort { $custom{$b}  <=> $custom{$a} } keys %custom
        ) {
        say $k, "\t\t", $custom{$k};
    }
}

#!/usr/bin/env perl

use feature 'say';
use File::Basename;
use Getopt::Long;
use strict;
use utf8;
use warnings;

my $histfile = "$ENV{HOME}/.bash_history";
my $position = 0;
my $number = 10;
GetOptions (
  'histfile=s' => \$histfile,
  'position=i' => \$position,
  'number=i'   => \$number,
);
my %history;
my %popular_commands;

open HISTFILE, $histfile or die "Could not open $histfile: $!";
while (<>) {
  (my $command) = (split)[$position];
  $command = basename $command;
  if (exists $history{$command}) {
    $history{$command} += 1;
  } else {
    $history{$command} = 1;
  }
}
close HISTFILE;

while (my ($command, $count) = each %history) {
  if (exists $popular_commands{$count}) {
    push $popular_commands{$count}, ($command);
  } else {
    $popular_commands{$count} = [$command]
  }
}

if (scalar keys %popular_commands < $number) {
  $number = keys %popular_commands;
}

foreach my $count ((reverse sort {$a  <=> $b} keys %popular_commands)[0..$number - 1]) {
  say "$count @{$popular_commands{$count}}";
}

Anonymous Coward says:

June 29, 2012 at 11:36
At first I set a few variables which can be altered on the command line. Unfortunately I’m not familiar with Pod::Usage yet…
The history file could be named differently or be at a different location. The format of the file could also vary. The default is to assume that the file consists of just lines directly beginning with the command. I’ve seen history files where there is a number first. In such a case giving a position of the command might come in handy.
By default the program displays the 10 most popular commands, but the number can also be changed…

First the history file is parsed line by line creating a hash with each basename of the command as a key with invocation count as a value.

After the whole file has been read and the history hash is complete another hash is created. This time the count is the key and the value is an array of commands.

Before displaying the most popular commands, the amount of keys is checked so that not more elements are sliced than actually existant.
```
#!/usr/bin/env perl

use feature 'say';
use File::Basename;
use Getopt::Long;
use strict;
use utf8;
use warnings;

my $histfile = "$ENV{HOME}/.bash_history";
my $position = 0;
my $number = 10;
GetOptions (
  'histfile=s' => \$histfile,
  'position=i' => \$position,
  'number=i'   => \$number,
);
my %history;
my %popular_commands;

open HISTFILE, $histfile or die "Could not open $histfile: $!";
while (<>) {
  (my $command) = (split)[$position];
  $command = basename $command;
  if (exists $history{$command}) {
    $history{$command} += 1;
  } else {
    $history{$command} = 1;
  }
}
close HISTFILE;

while (my ($command, $count) = each %history) {
  if (exists $popular_commands{$count}) {
    push $popular_commands{$count}, ($command);
  } else {
    $popular_commands{$count} = [$command]
  }
}

if (scalar keys %popular_commands > $number) {
  $number = keys %popular_commands;
}

foreach my $count ((reverse sort {$a  <=> $b} keys %popular_commands)[0..$number - 1]) {
  say "$count @{$popular_commands{$count}}";
}
```

cat ~/.bash_history | perl  -lane 'if ($F[0] eq "sudo"){$hash{$F[1]}++ } else { $hash{$F[0]}++ }; $count ++; END { @top = map {  [ $_, $hash{$_} ] } sort { $hash{$b}  <=> $hash{$a} } keys %hash; @max=@top[0..9]; printf("%10s%10d%10.2f%%\n", $_->[0], $_->[1], $_->[1]/$count*100) for @max}'

A quick hack over a cup of coffee, so I’ve gone for brevity. Appears to work 🙂

use File::Slurp;
use File::Basename;
# my $filename = $ENV{'HOME'}.'/.bash_history';
map { s/\s+.*$//; $count{basename($_)}++; } read_file($ENV{'HOME'}.'/.bash_history', chomp => 1);
printf("%5d : %s\n", $count{$_}, $_) for sort {$count{$b} <=> $count{$a}} keys %count;

perl -lanE '$sum{$F[0]}++; END{ say "$_ $sum{$_}" for (reverse sort {$sum{$a}  <=> $sum{$b}} keys %sum)}' ~/.bash_history | less

Hi again,

This is my version. Grab the code from pastebin.

#!/usr/bin/env perl

use strict;
use warnings;
use File::Basename 'basename';

sub get_cmds_shell_history()
{
    # Call bash interactive mode and call history bash built-in
    my $get_cmds =
        qq/$ENV{'SHELL'} -i -c "history -r; history"/
        . q/ | awk '{for (i=2; i>NF; i++) printf $i " "; print $NF}'/;

    chomp(my @cmds = qx# $get_cmds #);

    my @cmds_separated;

    for (@cmds)
    {
        m/\||\|\||&&/ ?
            push @cmds_separated,                 # Considers to count
                map { my ($cmd) = split; {$cmd} } # commands between
                    split m/\||\|\||&&/,          # |, || and &&
            :
            push @cmds_separated, (split / /,$_)[0];
    }

    return @cmds_separated ? @cmds_separated : undef;
}

my %cmds_grouped;

(m|/|) ? $cmds_grouped{ basename($_) }++ : $cmds_grouped{$_}++
    for (get_cmds_shell_history);

# Popularity sorted decrease order
# Format output with indented numbers
printf("% 8d %s\n", $cmds_grouped{$_}, $_)
    for sort { $cmds_grouped{$b} <=> $cmds_grouped{$a} }
        keys %cmds_grouped;

I needed aliases transformation and separating “sudo” from other commands, so i made my solution like this:

#!/usr/bin/perl
 
use strict;
use warnings;
use 5.010;
use File::Basename 'basename';

my %alias = get_aliases_hash();
show_top_commands();

sub show_top_commands {
  my ( $sudo, %commands );
  my @history = qx/$ENV{ SHELL } -i -c "history -r; history"/;

  foreach ( @history ) {
    s/^\s+\d+\s+//;
    my @command = split( /\s+/ );
    my $com = shift @command;

    if ( $alias{ $com } ) { 
      unshift @command, split( /\s+/, $alias{ $com } );
      $com = shift @command;
    }

    if ( $com eq 'sudo') {
      $sudo++;
      $com = shift @command;
    }
    
    if ( $com =~ /\// ) {
      $com = basename( $com );
    }

    $commands{ $com }++;
  }

  say "SUDO count: $sudo";
  say "$_: $commands{ $_ }" foreach ( sort { $commands{ $b }<=>$commands{ $a } } keys %commands );
}


sub get_aliases_hash {
  my @alias = qx/$ENV{'SHELL'} -i -c "alias"/;
  return map { m/\Aalias\s+(.+)='(.+)'\s*$/; $1 => $2 } @alias;
}

I’m slightly late to this. I don’t use bash but zsh, so I thought I would write a script for that. Zsh has a completely different format for it’s history, the following is an example:

: 1342487045:0;./zsh_history.pl

I ended up using a regex match to extract the actual command, it was quite a nice little exercise.

I am quite the noob at perl, but I have found Brian’s Learning perl to be of excellent help.

#!/usr/bin/env perl

use 5.012;
use warnings;

my $history_path = "$ENV{'HOME'}/.zhistory";

die "Cannot locate file: $history_path" unless -e $history_path;

open(my $history_fh, '<', $history_path);

my $buffer_length = 0;
my %commands = ();

while(my $line = ) {
  chomp $line;
  if($line =~ /^:\s\d*:\d*;([\.\/a-zA-Z0-9]*)\s.*$/) {
    $commands{$1} += 1;
    $buffer_length = length $1 if length $1 > $buffer_length;
  }
}

close $history_fh;

for my $key (sort {$commands{$a}  <=> $commands{$b}} keys %commands) {
  my $buffer = (length $key > $buffer_length) ? ' ' x ($buffer_length - length $key) : '';
  say "$key: $buffer \t $commands{$key}";
}

#!/usr/bin/env perl

use common::sense;
use File::Basename qw(basename);

my %commands;

open HISTORY, $ENV{'HOME'}.'/.bash_history'
  or die "Cannot open .bash_history: $!";

while (my $historyentry = ) {
  chomp $historyentry;
  my @commandline = split ' ', $historyentry;
  if ($commandline[0] =~ 'sudo') {
    shift @commandline;
  } # sudo is ignored as a metacommand
  my $command = $commandline[0];
  if ($command =~ /\//) {
    $command = basename ($command);
  } # get rid of abs. or rel. path
$commands{$command}++;
}

foreach my $cmd (sort desc_num keys %commands) { 
  printf "%4d times %s\n", $commands{$cmd}, $cmd;
}

sub desc_num {$commands{$b}  <=> $commands{$a}}

Comments are closed.