Redis provides lightweight, scalable persistent data structures

I’ve been having quite a bit of fun with Redis, a lightweight and simple data structure server. It’s easy to install locally, but you can also get a free server from redislabs. Services such as Heroku can spin up Redis instances and use them with your Heroku-deployed Mojo applications.

In “Lightweight Persistence”, I specifically avoided database servers and stuck to serialization formats. A chapter wouldn’t be enough to cover the Perl DBI and SQL, which meant I didn’t include SQLite either.

Redis is different. It’s a data structure server. It’s storing your lists, hashes, sets, and some other fancy structures without caring about the relations; sharing them between processes; and updating slave servers. For instance, If I were writing a web app, I might cache session data, share data between different processes in real time, and other more sophisticated things (see, for instance, What is Redis and what do I use it for?).

For this Item, I’ll jump into it and leave the details to the redis documentation.

The Redis Perl module gives me the access to the server. By default it looks for a local server on port 6379, but I can specify various ways to connect to a remote server (or set of servers), set timeout values, and many other options. It’s an amazingly flexible system:

use Redis;

my $redis = Redis->new;

Besides the Perl programs I show here, you can interact directly with the Redis server through the redis-cli command-line program in the Redis distribution.

Strings

Redis at its heart is a key-value store. The simplest store is what Perlers think of as a hash, but to Redis it’s the string data type since the value is a simple string. I’ll show other value types too.

The set method assigns a value to a key:

use Redis;

my $redis = Redis->new;

my %hash = qw(
	Buster Bean
	Mimi   Bean
	Cat    Hat
	);

foreach my $key ( keys %hash ) {
	$redis->set( $key => $hash{$key} );
	}

Although the Redis module documents some commands, the module can handle most of the Redis 2 command set. If I don’t see the method in the Perl documentation, I use the Redis command as the method name and pass it the right arguments to be handled by AUTOLOAD, which handles most of the commands.

This key-value store has some fancy features. For instance, I can set an expiration interval for a key. After the specified number of seconds elapses, the key disappears. The expire method is not documented in the Perl module, but I follow the protocol and it works:

# redis_expires.pl
use v5.10;
use Redis;

my $redis = Redis->new;

my %hash = qw(
	Buster Bean
	Mimi   Bean
	Cat    Hat
	);

foreach my $key ( keys %hash ) {
	$redis->set( $key => $hash{$key} );
	}
	
show_keys( "At start" );

$redis->expire( "Cat",   5 );
$redis->expire( "Mimi", 10 );

show_keys( "Right after expires" );

sleep 6;
show_keys( "After 6 seconds" );

sleep 5;
show_keys( "After 11 seconds" );

sub show_keys {
	my( $label ) = @_;
	
	my @keys = $redis->keys( '*' );
	say "$label: keys are [@keys]";
	}

Notice that I use a glob pattern in the keys method to return all matching keys (unlike the Perl built-in). After adding the hash, I set some expire. I sleep a bit then check the keys again. As time elapses, keys disappear:

% perl redis_expires.pl
At start: keys are [Cat Buster Mimi]
Right after expires: keys are [Cat Buster Mimi]
After 6 seconds: keys are [Buster Mimi]
After 11 seconds: keys are [Buster]

Lists

The Redis value can be a list, which is like storing an anonymous array as a Perl hash value (although Redis doesn’t do multidimensional data structures without tricks). Instead of set, I use rpush (like Perl’s push) or lpush (like Perl’s unshift).

# redis_list.pl
use Redis;

my $redis = Redis->new;

my $list_name = 'Cats';

$redis->rpush( $list_name => 'Mimi' );
$redis->rpush( $list_name => 'Buster' );
$redis->rpush( $list_name => 'Ginger' );

my @cats = $redis->lrange( $list_name, 0, -1 );

say "Cats are @cats";

When I run this program, I store the names of the cats in the list named Cats, then retrieve and print them.

% perl redis_list.pl
Cats are Mimi Buster Ginger

If I run my program again, I double the elements because the list retains the previous values and re-stores the ones I add to it:

% perl redis_list.pl
Cats are Mimi Buster Ginger Mimi Buster Ginger

If I want list elements to appear exactly one time, I can use sets, which I show later.

Once I use a key in a particular way, like I’ve used Cats to store a list value, I have to keep using it in that way. I can’t change it to some other data type like I could assign a new reference type into a Perl variable. If I want to change the value, I first have to delete that key:

$redis->del( $list_key );

Hashes

If I want multiple key-value stores in the same server, I can use a different set of commands to specify a hash name (and it can’t be the name I just used with the list):

my $hash_name  = 'Cats';
my $first_name = 'Buster';
my $last_name  = 'Bean';

$redis->hset( $hash_name, $first_name => $last_name );

my $value = $redis->hget( $hash_name, $first_name );

say "Name is $value";

I can all of the hash operations I expect from Perl hashes, but these aren’t stored in my program memory and other processes can share the hash in real time.

I can also fake this with some old Perl 4 tricks (see the $; variable in perlvar). I can make the Redis key a combination of values that I join with some special character:

# as a string value
$redis->set( "$hash_name.$key2.$key3" => $value );

$redis->hset( $hash_name, "$key2.$key3" => $last_name );

I still have to pack and unpack the joined keys myself, but that’s not such a big deal.

Sets

Sets are one of the more interesting data structures since Redis can perform set math:

my $set_a = 'Set A';
my $set_b = 'Set B';

my @a = qw( Buster Mimi Ginger );
my @b = qw( Nikki Addy Ginger );

$redis->sadd( $set_a, @a );
$redis->sadd( $set_b, @b );

my @diff = $redis->sdiff( $set_a, $set_b );
say "A\\B is @diff";

my @intersection = $redis->sinter( $set_a, $set_b );
say "A∩B is @intersection";

Redis has sorted sets. It’s almost like a hash since each set member has an associated score. Although it seems like a hash, I can still do set math on it. Additionally, I can extract set members in order of their score:

# redis_sorted_set.pl
use v5.10;
use Redis;

my $redis = Redis->new(
	server   => 'pub-redis-11592.us-east-1-4.4.ec2.garantiadata.com:11592',
	password => 'top1dog',
	);
die "Could not connect to Redis" unless defined $redis;

my $set_name = 'Pets';

my %members = 
	map { $_ => int( rand(100 ) ) }
	qw( Buster Mimi Ginger Addy Nicky Poppy );

say "Hash is: ", Dumper( \%members ); use Data::Dumper;

$redis->del( $set_name );
foreach my $key ( keys %members ) {
	$redis->zadd( $set_name, $members{$key}, $key );
	}
	
my @sorted = $redis->zrevrangebyscore( 
	$set_name,
	'+inf',   # starting score
	0,        # ending score
	'LIMIT',  # LIMIT the results
	0,        # start with element 0
	3         # go for three elements
	);

say "Top three elements are [@sorted]";

I first dump the %members hash so I know what’s going on, then show the top three set members I extracted:

Hash is: $VAR1 = {
          'Ginger' => 64,
          'Poppy' => 7,
          'Nicky' => 25,
          'Mimi' => 91,
          'Addy' => 29,
          'Buster' => 62
        };

Top three elements are [Mimi Ginger Buster]

There are also methods to increment or change the scores, perhaps from different programs all running at the same time. I could do this with a hash, but then I would have to sort the hash myself.

Even more you can do

Redis can notify me when things happen or change. I can subscribe to topics and run a Perl callback automatically. Each of the data structures have many more operations than I’ve shown and I haven’t bothered to go over any fancy use cases. You can investigate those on your own.