Recollective Stew

Page 2

Using matplotlib in Perl 6 (part 3)

2017-03-06 00:15, Tags: perl python matplotlib

This is Part 3 in a series. You can start at the Intro here.

So far this Perl 6 wrapper class for matplotlib is going well. With the first graph in the gallery under my belt, I moved on to second example, fill_demo.py. Here's the Python code

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 1, 500)
y = np.sin(4 * np.pi * x) * np.exp(-5 * x)

fig, ax = plt.subplots()

ax.fill(x, y, zorder=10)
ax.grid(True, zorder=5)
plt.show()

Being a rather simple graph, you might think that the Perl code would be fairly straight forward, and you'd be right. The only minor curveball is another numpy function linspace, and that rather hairy looking operation for the y value.

To reiterate what I've said in previous parts, I'm not familiar with numpy at all. Or at least, I wasn't before I started working on these graphs. I inserted a print(x) into the Python code to see what linspace did. In hindsight, the name is obvious; It simply creates a linearly spaced sequence of numbers. Here's a quick demonstration

>>> numpy.linspace(0, 1, 3)
array([ 0. ,  0.5,  1. ])
>>> numpy.linspace(0, 1, 5)
array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ])
>>> numpy.linspace(0, 1, 6)
array([ 0. ,  0.2,  0.4,  0.6,  0.8,  1. ])

In the case of x in the graph, it's a sequence of 500 numbers evenly spaced from 0 to 1. So how would I do this in Perl. It is a monotonic sequence, so maybe the Sequence operator can help us again. So far we've seen a couple simple Sequences. The most simple sequence defines the start of a number series, and Perl lazily generates the rest.

> 0, 2, 4, 8 ... 1024
(0 2 4 8 16 32 64 128 256 512 1024)

This is nice and all, but I had to define 4 numbers for Perl to understand that sequence. If I dropped the 8, then it would have generated a sequence of even numbers. Instead, I can define a function which calculates how the next value should be generated.

> 0, 2, * × 2 ... 1024
(0 2 4 8 16 32 64 128 256 512 1024)

Here we see our old friends, the Whatever *. This time I'm asking for the Sequence starting 0, 2, then I define a mini-function that takes Whatever the last number in the Sequence was and multiplies is by 2. Sequences are a fascinating part of Perl 6 that could occupy a blog post all their own, but for now, that's enough of a foundation to create a simplified linspace function.

sub linspace( $start, $end, $steps ) {
    my $step = ( $end - $start ) ÷ ( $steps - 1 );
    return $start, * + $step ... $end;
}

linspace( 0, 1, 3 ); # Result: (0, 0.5, 1)
linspace( 0, 1, 5 ); # Result: (0, 0.25, 0.5, 0.75, 1)
linspace( 0, 1, 6 ); # Result: (0, 0.2, 0.4, 0.6, 0.8, 1)

This function does what I need, but it could stand to be a little more robust. I'll make a module to house this function, just in case I need it for future plots. I'll also add type constraints to the parameters, and multiple dispatch functions to handle steps of 0 or 1. I'm calling my library Numpl. Any resemblance to actual libraries is purely coincidental.

class Numpl {

    proto method linspace( Real $start, Real $end, Int $steps ) { * }

    multi method linspace( $start, $end, 0 ) { Empty }

    multi method linspace( $start, $end, 1 ) { $start }

    multi method linspace( $start, $end, Int $steps ) {
        my $step = ( $end - $start ) ÷ ( $steps - 1 );
        return $start, * + $step ... $end
    }
}

So that was a very scenic tour around the Sequence operator, but now that we have a linspace function, the rest is smooth sailing. Getting back to the plot, the final code now looks like this.

use Matplotlib;
use Numpl;

my $plt = Matplotlib.new;

my @x = Numpl.linspace( 0, 1, 500 );
my @y = @x.map(-> $x {
    sin( 4 × π × $x ) × exp( -5 × $x )
});

my ( $fig, $ax ) = $plt.subplots();

$ax.fill( $@x, $@y, :zorder(10) );
$ax.grid( True, :zorder(5) );
$plt.show();

Oh yeah, there was that hairy looking operation for the y values. To refresh your memory, the python looked like this

y = np.sin(4 * np.pi * x) * np.exp(-5 * x)

I can pretty much copy this operation exactly as it appears and put it inside the map. I've put the map operation on it's own line as I think this aids readability.

You might have also noticed I'm using proper @arrays in this one, and there is a valid reason for this. Sequences are lazy, and typically can only be iterated over once. x is iterated over in the map and that would prevent any later iterations. Here I've assigned the Sequence to a fully reified Array, which also means the Sequence is evaluated eagerly at assignment (ie. not lazily). As covered in part 2, Python doesn't like Perl arrays as positional arguments, so I coerce them to scalar values when I pass them.

The other way I could have handled this was to coerce my Sequence to a List on assignment like so.

my $x = Numpl.linspace( 0, 1, 500 ).List;

Then I could have assigned it to a scalar variable and still be free to iterate over it as many times as I like. If you have no need for laziness from your linspace function, you could modify it to return a reified List either coercing the return value as above, or by instructing the Sequence to be eagerly evaluated.

return ( $start, * + $step ... $end ).List
# or
return eager $start, * + $step ... $end

But I like to be able to control the laziness of my linspace result from outside the function.

Whichever way you do it, the end result should look like this

The wrapper module is working out well... a little too well. I wanted to tackle something a little more complex, and scrolling through the matplotlib gallery I saw it. It's high time for a histogram.

To be continued...

Using matplotlib in Perl 6 (Part 2)

2017-03-05 23:00, Tags: perl python matplotlib

This is Part 2 in a series. You can start at the Intro here.

In Part 1 of this series, I managed to use matplotlib in Perl 6 to output a simple graph from the matplotlib tutorial. In this post, I will tackle the first graph in the matplotlib gallery, barh_demo.py.

import matplotlib.pyplot as plt
import numpy as np

plt.rcdefaults()
fig, ax = plt.subplots()

# Example data
people = ('Tom', 'Dick', 'Harry', 'Slim', 'Jim')
y_pos = np.arange(len(people))
performance = 3 + 10 * np.random.rand(len(people))
error = np.random.rand(len(people))

ax.barh(y_pos, performance, xerr=error, align='center',
        color='green', ecolor='black')
ax.set_yticks(y_pos)
ax.set_yticklabels(people)
ax.invert_yaxis()  # labels read top-to-bottom
ax.set_xlabel('Performance')
ax.set_title('How fast do you want to go today?')

plt.show()

Again, numpy is being used, but I figured Perl 6 can probably do whatever is needed natively, so I proceeded to convert the imports to Perl parlance (perlance?) and got started.

use Matplotlib;
my $plt = Matplotlib.new;

$plt.rcdefaults();
my ( $fig, $ax ) = $plt.subplots();

As mentioned in the tail end of Part 1, Python likes scalar values, so from here on in I'm going to stick to using mostly scalar variables in my Perl code. I may also provide alternative methods of doing things if I think they are noteworthy or interesting. Now on with the show!

I need to create the example data... people is just your average list of strings. Perl has always had nice quoting construct to create a list of words, so that ones is as simple as

my $people = < Tom Dick Harry Slim Jim >;

Now what's numpy.arange doing? I knew that len(people) was 5, but I didn't know what arange was doing with that 5. It turns out it simply creates an array of values starting at 0, incrementing by one and up to 5, ie. [0, 1, 2, 3, 4]. As seen in Part 1, I can use the handy Sequence operator for that, but creating lists like this is fairly common which is why Perl 6 also provides a convenient shortcut for it:

my $list = (0 ...^ 5); # Result: (0, 1, 2, 3, 4)
my $list = ^5;         # same thing

Of course, I can use a variable instead of a literal integer. As covered in Part 1, lists used in a numerical context are coerced to their number of elements. These features combined mean that my y_pos assignment looks like this.

my $y_pos = ^$people;

For the next two lists of values, performance and error, numpy is being used to generate some random values. Both values reference len(people), and this equates to the number of random values returned: 5. Perl has a fairly standard rand function that - without arguments - returns a random value between 0 and 1. However, I can use the "list repetition" operator (xx) to create a list of random values. Again referring to Part 1, I know that mathematical operations on a numpy array are equivalent to mapping that operation across each element, so I ended up with this.

my $performance = ( rand xx $people ).map( 10 × * + 3 );
my $xerr = rand xx $people;

In those 3 lines of code, The $people list variable is used in a numerical context so it is evaluated to it's number of elements: 5. You could always be more explicit in your code and say $people.elems if you prefer.

You may have noticed I called my list of error values xerr instead of error and there's a good reason. Perl 6 provides a convenient syntax for passing named arguments (kwargs in Python) when a variable has the same name as the named argument.
Oh, and one more thing. Remember that < word quoting > syntax I used before? I can use that when passing named arguments too. Here's a quick illustration using an example function foo() that takes 1 positional parameter, and 1 named parameter.

sub foo( $positional, :$named ) { ... }

# Calling the function
foo( 'bar', :named('baz') );

# Using the word-quoting construct
foo( 'bar', :named<baz> );

# or if I had a $named variable
my $named = 'baz';
foo( 'bar', :$named );

To some, it may seem odd to have so many ways to pass a named argument (and truthfully there's more) but I hope what comes through here is that the syntax tries to consistent in how it's used... But I digress... Less plodding and more plotting!

With the above out of the way, the rest of the code is pretty straight forward. Here's the final code.

use Matplotlib;
my $plt = Matplotlib.new;

$plt.rcdefaults();
my ( $fig, $ax ) = $plt.subplots();

# Example data
my $people = < Tom Dick Harry Slim Jim >;
my $y_pos = ^$people;
my $performance = ( rand xx $people ).map( 10 × * + 3 );
my $xerr = rand xx $people;

$ax.barh(
    $y_pos, $performance,
    :$xerr, :align<center>, :color<green>, :ecolor<black>
);
$ax.set_yticks($y_pos);
$ax.set_yticklabels($people);
$ax.invert_yaxis(); # labels read top-to-bottom
$ax.set_xlabel('Performance');
$ax.set_title('How fast do you want to go today?');

$plt.show();

This graph is based on random data, so the output will vary slightly with each run, but ultimately I get something along the lines of this when I run it.

Well that was mostly painless. The wrapper module is working as expected, and I was able to emulate a basic function of numpy with relative ease. I glanced back to the the Matplotlib gallery... Next in line was the fill_demo. Let's go!

Using matplotlib in Perl 6 (part 1)

2017-03-04 20:30, Tags: perl python matplotlib

This is Part of a series. You can start at the Intro here

Looking at the graphs in the Matplotlib tutorial, the first one is a straight line (boring). The second is a few red dots plotting a curve (getting better). The 3rd graph looked a more interesting: Three curved plots consisting of different patterns and colors. Here's the Python code.

#!/usr/bin/env python

import numpy as np
import matplotlib.pyplot as plt

# evenly sampled time at 200ms intervals
t = np.arange(0., 5., 0.2)

# red dashes, blue squares and green triangles
plt.plot(t, t, 'r--', t, t**2, 'bs', t, t**3, 'g^')
plt.show()

I ran it in python to confirm it worked as expected (it did). I then proceeded to copy and paste the code into a new Perl 6 file and changed the import statements to use statements.

use Matplotlib;
my $plt = Matplotlib.new;

Now for the data... The t values were generated by a numpy function called arange. I didn't know what that did, and rather than wrap numpy as well, I figured I could probably do whatever it was doing with Perl 6. As the comment suggested, it was a simple number sequence. Perl 6 has a handy Sequence operator for just this sort of thing.

my @t  = 0, 0.2 ...^ 5;

This defines a Sequence starting at 0, followed by 0.2, then increasing at the same interval until is gets up to (^) 5. If I wanted to include 5 in the Sequence, I would just drop the ^. Using some well placed print statements, I confirmed by @t values were the same as the python version.

I then pasted the rest of the Python code into my Perl script, inserted a @ in front of each t and ran it. Things did not go as planned. Well, I got a graph, but it wasn't quite right

Some of you might have already guessed where I went wrong. Spoiler, I don't have any experience with numpy arrays (or numpy for that matter). A few more print statements let me know where I was going wrong. You see, when referring to a list (or array) in a numerical context, Perl will coerce the array to it's number of elements

my @t = (2, 3, 4);
@t ** 2; # Result: 9

However numpy arrays will apply that operation to each value in the array. I could create an object in Perl 6 that behaves the same way, but for now I'll just use Perl's map function which is perfectly apt for this sort of thing. Perl 6 provides a number of ways to map values in an array: I can be very explicit, or very succinct.

@t.map(-> $x { $x ** 2 }) # Result: (4, 9, 16)
@t.map(*²)                # same thing

Note the lack of curlies {} in the second example. This is a special syntax where * takes on the value of whatever value it was passed. In fact, Perl 6 actually refers to that * as Whatever. This WhateverCode is best suited for simple operations, which perfectly describes a binary mathematical operation such as exponentiation.

Additionally, Perl contains quite a few unicode operators; My favourites are × for multiplication and ÷ for division, instead of the historical compromise of * and /. In the second example above, I am using the superscript of 2 (²) to raise Whatever (*) to the power of 2.

I modified my arguments accordingly and ran it again. Another graph! This one also had 2 dots... But they were in a different place to last time, so that's progress, right? I looked at the gist again and noticed the arguments to plot were inside square brackets. Accepting it was probably due a difference in how each language handled positional arguments, I diligently wrapped my arguments to plot in square brackets.

$plt.plot([
    @t, @t,         'r--',    # red dashes
    @t, @t.map(*²), 'bs',     # blue squares
    @t, @t.map(*³), 'g^'      # green triangles
]);

$plt.show();

This time when I ran it and was delighted when a graph identical to the one on the tutorial appeared before my eyes.

Wondering about those square brackets, I played around a little with the call to plot. I quickly learned that Python prefers to be given what Perl calls "scalar" variables. In simple terms, a scalar is a variable that Perl will treats as a single "thing". Passing a Perl array of 3 @things to a Python function would cause those 3 values to get expanded out to 3 positional arguments. Creating a scalar list $things or coercing $@things to be passed in a scalar context works much better.

With regard to that map, it seems that by giving the map directly to plot, it was more "listy" than scalar. By assigning the result to a scalar first, I could then pass those values to plot without having to wrap all the arguments in square brackets, like so..

my $t  = [ 0, 0.2 ...^ 5 ];
my $t2 = $t.map(*²);
my $t3 = $t.map(*³);

$plt.plot(
    $t, $t,  'r--',    # red dashes
    $t, $t2, 'bs',     # blue squares
    $t, $t3, 'g^'      # green triangles
);

$plt.show();

So that about wraps up Part 1. This graph was kind of interesting, but I knew matplotlib was capable of more. I felt no great excitement from the other images in the tutorial, so headed off to the gallery to see what else I could sink my teeth into.

To be continued...

Using matplotlib in Perl 6 (intro)

2017-03-04 20:00, Tags: perl python matplotlib

I'm a network engineer by trade, but I write a lot of scripts at work to automate or simplify aspects of my job for myself and my colleagues. Most of the tools I write are predominantly doing text parsing, and I use Perl 5 for that. I use Python on occasion where I feel like it's a better fit, but mostly I prefer Perl. I'm also a fan of Perl 6 and have been playing with it since the official release in December '15.

I've been aware of matplotlib (and numpy) for a while, but in my line of work I've never really had a need to use either. That said, I have the awareness that matplotlib is a great plotting library, something that Perl 6 is lacking, but not all is lost.

While perusing the comments in this HackerNews thread, someone linked to a gist that used Inline::Python (a module for executing Python code and accessing Python libraries from Perl 6) to wrap matplotlib in a Perl 6 class. Was this really all that I needed to do to use matplotlib in Perl 6? I headed to matplotlib tutorials page to try it out.


So that's what this series will be about. I found the process of figuring out how to use matplotlib in Perl 6 interesting, so decided to write about it. Now, I don't want to lead anyone down the garden path, so I'm putting this up front. After the first plot, I discovered that wrapper in the gist will fail when passing named arguments. I've added a comment to that gist about this. There are also more shortcomings to this module, but I'll cover those in later parts to this series.

The wrapper module I'll be working with for the first few diagrams looked like this, though it will change as I discover limitations.

class Matplotlib {
    use Inline::Python;
    has $!py;

    submethod BUILD() {
        $!py = Inline::Python.new();
        $!py.run('import matplotlib.pyplot')
    }

    method FALLBACK($name, |c) {
        $!py.call('matplotlib.pyplot', $name, |c);
    }
}

Essentially this class creates a new Inline::Python object and runs import matplotlib.pyplot when it get instantiated. There are no normal methods defined, so any class methods route to the FALLBACK method, which takes the name of that method and calls the matplotlib.pyplot method of the same name.

But what's that |c? That's the syntax for a special type in Perl 6, a Capture. Any additional argument are "captured" in the Capture object. A capture doesn't do any unpacking. It encases them in carbonite and ships them off to wherever to be unpacked later. Capture objects are very useful when you want to punt arguments to another function.

There's also an Inline::Python module for Perl 5 if you want to try to use that with pyplot. I took a stab at it but ran into difficulty with named arguments, but I gave up pretty quickly since Perl 6 was nicer for the number-y stuff.

Anyway, this wrapper module will cover most generic uses of pyplot, and I get a few plots in before I hit my first hurdle, so head on over to Part 1 to start the journey.

If you'd like to play around with my modules as they existed after this series, I've put them up in a gist. You're welcome to fork them and do whatever you will.

Later