Tags

, , , , , , , , , , , , , ,

Perl is my favorite language, probably because Perl was my first!

Perl has an enormous library of packages for every occasion. One of the bigger frameworks is POE. I had attempted to give it a spin a few years back, but gave up, I don’t remember why. This time I’m following through with it.

What is POE?

POE is a Perl framework for writing reactive programs. Cooperatively multitasked programs and networking programs are overlapping subsets of reactive programs.

POE implements a single API and bridges from it to other event loops. A program using POE can run under any event loop that POE supports, with little or no modification.

In layman’s terms

POE is something you can use to build event-driven applications. Event handlers are functions/methods listening for ‘Events‘ that may happen at any time, at any order.

In even simpler terms…

Think of a very simple program (not event-driven).

Start (Initialization)
   -> Run
      -> do something
         -> do something else
            -> exit (finished doing what needed doing)

This simple program has no potential of user interaction, it has a predetermined path that is already known at ‘Start‘, once it has finished ‘do something‘ it will ‘do something else‘ and then ‘exit‘.

Now think of the same program, if it was event-driven, ie. with a GUI.

Start (Initialization)
   -> Specify Event Handlers:
      -> listen for: do something
      -> listen for: do something else
      -> listen for: exit
   -> Run (Show graphical window with interactors)
      -> Wait for Events

It’s clear that a graphical program, like this simple example, must sit around and wait for user interaction. It will not ‘do something‘, ‘do something else‘ or ‘exit‘ on its own.

The user Starts the program… At any time afterwards, the user may trigger the ‘do something‘ event or the ‘do something else‘ event (ie. by clicking a button), the user can repeat the event as many times as the user wants! At any time after that the user can trigger the program to ‘exit‘.

Replace the graphics example with a networking application and the same analogy would also work. The application waits for network traffic/commands to trigger events, which can happen at any time… etc..

If you’ve played with Java you may have noticed the words Event and Action used in EventListeners/ActionListeners, Java is a very high level language that accommodates events in its core library. Perl doesn’t — or didn’t — accommodate them. POE is one of the modules that offer this functionality (there are a few others).

Asynchronous & Non-blocking

Terms mentioned often in POE are ‘Asynchronous’ and ‘non-blocking’.

The former means that when an event is triggered it will not be ‘executed’ immediately. Events will be queued and dispatched when convenient, and their handlers could be written in a way that they too trigger another event when they are done and want to ‘return’ a result to the user.

The latter means that when an event is dispatched it will not freeze the main program by making everything else wait for it to finish executing. Non-blocking programs in simple terms use threads or ‘fork’ processes to handle expensive operations, so the main program body stays free.

I do not yet know the nitty-gritty details of how this is actually done in Perl or POE. Some POE-related modules declare themselves as asynchronous and non-blocking.

If something is non-blocking, the asynchronous part is implied — the reverse is not true.

Preparing Perl

If under windows, ActiveState’s Perl works well, and its package manager (ppm) handles the nitty-gritty details of installing perl packages well enough. Installing POE under ActivePerl is as simple as launching their graphical ppm and selecting what you want to install.

Here’s an interesting observation: the ppm is not a non-blocking program, when it is processing modules for installation its GUI clearly freezes up.

I had trouble getting POE to install under Cygwin (I think my Cygwin broke) and even under Fedora 15 (a mess with older versions). But it was a breeze under Fedora 16. I think the specific environment and Perl version makes a difference.

First touch with POE::Component::IRC

An IRC Client or an IRC Bot is inherently an event-driven application. People send messages at any time; and the client must patiently listen for them and trigger the events associated with messages. The IRC server will periodically check that the client is ‘still there’ (with PINGs) and the client must reply to those (with PONGs) to acknowledge it’s still connected.

I chose to start with POE::Component::IRC, often referred to as ‘POCOIRC’ for short, because it deals with something I’m familiar with — scripting for IRC — and is an easier task to tackle than a full GUI app.

To play with this I first wanted a local IRCd as a local playground. And POE::Component::Server::IRC (notice: different module) gave me exactly that! In the /examples/ folder of its distribution archive there is a test-ircd.pl script that works fine as-is.

help topics

My first goal was to write a simple ‘help topic bot’ that responded to user commands like ‘!help something’ with the help topic’s content. I had previously written a help topic script like that, as a script for Irssi, the code was in Perl already since Irssi can load Perl scripts as ‘modules’. I just needed to find the equivalent functionality for POE. This was easy to tackle.

multiple bots

But, I also wanted to look in how to have one script handle multiple connections to IRC at the same time. Simple enough concept; So it shouldn’t be too tough to tackle, I hoped…

That took a bit of reading, a lot of trial and error and some rummaging through other people’s code to see how it could be done — particularly Bot::BasicBot.

timers

The last big thing that any self-respecting bot must do is have timers. Figuring out how that worked was a brain twister that took a lot more reading, and a lot more trial and error to get right. I made an example of four predefined timers at 1, 5, 10 and 20 minutes intervals.

Et voilà! Here’s a code sample of how to do that (note: it’s full of debug lines left in).

Bot000/Bot.pl (view | download)

The result was a bit interesting, looking at the code as it is you may notice that each ‘bot’ may be in many channels, but is ‘responsible’ for only one. As a side-goal I wanted to see if I could make them all, collectively, reply to help requests so that whichever bot is least busy should send the reply.

‘Busyness’ is the number of outgoing messages waiting in the bot’s message queue — which POCOIRC has built-in as a way of protecting itself against flooding.

That last bit is handled in

sub communal_queue { ... }

Whenever one of the bots catches a command — in its own channel — and wants to send a ‘/notice’ reply to someone, it performs:

$_[KERNEL]->yield( 'communal_queue', 'notice', $nick,
                      '....message to send here....');

The communal_queue event will be ‘enqueued’ (yield sends events asynchronously, see POE::Kernel). When dispatched — in theory — it will look at each bot’s queue and use whichever bot is least busy, to do the actual responding. That’s a queue within a queue that checks a queue against a… .

This is not yet perfect and needs some ironing out. It seems to favor the first bot in @myBotHashes, unless under a heavy message load (>10 outgoing lines). My intended goal was to have each outgoing line go out via a different bot, but I did not meet that goal (yet).

Here’s some test results.

Querying with POE::Component::SimpleDBI

POE::Component::SimpleDBI‘s tagline is: Asynchronous non-blocking DBI calls in POE made simple.

That tagline drew me in and I wanted to give it a try.

I implemented two commands: portfolios and stocks. The DB used in the queries is from my solution of CS50’s pset7. The !stocks command can have a large output (I limited it to 200 rows), I mainly put that in to test how the bots behave when they’re really under a heavy outgoing msg load.

Eventually I got it to create the effect I initially intended for, which is: for each line that needs to go out, loop through all available pocoirc instances and send one each, rinse, repeat… But I didn’t want to do it that way explicitly, I wanted to see if it could be done implicitly, via weighing the queues of each pocoirc instance and choosing the lightest.

It turned out almost perfect (with a few re-arangements seemingly at random) and the output is no longer sorted when it reaches its destination (the $targetNick).

If this was a real irc network I’d say it was due to lag, network issues, etc; but it’s all local so I suppose it gets mixed around while still in the queue. This is still good for the help topic viewer, but not so good for sending lots of rows that should have certain predefined order — either for practical or aesthetics reasons — ie. multi-paragraph help topics, the  alphabetically ordered results in !stocks, or the ranked order in !portfolios.

The easy/practical solution I though for this is to just find the least busy bot once at the start, and then send all output through that one bot. I’ll do that in my next journey through POCOIRC and leave the sample code as is for now…

Bot002/Bot.pl (view | download)

and here’s some test results.

What next?

POE::Wheel and POE::Wheel::Run are used to deal with multiple processes, from some code examples I looked at a ‘session’ can have multiple wheels in it… I need to see how that works and maybe launch each separate POCOIRC in its own Wheel… if possible.

Enjoy!

P.S. I’m a beginner with POE and I would welcome any scrutiny of the source code from anyone experienced with POE. I’m pretty sure the startup and shutdown code can be improved. I’d love some tips and pointers 😉

Advertisements