I just mean to point out this great post by Mauricio Arango on “Coordination in parallel event-based systems”. He mentions message passing, actors, message queues, pub/sub and tuple spaces, but seems to conclude that we’ll be seeing more tuple space usage in the future.
New Direction: Computational Biology
After an absurdly long job search, I’ve finally found myself a comfortable place in a computational biology lab. I’ve been here a bit more than a month and thought I should mention something about what I’m doing.
I’m working for Dr. Michael Brudno in the Computational Biology Lab at the University of Toronto. At the moment, I’m developing an application for visualization and analysis of biological sequence and annotation data with a graduate student named Marc Fiume. (We just chose a name for our project today: SAVANT. I like it.) I’m also sitting in on a graduate seminar on analysis of high throughput sequencing data and attending the occasional presentation on related research at The Centre for Applied Genomics. I’ll be spending one day a week at Sick Kids hospital, in order to interact with biologists and bioinformaticians who are among the target users of SAVANT.
I’m having a great time.
This is all a huge change from the enterprise web development that is more or less what I’ve been doing since 1996. A huge change that I really needed. Sometimes you just need to start over, you know? It was getting to the point where I honestly couldn’t picture myself actually taking any of the jobs I was applying for. I couldn’t face the same-old, same-old any longer.
I’m not sure where this is all going to lead, but I’m kind of hoping to make a career in this relatively young field. I believe that my many years of experience in commercial software engineering will be useful here. I think I can have fun and make a difference. The territory is huge; the problem space practically inexhaustible. I can’t imagine getting bored any time soon. Heading off in a new direction feels exactly right. So work-wise right now, it’s all good.
A “New Space Opera” Reading List
Many of my friends and acquaintances read SF, but there are a lot of sub-genres and I’ve noticed recently that many people either don’t know of “New Space Opera” or have just discovered it and want help finding more. Since I read as much of it as I can get my hands on, I figured I’d make a short, annotated, reading list to give those so-inclined a leg up.
First, the inevitable question: what is “Space Opera?” As far as I can make out, “Space Opera” was originally a derogatory term for the type of “cowboys in space” stories popular in the pulps once upon a time. Of course, low-brow or not, Space Opera is both very common and highly entertaining in television and film: “Star Trek” being the most obvious—in all its incarnations—as well as “Stargate” (likewise in 3 different series), “Firefly”, “Babylon 5”, and others you either know of if you’re a fan, or don’t need to know about if you’re not.
So what’s “New Space Opera?” This seems to be a much more controversial and slippery concept. For my part, I think of it as an updated, more high-brow version of Space Opera. It’s still primarily about adventure in space, but space isn’t just the setting anymore. The usual concerns of humanity far away from here and now—wild new technologies, bizarre and hostile environments, cultural evolution and upheaval—that you find in other types of SF are also found in New Space Opera. Since the late eighties, writers like Iain M. Banks, Alastair Reynolds, Peter F. Hamilton, Neal Asher, and others threw out the laws of physics as regards FTL travel (in “real” or “hard” SF you obviously cannot go faster than light) and then decided to see what kinds of ripping good yarns they could tell while sticking to most of the other rules of the SF genre. Usually, the canvas must be big: inter-stellar or even inter-galactic wars (including the cold kind); smuggling, piracy, espionage and terrorism; the rise and fall of empires; vast spans of space and time—that kind of thing.
So without further ado, here’s my short list of “New Space Opera” for anyone who wants to give it a go. This is by no means an exhaustive list (that would be impossible), nor is it necessarily a list of “the best” New Space Opera. It’s just my idea of some good places to start. Enjoy!
- Iain M. Banks. Not to be confused with Iain Banks, who is the same person writing non-genre fiction. To the best of my knowledge Banks was among the very first (if not the first) to kick of this new kind of space adventure. His first space opera Consider Phlebas is a classic, but my personal favourites are the later Use of Weapons and Feersum Endjinn. Both Consider Phlebas and Use of Weapons are so-called “Culture” novels (involving a human civilization Banks calls simpy “The Culture”), while Feersum Endjinn is set in a universe of its own. Banks’s more recent work is still worth reading, but it seems to me to be running out of steam.
- Alastair Reynolds. Reynolds has to be the undisputed master of vastness. His stories take place over such enormous spans of space and time that they sometimes feel a little bleak. Nevertheless, it’s brilliant and imaginative stuff. His debut series of novels Revelation Space, Redemption Ark, and Absolution Gap are must-reads. But if you’ve only got the time or taste for a singleton, I’d recommend the recent House of Suns
- Peter F. Hamilton. Hamilton tends to write long books with lots of characters. It’s the many points of view that bring the stories to life and make them so much fun. Probably the best place to start is with the two-volume “Commonwealth Saga”: Pandora’s Star and Judas Unchained. He has what promises to be an even better series in the works, but as it’s unfinished, I wouldn’t start there (ok, it’s The Dreaming Void and The Temporal Void…so far.)
- Neal Asher. Asher took a little while to grow on me. I think he grew as a writer, too, which certainly helped. His first novel Gridlinked was well-reviewed and is probably worth reading because it’s the first “Agent Cormac” novel, but I didn’t think much of it at the time. An ultra-violent high-tech secret agent yarn. **yawn** But things definitely got better after that. Asher has a marvellously creepy bio-aesthetic that’s unlike any other writer I can think of. Asher’s “Polity” owes much to Banks’s “Culture”, in that it’s a utopian society ruled by (allegedly) benevolent AIs often embodied in warships (?!) But that’s about all they have in common. My favourites are probably the Agent Cormac novels, which are, in order: Gridlinked, The Line of Polity, Brass Man, Polity Agent, and Line War.
- Charles Stross. Stross does more than just Space Opera, but he’s written a couple of decent ones: Iron Sunrise, Singularity Sky, and by stretching the definition of Space Opera a bit, you could probably include Accelerando
—which is worth a read anyway, because it’s the only attempt I know of to write up to and through the “Vingean singularity”. Deserves a prize for bravery, that. - John Scalzi. This series is a must-read: Old Man’s War, The Ghost Brigades, The Last Colony and Zoe’s Tale. The last is a bit of a do-over of The Last Colony, but even though I was originally doubtful, it turned out to be well-worth the read.
- David Weber. This is probably the most controversial entry on my list. Weber writes military SF (and fantasy), and he’s very prolific. Military SF is sort of a sub-genre of Space Opera (itself already a sub-genre). Still, it’s got all the right elements: lots of big space ships getting blown to smithereens. Honor Harrington, the heroine of an apparently endless series of novels, owes probably everything but her gender and her Queen to Horatio Hornblower. Nevertheless, the first 4 books in the series, On Basilisk Station, The Honor of the Queen, The Short Victorious War, and Field of Dishonor are highly entertaining.
- Gardner Dozois and Jonathon Strahan (Eds.) If you’re an SF fan, you’ve probably noticed the ever shrinking shelf-space for SF in bookstores. The “Science Fiction & Fantasy” section is typically 99% fantasy and most of the SF is video-game and television tie-ins. The industry seems to be in some kind of (hopefully non-fatal) tail-spin, so the real writing and publishing effort is going into (or should I say: returning to) short fiction. The New Space Opera and The New Space Opera 2 are two excellent anthologies of short fiction, some of it by the aforementioned authors, and many more besides. I can’t wait for the next instalment!
Note that all of the authors mentioned above (well, most of them, anyway) have written many more books than those I have listed. Those are just my suggestions for places to start. After that, Amazon’s search feature is your friend. Enjoy!
Gruple: A Tuplespace for Groovy
I’d like to announce the first release of Gruple, a tuplespace implementation for Groovy (and Java, of course.) Release 1.0 is an *in-process* space only, meaning that it can be used to co-ordinate and synchronize threads, but not separate processes. It’s intent was to add one more tool to the toolkit for making concurrent programming simpler. (A remote space, allowing co-ordination among separate processes and/or nodes is on the roadmap, but not near the top.)
You can find the project at http://gruple.googlecode.com
After downloading and building the source (there’s an ant buildfile to make it easy enough), please read the Demos page for some instant gratification. 😉
Please enjoy it, and I appreciate any feedback (including bug reports!)
Happy concurrent programming!
Contacting the Author
I realized late that the only way to contact me is by my now thoroughly expired iName (what was I thinking?!) The About page has been fixed. Sorry for the inconvenience.
Loosely-Coupled Actors: In Brief
Actors are a popular way to write concurrent & distributed programs. Immutable messages are passed between actors which do the required processing, avoiding the difficulties inherent in sharing data among threads/processes.
Tuplespaces (best exemplified in current times by JavaSpaces) are a way to decouple cooperating processes by using pattern-matching to share immutable and persistent data objects.
Many distributed algorithms can be modelled as a flow of objects between participants.
Or, if you prefer, substitute “parallel” for “distributed” and “messages” for “objects”.
So what do you get if you combine the pattern-based communication of tuplespaces with the message processing paradigm of actors? Loosely-coupled actors.
Actually, as I see it, you get two things:
- An easier way to write tuplespace programs: the actor DSL provides a formalism for describing reading, writing, and consuming of tuples and the behaviour associated with these operations.
- Pattern-based communication between actors with no explicit reference to each other: actors are decoupled in both (address)space and time (since messages are persistent objects.)
In both cases, a (hopefully) better way to write concurrent programs.
To the best of my knowledge, no one has tried this yet.
Note: I wrote a longer piece on this topic yesterday with more explanation and references. For those wanting a little more…
Loosely-Coupled Actors
Recently the growing concern with effective use of multicore processors and the subsequent popularity of the actor model came together with my desire to do a project in Groovy. Adding to the mix, I had an in-JVM tuplespace library in Java I’d never released because it seemed thread co-ordination was just not enough of a big deal. Now, with the need to simplify and promote good multi-threaded programming practices, I thought it might be worth translating it into Groovy as a learning exercise (I’m new to Groovy) and releasing it as open source.
But the actor model and tuplespaces kept getting mixed up in my head. Like chocolate and peanut butter, I just wanted them both at the same time. What I really wanted was loosely-coupled actors.
Actors
The actor model uses the passing of immutable messages to avoid the problem of shared state in concurrent and parallel programs. This is a gross oversimplification, but a deeper discussion is beyond the scope of this post. If you want a better explanation, this two-part article (part 1, part2) by Alex Miller really helped me out. (Don’t let the Erlang stuff in Part 1 turn you off, he gets to Scala, Java, and Groovy eventually.)
Tuplespaces
In a tuplespace–again grossly simplified–there are no shared variables, but there is a shared data space filled with immutable objects. These objects are accessed not by address, but by their contents. A tuplespace is an associative shared data store. These objects, or tuples, can be thought of as messages. They are matched by templates inserted into the space by other components or processes, rather than delivered to a named end-point. A tuplespace is a realization of the “generative communication model” because messages are first-class, shared objects. But in this model communication is uncoupled rather than point-to-point.
Digression: Assembly by Diffusion
A passage in Manuel Delanda’s Intensive Science & Virtual Philosophy, does something to explain why I am so attracted to spaces as a communication mechanism. It’s not an exact analogy, by any means, but it gives a flavour of the “open combinatorial spaces” that the generative communication model creates. Here’s an excerpt:
[Biological assembly] permits the transport processes not to be rigidly channelled, using simple diffusion through a fluid medium to bring the different parts together. Components may float around and randomly collide, using a lock-and-key mechanism to find matching patterns without the need for exact positioning.
I think that’s a pretty fair description of “loose coupling”: no rigid channels; no exact positioning; parts brought together by pattern matching. Tuplespaces would seem to act like the aforementioned fluid medium, allowing components to work together in an almost organic manner.
ActorSpaces
Agha and Callsen describe the idea of an ActorSpace. Have I finally found what I’ve been looking for? Well… not quite. Agha and Callsen do recognize the importance of “pattern-based communication between processes which have no explicit reference to each other” for open distributed programming. However, they chose to base their pattern matching on attributes of actors themselves. This introduces a new element to the actor model–attributes–and it is not entirely clear what these attributes might be or how to use them to abstract communication. Interesting paper, but it doesn’t quite scratch my personal itch.
Tuplespaces + Actors
Many distributed algorithms can be modelled as a flow of objects between participants.
We could as well substitute “parallel” for “distributed” and “messages” for “objects”.
Considering the quote above, the actor DSL provides a way to define the “participants”, describe their behaviour, and manage their lifecycle. The tuplespace operations provide the pattern-directed flow of messages.
The result is:
- An easier way to write tuplespace programs: the actor DSL provides a formalism for describing reading, writing, and consuming of tuples and the behaviour associated with these operations.
- Pattern-based communication between actors with no explicit reference to each other: actors are decoupled in both (address)space and time (since messages are persistent objects.)
In both cases, a (hopefully) better way to write parallel programs.
So the bottom line of all this is that my little Groovy tuplespace project is trying to grow into something more: using a tuplespace paradigm to provide pattern-matching based on messages and processing of these messages by actors: loosely-coupled actors.
Handheld Geometry
I’ve been reading The Symmetries of Things and came across the sculptor Bathsheba Grossman, an artist “exploring the region between art and mathematics.”
In addition to the artist’s site, here’s a ton of images available elsewhere, including the artist at work.
Blogfail
Ok, I know a bunch of people are waiting for me to comment on Opera Unite and what it does (or does not) have to do with oponia. That and the embarrassing last post from January promising to post more. So, yeah, yeah, yeah, I fail. What can I say? I’m job hunting. If you want me to blog more, hire me 😉
IE and JPEGs
I recently discovered that Internet Explorer 6+ reports all jpeg images as having the content type image/pjpeg, thus breaking every web application that allows the uploading of files and checks for certain allowed mime types (typically image/gif|jpeg|png). Officially, there is no image/pjpeg mime type. It’s just made up. It adds no value. It is a pain in the behind.
While it doesn’t surprise me that Microsoft would do such a thing (though why break that in particular?), it does surprise me that there isn’t more outrage among developers about it. Have we just become so used to MS playing dirty tricks on us that we eat the time finding a workaround and move on without complaint? Personally, I feel deeply offended. An over-reaction, no doubt, but it seems to me that life is too short to have to deal with this kind of petty crap.
Rant off. I feel marginally better. Sorry for the dump, but silent fuming isn’t good for the body or the spirit.
And BTW, I made a New Year’s resolution to blog more. I’ve been so busy lately, I let it slide. I’ve got a couple of technical posts in mind, so I’ll get them out soon. Cheers, and Happy New Year to all!