This started out to be a single post, but writing about code gets you thinking about code, and I’m now making some changes to the program under discussion. Rather than wait until that’s done and post one long piece I’m splitting it up. Part one is mostly background and some observations on using and writing about Processing, part two is more about the code for a Processing program called OSC Commander. At some point in the near future I’ll put it up on Github, perhaps when I post part two.
I’ve been working with Open Sound Control (OSC) for a while, mainly to control music software, but also for visual applications. Typically the OSC message is sent from an application that is reacting to input from some kind of controller, such as an XBox Kinect.
I started writing about OSC for an E-book, Just the Best Parts: Kinect Hacking for Artists.
The book is in beta; you can read and comment on it on-line as I write it. The book describes how to use the XBox Kinect to control audio/visual applications, such as Animata and Renoise. It explains how to get and install the required core software so as to use a standalone Kinect with a computer running OSX or Windows, and then how to do fun stuff. The “do fun stuff” part involves converting the Kinect data into either OSC or MIDI. Some explanation of OSC was required.
It occurred to me, though, that this might be a serious digression from the main focus of the book. One of my goals with Just the Best Parts is to avoid a common flaw in many tech-leaning books: cover-everything-ititis.
It’s tricky to know what to include and what to omit, and whether to explain something in detail or just defer readers to some other source.When I was doing some writing for Wrox (the first, now-gone-bankrupt, version) they seemed pretty adamant that if I could not refer the reader to another Wrox book then I would have to explain things myself. There’s a reason those books with the red covers and yellow titles were so large.
I couldn’t find a suitable resource that explained OSC well enough for non-geeks so started writing a companion book, Just the Best Parts: OSC for Artists.
By the way, as the books get closer to completion I’ll starting making them available in various formats, such as epub, mobi, and PDF.
OSC is extremely handy. Think of it as MIDI on steroids. Comparisons between the two are interesting. There may be some things that MIDI handles better than OSC, but OSC wins easily as a general-purpose inter-application control protocol. In short, the word “sound” in the name is a really misleading.
I’m a big fan of the DAW/tracker application Renoise. It has both MIDI and OSC interfaces. The OSC aspect is somewhat limited, but Renoise as a full-blown API, scriptable via Lua. This allows you to add your own OSC message handlers.
There are various client tools for sending OSC commands, such TouchOSC, and Control, and they’re great. They have some limitations, though.
One is that they are generally hard-coded to send specific OSC address patterns. (This is true of TouchOSC; Control will do what you want, but if decide to change the address patterns in use by a particular screen you need to go change code.)
My way of handling this is to use a proxy to convert incoming OSC messages into more suitable out-going messages. But that’s still making you use this or that client controller to initiate a message.
One problem with relying on certain controllers is that getting one to send exactly what you want, when you want it, can be a problem. For example, using OSCeleton I can turn Kinect skeleton tracking into OSC messages describing body-part locations. But if I want to send a specific location message, well, that’s tricky.
Another problem is trying to figure out how best to convert, say, those OSCeleton tracking messages. Do they need to be scaled? Adjusted in some way? What’s the end-point OSC server expecting? Why is not working, dammit?
In order to get what I wanted I did what I often do: I wrote my own tool. It’s called OSC Commander and it’s written in Processing
Actually, the current version is written in Processing. It started life as a small Ruby script. Run from the command line, I could pass it any address pattern and values and it would send it off to some OSC server. At first I hard-coded the server IP address and port, but as found myself using it for different servers I had it read in a config file. There was some sort of switch to tell it what server config to use. It was quite handy, and I still use it occasionally.
When I started writing the OSC book I wanted to include my Ruby tool, but that added some complications. First, I was already using Processing in the Kinect book. Since that, and the OSC work, are meant for people who may be, at least initially, uncomfortable dealing with too much technical material I was already walking a fine line with how much geekery to include. Processing, at least, was intended for artists, and the intro course at Kahn Academy is a great learning resource. Introducing another programming language, and explaining how to use it fro the command line, seemed a bad idea.
I decided to port the code to Processing, where I could add a decent GUI. I did a bit of Googling for a GUI toolkit and settled on G4P AKA GUI for Processing. There’s also controlP5 which I think I used for this or that little practice code.
I’m not a long-time, hard-core, Processing expert. I’m pretty good with it but I’m sure there is more I can learn. (Yes, that’s always true for almost everything; I mean it in a non-trivial sense.) I’ve written a fair number of “sketches”, as they’re called by Processingites. (I just made up that term; no one actually ever says that.) I’ve read (most) of some books on Processing; as best I can tell no one writes anything really big in Processing. I mean, you could, I suppose; it’s Java under the hood, and it’s fairly easy to use Java libraries, but I don’t think I’ve ever seen or heard of a truly big Processing sketch. Hell, that name alone, sketch, tells me that no one is supposed to make something gigantic. (If any reader can point me to a large Processing app I’d love to see it. All the large-ish sketches I’ve seen were large because they had scads of computations and special conditions. The application logic itself was not so big, but space was taken up, for example, by many methods for drawing this or that shape.)
That’s fine, small is beautiful, etc., but if you find your sketch growing past a page or two and you want to organize it better what are your options?
Pretty much every sketch I’ve seen employs some cring-y coding habit. Global variables are the biggest offender. Here’s the thing, though:If you’re trying to get artists and musicians writing code, do you want to beat them about over best-practices, object design and encapsulation, etc.? OK, you may be silently mouthing “Yes!” right now. Sure, I’d prefer that people learning to code learn to do it well, and learn practices that will, over time, save them some grief. But the reality is that for small programs that do very specific tasks (e.g., make swirly smoke images), one-file, big series of procedural code apps are fine.
You can create your own classes, using a syntax similar to that in Java. But it’s not quite the same, and the classes you create in a sketch end up as Java inner classes. You can create full classes by using actual Java files in your sketch, but that opens another can of worms if you plan on explaining that code to a coding newbie (or if you yourself don’t know Java).
As I was describing the various demo sketches for the books I tried to clean up code that had evolved over time. The code wasn’t bad, but as one decides to release something there’s always that thought that maybe things could be a little cleaner, maybe demonstrate whatever might count as better practices.
One easy way to manage code is to break your sketch into separate files in the root directory. When you run the sketch they are all slurped in, apparently in alphabetically order.
My general approach, then, has been to create classes to provide some sensible, reusable data structures (e.g. a Point class) and try to put related code into separate files (e.g. OSC initialization, and OSC message event handling).
OSC Commander, though, really doesn’t have too much of this. It’s basically a bunch of G4P controllers that feed an OSC client. There’s also an OSC server to catch replies that may come back.
I’m using the oscP5 library to send and receive OSC messages. It seems to be the Processing library to use for OSC. It’s easy to use, and while it does not support all of the OSC spec (no built-in pattern matching, for example; you have to handle that yourself) it does it make it quite easy to send and receive messages.
I put the OSC stuff into an osc.pde file. The sketch loads a config.txt file and uses loadStrings to get an array of lines. The upside to this: It’s really easy. The downside: Order matters.
At first I had IP addresses and port numbers on separate lines. That made it easy to load, save, and set assorted text fields in the GUI, but I used some goofy line ordering that ultimately confused me. I’m pretty sure “Your own code confuses you” is a code smell.
I changed it so that address details used this format: 127.0.0.1:8001, and did a split on the “:”.
Once the configuration data are loaded into the GUI, an OSC client is created. You do not have to specify the IP address of the client machine. You do have to say what port you are using.
import oscP5.*;
import netP5.*;
OscP5 clientOscP5;
NetAddress oscServerAddress;
void createOscStuff(){
oscServerAddress = new NetAddress(serverAddressText.getText(),
int(serverPortText.getText()) );
println("Listening on " + clientPortText.getText() );
clientOscP5 = new OscP5(this,
int(trim(clientPortText.getText())) );
}
What’s nice is that the same OscP5 instance can be both client and server. As a client, you use it to send OSC messages to some other address and port; these you specify at the time you send you message. As a server, if you implement an oscEvent() method you can handle any messages sent to you.
Part two of this post will get into the details of that and other code, once I make a few changes.