DataEngine+Service

I'm a big fan of making it easy for artists to do art, interface designers to work on interfaces and coders to work on all the plumbing underneath. This means that each needs to be able to work somewhat independently, such that a coder isnt' forced to do interface work just to see their stuff in action or so that an interface designer isn't forced to request code changes beneath the UI just to change things around a bit. With an open source project that is distributed around the world, it's even trickier as you can't even guarantee that all three skills will be available simultaneously in the same geographic location. So the process must allow for different pieces to happen at different times.

Another thing I noticed when looking through other attempts at solutions in the "widgets" space was that pretty much everyone provided an API for things like "what's the speed and heat of the CPU" and "am I on the network?" and "fetch me this RSS feed". Unfortunately, these API sets were generally monolithic lumps of interfaces that weren't easily extended. Which means that as soon as you wanted to do something not in that API, you had to resort to cobbling it together yourself ... for each and every widget. At that point one is right back to the traditional development model, pre-widgets era, and life begins to suck again.

In designing Plasma, I decided that one of the (many) goal(s) would be to find a way to fix all of the above. Thus came DataEngine, and eventually Service, as vital pieces to that puzzle. One way to view DataEngine is that it allows ones to extend the widget API at runtime even through third party modules without runtime bloat: only what's needed is loaded, and what isn't there can be added as an after-market piece.

I blogged about KTorrent's plasmoid the other day, and something I don't think I mentioned is that they provide a DataEngine as well. That means that once you have KTorrent installed, any widget can start displaying torrent data. What if KTorrent isn't installed? If a widget asks for the KTorrent engine but it isn't installed, a dummy engine that just spits out null data is given in its place. This allows widget developers to be a little more lazy and lot more safe (you can always assume you'll get some engine back when you ask for it).

DataEngines are designed to be accessed by multiple separate data visualizations at once, support polling updates as well as push-on-demand updates and other fun stuff. All this comes for free to DataEngine implementations, but there is one catch to this all-shared magic dancing bear approach: writing to a DataEngine isn't supported, as that could have unexpected (and unexpectable) consequences to other visualizations.

That's where Service comes in. Services can be loaded as plugins and used independently, but there's another very useful pattern of usage: a DataEngine consumer asks for the Service associated with a given bit of data published by a DataEngine. Code wise, it looks like this: Plasma::Service *service = engine->serviceForSource("nameOfSource");. As usual, you are guaranteed to get a Service, even if it's a Null (aka "doesn't do anything") Service. This Service will arrive pre-configured to the data it is associated with.

For instance, if you are using the Twitter engine you can request a Service for a feed that provides both an authentication and a new tweet posting operation. That Service already knows the account it should use because it knows what data it was associated with: twitterEngine->serviceForSource("Timeline:aseigo"). So now we can interact with whatever that bit of data represents, which might be a media player (the NowPlaying engine provides such a Service) or a torrent or a Twitter account or the powermanagement daemon running on your laptop or ...

Each Service exposes a KConfig based API that lets you see what operations are supported and the details for that operation. Writing a Service is pretty easy too, especially if you provide a KConfigXT XML file named after the Service as that populates and registers the Service details automagically.

Services, unlike DataEngines, are not shared. They belong to whomever asks for it. Services spawn jobs, so a Service can be re-used over and over by the same owner.

Ok, enough about Services and DataEngines. This design allows coders to write these nice little bits of glue to all sorts of technical things that interface designers and artists can then very easily bind their work to using straight-forward and standardized API hooks. Everyone can work together, but also stay out of each other's way.

There's one more catch here, though: how does the programmer test their DataEngine or Service? Remember, we don't want them running off to write a user interface, especially not during testing since that results in some of the more hideous results. (I'm just as guilty of that, btw. =)

I hear some of you say, "Unit tests!" Sure, that's a nice idea, but not everyone's into unit testing and some DataEngines just aren't very unit testable (at least not easily; this can be particularly true for things like integration with 3rd party online services).

Recognizing the laziness inherent in developers, I wrote a small utility called plasmaengineexplorer, or Plasma Engine Explorer for the spaces-and-capital-letters crowd out there ;), to encourage programmers to test without creating "testing UIs" which is a synonym for "a looks-like-roadkill-on-a-hot-day interface".

Here's a shot of the engine explorer in action:



In that shot you can see it looking at the Twitter engine, the context menu on a data source and the service associated with that source in another window.

With this tool you can interact just like a plasmoid (or other user of DataEngines) can. You can request new sources, poll for updates, see what sources are available, request services, start operations in those services with custom parameter values ... It makes testing a DataEngine really rather easy. It also makes it really simple for someone writing a component that will use a DataEngine to see what is available and how it all fits at runtime.

There are still some TODOs, like providing per-DataEngine/per-Service runtime documentation and drag-and-drop from the sources view, but it's already a useful tool. You can find it in kdebase-workspace along with the rest of Plasma.

The Plasma team has a few other tools available as well, such as plasmapkg (for managing Plasma packages) and plasmoidviewer (a stand-alone Plasma widget viewer), and we have a few more planned. The goal is to continue to encourage people working on the Three Disciplines (code, art and interface) to work together while maintaining their independence.