Friday, January 04, 2008

talking bluntly

Now that 4.0.0 is tagged and out and that bit of worry and concern is behind me for the moment, I wanted to take a moment to talk really bluntly about 4.0. In particular, I'm going to address some of the common memes in fairly random order that i see about kde 3.5 and 4.0. I'm going to speak bluntly (though not rudely =) so prepare yourself ;)

Meme 1: What is the future of 3.5?



This year, as with most years since KDE3 emerged, there have been huge deployments of KDE 3 based software. These deployments will not shift for years to come, no matter what KDE4 is. This is because large institutional deployments (government, corporate, educational, etc) typically have 3-7 year cycles (sometimes even longer) between major changes. Patches and security fixes? Sure. Major revamps? No. This alone ensures that KDE3 will remain supported for years. Why? Because there are users. That is how the open source dev model works: where there are users, there are developers; as one declines so does the other. The developers tend to be a step ahead of the users for software that is progressive, but you'll also find that they have a foot in the here and now too (as well as the past, often).

KDE3 is still open in our svn so that bug fixes, security fixes, etc. can continue to be made. KDE 3.5.x is a rather solid desktop system and really doesn't need a huge amount of work given what it is today; the work to move it to the next level is what we refer to as KDE4, of course. This means that the efforts needed to put into it aren't huge to keep it viable. However, efforts that do go into it are welcome.

While the core KDE team will continue to concentrate our work on KDE4 since that is the long term direction of things, it is fully expected that our partners (which include some KDE core team members as employees/members) will continue supporting and even developing on KDE3 issues. The central project will also be around to lend a helping hand with advice and what not; I did that for a person the week before I left for holidays in December, actually, so it's not wild hypothesis but solid theory.

For those familiar with the open source method, the above probably sounds .. well .. obvious. That's because it is .. for those familiar with the open source method. We will find in this blog entry that many of the concerns people raise come from not acknowledging how Free(dom) software is created via the open source method.

Meme 2: KDE 4.0 isn't what a business would do



I've read exactly this statement with those literal words, but I've also read and heard what are essentially the same things put slightly differently. Well, not to play Captain Obvious too much here, but: KDE is not in the business of proprietary software. There are two parts to that statement:

KDE is not a business: we are not selling a product to the mass market. We are a development team creating the resource which can be sold to the mass market. This is an important distinction since we go through an R&D process that is very open, something that a business would have a hard time doing. We also don't pay volunteers per hour, commit or line of code. There are many things, you see, that we don't do that a business does, and vice versa. The fact that people are getting confused on this point shows how well we've done presenting KDE to the world, but we're not a business and we're not about to start pretending to be one to satisfy chin-waggers at the expense of what works for us.

... and that's a salient point: By asking KDE to behave like a proprietary company these people are asking KDE to abandon what has worked for us all these years. They are asking us to abandon our identity, to cease doing what resulted in the Free software desktop going from non-existent in the mid-90s to parity in just over 10 years. Remembering that we started 15 years (and multi-billions of dollars) behind our competition that's a pretty impressive success story.

At the same time, KDE works with business. We have relationships with companies at many levels, technical and otherwise. In order to provide good guidance to our partners we've been pretty blunt about what 4.0 is. That is because while KDE itself isn't a business, we have a large business ecosystem around us. We are a good business partner, even if we ourselves aren't a business. I know, this is rather paradigm shifting for a lot of people out there, but that's what makes it fun and enjoyable for so many of us.

KDE is not a proprietary software product: this is another obvious statement, but it's one people seem to forget. While there is proprietary software that gets written using KDE technology, KDE itself is not and never will be proprietary.

In the open source method you release early, you release often. By doing so, a progression is presented that people can follow with fairly blunt (often overly pessimistic) guidance along with it, e.g.: "foobar v 0.0.1: will eat your children". In theory you can't do that with proprietary software due to the distribution mechanism and economic repercussions (though many companies do anyways), but with open source it is exactly what one must do to get the production wheels turning.

KDE 4.0.0 is our "will eat your children" release of KDE4, not the next release of KDE 3.5. The fact that many already use it daily for their desktop (including myself) shows that it really won't eat your children, but it is part of that early stage in the release system of KDE4. It's the "0.0" release. The amount of new software in KDE4 is remarkable and we're going the open route with that. Which brings us to the next meme:

Meme 3: Just keep releasing alphas until it's ready



Ah, the "until it's ready" idea. Some would say 3.5 isn't ready; software never really is from a perfectionist's standpoint. It's so complex and full of ever springing promise that one can never reach that point of perfection; usually we are just happy with "better than good enough" and call it a day at that point.

KDE 4.0 isn't yet "better than good enough"; so why don't we just release more betas? When one perpetually releases alphas/betas a few things happen: people don't test it aggressively enough, third party developers don't get involved, core developers continue doing blue sky development rather than focusing on release qualities.

Between the rc's and the tagging of 4.0.0 the number of reports from testing skyrocketed. This is great, and shows that when I assert "people don't test when it's alpha or even beta" I'm absolutely correct. This is not about tricking people either: people seem to forget that the open source method is based on participation not consumption. So testers look for a cue to start testing; that is their form of participation. "alpha" and even "beta" is often not enough of a cue, especially today when so many of our testing users are not nearly as technically skilled with the compiler, debuggers, etc as the typical Free software user was 10 years ago.

The KDE4 libraries are ready for application development, as testified to by the quality KDE4 apps that exist today. However, third party application developers tend to be a conservative lot, and rightly so. They wait for user base migration, they wait for stability in the APIs, etc. They want to know when to start working with the new awesomeness, and for most of them that isn't "alpha" or "beta". The libraries crossed that stage in quality and reliability many months ago and so it is only fair to mark them as such.

Finally, the amazing maturation at all levels of KDE 4.0 software that has happened since the last beta shows just how focusing developers off of blue sky development and onto release quality code is important. The delta speaks for itself.

Meme 4: KDE doesn't do time based releases



When I hear this statement, I know I'm either dealing with someone who has been around the community for less than 2 years or has a long term memory problem. KDE has traditionally done time based, not feature based, releases: the project would set a target date, people would create a list of expected features they can do in that time, features that didn't make the target date would get punted to the next release.

In my time with the project we have done 2 feature based releases: both for major overhauls of the entire system. The first one resulted in the pedigree that would become KDE 3.5. The second one is KDE 4.0.

In that same time period we did 8 time based major releases (2.1, 2.2, 3.0, 3.1, 3.2, 3.3, 3.4, 3.5) and countless time based minor releases (8 for 3.5 alone). In each of these releases a target was set and in the overwhelming majority of these cases that target date was hit.

With 4.0 behind us and marching towards 4.1, we'll be back to these time based releases.

Meme 5: What's the quality of KDE 4.0?



KDE 4.0 rocks in a number of ways. Whether one looks at the new frameworks (solid, phonon, akonadi et al) or the revamped existing ones (kconfig getting multiple back end support, the UI-less kdecore), or examines the apps like okular or kdeedu or the games or dolphin or ksnapshot or konsole (ok, I won't list every app) or many of the new workspace features like composite and widgets or the new artwork or ... you get the picture. There's a lot that is just amazing.

What leaves people wondering about quality is that there is a disparity between our stated end goals and 4.0. This is, to be blunt, due to a lack of experience on their part: most people have never been involved in the creation of something great. We're involved in making something great that will end up spanning a decade of effort and be used for even longer than that. To be able to accomplish such a thing one requires the ability to see beyond today and into the uncertain future. They also need to be able to adjust and shift that vision as things evolve (ergo the shift from tenor to strigi/nepomuk, even though the end result is essentially the same ideas). It is simply not possible, without extreme luck similar to winning the lottery, to create something great without that vision. This is not my idea, this is the result of pretty much every bit of research and practical analysis from the business operations world.

More on the concept of vision



To re-affirm: our stated goals for KDE4 remain and they haven't gone anywhere. In fact, KDE 4.0 is the first proof that this is not only vision or, worse, vapour: we're putting it into action. However, long term vision is not met in a short term effort. Vision of the end is what directs immediate efforts into mid-term and eventually long-term successes. We will get there, and probably beyond what we currently imagine, with the releases that will follow. Each stop along the way will rock harder, and none of them will suck. Importantly: nont of it would be possible without the vision.

So it's ironic that some would see the vision we are committed to as reflecting on us in a bad way, since it's what is enabling us to deliver a great product not just today but in the future. Companies usually keep this vision internal and you never really get to see it from the outside. Sure, the ghosts of the vision get communicated eventually through marketing slogans (if they do their job right, anyways =), executive communications, AGMs, etc. but generally it's internalized.

KDE is an open project, however, so we can't talk about our vision without it getting "out there". There is simply no way for us to "keep our light hidden beneath a basket". So while it's ironic, it's not an unexpected consequence: most of our audience is simply used to being on the outside. They are not used to freedom, they are not used to openness, they are not used to being privy to the internal world of others.

For better or worse, there is no way we can shield them from being able to see our vision. As a project we need to talk about it with each other a lot: openly, loudly, even argumentatively at times. So everyone gets to see it, and some mistake the vision creation and maintenance process for marketing effort or spin; they are unrelated. What's funny is that community news sites will actually pick up the evolution of our vision as news events; it's undeniable that people find it interesting, which is pretty cool.

So to achieve what we want to, I've come to realize that I'll take it on the chin, so to speak, from some people who aren't able (yet?) to internalize what this process is all about. I can't in good conscience suggest we divert in response to this particular set of feedback, though, as it would cripple the project in the long term.

Many companies in Europe and North America have been criticized by the business management community for a couple decades now of not investing enough in mid-term (let alone long-term) projects. They have trouble doing so due because they allow themselves to be led by the nose by the financial and consumer markets copuled with vision-lacking internal assignment of resources. Yes, you're reading that right: listening to the short term consumeristic demand of the populace has been a major component of the march towards much of the stagnation and crappy products and services we get to deal with today. Ignoring the short term is foolish, but not investing in the mid-term is equally so.

I don't expect the populace to suddenly get long-term vision; I do expect serious organizations to stop setting their agendas by the flawed clock of the short term thinking that (inevitably?) dominates large societies of people.

To bring it into high-relief: KDE3 is our current product line for production, and KDE4 is our mid-term production line. For there to be any KDE worthy of succeeding KDE 3.5, we needed a mid-term project. No short-term project would cut it. We're at the beginning of where we can bring KDE4 into "current produce line" condition, which is to say that KDE4 is that transition period from mid-term to short-term project. That's exciting, and one more reason 4.0 rocks.

Prologue Epilogue



To close I'd like to recognize that KDE as a project is not perfect; we are made of fallible humans engaged in an amazingly complex process. All the same, the people involved are pretty amazing and competent. We're on a good path right now as a result of those people. If you find this process hard to understand that, try to adjust your assumptions and deeply internalize the concepts of the open source method since that is our guiding light. In spite of some of you finding it hard to understand this process, we won't betray you by switching to an inferior plan just because it fits your assumptions better. Even those who are most concerned today will thank us further on down the road.

Ok, enough about that. I've said what needs to be said and won't say more about it from here on out. I have a huge backlog of blog topics to cover that are more interesting and positive in nature. I'll do my best to keep them shorter than this one .. but no promises there ;)