Why do we have a shared source code repository for KDE development?
That is not a rhetorical question. :) Perhaps even stop reading at this point for a moment to think about it a bit.
Benefits, imho
For me the reasons for having a shared source code repository are many. Some are historical (back in the day, pretty well everyone self-hosted due to lack of credible options elsewhere) and some are convenience related (I always have a place to start a KDE related project without much effort thanks to svn.kde.org). These reasons are not particularly compelling to me because they are either not benefit related or are easily fulfilled via other options out there, particularly as project hosting options such as gitorious.org have gotten as good as they have.
There are other reasons, however, that have everything to do with the source code being close together that can not be so easily replaced in my opinion.
There is, first and foremost, the joint issues of barrier to entry and transferability of knowledge. It is amazing that if I want to hack on a given application or library that is produced by the "central" KDE community, there is exactly one place I need to look to, there is exactly one workflow I need to figure out and there is exactly one set of tools I need to learn (cmake, svn; maybe bugs.kde.org and reviewboard). Even our APIs tend towards, quite purposefully, similar idioms for the same reasons: it makes it easier to get up to speed with them.
What we learn in one KDE experience, we can apply again immediately to another. Even as we lower the barrier to first entry to KDE, there has historically been a near-zero barrier to entering a second, third, fourth, etc. KDE project.
It also means that when I want to fix or improve something, as I did with katepart for the 4.5 series, I could dive in immediately. That was the difference between me accepting a poorer solution, working around the problems and actually reaching out to fix it. Admittedly it wasn't a huge contribution, but for the use of katepart in plasma-desktop it was pretty important and it was, after all, a contribution.
We are able to coordinate things like release management much more easily as well: there is exactly one place to look for the official sources, one place to deal with tagging and other administrivia. We have entire workflows built around being able to land code in playground, move it to kdereview for feedback from any interested party and then into trunk proper. We catch more localization issues this way than probably any other single mechanism. These workflows, in turn, are documented on Techbase for people to read, learn and use as guides to getting involved.
There is also the issue of simple visibility: when something lands in a shared space like svn.kde.org many more people notice than when it lands in gitorious.org. For those who watch the commit lists, or when it was alive the commit digest, the ongoing development of a project really keeps it in our sights. It also helps us notice when it isn't being developed so actively. Visibility is a medium of passive communication.
Without a common set of tools, including a revision control system, all of the above is eroded. We begin to lose the things that allow us to easily behave as a coordinated community when it comes to creating software.
Why this question now?
The reason I ask the above question at this point in time is that KDE developers are voting with their feet and moving to git, one project at a time. Moving to a revision control system that gives us better tools is not a bad thing. Quite the opposite, in fact. We also should not try and shut our eyes to the fact that git is becoming, slowly and surely, the new consensus (which does not mean unanimous :) choice for KDE developers. What is interesting about this trend is that it is happening piecemeal: one project here move, one project there moves.
Fortunately, most projects are moving to gitorious.org where we have a kdedevelopers group that can be added to a project to allow some measure of group involvement again. It is inescapable, however, that our community is slowly dividing into two pieces when it comes to where our source code lives, like two continents moving apart on separate tectonic plates. (What's really unfortunate is that some projects have moved to completely different revision control systems that throws them right to the fringes of our community in terms of cross-project activity.)
Let me open about this: I'd like to see plasma hosted in a git repository. There are, however, no benefits to that which trump being close to the rest of the KDE community's efforts.
Other risks
For those moving to gitorious.org, there are other risks involved. While gitorious.org would be great for a few reasons (proximity to qt, large pool of people who already have accounts there) there are also drawbacks to it. We are currently in friendly discussions with the company behind gitorious.org to have those drawbacks addressed. Part of that solution involves arriving at a financially acceptable agreement for commitments to availability, hosting and ensuring we have access to certain bits of information we must have in case of disaster (e.g. a back-up of user accounts so KDE sys admins could quickly set up a replacement server should something unforeseen happen to gitorious.org). Given that revision control is so central to what KDE does, these are not issues to be taken lightly.
They are, also, not issues that have been resolved yet. I had a conference call during Tokamak 4 with gitorious.org staff and KDE e.V. board members to discuss these issues, so progress is being made towards a decision, but right now it's not a foregone conclusion that KDE will move to gitorious.org. We may well end up hosting a gitorious installation on our own servers.
Having projects jump onto gitorious.org prematurely may end up being more of a hassle than they expect as a result. It is very easy to move a local git repository from one place to another, but then we have to communicate that to all of our contributors and project followers and figure out what to do with things like pending merge requests.
I also see some projects moving to git as a way to work around build system issues: Kate is moving to gitorious because they want to make it easier to build katepart, kwrite and kate outside of kdelibs and kdebase (kdesdk is already simple: just cd into the kate dir and build). That is a huge (and unnecessary) workaround. cmake already supports builds starting from different levels in the host module; that's how we currently manage kdebase as one build vs kdebase-runtime/-apps/-workspace as three separate builds. Working around such issues fixes the problem for some but not for others while causing rifts in our collaboration community.
This is just one more set of reasons to not rush headlong into things just yet.
What are the alternatives?
The alternative is quite obvious in my mind: take the time to coordinate.
There is a git migration project that is making progress week by week. The project is documented on Techbase and can use more hands. It can also use our support, e.g. those two projects who still have svn externals in key places that we can't just ignore (Oxygen and kget) really need to get moving on that (or face more dire results like finding the code migrated one day in a poor state, e.g. with code being copied around or files commented out of the build).
svn.kde.org is not so horrible that we can't live with it for another release cycle. We obviously don't want to live with it indefinitely, however, so a final move-by date must be established, in my opinion, to give all of us something to work with, to ensure all of us that it will indeed be happening within a reasonable time frame. Deadlines rarely hurt. :)
Instead of moving projects one at a time in an uncoordinated, maldocumented mess, let's rally together to coordinate on the move to a new revision control system (which will be git, by popular acclaim) so it gets done well, right and without losing the benefits we've so carefully gathered to this point.
16 comments:
First off everyone I've talked to agrees that Kate moving to Git is just really weird (obviously I haven't talked to Kate devs). An app or a library in the main modules really can't move to Git before all the main modules do, due to release coordination. Plus they're planning to move to git and then back to SVN or something? SVN branches can't be *that* bad to go through all that. :D
That said, when it comes to extragear apps moving to Git we should keep in mind that, since it wasn't obvious from your blog:
* Translations continue to work as normal
* There's still commit emails going out to kde-commits, and the old SVN-based commitfilters still work
* Merge requests actually make it easier to work on a KDE project that isn't yours, since there's a standard way for an outside dev to contribute. In KDE it's always been the case that just because you have access to commit to everything doesn't mean you do so without somehow getting in contact with the devs, but the mechanism hasn't been standardized (IRC, some mailing list, private emails...).
In short I don't think anything has been degraded by Konversation, Amarok and shortly KOffice and KDevelop moving to Gitorious.
On the issue of "what if KDE hosts our own Gitorious instance". The transition from one Gitorious repo to another really won't be a big deal. Obviously if there's a local instance we are *already* going to have to solve the problem of creating accounts for everyone. Users coming from gitorious.org will *clearly* have an easier transition to another gitorious server then users coming from SVN.
So I agree with your conclusion that we should double-down and finish the transition (join us on #kde-git! please don't join the bikesheds on kde-scm-git!), but I really see no disadvantage in the non-main module projects moving over to Gitorious in the meantime. It makes the final transition easier and it increases the community-wide knowledge of git and gitorious.
-Ian Monroe
Also I really like the idea of deadline. :)
Hi Aaron,
I, as a basic KDE user with very limited code contributions, if any, just wanted to tell you how lucky KDE is to have you.
Most of your posts have the distance and the vision that lead KDE well above just a bunch of coders working together. On the particular matter of code base "diaspora", or on others, I'm often impressed and generally agree with your points of view.
I'm not really used to be that emphatic about someone, particularly someone I never met :), but I figured I had to tell you how much I respect your work in general.
Thanks so much !
Alban
And always keep in mind that gitorious is provided for free by a company which has to make money. So it's not a given they accept more and more kde projects moving there.
I expect at one time they either refuse to host a project or have to tell a project to go away or pay because it consumes to much of their ressources.
This would be no ill will on their side. They have to make money and pay the stuff we will use for free.
Mike
From what I can see, as more and more of these pieces are moving to git the developers are simply telling (no screaming) for the transition to happen. Obviously there are details to finalize with a gitorious move, but at the same time... maybe it needs to be sped up a bit a just finished? Git workflow is just far superior to svn that the devs themselves are pushing for it to happen.
As for a customized gitorious on kde servers...It could be something interesting to consider, but then you are just fragmenting the gitorious users. Since Qt is already hosted there and all. Is it possible to just host the kde subproject pages on the kde servers? but still be searchable from gitorious?
@eeanm: "In short I don't think anything has been degraded by Konversation, Amarok and shortly KOffice and KDevelop moving to Gitorious."
everything i mentioned in the blog entry about losing cohesion across KDE is what gets degraded. i don't think you actually appreciated what was written, but rather just agreed with the part you liked and skipped over the parts that don't support what you want. :)
as just one very small data point, i hardly ever follow amarok development now. in fact, at the moment, it doesn't even launch. what brought that about for me? the move to git before the rest of kde.
i understand the need to have a test case and amarok did well there. it hasn't come without it's problems and fragmenting our developer base further is, tbh, rather silly.
those other projects can wait and/or get involved in helping speed up the git process.
that's a much better option that losing testers, participants and visibility.
"The transition from one Gitorious repo to another really won't be a big deal."
not on the technical side (git makes that really easy) but this does mean potentially communicating and coordinating a shift in where development happens *twice* (once from svn.kde.org -> gitorious.org, then to whatever we move to) instead of just from svn.kde.org -> whatever we move to.
if we do move to gitorious.org wholesale, then it's a non-issue. but if we do, that's a real inefficiency that should be avoided if possible as it raises the chances of losing people as transitions happen. as such, it's just not a risk worth taking until we know for sure where we will land.
@Roman Shytan: "more and more of these pieces are moving to git the developers are simply telling "
yes, but this doesn't mean we should do it poorly.
"Git workflow is just far superior to svn that the devs themselves are pushing for it to happen."
what is the "it" they pushing for? right now it is "we want git for ourselves" without paying any attention to benefits of a coherent KDE community code base. that is very short sighted.
the question is NOT "should we move to git", the answer to that is already well known: yes. the question is how and when.
"but then you are just fragmenting the gitorious users."
gitorious.org has definite benefits. but if they are not able to support the needs of KDE, then it is a moot point, isn't it? :)
Well I'm sorry that you don't follow Amarok as closely. But I really don't understand why. Is it because you checked out extragear/multimedia? Well sorry, we only supported this use case even in SVN because the 10 people who used it this way whined like hell whenever the cmake broke.
But I think we've done a good job keeping us in the KDE workflow as much as we've always been. Kde-commits works exactly as before, we still use the bugzilla infrastructure, we're back on the EBN. We already had an independent release cycle
Is communicating a move twice to the (at the most) half-dozen communities that go to Gitorious first really so hard? Especially since all it requires is fixing your git remote. Yes okay, the hard part isn't the technical side of things its the communication with the community. But it is really easy to communicate with the only thing to communicate is a changed URL.
If we do a cost-benefit analysis I think the benefits of having whole communities of git-knowledgeable people able to help out with the git transition clearly outweighs the (in my mind) dubious cost of telling people to update their bookmarks.
I feel like I'm repeating myself, but apparently I didn't address any of your points in my previous worthless comment... *sarcastic emoticon*
"what is the "it" they pushing for? right now it is "we want git for ourselves" without paying any attention to benefits of a coherent KDE community code base. that is very short sighted."
This is just willful ignorance. You surely know much of the inertia for the switch, such as it is, comes devs whose projects have already switched. Or like the KOffice team put *a lot* of work helping the KDE-wide git switch effort before deciding they needed to switch sooner rather then later on their own. And I have no doubt they will continue to help after their own switch.
@Ian Monroe: "But I really don't understand why. Is it because you checked out extragear/multimedia?"
no, it's because it's less convenient for me to hold two completely different work flows with two completely different tools: svn with svn.kde.org vs git with gitorious.org.
it moved amarok from being something i could follow very casually to something i had to switch workflows to keep up with. and not just once, but i have to switch which tools i'm using every time i want to keep up with amarok.
since my main workflow remains with svn, and will until KDE as a whole moves over, amarok loses.
"Yes okay, the hard part isn't the technical side of things its the communication with the community. "
exactly. this is a set of social issues. talking about them as if they were technical issues misses the real point. i'm glad we can all see that :)
"But it is really easy to communicate with the only thing to communicate is a changed URL."
then explain why it is so hard to get people to learn these kinds of things from techbase.
it's easy to tell all the amarok devs, sure.
it's less easy to tell the people who git pull anonymously from gitorious.
it's even less easy to tell the people who'd like to do so and find references to this randomly on the internet in their research.
and it's cheeky to ask people to sign up on gitorious.org then a few months later on git.kde.org (in the case we don't end up on gitorious).
"Is communicating a move twice to the (at the most) half-dozen communities "
it used to be one community: Amarok.
now it's several. and growing. with no end point in sight and you even suggesting that all the extragear apps should follow sooner rather than later.
to say that it's "at most half a dozen" is not being honest with ourselves: the scope has grown to "half a dozen" and will almost certainly grow beyond that (if it hasn't already) unless we set some limits and project-wide-migration goals right now.
"If we do a cost-benefit analysis I think the benefits of having whole communities of git-knowledgeable people able to help out with the git transition"
in which case, we should be done with the git transition or see more people actually helping out. maybe i'm missing it (and i do hope i am) but i don't see that.
with my social engineering hat on, i do see how giving ourselves what we want now that doesn't require us to do it right in the first place will actually remove incentive to do it right and keep the process slow.
"You surely know much of the inertia for the switch, such as it is, comes devs whose projects have already switched."
It would come even more so if they were still at the "want to switch" state; given that there is not enough value being placed on the coherent ecosystem to give pause to "just moving" as we can see with, e.g. kate (not to mention the various extragear and apps-that-would-have-been-in-extragear-but-just-went-straight-to-git-instead), i don't see how those projects getting what they want now increases in any way the imperative to keep our ecosystem together.
they already have what they want. why push harder on getting the rest over?
iow, where is the system thinking?
and at the end of day, if anyone really tries to make the case that the move to git is so critical for them that they couldn't instead put a bit of effort into the git migration and wait a couple more months, i'll show you an aseigo who will be laughing in disbelief. ;)
in plasma-land, we've been waiting to migrate and helping out as we can with the git migration from day 1. we're also precisely the kind of project that would benefit immensely from the switch as our workflows already match git far better than they do svn.
"then explain why it is so hard to get people to learn these kinds of things from techbase."
I was talking about switching from one gitorious to another gitorious. I really haven't found URLs hard to explain...
cheeky, okay maybe.
(btw we might switch to git.kde.org even if we hire Shortcut)
"in which case, we should be done with the git transition or see more people actually helping out. maybe i'm missing it (and i do hope i am) but i don't see that."
have you even tried to look? I don't remember seeing you around #kde-git...
"with my social engineering hat on, i do see how giving ourselves what we want now that doesn't require us to do it right in the first place will actually remove incentive to do it right and keep the process slow."
with my historian hat on, I don't remember the year previous Amarok switching being much more active.
"and at the end of day, if anyone really tries to make the case that the move to git is so critical for them that they couldn't instead put a bit of effort into the git migration and wait a couple more months, i'll show you an aseigo who will be laughing in disbelief. ;) "
a couple of more months? how you been paying attention even a little? I'm sure KOffice wouldn't be planning to switch if they thought it would take a couple of months.
"in plasma-land, we've been waiting to migrate and helping out as we can with the git migration from day 1. we're also precisely the kind of project that would benefit immensely from the switch as our workflows already match git far better than they do svn."
Well you could help out by not adding to the drama. It's amazing how many times the same topic is discussed on kde-scm-interest (including this one), it's why I mostly ignore it and stick to #kde-git.
@Ian Monroe: "I was talking about switching from one gitorious to another gitorious."
yes, i know; i think you're underestimating the effort it takes to make these things clear to a broad audience.
right now it's getting more and more confusing overall.
"have you even tried to look? I don't remember seeing you around #kde-git"
i have been triaging with the two projects who have svn externals and have been pushing forward the contract negotiation bits between Shortcut and the KDE e.V. board. i've also been to a couple of the git meetings (though not as many as i'd like). so, while not trying to blow my own horn, i've been contributing more than the average KDE participant in this, though not as much as people such as Chani or yourself.
"with my historian hat on, I don't remember the year previous Amarok switching being much more active."
*sigh* Ian, you are arguing something completely orthogonal to what i'm trying to discuss here and frankly it's not productive.
i completely agree that it's useful to move to git. i'm not sure why you keep bringing that up. it's not even up for discussion afaic.
but i don't agree that doing it in such a random, uncoordinated fashion is the best way to go about it. and -that's- what i'm trying to raise awareness about here, before even more kde projects go off willy-nilly.
It is true that it is too complicated for developers unfamiliar with the code base to set up a working build environment for applications and that application development should have very little to do with Desktop Environment development. The Kate developers stressed that. Techbase explains you 5 different ways to get a KDE build and none of them work out of the box for whatever reasons. Amarok already uses GIT, others will follow. It contains some problems. Every problem or misconfiguration you run into leaves people behind who do not reach the stage of participation in development. As a matter of principle development of applications ought to take place using a stable productive system. Because what could break usually would break.
Git has some good and some bad sides.
The bad sides:
project-silk
gluon
qtscript-smoke
blogilo
kde-forum-mods
scripty
kobby
rekonq
bangarang
They are real KDE-projects, some of them were started in Gitorious, some of them moved from SVN, they are developed by kde-developers, they are using KDE's bugzilla, mailing-lists, irc-channels, wikis, are often mentione at planetkde and some of them are shipped with standard-kde-repos, but they are not listed on the kde-developers-page, although some of them give members of +kde-developers write-access.
Why? I do not know it exactly, but I think it is because the maintainers want to have some kind of control, one person told me that he does not move his repo to +kde-developers because he would not be able to change the distription any longer. In my opinion it is contra-productive to hazard to split the community and to make it much more difficult to get an overview over kde-development.
The User
I think what Ian is trying to argue (without implying that I agree with him) is something like this.
Scenarios, in decreasing order of desirability:
1. KDE gets its act together and moves to git en-masse in a coordinated fashion in a couple months' time
2. KDE projects which really want to, move to git on their own, and everyone else eventually gets around to it as well once they tire of dicking around
3. everyone eventually moves to git, but spends a real long time dicking around on svn in an uncoordinated fashion first
Basically, I think the argument is that by aiming to do 1 you run the risk of 3, and it's better to play it safe and just go with 2; and further that it's actually easier to achieve 1 by way of first doing 2.
The disagreements, as far as I can tell, are:
- Whether KDE is capable of getting its act together and moving over in a reasonable amount of time; Aaron is optimistic on this front, Ian is pessimistic.
- Whether projects who move over on their own increase or decrease pressure/momentum/inertia for the remaining projects to also move over. Aaron says decrease, Ian says increase.
Again, I'm not taking sides; just trying to summarize/clarify arguments. Correct me if I got anything wrong.
As a long-time casual observer, I remember when KDE moved from CVS to SVN (around KDE 3.2 or 3.3, IIRC). The codebase was probably a bit smaller then, but still: it happened, and more or less in a coordinated way. Since it clearly can be done, I guess it will be done; it's just about having a little patience while logistic issues are sorted out so that everyone can jump at the same time.
I think this was the main message Aaron was trying to get across.
Post a Comment