something i've learned in my time as a developer (yes, thankfully i've learned one or two things in that time ;) is that all code bases will, if developed on long enough and added to enough, will grow crufty. the code will start to creak and show signs of age.
partly this is because while coding you are not only doing but you are also learning, especially if it is something that hasn't been done by the team involved 100 times before. so you look back at things written years ago and think, "i would do that differently today." this is true of most creative endeavours.
partly this because technology moves on: the tools improve, new languages emerge, new development methodologies and strategies appear, new algorithms are arrived at even. these advancements open new doors, some of which would have been nicer earlier on in the development of the code base and some of which get added, or even just bolted on, to the existing code base.
partly this is because bug fixing, optimizing and "organic" feature additions end up muddying the code. these external and valuable improvements often come with a cost on the clarity, beauty and even at times quality of the code base.
so eventually in walk the knights of the great refactoring. this is often a turning point since if the code was written with care, the refactoring can often go deep and produce good results. the code base can keep its features, bug fixes and optimizations; in fact, new ones can be added .. with a resulting code base that is tidy, clean and ready to get messed up again. ;) it's sort of like spring cleaning and getting all the books back on the right shelves, the cups and plates in the right cupboards and all the dust out from the corners.
but if the code base has been neglected or wasn't built with care, sometimes refactoring can be extremely difficult, even impractical. this is especially true when the refactoring is intended to not only perform spring cleaning but also to pave the way for a new set of capabilities. the more the new capabilities diverge from the existing, the more robust and flexible the code base needs to be to withstand the gales of change that it will face.
that is why the early structure of a long lived code base is so important. a lot of recent research on development methodology surrounds quickly creating what are often one-off productions with enough quality (there are economic drivers pushing this research over other sorts). it seems that creating successful long lived code bases is still a bit of a black art, though. at least, judging by a lot of the code i see getting written in industry it is. ;)
i don't have definitive answers at all, but i've learned that i experience less pain years later in a project when the code base has started with a well defined purpose (not just a requirements doc), clear implementation strategies that are maintained fairly strictly, code level consistency and a design that allows room for realistic possibilities without making the current implementation impractical (that "don't over design, but don't under design" balance).
in kde 4.0.0 we are releasing a number of new code bases as well as some old ones that have been spiffied up and even refactored in places.
for the substantial quantities of code we've brought through from kde3 (or even kde2) into kde4, i think it says a lot about the care and thought that went into the initial creation of those classes, libraries and applications. there are many places i've looked at code in kde 4.0.0 that has its roots in kde3 that looks and performs better than the kde3 version. to simply bring along code year after year is one thing; to add to that code and improve the feature set is another; to do so and have the results shinier than what was there .. impressive =)
the new code bases were what really got me to thinking about this in the shower yesterday, though. how will they stand up over time? what care will we apply to them as we take the kde4 journey over the next decade? when will it be likely that refactoring will become a necessity? do they have enough capacity now to absorb future development efforts?
while i considered things like the new stuff in kconfig that will continue to need love, the new pillars of kde libraries, etc ... much of my thoughts were turned towards libplasma and the technologies it also needs to see come along, such as the new dxs and openID stuff.
to be honest, there are some internal complexities in libplasma right now that i'm not overly fond of. most of these are the result of short-term scaffolding we erected whilst waiting for things like qt 4.4 to become available, so that doesn't bother me too much as it will soon get washed away rather nicely.
still, one of the things i will be doing during the post-4.0.0 development times is to go through libplasma with an eye to keeping it maintainable and extensible. it's replacing code that had a really good 7 year run, and i want it to be able to beat that. given the heritage of what it builds on, we have learnt from and what it is superceding ... it needs to be awesome. =)
on a related note, i found some hand written notes in a sketch book from 4 or so years ago that were full of thoughts on kicker and how to improve it. it was interesting because it was easy to see the start of the ideas that would become plasma in so many of the lines on those pages. when writing and sketching those thoughts down i obviously didn't yet see where it was heading, but looking back on them now one can see the path was already laid out just waiting to be discovered and then walked on.