I've been thinking a lot about this topic lately, especially since I'm just finishing up a very intense 4-month development cycle. You can't call a 4-month project a death march, really - perhaps death sprint is a more appropriate term?
At any rate, I think that we're seeing the beginnings of a shift in software development. You have to navigate through the hype from the gurus and the quacks to see it, but if you cull out what's interesting and important from current fads and fashions, you start to see something emerging from the following trends:
- Extreme/Agile Programming
- Open Source
- Outsourcing
- Web Services
- Generative Programming
- Software "Factories"
- The Rise of Dynamic Languages
- New Internet-Based Innovations (e.g., RSS, Podcasting)
- Microsoft's Slide Into Irrelevance
On the face of it, a rather cliched list. But each of these trends will contribute something crucial to the art/craft/science/business of creating software - and those contributions will not be the aspects that are getting the most hype right now.
Each trend deserves serious consideration, and I'll do that over the next few months. But for right now, I'll give you both the punch line (since every attempt to predict the future is a joke) and the setup.
The punch line:
There will never, ever be a substitute for talented software developers. No technology, no process, no other innovation will ever change this.
The demand/need for software will continue to accelerate. Software will disappear into the fabric of our everyday lives as virtually every man-made device becomes a computer. This will fuel the demand for effective integration of every thing.
The level of abstraction will continue to rise and so will the capabilities of our applications - but this will not significantly (or even slightly) lower the complexity of creating usful software.
The new software will be designed and created at an extra level of indirection, and we will have to evolve from programming systems to meta-programming of meta-systems.
This is good news for talented people who keep their skills sharp and relevant. Only really smart people with really good skills can tame software complexity.
The Setup
Christopher Alexander's single overriding rule for A New Theory of Urban Design:
Every increment of construction must be done in such a way as to heal the city.
I've quoted this before, in a slightly different context. At the time I thought it only applied to software maintenance. But I've come to an important realization:
Dividing development from maintenance is artificial and arbitrary. All software creation is software maintenance, and vice versa.
Alexander again:
...the word "heal" must be understood in its old sense of "make whole". It includes not only the repair of existing wholes which are there already, but the creation of new wholes.
Thus, Alexander's overriding rule is directly applicable to software. All increments of software creation must be done in such a way as to heal the software - that is, to make it whole..
Refactoring is only the best and most obvious example of this. In fact, all aspects of software creation should be indirectly driven by this overriding rule. Test-driven development is a way to heal the software. Inspection is a way to heal the software. Patterns are a way to heal the software. Good software engineer processes are a way to heal the software. Aspect-Oriented Programming is a way to heal the software.
All of the worthwhile techniques of software creation are deeply interrelated parts of a process that produces an organic wholeness in our software. When I say "process" , I don't mean it in the way we usually use it in software engineering. I mean something more fundamental - not a series of procedures, like a manufacturing process, but a set of actions, functions, and changes that advance us towards a goal. Exactly what this process is remains unknown; we can see and understand fragments of it - the trends listed above represent those fragments - but we don't know how to describe it yet, much less practice it.
This process will be interpretative, not mechanical, and thus can only be discovered and executed by talented software developers. All of the mechanistic aspects of the process will be automated, leaving only those aspects that require taste, judgement, and talent.
