Alan Greenspan has long praised computer technology as a tool that can be used to limit risks in financial markets, but recently he acknowledged that the data fed into financial systems was often a case of garbage in, garbage out, indicating that this has led to huge trouble. Have bad IT systems been deployed elsewhere on this planet? Yes. Will the world continue to do so? Yes. Why?
If you look at a number of ideas for software systems, then some of these will definitely not make sense, some of them will make a lot of sense, and then there is a huge group in between. In this middle group, it is difficult to evaluate them, sometimes even after deploying the software. A famous person once said, it is often easy to measure things, but difficult to understand what is being measured, and this applies very well to software.
The dot com boom was based on the assumption, that the productivity gains in Software are so huge, that the value of many things would go up, a lot. Expectations were too high. Why? Because software doesn't deliver that kind of results, that fast. Resources for Software projects are allocated for the wrong projects, and many projects are doing something wrong.
Why cannot we just do it the right way? There are many reasons, but the single most important reason is, that there is no single right way that fits all purposes, and it is therefore impossible to make one recommendation for all. The best "single right way" that I have seen so far is real "Agile", meaning that you need to adapt all the time. In other words, a very difficult concept to teach. And now we're at the core of why not all Software projects are a huge success:
Humankind is unable to do everything right. We will never get rid of that middle group of software systems, where we don't really know if they were a success or not.
Software is not much different than other technologies, like chemistry or electronics: Some people are making huge progress, others not, and when the good ideas get deployed, world productivity improves. Some software is great, but software as such is not a silver bullet by itself.
I think we should start to try to identify the biggest successes, in order to learn from these. Maybe we should have a Nobel prize for Software?