Has software development become too complex?
Jaco van der Merwe, CEO of DVT.
When I started my career in IT, I was fortunate enough to build and deploy several custom enterprise software solutions. In my first real job as a software developer, I coded in Delphi using an Oracle database. It was a relatively large system with a fairly complex data structure, but with a very small team we were able to get it into production in less than six months. These days, systems of similar size take much longer to grow, often with much larger teams than before.
It’s not like our small team of Delphi developers back then were so special. I’ve seen the same with teams of Visual Basic developers, Oracle Forms developers, and many more.
So why does it take so long to create software? Or more accurately, a question that many of our customers ask themselves: why is software development so expensive?
With the growth of available libraries, cloud-based services, open source frameworks, and advancements in software development tools, one would be forgiven for assuming that the time required to develop software would decrease, not increase, like this. would appear to be the case for the last decade or so.
I can’t help but wonder if software development has gotten too complex? The reality is that software development is orders of magnitude more complex today than it was a decade or two ago.
Why has software development become so complex?
The short answer to why software development has become so complex is due to the additional demands we have on software today compared to a decade or two ago.
These additional requirements of our software were first described by computer architect Fred Brooks in 1986, in an article titled “No Silver Bullet”, in which he observed that software engineering did not produce the same gains in software. productivity than hardware engineering. Brooks described that in software development there were two major hurdles to overcome: essential complexity and accidental complexity.
Essential complexity vs accidental complexity in software development
The core complexity is, in essence, the business problem that needs to be solved.
It describes what the system should do, or the “scope” of the business problem. While there may be some flexibility in the scope of the trade problem, we have to solve the complexity essential to have a system that solves the trade problem. While the real business issues have changed over the years, they haven’t become more complex. Essential complexity is not our challenge.
Accidental complexity is the complexity that developers unintentionally create for themselves when trying to solve a problem. For example, if a team of developers are highly skilled and experienced in using Angular as a web framework, but that same team now has to develop a system using React, it adds considerable accidental complexity to the system. software development project. Another example could be a team used to developing web solutions and now needing to develop an interface for mobile devices.
The hard life of a software developer
Learning a new development framework or solving the challenges of producing software for a new type of device isn’t exactly a new challenge in software development. I have learned to code in several programming languages over the years of software development. So what exactly has changed between my days of simple two-tier, client / server software development, and today?
The answer is that the demands we have on software today have resulted in additional accidental complexity in exploding software development.
Here are some of the accidental complexities a developer faces today that weren’t there 10 or 20 years ago:
Multiple devices and platforms
Developers today have to develop software that runs on multiple devices and platforms on form factors that include desktops, laptops, tablets, phones, watches, and IOT devices. While there are various frameworks to tackle this challenge, it still adds considerable accidental complexity to a software development project.
The first time I experienced this complexity was when my Microsoft ASP based web system worked fine in Internet Explorer but behaved completely poorly in Netscape!
Containerization is great for wrapping or packaging software code and all of its dependencies so that it can run uniformly and consistently on any infrastructure, but does it add complexity to a project! The first time I saw this was with the introduction of J2EE.
When I first started working with J2EE, I understood the value of an application server, but saw how quickly the speed of getting software into production slows down.
User experience (UX) and user interface (UI) design
Just 10 years ago, most developers hadn’t heard of UX / UI design. It did not exist as a software development practice.
This meant that the software was generally functional, but sucked from a UX or UI perspective! Great software today has professional UX / UI designers behind it – to create designs for the software that make it very easy for users to use. UX / UI design is essential to software development, but UX / UI design implementation adds more complexity.
Today, all systems are vulnerable to cybersecurity threats. This means developers need to design with cybersecurity in mind and have their software tested for security vulnerabilities.
Source control is by no means new to software developers, but it has not been so widely adopted in the past or has not addressed the complexities of multiple branches of code and the complexities of deployment and deployment. continuous integration.
The ability to respond to changing business demands
Continuous deployment and integration and more regular deployments help create software that can respond quickly to business needs and generate business value faster. No debate.
Just a lot more complex.
Software capable of handling tens of thousands of users, sometimes almost simultaneously, was unheard of two decades ago. Add to that the ability to work with virtual hardware in cloud-based environments that scale up or down as demanded, and you have some very impressive software. This very impressive software forces developers to deal with a lot of additional accidental complexity to understand and work with the software development tools and architectures and design patterns to support this level of scalability.
These days, every user expects flawless software. This results in a considerable effort to reduce software defects. Test-driven development, test automation, performance testing, etc. all add up to better software, but certainly impact how quickly the software is deployed to users.
In my experience, whenever a software development team starts a new project, they have to manage and learn a lot of new technologies (languages, frameworks, design patterns, tools, vendor software, etc.).
Developers love to learn new technologies, but the rate at which they are deployed is currently slowing software development.
How to deal with the complexity of software development
There are many more examples of the complexity of software development than those listed above. Does this mean that software development will become more and more complex to the point that it will no longer be feasible or is worth it?
I believe that we are in a time when the complexity of software development is increasing, but the current complexity will be reduced through the development of new technologies. Low-code software development tools are already making it easier to solve certain types of business problems, and AI will automate some of the complex tasks that developers face.
But for now, think about the ever-increasing accidental complexity that software developers face in meeting the demands we have for software.
Finding a software development team that knows when to reduce complexities for efficiency, productivity, and quality means turning to the right people for advice. Look for people who have had multiple successes, not just one-off successes. The company should also have established remote best practices and reliable distribution of technology within the engineering team.