Fabio Manganiello on Nostr: Technological innovation is increasingly held back by the engineering time wasted in ...
Technological innovation is increasingly held back by the engineering time wasted in managing dependencies, breaking changes and deprecations.
It seems to me that software has accelerated it process of decay and expiration without thinking much of it downstream dependencies. And, in a world where dependencies have multiplied compared to a few years ago (my node_modules directories can witness that), it means that engineers have to spend most of their time to fight against transitive decay of their own software.
Take the example of my main Platypush dashboard in my living room. A couple of years ago it sported a script that could automatically take pictures (and show an on-screen preview) with a picamera and the press of a button and send them to my mobile device.
Nowadays? Well, that picamera library has been abandoned and it’s not even included on recent versions of RPi OS. And the new one doesn’t yet have many of the features of the old one. And the Google Assistant library that I used to process the “take a picture” voice commands has been abandoned by Google. The Pushbullet API that I used to send pictures has gone through several breaking changes. The Android Wear mechanisms that showed photo previews on my smartwatch from a notification are gone. The Bluetooth buttons that I used to trigger those commands are no longer produced, and the library has been abandoned. I switched to Zigbee in the meantime - but hey, zigbee2mqtt 2.0 has also a lot of breaking changes that I still need to adapt into my code.
Or take the example of the code I’ve been working in my daily job lately. It has dependencies that want the AWS SDK 2.0 (namely OpenSearch). But importing the SDK 2.0 means breaking other AWS dependencies (namely Neptune), and it also breaks our logging dependencies. So in the meantime we stick to the SDK 1.0, knowing that it’ll be deprecated by EOY, and we’ll have again to spend time later to manage the migrations.
And let’s not even get started with the tragedy of frontend frameworks. I start a new Vue project more or less every 6 months, just to acknowledge that the way I started it 6 months before and the frameworks and patterns I use are completely deprecated, and I should migrate to something new.
And I could report many more examples just from my experience as a software developer in the past couple of months.
Looking back at the past year, I feel like 90% of my time has literally been spent preventing software from rotting because of rotting dependencies.
I don’t recall things being like this ~10-20 years ago.
The number of removed/archived repos on Github seems to also point at the fact that software has a shorter lifespan nowadays. And, on top of that, breaking changes are pushed through without much thought. And developers downstream are left alone to fend off this eternal stream of change just to keep the lights on and prevent their projects from decaying.
I have the impression that this is leading to widespread developer fatigue, frustration, and in general a sense of not being able to innovate.
It’s even hard to explain to non-tech people that the reason why you can’t give them the technology that they want is because 10 of your upstream dependencies have either been abandoned or changed so much that they’d require you to change nearly all of your code, or that the concurrent breaking changes introduced by some of your dependencies mean that you have to find the right alignment of versions that would allow you to keep the lights on.
It seems to me that software has accelerated it process of decay and expiration without thinking much of it downstream dependencies. And, in a world where dependencies have multiplied compared to a few years ago (my node_modules directories can witness that), it means that engineers have to spend most of their time to fight against transitive decay of their own software.
Take the example of my main Platypush dashboard in my living room. A couple of years ago it sported a script that could automatically take pictures (and show an on-screen preview) with a picamera and the press of a button and send them to my mobile device.
Nowadays? Well, that picamera library has been abandoned and it’s not even included on recent versions of RPi OS. And the new one doesn’t yet have many of the features of the old one. And the Google Assistant library that I used to process the “take a picture” voice commands has been abandoned by Google. The Pushbullet API that I used to send pictures has gone through several breaking changes. The Android Wear mechanisms that showed photo previews on my smartwatch from a notification are gone. The Bluetooth buttons that I used to trigger those commands are no longer produced, and the library has been abandoned. I switched to Zigbee in the meantime - but hey, zigbee2mqtt 2.0 has also a lot of breaking changes that I still need to adapt into my code.
Or take the example of the code I’ve been working in my daily job lately. It has dependencies that want the AWS SDK 2.0 (namely OpenSearch). But importing the SDK 2.0 means breaking other AWS dependencies (namely Neptune), and it also breaks our logging dependencies. So in the meantime we stick to the SDK 1.0, knowing that it’ll be deprecated by EOY, and we’ll have again to spend time later to manage the migrations.
And let’s not even get started with the tragedy of frontend frameworks. I start a new Vue project more or less every 6 months, just to acknowledge that the way I started it 6 months before and the frameworks and patterns I use are completely deprecated, and I should migrate to something new.
And I could report many more examples just from my experience as a software developer in the past couple of months.
Looking back at the past year, I feel like 90% of my time has literally been spent preventing software from rotting because of rotting dependencies.
I don’t recall things being like this ~10-20 years ago.
The number of removed/archived repos on Github seems to also point at the fact that software has a shorter lifespan nowadays. And, on top of that, breaking changes are pushed through without much thought. And developers downstream are left alone to fend off this eternal stream of change just to keep the lights on and prevent their projects from decaying.
I have the impression that this is leading to widespread developer fatigue, frustration, and in general a sense of not being able to innovate.
It’s even hard to explain to non-tech people that the reason why you can’t give them the technology that they want is because 10 of your upstream dependencies have either been abandoned or changed so much that they’d require you to change nearly all of your code, or that the concurrent breaking changes introduced by some of your dependencies mean that you have to find the right alignment of versions that would allow you to keep the lights on.