Continuous Delivery(CDe) | ‘Release early, Release often’ (RERO) — part-1

Summary aka Should you read this article?

If you find yourself discussing/selling merits of Continuous Delivery (CDe), to a technical or non-technical crowd, and you are looking for a brief article on merits, requirements, and patterns of CDe, then you should read on.

Over the years, when a software engineer or a software engineer, manager learns with practice from releasing products in a real world, many of those learning are intuitive and some are not. I bet ‘continuous delivery (CDE)’ does not fall in the intuitive bucket, for most of us. It is as counter-intuitive to release more often, and before finishing your product fully as it is to hold your breath while swimming or lean forward while skiing. Stemming from multiple discussions where selling, ‘Release early, Release often’ (RERO)’ [6] is countered with cries of, ‘love your customers’, I choose to invest my sunny Saturday to repeat the plentiful literature on CDE.

In software engineering at least, the proof is always in the pudding, not in the recipe. In a startup, the difference between almost-done(95%) project and a productionalized project is always binary, either you have it or you don’t. If you or your team is inheriting a project, never fall for the trap of ‘This project is almost done/95% done.’ Majority of the time it is a bullshit estimation, and other times it is laziness to not finish the remaining 5%. Unless you work with hourly paid contractors, there is a solid reason why the project couldn’t have been completed. A good engineer abhors, to leave a project at 95% completion. Sunk cost fallacy is all fine, but for small projects, I value engineering satisfaction equally. Far too many people have quit companies for that reason.

Also from our experience, the longer a project takes to go to production, higher chances are there for that project being abandoned.

From these two reasons alone, both of which point to survival, it should be clear that a project is better structured as a set of smaller deliverables instead of one big bang release. From my personal experience of leading the rewriting of eBay’s pricing-engine (the service that powers shopping cart total) over a year to the rewriting of the menu-authoring software for UberEats over 9 months period, I can attest that these projects would have been abandoned without multiple release-to-production events.

In a layman term, CDE is releasing your service and product to customers, in the smallest feasible chunks, without incurring heavy technical debt. This is also called in Uber, ‘We deliver cupcakes, instead of wedding cakes (too slow) or just layers of cakes(no one will buy-it)’. The smallest feasible chunk is that logically minimalistic piece that a customer would rather have it than not. You can call it, ‘better than before’ test. Say you are releasing a calculator web site, it wouldn’t make sense to skip implementing/showing any of the digits but it might be ok to release the calculator site with only ‘+’ operation. If that is the world’s first calculator website, customers would prefer a site with one operation than no-site at all, or simply put it will be ‘better than before.’

Of course, I don’t want to trivialize efforts that go into software distribution, but there will always be a customer segment (beta-tester) who would prefer “something” before other polished things.

Why should you know about CDE

In large companies, big multi-year long project getting shut-down has no big consequence. I have seen company divisions not delivering a single project to the customer over three years, with projects and divisions being shut down. While this may not get people fired, it certainly would slow down your professional growth and learning experiences that come by working on production issues. In smaller companies, this kind of project shutdown can be fatal. As a game player, Diablo-III [5] with 12 years of development lifecycle sort of killed the Diabolo continuity. Now imagine as a developer or team member on that project, how much experience you would have raked releasing a single project between 2001 and 20012. If not for the shareholders, at least for your team members professional growth sake ‘Release early, Release often’ (RERO)’.

Many of us have released a product to production only to discover that customer engagement is not what was expected. In that long famous list of iTunes-Ping, Clippy the assistant and Fire-phone, Google-Plus, every single extra day spent towards releasing the product, was a waste of company’s resources. Of course, the complication is not knowing which product will actually succeed (Slack, Airbnb, Skype, Whatsapp). In such a world it is best to be humble and go with the assumption that, while we hope our product will succeed, there is every chance that what the actual customer wants could be entirely different. The mantra of ‘Release early, Release often’ (RERO)’, will help you with those uncertainties.

One of the most important jobs that the engineering manager has is, ‘project status communication’. Its hard to believe that there is anything like ‘95% done project’, it is a project where 95% of the known problems are solved. It is no guarantee that by the time we finish the remainder of 5%, there wouldn’t be another scope discovery of 30% or 50% of the original estimation. The only project status that is ‘a fact’ is something that is released to production and stable. RERO so that the stakeholders know that the status.

My personal reason for knowing about CDE is it is always a good idea to know something before you actually need it. If life gives me a cheat code to not-fail in my project, I would take it. After all it would take investment of reading one book, in this case ‘Continuous Delivery’ ([3]) by Jez Humble and Farley, and then hope to apply what you have learned to the situations you come across.

What is Continuous Delivery(CDE)

CDE is the art of finding the smallest logically complete piece of software/service that is adding value to the customer’s life, and actually releasing it to the customer, without adding significant tech-debt. The net effect of this is RERO.

i. Find the smallest logically complete component

This requires intense collaboration between engineering and product, and most importantly acceptance of CDE philosophy by all the parties. It is not uncommon for your engineering or non-engineering partners to have never heard of CDE, so be patient. You will also have to think creatively to carve out functional pieces that are logically complete. In UberEats initial days, the money to the restaurant paid was calculated from excel sheet, and the team didn’t wait for an automated accounting system which mostly correct. Restaurants as a customers didn’t mind, as they would prefer to do some business over none-at-all.

ii. Release it to the customer

You also have to believe that the customer would prefer your partially completed product over future fully finished product. But they can’t use it and give you feedback until you actually release it to them. If there are worries about your product perception, you can always start with a smaller group of volunteer customers who are ok to test the product. Not since we were living under rock, larger established companies have been using beta testing to release the product, and if you don’t have such a program then you are behind on the strategy.

iii. Beware of the tech-debt

CDE doesn’t encourage releasing a product that will actually increase the tech-debt so much that it will bog down the future releases. If the compromises to achieve a CDE are bridging the gap between what customers will have now vs what they will eventually, that is one acceptable thing. Examples are putting a banner on your webpage for features not yet implemented, as ‘Coming soon’. It does not mean you release your first version of product in C# when you plan on maintaining it in Java. It does not mean using ‘react native’ when you don’t have any plans of using it in the near future. To be amply clear, CDE is not the same as releasing your proof of concept prototype to the customer. Perhaps the reason why PoCs are encouraged to not to use the same technology as the one used in production. As a devil’s advocate, I will mention, there will be situations when you are ok to take large tech debt that could take half or more of the engineering team bandwidth in order to launch fast, but be aware of such trade-offs.

Deployment in this context can relate to any of the various things such as a full-stack production deployment(mobile/web-gateway-backend-DB), a combination of full-stack, and sometimes just config changes that tweaks the behavior.

Heeding my own suggestion, I will split my write-up in multiple parts, and end the first part here. In the next parts I will write about a. Benefits of CDE b. Patterns and best-practice of CDE c. Pre-requisite for CDE

References

[1]: Fowler on Continuous Deliver — https://martinfowler.com/bliki/ContinuousDelivery.html

[2]: Wikipedia Continuous Delivery — https://en.wikipedia.org/wiki/Continuous_delivery

[3]: Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler)) https://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912/ref=sr_1_fkmrnull_1?keywords=Continuous+Delivery%3A+Reliable+Software+Releases+through+Build%2C+Test%2C+and+Deployment+Automation&qid=1553381361&s=gateway&sr=8-1-fkmrnull

[4]: Four Principles of Low-Risk Software Releases http://www.informit.com/articles/article.aspx?p=1833567

[5]: Diablo-III https://en.wikipedia.org/wiki/Diablo_III

[6]: Release early, Release often’ (RERO) https://en.wikipedia.org/wiki/Release_early,_release_often