|title:||Data-Intensive Software Development without Downtime|
|keywords:||RDBMS, Oracle, Postgres, Downtime, DSL, Java, Kotlin, Spring, Kubernetes, Docker, Rest AP|
|topics:||Case studies and Applications , Dependability, security and performance , Software Technology|
Maurice van Keulen
Traditional banking has evolved from manual labour to digital commodities. In order to stay relevant in the market of financial institutions, it is a necessity to rapidly evolve banking products to adapt to business requirements and adhere to market standards. At this very moment, banking software is released in a planned fashion and infrequently. This release process hinders the current management strategy in becoming an agile company, where adapting and learning fast are key. At this point bringing new software to production that imposes structural changes to the underlying data model, yields downtime of services. Currently we are investigating how to negate downtime with a technical solution that intervenes in the development/maintenance process.
A strategy to cater for zero-downtime software deployments is to leverage capabilities in the underlying database, such that it supports two versions of the same application at the same time. This means that changes that might impose downtime, need to be modeled differently (temporarily) such that they bridge the previous and subsequent model. Whenever the very first version is of no use anymore, it can be taken down and the actual change can be finalized by another deployment.
Applications typically expose REST API's using semantic versioning. Breaking changes to an API consist of a major version change.
Company: ING Group
There are several projects that address spects of this problem. All of them are to be carried out at ING. If you choose one of these projects, you will be employed as an intern at ING for the duration.
ING group is a large financial institution based in The Netherlands. It provides services in over 40 countries and houses ~54K employees. 97% of the customer contact is digital, which clarifies the strong need for thorough IT solutions.
Project 1 - Using database capabilities to overcome downtime
A strategy to cater for zero-downtime software deployments is to leverage capabilities in the underlying database, such that it supports two versions of the same application at the same time. This means that changes that might impose downtime, need to be modeled differently (temporarily) such that it bridges the previous and subsequent model without hampering performance. Whenever the very first version is of no use anymore, it can be taken down and the actual change can be finalized by another deployment.
In this project the idea is to come up with solutions for Oracle and Postgres databases to nullify downtime for different types of changes. Different solutions leverage different capabilities and might deviate in terms of their impact on throughput and latency. Ideally this will be measured and improved.
Depending on time: the ultimate goal is to extend an existing DSL for database change management (Liquibase) by a plugin, such that these alternative solutions can be used to overcome downtime in a Java/Kotlin Spring Boot application. Testing would go in a Kubernetes cluster by deploying different versions of the same dockerized application and validating in/output of REST API’s.
Project 2 - Designing applications to minimize downtime by semantic versioning
In this project the idea is to (re)design the current architecture of our typical Spring Boot applications, such that it caters for zero-downtime deployments. The correspondence between the semantic versioning of the API and the backing data-models has to be strongly linked. The ultimate goal is to have guidelines on how to use semantic versioning for zero-downtime changes properly. A PoC might be desirable where different types of changes can be deployed without downtime by leveraging semantic versioning and providing guarantees using that. Backward compatibility might be guaranteed by using specific principles in REST, whilst forward compatibility can only be in some cases.