Adopting and Sustaining Microservice-Based Software Development

For many, the decision to adopt MS architecture generally emanates from out-of-control codebases in large and complex monoliths.²³ While many applications start small and are expected to fulfill a specific business need, over time they become larger and more complex as new requirements are added and bugs are fixed. Code refactoring to counter the effects of mounting complexity is often pursued, but barriers to refactoring in a monolith setting are well known.^1,25 Ultimately, the decision to adopt MS architecture rests on the profile of the organization’s application suite.

Although extreme cases of spiraling codebases and escalating complexities are likely to be self-evident, other situations complicate the decision to adopt MSs. In practice, the decision should not be based on transforming all monolithic applications all at once to MSs; rather, the first step should be identifying specific business functionalities and associated code segments as initial candidates for modernizing to MSs. Monolithic applications create enormous bottlenecks when frequent code changes warrant the need to deploy them often.

When applications are already subjected to significant enhancements in a relatively short period of time or are expected to be in the future, management should consider adopting MSs. The case for modernizing to MSs should also be made when monoliths take up greater testing resources than the development effort. When an application is identified as a differentiator in the industry and significant investment has been made for its success, the case ought to be made for refactoring it under MSs. When monolithic applications continue to struggle to meet nonfunctional requirements (NFRs) and user expectations, there are more opportunities to reap the benefits of modernizing to MSs. When a higher value is placed on software scalability, refactoring to MSs allows each MS to be scaled independently of the others that make up the application (vis-à-vis a monolith). While scalability in general is desirable, predicting future scalability requirements is not trivial.

Nonetheless, the biggest challenge for MS proponents is to make the value proposition to upper management. The key to making the case for adoption lies in quantifying the benefits of MS and comparing them with costs of implementing an MS infrastructure, including recruiting and retraining personnel. The decision to modernize to MSs must consider all parameters in a holistic, rather than a piecemeal, fashion. Intangibles such as changes to the organizational culture, where a team of individuals is responsible for each MS from development to operation, must be considered as well.

While the MS model offers greater resiliency in terms of the overall system’s ability to continue to function even when a noncritical MS fails,⁷ it still needs to be programmed into the code.¹² Unlike a single monolith, MSs allow for setting more finely granular service-level agreements (SLAs). When SLAs for an individual MS are considered for the system, however, they become multiplicative instead of additive from an end-user point of view. Hence, quantifying SLAs for an application made up of a suite of MSs depending on each other can be quite challenging.

As companies look for ways to cope effectively with organizational changes that manifest from a demand such as new product development and the corresponding IT/IS changes, MSs offer a novel approach to align organizational and IT/IS changes more seamlessly.^7,18,29 For many, however, the decision to adopt the MS architecture is not a trivial one, and not all applications are candidates for this modernization.

In comparing monolith and MS approaches to software development, Thomas Hunter, author of Advanced Microservices,¹⁴ asserts “there are various reasons to choose one approach instead of the other, and neither approach is absolutely better or worse than the other.” When a company has a large suite of applications that historically have had one or two deployments a year, there is little reason to make the leap to MSs. In making this important decision to adopt the new paradigm, companies must examine their application portfolio and consider all aforementioned criteria. Over time, the experience will help organizations navigate the decision-making landscape to either adopt MSs or retain the conventional monolith-based development.

To realize the benefits of the MS paradigm, companies look for ways to break up their existing monoliths. Practically, it makes sense to identify larger MSs that could later be broken into smaller ones. Our experience is that breaking a larger MS into smaller ones is easier than combining two smaller MSs to create a larger one. Others have made similar observations, that most organizations tend to start with bigger MSs and then split them up into smaller ones.²⁶ The challenge is to develop guidelines for systematically carving MSs out of the monolith.

Segmenting code into units where each represents a single function or business capability is referred to as bounded context, a term that has roots in domain-driven design.^3,9 Isolating frequently changing code segments within a monolith serves as the starting point in identifying bounded context for segmenting part of the monolith to create MSs. As alluded to earlier, the code segments that are most used, take up significant testing resources, serve as differentiators, and struggle to meet NFRs and user expectations also provide clues about how to segment a monolith to build MSs.

This transformation exercise requires adherence to conventional principles in strong cohesion and loose coupling. Nonetheless, the goal should never be to convert the entire monolith into MSs. Instead, in an iterative fashion, suitable code segments should be identified and refactored into MSs.

Implementing stopping rules for the monolith-to-MS transformation plays a crucial role in reaching desired objectives without getting carried away by MS mania. Moreover, this should not be an exercise in simply copying a codebase from a monolith and pasting it to an MS. It needs to be looked upon as an opportunity (albeit with certain challenges) to improve that code segment in the form of an MS that provides a specific business function with a well-defined interface.

In most instances, not all the original monolith is converted to MSs. The overhauled monolith now must coexist with newly created MSs. Calls that initially went to the monolith may now have to be rerouted to the corresponding MS. The new ecosystem creates an added challenge that is not present in a solely monolithic environment: The user interface of the existing monolith must be updated to point modernized/new functions to the MS-based application delivering that business function. This coexistence of the monolithic application and the MS-based modernized parts requires sharing context between them, including authentication/authorization information, thereby further exacerbating complexities in software development and operation.

Breaking up a monolith invariably involves segmenting data elements in the enterprise repository to individual MSs. While designers are generally well versed in principles of distributed database design, the new ecosystem consisting of monoliths and MSs requires training in data segmentation that transcends our extant knowledge. In this exercise, the challenge is to avoid data duplication and possibly learn to disregard conventional principles such as data normalization. Once data is partitioned, the monolith must be tested to make sure that it does not break. While each MS containing its own data resources has it benefits, practitioners face significant challenges in partitioning the enterprise database.

MS-based development requires additional levels of testing. First, contract testing refers to testing APIs. When a consumer links to an MS, a contract is formed. This contract defines the inputs, outputs, and performance parameters that need to be thoroughly tested.

Second, chaos testing refers to testing for chaos inherent in the natural world. Generally, this exercise involves defining what is “normal” and testing the system under those conditions before introducing chaotic conditions. More chaotic conditions manifest in the new paradigm because unlike a monolith, which has only two states of availability, some of the MSs in the application might be up and running while the others may not.

To mimic this reality in chaos testing, each MS is brought down deliberately to test its implications on the overall system. This testing also involves intentional introduction of network latency to assess its impact on performance. MSs communicate with each other over the network as opposed to the in-process function calls made in a monolith. Testing circuit breakers¹¹—put in place to decide whether to continue operating under normal conditions or resort to operating under limited capacity—places added complexities on the emerging paradigm.

As new requirements emerge, developers must decide whether to build a new MS or modify an existing one. When modifying an existing MS, the challenge is to preserve the bounded context of that MS (that is, high cohesion of what is included in the MS). Unless developers are trained in the new paradigm, they could resort to what is technically expedient rather than maintaining core principles of MS-based development by focusing on business aspects. For example, adding new code to satisfy a requirement might break the tenet that each MS should support a single function around a business capability. Or lack of due diligence in adding code might result in unintentional dependencies (that is, coupling between the modified MS, other MSs, or even the refactored monolith), thereby stymying the ability to change, test, and deploy it independently and quickly.

The key is to educate developers to handle business requirements in the new paradigm and instill discipline to maintain its fundamental tenets. The technical lead in each development team must verify that creating new MSs and modifying existing ones does not break key assumptions of the emerging paradigm. Conventional techniques such as code reviews need to be expanded to ensure the new configuration does not violate the bounded context. Failure to do so would result in the breakdown of the MS architecture and thwart the realization of its much-touted benefits.

The new paradigm operates at the application level rather than at an enterprise level, as is the case with service-oriented architecture (SOA) and Web services. Hence, there could be some duplication of effort. Although one application team might have created an MS that could be used by a second application team, in the absence of an enterprise-wide repository of MSs, the latter might create an identical MS.

These application-based teams tend to move at a fast pace and tend to do little sharing of codebases among them. The modest sharing that does occur manifests by word of mouth or through personal contacts among developers and project teams. In case one team finds an MS already created by another team useful, the former team tends to create its own “copy” of the existing MS to avoid being dependent on the MS created by the latter team. The team essentially forks the codebase and is now responsible for maintaining its own fork of the MS. The best way to promote sharing, however, is to catalog all MSs.

The addition of new MSs creates new challenges in monitoring them. Developers must define metrics for monitoring performance of MSs. One key question is to define what it means to say that an MS is in “good” standing. Monitoring MSs for their availability, reliability, ability to fulfill SLAs, robustness, and resiliency requires metrics and benchmarking. It might require creating synthetic transactions to test the health of each MS periodically. Performance logs of MSs need to be continually monitored to assess their performance and take corrective action when needed.

The MS paradigm creates greater operational complexity because of the numerous moving parts that need to be tracked (vis-à-vis conventional monoliths). Each MS provides a specific business functionality and a suite of MSs must work in concert to provide a more overarching business function. While MS-based development relies on developing autonomous services offering some business functionality, designing decoupled application systems is no easy task. It involves messaging and data exchange among corresponding MSs that must be monitored and managed. Unlike maintaining enterprise databases commonly used with monolith systems, maintaining the availability and consistency of data partitioned into MSs creates additional demands.

Versioning brings significant challenges in the new paradigm. Instead of having to deal with a few versions of a monolith, now developers and analysts must deal with multiple build and interface versions of many MSs.²⁰ As developers create new versions, they must provide support for both prior and new versions. Users need to be informed of differences with older versions in terms of functionality and interface. Some developers choose to list a new MS as experimental, beta, or general.⁷ There must be clear guidelines for informing users in case of discontinued support for older MSs. When the service registry contains multiple versions of an MS, locating the right one for a given application creates additional complications. Versioning strategies and guidelines must be in place.

The network has a greater effect on an application with many MSs as opposed to a single monolith where function calls are in the same running process.⁸ Networks inherently introduce latency issues that could adversely impact the performance of an MS-based application. MSs manifest in the form of distributed systems with calls among them managed via the network. Unlike monoliths, an application consisting of a suite of smaller MSs with messaging among them could amplify network latency.

Running performance tests to identify the biggest sources of latency could hold the key to addressing this challenge. Choosing a platform that helps identify potential bottlenecks could ease network latency. While network latency is unavoidable when dealing with applications composed of MSs, greater insights into the inner workings of MSs would aid in identifying markers on acceptable latency levels.

In addition to technical challenges, companies face obstacles to changing the prevailing culture surrounding monoliths when they begin to develop MSs, where incentives and accountability differ considerably. The new paradigm promotes principles of DevOps teams that are charged with both developing and operating each MS. DevOps correspond to a set of practices that strive to unify software development (Dev) and software operation (Ops).²

Unlike conventional teams based on disciplines such as marketing, DevOps teams are organized by business capabilities. They are responsible from inception to the entire operational life of a product, whether it is small or an end-to-end vertical slice through the product implementing a feature of a larger product that includes all the MSs that are part of it.

Instead of seeking technology-focused solutions in monoliths, these DevOps teams seek business service-focused solutions. Before an MS is implemented, a clear business service (that is, function) that the MS is expected to represent must be delineated. The cultural changes, such as willingness to work with cross-functional teams and to take on expanded roles demanded by the emerging paradigm, are likely to face resistance from some quarters. Moving to a new culture based on the principle “if you build it, you run it and support it” could prove challenging. Unless the incentives are aligned to promote cross-functional collaboration and accept expanded roles as in DevOps teams, some may be reluctant to embrace the organizational changes necessary to facilitate MS-based development.^7,28

As with any new paradigm, the use of MSs for software development and operation warrants justification on a periodic basis. Top management is likely to insist on evidence of performance gains to furnish ongoing investment in people and MS-related technology. This requires initial benchmarking and continuous performance assessment.

While conventional metrics have relevance, new performance metrics need to be developed and instituted. For example, SLAs for an MS-based application as well as SLAs for each MS must be tracked for compliance. In addition to these traditional nonfunctional requirements, other baselining and tracking metrics include an increase in deployments with the modernized application, reduction in the amount of downtime during deployments, reduction in the number of defects/bugs being reported, decrease in the cost of making updates to add new features, among others.

Moreover, processes must be in place to review and refine applications and MSs within each application. Fundamentally, each MS is expected to represent a single business functionality. Checks and balances are needed to confirm that each MS conforms to this core principle. Generally, companies have not instituted such processes, although there is tremendous value in doing so to attest that the company stays true to itself in embracing the new paradigm. Often when existing MSs are modified, some may contain meta functionality and dependencies (that is, tight coupling) with other MSs—two characteristics that are contrary to guiding principles of MS-based development.

On a periodic basis, each MS should be inspected by a different party from the one that developed it to make sure it complies with key tenets of MS-based development. The same holds true for data embedded in MSs. Without clear focus and discipline, revising existing MSs to add a new requirement or fix a defect might result in unwanted data dependencies between MSs and enterprise databases.

With ongoing pressure to move software products out the door, some developers might be tempted to cut corners and abandon the founding tenets of MSs in the process. The developer pool needs continuous training in maintaining core principles of MS-based development.

Moreover, companies need to continue building talent to develop the next generation of MSs that correspond to a single function around a business capability, encompass their own data resources, and are quickly deployable. However, few college graduates receive training in MS-based software development. These are some of the challenges organizations face in sustaining MS-based development over the long run.

Increasing globalization and hypercompetition have resulted in dynamic business environments that require companies to adapt quickly to rapid and frequent changes. These environmental forces necessitate changes to business strategy that manifest in internal organizational changes. Software systems are intimately intertwined with all aspects of a company and, therefore, organizational changes warrant system changes.^4,7,21,24

Companies continue to struggle, however, with effectively mapping organizational changes with IT/IS changes.^7,29 This has been attributed to a company’s inability to deploy software applications independently and quickly.⁷ The software industry is enamored with MS-based development’s ability to overcome these perils.

Each MS contains its own data and represents a single function that corresponds to a business capability. This allows for seamless alignment of business services with software services (that is, microservices). While the new paradigm has considerable advantages, it can impose substantial costs as well.^1,7,22,27 Hence, the challenges highlighted in this article must be considered to improve a company’s chances of success in adopting and sustaining MS-based software development.

This research was funded by grants from the Robert H. Brethen Operations Management Institute and the Earl V. Snyder Innovation Management Center at the Whitman School of Management, Syracuse University.