Team Topologies - Organizing for fast flow of value

View Original

From Monolith to Agility: A Team Topologies Journey in a Leading Financial Institution

This short case study describes a personal journey undertaken as a Team Topologies Advocate at a global financial services organization.

Background

The client had a core case-management system that had evolved over 20 years. This product had a number of components that included an insurance-broker web portal, a financial and fiduciary section, a set of management reports, and a desktop client that managed cases from quotes to renewal and amendment. It was the absolute definition of a monolithic system. Its users were what you might typically expect in an insurance business; sales staff creating new policies, case managers dealing with claims, risk managers assessing risk returns, and brokers reselling white branded products as their own.

Complicating matters, development took place by grabbing and assigning developers from a pool of around 70 software engineers, business analysts and testers.

To make the situation more complex, a distinct database administration (DBA) domain was created to separate data from business rules and management. This was done to adhere to the Sarbanes-Oxley Act (SOX), which interprets “separation of duties” as a requirement that no single person or team should have control over two aspects of a business transaction. In practical terms, this interpretation of SOX compliance meant that the software development team should not have direct access to production systems. Consequently, the development team lacked control over the pipeline for releasing new software to production. (It should be noted that the effect on this organization’s software release process reflects their interpretation of SOX compliance rather than a definitive interpretation.) 

Figure 1: A model of the starting state

Navigating the Path to Agile Excellence with Team Topologies

Complex change cannot happen instantly or via a revolution. You cannot define an end state, implement it and simply wish for success. Instead, the quest for organizational agility begins with a reasoned, systematic approach. The pathway to implementing Team Topologies unfolded in the following distinct phases, each contributing to the transformational voyage:

Education on Concepts

Foundation training: Teams immersed themselves in understanding the core principles of Team Topologies. They explored the concept of team types, interaction modes, cognitive load and Conway’s law.

Shared language: The creation of a common language bridged gaps between developers, architects and business stakeholders. Concepts like stream-aligned teams, interaction types and cognitive load became part of everyday discourse.

Education Feedback: Teams were very receptive to this phase. Team Topologies introduced some new concepts and terms, creating common ground that helped to accelerate the next phase, that of defining experiments.

Defining and Implementing Experiments

Hypothesis crafting: Teams formulated hypotheses based on their contexts. What if we reorganize around product boundaries? What if we limit cognitive load by adjusting team sizes?

Experiment design: Teams designed lightweight yet structured experiments, working collaboratively to ensure collective ownership. Given the nature of the business, we included non-software departments such as legal and marketing in this design process. My responsibility as a transformation coach was primarily to facilitate the process, and I only needed to  step into the mentor role. 

Feedback: Empowered by an understanding of the Team Topologies concepts, all of the teams fearlessly embraced experimentation. They identified monolithic structures and interactions, empowered autonomous teams and fostered cross-functional collaboration. They aimed to uncover patterns that resonated with their own unique challenges.

Examining Experiment Outcomes

Data-driven insights: Metrics like cycle time, lead time and deployment frequency became key to driving improvement. Teams scrutinized the impact of their changes. Did cognitive load decrease? Did stream-aligned teams enhance delivery speed?

Qualitative feedback: Conversations really mattered. Teams engaged in retrospectives, seeking qualitative insights. How did the shift affect morale? Did communication improve? Was there a surge in innovation? Again, they included non-software colleagues in this process.

Feedback: The feedback during this phase concluded that this was the tipping point in the transformation. The organization now had metrics against which to stop or start ongoing experiments. This led into the last phase of the process.

Adjusting Based on Feedback

Adaptive course correction: Teams pivoted based on evidence. If a particular team called for collaborative interaction, they explored how this could be developed and improved. If a practice hindered flow, they tweaked it.

Continuous learning: The journey was iterative. Teams learned, adapted and iterated anew. They remained open to unexpected discoveries and emergent patterns. 

Experiments That Forged Our Path: Key Turning Points

While this journey took 18 months and included dozens of experimental changes, some turning points proved key in the journey. These were the pivotal moments:

Goal: Reduce Cognitive Load by Focusing on Two Discrete Platforms

As mentioned previously, there was a pool of cross-functional development team members. They regularly shifted focus, perhaps working on the desktop app one week, then the web-based broker portal the next. This engendered long-term context switching and high cognitive load. It became apparent early on that they could improve performance by structuring around the two platforms.

Figure 2: A broker team and a desktop team emerge

The challenge with adopting this model was the highly coupled nature of these two systems. They chose a collaboration interaction to ensure that one team’s changes didn’t impact the other team. The DBA fiefdom remained, and this was a key area for the collaboration to focus on: retaining a data model that worked for both platforms. They also extended release management to the two platforms, though the act of release remained with the separate release team.

Goal: Further Reduce Cognitive Load by Decoupling

The financial services domain had common entities irrespective of the product being sold: notions of a customer, client payments, correspondence and, for insurance products, a policy. The client wanted to build a decoupled architecture, which led them to consider the idea of the inverse Conway maneuver. If they formed a team for application services, Conway’s law would suggest that this decoupled architecture would naturally emerge.

Figure 3: The inverse Conway maneuver

This state was an evolutionary journey. Initially, there was no clear understanding of the functions that the Application Services platform would provide, so the teams initially formed a collaboration interaction. This evolved into an as a service” interaction as clarity emerged. While this progressed, the cognitive load on their broker and desktop teams fell. The teams learned that they did not need to worry about maintaining comprehensive communications with the DBA team as this would be provided in a series of well-defined data services.

A Complex Subsystem Emerges

In the insurance domain, the rating engine is the core IP of the business. This analytical process evaluates various risks and works out the optimal premium. These premium calculation models are closely guarded secrets. The client identified this element, which required complex actuarial knowledge, as a complex subsystem.

Figure 4: The complex subsystem

Furthermore, the client wanted to accelerate flow and so explored implementing test automation across the system. To achieve this, they created a team of test engineers who used a facilitation interaction with the stream and platform teams. The intent was that facilitation would create a legacy and test-automation culture within each team which would lead to a more permanent shift-left approach to testing. The last thing they wanted was to create a decoupled “testing as a service” model.

  1. Real-World Streams Emerge

“Insurance is all the same, but all insurance is different.” This saying identifies the paradoxical nature of insurance. All insurance essentially functions the same: a customer pays a premium to insure against a risk, and if that risk occurs, they receive a payout. If the insured client’s situation changes, then they would have an endorsement applied to their policy.

However, a client who wishes to insure their house, must provide significantly different information from a client wanting pet insurance for a cat. This paradox helped further refine the Team Topology models around more targeted stream teams for different insurance products. 

Figure 5: Real stream teams focus on customer demand

In this state, they were approaching a final vision. They created small teams, each focused on a particular insurance product. This allowed the teams to focus on increasing their germane knowledge of their own sector. They further reduced the cognitive load on the Application Service platform team by moving their interaction with the DBA into a Data Services platform team. In a similar manner to other as a service” interactions, these interactions began as collaboration but evolved as the service contracts came into agreement.

The Outcome: 200% Improvement in Lead Time

The client undertook a long and challenging transformation program. The process impacted more than 70 people in the development space, along with a dozen or so others in marketing, legal and other teams. It took over 18 months to evolve, with some failed experiments along the way. But ultimately, from a flow perspective, it was an overwhelming success.

Prior to this initiative, the business delivered 95% of changes in around 270 days. After this transformation, 95% of changes are being delivered within 67 days: an impressive 200% improvement in flow. The emergent decoupled architecture also simplified the creation of new products.

Experiments that didn’t work

Some of the key turning points that were pivotal are outlined above. However this was a long experimental journey and there were some ideas that didn’t achieve the expected results. For reference some of these are included below, and some are quite specific to the context of the organization. 

  • Too much too soon - This was a marathon not a sprint. We had success with one experiment and wanted to try and scale it quickly across multiple teams. This didn’t alway go to plan, and sometimes challenges only emerged during team interactions.  

    • Key Learning - Take your time. Let each change really bed in before trying to adopt the change at scale. Make sure your chosen metrics stack up consistently.

  • Trying to change the DBA team too early - One of the biggest impediments to flow was a handover of database changes to the DBA team. There was an experiment that suggested this team could operate via an ‘as a service’ model. Time was wasted trying to change a team over which we had little influence.

    • Key Learning - Focus on team changes where you have influence

  • Facilitation is hard - The Team Topologies book mentions this, but real facilitation is a hard skill to master. We fell into the trap a couple of times of trying to create interactions, where the facilitator ended up doing most of the work.

    • Key Learning - Understand the difficulty in facilitating. Set clear success measures, and establish a fall back state.

About the author:

ANDY SPAMER, TTA

Andy Spamer is an organizational agility coach and delivery lead based in Western Australia. He has worked with many of the largest blue-chip companies in Australia, and has had exposure to global delivery projects in mining and finance. Andy is a Team Topologies Advocate and also a proud geek who is always tinkering with tech and software.