Visma delivers software that simplifies and digitises core business processes in the private and public sector. With presence across the entire Nordic region along with Benelux, Central and Eastern Europe, Visma one of Europe’s leading software companies.

We spoke to T. Alexander Lystad, Chief Cloud Architect at Visma, about the ways in which Team Topologies has helped to shape teams and software delivery within Visma.

Background

Prior to 2015, Visma released a new version of our software products 6 times a year (on average). Today, our Service Delivery Teams deploy to Production around twice a week, and we have many teams who deploy changes to Production multiple times a day.

We have transformed how we develop and operate software, based on our own experiences, circumstances and intuitions, but also based on guidance from excellent books such as Continuous Delivery, The DevOps Handbook, Accelerate and more recently Team Topologies. In Team Topologies, Matthew Skelton and Manuel Pais explain how to organise effective business and technology teams, introducing several useful terms and concepts in the process. In this article, I will reflect on how we do software development, through the Team Topologies lens.

Stream-aligned teams

Visma consists of more than 11,000 employees that spend most of their time developing, delivering, and operating cloud software that automates and simplifies business and administrative processes for nearly 1 million customers across Europe and Latin America.

In 2015 we started organising our software development in what we call Service Delivery Teams (SDTs). These are teams that have full and exclusive ownership of their own codebase and infrastructure, as well as full responsibility for continuously delivering a high quality service to their customers.

Each team typically owns a single monolithic application or a group of microservices, and the team is responsible for design, development, testing, deployment, monitoring, handling incidents in Production and everything in between.

Our SDTs use Amazon Web Services, Microsoft Azure, or Google Cloud and are in full control of their own test and production environments instead of relying on a separate operational team. This level of autonomy has been a critical success factor for us.

Teams that have the competence and authority to handle most tasks within the team, work more effectively and make better decisions. In Team Topologies parlance, these are our stream-aligned teams.

Cognitive load

In Continuous Delivery, Dave Farley and Jez Humble pointed out that one of the benefits of deploying small changes often is that when something goes wrong, it’s much easier to reason about a change that consists of 50 lines of code compared to 500 or 5,000 lines of code. This is similar to one of the benefits unit testing has over integration testing; if something fails, it’s easy to pinpoint where the problem is.

In 2015, Dan North used the phrase “software that fits in your head” to advocate for simplicity over complexity. With help from psychology, Team Topologies applies this principle to software development teams: in order to build an effective development organisation, we need to consider the cognitive load on our development teams.

In Visma, we have several enabling teams that support our stream-aligned teams in different ways to accomplish this.

Platform services

Alert Center - OnGuard as a Service

In Visma, our Service Delivery Teams are responsible for designing, implementing and reacting to their own monitoring of their own cloud service(s). Why? Because it’s the team that created the software and that knows the technology, the customer, and the domain, that is best equipped to handle this responsibility.

We follow the principle of “you build it, you run it”. However, SDTs can choose to send monitoring alerts, along with instructions on how to handle those alerts, to a central Alert Center. The Alert Center team consists of many people, but there is often just one person on guard outside of office hours.

It is impossible for one person to be an expert on all our products and all the technologies involved, but by purely acting on documented instructions from the SDTs, we ensure that the SDTs retain complete responsibility for their own service(s) and we lower the risk of the Alert Center becoming a traditional siloed operations team.

When the Alert Center receives an alert, there are three possible outcomes:

They identify that the alert is a false positive and do not escalate immediately to the SDT
They identify the problem, resolve it according to instructions, and do not escalate immediately to the SDT
They identify the problem, but are unable to resolve it, and escalate immediately to the SDT

As a result, the Alert Center can protect the SDTs from additional cognitive load a significant percentage of the time a monitoring alert goes off. We measure the number of incidents not discovered by monitoring as well as the number of false positive alerts for all SDTs so that we can continuously improve our monitoring accuracy.

To collaborate efficiently, SDTs and the Alert Center communicate through a well-defined “Team API”, a concept defined in Team Topologies. When an SDT wants to send a monitoring alert to the Alert Center, they send it to OpsGenie, our alerting and on-call management tool.

SDTs use a standardised template for documenting their instructions to the Alert Center. When the Alert Center needs to escalate to an SDT, that is also documented in a standardised way in a single centralised place.

Developer Efficiency - Tools as a Service

Instead of every SDT having to host and operate their own tools, we have a centralised team that offers a number of centralised tools, most of them optional. Examples include Confluence for internal documentation, Jira for issue tracking, BitBucket for version control, TeamCity for build pipelines, SonarQube for code quality scanning,, Octopus Deploy for deployment pipelines, and many more.

Our Developer Efficiency team ensures that these tools are professionally operated and can also support SDTs in getting the most value out of the tools on offer. Not only does this have commercial, security and reliability benefits, but it takes cognitive load off our SDTs.

Visma Application Security Program - Application Security as a Service

All of our SDTs have a Security Engineer. This is often a developer or architect that dedicates about 35% of their time to distributing security awareness and competence in their team and making sure that the Visma Application Security Program is implemented in their team and for their cloud service(s).

The SDTs are fully responsible for their own application security, but a central Application Security Team provides several services to the SDTs, most of which are mandatory. These services include automated static application security testing, automated dynamic application security testing, automated identification of known vulnerabilities in third party components and manual penetration testing. The underlying tools are operated centrally, but offered as self-service as possible to our SDTs.

Enabling teams

Architecture, Technology and Cloud - Modern Engineering Practices as an Enabler

Developing and delivering software in 2020 is not the same as developing software in 2015, 2005 or 1990 - which is why we have a central team that enables SDTs to adopt modern engineering practices and improve software delivery and operational performance.

Examples include adopting Continuous Delivery, adopting Public Cloud technology, developing a business continuity plan, developing a modern API that provides a good integration experience, and maximizing the benefits of SaaS. Our stream-aligned teams should not have to reinvent the wheel!

Scaling the organisation

Although our SDTs have broad responsibility by design, you have seen how enabling teams helps reduce the cognitive load they have to deal with. However, the cognitive load on a team still grows proportionally with the size of the service(s) they are responsible for – and the size of the team itself!

As an example, one team of 20 people working on a monolithic application rarely works well – too many cooks, too much coordination. We also don’t want multiple teams working on a single monolithic codebase, because that breaks with our DevOps principle of full and exclusive ownership which would bring another set of problems we have a lot of experience with.

This means that there are organisational reasons to change our software architecture. Yes, you may want a microservices architecture in order to automatically scale services independently, or to be able to choose different technologies for different microservices. However, the biggest reason to adopt microservices architecture will often be to scale the organisation; To be able to have two teams of 8 working on a few microservices each, instead of one team of 16 working on a beast of a monolith.

We have learned that it is much cheaper to think about and facilitate this earlier, rather than later; splitting up a monolithic application supported by a group of 50 people is not a 5 month job, it’s a 5 year job. The typical size of development teams in Visma is about 9 and we recommend somewhere between 6 and 10.

Concluding remarks

Team Topologies has changed how we form development teams in Visma. We know we still have lots to improve, but now we have more arrows in our quiver when we ask whether a team is organised for success: Does it have the right competence? Does it have the right size? Does it get the right support from enabling teams? What is the size and complexity of its responsibility? What is the sum of the cognitive load on this team?

We now have more than 60 Service Delivery Teams in Visma following our Visma Cloud Delivery Model, which is based on the core principles of Continuous Delivery, DevOps and Public Cloud. In a couple of years we expect to have over 100. Our transformation continues.

T. Alexander Lystad, Chief Cloud Architect, Visma

Follow Alexander Lystad

Twitter: @alexanderlystad
LinkedIn: alexanderlystad

If you are interested in having Visma speak at your event about how we work, contact alexander.lystad@visma.com Interested in a career in Visma? Visit our Career pages and see our vacant positions: https://www.visma.com/careers/

Team Topologies at Visma - Organising for Autonomy and Sustainable Cognitive Load