## migrations an enabler of sustainable velocity and high quality for software teams

# hello. I'm Luke

### migrations are not just for big projects, useful for all sorts of technical change on running systems - data schema change - infrastructure change - code logic change notes: Can be very large, or very small. Migrations aren't something you do once in a blue moon when you go from on-prem to AWS. Migrations are a really powerful concept, framework and tool we can apply to many problems in software, tech debt, legacy etc etc

## so what is a migration? 1. deploy new version iteratively 2. cut over via some strategy 3. decommission the old version

## cutover strategy: big bang - in-place - blue/green notes: this is the simplest, either deploy it in place or use a blue/green strategy where you deploy a new copy and immediately cut over to it

## cutover strategy: phased - canary - rolling notes: partition the change to reduce the blast radius of failure. examples of this are: deploying the new version to 1/n nodes ("canary"), observing then rolling to the rest we used this strategy recently at Uneeq with an infrastructure migration, we operate data centers across 4 global regions. We migrated them one at a time to reduce the number of customers impacted if we made a mistake At uneeq we pivoted from video calling to digital humans. We used this strategy recently at to remove an abstraction from our domain which no longer made sense. We used a multi-step, phased approach to the schema migration to ensure the applications were compatible and we avoided downtime.

## cutover strategy: parallel - feature flags - A/B tests - [scientist pattern](https://github.blog/2016-02-03-scientist/) notes: this is the least risky, but most expensive. it allows you to observe the old and new system running at the same time. we have been doing this at uneeq lately with our digital human synthetic behavior, experimenting with different machine learning models

## what are the alternatives? - deploy the change in place - don't make the change and work around it notes: This all sounds like a lot of work. Do I have to do this?

## there aren't really any alternatives - deploying the change in place is a migration with a big bang cutover - not making the change is how we get system rot notes: Any change to a running, live system is a migration if you think of it that way

> the code still works, but has gradually become complicated and hard to work with https://henrikwarne.com/2017/04/28/code-rot/

## symptoms - very wide database tables - names or abstractions that aren't correct but haven't been changed - code comments which indicate code doesn't run, but the author is too afraid to delete it - ...

> Lean, spare, fast programs are almost always the result of strategic breakthrough, rather than tactical cleverness. [...] often, the breakthrough will come from redoing the representation of the data or tables. Representation is the essence of programming. The Mythical Man-Month, Fred Brooks, 1975

 [Jack Danger, 2019](https://medium.com/dangerous-engineering/a-reference-guide-for-fintech-small-data-engineering-bd65b9796d90) notes: If we’re building Twitter or Instagram we’ll need to process hexabytes of simple information. If we’re building Flexport or Gusto or even Square¹ we’ll be hard-pressed to even find terabytes of information. There’s a huge set of companies that have small amounts of critical data in complex domains. These companies have different problems than big daya companies. They have 4 things to focus on: - Prioritizing domain model correctness over computational efficiency - Refactoring the data model early and often - Ensuring data is encrypted by default everywhere except in your product’s memory space - Separating data by sensitivity (PHI, PII, PCI) to empower your Security team

## building a culture which embraces migrations enables sustained velocity and quality notes: migrations are a way of making scary big changes safe. having the confidence to make data model changes is key one thing that stood out to me when I joined the uneeq team was the confidence with which they take on technical change.

## your process, tooling or culture might prevent this by strongly preferring one cutover method notes: It's probably big bang. It might be phased. Whichever it is, your teams need to have the flexibility to chose an alternative approach if it makes sense. They need to have the tools to enable them to chose when to re-write a method and push it to production or when a new method should be executed in parallel with the old one and results compared for a period of time to iron out any risk of bugs. They need to be able to choose when it's the right time to do a database table migration via multiple steps to ensure uptime or when simply adding a new column is fine.

#### this sounds like a lot of up-front planning, is it agile? > Continuous attention to technical excellence and good design enhances agility. > Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. https://agilemanifesto.org/principles.html notes: The agile manifesto outlines 12 principles. Two of them talk directly to the benefits migrations give. Migrations can be executed incrementally. Being able to safely make changes to the design of a system enables a team to adapt based on the feedback they get from production, users, the market...

## don't forget the last step - decommission the old version notes: this is often where the value is. cost to serve. less complexity. less to maintain. less to secure.

thank you