How Ivan stand conversion investigated

After Ivan met with the cohort analysis , he hated any kind of sugary metrics.
But the irony was that the leadership did not know anything else, and categorically did not want to know. I had to step over myself and stupidly go to a meeting with the "requests" of the chief in order not to earn the reputation of a bad person who disobeys the instructions of the wise men.
Sometimes even very interesting results were obtained from this. About one such case now and will be discussed.
Once the manager asked Ivan to find out why, within 3 weeks, the conversion of stand passing by the teams continuously decreases: 3r3207.  
How Ivan stand conversion investigated
I must say that there were many teams in the department of Ivan, several dozens, they did hundreds of builds of their distributions daily and checked them on the stands.
Conversion counted the ratio of the number of assemblies created by teams per week to the number of assemblies that passed the target stand.
One of the main disadvantages of sugary metrics is that it is impossible to determine anything by them. The conversion used by Ivan’s manager turned out to be a typical sugary metric. Conversion fell, but it was completely incomprehensible why.
Naturally, all the teams were put on their ears with an order to by all means improve the stand passing indicators. For this, one more sugary metric with a list of bad guys was compiled very quickly: 3r3207.  

Under each of the columns in the picture in reality, the names of the teams are written.
The teams howled and began with all their might otbranhivatsya from the new misfortune fell on them.
Charter to drive a team leader called Ivan to understand the reasons for the fall of the conversion.
And that's what came out of it
3r3195. Understand the essence of metrics
Metrics must be understood. At least understand how they are considered. Then you can dig deeper and figure it out. So Ivan did: 3r3207.  
3r366. Conversion = Created assemblies /Stand 3r367 passed from them.
Understanding the formula, Ivan first of all displayed on the chart two components of the conversion: 3r3207.  
It immediately became apparent that the number of assemblies created was almost unchanged, but the number of assemblies that passed the stand decreased, and it began to decrease at the exact moment when the conversion began to decrease.
Conversion has changed due to the fact that the difference between the number of assemblies being created and the number of those who passed the stand increased, but because conversion is the result of dividing one into another, then with the increase in the difference the conversion value synchronously changed (decreased).
The difference between the values ​​on the graph is the dark line.
Those. the increase in the difference between the red and blue columns in the graph indicated an automatic decrease in the conversion.
3r3195. Ponder the results 3r3196.
Ivan understood that the findings were still not enough to determine the cause.
Previous experience on metrics gave him one important thought: you need to dig to the root, to the true root cause of any problem.
The reason for any change in the DevOps conversion is actually people. Developers and devopery teams. It was up to them that Ivan finally wanted to get to.
Looking at the growth of the difference between the assemblies created and passing the stand, he asked himself: why is this happening and who is the “generator” of assemblies that have not reached the stand?
The read book "Tao Toyota" gave a hint: you need to look at the "stock balance" or "work in progress".
Since the assemblies went through several stands and could stay there, Ivan decided to count not only the assemblies of one stand, but to see the true “real balance”, i.e. count how many assemblies were not used at all in one stand and were left to lie dead weight:

The dark line shows the amount of residues, the yellow line shows the number of assemblies that have passed the target stand, which is taken into account in the initial conversion chart.
There was no reason to even guess. The apparent synchronous movement of the two lines was confirmed by calculating the correlation:

It turned out that the more remnants of the assemblies remain, the lower the desired conversion.
3r3195. Find the root cause
Determining who creates the residuals was not difficult with the help of the simplest table:

In the left column - the name of the team. Selected column - the number of residues created by this team in a week.
Top 2 leaders got out immediately and Ivan instantly ran to them to understand.
The reason, of course, turned out to be trivial: the teams simply began a new development cycle and, creating functionality, began building assemblies to verify its correctness.
3r3195. The main disadvantage of sugary metrics is
In fact, the change in conversion turned out to be closely tied to the cyclical nature of the development process and was not fatal and bad.
Three conversion waves (cycles) are viewed on the conversion graph.
This is a natural process that should go.
And the teams that cursed with the manager were absolutely right: with the current development process used in the company, an increase in conversion would not only be an “incomprehensible” action, but could completely disrupt the process and lead to significant delays in software delivery.
This is the main drawback of the sugary metrics - they turn the positive sides into negative ones, and instead of clarifying the situation, they only confuse and worsen it.
3r3195. Conclusions 3r3196.
In general, the experience for Ivan was interesting, so he even with some pleasure prepared a presentation for the management, in which he explained that the metric used was not suitable for managing the department, was misleading, and could cause great problems.
That's all.
If Ivan's experience seemed interesting to you, he will be very grateful for your feedback.
By the way, Ivan now wants to enclose himself and his knowledge in an inspiring and incendiary project, therefore he accepts interesting proposals in a personal.
+ 0 -

Add comment