m&ms are not just for eating, it thought me something about establishing SCRUM metrics.

As part of my Operations Management module at MBS, we studied the concept of performance and quality measurement. We played a game whereby a team of four separated into two quality inspectors and two operators. I was an operator. The inspectors were told to develop two measures of product quality. The product? m&ms.

m&m's are not just for eating

The M&Ms were laid out and given the quality metrics: fail any m&m if the 'm' was less than 50% visible and if there exists any visible cracks. I diligently went through each m&m, examining the quality of the printed 'm', its opacity, the shape of the 'm'. I then did a 360 degree check on the m&m checking for uniformity in the shell. This was repeated for my colleague and we each repeated the check for consistency.

In the end I failed 40% of the m&ms. So did my colleague. The quality inspectors only failed 5%.

This example illustrates the difficulty in establishing performance and quality metrics that furthers and organizations goals. As an operator, I wanted to do the best job that I can to the established metrics. I also let pass m&ms that had pinhole defects, but did not fail them as they were not specified in the metrics, although they were clearly defects (still as yummy though). The danger in metrics is that they may create an imbalance between performance and quality, in this case, the number of sellable m&ms and clear defects (e.g. missing 'm').

SCRUM performance and quality metrics

This made me wonder as to the effect of scrum's performance and quality metrics have on a team. Increasing and maintaining the velocity is a performance metric. That's the amount of work a team is able to accomplish during a sprint. If a scrum team's only key metric is velocity, the team could overly focus on cutting code, thereby increasing velocity. The cost would then be defects introduced. The performance metric of velocity would then have to be balanced by the number of defects introduced, this would promote creation of unit tests and developer testing.

The metric of defects introduced would have to be denominated by velocity (defects/velocity) to provide an apple-to-apple metric, whereby a team's velocity increases, but so does the defects introduced as the team is producing more per sprint.

There are a couple of challenges with the defects introduced metrics. Firstly, it is a lagging indicator. Defects introduced would only be found beyond the current sprint, thus would reflect on past work done. Secondly, if there are testers on the SCRUM team, then there is a conflict of interest for testers to report found defects. Perhaps a relevant control metric would be number of test cases executed against the story size (via points).

Unfortunately, defects, unlike m&ms, are not as tasty.

Is your SCRUM team too big?

How big is too big? Some people say you can never have it too big, but in this case, it is.

I'm talking about team size. The team that I am currently working on supporting a large eCommerce portal, has 18 members. We also practice SCRUM. This is a big no-no in SCRUM teams as it becomes difficult to manage. These were the signs that our SCRUM teams were not lean and agile:

1. Standup meetings

Our standups were taking 30 minutes or longer. Team members were not engaged as it takes a while to get to their turn to update. People were bringing in laptops to do "work." People stopped coming because they had more important things to do. Daily updates were sent via email to be "more efficient." The team was SCREAMING that the team was too large, but it takes a skilled scrum master to listen.

2. Sprint planning

Planning was taking two days, sometimes longer. Many tasks were added to the sprint after start of sprint. Many of these could've been identified if we had performed better planning upfront in discussing stories and breaking them down to bite size pieces. Many members did not have the information they required to start working on the tasks before sprint start.

3. Team Dynamics

Our testers did not fully understand the stories in the current sprint or even which developer were working on them. The developers, business analysts and testers were not working in concert to deliver a feature.

4. Sprint Status: Burndown

With every sprint, we have problems burning down to zero. Of course many of these problems lie with the quality of the requirements, but it goes back to team dynamics where the business analyst, developer and tester were not working in concert to deliver a feature. By having the three roles in close communication during the feature development, information is more quickly shared and know-how is transferred.

As what Jesse Fewell has thought during my SCRUM product owner training, SCRUM is a diagnostic tool. SCRUM will not solve our problems, but rather explicitly display it. It just takes the team to listen and bite the bitter pill to improve.

We did listen and bit the bitter pill and took the decision to split the team. In the spirit of empowerment, the manager set the stage for why we need to split and promptly left the room. It was then left to the team to self-organize in to two teams. Typically a manager would make the decision, as in "you go here, you go over there, you I don't need." By giving us the opportunity to self organize, we discussed in length about the pro and cons, how to address the cons and it resulted in a better understanding of the WHY and the HOW.

All this without having a manager in the room.