Articles

Agile Scrum in action 4: Reporting and monitoring

By Wunderer

Proper monitoring of a project and correct reporting of the progress are key to success and open communications. Let’s take a look at how we can achieve this.

Project & sprint objectives

At the outset of the project, we need to know and understand the goals of the overall project. What does the client expect to achieve from it?

These objectives are the main steer behind all decisions we make throughout the life of the project.

In our previous post, we discussed the need to review and report on quality and risk. In this post, we’ll look at reviewing and reporting the overall project progress towards its objectives.

The project is broken down into sprints. Each sprint should have one or more objectives.

We outline these at the start of the planning session and select the stories for the sprint which focus on those objectives.

As we progress through each sprint we use its objectives to make sure that our decisions are geared towards the objectives. When we report on the progress made at the end of the sprint we’re measured against those objectives.

The audience at the sprint demo will often be chosen based on the relevance of their jobs to the objectives. For example, if the sprint is focused on editorial functionality, we’ll invite authors and content creators.

Others are welcome to join the review but the key users will provide the most valuable feedback in helping the Product Owner understand whether the stories he described meet the users’ expectations.

Mid-sprint review

We’ve recently introduced mid-sprint reviews, as a result of a communication request and to keep a wider audience up to date. The project in which we’ve implemented this is fast-paced with a lot to achieve, so it would be easy to lose sight of the overall picture.

We’ve agreed that, as a distributed team, we need to take time to get everyone together – the team, Product Owner and stakeholders – and summarise the progress of the sprint so far. It’s an excellent way to keep everyone up to date, to understand the progress towards the objectives and to make sure that the project is communicating clearly.

Agile provides a great means of delivering new stuff regularly. This mid-sprint review helps keep the team focused and helps us to achieve our sprint objectives.

Those people more removed from the project, such as stakeholders, often don’t want to wait until the sprint demo to communicate progress. By keeping them updated weekly they communicate that progress through their teams.

The upshot is that Agile gets a great name, as people get to hear of the weekly progress the project is making. That communication helps people understand the power and effect of Agile delivery.

Backlog, user stories, progress, notes, decisions

There are some seemingly obvious tasks that need to be continually carried out on the backlog but often aren’t.

The first is that all known stories and epics need to be included and their priorities recorded.

Secondly, as we progress through a sprint, it’s important to be looking ahead to the next sprint, considering the objectives, identifying the stories and epics and grooming. By grooming I mean that the epics need to be expanded, comprehensive stories are written to support the epics, and each story checked for clarity.

As we progress through the sprint it’s important to keep track of story progress. It’s obvious that we need to update the progress in terms of story points completed, but we also need to record comments against stories.

These comments help us understand why certain decisions were made and show progress towards story completion. All of this is part of our monitoring and reporting.

By keeping our stories up to date, we provide the data that will be used to report progress throughout the team and to stakeholders.

Visual Metrics For Reporting

Progress is far easier to understand and explain when we have a clear picture or graph to represent it.

Key reporting metrics for most projects are burndown and risk. Written documents are all very well but graphical representations summarise and communicate more simple, and are more pleasing to the eye.

It’s important for all involved in the project to understand the progress that’s being made and the easiest way to do this is to report progress in a series of graphs.

For example, in steering group meetings, we discuss the progress of the project and support this with a series of simple slides including:

  • The mission – It’s always good to remind everyone why we’re doing this
  • Project goals – Specific reasons behind the blood, sweat and tears
  • Sprint summary – A quick, easy-to-digest summary of sprint progress against sprint objectives
  • Project status – A traffic-light system showing the high-level status of goals, schedule, budget, blockers and risks
  • Risks – Don’t forget them, show how we’re managing them and any movement since the last communication
  • Schedule – Just a simple indication of where we are on the timeline of the project schedule
  • Burndown – If we’re making good progress then show it. If not, it’s a chance to explain why and what steps are being taken to improve
  • Project burndown – How the burn rate looks in relation to the overall project
  • Feedback – Communication is two way, make sure the client airs any concerns with you or the project in general
  • Coming up – What can be expected next, what the next steps are in the project

Project stakeholders are a busy lot and their time is limited. You need to prepare for this type of meeting and make sure it’s timeboxed. Be clear on progress, risks, mitigation plans and any issues on the project.

The metrics you report here need to be concise, accurate and assuring. If there are questions you need to be prepared. This is all part of clear communication and reporting. Using graphs and pictures provides focus points around which we can discuss.

Estimates and expectations

Setting realistic customer expectations is an important way to build a good relationship. In Agile delivery we set expectations in every sprint planning session, accepting a number of stories and setting the expectation that the client can expect to see them in the next demo.

When we develop an estimate, we know it’s just that. Although we might have a lot of experience, there are always other factors that can affect our initial estimates and make the delivery of the accepted stories difficult or even impossible.

So we keep a record of our progress and map our estimation accuracy over the life of the project by recording our estimates and using retrospectives to understand the reasons behind any inaccuracies. We also record changes we implement to improve estimates.

This level of monitoring and reporting is shared with the customer to help them understand the lengths we go to in setting expectations and accuracy. We also use it ourselves to set goals for accuracy and feed them into a central company report to show our overall accuracy and progress towards improvement.

Burndown

Understanding sprint progress is important. Recording and reporting progress on a daily basis helps us to identify potential schedule risks that we need to be aware of.

Maintaining a log of all sprint burndowns allows us to understand the trends we can expect in velocity, whether there is more activity at the beginning, middle or end of a sprint. This adds to the open communication with the client and helps them to keep the shareholders and their users updated.

Burndown charts are often included in Agile software such as Jira or Pivotal. Alternatively, they can easily be created in MS Excel or Google Spreadsheets.

I’ve created an example spreadsheet on Google Drive with some dummy data. It includes a sheet for each sprint, together with a summary to chart the overall progress. It’s simple but meets the requirements of the client.

Sprint review, retrospective, steering group and signoff

At the end of each sprint, we invite members of the wider team to view a demo of what’s been developed in the sprint. The review is a mix of actual product demonstrations and discussions about the `goals and objectives of the sprint.

The sprint demo needs to focus on the objectives that were set for the sprint, to put it in context and to help people understand why certain stories were selected. The demo is effectively a report to the users and stakeholders, so it’s important that it’s relevant to them.

Prepare the demo with the team ahead of time and select who will be best to demonstrate which parts of the product. Help the audience to understand the importance of the delivery by making references to how the new site will help them in their work.

When preparing for the demo, the steering group meeting and the signoff, we need to refer to the project metrics. Look at the objectives that were set, at the completed stories, understand the progress that’s been made towards the objectives, details of any deviations made and how risks have been dealt with.

Look at the best ways to communicate this information to your audience. Try to make it as easy to understand, interesting and relevant as possible.

As for the steering group, we need to show that we understand what we’re trying to achieve and that we’re in control of the project. Very few projects go exactly to plan but it’s how we show our control that will provide the assurances with the client needs.

Project Repository & Consistency

When recording information it’s important that we maintain consistency. We need procedures that prompt us to record data, tools to record that data consistently and accurately, and a place to store all project data together.

A simple tool such as Google Drive can provide this. We can create a project repository with individual folders for Consultancy, Contracts, Risks, Meetings and Reviews, and grant access on an individual basis. Within here we can create specific documents for recording meetings, backlogs, risk logs, assessments and details of sprints.

With documents all in one place, it becomes easy to refer to specific items as we become accustomed to the structure. We can interlink documents and provide these to clients as an easy way to follow sprints, reviews and meetings.

However we structure project documents, we need to make sure we concentrate on consistency, simplicity and accuracy.

Reporting & monitoring summary

The purpose of monitoring is to make sure that projects remain on track. To monitor properly we have to understand the goals we need to achieve. The goals, together with the monitoring, help us to keep the project on track.

The data that we capture through a project provides us with metrics to accurately report progress to the client. At the outset of the project, it’s important to put in place the tools and the procedures for capturing this information, so we can provide the reports that the client has requested.

We also need to report back to ourselves. By monitoring what we do and reviewing that information, we’re able to track our progress and our improvements – not just on individual projects but across all projects.

Storing real project data so it’s easily retrievable and consistent means we have information based on real experience to use in making informed decisions.

By recording, monitoring and reporting effectively, we help develop relationships, provide assurance, maintain control and create the history we need to make decisions and improvements.