Monday, 23 September 2013

Reporting Accuracy – Greater Than or Less Than


Nothing is more frustrating than publishing your metrics only to find that there are some inconsistencies in the values you have reported on. You may ask how that is even possible but it happens more often than some would like to admit.



Firstly, let me tell you, while you may feel only a few apples tall these erroneous numbers should have at the very least taught you a few things:

  1. There may be a process gap that is contributing to your metrics in some way
  2. That the vehicle for reporting has some “issues” that need to be addressed
  3. That you simply are not reporting the right “things”
  4. That there is a disconnect between you and your customer

To describe in some detail what I am talking about I will run through each in a little detail.

Process gaps

Let us suppose that you are reporting against the number of escalated critical incidents to your support teams. In a given month you run the report against all Priority 1 incidents created. Since these are always escalated this should be pretty simple right? After publishing the values your operations manager indicates that while the numbers for the most part look sound, the number of escalations to the database group looks to be only in half of what they should be. Puzzled you rerun the numbers and get the same results. You contact the Database Manager to see if she can shed some light on the situation. “Where are the automated alerts?” she asks. What we forgot was that our ticketing system also creates Incidents based on up/down traps from our database event management tool. The system automatically generates the alerts if a database appears to have lost connection for “x” period of time. The difference is that until reviewed the priority always remains as a priority 3 (moderate). This is why these never shown up in the reports. The process was to reprioritize all escalated events to Priority 1 should they need to be addressed.  Having found that we also need to ask:

  • Do we want to report against all escalations despite priority
  • Does our database team need to better understand the process for reassigning priority
  • Is our event management system trapping up/down correctly
  • Does that system need to be evaluated for information alerts as well

Reporting Issues
Not all systems are created the same. Suppose you want to run a report against all network outages for the customer last month. There is even a nicely “canned’ report which handles this. When you run the report it indicates that there was a 2 hour downtime that was the result of a hardware failure. Despite this your leadership team indicates that there was also a large outage at the beginning of the month, they are asking where this statistic is. Taking a closer look at your report, while it looks sound you see that it reports against the start time of last month. The large network outage that they are referring to started technically on the last day of the month previous. This is why you did not see it in your uptime report. This will need to be adjusted for the next time you run the report.

The Right Things
There are also times where the reports you run are correct but the audience just simply wants something else. For example you may have run a report which outlines the duration of Service Requests for a given period. Your audience however, while interested in the total duration, is more concerned with the time allotted to each team. You may need to revise the report to reflect these particular needs.

Customer Disconnect

You publish the monthly report for your customers which show that application “Y” had an uptime of 99.98% last month. The feedback that you received on this was that these numbers are greater than they should be. “We had several small but significant outages last month”, they indicate. After going back, the numbers look to be right, there appear to be no process gaps, reporting issues and the right things are in there. Even a discussion with your Service Desk manager seems to agree with the stats. However further discussion with your Problem Manager, and a customer stakeholder indicates that there is a consistent login problem with the application and that these issues may not always get escalated and in turn are not being tracked. This is why your numbers don’t match up to your customer’s experience. A discussion with the Problem Management, Service Desk, and the Business Relationship Manager helps to facilitate the need for escalations for these issues, even when they seem small.


Overall getting the discussion going on these reporting issues allows you to continuously improve not only what is reported, but gives insight into various processes and discussions with stakeholders to improve the quality of service you provide.



  
Feel free to connect with me on Twitter @ryanrogilvie and/or on LinkedIn
 
If you like these articles please take a few minutes to share on social media or comment
 
 

No comments:

Post a Comment