How Do You Do QA for Perfomance Management?

December 15, 2007

Doing performance management correctly is like trying to get my kids to wash their hands. Sure, I can create a process or rule for it: When our hands are dirty, we wash them. Then I can ensure that the components of the process are available and responsive: There’s soap by the sink and the hot and cold water turn on easily. But even when I’ve set everything up, and have the metrics established (we need to wash our hands daily!), how can I be sure that they’ll be achieved? And if they are achieved, are they being achieved correctly?
For example, it’s pretty easy to identify when my older son isn’t meeting our performance goals: His hands are clearly filthy. But it’s a bit tougher with my younger son: He did wash his hands. But did he use hot water? Did he use soap? Did he actually scrub for a reasonable period of time (or did he simply pass them under the water)? It can be hard to tell. I’ve also discovered that there’s usually no adequate feedback mechanism to help me identify when there are user or system problems. For example, we can be out of soap for days in their bathroom and I’ll never hear about it. Or they may decide on their own to use one of those liquid hand sanitizers instead of actually washing their hands. It’s hard to know sometimes.
The point is, even with the right process (or application) and tools, the effectiveness of the result really depends on actions of the users. If there are user failures (like failing to report missing soap, using cold water, etc.), then the resulting process is going to be ineffective.
As I will explore in my next column, it’s the same with enterprise performance management and an organization’s applications. When enterprise applications like SAP, Oracle or Seibel aren’t being used effectively and efficiently by users, the business results will suffer.
One solution to this problem is putting solutions in place that can collect and manage information not only on your applications, but on the user experience of the users using those applications. Let’s look at what that means.
Of course, you’d want to start by collecting traditional application management-related information like response time and availability. You’d also want to know the transactional response times-for example, how long does it take the application to respond after the “enter” key is pressed? But-and here’s where it starts to go beyond traditional management solutions-you’d also want to know how long it takes for users to navigate through the application screens.
Are they spending too much time on difficult to navigate or complete screens? Of course, you’d also want to collect information on which applications (and transactions) what people are using. It would also be helpful to collect information about quality of the users’ experience and any errors in the infrastructure or application, whether they’re cause by the user or by something in the IT environment. For example, perhaps users are getting errors trying to connect to certain applications or databases, but such errors aren’t getting reported.


Leave a Comment

Previous post:

Next post: