Contexts differ.

• Testers provide information to our clients (stakeholders) about the product, about how we tested it, and about what we found.
• Our clients get to decide what information they want. We don’t get to decide that for them.
• Testers provide services to software projects. We don’t run the projects. We don’t control those projects’ contexts.

In context-driven testing, we respect the fact that contexts differ.

What Does “contexts differ” Really Mean?

I think it means that in different contexts, the people who are our clients:

• are going to want different types of information
• are going to want us to prioritize our work differently
• are going to want us to test differently, to mitigate different risks, and to report our results in different ways.

Contexts don’t just differ for the testers. They differ for the project managers too. The project managers have to report to other people who want whatever information they want.

We don’t manage the project managers. We don’t decide what information they have to give to the people they report to.

Sometimes, Our Clients Want Metrics

Sometimes, a client will ask how many test cases the testers have run:

• I don’t think this is a very useful number. It can be misleading. And if I organize my testing with this number in mind, I might do worse testing.
• So if a client asks me for this number, I might have a discussion with her or him about why s/he thinks s/he needs this statistic and whether s/he could be happy with something else.
• But if my client says, “No, really, I need that number“, I say, OK and give the number.

Sometimes a client will ask about defect removal efficiency:

• I think this is a poor excuse for a metric. I have a nice rant about it when I teach my graduate course in software metrics. Bad metric. BAD!
• If a client asks for it, I am likely to ask, Are you sure? If they’re willing to listen, I explain my concerns.

But defect removal efficiency (DRE) is a fairly popular metric. It’s in lots of textbooks. People talk about it at conferences. So no matter what I say about it, my client might still want that number. Maybe my client’s boss wants it. Maybe my client’s customer wants it. Maybe my client’s regulator wants it. This is my client’s management context. I don’t think I’m entitled to know all the details of my client’s working situation, so maybe my client will explain why s/he needs this number and maybe s/he won’t.

So if the client says, “No, really, I need the DRE“, I accept that statement as a description of my client’s situation and I say, OK and give the number.

One more example: ratio of passing to failing tests. Michael Bolton presents several reasons for disliking this metric and I generally agree with them. In particular, I don’t know what the ratio measures (it has no obvious construct validity). And if the goal is to make the number big, there are lots of ways to achieve this that yield weak testing (see Austin on measurement dysfunction, for discussion of this type of problem.)

But Michael takes it a step further and says that using this metric, or providing it to a client is unethical.

UNETHICAL?

Really?  UNETHICAL?!?

If you give this metric to someone (after they ask, and you say it’s not very good, and they say, really-I-want-it):

• Are you lying?
• Are you taking something that doesn’t belong to you?
• Are you oppressing someone? Intimidating them?
• Are you hurting someone?
• Are you cheating anyone?
• Are you pretending to skills or knowledge that you don’t have?
• Are you helping someone else lie, cheat, steal, intimidate, or cause harm?

I used to associate shrill accusations of unethicalness with conservatives who were losing control of the hearts and minds of the software development community and didn’t like it, or who were pushing a phony image of community consensus as part of their campaigns to get big contracts, especially big government contracts, or who were using the accusation of unethical as a way of shutting down discussion of whether an idea (unethical!) was any good or not.

Maybe you’ve met some of these people. They said things like:

• It is unethical to write code if you don’t have formal, written requirements
• It is unethical to test a program if you don’t have written specifications
• It is unethical to do exploratory testing
• It is unethical to manage software projects without formal measurement programs
• It is unethical to count lines of code instead of using function points
• It is unethical to count function points instead of lines of code
• It is unethical to not adopt best practices
• It is unethical to write or design code if you don’t have the right degree or certificate
• It should be unethical to write code if you don’t have a license

It seemed to me that some of the people (but not all of the people) who said these things were trying to prop up a losing point of view with fear, uncertainty, doubt — they were using demagoguery as their marketing technique. That I saw as unethical.

Much of my contribution to the social infrastructure of software testing was a conscious rebellion against a closed old boys network that defended itself with dogma and attacked non-conformers as unethical.

wrong versus Wrong

So what’s with this “Using a crummy metric is unethical” ?

Over the past couple of years, I’ve seen a resurgence of ethics-rhetoric. A new set of people have a new set of bad things to condemn:

• Now it seems to be unethical to have a certification in software testing that someone doesn’t like
• Now it seems to be unethical to follow a heavyweight (heavily documented, scripted) style of testing
• Now it seems to be unethical to give a client some data that the client asks for, like a ratio of passing tests to failing ones.

I don’t think these are usually good ideas. In fact, most of the time, I think they’re wrong.

But  _U_N_E_T_H_I_C_A_L_?_!_?

I’m not a moral relativist. I think there is evil in the world and I sometimes protest loudly against it. But I think it is essential to differentiate between:

• someone is wrong (mistaken)
• someone is wrong (attempting something that won’t work), and
• someone is Wrong (unethical).

Let me illustrate the difference. Michael Bolton is a friend of mine. I have a lot of respect for him as a person, including as an ethical being. His blog post is a convenient example of something I think is a broader problem, but please read my comments on his article as an assertion that I think Michael is wrong (not Wrong).

To the extent that we lose track of the difference between wrong and Wrong, I think we damage our ability to treat people who disagree with us with respect. I think we damage our ability to communicate about our professional differences. I think we damage our ability to learn, because the people we most agree with probably have fewer new things to teach us than the people who see the world a little differently.

The difference between wrong and Wrong is especially important for testers who want to think of ourselves (or market ourselves) as context-driven.

Because we understand that what is wrong in some contexts is right in some others.

Contexts differ.