So, did I really say that context-driven testing is dead? No, that was some other guy (Scott Barber) who’s using the buzz to launch a different idea. It’s effective marketing, and Scott has interesting ideas. But that’s his assertion, not mine.
What I wrote a few days ago was this:
If there ever was one context-driven school, there is not one now.
A “school” provides an organizing social structure for a body of attitudes and knowledge. Schools are often led by one or a few highly visible people.
Over the past few years, several people have gained visibility in the testing community who express ideas and values that sound context-driven to me. Some call themselves context-driven, some don’t. My impression is that some are being told they are not welcome. Others are uncomfortable with a perceived orthodoxy. They like the approach but not the school. They like the ideas, but not the politics.
The context-driven school appeared for years to operate with unified leadership. This appearance was a strength. But it was never quite true: Brian and Bret left early (but they left quietly). I’ve repeatedly raised concerns about the context-driven rhetoric, but relatively quietly. James and I haven’t collaborated successfully for years–this is old news–but for most of that time, our public disagreements were pretty quiet.
I think it is time to go beyond the past illusion of unity, to welcome a new generation of leadership. Not just a new generation of followers. A new generation of leaders. And to embrace their diversity.
There is not one school. There might be none. There might be several. I’m not sure what our real status is today. There will be an evolution and I look forward to seeing the result.
For now, I continue to be enthusiastic about the approach. I still endorse the principles. But what I understand to be the meanings and implications of the principles might not be exactly the same as what you understand. I think that’s OK.
In terms of the politics of The One School, my perception is of an exclusionary tone that has become more emphatic over time. I think this can make good marketing–entertaining presentations, lots of excitement. But does it serve its community? What is the impact on the people who are actually doing the testing: looking for work; looking for advancement in their own careers; striving to increase their skills and professionalism?
For many people, the impact is minimal–they follow their own way.
But for people who align themselves with the school, I think there are risks.
I wasn’t able to travel to CAST last year (health problem), so I watched sessions on video. Watching remotely let me look at things with a different perspective. One of the striking themes in what I saw was a mistrust of test automation. Hey, I agree that regression test automation is a poor bases for an effective comprehensive testing strategy, but the mistrust went beyond that. Manual (session-based, of course) exploratory testing had become a Best Practice.
In the field of software development, I think that people who don’t know much about how to develop software are on a path to lower pay and less job security. Testing-process consultants can be very successful without staying current in these areas of knowledge and skill. But the people they consult to? Not so much.
It was not the details that concerned me. It was the tone. I felt as though I was watching the closing of minds.
I have been concerned about this ever since people in our community (not just our critics–us!) started drawing an analogy between context-driven testing and religion.
As James put it in 2008, “I have my own testing religion (the Context-Driven School).” I objected to it back then, and since then. This is deeply inconsistent with what I signed up for when we declared a school.
An analogy to religion often carries baggage: Divine sources of knowledge; Knowledge of The Truth; Public disagreement with The Truth is Heresy; An attitude that alternative views are irrelevant; An attitude that alternative views are morally wrong.
Here’s an illustration from James’ most recent post:
This illustrates exactly what troubles me. In my view, there are legitimate differences in the testing community. I think that each of the major factions in the testing community has some very smart people, of high integrity, who are worth paying attention to. I’ve learned a lot from people who would never associate themselves with context-driven testing.
Let me illustrate that with some notes on my last week (Feb 27 to March 2):
- My students and I reviewed Raza Abbas Syed’s M.Sc. thesis in Computer Science: Investigating Intermittent Software Failures. The supervisor of this work was Dr. Laurie Williams. If she identified herself with any school of software testing, it would probably be Agile, not Context-Driven. But, not surprisingly, the work presented some useful data and suggested interesting ideas. I learned things from it. Should I really stop paying attention to Laurie Williams?
- Yesterday, Dr. Keith Gallagher gave a guest lecture in my programmer-testing course on program slicing (see Gallagher & Lyle 1991and Gallagher & Binkley, 1996). This is a cluster of testing/maintenance techniques that haven’t achieved widespread adoption. The tools needed to support that adoption don’t exist yet. Creating them will be very difficult. This is classic Analytical School stuff. But his lecture made me want to learn more about it because it presents a glimpse of an interesting future.
- This evening, I’m reading Kasurinen, Taipale & Smolander’s paper, Software Test Automation in Practice: Empirical Observations. One of my students and I will work through it tomorrow. I’m not sure how these folks would classify themselves (or if they would). Probably if they had to self-classify, it would be Analytical School. Comparing myself to a modern Newtonian physicist and them to outdated Aristoteleans strikes me as one part arrogant and five parts wrong.
I think it’s a Bad Idea to alienate, ignore, or marginalize people who do hard work on interesting problems.
James says later in his post,
I respect the right of any individual to seek his or her own level of ignorance.
But I see it as a disservice to the craft when thought-leaders encourage narrow-mindedness in the people who look to them for guidance.
When I was an undergraduate, I studied mainly math and philosophy. Of the philosophy, I studied mainly Indian philosophy, about 5 semesters’ worth. My step-grandmother was a Buddhist. Friends of mine had consistent views. I was motivated to take the ideas seriously.
One of the profound ideas in those courses was a rejection of the law of the excluded middle. According to that law, if A is a proposition, then A must be true or Not-A must be true (but not both). Some of the Indian texts rejected that. They demanded that the reader consider {A and Not-A} and {neither A nor Not-A}. In terms of the logic of mathematics, this makes no sense (and it is not a view I associate with Indian logicians). But in terms of human affairs, I think the rejection of the law of the excluded middle is a powerful cognitive tool.
I have thought that for about 40 years. I brought that with me in my part of the crafting of the context-driven principles. Something can be the right thing for your context and its opposite can be the right thing for my context.
I think we need to look more sympathetically at more contexts and more solutions. To ask more about what is right with alternative ideas and what we can learn from them. And to develop batteries of skills to work with them. For that, I think we need to get past the politics of The One School of context-driven testing.
I was sitting at my desk, having a bit of lunch and reading when I was pinged that a new tweet had come in. Lo-and-behold – it was your tweet on this blog post.
I mention this because I’m rereading Miyamoto Musashi’s A Book of Five Rings. What can a book, ostensibly on swordmanship, kendo, written in 1645 help me understand in software development and testing?
The passage I was reading was discussing this very topic – the concept of all things with no teacher – one school that is not a school or a way. Musashi was describing how difficult it is to master any Way without great effort and that many fall into the trap of believing they are following the true path of the Way when they are not.
I believe part of what you descibe above is the corruption of the Way of “Context” – because in our situations, certain things may work, or may not work, we become comfortable with this as what is right. At times, I have been guilty of what you describe, and I try and be aware of it.
What made me comment was partly that idea and partly the idea expressed above in the {A and Not-A} discussion. Musashi wrote “The Way of the warrior (swordsman in this context) does not include other Ways, such as … But even though these are not part of the Way, if you know the Way /broadly/ you will see it in everything.”
Best Regards –
THANK YOU.
Cem, I have been deeply troubled by this idea of “schools of testing” for years now, and for exactly the reasons you cite. What some call schools I call strategies. Read Sun Tzu’s discussion of strategies of war for the key difference. A person may belong to a school; smart leaders (such as managers or generals) choose strategies that will work for a given context.
I like the fact that you and I agree on this point. Let’s keep the discussion going. It’s good for the testing profession.
Rex:
I don’t think we fully agree, but I’d be happy to post an article of yours on this blog if you want to write further about this.
— Cem
Good comments. In the discussion whether there is one school, you might question “should there be?” Although uniformity and consensus might seem to create a nice world that we can cope with more easily. Tranparent and clear.
I think, like you say, there is much value in diversity. Whether this is found within our outside the school boundaries. Who sets the borderlines anyway?
Cross the border, leave the beaten track and communicate, learning and be astonished by what and how other accomplish their things.
Although context-driven by nature and in practice and an admirer of its leaders’ work, I have always refused to identify myself with the Context-driven School, precisely because of what I saw as its religious fervour. Whether or not its leaders actually had messianic pretensions, many of its adherents sat at their feet — and to my mind, did so unthinkingly. As a life-long atheist, I was and am put off by the “you’re either my follower, or my enemy” pronouncements and behaviour, which I found repellently church-like. I have always reserved the right to talk to, and learn from, anyone I could, whether or not I agreed with them.
Someone — I think it was Ajay Soundarajan — tweeted today that sometimes we learn most from talking to people we don’t agree with. Certainly, I have found things to respect and learn from in the ideas and dedication of testers I don’t fundamentally agree with.
I’ve been dismayed at seeing context-driven friends — intelligent, passionate testers I respect and am fond of — dismiss contexts and solutions outside their own experience when they don’t appear to fit the One True Way mold. And yes, I too have felt that “Manual (session-based, of course) exploratory testing had become a Best Practice.”
I once considered writing a blog post founding the Rugged Individualist School of software testing — until I realized other people might take it seriously and want to join. I might be forced to say, “Please don’t follow me! Think for yourself!” 🙂
I hope that thinking testers everywhere will take encouragement from your recent posts to go their own way: explore, invent, discuss, dispute, experiment. I hope also that they will continue to come together in a spirit of learning, sharing and collaborating. Context is obviously not dead, and nor is context-driven testing.
Best regards…/Fiona Charles
I see I misquoted Ajay’s tweet in my comment above. He actually retweeted someone else saying: “People that are willing to consider an opinion different from their own learn much more than those who won’t.”
Yup.
Cem,
I owe a huge debt to context-driven testing:
– it taught me to quesion my context, instead of making assumptions
– it taught me to think, instead of blindly assuming the best way
– it taught me to be aware of my own biases, instead of letting them blind me.
Yet I’ve witnessed first hand the kind of behaviour to which you refer – a supposed member of the standards school who, when attempting to discuss an aspect of testing, was met with a tirade about what they stood for. That is not about testing, and it’s not what I signed up for when I chose to identify as context-driven.
For me being context-driven is, above all else, about having an open mind. Sadly, we seem to have lost that somewhere.
–Iain
Pingback: Five Blogs – 3 March 2012 « 5blogs
Pingback: It’s About The Testing, Stupid | Exploring Uncertainty
Cem,
Sorry to hear about the fallout in the Context-Driven “school”. The ideas and principles this approach has espoused over the last 10-15 years has helped to expand the practice of software testing. Which is a good thing, but all good things must come to an end. Or better yet… transform.
I’ve met you and James Bach at different conferences/workshops over the years and you both always bring up great ideas and presentations. Thank you.
But I can see the personal approach differences you have. And unfortunately it appears that is what caused things to go kaboom. I think you both will go on and do more great work, I hope so.
Anyway… before I put my foot into my mouth. Best to you and I look forward to how your ideas for Software Testing (not just Context-Driven alone) will tranform and give the rest of us new ideas to try and discuss.
Regards,
Jim Hazen
Pingback: Schools of (pro)testing « Pro-testing
At first, I though I’d comment here, in your blog but, since I’d had a blog post of my own brewing in my mind for the past couple of days concerning your announcement, I decided to post the response in my blog instead. Here’s a link to the post should you be interested to read it:
http://pro-testing.arabuusimiehet.com/archives/273
Now that’s leadership. Thanks Cem.
This clarification (context-driven as an approach, not as a school) is useful for the testing world. And it was not easy. Thank you, Cem.
Pingback: The Seven Basic Principles of the Context-Driven School – part not-three « the testing curve
I have the same feeling with Agile/Scrum. There is not a single thruth!
A tester must be practical and creative, adapt to the circumstances. The first rule of a tester is do not assume anything. Applying that to practice this means that you cannot assume a method is old fashion, or another one is the only solution.
You cannot assume one automation tool is the only solution.
When pride comes, then comes disgrace, but with the humble is wisdom.
Pingback: Are Testing “Schools” a Good Idea?