Articles

Testification - Learning Testing Through Gamification

This article was written for EuroSTAR Software Testing Conference and is based on a speech given there in 2016.

Know Thyself

Let us start with a philosophical question that all of us are used to from our near and dear:

What do you do?

Perhaps you answer “testing”. If you have met another tester, you will probably have enough to talk about for the rest of the evening, but what if you talk to a non-tester? Do they smile politely and change the subject to conceal their ignorance of what you do? Do their eyes roll as they try to process the meaning of what do you do? Or do they follow up with more questions to learn more of what you do?

I typically get the first reaction from people I meet for the first time and the second reaction from my spouse. But what if you meet a curious person who asks good questions and challenges me for good answers, such as a three-year-old child?

Where is your goal?
Why is it difficult?
How do you reach it?

Would you be able to answer those questions? You probably know your work so well that you never reflect on those questions. But if you are unable to tell another person what you do, how can you train another person to do what you do? This is the major challenge for all trainers, yet the solution was elegantly formulated already by the Ancient Greeks:

Know thyself

Don't Learn how to Do, Learn how to Think

Now that you know yourself and know what you do, it is time to examine how you can enlighten other people. If we return to our clever three-year-old child, he or she learns by observing how adults do and does the same.

This works well for basic learnings, such as reading and writing, but how about more advanced learnings, such as reviewing a book or writing one yourself? Knowledge of other authors’ books may certainly help you avoid repeating other authors’ mistakes but will it help you writing a book of your own? Or will it steer you towards already known paths and restrict your own creativity? It is no coincidence that the words “create” and “creativity” have the same origin.

The same applies to testing. Knowledge of testing methodology, such as the ISTQB Syllabus, may certainly help us find the expected defects but it is the unexpected defects that are hard to find. The less open-minded we are, the harder it is to find those.

This challenge has been very well expressed by a famous Doctor in Mathematics, Reiner Knizia. He stated that “when you know a solution to a given puzzle, it’s hard to find another one. When you don’t know the solution, you’re much more likely to find your own. Not knowing others’ solutions helps keep your own ideas fresh.” Basically, he told us not to learn how to do but to learn how to think.

Encourage Creative Thinking

Now that we know the learning objective, we are ready to move on to the question on how to get there. How do we teach the students to think like testers? How do we stimulate their imagination and creativity? How do we ensure that the learning experience is engaging and memorable?

There is no single answer but rather several attributes that together help forming a good learning atmosphere.

Motivation

Motivated students are good students. If they learn through fun and engaging activities, they are more likely to participate whole-heartedly and remember what they learned afterwards. A typical example is team-building activities, where the students practice team-building first and discusses characteristics of good teams afterwards.

Context

Students that understand the big picture are more likely to understand the small details. To them, the learning “makes more sense” and they will remember common situations where it is applicable. A typical example is case studies, where students learn to find solutions in real life examples.

Concrete

Learning together with others’ let the students share ideas and see things from new perspectives. In addition, it also creates bonds that last beyond the learning session. A typical example is group work, where the students cooperate to deliver.

Answers

A good way to learn is to ask questions and get answers. The proverb “there are no stupid questions, only stupid answers” may have become a cliché but it is important that teachers create an atmosphere that encourages students to ask questions. A typical example is the Q&A session that usually follows lectures.

Feedback

Feedback serves a receipt that the students actually achieved something. More importantly, feedback also allows the students to “fail” in a safe environment and learn how to do better next time. A typical example is the school grades.

Rewards

You do not need to be an expert in behavioral psychology to understand that rewards encourages learning. Learning may be a reward in itself but additional rewards never hurt. A typical example is the graduation party.

This is a long list of attributes indeed. Is there really a setting that captures all of them? As a matter of fact, there is. This list is actually a list of attributes commonly found in modern board games!

Motivation: Games are fun and encourage players to do their best.
Context: Games provide a context for the players’ actions.
Concrete: Games take abstract concepts and turn them into concrete actions.
Social: Games let players interact with each other.
Questions: Games provide answers to “what if” questions.
Feedback: Games provide a safe environment to try different actions.
Rewards: Games end with the players having accomplished something.

Represent Test through Games

Can test really be represented by a game you might be wondering. The idea is not far-fetched at all. Games have represented life throughout the History of Man. The Mancala game symbolizes sowing and harvesting while the military plans of Chinese warlords are said to have been the origin of the game of Go.

More modern examples include the game Robot Turtles, invented by the software entrepreneur Dan Shapiro to teach his 4-year old twins programming. Basically, the players play cards to program their turtles to find crystals on a board. Robot Turtles was a huge Kickstarter success with $630,000 raised and 25,000 games shipped.

Do you still wonder if test can be represented by a game? If not, let us return to our initial question about what we do as testers and see how a game can help us answer.

A game should put you in a testing role
A game should give you a testing goal
A game should let you face typical testing challenges
A game should offer you meaningful testing decisions

If our game can answer those questions, it will also teach us to think as testers. Our friend Doctor Reiner Knizia has expressed this very well by stating that ”it is the goal that is important, not the winning”. As you may have guessed, the good Doctor is also a designer of many good games.

Now let us start designing our test game!

Find the Bug!

For our test game, the two first questions are easy to answer. Your role is a test lead and your goal is to find bugs. To ”gamify” those concepts, we convert them into game components by letting our testers be represented by game pieces and our test environment be represented by a game board. To play the game, simply place a tester on a square and draw a tile to see if you find a bug.

So far, so good, but how about the other two questions? Where is the testing challenge and where are the meaningful testing decisions? As testers, we know that we never have enough time and resources for our testing, and our game must capture and convey this knowledge.

What if we limit the number of testers each player has? Then there will not be enough testers to place on all squares and hence not enough testers to find all the bugs. This is a bit better but now the number of bugs found is dependent on luck. We all know that good testers find more bugs, don’t we? We also know that the difference between a good tester and a bad tester lies in the decisions taken. Clearly, our game must answer the fourth question about meaningful decisions as well.

Now go back to your own everyday work and ask yourself how you test. Do you prioritize test areas where bugs are not likely to be found or to be critical? No, you probably apply some kind of risk-based testing and prioritize functionally complex or business critical areas. Why not let our players do the same?

What if we let our players use a tester not for testing but for analyzing the board and learning which areas are defect-prone? And what if we let them save a tester for retest or even automatic tests, covering several squares at the same time?

Our game now lets the players face several testing decisions, they must think like testers!

Find the Bug! deals with traditional V-model testing but how about modern agile testing? Can that be gamified as well? Of course!

Find the Bug! Agile

Our agile test game should answer the same questions as before about role, goal, challenges and decisions. Since agile testing activities are closely linked to development activities, let us put our players in the roles of scrum masters with the goal of coding components through test driven development and continuous integration. The code is represented by colored cubes and the components by tiles illustrating input codes and output codes.

The challenge for the players is to ”develop code” (acquire cubes) and ”test components” (place cubes) until the user story can be considered ”done” (the input codes placed on the components match the input codes required by the user story).

The decisions for the players are all about testing effectively (test so that you always have the code necessary to continue testing) and efficiently (test so that you pass as soon as possible).

Again, our players must learn how to think to ensure the early and continuous delivery of the Agile Manifesto.

Learn the Experience of a Tester

Find the Bug! and Find the Bug! Agile are only two of many examples of how game elements can be turned into testing exercises. As we have seen, games can teach students how to think like testers by letting accessible and memorable games represent test concepts. But testing is not only about creative thinking, it is also about experience and gut feeling, about the ability to expect the unexpected. Can games be used for that as well? They sure can!

Newly graduated students typically have a strong foundation of general skills, upon which specialized testing skills can be built. We are now ready to move on to the next skill step, that of experience, and this time we will do it the other way around by turning testing exercises into game elements.

Our journey will start with the fundamental test process of Plan-Analyze-Design-Execute-Close. For each of those activities we add game elements to strengthen the learning:

Motivation: The students take the roles of experienced testers
Context: The students work in a realistic project
Concrete: The students get real testing tasks
Social: The students work together in a project team
Answers: The students get to search for answers themselves
Feedback: The students get ”game benefits” for well executed tasks
Rewards: The students get to use their ”game benefits” to execute tasks better

Plan with the End in Mind

In the planning activity, a role play is set up where the teachers play stakeholders that the students interview to collect information. The better the students perform, the more artefacts do they receive. The artefacts are typical documents that we testers manage on a daily basis, such as requirements, design specifications etc.

Using those artefacts, the students write a test plan and get it graded based on how well it covers test activities as well as the project risks. The grades come in the form of ”mitigation cards” that the students may play later during the project when they encounter typical test issues. We will come back to those in the Execute activity.