For the first stop on my tour I consider myself really lucky and honored that Maaret Pyhäjärvi agreed to pair test with me. I admire her work in the community and was glad to meet her personally at last year's Agile Testing Days, where I could see her testing and critical thinking, coaching and feedback skills in action; to name only a few. With every day of our session coming closer, I grew more nervous about it; even though rationally I knew that this was an awesome opportunity to learn even more from her, and that I have nothing to lose. In hindsight, our session was absolutely worth it as I indeed learned a lot in a short period of time.
Just did the first stop on my #testingtour: #pairtesting with @maaretp. Learned a lot already, thanks Maaret! Now's the time to reflect, will blog about my lessons soon.— Elisabeth Hocke (@lisihocke) January 10, 2018
The SessionWe both wanted to dive deeper on exploratory testing and practice our skills. Maaret proposed to tackle the Freemind version for Mac; according to her it's "buggy as hell and therefore really rewarding for newbies". This makes it a great system under test when teaching about testing (idea noted ;). To reduce the scope of our session we focused on the find & replace feature of the application.
The identified issues were spread all of the place. Some questions we asked ourselves several times:
- What does this mean? What does this message try to tell us?
- Why does this feature behave in a completely different way than similar features in comparable applications?
- Why are things so inconsistent? Why do labels or icons imply different functionality than they actually offer? How come even a simple form shows different spacing between input fields, just a few pixels below each other?
- Why is this data displayed in this location? It does not have anything to do with the feature. Why not put it in a different place where it might actually provide value?
- Why would we need this data or feature in the first place?
- Where is the functionality I am looking for? Why is it so hard to find things and learn this tool?
What I Learned about TestingReflecting together on our testing session, we addressed several important topics. Here are my lessons learned about testing.
- Where's our box? Who says what's in scope and what not? In this session we defined our box fluently on our way, calling each other out in case we started to divert from it. This was totally fine for us, but in the end we would have loved to have an overall picture of our box to see what we intentionally left out and postponed for further sessions.
- You discover there are too many combinations requiring a lot of time effort? Check if you can automate your test data setup. We didn't go this way in this session, but this was one of the things we could do in a future one.
- Whatever you document - always chose a notation that lets you quickly identify how many issues, questions, follow-up charters there are and what your feeling about the level of quality is.
- During the session we talked a lot about different aspects of exploratory testing - without even mentioning the approach itself. Observation as a skill, heuristics, and more. It's quite easy to start with exploring, but it's always experience that brings you further.
- Last but not least: testing and finding bugs is fun! I really love hands-on testing and close collaboration with people to get the best out of us.
What I Learned about Myself
In this session this fact became obvious when it came to note taking. In the beginning of the session I told Maaret I'm just shortly noting down what we learn. To do so, I had a physical notebook right beside me and scribbled down on it as we proceeded. When reflecting on our session, Maaret asked me whether I had used a certain notation to quickly get a summary from my testing notes. I had not. Which stunned me. When I do testing solo, I always use a digital form for my notes, be it a comment on a ticket, a mind map, or anything; but always including a certain notation to quickly let me know how many issues I found, which questions are still open, what other things I postponed for further sessions, what should be automated, and so on. But this time I totally diverted from it.
So Maaret asked me why I had chosen a physical notebook in the first place this time? The best answer coming to my mind was that maybe I fell into a habit. As I only use a physical notebook in meetings, be them at the same location or online. Like in interview calls. Did I fell into a habit because of the situation I was in? Why did I divert from my normal testing behavior? After reflecting a bit further, my intentions were the following:
- I told Maaret I was taking notes as I wanted to let her know what I'm currently doing as we couldn't see each other (we turned off video to improve call quality).
- I wanted to roughly keep track of what we were doing to help our testing.
- I wanted to have notes of our session to make it easier for me to blog about it later on; as this was part of my experiment.
What I Learned about Pairing
Also, we didn't really talk about how to do pairing. We both assumed we would do strong-style pairing, with one being the navigator and one the driver on the keyboard. We both share this knowledge and are used to this approach (probably Maaret more than me, but I made great experience with this style over the past year). However, in future sessions with other testers I have to set the stage first so we have a shared understanding how to pair.
What I Learned about This Experiment
Where to Next?
Thanks so much @maaretp, I really enjoyed it as well! You've given me a lot of food for thought, thanks for sharing your observations and feedback. Invaluable.— Elisabeth Hocke (@lisihocke) January 10, 2018