Saturday, June 2, 2018

Testing Tour Stop #11: Pair Exploring Test Ideas with Viki

The first time I met Viktorija Manevska was back at Agile Testing Days 2015, the very first conference I've ever attended. Before the first conference day started, I joined the lean coffee session hosted by Lisa Crispin and Janet Gregory. Both Viki and I found ourselves at Lisa's table, sitting next to each other. This is where we started to exchange experiences. Since then, we met each other each year at the same lean coffee table on the first conference day! :D Viki helped me a lot by sharing her story as first-time conference speaker and providing feedback on my own first steps towards public speaking. We had several Skype calls over the last year which proved invaluable regarding knowledge exchange, just like all my sessions with Toyer Mamoojee. I really love how I have met up with more and more other awesome testers over the last months! I signed up for a virtual coffee to get to know other testers like Milanka Matic, Amber Race or Kasia Morawska. I really enjoyed the calls I had with Mirjana Kolarov. And I loved meeting Anna Chicu finally for the first time! All those meetings were invaluable, full of knowledge sharing, story telling and having a good time together.
Now, back to Viki. She just recently moved to Germany to work for a consultancy. Having worked in product companies before, her current project provided new opportunities as well as posed new challenges for her. We decided to focus our pair testing session on one of those, on a part of testing that might not come to mind as first: generating and challenging test ideas as input for exploratory testing sessions.

A Session of a Different Kind

This time, there was no software under test - instead, we tested and challenged ideas. Test ideas that Viki had already brainstormed and provided as input for our testing. The result? We discovered more ideas. We restructured current ideas. We removed duplicates. We challenged procedure and more. All with the goal to improve and make it better, like we testers usually do with everything that gets into our hands.

While brainstorming and discussing, we talked a lot about testing itself. Have you used tours as a method to explore a product? I normally use cheat sheets heavily to generate further test ideas, identify further risks, come up with further questions about things we haven't talked about yet. Cheat sheets like the infamous test heuristics cheat sheet by Elisabeth Hendrickson, among other great resources. This way, wee started talking about exploratory testing itself. How structured should it be? Sometimes I feel my own sessions would benefit from a bit more structure, but I'm still happy with starting out to look for risks in the area of A and ending up with discovering a lot in section S. However, exploring is not unstructured and seldom goes without a clear mission or note taking. What I really love is Elisabeth Hendrickson's definition of exploratory testing, as stated in her most awesome book Explore It!.
Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next
In my opinion, too much structure or being too strict about it removes the part of adapting your way according to what you learned. Having the product show you the way, or better multiple ways, following them to explore areas of unknowns. At times I wonder whether Maaret Pyhäjärvi was thinking of that when saying that "the software as my external imagination speaks to me"; a notion I often pondered over.

Now, the system to brainstorm test ideas for was a typical untypical web application. Something known like signing up for a user account, something unknown due to the specific domain. We started with ideas around negative testing, UI testing, smoke testing, and workflow testing. Going forward, we produced a lot more ideas, asking so many questions about the product, and also about testing itself.
  • Would you consider different browser settings like turning JavaScript off as negative testing? It could be a normal use case for certain users.
  • What about high security settings or incognito modes? Some users just care more about privacy than others, still they don't mean any harm to your application.
  • What about popup blockers or ad blockers? They are quite frequently used.
  • How to best limit the input for a name field? This made me think of Gojko Adzic's recent book Humans vs Computers as well as his famous Chrome extension Bug Magnet, a really useful tool to quickly test out valid or invalid input values.
  • Or the Big List of Naughty Strings! I've known it for a long time but haven't used that one much yet. There's definitely more to it.
  • This brought up the topic of SQL injection, a common vulnerability to check for. In general, user data must not be at risk, so let's see if we can get login information, take over user sessions, or simply can view other user's unsecured pages.
  • What about tampering with system settings like your local time, could you trick the application to provide you better offers that are based on dates?
  • You send emails? Really important to test those properly then. In one of my previous teams, formatted emails were displayed very differently using different email clients. Are the parameters of the email template filled with the correct values depending on given conditions? Are potential images attached that need to be downloaded? Are the links correct? Could the email be considered as junk and filtered out? How long does it take to receive the email?
  • What happens if you bookmark the URL while being in the middle of a process to share it with someone else?
  • What about the browser back and also forward buttons, could they lead to unexpected, inconsistent, or simply unpleasant behavior?
  • What about consistency regarding styling? Is the UI accessible, e.g. for color-blind people?
  • Which kind of browsers, browser versions, operating systems/devices, screen resolutions do we really want to support?
  • Would you have manual smoke tests? Why not automate those sanity checks instead of repeating them over and over again?
  • What about alternative paths through the application, mixing up the order in which you can do things or skipping steps - would you still reach your goal as a user? Should you be able to do so or is there a prerequisite for critical things?
  • Have you ever tried the nightmare headline game to discover what would be the worst to go wrong?
  • Oh and: although we received so many emails about it lately - have you really considered GDPR compliance?

Looking Back

Both of us loved the session! Viki said she was really satisfied with the outcome. It was great to hear a different opinion, especially from another fresh tester point of view coming from the outside. Sharing experiences, experiments and stories provided lots of value for both of us. I was really glad my input helped her further, especially regarding testing emails, SQL injection and GDPR. I personally really enjoyed to have a completely different pair testing session, exploring mind maps, challenging ideas and thoughts, discussing structures, procedures, and strategies. Exchanging information about different positions, roles, expectations, and ways of collaboration - which is so important as well when testing. Talking about exploratory testing, what we understand, what we do, how we give feedback and create transparency, how we provide value. Just as Viki said, quoted freely: "Name it like you want, but it's important to deliver value." I so much agree.
Posted by at
Labels: , , , , , , , , , , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)