Showing posts with label mindmaps. Show all posts
Showing posts with label mindmaps. Show all posts

Friday, October 5, 2018

Testing Tour Stop #21: Pair Accessibility Testing with Alex

Alex? Now, this Alex is not the Alex I already paired with. This Alex is Alex de los Reyes, whom I met in person at CAST 2018. He had listened to my talk "Cross-team Pair Testing: Lessons of a Testing Traveler" and therefore learned about my testing tour. And was intrigued! Inspired from the moment, he agreed to pair test with Amit Wertheimer.
But how to get started? Alex and I had a call where I explained in more detail how I found partners to test with, how I prepare sessions, how to run them and how to follow-up.
Even before we had this first call, Alex scheduled a pair testing session with me. Since then I was looking forward to this one and it turned out to be awesome!

Testing for Accessibility: Why, What, How

As with most partners, we had several topics to pair on in mind. In the end we decided to go for accessibility as we both wanted to get better in this area and for Alex it's becoming really relevant in his current project. Best time to practice together!

In the beginning of our session we decided to first try our skills on a practice application and afterwards tackle a real productive website to apply what we learned. The selected demo we chose was Accessible University 3.0.
"Accessible University (AU) is a fictional university home page designed to demonstrate a variety of common web design problems that result in visitors with disabilities being unable to access content or features. [...] Use the AU site to
  1. demonstrate common web accessibility principles at trainings, presentations, and workshops on accessible web design.
  2. learn common web accessibility problems and solutions in an easy-to-understand way." (Accessible University 3.0)
The demo offers an inaccessible site where you can detect at least 18 accessibility issues, a list of all issues including detailed explanations and the accessible version of the site having all issues fixed.

What can I say? We had a blast while discussing our experiences, sharing our accessibility knowledge and finding issue over issue! First, we looked for quick wins using our eyes (we're biased beings), things that instantly stood out that we knew could cause trouble for people who have to rely on a different access to technology. Second, we inspected the page structure and found even more issues using the experience and knowledge we had. And third, we used tooling to discover further issues and get inspired to explore more. Alex does not like the WAVE Chrome extension too much and rather prefers Axe. However, for this session we decided to go for another option, letting me get to know another feature of the wonderfully useful Chrome DevTools: Lighthouse, allowing you to run audits for accessibility. The report provided an overall score, automatically detected issues and their instances, as well as a useful checklist of what to explore for as it could not be detected manually. Super easy, fast and really worth it.

As I can really recommend to try this demo app on your own, I won't list our findings directly here in this blog post. If you'd like to compare your results with ours then check out our mind map of potential issues. In the end, we compared them with the provided list of 18 accessibility issues and found that we had identified a lot of them! At the same time, we learned about many more problems we had not been aware of at all before and solutions we even didn't know existed. Awesome!

Having still 15 minutes to go, we decided to follow our original plan and tackle a productive site as well. We selected agoda, a travel booking platform. Having only a very limited time box, we decided to instantly use Lighthouse and discovered the page's score was not much higher than the one of the inaccessible practice app. This way we found so many issues very quickly already. We also stumbled across other usability issues like changing navigation or not responsive design. Last but not least we tried to tab through the homepage which offered a navigation menu and a form to book a hotel room - and here's what happened. Just try it yourself!
  • For the first few tabs we saw where we were as navigation menu elements showed a blue highlighting frame.
  • The next few tabs suddenly did not show anymore any focus, so we got lost.
  • The next tab opened the calendar for the arrival date at the hotel - strange, as we had skipped the first form element to enter a destination.
  • We tabbed to the second calendar for the departure date, tabbed again and landed in a sort of drop-down / dialog / sub-form / whatever element to provide the number of rooms, adults and children.
  • We tabbed on. And found ourselves trapped in this very UI element! We could neither tab forward nor backward.
When reproducing the issue for this blog, I tried to see whether we really skipped the destination field, trying to tab backwards from the calendars - and noticed Shift+Tab did not allow me to tab backwards but had the same effect as Tab itself, both tabbing forward! I'm speechless.

Looking Back

As for each pairing session, we took some time in the end to reflect on what we learned and share our thoughts. First of all: The demo app was a really valuable target to explore, it raised awareness of the multiple traps to fall into when it comes to accessibility. Interestingly, we noticed we didn't even look at the accessible version of the page to see an example how to do it well! This is definitely still worth looking into. In any case, we both loved our approach of first practicing with the demo app, getting our brains prepared, and then switching to a productive application, being instantly able to spot issues there with the knowledge gained. We identified a lot within only 15 minutes - time worth spent.

It turned out again that bad accessibility design is often also bad user experience for everyone. Designing accessible sites improves the experience for everyone else as well. However, accessibility has to be considered already during design. Nowadays there are so many tools to support us, like checking for sufficient color contrast already in mockups.

Talking about our collaboration, it was once again a very easy session for both of us. We had agreed to use strong-style pairing and a mob timer to frequently switch roles. Alex also had made some experience with this approach already and felt comfortable with it. Our flow was very natural. Especially one fact caught our eye: we followed up on each other's ideas, using the "yes, and ..." rule from improvisational theater. Alex had already noticed that ignoring this is a common problem when pairing. In all the sessions I did on my testing tour, I found this to be one of the most important key points to pair successfully with strangers, being productive from the first minute on.

The session was thoughtful. It was super productive. It was so relevant for Alex' current project, where accessibility is a big part of the testing strategy. And the session was fun! Like several others before him, also Alex advised to "do it with someone else" as this makes it so much easier. "Don't test alone, for your own safety." I have nothing to add.
Posted by at No comments:
Labels: , , , , , , , , , ,

Monday, October 1, 2018

Testing Tour Stop #20: Pair Exploring Online Invitations with Claire

Pair testing with Claire Reckless was the next stop on my testing tour. We knew each other only from Twitter and Slack, have never talked or even met in person. Still, when I saw the scheduling notice, I knew this was going to be great. And it was!

The Goal: Explore

Claire did not aim to focus on a specific topic. She shared that she recently moved into a test lead role where she's not quite so hands on so it would be great to have a testing session where we could focus on doing some actual testing. She was happy to pair with me and see what happens.

Being for longer on this testing tour already with several session done, I now have a whole list of potential test targets ready to be tackled. For our session, I selected a sub-set of 5 applications that might fit our general exploration goal. At the beginning of our session, Claire decided to go for Evite, an online invitation service we both did not know.

We had already agreed in advance to pair in the strong-style way. Claire was familiar with this approach, so I simply shared screen control and explained shortly the mob timer that I prefer to use when pairing with new people. We set the timer to 4 minutes and off we went.

Stumbling and Frowning

Throughout our session, we stumbled on several issues, frowned on even more and documented all note-worthy points in a mind map. Here's what we found.
  • We started off on the Evite homepage, wondering whether we needed to register to create an invitation or how far we could come without signing up.
  • Right in the beginning, as first navigator, Claire asked me to open the browser's DevTools. When scrolling through the homepage, having a look around what this online service offered, we viewed several tabs highlighting the platform's features. When hovering over one them, showing you can handcraft invitations, we noticed in the DevTools that errors got thrown. An image was not found and a video not loaded.
  • As we had the DevTools opened and docked to the right side of the screen, its width got reduced. We noticed that this resulted in having the top navigation menu displayed as menu bottom on the top left. When opening it, we found a completely differently looking menu with differently ordered items. This felt strange and definitely not consistent.
  • We decided to search for invitation designs, using not the usual suspects that got offered but something different. We searched for "life role-playing" which gave us 0 search results back. We got suggestions to design our own invitation as well as several design proposals to start from, however, we missed what we knew from other applications to give us suggestions based on the search (like Google or Amazon does, for example). We noted it down as feature idea.
  • Going through the suggested invitation designs, we noticed that some of them had been replaced by advertisements instead, looking a lot like the other invitation designs. We did not like that at all, being a dark pattern tricking the user into clicking on these advertisements.
  • We browsed the offered categories, choosing "Birthday for Kids". The suggested designs seemed to be indeed filtered by category, and everything looked like a new search, our previous search term not maintained. We decided to use more filters and selected to see only free designs. Doing so, the result page told us that no templates matched our search although we had seen free templates for the category before. We realized the reason was that it indeed had maintained the former "life role-playing" search!
  • We removed our custom search term and then tried out further filters, like for color or animations. When filtering for themes, we noticed sometimes strange gaps in the search result grid, leading to an inconsistent layout.
  • We viewed the details of an invitation template. It showed the selected design on top, with navigation arrows on both sides. Browsing through designs this way, we found designs that had not been in our previous search result. Later I saw the tool-tip indeed says "Previous Design in Category", still, this felt not intuitive.
  • We chose to have a deeper look into the invitation template itself. We found that the design on top displayed placeholders for information we could fill in a form below. We instantly thought of security testing, if that would be our product to test.
  • The template offered a preview button, so we decided to use it to see how the invitation would look like. As we still had the DevTools opened, we saw that validation errors had been thrown. Only by scrolling down further we now also saw fields marked as required - not instantly obvious for users.
  • The required fields of the form were title, host and date/time. We filled title and host and decided to add a co-host. Interestingly, the co-host showed not only a name field but also an email address. Strange. We decided to leave the email out for now. Date and time were two different fields; we went for only providing a date. Trying to preview now, we got once again validation errors. The co-host email was required which was not obvious before, and the time as well. Well then, we added a time and any value in the email address field, having the validation errors vanish. Preview again failed due to the invalid email address. Okay, provided a valid email format.
  • We had selected an invitation template including a photo. It offered a default picture, but required us to upload a custom one before previewing. The validation message said "Please upload a photo first by tapping on the camera" - but which camera? Having scrolled down, viewing the form, it was not clear what was needed to do. Scrolling up, we saw a camera icon next to the photo.
  • Clicking on the photo asked us to choose a file from our file system, with a file extension filter on image files. We decided to view all file types instead and selected a pdf file to see what happened. A transparent overlay appeared on the whole screen, nothing else to indicate the user what went wrong. Again, as we had the DevTools opened, we saw what had happened. A JavaScript error informed us that they could not cope with the provided input. Clicking anywhere else on the page threw another error as the not-existing dialog could not be closed. We even had us thrown to a blank page when clicking on the zoom levels offered for the non-existing image. All this made us worry. If the application allowed unwanted file types and then could not cope with them this was a security issue. Selecting such a file type can happen by mistake, but also with malicious intent.
  • Back to the start. Choosing a template, filling out required fields, uploading an actual image this time. Clicked somewhere else on the page by mistake - and saw we lost our photo. The upload did not actually upload the photo yet.
  • We learned we had to confirm the upload by pressing a "Done" button which saved the image. However, hovering over our uploaded photo, the tool-tip still showed "No file chosen".
  • We noticed the upload photo button had changed into an edit button. Clicking it, a dialog was opened, titled "Custom Image". Hm, we couldn't select any standard image, so why custom? Also, the question asked whether we'd like to "modify" our image or upload a new one; the related button was labeled "Edit", however. Inconsistent.
  • We chose to edit our photo, noticing it shortly displayed the previous photo before our custom image which felt strange. We could zoom it, we could rotate it - in 45° angles! We would have expected 90° as in other applications, however, it made sense to us that more angles are offered for customizing invitations. We could drag and drop the image to a new position. Editing the photo again, we noticed the edit mode had discarded our selected zoom level, showing the image again in original size. Not nice.
  • Now, having fulfilled all prerequisites, we were ready to try the preview. Pressing the button opened a dialog telling us we could now preview the invitation - this came as a surprise to us! There was no loading bar or any other signal that would justify a separate dialog instead of simply loading the preview.
  • The preview was opened in a new tab, informing us on the tab that we're viewing a preview. The page was displayed as read-only, we could not interact with it. Strangely, the entered co-host was not displayed anywhere.
  • We decided to add more information and see what gets displayed on preview. The message from host was displayed. The created poll did not. Strange, as we would have expected to see a preview of the actual invitation including all information.
  • We found that the preview tab could not simply be reloaded, another preview had to be requested, getting a different URL.
  • We created a what to bring list; and found the field validation there to simply be incorrect. Filling only items to bring without quantity would validate the name field of the first item to bring, telling us to "Please fill out at least one field" - which we clearly had done. We added the quantity for the first item, left it empty for the other items, and the list got created successfully; only including the first item.
  • We found we could still change designs and our details got maintained. Phew. We decided to save the draft by clicking the related button, and got referred to a sign in page. The offered Google login button was framed by a red border, making it look like one of the field validation errors that we had received before.
  • Well, we did not want to register, but there was no option to cancel or return to the previous screen. So we used the browser back button - and found ourselves back at the selected template, but with all our previously added information lost!
This was a perfect point to end our session, exactly meeting our time box.

Focused, Fun, Fruitful

Claire shared that she enjoyed the session. It got her tester brain going again! She's doing lots of management these days so this was refreshing and welcome to her. She felt that the session was really productive, too. We found lots of things on all kind of levels, functionality, usability, even security. And time flew so fast! Using the mob timer set to 4 minutes, she was often surprised how fast these 4 minutes passed. She said this kept us focused, switching our roles frequently helped a lot. Also, we didn't go down too many rabbit holes and stayed on track.

From my point of view, it was very easy to collaborate with Claire. We knew each other only vaguely from online, we haven't even met yet, but I felt we spoke the same language. We did not have to explain what we meant, we had a shared basis and could instantly test together. It was fun! Another interesting point for me was that Claire was the first of my pair testing partners to ask me to open the DevTools from the very start. I really loved that fact as this is exactly what I do when testing on my own. When pairing, I try not to impose myself too much and see how others tackle testing problems. This time it was refreshing for me to see her having the same approach. And these tools let us discover so many things already, not even diving deep, simply by having the console and network tabs open.

Reflecting on what kind of issues we found testing for 75 minutes only, considering that this is a productive application, I really wonder how the people developing it approach testing and quality. I don't mean to assume bad intention, I am not the one to judge their circumstances and challenges. Seeing an application like this just makes me very curious. When we find so many clear issues on the surface, not even deeply knowing the application or testing further regarding security or accessibility or anything else, it really makes me wonder what else is there to find.

All in all, our session was again enlightening, we practiced our testing and collaboration skills, exchanged ideas and approaches. It was great. And the best thing: We will meet each other in real life quite soon at Agile Testing Days 2018 as both Claire and I are speaking there!
Posted by at No comments:
Labels: , , , , , , , , , ,

Tuesday, September 4, 2018

Testing Tour Stop #19: Session-Based Pair Testing with Guna

Guna Petrova had listened to my testing tour talk at a local meetup end of July. Right afterwards, on the same evening, she scheduled a pair testing session with me. What a wonderful feedback! We had a great time testing together.

When it came to selecting our topic, we had lots of ideas and areas we would like to dive into or improve our skills in. For our session we agreed to try an approach we've known for a long time but never actually tried out ourselves: session-based testing.

Prepare and Align

As neither of us was really familiar with session-based testing or session-based test management we had to first read about the approach. The following resources helped us prepare as well as guide us during the session.
We agreed to try this approach to support our exploration, so we needed a test target. We decided to go for the Hours time tracking web application. The domain is not unfamiliar, the product still unknown to us.

We also needed to decide where and how to take notes. As far as we could see, usually a simple text file is used which can easily be parsed for automatic reporting. For our session we did not care about the automation part, so we decided to have the sample session reports guide us, but not limit us. We wanted to create a mind map instead.

Taking the template and making it our own, in a visual way, was a great exercise in itself. We had to discuss and align which information to take over, where to put it and how to structure it in the beginning, knowing we will adapt it as needed during our session. We agreed to use child nodes along with notes on them to store more detailed information. Later during the testing session we also decided to use icons to mark issues so we could quickly get an overview on our findings.

While creating the mind map and aligning on everything, we agreed on our mission and our first charter to focus on.
  • Mission: Evaluate the web app whether it's suitable for a first time user
  • Charter: First interaction with the system, namely the
    • Web home page
    • Registration
    • App landing page & content
We noted down areas to test and session metrics. We planned to report issues and opportunities for future charters, as well as test data and tools used.

We agreed to time box our testing session to 45 minutes. We also wanted to pair the strong-style way, frequently switching the navigator and driver roles, having a mob timer support us. Finally, all agreements were made and we felt ready to start our session.

Time Flies

We started testing, tackling the very first contact point of most first time users of Hours: the web home page. If we would want to evaluate the product, this is where we would start to learn more about the application and create an account.

We found several issues already on this home page. We frowned upon lots of things we would not have expected. We also found praise! We took note of all these findings in the report section of our mind map. Besides that, we also stored details on what we tested in the notes section of our charter. I wondered whether I really would put down so many notes myself (I normally don't). The benefit I saw was that it would definitely help tell the testing story in the end when discussing findings. Guna added that this was also proof what we did, it felt safe to do it, especially in situations were you needed to show evidence of your testing.

Our mind map evolved quite naturally. We discussed and aligned ourselves. Suddenly we realized that not much time was left and we were still on the web home page, not even close to the other parts we wanted to look at. We decided to keep it with that, and still we went overtime. We tested for 55 minutes and therefore exceeded our time box by 10 minutes.

Debrief!

In session-based testing the session is debriefed right afterwards. We agreed to not use questions of the offered debrief checklist, but try the PROOF approach of Jon Bach.
  • Past. What happened during the session?
  • Results. What was achieved during the session?
  • Obstacles. What got in the way of good testing?
  • Outlook. What still needs to be done?
  • Feelings. How does the tester feel about all this?
This felt pretty fast and easy, straightforward, and indeed helpful. Mental note to myself: Do debriefs more often for your own testing; you might learn a thing or two.

Looking Back

In retrospect, both Guna and I liked our session. It was great to try something - together. It's way easier this way. As Guna put it: She wouldn't do session-based testing alone. Cassandra did it alone and struggled. But together it provided value. It's not a bad format.

What I liked about our approach was that we took the format and template, tried to understand the essence and then transferred it to a visual mind map, thus making it our own and adapting it to a setting which might be useful in our everyday life. Guna said documenting testing and findings like this seemed to be a neutral way to not force your own style on the other person. It felt great for multiple people working on the same stuff and it was a good reminder that there's more to consider for testing sessions.

Regarding our strong-style pairing, Guna offered interesting thoughts on it as well. We had to communicate and align ourselves. Whenever we felt we were not aligned, we explained why we would want to do this or that. We had to hold back sometimes, especially when we wanted to bring up something but it wasn't our time yet. These cases were the only time she noticed the pairing part. Besides that we were still having a natural conversation and flow. Her conclusion: we should do more pairing. I agree!
Posted by at No comments:
Labels: , , , , , , , , , ,

Monday, September 3, 2018

Testing Tour Stop #18: Mob Exploring with Amitai and codecentric

This stop on my testing tour was really special. Why?
  1. I saw Amitai Schleier again! We had read each other's tweets since longer and first met in real life at the Mob Programming Conference 2018.
  2. Amitai went on a coding tour this summer, joining several companies for one week each to learn together (find more about it on his blog). Most of the tour happened to be in Germany, and the last two days in Munich at codecentric. He knew about my testing tour and imagined us to have a common stop in my home town.
  3. We mobbed. My original hypothesis for my tour was about either pairing or mobbing, however, I only had the chance to pair before.
  4. The people on the mob identified as developers and had no experience with exploratory testing. My tour was about learning from testers, still, this promised to be interesting. I did not join the mob myself but stayed the facilitator. Though my tour was about hands-on testing, with me included, I learned a lot about teaching testing.
  5. My previous stop with Simon Berner was about pairing and mobbing itself. Only a few days later I could put our exchange into practice!
At first I saw this session as guest visit on Amitai's coding tour. Amitai, however, wished for counting it as stop on my testing tour as well. Even though I was not on the mob myself, this indeed felt like a bonus stop on my tour. So be it! :)

Setting the Stage

Amitai and Thorsten Brunzendorf already welcomed me at the entry. We had a nice chat, set up the room intended for our mob and waited for interested people from codecentric to join our session. They came, and even brought a test target we could use where they would value feedback on. Great, I really like to have our sessions to not only have the training effect but also have real-life impact.

After Amitai and I introducing ourselves and sharing some words on our tours, the session started. I decided to have a short introduction round in the mob as well, asking them for their name and core expertise. I knew everybody was identifying as developers, however, it was still very interesting to hear the different backgrounds, interests and specialties. Some people were into DevOps, some preferred backend development, or even tended towards security testing. I was pleasantly surprised to hear most of the group already had mob experience, only two persons didn't. After giving them a short heads-up, I went over to introducing testing, and especially exploratory testing. I presented my preferred definition by Elisabeth Hendrickson, as well as her charter template that I find useful especially for people new to exploration.
"Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next."
"Explore (target)
With (resources)
to discover (information)"
(Elisabeth Hendrickson, Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing)
I wanted to keep the basic theory short and have people learn by hands-on testing as soon as possible. I asked the ones who provided the test target to shortly introduce it and afterwards asked the mob to write their charter. They, however, were really unsure what they wanted to focus on as they felt they didn't know the product well enough yet. So I decided to have them do a recon session for the first rotations in the mob, mapping out the system. I also recommended to already jot down notes together as a mob, for example in a mind map. And off they went!

Let the Mob Explore

It was really interesting to observe this mob explore a product most of them saw for the first time. They were unsure about their procedure, trying to align themselves as a group. They jumped all over the place, already finding great issues; not taking notes of them. They started a list of assumptions and wishes they had for the product. And they shared lots of implicit knowledge with each other, just as I have witnessed it in so many mobs before.

After nearly one full round of rotations, I had them stop. I shared some of my observations and provided recommendations. I also asked them whether they now felt ready to write a charter so we could do more focused exploration, using a simple Roman vote for quick decision making: thumbsup - I feel we're ready, thumbsdown - we need more information, thumbs sideways - not sure which way to go. Half of the group voted up, half sideways. I could have used the mostly positive result to have them write a charter now, but decided to have them finish a full round of rotations before asking again. Now all thumbs went up! Chartering didn't come easy to them. I helped them through, trying to not enforce my own suggestions or opinion. They managed to come up with a first charter to start with, providing them the needed focus. Interestingly, they decided to explore user experience with the help of the source code! That promised to be interesting, and it was indeed.
  • They used the code as oracle - great.
  • They looked for existing unit tests - nice.
  • One person shared: "I still don't understand the test, can you shortly walk me through?" - liked that.
  • They fed different values to unit tests, in an attempt to use it for exploration - awesome! Unfortunately, they did not follow up on it, or left a remark for future sessions.
  • They said they should also test the feature from user perspective - exactly! It might be the code they saw was actually never executed, right?
Towards the end of our mob session they started a discussion of whether a certain feature was useful at all. They brainstormed ideas how a different solution might be way more valuable to the user in the end. They questioned the context the product might be used in. Many great points were made.

Retro? Debrief, Discussion, Feedback

On my testing tour sessions, I normally do a short retrospective in the end so we can learn how to improve (I learned their value on my very first stop with Maaret). This time, I did a debrief of the testing session with the mob instead, to have them reflect on certain aspects. It proofed really valuable to get them thinking about what they did. Mental note to myself: I should do that more often for my own testing as well.
  1. What was our mission?
  2. What did we test and what not?
  3. What did we learn?
  4. What feedback can we provide about the product, can we quickly derive it from our notes?
  5. What to focus on as next highest priority with the knowledge we have now? 
The response on the last question was my personal favorite. They wanted to talk with the product owner as next step to see if they were going in the right direction or whether other areas might be more risky.

After a short closing, the mob session was officially over. Still, the group stayed and started an informal discussion. One of the persons who proposed the test target now shared that he didn't want to spoil the session, but the tested product version was outdated already and they had indeed agreed to remove the challenged feature - so the mob's feeling was indeed correct here!

It was great to hear people's experience with testing, mobbing and knowledge sharing. I really enjoyed the conversation to conclude a great session. Afterwards, Amitai shared personal feedback with me. He said he could clearly see I did these sessions very often already. This feedback took me by surprise. I had facilitated several mob sessions since beginning of the year: three in my company, one at a meetup, and three at conferences. So this session had been my eighth only. Amitai shared it was received as really professional and it was exactly what he was hoping for. This showed me I had learned a lot and it also triggered me to reflect. The session had indeed felt good. Maybe because I taught developers new to exploratory testing? Would it have been the same in case they would have been other testers? Still to be answered. This I know for sure: I had shared a lot about testing and testers during the session while people were doing hands-on testing. The mob approach is just an awesome way not only to learn but also to teach testing.
Bonus: Amitai also shared what he learned on his blog as well as recorded a conversation with codecentric sharing their highlights from this stop; including my exploratory testing session! :D

Posted by at No comments:
Labels: , , , , , , , , , , , , , ,

Sunday, August 26, 2018

Testing Tour Stop #17: Pair Evolving a Mind Map about Pairing and Mobbing with Simon

On my latest stop on my testing tour I had the pleasure of pairing up with Simon Berner on the topic of pairing and mobbing.
We only had contact via Twitter before. It's always a great experience to put a real face to this sort of virtual connection and have a meaningful conversation with them.

Going on a Meta Level

Simon put a lot of thought into our session already in advance and proposed our topic as follows.
Hi Lisi
There are so many great things out there that I would like to discuss and explore with you, but so little time. I have something specific in mind where I know little about yet: to Mob. As you seem to have quite some experience around this topic, I would love to pair up with you on the topic of “to Mob” and elaborate a bit on it, if you are up for it. What are our experiences with MobTesting and MobProgramming in general? What are its benefits? When can it be applied best, when not? Are there different forms of it? Can a whole Sprint-Iteration completely be developed in a Mob? How can you introduce “to Mob” to a team who knows nothing about it? How do we learn best from its outcomes? … and many more interesting questions to discuss.
My intention in this session with you is, to create a MindMap where we both can share ideas/experiences and grow up on it (maybe me a bit more than you as you seem to be already a pro on it)
I’ll create a MindMap in advance with some of my ideas around the topic, so that we can get started on them.
Hope that sounds interesting to you
Cheers
Simon
If that sounds interesting to me? Indeed it does a lot! Our session could only last that long, but I'd love to continue the conversations around it.

Now, originally my goal for the testing tour was to practice testing hands-on. I read and talked a lot, but felt I didn't practice or try out new approaches and tools enough. In this sense, this pairing session was a bit different, as we had a paired conversation around a certain topic. Still I was eager to give it a try, betting there's much to learn out of it as well.

Evolving a Pairing and Mobbing Mind Map

As promised, Simon already prepared a mind map. And it was huge! It contained most of the things already, so it was great to get our discussion started, elaborate on certain points and develop it further. As words cannot express what mind maps can, let's start at the end and show you our end result. I decided not to edit it after we stopped, so this is exactly as we left it. The original file which allows to follow the links can be found here: TestingTour_2018#17_Pairing&Mobbing.xmind.
I really enjoyed that Simon thought of including an "introduction" node for us to get going. After all, we didn't know each other and this made it easier to start and learn one or two interesting bits about each other. This way, I got to know that he just started in a new team with huge potential where he's the only one to drive things forward regarding testing and collaboration. Simon already pairs with his teammates and now got intrigued to try the mob approach as well.

We talked a lot about styles of pairing and what comes with them. We both enjoy and prefer strong-style pairing the most. When it comes to explaining this form, I normally shared this blog post by Llewellyn Falco: Llewellyn’s strong-style pairing. Great to see Simon had it already in his mind map. And he also had the following post by Maaret Pyhäjärvi right next to it: Being a navigator in Strong-style Pairing. I really enjoyed reading it afterwards, as well as her previous posts in this mini-series The pairing experience: foundations and Being a Driver in Strong-Style pairing. (Update: Maaret just compile an article from her notes on strong-style pairing: The Driver-Navigator in Strong-Style Pairing; my new favorite post to share.)

Simon shared that when it comes to understanding the roles of driver and navigator in strong-style pairing, the metaphor of the rally driver and navigator was enlightening to him. When driving a rally, the driver has no time to read the map or think about what else to consider, they are fully occupied with getting them there. It's up to the navigator to take care of directions and instructions and thinking ahead. This metaphor was flipping the switch for him.

Another interesting point Simon made was that "listening is an art". I so much agree! Especially when he shared his observation when meeting people. He's interested in people and the one asking questions about the others. People have a tendency, however, to tell you about their lives and then not ask in return to learn who you are. I made this observation myself so many times as well. However, I also fell into the trap of the other side so many times already, being guilty of not asking back myself. Best example was when I was speaking at a conference and someone approached me asking questions, I was really happy I could talk about things I love for once and not only listen to others, so that I often forgot to ask back. Another factor here is that exhaustion makes it worse as Simon added. When I'm tired, I fall back into my worse habits and less social behavior.

Another interesting discussion point was what to do when you see that people are falling back into a traditional style of pairing, having the navigator take over control of the keyboard again. We both experienced these kind of situations already which can be tricky to solve. I realized that I personally grew with experience, so when I am the driver, I won't allow my navigator to take over. It's really hard for me, however, to be on a mob and watch the driver freely handing over full control to the navigator. I normally intervene, referring to the learning aspect for both. Sometimes this is helping people find back on track, sometimes my voice is not heard. Now, the worst thing is if I'm myself the navigator (maybe also tired) and have to actively stop myself and refrain from taking over. At least it's fully in my hands to stop it then.

When focusing on mobbing, we quickly came to the question how to introduce it to a team and tackle the likely question regarding productivity. I referred to a great keynote by Woody Zuill at the Mob Programming Conference 2018: "Mob Programming and the Power of Flow". Unfortunately this talk cannot be found online, however you can find my notes about it in my blog post about the conference, and Woody's webinar Enhancing Team Effectiveness with Mob Programming seems to be sharing about the same content.

In a mob, it's important how we treat each other. Kindness, consideration and respect are the often-shared ground rules for a mob. Simon and I felt "kindness" is kind of straightforward, and "respect" understandable as well; "consideration", however, is not as easily graspable. I admitted I had to look this one up the most often before explaining it to a new mob. I explained it with being aware of the others and their context and considering that everyone's doing everything to the best of their knowledge. And instantly admitted I would love to look it up again right then to see if that's correct. Now that I have written this down in this blog post, I really went back to the Mob Programming Guidebook and looked it up. And found that once again I indeed mixed "consideration" up with "respect"!
"Consideration is really about listening. [...] The place it is going to show up the most is at the driver seat. Often the driver will start by not listening to the navigator. [...] Another way that this will manifest itself is that good ideas will be spoken by members of the mob and will be totally ignored. [...] As a facilitator, it is your job to call attention to those ideas and make sure that everyone gets heard. Over time the mob will learn the habits of listening to everyone and people will find their spots to contribute. [...] Another aspect of consideration is to remember to allow other people to shine. [...]
[Respect:] We always assume that the person who wrote the code before us did the best they could with the knowledge and circumstances they were in at the time they wrote it. [...] We want to create a space that is safe."
(Maaret Pyhäjärvi & Llewellyn Falco: "Mob Programming Guidebook" version 2018-03-24, chapter "Working in You First Mob", section "The Rules for Working with Each Other")
The next interesting topic was the size a mob should have and when it stops working due to the number of people involved. I learned that three persons already form a mob, however, if one leaves the mob for a break you are back in a pair. Therefore, a minimal size of four people is working out really nicely. When it comes to the maximal size, the general idea that I picked up was that the size of a mob is still right when everyone's either learning or contributing. If you have a normal-sized agile team it's perfect. I heard from two team mobs temporarily joining forces in a bigger mob, or from people trying a mob with a full meetup group. So far I still made good experience with a mob of ten people myself.

We were running out of time and still had many more topics on the mind map. So I asked Simon what would be the last most valuable topic he'd love to have input on before trying to mob with his team himself. Simon wanted to learn about the "don'ts" in a mob. Two points came to my mind instantly.
  1. Don't force anyone into the mob. If they are not willing to give it a real try, leave them out and only do it with the people who want to. Make it fun so they might want to join in next time, as I learned from Maaret. In my team's case, the two most skeptical persons who did not want to be on our mob either changed team or the company soon after, so I cannot tell from experience how it might have evolved with them. Still, we cannot force people, only convince them.
  2. Don't force anyone to stay in the mob. Besides from the very first mob sessions, we made very good experience with allowing people to take any break they need and temporarily retreat from the mob. These can be as simple as toilet breaks or getting some water, with them being back on the mob very quickly. These can be meetings that just happen to take place at the same time, like a one to one with our manager or an interview. These also can be times people need to fall back to solo working mode and tackle other tasks they have. With this rule we made it easy for everyone to opt in and out, and the result was we were even longer together in the mob. Just the freedom was needed. And the coffee breaks? Since then we nearly always do them together, staying synchronized, and having that extra bit of socializing that helps so much when working together.

Retrospective Time

Simon shared it was a wonderful experience for him, joining me on my testing tour. He was nervous in the beginning as we didn't know each other (just like me always!), but it turned out to be a cool session. He also got a lot of information and value out of the session, which I loved to hear. He didn't think we should have improved anything in our session, as a lot depends on the energy flowing between two persons and ours was great.

I really enjoyed our discussion and the additional food for thought when it comes to pairing and mobbing. The preparation work Simon had put in already in advance was simply awesome! So much more to dive deeper into, lots of resources new to me. It was really interesting to try strong-style pairing on such a meta level topic, evolving the mind map with our results together. Because this is what we did! We used a timer and switched roles, however, for this kind of a topic it was quite hard to do and we often lost discipline. Still, both of us could contribute and learn. Simon made a good point here: it's the natural flow that matters, we don't have to force people into a box. It was as if we would have done it before.

All in all, it was once again a pleasure to pair up with great people of our wonderful community. There's so much to learn from them, to give back, and to discover new insights together.
Posted by at 2 comments:
Labels: , , , , , , , , , ,

Wednesday, July 18, 2018

Testing Tour Stop #15: Pair Evolving a Test Strategy with Toyer

The testing tour continues. Today I had the honor to pair test with Toyer Mamoojee. Since end of 2016, when we agreed on our first pact to try ourselves as conference speakers, we are having a call once every two weeks. We talk about all things testing, exchange experience, trigger thoughts, provide feedback and support. Still, we haven't ever tested a product hands-on together. You can imagine how happy I was when Toyer took the chance and scheduled a pair testing session with me on my testing tour!

What topic to pair on?

That's one of the common first questions that come up after scheduling a session. In this case, Toyer said he would love to do something from scratch, gather test ideas together, align our thinking. We know each other and our viewpoints quite well, but we haven't yet practiced testing together. He wanted to see how we really go about testing certain things, ask questions and see each other's thought processes when actually testing.

Based on his input, I thought let's tackle an application we both don't know, explore it and come up with a very first test strategy based on the gathered knowledge. As I knew that Toyer discovered mind maps for his work some time ago and learned to love them for many purposes, I thought that this could be our way to document our very first strategy draft, keeping it lightweight, easily editable, visual.

Having a list of potential applications to tackle at hand, I reached out to Toyer and asked whether he wanted to learn more details about my idea before our session, or rather not. He answered "I'm tempted to say share more information.. but I would like to be surprised too, as I want to see how I can tackle something without preparing". So he chose the latter option to which I can really relate. Over the last years I am continuously learning to tackle things without over-thinking; and I'm not done with learning this yet.

Evolving a Strategy While Exploring

At the beginning of our session I presented my topic idea to come up with a test strategy for a new product, and Toyer agreed to go for it. Lucky me, otherwise we both would have had to cope with an unprepared situation! ;-) So I provided different options as our system under test of which Toyer chose Blender, an open source 3D creation tool. I had some rare encounters with this application back at my first company when we developed an AI middleware for game developers, but had hardly touched it ever since. Toyer thought it looked really promising as we normally don't get to test these kinds of applications.

Toyer shared that first of all, he would ask which kind of need this application wants to fulfill and do related research upfront. For the limited time box of our session, however, we decided to skip this and explore ahead. Toyer accepted my suggestion to draft our strategy in a mind map, so we created it and continuously grew it according to our findings. He also agreed to do strong-style pairing while exploring, so I started up my favorite mob timer, set the rotation to four minutes, and off we went. It quickly became clear that we knew each other well. Collaboration was easy and communication fluent. We could fully focus on exploring Blender from a high level point of view, trying to grasp its purpose and main capabilities, identifying limitations and potential risks. We were actually doing a recon session, just as Elisabeth Hendrickson describes in her awesome book Explore It! Reduce Risk and Increase Confidence with Exploratory Testing.

Throughout our session we gathered lots and lots of findings and discoveries, adding more and more important points to our test strategy.
  • Learnability. The application is not intuitive at all. It's a real expert tool. Still, everybody is a first-time user once, and even if you know the domain the user experience this product offers is not that great.
  • Functional scope. The more we explored, the more functionality we discovered. The whole tool seems really powerful, but again, is not easy to understand.
  • Screen resolution. The GUI is cluttered with many UI elements, sidebars, popups and more. On our laptop screen that was already a challenge, and it will still be one on larger screens.
  • Usability.
    • Menus, popups and tooltips looked very similar which made it hard to distinguish the purpose of each.
    • Feedback on actions was often missing or confusing.
    • Some sidebars displayed content related to views we previously visited, not getting updated with information of the current view. This way, they sometimes obscured needed information.
  • Consistency.
    • Some actions worked in one area but not in the same way in another.
    • Some sidebars were named x but the header label said y.
    • Some delete actions asked for confirmation, others just instantly deleted the item.
  • Portability. We tested Blender on macOS. The product is also offered for Windows and Linux. At several points we found strange unexpected behavior and made the assumption that it might have been due to porting issues for the macOS version. For some points, I could even confirm that assumption when writing this blog post and checking the Windows version of Blender.
  • Maintainability and reusability. The GUI offered many sidebars, popups and views that shared similar layouts and menus. We noted to investigate whether they were duplicated or re-used components.
  • Robustness. We encountered error messages on invalid input that was not caught or prevented.
  • Automatability and testability. The application offers a Python API. We found Python commands offered in tooltips, the API reference in the help menu and an integrated Python console. The console behaved differently than we knew it form other terminals, but still it was very interesting that you could automate operations; which would also increase the product's testability.
  • Discoverability and documentation. The help menu offered an operator cheat sheet; clicking on it triggered a temporary message to look at the OperatorList.txt which we could not find. Only later I learned that we had not come across the text editor where you could open named text file. What a hidden support feature. Also, we found the linked release notes page to be empty. We didn't dive deeper into the manual, but all available documentation would have to be tested as well, especially for an expert tool like this.
All in all, we made our way through many parts of the application. We made quite some assumptions. And we found we still haven't seen a lot yet. In the end, we didn't have a final test strategy, but a good starting point to be iterated over.

Time to Reflect

We covered a lot in the limited time. We gathered lots of insights, ideas, assumptions to verify. We tested a product in a domain we both don't know much about, being a desktop application instead of our usual web applications. We tried to gather information and keep a holistic view on things, not diving deep yet, focusing on uncovering the different aspects to test for such kind of a tool. All the way mapping out our world and the points to tackle in our test strategy. As we learned more, our strategy evolved. We didn't reach an end by far. If this would be our product to test, we would iterate over it while learning more.

The unknown domain had its own charm. We approached the product as a black box, not looking under the hood in the first place. We brought lots of testing knowledge, but quickly saw we lacked the domain knowledge. Toyer made an important point here: when hiring a tester for such kind of a product, it would be best to look for someone who was already exposed to these kind of tools or related areas of expertise. We could still provide lots of value and ask questions which might go unasked otherwise, but we would quickly pair up with a product person or business analyst to model the product from domain point of view. And also sit with developers to model the product's architecture.

Pairing up helped once again a lot. To see different things at the same time, by looking at different parts of the screen. To grow the mind map way faster than any of us would have done on their own. And to include different thoughts and viewpoints.

Enrich Your Experience

This was the fifteenth stop on my testing tour so far. In the beginning I had only planned ten sessions, one per month from beginning of this year until end of October. Although I already overachieved my initial goal, each further session enriched my personal experience and brought me in contact with different approaches to learn from; all that while practicing my skills hands-on. Right now, I am reflecting on my whole journey so far as I am crafting a talk about my experiences on this tour which I have the honor to give at CAST and SwanseaCon this year. And just while doing so, another tour stop was scheduled with me, further people indicated interest to pair up or listen to my lessons, and I'm having further test sessions with awesome people. I'm curious where else this will lead me. What a wonderful time.
Posted by at 2 comments:
Labels: , , , , , , , , , ,
Subscribe to: Posts (Atom)