Showing posts with label accessibility. Show all posts
Showing posts with label accessibility. Show all posts

Friday, September 1, 2023

SoCraTes 2023 - A Place Where I Belong

I nearly didn't go to SoCraTes this year, the "International Conference for Software Craft and Testing". My speaking budget was already strained, my schedule overbooked, and it would have meant going on vacation time. But then the organizers reached out and offered me yet another slot on the training day this year. They were even fine with me giving a workshop I already had prepared, and I changed my mind. I seized the opportunity and went to SoCraTes on vacation. What can I say, I don't regret it one bit! Granted, I'm super tired, and at the same time I'm super happy. It was so much worth it and convinced me to reserve this time of the year for 2024 as well!


Coming Back

It's always super exciting to join a conference for the first time. The second time around, a few things are already clear - you know the venue, more people, the procedures and other things that reduce your cognitive load. Still, the second time is curious as well - how will they welcome me this time? How easily can I reconnect to where we left things last year?

The first moments are awkward for me, at any conference. At SoCraTes, this very quickly vanished into a feeling of belonging. I felt welcomed, I was included, I had a right to be there. This is the foundation for everything coming afterwards.

My first contact after seeing familiar faces at registration was a person being there for the first time, Lydia Leifels - such a pleasure right from the start! At dinner, I met dear people I already know for a while, like Tobias Göschel, Thierry de Pauw (along with their daughter), Juke Trabold, and Woody Zuill. There were so many awesome folks I got to know or meet again over the course of the conference! Like Janina Nemec, Marc Kalmes, Stefan Scheidt, Claudius Link, Lea Rosema, Waldemar Tomme, Martin Schmidt, Markus Tacker - well, the list could go on and on.


Training Day

The second edition of the training day was even better than the first one. Trainers were amazing both times, yet for this year organizers listened well to the feedback and crafted a schedule of three tracks with each session having enough time to dive into the topic and generous breaks in between. Just awesome. Here's the choice of sessions I made.

  • "Enforcing Architecture Using Tests" by Javiera Laso. This was the first time I heard about ArchUnit, a library to check for conventions like file names, structure, dependencies and more.  Writing tests was quite straightforward, and I can see how these could support maintainability and reduce friction by codifying agreements.
  • "Modernize CI/CD Session" by Raimo Radczewski & Chris Neuroth. Very interesting talk sharing fundamental principles for a pipeline optimized for quick feedback while going small, safe steps. They demonstrated live how fast a change can be on production. A few statements I really related to were these: "minimize the time from code written to code on mainline, deployed to real users, running in a real environment, ready to be evaluated - there is no other way to develop software sustainably", "influence the loops you can", "the moment we stop slicing we deliver slower", "skip the review, pair with someone".
  • "Ensemble Exploratory Testing" by me. Fun fact, I've given this workshop now for the 10th time so it became my most repeated conference session ever so far. Good thing is, it seems it doesn't get old! This time again, people were eager to join and seemed to enjoy the learning experience (which hopefully convinced them to try new approaches back at work). Well, I had fun observing people having lots of aha moments together.
  • "Take a mess, make a mess, fix the mess" by Aki Salmi. A very interesting session on refactoring code that's untestable and contains hidden domain concepts, code that's still valuable yet needs to be modified. How? Together, we tried an approach many of us had not seen before: not even trying to understand the code in the first place. Instead, using the IDE's automated refactoring tools to slice it up first, turning hard to test code into easy to test code. Then we can document its behavior in tests, and hence gain our safety net to make the required changes. Check it out, you can follow every small commit Aki made. Nicolas Carlo also wrote a great post on this where you can follow an example: "Another way of refactoring untested code" (by the way, his newsletter is super insightful and a clear recommendation).

After the training day ended, the main part of the conference was opened with a world café. Find a group of people you don't know, take a question as starting point, see where the conversation leads you and doodle your insights on a shared canvas. After a given time box is over, everyone find themselves new groups besides one person staying at the table and getting the newcomers on the same page. Repeat until you finished three rounds. Really nice exercise, perfect to get to know first people ahead of the main conference part and have interesting conversations emerge. One main theme we had was on people, community, culture and how that's foundational. In case you're wondering where to start, I have a resource collection on inclusion that really helped me.

That's not the end of the day, of course - SoCraTes goes all in! Everyone is at the same place, we're together the whole time, so of course there's an evening / anytime during the night / morning schedule with bonus activities suggested by anyone. Such a good thing our hotel rooms are right there to retreat and rest any time. I love how Juke as facilitator and also organizers continuously emphasized the importance of caring for your needs and taking breaks.

Having learned from last year that joining all sessions can be quickly very exhausting, I decided to use the evening time for conversations and enjoying both atmosphere and company of wonderful people.


Open Space Day 1

Days start early at SoCraTes for me being a night owl, yet it's worth it. In its core it's an unconference offering an open space for everyone to bring their topics. Things they want to share, conversations they'd like to have, apps they want to build, tools to try out, skills to practice - and also challenges they'd like to get help on. With so many different people, there's a whole range of topics offered, covering a spectrum of deep dive tech topics to humans and culture as foundation for everything. No matter if it's very personal, related with our professions, challenges in society, or all combined, everything is represented. There are also sessions like doodling together, painting your nails, talking about sheep, whatever is most valuable to people right now. Be prepared to be surprised! This format of building your own schedule together on the fly works amazingly well. It definitely worked out well for me again this year.

  • "Web accessibility - building beautiful web sites that don't make you puke" by njan Völker and Lina Sievering. I had hoped to learn more about accessibility at the conference, and already the first session was right on spot! Awesome workshop focused all around motion sickness induced by websites and apps. A topic close to my heart as I'm affected myself. Njan and Lina provided mindful and enlightening exercises to convey the problem and think of more accessible options together. Thanks to them, I also learned about the European Accessibility Act becoming effective as of 28 June 2025 - which means accessibility will be enforceable in Europe which hopefully gives us more leverage for accessible solutions when making product decisions.
  • "Capture the flag together" by me. As part of my personal AskAppSec challenge, I recently tried out further services offering hacking labs like TryHackMe and Hack The Box to practice penetration testing. The first challenges were good fun to me, so I thought why not offer a session and do it together during the open space. I was positively surprised how many people came and joined me! I opted for Hack The Box and their starting point machines. It worked super well, people were engaged, shared lots of knowledge and we captured a flag together (the second we missed only due to my VPN interfering). A very insightful experience, validating my assumption that there's interest and these could be great sessions for more people to learn about security in a fun way.
  • "Security for devs (& everyone)" by Claudius Link and me. The more security sessions the better, right? So why not host another one, together with Claudius who had the same idea. We had a great conversation with people bringing up all kinds of insights and common challenges. Nothing was immediately new for me, and yet it was validating to hear experienced people share similar views.
  • "Documentation as Code" by Markus Decke. We work together at the same company, and we are transitioning to having more and more documentation as code, so I was curious about other people's ideas, struggles and in general experiences. We talked about benefits and use cases, shared tooling options and their limitations. One of the main themes was around what problem we're trying to solve and then aim for tackling exactly that, nothing else not to document only for the sake of documentation.

What about the evening? Well, after open space is before open space! As mentioned, there's always an evening schedule people are building up. Once more, I opted for dinner, conversations and more fun time! Last year I discovered that Janina Nemec is an absolutely pro in playing Set, a card deck game I've learned to love through the testing community. At first we didn't spot any deck so decided to opt for a round of Exploding Kittens (so much fun) - and finally discovered a Set deck in the end. Well, next year we're prepared to bring our own decks. Such a good way to close the day.


Open Space Day 2

New day, new schedule. Every day starts a tiny bit later, which is still early for me. And yet it was awesome again and worth the early start.

  • "Stop being a superhero!" by Janina Nemec. She did an early dry run of her upcoming talk to be presented at Agile Testing Days this year - which is your chance to catch this talk, you'll be in for a treat! Janina has vast experience of working in an ensemble full time for many years. In her talk she describes (superhero) behavioral patterns she's observed (like the architecture wise or the coding wizard - way too relatable) and the pain points that result from them (like lots of work in progress without things getting done, or the behavior not helping the team grow and work sustainably). There's a solution for this: working together as an ensemble, or team programming as she calls it - saving the world together.
  • "Build a minimal showcase app" by me. I have a recurring argument around a security topic and wanted to finally start building a minimal app to demonstrate good security practices. So what could be better than get this started together right at the conference? Nonetheless, I nearly didn't dare to suggest the session. And then I thought no one shows up (I didn't realize it was break time). Until people did show up and we formed a wonderful little ensemble helping me get started on a good way. Everyone else enjoyed setting things up and realizing that we all struggle in certain areas. When doing it together, however, we usually have the missing piece of knowledge in the round to avoid friction, solve problems without frustration, learn with fun, and get to value fast. A pleasant experience, would have loved to continue together.
  • "Security scanning in pipeline" by Raimo Radczewski & Chris Neuroth. Both wanted to try out security scanning tools like Trivy, Syft and Grype on a real case example and see how they work and what value they bring. A really interesting session that then also sparked a serendipitous hallway conversation on why attack trees might work better compared to threat models in order to get people to think like malicious actors and consider risk.
  • "Security games" by Claudius Link. He brought a whole bunch of games to teach security in a safe space with fun. We could try out a few of them and gain experience how they could be used for educational purposes. Games like Elevation of privilege and OWASP Cornucopia, yet also [d0x3d!] and a lot more I can't remember. Now I know there are a lot more out there to try out!

This was the last day for the main part of the conference, so we closed it with a retrospective. The best part here was that it didn't merely gather feedback for organizers and Juke as facilitator, it also was  intended to provide feedback for each other as participants. Over half of my group were here for the first time, and I just loved hearing their feedback and also input to make it even better for each other.

A gratitude round followed. The challenge was to thank five people we have not thanked yet. Honestly, this experience was a bit overwhelming, in all the best ways - and more than one feedback took me by surprise. I haven't mentioned yet, this conference goes big on hand-written kudos cards you can hand out any time for anything you appreciated the other person doing. The ones I received I will keep with me for long, they present such a dear memory.

Dinner followed with more great conversations. Then further sessions were hosted (remember, the evening schedule). I had a great time joining a code kata ensemble. We did the "Vending Machine Kata" which was an insightful exercise itself, yet my main takeaways were on the collaboration part. It was fascinating to see an ensemble start off quite free style and converge to more structure like having a dedicated navigator, using a timer for rotations, having the navigator stand up to be more prominent, etc. It just worked better with folks who have never worked with each other in this way. Also, we used the fish-bowl approach to ensembling, having outside observers and always open spots to join the ensemble. This exercise made me realize once again that this approach is simply not for me, despite having super kind and safe people around me it felt exclusive. I'm very sure it's actually more inclusive for other people to opt in. For me I much prefer the "all together just one ensemble without observers" format.

Last but not least, a retro gaming session! We played the old adventure game "Zak McKracken and the Alien Mindbenders" all together on a C64 system provided by Tobias Göschel - how awesome was that?


Workshop Day

Have I said the conference is over? There's still the additional workshop day! And what better format to have than asking people to bring their topics for hands-on sessions also on this day. It's also the time of the traditional code retreat, practicing the whole day solving the same kata in various ways. Originally, I was adamant to join the code retreat again, it was simply an amazing experience last year. And then Claudius Link came and suggested the only thing that could possibly lure me away: co-facilitating a security workshop. Well, what shall I say? I couldn't resist, this fit way too perfectly to my personal AskAppSec challenge this year.

We aligned on our main thoughts of what to aim for, pitched it to a fellow participant, incorporated the feedback, and came up with a workshop on "Painless Security". We crafted the agenda shortly before and went ahead with it, playing it by ear and experience in giving workshops. Having co-hosted a session the other day helped, too. Co-facilitation worked super well together, we often thought along the same lines and built on each others ideas. People shared many painful experiences and we gathered potential things to try out and what we related with most to bring back to work. I took a lot with me myself.

During lunch time, the security theme continued for me. First thinking about security conferences and ideas to contribute to the space together with Claudius Link and Susanne Neunes. Heading back towards the conference rooms, I noticed another table where Martin Schmidt and Philipp Zug, who also both participated in our security workshop, were scheming on a new security card deck. I loved the idea and they were so kind to invite me in. Well, it seems I got myself involved in two new topics - and yet I feel these are very much worth it.

I decided to check out what other sessions might still run that I could join. First, coffee though - and that's when Matthias Klass approached me and asked whether we might have another capture the flag session together, as a couple of people expressed their interest. Well, I didn't need to check the schedule anymore, security theme it was for the whole day! Of course I'll host another capture the flag session. What can I say, it was awesome! We spent the whole time until we officially had to leave the room and head for dinner. Nearly instantly, the idea popped up whether we could ask the hotel for opening the room once more during the evening. Asking was worth it, we got the keys and went all in after dinner. Many more rounds of capturing flags followed until everyone was so tired we couldn't think anymore - while being just super happy. Some of the challenges tackled were very familiar to us and hence solved a lot faster, others required us to piece together knowledge we usually don't need. All of them were very insightful and fun. So much fun. I loved that we all worked together so effectively as a big ensemble again. I really want to do more of this.

There's a traditional count of code katas done at SoCraTes, counting each exercise by everyone. I was really moved seeing so many people join me on practicing penetration testing, staying with me for so long and sharing my enthusiasm. It felt we just established a new counter at SoCraTes next to the code kata counter: the captured flags counter, aka the number of security secrets discovered. We collectively increased it to 8 overall! 


Why again next year?

Heading home, my heart was full, my brain energized, my body tired, and me super happy. I was certain that if I have any chance to be back next year I will take it.

Besides the obvious reasons of self-selected, very insightful content and just amazingly kind and inspiring people, there's a reason that is even more important. This conference is the best I've seen so far in intentionally designing welcoming and inclusive spaces. They continue to reduce friction and make it more accessible and safer to more and more people. Literally every year. There are lots of aspects of this to be found everywhere. Not only in the code of conduct that's actually being lived and enforced, or offering all gender toilets, or giving away tickets based on a lottery to level the playing field. It's in every little detail. Less noisy applause by waving hands. Exact food labelling and plenty of options for everyone. Child care service so lots of parents could join this year. Sharing Covid tests upfront for every day and encouraging masks.

The level of diversity already achieved has a huge positive impact on the quality of conversations and insights gained. A lot to learn and take with me to do better myself.

Sunday, October 6, 2019

TestBash Manchester 2019 - About Learning, Daring and Enjoying

Looking back at the last week of TestBash Manchester 2019, I have to say this was an absolutely amazing week! Thoroughly enjoyed this experience. Brace yourself for quite a lengthy report. ;-)

Arriving in Great Company

Already when boarding my plane to Manchester there was the first nice surprise. Sven Schirmer was going to TestBash as well! The conferring part of conferences officially started for me then.

After arriving at the hotel, we joined the first meetup preceding the workshop days. I really enjoy meeting a few people already before the official program starts. This gets me into the mood and it's not as overwhelming as "meeting" everyone at once.

Together with Sven, Richard Bradshaw, Mark Winteringham, Maaike Brinkhof, Göran Kero, Rick Scott and several more we had a great dinner and lovely conversations. I felt emotionally prepared for the workshop day.

Learning: Workshop Day

I really enjoy workshops for learning, especially when they are hands-on and interactive. This time, I could finally catch a workshop I wanted to join for some time now: "Using Dependency Mapping To Enhance Testing Techniques" by Melissa Eadon. It was great! Exactly met my expectations. Mel emphasized how dependencies are a huge risk in modern development and how everything is about communication. In working groups, each of us drew a dependency map of their team, product or a problem they had. The rest asked questions for clarifications, trying to identify where the problems are and what could be done about them. As simple as it may sound, this visualization exercise was extremely powerful. It helped to get the thoughts and knowledge we have from our brains onto paper so we could discuss about it together. It triggered us to describe the situation while drawing, and the resulting questions caused interesting insights. For example, I realized what part of our product I had nearly forgotten to draw at all; and indeed, this is a part that's not well covered with testing. Another example: by casually answering a question I realized that I had neglected testing one of the most crucial parts of our application! The identified risk areas are a great input to focus further exploratory testing on. Besides that, this exercise would be very interesting to do with lots of people in my team and also across teams. Everyone has a different mental model and this makes it visible, therefore serving as discussion base. As a plus: it's also a relationship-building exercise!

In the afternoon I had the chance to see Emily Bache in action in her workshop "Getting High Coverage Regression Tests Quickly". She introduced us to the concept of approval testing and its characteristics. How you can still do test-driven development, how to handle test failures, how code coverage and mutation testing can guide us. This approach especially comes in handy when it comes to legacy code, yet is also convenient for new projects. The best part of the workshop: Emily provided us four practice projects to work on, in the language of our choosing. Trying out approval testing and covering existing code with tests. One of the persons at my table could not get their IDE setup to work for approval tests, so we decided to pair up. Even better: he was familiar with strong-style pairing and showed great communication skills, so it was a real pleasure to pair up and solve the challenges together. It's just a lot more fun this way! :)

For the evening, I had missed the opportunity to sign up for another meetup, hosted at the BBC. So I ended up with plan B (which should have been plan A in the first place): dinner with Sven, Maaike, Göran and Emily! I'm so much enjoying spending valuable time with my community peers.

After dinner, we decided to go for a drink to slowly end the day. And suddenly, to my huge surprise, Patrick Prill was standing in the room! He is the one I test all my talks with at a local meetup. Same about two weeks ago. Yet somehow I haven't asked him if he would also be at TestBash Manchester - I should have! This was an absolutely nice surprise. On the one hand because I really appreciate him as one of the kindest and most insightful human beings on earth. And on the other hand because of my talk. Granted, his presence also made me a tad more nervous, and yet I knew this would be the chance to learn whether I managed to improve my talk. What a nice surprise!

Daring: Conference Day

Then the time had come. The main conference started. Our compère: the great Leigh Rathbone! Once more I sketchnoted most of the talks - besides the one directly before mine. Such a pity, I would have loved to learn from Areti Panou's story! Yet I knew I wouldn't be able to focus. I did so many conference sessions already, and yet I can be certain I will be super nervous just before, and super distracted right after my own session (so I didn't catch the 99 second talks either). I learned not to beat myself up because of that anymore.
This time I went last - a great honor, yet my nerves were on edge and showing. Also, this time it was not only a talk for me - there was an even more daring part included as well. On the one hand I shared my lessons learned on my #CodeConfident journey, and on the other hand I proved my increased confidence live on stage with my very first live coding demo! Small and short, and yet really extending my comfort zone. The demo gods were kind to me and all went well, so next time I'm ready for more! For here, I'll let the tweets speak for themselves.
After the conference we all came together for a meetup at a nearby location. Great food, even more great conversations - yet it was extremely noisy and I felt drained. So, a shorter evening for once; the next day another conference day waited for me! Time to get some rest.

Enjoying: Test.bash();

The next day it was time for the second edition of Test.bash(); overall, and my first one. It can be considered a more technical and hands-on focused version of TestBash - and I really enjoyed it. Lots of great talks all over the place, lots of live demos, too. Hosted by the wonderfully energetic Gwen Diagram! I got out of the day with more knowledge and even feeling refreshed.
Right after a great conference day, we all continued with a meetup hosted directly at our venue, The Lowry. Not only that, it was indeed inside its gallery! I love art, so this was a special treat. I thoroughly enjoyed the opportunity to have both great conversations as well as some quiet contemplating time when looking at the great paintings and sketches by this artist from Manchester. Simply awesome. I loved this.

And once more: A great dinner with great friends. Elizabeth Zagroba, Joep Schuurkes, Patrick, Sven, and Geert van de Lisdonk. Thank you for a great time

Open Space

Only one more day to go before TestBash Manchester was finally over. I really looked forward to the open space day, hoping I still had enough energy. Lots of great topics made this easy, I learned a bunch.
  • "Testing without touching" by Joep Schuurkes and Elizabeth Zagroba. What a great session! Our group generated lots of great testing ideas and assumptions to verify, just by looking at the first page of an application. Great exercise, looking forward to taking this back with me to work.
  • "Threat Modelling" by Saskia Coplans & Jay Harris. Always awesome to learn from great security people. This time we threat modeled a service of our choosing to learn which requirements to fulfill to mitigate those threats.
  • "Motivation & Productivity" by Rick Scott. Great exchange about all things productivity hacks and self-motivation. It's not easy to get things done, and everyone is different so we need to learn what works for us.
  • "Accessibility Quiz" by Ady Stokes. This was super insightful! Ady handed out a page full of UI examples illustrating different types of accessibility issues. Great way to learn more how we can include all people and at the same time make the lives of everyone easier!
  • "Tester Growth" by Melissa Eadon. Mel asked all of us three questions: Where do you want to go? Where do you come from? How did you change? Great opportunity to reflect on our own situation as well as listen to the experiences and wishes of my peers.
  • "Practice Mob" session by me, additionally initiated by Joep Schuurkes and Elizabeth Zagroba (who mob with their testing community once a week). We wanted to work on something hands-on, practicing together - using the opportunity of having your peers in one place. Most people were simply interested to experience a mob for the first time, so we went for Joep's idea: let's extend MobTime, my (so far) favorite mob timer! Its drawback: an annoying alarm tone. On our endeavor to change it, we faced several setup issues as the project had not been maintained anymore for longer. Still, the session was great, we made progress, and most of all: so much knowledge was shared within just two hours of mobbing. Loved it!
How to end this evening best? Of course with a nice dinner and some goodbye drinks, with Mel, Joep, Elizabeth, Rick and Geert.

Now this leaves me only with one more thing to say: a huge THANK YOU to organizers and volunteers! You did an amazing job. This was my favorite TestBash so far, and I'll remember it dearly.

Friday, October 5, 2018

Testing Tour Stop #21: Pair Accessibility Testing with Alex

Alex? Now, this Alex is not the Alex I already paired with. This Alex is Alex de los Reyes, whom I met in person at CAST 2018. He had listened to my talk "Cross-team Pair Testing: Lessons of a Testing Traveler" and therefore learned about my testing tour. And was intrigued! Inspired from the moment, he agreed to pair test with Amit Wertheimer.
But how to get started? Alex and I had a call where I explained in more detail how I found partners to test with, how I prepare sessions, how to run them and how to follow-up.
Even before we had this first call, Alex scheduled a pair testing session with me. Since then I was looking forward to this one and it turned out to be awesome!

Testing for Accessibility: Why, What, How

As with most partners, we had several topics to pair on in mind. In the end we decided to go for accessibility as we both wanted to get better in this area and for Alex it's becoming really relevant in his current project. Best time to practice together!

In the beginning of our session we decided to first try our skills on a practice application and afterwards tackle a real productive website to apply what we learned. The selected demo we chose was Accessible University 3.0.
"Accessible University (AU) is a fictional university home page designed to demonstrate a variety of common web design problems that result in visitors with disabilities being unable to access content or features. [...] Use the AU site to
  1. demonstrate common web accessibility principles at trainings, presentations, and workshops on accessible web design.
  2. learn common web accessibility problems and solutions in an easy-to-understand way." (Accessible University 3.0)
The demo offers an inaccessible site where you can detect at least 18 accessibility issues, a list of all issues including detailed explanations and the accessible version of the site having all issues fixed.

What can I say? We had a blast while discussing our experiences, sharing our accessibility knowledge and finding issue over issue! First, we looked for quick wins using our eyes (we're biased beings), things that instantly stood out that we knew could cause trouble for people who have to rely on a different access to technology. Second, we inspected the page structure and found even more issues using the experience and knowledge we had. And third, we used tooling to discover further issues and get inspired to explore more. Alex does not like the WAVE Chrome extension too much and rather prefers Axe. However, for this session we decided to go for another option, letting me get to know another feature of the wonderfully useful Chrome DevTools: Lighthouse, allowing you to run audits for accessibility. The report provided an overall score, automatically detected issues and their instances, as well as a useful checklist of what to explore for as it could not be detected manually. Super easy, fast and really worth it.

As I can really recommend to try this demo app on your own, I won't list our findings directly here in this blog post. If you'd like to compare your results with ours then check out our mind map of potential issues. In the end, we compared them with the provided list of 18 accessibility issues and found that we had identified a lot of them! At the same time, we learned about many more problems we had not been aware of at all before and solutions we even didn't know existed. Awesome!

Having still 15 minutes to go, we decided to follow our original plan and tackle a productive site as well. We selected agoda, a travel booking platform. Having only a very limited time box, we decided to instantly use Lighthouse and discovered the page's score was not much higher than the one of the inaccessible practice app. This way we found so many issues very quickly already. We also stumbled across other usability issues like changing navigation or not responsive design. Last but not least we tried to tab through the homepage which offered a navigation menu and a form to book a hotel room - and here's what happened. Just try it yourself!
  • For the first few tabs we saw where we were as navigation menu elements showed a blue highlighting frame.
  • The next few tabs suddenly did not show anymore any focus, so we got lost.
  • The next tab opened the calendar for the arrival date at the hotel - strange, as we had skipped the first form element to enter a destination.
  • We tabbed to the second calendar for the departure date, tabbed again and landed in a sort of drop-down / dialog / sub-form / whatever element to provide the number of rooms, adults and children.
  • We tabbed on. And found ourselves trapped in this very UI element! We could neither tab forward nor backward.
When reproducing the issue for this blog, I tried to see whether we really skipped the destination field, trying to tab backwards from the calendars - and noticed Shift+Tab did not allow me to tab backwards but had the same effect as Tab itself, both tabbing forward! I'm speechless.

Looking Back

As for each pairing session, we took some time in the end to reflect on what we learned and share our thoughts. First of all: The demo app was a really valuable target to explore, it raised awareness of the multiple traps to fall into when it comes to accessibility. Interestingly, we noticed we didn't even look at the accessible version of the page to see an example how to do it well! This is definitely still worth looking into. In any case, we both loved our approach of first practicing with the demo app, getting our brains prepared, and then switching to a productive application, being instantly able to spot issues there with the knowledge gained. We identified a lot within only 15 minutes - time worth spent.

It turned out again that bad accessibility design is often also bad user experience for everyone. Designing accessible sites improves the experience for everyone else as well. However, accessibility has to be considered already during design. Nowadays there are so many tools to support us, like checking for sufficient color contrast already in mockups.

Talking about our collaboration, it was once again a very easy session for both of us. We had agreed to use strong-style pairing and a mob timer to frequently switch roles. Alex also had made some experience with this approach already and felt comfortable with it. Our flow was very natural. Especially one fact caught our eye: we followed up on each other's ideas, using the "yes, and ..." rule from improvisational theater. Alex had already noticed that ignoring this is a common problem when pairing. In all the sessions I did on my testing tour, I found this to be one of the most important key points to pair successfully with strangers, being productive from the first minute on.

The session was thoughtful. It was super productive. It was so relevant for Alex' current project, where accessibility is a big part of the testing strategy. And the session was fun! Like several others before him, also Alex advised to "do it with someone else" as this makes it so much easier. "Don't test alone, for your own safety." I have nothing to add.

Saturday, June 2, 2018

Testing Tour Stop #11: Pair Exploring Test Ideas with Viki

The first time I met Viktorija Manevska was back at Agile Testing Days 2015, the very first conference I've ever attended. Before the first conference day started, I joined the lean coffee session hosted by Lisa Crispin and Janet Gregory. Both Viki and I found ourselves at Lisa's table, sitting next to each other. This is where we started to exchange experiences. Since then, we met each other each year at the same lean coffee table on the first conference day! :D Viki helped me a lot by sharing her story as first-time conference speaker and providing feedback on my own first steps towards public speaking. We had several Skype calls over the last year which proved invaluable regarding knowledge exchange, just like all my sessions with Toyer Mamoojee. I really love how I have met up with more and more other awesome testers over the last months! I signed up for a virtual coffee to get to know other testers like Milanka Matic, Amber Race or Kasia Morawska. I really enjoyed the calls I had with Mirjana Kolarov. And I loved meeting Anna Chicu finally for the first time! All those meetings were invaluable, full of knowledge sharing, story telling and having a good time together.
Now, back to Viki. She just recently moved to Germany to work for a consultancy. Having worked in product companies before, her current project provided new opportunities as well as posed new challenges for her. We decided to focus our pair testing session on one of those, on a part of testing that might not come to mind as first: generating and challenging test ideas as input for exploratory testing sessions.

A Session of a Different Kind

This time, there was no software under test - instead, we tested and challenged ideas. Test ideas that Viki had already brainstormed and provided as input for our testing. The result? We discovered more ideas. We restructured current ideas. We removed duplicates. We challenged procedure and more. All with the goal to improve and make it better, like we testers usually do with everything that gets into our hands.

While brainstorming and discussing, we talked a lot about testing itself. Have you used tours as a method to explore a product? I normally use cheat sheets heavily to generate further test ideas, identify further risks, come up with further questions about things we haven't talked about yet. Cheat sheets like the infamous test heuristics cheat sheet by Elisabeth Hendrickson, among other great resources. This way, wee started talking about exploratory testing itself. How structured should it be? Sometimes I feel my own sessions would benefit from a bit more structure, but I'm still happy with starting out to look for risks in the area of A and ending up with discovering a lot in section S. However, exploring is not unstructured and seldom goes without a clear mission or note taking. What I really love is Elisabeth Hendrickson's definition of exploratory testing, as stated in her most awesome book Explore It!.
Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next
In my opinion, too much structure or being too strict about it removes the part of adapting your way according to what you learned. Having the product show you the way, or better multiple ways, following them to explore areas of unknowns. At times I wonder whether Maaret Pyhäjärvi was thinking of that when saying that "the software as my external imagination speaks to me"; a notion I often pondered over.

Now, the system to brainstorm test ideas for was a typical untypical web application. Something known like signing up for a user account, something unknown due to the specific domain. We started with ideas around negative testing, UI testing, smoke testing, and workflow testing. Going forward, we produced a lot more ideas, asking so many questions about the product, and also about testing itself.
  • Would you consider different browser settings like turning JavaScript off as negative testing? It could be a normal use case for certain users.
  • What about high security settings or incognito modes? Some users just care more about privacy than others, still they don't mean any harm to your application.
  • What about popup blockers or ad blockers? They are quite frequently used.
  • How to best limit the input for a name field? This made me think of Gojko Adzic's recent book Humans vs Computers as well as his famous Chrome extension Bug Magnet, a really useful tool to quickly test out valid or invalid input values.
  • Or the Big List of Naughty Strings! I've known it for a long time but haven't used that one much yet. There's definitely more to it.
  • This brought up the topic of SQL injection, a common vulnerability to check for. In general, user data must not be at risk, so let's see if we can get login information, take over user sessions, or simply can view other user's unsecured pages.
  • What about tampering with system settings like your local time, could you trick the application to provide you better offers that are based on dates?
  • You send emails? Really important to test those properly then. In one of my previous teams, formatted emails were displayed very differently using different email clients. Are the parameters of the email template filled with the correct values depending on given conditions? Are potential images attached that need to be downloaded? Are the links correct? Could the email be considered as junk and filtered out? How long does it take to receive the email?
  • What happens if you bookmark the URL while being in the middle of a process to share it with someone else?
  • What about the browser back and also forward buttons, could they lead to unexpected, inconsistent, or simply unpleasant behavior?
  • What about consistency regarding styling? Is the UI accessible, e.g. for color-blind people?
  • Which kind of browsers, browser versions, operating systems/devices, screen resolutions do we really want to support?
  • Would you have manual smoke tests? Why not automate those sanity checks instead of repeating them over and over again?
  • What about alternative paths through the application, mixing up the order in which you can do things or skipping steps - would you still reach your goal as a user? Should you be able to do so or is there a prerequisite for critical things?
  • Have you ever tried the nightmare headline game to discover what would be the worst to go wrong?
  • Oh and: although we received so many emails about it lately - have you really considered GDPR compliance?

Looking Back

Both of us loved the session! Viki said she was really satisfied with the outcome. It was great to hear a different opinion, especially from another fresh tester point of view coming from the outside. Sharing experiences, experiments and stories provided lots of value for both of us. I was really glad my input helped her further, especially regarding testing emails, SQL injection and GDPR. I personally really enjoyed to have a completely different pair testing session, exploring mind maps, challenging ideas and thoughts, discussing structures, procedures, and strategies. Exchanging information about different positions, roles, expectations, and ways of collaboration - which is so important as well when testing. Talking about exploratory testing, what we understand, what we do, how we give feedback and create transparency, how we provide value. Just as Viki said, quoted freely: "Name it like you want, but it's important to deliver value." I so much agree.

Friday, May 11, 2018

Testing Tour Stop #8: Pair Accessibility Testing with Viv

On today's stop on my testing tour, I had the pleasure of pair testing with Viv Richards. I got to know him via SwanseaCon. He was the first one to accept me as newbie speaker last year and gave me the opportunity to speak again at this fabulous conference this year! I'm really glad it worked out to have him as part of my tour. It was a fun session full of insights.


Accessibility - The Neglected Child

Viv left it to me to choose a topic for our pairing session. He said he would be happy to explore any area I prefer or am comfortable with as sees himself as "jack of all trades master of none". I so relate to that! Well, I decided to go for accessibility testing this time. Why? In my opinion this is a very important topic, often overlooked or postponed. I have never had the opportunity to actively work on a product where this was a requirement, or even considered in any way. I read some things about it, but really lack practical experience. To add to that, I knew Viv had experience in this area. Back when he was still in a developer role, he worked on a product where accessibility was a big topic.

To prepare for our session, I researched some pages which would help us kick it off. As shared in the post about my last stop, I don't like to limit the scope of our sessions too much. I prefer to keep enough freedom for us to explore in any direction it might lead us; the main goal is learning. Still, I'd like to have some options prepared upfront. Here's what I found.

Hands-on Testing

For our session, I decided to not go for one of the demo pages, but rather try a productive application and see how accessibility looks like in the real world. I chose the web version of the todo application Remember The Milk. I'm not using it myself, but tried it out years ago when searching for a task management solution fitting my needs.

We started the session by imagining we had no mouse available but can only use the keyboard to navigate the application. We could successfully sign up for a new account this way, but then quickly faced problems. It was not obvious at all how fields are ordered and we most often missed visual feedback where the current focus was. Viv shared that a screen reader tool would have problems with that. But even only by just not using a mouse we failed to navigate to certain fields, like setting additional options when creating a new task. As we stumbled heavily from the beginning on, we decided to switch to simulating a different kind of user experience.

What if we were just shortsighted and didn't have an optical aid at hand? We set the browser zoom to 200%. The page looked not as nice anymore, but was still fully functional. We could reach all page areas and elements. Same when reducing the zoom to a lower value than 100%.

But what if we only had one hand available (maybe carrying a child in our arms), and that might not be our usual one? I'm right-handed, so I tried to use the mouse with my left hand while using the application. Though this was slower it worked out well. Interestingly, during this time we came across functional application behavior which we would not have expected.
  • We just wanted to add a reminder to one of our todos, but doing so the application took us to the settings page. We should first define a device to get reminded on. Hm. Okay, we chose the computer. And the app instantly opted us in for all kind of notifications. I don't like it when they sign me up for everything by default, it just leaves me with a bad feeling.
  • The settings dialog showed a save button - but inactive. Why? We found it was only meant to save any changes made on the kind of notifications we'd like to receive. Not obvious, not nice.
  • Going further, we failed to define a reminder for a specific time; only days or weeks before our due date were available. For me this would be an important feature for a task management tool. But okay.
  • Then we discovered that subtasks can be sorted by drag and drop. There was a configuration menu, but it only offered one option, drag and drop sorting, and it could not be unchecked. Really strange! Only later I found that the related help text explained that subtasks can only be sorted the same way as the original task list they belong to.
Well, the drag and drop functionality triggered the next idea. What if we could not use JavaScript? Viv shared that in his experience this was a valid case. They had to first develop without using JavaScript at all, which meant they needed a lot simpler UI. To simulate this in our case, we opened the Chrome Dev Tools and and disabled JavaScript in the settings. We learned that you need to keep the Dev Tools open to make this work. After refreshing the application page, we found it could not be loaded at all. However, it also did not provide any further feedback why. At least a notification that JavaScript needs to be enabled would be needed to not get people lost.

We decided to start using tools in general to get an overview of existing accessibility problems. Viv recommended the Chrome extension WAVE Evaluation Tool. It really presented a nice overview of the current page's accessibility issues as well as explanations why these points are considered problematic. This way we found issues like missing labels for input fields, missing alternative descriptions for images, structural issues like having h2 headers but no h1 header to start from. We found that ARIA roles, states, properties and labels were provided as expected. To my surprise, the tool also pointed out that an unordered list was used! When researching later on, I learned that incorrectly defined unordered lists are easily comprehensible as a formatting element for sighted users, but present a problem if you have to rely on a screen reader that interprets them as single paragraphs without providing an outline. WAVE also offered the option to see the page without any styles as well as to test for sufficient contrast ratio between foreground and background colors. In the case of Remember The Milk some elements did not provide sufficient contrast to fulfill the AA or AAA levels defined in WCAG 2.0.

Evaluating the contrast triggered us to consider color-blindness. There are Chrome extensions to simulate this as well. We tried Spectrum and I want to see like the colour blind, both offering different display modes for the page. We realized we didn't know the technical terms to describe different experiences when it came to colors!
What helped me to get a quick overview on these different types was the color blindness table of the color-blind npm package. All in all, Remember The Milk did quite well, only with low contrast we deemed it hard to use.

Another idea came to mind: what if we saw everything but have a hard time to digest the information? Like if we struggled with dyslexia? Chrome extensions to the rescue! Also for this case we found simulators. We tried dyslexia simulator first but couldn't get it working on our application page, also when adapting its settings. We were not sure if maybe our brains sorted everything out automatically, so we tried another Dyslexia Simulator - and instantly got closer to understanding what it means to have dyslexia and view a web page! This simulator scrambled all texts constantly, we had a hard time to focus. It took us way longer time to recognize the words, and we couldn't watch it for long.

So what about screen readers? Viv recommended the free NVDA (NonVisual Desktop Access) so went for it. I was surprised by the audio feedback we received while the application was installed and set up! Of course this only makes sense. It just showed me again how much I do not know about different kind of technology experiences. Also, I instinctively used the mouse first, and the screen reader instantly commented on everything I hovered over - until Viv told me blind people would use the keyboard not a mouse. So we tried it on Remember The Milk. The speech output was very fast and I had a hard time understanding but at least could grasp some parts. Especially I understood that the output did not provide helpful information. For example, after adding a new task I heard that "the input field is empty." So what, how should that help me? Why not providing the information that I could instantly create yet another task? Well, here it showed in practice what WAVE pointed out - the input field did have any contextual label.

As final part of our session, we went half-way through a list of tip when testing for accessibility that Viv had found. We noticed that some of the listed points were considered in our application, like not labeling links with "click here", and other points had been disregarded, like the missing h1 tag. Font size was another remark triggering the idea that although we did try to increase the browser zoom, we haven't tried to increase the font size in general on the operating system. When trying to do so, we had a hard time finding the related setting in Windows 10! Seems there is only the option to scale everything at once, text, apps, and other items. I cannot tell whether this might be a good way to handle it or not.

After our session, Viv provided me his notes and thoughts. Here's what I did not mention already.
  • W3C Accessibility
  • JAWS (Job Access With Speech) - a very good screen reader
  • Another idea: Does the page have regions defined to enable a user to quickly jump to sections of the page using a screen reader? (This would have been something with more time to test in the screen reader)

What worked well, what to improve?

In the very beginning of our session we struggled with the technical setup. No matter how often I had video calls with many different persons, this just happens and I have to remember to take it into account. At first the computer wanted to install updates. Then the call could start but the microphone was not recognized. Finally this could be solved but screen sharing failed to actually show the screen. In the end it worked and continued to work until the end of our session including sharing control.

As with several previous pairing partners, we chose to pair the strong-style way with one being the navigator and one the driver, switching roles every four minutes. Although we didn't always switch in time, this worked out pretty well. Conversation flow and collaboration was once again very smooth.

The session proved very valuable for both of us. To further improve it, Viv came up with the idea to maybe rather focus on a small part of the application. Accessibility is a huge area in itself. Spending more time on a smaller feature might have helped us. Something to keep in mind for future sessions!

Viv sees accessibility as a topic similar to security. Many times we are facing lots of technical debt in these areas. We should be mindful about it from the beginning when starting a new application. I totally agree with him. When starting a product from scratch accessibility is often neglected, and then you end up with a legacy system where it's hard to build it in afterwards.

For both of us it was interesting to pair with another tester remotely and see other people's approaches. At work, we both rather pair with developers which is really valuable but has a different outcome. Viv also pairs with other testers but rather to instruct and provide support. Not being on the same level causes different dynamics. He shared that the strong-style way of pairing would help a lot, with the back and forth you really have to contribute. Often people don't see which kind of value they can provide, like when pairing with developers to write unit tests. However, there's always something to be shared, always something to offer. Wise words.

Something to Keep in Mind

If you take one thing of this post with you, then let it be this. We all experience the world around as and technology in a different way. Most people encounter the one or the other barrier when doing so. This doesn't have to be permanent, it can also be temporary or situational. So let's keep accessibility in mind to develop valuable products.