Showing posts with label swanseacon. Show all posts
Showing posts with label swanseacon. Show all posts

Tuesday, October 23, 2018

Testing Tour Stop #24: Pair Exploring Voice Experiences with Gem

At this stop of my testing tour I had the honor to pair test with Gem Hill. I got to know her as active podcaster and enjoyed listening to several of the already 100 episodes of Let's Talk About Tests Baby. This year, I had the pleasure to meet Gem in person for the first time at SwanseaCon. We had attended each other's talks, and right after she heard me speaking about my testing tour she scheduled a session with me!
Gem shared with me that she also experienced a pair testing session with Maaret Pyhäjärvi, just like I did on my very first tour stop. She even did a full podcast episode about it, which I can only highly recommend! Since I've met Gem, I was looking forward to our pair testing session. With good reason, I learned a bunch.

Test Setup

Gem proposed to pair test on voice experiences. As we paired remotely we agreed to use a simulator instead of a real device for testing. My experiences with voice apps are very limited, they narrow down to a simple learning project during one of my company's hackrdays last year. Also, I am not a user of voice apps myself. Therefore, I was eagerly looking forward to our session. Some interesting blog posts I came across during preparation were the following.
Gem kindly agreed to prepare a test setup. As she is currently working on voice apps at the BBC, she suggested to tackle the BBC Alexa skill which is basically a player for all BBC radio stations and podcasts. We used the Alexa Simulator and the latest released skill version, publicly available for everyone.

Challenges of Testing Voice Applications

At first, we checked the happy path to start the app and play a radio channel or podcast. Just by doing so, we discovered that the audio player functionality showed issues. Instead of playing the requested source, it triggered a warning that "AudioPlayer is currently an unsupported namespace". Interestingly, the warning message box was not completely displayed on our screen. Well, we're not testing Amazon's services here. Checking later, I found that this is a known limitation of the simulator and audio playback would work on a real device.

Then we tried to change from one radio station to another one and stumbled again. Asking to "switch" to another station opened another skill, TuneIn, attempting to play the requested station there. Unexpected and not desired! We thought maybe native speakers would not say "switch", but rather "change"; however, this command was unknown. What about "go to"? Again, we switched skills. Interestingly, the feature to switch stations is indeed advertised in the skill description.

Let's try to ask for the news, a use case we deemed quite common when thinking about radio stations. To our surprise, the player started the podcast "Breaking the news". Seems a synonym had been stored for "the news". Hm, what about only "news"? The skill told us that this radio station was unknown. We tried "weather" instead, another common information you might expect to get from a radio station. To our surprise, the skill answered with "I can't do that". Strange, we thought if the request was unknown it would fall back to the "unknown radio station" case. We tried "cheese", which again was interpreted as unknown radio station, just as expected. To test the simulator a bit, we tried a clearly misspelled input, "ra7dio 0n3", and this got recognized as unknown podcast now. But why? We tried several cases, using written words as well as voice input, however, we could not determine why the skill reacted in three different ways to unknown input values.

We looked at the JSON input and output the simulator showed us. Everything looked good here. So we moved on and tried different languages, like asking for the Welsh radio station "nan Gàidheal". The skill understood "nine gale" and asked "Should I play nine gale?" Yet another response we hadn't triggered before! We confirmed, and the skill answered that it couldn't find the station and included a long error message within the response. Interesting finding! However, it was again caused by the simulator and not a real-life example. We used written input, and the API could not deal with the special character included, which would simply not happen when using voice input.

We tried a few more things like seeing whether the station "WM 95.6" got recognized when pronouncing the "." as "point" as well as "dot", or only using "WM". All worked. We tried similarly pronounced words and unclear mumbling to see whether one could be mapped to a station. There were many more options we did not try, like checking any customer reviews on the skill, which in themselves provided lots of useful input for testing.

Throughout our session, Gem shared bits and pieces of wisdom when it comes to testing voice apps.
  • The simulator and real devices behave very differently, so normally they always test on real devices. Furthermore, the simulator is way slower than real devices; another reason to use the latter.
  • They do lots of API testing, checking the JSON output, to ensure the implemented functionality is working as expected.
  • They learned how to test efficiently. Why? Well, when you explore without automation you have to listen to the skill welcome and introduction text again and again and again and again... This does not only take a lot of time, it is also very quickly annoying. Very. Quickly. So, exploratory testing without automation can indeed by very inefficient in this case so they try to automate as much as possible on API level.
  • As the team got more and more mature, they now started to think more and more about how to increase testability. With all automation in place, you still have to test with the real devices as it's the only way to get the real experience.
  • It is really hard to design voice experiences. For example, you have no idea whether you have a very experienced user who might get annoyed by having to listen to too much navigation, or a newbie, who simply needs more instructions.
  • Always ask yourself: Are you still testing the skill, or already the device? Testing the device does not make sense, we can't change it anyways. Going through the list of things users actually say and what Alexa understands, however, does make sense as it allows you to add those as synonyms and make the skill more user-friendly.

Retro Time

We quickly found ourselves at the end of our 90 minutes time box. Gem felt the session went very well, using Zoom to share screen control was amazing, she had lots of fun and the time flew by. For another session she would spend more time in the children's skill BBC offered as it had more branching narratives.

For me the session was awesome, Gem provided a perfect test setup for us to practice. I agreed that a more complex skill would be interesting, however, it was also nice to start with such a "basic" skill as the radio player. This challenged us to come up with good test ideas and we still found issues for such a basic skill. I really appreciated Gem for sharing her knowledge with me, I learned a lot in our session and my respect for people designing and testing voice experiences grew even further. Gem shared it's a real challenge and they are learning something new every day. By the way, a fun fact I learned today: both of us are normally listening to audiobooks when at home! It was a real pleasure testing with Gem and I'm already looking forward to seeing her again in real life.

End of October, the end of my experiment, is getting closer. This was the second last stop on my testing tour 2018. One more to go, so stay tuned!
Posted by at No comments:
Labels: , , , , , , , , ,

Wednesday, September 12, 2018

SwanseaCon 2018 - Celebrating One Year of Public Speaking

It's this time of the year again, the SwanseaCon time. What can I say?
This conference is dear to my heart. I am most probably biased here, so I'd like to invite you to come and judge for yourself! I'm biased as this was the first conference that accepted me as speaker. Last year, on 26th September 2017 I gave my very first session on a conference stage. Coming back this year, again as a speaker, feels like closing a circle. Even though it's already more than a year of public speaking, and only nearly a full year of conference speaking, I consider my return to SwanseaCon as a personal anniversary. Biases aside, I overheard several others that they enjoyed the event just as much.

This year the conference took place on one day offering four tracks instead of two days with two tracks. Shorter with more options to chose from can be both better and worse, depending on your personal preferences.
The good thing of a one-day experience is that it's nice and condensed, and afterwards you can celebrate all the learning with all the great people. As a speaker it's nice to relax, too, you simply can't be scheduled for the next day! Still, more tracks mean more options and increased fear of missing out. It was indeed hard to chose the most favorite talk of this year's awesome SwanseaCon program. I love the fact that it was really diverse regarding content and speakers again, just like last year. The audience was quite diverse, too! A nice mixture of local and international community, of many different roles. Lots of women around as well. My personal impression, was a good one and I perceived the atmosphere as very friendly and open.

Now what about the content? The sessions I've joined were very insightful, I learned from each of them.
  • It started with the opening keynote of Hadi Hariri: "Where's My Free Lunch?". He vividly made clear how people in today's society pay for free apps with their privacy and time, becoming more and more indifferent; just because it's so convenient. "Personalized experience" a.k.a. manipulation. It's also convenient to not contrast any presented "fact", validate it, and only then share it; we have our part in producing and spreading fake data. Hadi's conclusion: We're screwed. So what we can do? Raise awareness and think before we act.
  • Next up: Gem Hill with "Anxiety Under Test". Gem talked very openly about her mental health issues and how she learned to live with them. Companies like BBC discovered the benefits of mental health first aid programs, offering trained contact persons for people in need. There are many more places we can reach out to, Gem shared a list of resources in her slide notes. Key is to learn our individual signs of upcoming issues, and find out what helps us mitigate them. Self-care is essential in these cases. What a very hard topic to talk about - and so incredibly important.
  • An interactive workshop! That's a first-timer for SwanseaCon as far as I know, and it was great. Lisa Crispin and Kristine Corbus took us on an experiment of "Questioning requirements: improving quality for everyone". The small room was crammed with interested people! Time flew while all groups tried different approaches to come up with a first release plan for a new product, given only a few ambiguous requirements. It was great fun and we learned a lot!
  • After lunch, it was time for my own talk, "Cross-team pair testing: lessons of a testing traveler", sharing lessons learned on my testing tour.
    The goal of this talk is to inspire people to go on their own learning journeys by giving an example. An idea to take, adapt to their context and make it their own. And it worked, just like the last times I've given the talk. Right after the Agile Testing @Munich meetup Guna Petrova signed up for my tour. After CAST Alex de los Reyes scheduled a pair testing session with me, and now it was Gem Hill! Also, further people came and discussed how they could start their own experiment. This makes me unbelievably happy. Moments like these made my journey out of my comfort zone by public speaking and the testing tour so much worth it.
  • This time I was eager to give my best to not fall into my usual post-speaking state of total distraction and lack of focus but instead closely listen to Nick Tune's talk "Strategic Autonomous Design: Patterns and Heuristics". I knew Nick as great story teller, he could really engage and entertain me last year. Now, following the talk worked out well at first. But then: technical problems. The projector refused to show the slide deck. One slide was displayed, the next one not. They had to reactivate sharing, it worked for one slide, and so on. In the end they had to use another computer. Nick got constantly interrupted and it cost a lot of time overall so that he could not present the whole talk - what a pity.
  • Erkki Salminen and Mari Wuoti's talk "Culture for the win" was my last choice of the day. They shared how the culture at their company Gofore evolved. They values they have are used as guideline for everyone to decide to the best of their and the company's needs. The nearly 500 people only have one "middle manager" - a Slack bot! Culture is strategy for them, it's a lot about the mindset. Great example that you can indeed grow your own culture and related structures; maybe inspired by others but not copied and pasted into your context.
  • The closing keynote was given by Seb Rose: "Whatever your job title, you’re actually being paid to think!". Wonderful talk around his rant for the FFS Tech Conference, sharing why metaphors, acronyms, templates and models can be confusing, ambiguous and generally unhelpful - if you don't think. That's our job, we're creative thinking professionals! Seb asked everyone to not just belief him, but really think about it first and make up our own minds. Wonderful closing keynote that was fitting so well to the opening keynote.
There were several more talks I would have loved to see. The good thing: There was again a post-conference meetup hosted by Equal Experts - with great lightning talks!
A great conference ended with even more great and fun talks and long conversations with awesome people. Just perfect.
Posted by at No comments:
Labels: , , , , , , , , , ,

Saturday, September 30, 2017

SwanseaCon and My First Conference Talk Ever

Let me share some of my adventures at and around SwanseaCon 2017 where I had the honor to give my first conference talk ever.

Before SwanseaCon, or: Off to Swansea!

February this year I was invited by Viv Richards to submit papers for SwanseaCon. SwanseaCon? To be honest, I've never heard of it before. During my research I found that it's an "Agile Development and Software Craftsmanship" conference in South Wales. I was honored to be asked to submit to a developer conference! Judging from the past years, it sounded really interesting - so I did submit. Only a few weeks later, Viv came back to me with the great news that I've been selected! That was the beginning of a great adventure to the north of Europe.

When planning my trip from Munich to Swansea, I decided to fly to Cardiff directly, which would mean to travel a few days earlier on Friday evening. This decision proved really valuable! First of all, I was excited enough already about the conference and my upcoming first talk without the
additional time pressure.
After arriving in Wales, the way from Cardiff airport to Swansea was not completely obvious for me, despite previous research on the internet. However, it was made easy for me as I met some great people on my way, traveling to Swansea together. The next day, I enjoyed a lovely sunny Saturday to discover not only Swansea's city center, but also the nice district Mumbles with Oystermouth Castle and some great ice cream at Joey's. Really recommended!
On Sunday my plan was to practice my talk again and maybe do some final adjustments, but well, the nature of plans is that things can easily go differently than we planned. In my case, I discovered that the travel adapter I brought with me did flawlessly charge my mobile phone, but didn't fit to my laptop cable! My bad. I had prepared well to avoid such incidents, but this one I had not tested out before. Lesson learned for the next travel to a country with a different socket type. So I asked at the hotel if they have any adapter available - and yay! They could provide one that fitted my cable. But: My laptop did not charge. :( So instead of practicing my talk or relaxing, I ran around Swansea to find any other travel adapter which might possibly fit my laptop cable. Lucky me - the fourth adapter finally fit AND charged my laptop. Phew! Well, I knew that Viv would have gladly supported me, but I was eager to try solve my self-induced problem myself before bothering him as I knew he had a lot to prepare for the conference.

On Sunday evening I had the chance to meet up with some early arrivals for the conference. That meant Viv himself (a pleasure to finally meet him!), Richard Bradshaw (whom I met in his tutorial at Agile Testing Days 2016 already), Carly Dyson (whom I only briefly knew from Twitter and had spoken at Nordic Testing Days) and Georgina McFadyen (whom I had not yet met and who was about to do her second conference talk at SwanseaCon). A lovely group to start with! As an introvert I was happy to get to know some friendly faces upfront to the conference, which instantly made me feel so much more relaxed. Ready for the conference to start!


During SwanseaCon

Conference Day 1 - Diving In

The conference took place at Liberty Stadium in Swansea. A nice venue! We were greeted with tea, coffee and cookies. The two session rooms were quite large and showed a friendly cabaret setup, having people group around round tables. All the sessions I attended were really great, each session in its own way. However, in the following I will only list my personal highlights of each day.
  • Let the conference begin! After a warm welcome and some introductory words by Viv, Sander Hoogendoorn kicked it off with his opening keynote "It's a small world after all". A great talk about how quickly time is changing and how everything is becoming smaller and faster. I especially loved the idea of "continuous culture".
  • In her talk "Don't Diss the Discipline", Georgina McFadyen shared her stories about how her company changed from waterfall to agile, and from being professional software developers who did their job, to software craftsmen who really cared about the product they were creating, about its quality and its value to the ones using it. Awesome experience report, I really loved her stories and presentation!
  • Seb Rose of Cucumber shared "10 things you need to know about BDD, Cucumber and SpecFlow". There were several misconceptions to be addressed and clarified! Though not everything was new to me, he did a well-rounded talk covering many essential topics around BDD and the core concepts or benefits of it. Great job!
The first day ended with some snacks and drinks, a nice socializing and networking opportunity to share and discuss our thoughts about what we heard with other people. Right afterwards several attendees decided to join a special meetup in town, where several of the conference speakers or attendees had the chance to give a lightning talk about any topic; prepared or unprepared. It was simply awesome! Many valuable insights given in short time. Nice way to close a conference day.

Conference Day 2 - My First Talk and Other Highlights

My talk was scheduled first in the morning, so I had the task to kick off the second day. In the beginning, not so many people had entered the room yet, but they grew more and more over time. And they were a lovely audience! In retrospect, when reviewing the received feedback and reflecting on what could be improved, I have to admit that it went really well, especially for my first conference talk ever. The dress rehearsals at my company and at local meetups really paid off (also regarding my level of nervousness). I forgot to mention some points (as always) and noticed some things I should work on (like me posture). But overall I cannot complain, it was a great experience! By the way, my slides are now publicly available.
The rest of the morning I attended further great sessions, but found myself not being able to focus on them as they would have deserved it. I got distracted by thoughts about this scary thing I've just done - public speaking, at a conference. I still catch myself wondering if this story really became true after all.

After a while, my focus was back; just in time to enjoy the following most awesome talks.
  • We had the honor to listen to professor Dave Snowden from Bangor University, the creator of the Cynefin framework. He gave his very insightful talk "Making Agile safe again: resisting the Borg" without any slides - just once using a whiteboard. Really impressive!
  • Afterwards, I heavily enjoyed columnist and author Kevlin Henney's talk about "Turning Development Outside–In". Such a great and entertaining presenter! He took us on a lively history of how development processes evolved and how we have to look closely for whom we are doing all the things we do and why.
  • Another talk I really enjoyed was "The Domain-Driven Coding Architect" by Nick Tune. Great storytelling of his lessons learned while finding out how to properly (!) read the DDD book written by Eric Evans! :D
  • The conference was concluded by Sallyann Freudenberg and her talk "The trouble with culture", talking about neurodiversity and how teams benefit from it. What an awesome closing keynote about a really important topic everyone should learn about! And definitely lots of food for thought.

After SwanseaCon Is Before SwanseaCon?

After the second day, most people went home directly. I took this chance to review the #SwanseaCon twitter stream, see what people shared about the talks I missed. So many great sessions! Even for a two-track conference it was really hard to choose between all those great speakers and topics. A pity that not all sessions had been recorded, but sometimes it's also the charm of a conference that you had to be there to experience them. Only there you can feel the whole atmosphere - and especially connect with people. You can't do that when watching a recording alone at home. I did connect with several people at this conference and am already curious what they will be up to next.

The trouble or sad news: This might have been the last SwanseaCon. Believe it or not, Viv does all that work on his own. The organization was awesome and his support outstanding, in any way. He engaged family and friends to help and everything went so smoothly and professionally. Still: There's lots of money included when you organize a community conference by your own. A conference which supports speakers with travel and accommodation costs, but still comes with very affordable prices for attendees. If no one buys a ticket - you basically lost a lot of money. As attendance on this third edition of SwanseaCon was worse than last years, and there's so much work and blood and sweat and money in it - he understandably considers not repeating it. Especially as this man volunteers with so many other great things for the local community already, while working a full-time job and having a family with soon to be five kids! All my respect goes to him.

What I loved about this special conference was the speaker and topic selection itself. Because in the end, it's all about the people. I you have a chance to come to Swansea, I really recommend this event. Would love to return to this wonderful conference where so many great people met to learn together.
"Having attended a fair few technical conferences, it felt refreshing and important to be part of something a little softer. Software development is about community, communication and culture, and SwanseaCon scored top marks on all three." - Thomas Guest, SwanseaCon 2017
Posted by at 2 comments:
Labels: , , , , ,
Subscribe to: Posts (Atom)