Saturday, July 21, 2018

Testing Tour Stop #16: Pair Exploring an API with Thomas

Yet another stop on my testing tour? Yet another stop on my testing tour! Yesterday I learned together with Thomas Rinke. We shortly met each other at Agile Testing Days 2017 where we had a great conversation over lunch. Like many of the other people I paired with, Thomas is really engaged in the community as well, something I really appreciate. He is on the conference board for the German Testing Day, and also actively helping his colleagues grow. He already did what I wanted to do with my own company's testing community for a long time: watching great conference talks and discussing them together. Still on my "to try" list!

The Preparation Phase

About a week before our session, we started brainstorming and aligning ourselves on our topic. Thomas was up for exploration as well as automation. I proposed to go for an exploratory testing session, pairing the strong-style way. Thomas shared he never experienced it himself but was intrigued to give it a try. He had come across Alan Richardson announcing Thingifier, his new demo app for API testing. Awesome! I wanted to have a look at that myself already, so we chose it as our system under test. We directly went for version 1.1, however, as the previous one did not start up successfully on our computers.

In case you haven't explored an API yet, I recommend Maaret's fantastic article Exploratory Testing an API. For our session we agreed to use Postman as our tool of choice and to run the app on my computer, sharing screen control. Have I mentioned already how much I love Zoom for not only offering great video quality and recording functionality, but also free control sharing that's easy to use and actually works?

The Session Introduction

We started out setting everything up and getting any open questions out of the way. Interestingly, Thomas shared with me that he was nervous and excited, thinking "well, what could go wrong, it's going to be great". Especially testing with "strangers" was something extraordinary, and we didn't know each other so well yet. The thing was: I was nervous, too. I always am for my pair testing sessions. Fear is even one of my reasons that I started the whole testing tour. Doing this is still scary for me. However, I already see that it is continuously getting better over time. "If it's scary, do it more often" really applies for me here.

Thomas also shared he prepared for the session upfront. For example, he read Llewellyn Falco's blog post about strong-style pairing and listened to Woody Zuill's podcast episode about mob programming on Agile Uprising. Both really recommended!

I learned Thomas was running Windows as operating system, so I advised him to rather not use shortcuts as they are mapped differently on macOS and this doesn't work easily when sharing control, as I had learned in my session with Pranav. Due to the preparation done beforehand, everything could be quickly clarified so we could activate the mob timer and start testing together.

The Testing Part

Thingifier is described as a "hard coded model of a small TODO application". Our first goal was to get an overview of the capabilities of the application and understand its model. We started out using the given documentation and sent first requests to see how the application worked and behaved. Interestingly, later on I saw that the app's release note for version 1.1 said "When the app is running, visit http://localhost:4567/documentation in a browser to see the docs." This was in contradiction to the actual documentation that could be found directly on "http://localhost:4567/".

Thomas started out as navigator, but when it came to him to take over keyboard control as driver, we found that Postman and Zoom screen control sharing did not work well together. He could do right-click actions with his mouse, but not left-click. We discovered that it worked in Chrome, however, and when going back to Postman it now worked here as well. Would be an interesting topic to explore itself!

When getting to know Thingifier's Rest API, we came across the following findings and starting points for conversations.
  • The documentation spoke of todos and tasks. On first scanning over the documentation, it looked like these terms had been used for the same thing, so it seemed like an inconsistency. However, on second read it became clear that you had standalone todos on the one hand, and projects that have todos as so called "tasks" assigned on the other. Nonetheless, this naming differentiation caused us to stumble. Later I found that when using a category guid as resource instead of a todo guid, it's successfully accepted as task, but then calling the task collection returned a 400 (Bad Request).
  • When adding a task to a project, the request returned a 201 (Created). Repeating the same request did not add the task a second time which was good, but the request still returned 201 (Created). We considered that not critical but also not completely clean, we would have rather expected a message that the relation already existed.
  • We stumbled a bit regarding the naming of resources. For collections the singular form was used (e.g. "/todo"), for sub-collections of an element the plural form (e.g. "/todo/:guid/categories"). We wondered whether there was a related naming convention for Rest APIs. Intuitively we would have used the plural form for all kind of collections, as the request was returning an array of all items. For example "/todo" returned an array of "todos", so we would rather have named it "/todos". Further research showed that it indeed seems to be best practice to use the plural form for collections.
  • For testing purposes we would have loved to see existing relations in the request responses, e.g. when querying a project to see its assigned tasks and categories.
  • Creating a todo created it with a random guid, and it was marked as not completed by default; just as expected.
  • When checking all given relationships between entities, we found them to be inconsistent. Categories and todos were implemented in both directions, so you could see all todos of a category as well as all categories of a todo. However, for example, you could see all projects of a category, but not all categories of a project.
  • We had already tried three of the four offered relationships, and thought everything seemed okay. To complete our understanding we tried the fourth as well - and it failed. "/todo/:guid/categories" resulted in a 400 (Bad Request). Fun fact number one: It's so often the last thing we try that unveils an issue. Fun fact number two: I could not reproduce this issue anymore when writing this blog post; it seems we missed something here.
  • When querying a collection, we would have liked to directly see in the response how many elements were returned, at least to help testing.
  • When requesting a resource that did not exist, we received an error message as expected. Its wording was quite technical but expressive.
  • We deleted all todo items and then queried for the collection of all todos. The request returned only an empty response without value ( i.e. {} ). We would have preferred to receive an empty array of the queried entity instead to keep the response expressive (i.e. {"todos": []} ).
After we had gotten more familiar with the application, we decided that it was time to go deeper on a certain part of the API. We agreed to have a more detailed look at creating todos as we perceived it as core part while still being small and graspable.
  • Sending a request with an empty body returned a 400 (Bad Request), as expected.
  • When providing the framing brackets {} as body we received the error message that the field "title" was mandatory.
  • When providing a body with the title field set to an empty string, we received the error message that the title must not be empty. Providing a blank resulted in the same message.
  • Providing a dot (.) as title worked, just as the Polish special character ł. Previously I learned those additional Polish characters are not included in the basic Latin character set so they can can cause problems e.g. when storing values in a database.
  • Providing a double quote (") returned a MalformedJsonException. We tried to escape the character, using a backslash (\), but it returned "Expected ',' instead of '"'". We wondered how to escape a character in a REST API request body but left that for future investigation.
  • We provided a quite long title of more than 256 characters. The request returned a 201 (Created) as expected. However, when then querying for all todos, the request only returned "Expected ',' instead of '"'". We found that this was not related to the long value provided, but due to our previous tries to escape characters. We indeed had to restart the app to reset the given test data.
  • We switched to the guid field. We found that if the guid is explicitly specified on todo creation, the given one is taken. In case the guid already existed, the todo is not created again but the existing todo is updated. We discovered that there seemed to be no validation regarding the length of the guid. Later I found you could even provide an empty string as guid which made it impossible to request the specific todo.
  • According to documentation, both POST and PUT methods were offered. However, their purpose felt mixed up. Both should create todos in case they did not exist yet, and both should update an already existing todo. However, I found that my gut feeling is mixed up here as well, as I thought in terms of HTTP, however REST only demands a uniform interface.

The Retrospective

As for all test sessions, we left some time for a short retrospective at the end to reflect how it went and what could be improved. Thomas started with sharing his thoughts, and by doing so provided wonderful feedback! Here's what I understood.
  • It was a lot of fun and overall a great thing.
  • He had seen on Twitter that I invited basically the whole world to pair test with me and thought that was a really cool thing. He told himself he could not lose anything there and wanted to join in.
  • The communication beforehand was great. He really liked that I reached out in time upfront, about one week before our session. That we clarified the technical setup, aligned on the topic to pair on, agreed on the style of pairing. That both of us could contribute when choosing our topic and how we wanted to collaborate.
  • He learned something during the session which was great. He had the feeling that it was also worth it for me, not that I thought "oh no, Postman, exploring an API, how boring".
  • He found that I was open, friendly, and welcoming. He really liked that I was looking forward to our session myself.
  • The session offered lots of first-timers for him. It was his first time of using Postman, first time of having a REST API, first time of trying the strong style way of pairing, first time exploring an API. He considered these quite a lot of firsts and therefore felt triggered to prepare for the session which he thought was good. He felt a certain sense of commitment and bindingness to do so. He wanted to explore an API for so long already, and our session triggered him to really do it now.
  • Thomas said the execution of the session and our alignment were good as well. Strong-style pairing was really great, too, he had a good feeling about it. He shared that he felt the trust given and had trust in me himself. He referred to Llewellyn's blog post which emphasized the importance of trust as well.
  • What was not completely perfect from his point of view was that we worked on different operating systems. Thomas felt he was slowed down as he always had to use the GUI for operations like copy and paste. This was not optimal as he felt that I was so much faster when I was driving. He said that the upfront warning about the mismatched shortcuts was great, and in the end this was okay for him, but he still felt I was always faster.
  • Thomas valued the results of our session. We found some behavior working as expected, some surprising things, and some classics.
First of all I had to thank him for his great, structured, constructive feedback. It is so helpful to receive detailed feedback also on the positive things, showing which things are going in the right direction besides those that could be improved. Really appreciated!

Besides that I shared that for me nothing is too basic to pair on. I learned that I always learn something, especially with different partners who bring different perspectives. Sometimes it's great to see a different approach, but sometimes it's also great to see when we share approaches or think in similar directions. I started out on my testing tour because I did not know where I stood regarding my own testing skills and knowledge, if what I was doing made sense at all.

We both shared the approach to first start with the base frame and then dive into the details. As Thomas added, there's also the other approach of going fully destructive at once. However, he normally likes to see the happy path first, how the application was intended to behave. He made better experience to first provide feedback around that to developers instead of instantly coming up with rare strange cases causing the implementation to fail.

The whole communication with Thomas upfront and throughout the session was so easy, I really enjoyed it. Furthermore, I really appreciated that Thomas created a safe environment for both of us from the very start of our session. He shared right at the beginning that he was nervous and excited, which made it really safe for me to share my thoughts and feelings with him and put everything on the table. Neither of us had a problem to call something out during the session, as this trust had been established beforehand already. 

Thomas shared that he loves to experiment and try new things in order to see which of them work. He also wanted to give mob testing a try, especially after having experienced strong-style pair testing now. With the strong-style approach you don't have any chance to lose focus, it's simply awesome to have two people or even the whole team fully concentrating on the task at hand.

The Inspiration

In the very end, Thomas gave me some feedback on my whole tour and what I am doing. He said it's a real inspiration for him to also get out of his comfort zone. For example he had recently decided to invest the time to join the Tuesday Night Testing and really valued the event, having great conversations and experience exchange. Honestly, this is so great to hear as it's exactly what I am trying to do: show people how to safely get our of their comfort zones and how valuable this can be for their personal development as well as for the rest of us learning from them.
Thomas also presented the example of Kim Knup, who just decided to tackle Alan Richardson's note taking experiment and made this decision a public commitment. This created the same sort of bindingness. Thomas shared he really wanted to use some of his time focusing on learning. We both encountered people who do not take any time to learn, not even half an hour. Many seem not to belief they are entitled to use their time for learning, "away from their tasks", a few might also not want to grow (which is a pity). Thomas said that sometimes we should do the things that we think are great fun, and sometimes we should do things that are on the edge of our area of responsibility. I so much agree.

I am already looking forward to meeting Thomas in real life again at the next Agile Testing Days end of this year. We are both going to join the Web Application Security tutorial by Dan Billing. Also, Thomas plans to join Toyer's and my workshop of Finding a Learning Partner in the Testing Community and hopes to find an accountability partner for himself there! So awesome. I really hope he will, alongside many more.

No comments:

Post a Comment