Tuesday, September 12, 2023

AskAppSec - Input Validation

Input validation is a topic that's been following me around for years. I've came across countless resources speaking about the importance of input validation, or input filtering as it's called at times. What stuck with me is the recommendation to validate any input coming from any source, no matter if we're speaking about third parties, public interfaces we offer ourselves, internal services behind a firewall or accessible only from inside a private cluster. No matter if the input comes via clients, APIs, messages, data sources, or anything else. Based on my experience of working in testing and quality focused roles for over 14 years, I couldn't agree more. All that makes a lot of sense to me. Not only from a security point of view, yet also from a holistic quality perspective as input validation can help prevent errors, improve usability, increase observability, and more.

Here are a few interesting resources speaking about input validation from a security standpoint.

Well. Seems convincing people to validate input is also a common challenge in AppSec; it definitely is in my bubble advocating for better quality outcomes. Most frequently I've seen these discussions when working with backend for frontend (BFF) services. This architectural pattern is often applied when you develop mobile applications, yet not limited to it. It is usually found along with having a bunch of downstream services that are all tasked with different duties. The BFF acts as the main entry point or proxy for a single client (e.g., the mobile app), hence only this API is public to the outside world. Any requests are routed through the BFF to the respective downstream backend services (which are usually protected further). Doing so, the BFF orchestrates incoming requests to different services; it can take care of authentication and authorization, as well as filter and aggregate data in order to respond with the needed information.

If you'd like to learn more about BFFs and see visual or code examples, I found the following resources useful.

What about input validation for BFFs now? What I've heard frequently from colleagues can be summarized in the following statements.

  • "The BFF is just an API gateway."
  • "The BFF should not contain any logic, just pass through anything it receives to the backend services and vice versa."
  • "Only the downstream backend service behind the BFF should validate input as it's their responsibility, otherwise we replicate the same logic everywhere."
  • "Well, it's okay for the BFF to do sanitization, yet not validation."

Here's my viewpoint, and I'm very curious to hear further opinions.

Any modular component of our system needs to sanitize and validate input coming from outside in order to prevent falling into an unknown state. This is both causing poor user experience as well as presents an attractive situation for malicious actors looking for further insights that can be used for exploits. Components include frontend clients in favor of usability, even though malicious actors can easily circumvent them. As long as there is an interface accepting data from another source, there should be validation. Under that premise, I do think that also BFFs should validate incoming data, especially from the public facing side, yet also from internal backend services or other data sources. The BFF is one of the first layers of defense we have, hence if input is validated, we leave the door less wide open. We cannot rely on underlying backend services having foreseen anything that could enter the system from the outside and having sufficient mitigations in place; especially if they are developed by other teams in other contexts who might not even realize the impact of their local decisions. Also, lots of people tend to underestimate threats coming from within the company, like malicious insiders or mere human mistakes. I do understand that it's not always pragmatic or feasible to validate input on all boundaries. Yet if we have to decide, I choose validation at trust boundaries, like between the BFF as public interface and the outside world, as a minimum.

That being said: context is crucial, as always. Maybe the policy to only validate on the most downstream service works well in your situation. Maybe this is considered way too dangerous as you might be aware that specific services are not in good shape. Or maybe your product domain's nature means you're dealing with lots of confidential, sensitive data and you are more invested in keeping people out right at the door (aka the BFF API) without letting them any farther in. As usual, it depends on the risk appetite of the company, combined with your own ethics of what potential impact and harm you deem acceptable or not.

One thing I learned over and over again in my career is that arguments might convince rationally, yet they often don't reach people in a way that they change their behavior. They usually need to experience it, and usually a few times (and I'm not excluding myself in this equation). The trouble with security and similar quality aspects: I want to prevent the experience of harmful impact as much as possible. Speaking of the topic of whether it makes sense to validate input also for BFFs. I could of course invest in exploiting lack of validation, or showcasing a close to real situation, yet it's effort that still does not easily change the narrative and then behavior. If you have any further idea or tactic for these kinds of situations, your input is appreciated.

All in all, I do have a strong opinion on this topic, yet I hold it loosely enough to allow myself to be convinced by better ones. I did find posts that support my standpoint, like "Web App Security: Understanding The Meaning Of The BFF Pattern" by Syed Wahaj - yet that might be pure confirmation bias. So, I'd sincerely love to hear your thoughts and experience about this and learn more: should BFFs validate input?

UPDATE: I've shared this question with the wider community and received some validating feedback. My appreciation to everyone who offered their thoughts!

What I found especially insightful was the following input from a We Hack Purple Community member, hence sharing it further with their permission so it can help more folks besides me. I think they nailed it, so I mostly maintained the original take with slight format editing from my side.

I think the answer to "Should BFFs validate input?" really depends on what it does with the data. The BFF will need to validate some input, but not necessarily all of it.

Generally, anything that looks at the data, parses it or interprets the data in any way will have to validate input.

A BFF will likely look at the HTTP request headers, so it has to validate those. It cannot assume that the request headers will be sensible, or not malicious. It may also have to decide how to deal with duplicate request headers, etc.

But maybe the BFF does not look at the request bodies, and just passes them through to the backend.

It probably would not make sense to duplicate lots of application logic in the BFF to perform application specific input validation on data the BFF itself does not process. Unless, maybe there are very common things that may make sense for a BFF to filter out before bothering the backend with it. But that would then be a bit more like a WAF that may do some general input validation, like looking for common SQL injection patterns. And this still does not absolve the backend from its input validation responsibilities.

The backend services will always have to validate the input they are handling, but even backend services may pass some data through to other downstream backend services. The important thing is that everything that looks at the data and processes it performs validation, whether that is a web server, an API gateway, a web API, or a BFF.

My deepest thanks go out to the person who took the time and energy to elaborate on this. They made the distinction I was looking for (without knowing I was): what exactly makes sense to validate where and why, given the specific context at hand. I think this is what I struggled with myself and hence struggled to convey more clearly to colleagues. Taking this explicit distinction, I feel enabled to map it to our context to make better informed decisions, and I also feel equipped to bring more clarity to the next conversation on input validation!

As a bonus, here's one more thoughtful response allowing us to weigh further aspects against each other.

Friday, September 1, 2023

SoCraTes 2023 - A Place Where I Belong

I nearly didn't go to SoCraTes this year, the "International Conference for Software Craft and Testing". My speaking budget was already strained, my schedule overbooked, and it would have meant going on vacation time. But then the organizers reached out and offered me yet another slot on the training day this year. They were even fine with me giving a workshop I already had prepared, and I changed my mind. I seized the opportunity and went to SoCraTes on vacation. What can I say, I don't regret it one bit! Granted, I'm super tired, and at the same time I'm super happy. It was so much worth it and convinced me to reserve this time of the year for 2024 as well!

Coming Back

It's always super exciting to join a conference for the first time. The second time around, a few things are already clear - you know the venue, more people, the procedures and other things that reduce your cognitive load. Still, the second time is curious as well - how will they welcome me this time? How easily can I reconnect to where we left things last year?

The first moments are awkward for me, at any conference. At SoCraTes, this very quickly vanished into a feeling of belonging. I felt welcomed, I was included, I had a right to be there. This is the foundation for everything coming afterwards.

My first contact after seeing familiar faces at registration was a person being there for the first time, Lydia Leifels - such a pleasure right from the start! At dinner, I met dear people I already know for a while, like Tobias GöschelThierry de Pauw (along with their daughter), Juke Trabold, and Woody Zuill. There were so many awesome folks I got to know or meet again over the course of the conference! Like Janina NemecMarc KalmesStefan ScheidtClaudius LinkLea RosemaWaldemar TommeMartin Schmidt, Markus Tacker - well, the list could go on and on.

Training Day

The second edition of the training day was even better than the first one. Trainers were amazing both times, yet for this year organizers listened well to the feedback and crafted a schedule of three tracks with each session having enough time to dive into the topic and generous breaks in between. Just awesome. Here's the choice of sessions I made.

  • "Enforcing Architecture Using Tests" by Javiera Laso. This was the first time I heard about ArchUnit, a library to check for conventions like file names, structure, dependencies and more.  Writing tests was quite straightforward, and I can see how these could support maintainability and reduce friction by codifying agreements.
  • "Modernize CI/CD Session" by Raimo Radczewski & Chris Neuroth. Very interesting talk sharing fundamental principles for a pipeline optimized for quick feedback while going small, safe steps. They demonstrated live how fast a change can be on production. A few statements I really related to were these: "minimize the time from code written to code on mainline, deployed to real users, running in a real environment, ready to be evaluated - there is no other way to develop software sustainably", "influence the loops you can", "the moment we stop slicing we deliver slower", "skip the review, pair with someone".
  • "Ensemble Exploratory Testing" by me. Fun fact, I've given this workshop now for the 10th time so it became my most repeated conference session ever so far. Good thing is, it seems it doesn't get old! This time again, people were eager to join and seemed to enjoy the learning experience (which hopefully convinced them to try new approaches back at work). Well, I had fun observing people having lots of aha moments together.
  • "Take a mess, make a mess, fix the mess" by Aki Salmi. A very interesting session on refactoring code that's untestable and contains hidden domain concepts, code that's still valuable yet needs to be modified. How? Together, we tried an approach many of us had not seen before: not even trying to understand the code in the first place. Instead, using the IDE's automated refactoring tools to slice it up first, turning hard to test code into easy to test code. Then we can document its behavior in tests, and hence gain our safety net to make the required changes. Check it out, you can follow every small commit Aki made. Nicolas Carlo also wrote a great post on this where you can follow an example: "Another way of refactoring untested code" (by the way, his newsletter is super insightful and a clear recommendation).

After the training day ended, the main part of the conference was opened with a world café. Find a group of people you don't know, take a question as starting point, see where the conversation leads you and doodle your insights on a shared canvas. After a given time box is over, everyone find themselves new groups besides one person staying at the table and getting the newcomers on the same page. Repeat until you finished three rounds. Really nice exercise, perfect to get to know first people ahead of the main conference part and have interesting conversations emerge. One main theme we had was on people, community, culture and how that's foundational. In case you're wondering where to start, I have a resource collection on inclusion that really helped me.

That's not the end of the day, of course - SoCraTes goes all in! Everyone is at the same place, we're together the whole time, so of course there's an evening / anytime during the night / morning schedule with bonus activities suggested by anyone. Such a good thing our hotel rooms are right there to retreat and rest any time. I love how Juke as facilitator and also organizers continuously emphasized the importance of caring for your needs and taking breaks.

Having learned from last year that joining all sessions can be quickly very exhausting, I decided to use the evening time for conversations and enjoying both atmosphere and company of wonderful people.

Open Space Day 1

Days start early at SoCraTes for me being a night owl, yet it's worth it. In its core it's an unconference offering an open space for everyone to bring their topics. Things they want to share, conversations they'd like to have, apps they want to build, tools to try out, skills to practice - and also challenges they'd like to get help on. With so many different people, there's a whole range of topics offered, covering a spectrum of deep dive tech topics to humans and culture as foundation for everything. No matter if it's very personal, related with our professions, challenges in society, or all combined, everything is represented. There are also sessions like doodling together, painting your nails, talking about sheep, whatever is most valuable to people right now. Be prepared to be surprised! This format of building your own schedule together on the fly works amazingly well. It definitely worked out well for me again this year.

  • "Web accessibility - building beautiful web sites that don't make you puke" by njan Völker and Lina Sievering. I had hoped to learn more about accessibility at the conference, and already the first session was right on spot! Awesome workshop focused all around motion sickness induced by websites and apps. A topic close to my heart as I'm affected myself. Njan and Lina provided mindful and enlightening exercises to convey the problem and think of more accessible options together. Thanks to them, I also learned about the European Accessibility Act becoming effective as of 28 June 2025 - which means accessibility will be enforceable in Europe which hopefully gives us more leverage for accessible solutions when making product decisions.
  • "Capture the flag together" by me. As part of my personal AskAppSec challenge, I recently tried out further services offering hacking labs like TryHackMe and Hack The Box to practice penetration testing. The first challenges were good fun to me, so I thought why not offer a session and do it together during the open space. I was positively surprised how many people came and joined me! I opted for Hack The Box and their starting point machines. It worked super well, people were engaged, shared lots of knowledge and we captured a flag together (the second we missed only due to my VPN interfering). A very insightful experience, validating my assumption that there's interest and these could be great sessions for more people to learn about security in a fun way.
  • "Security for devs (& everyone)" by Claudius Link and me. The more security sessions the better, right? So why not host another one, together with Claudius who had the same idea. We had a great conversation with people bringing up all kinds of insights and common challenges. Nothing was immediately new for me, and yet it was validating to hear experienced people share similar views.
  • "Documentation as Code" by Markus Decke. We work together at the same company, and we are transitioning to having more and more documentation as code, so I was curious about other people's ideas, struggles and in general experiences. We talked about benefits and use cases, shared tooling options and their limitations. One of the main themes was around what problem we're trying to solve and then aim for tackling exactly that, nothing else not to document only for the sake of documentation.

What about the evening? Well, after open space is before open space! As mentioned, there's always an evening schedule people are building up. Once more, I opted for dinner, conversations and more fun time! Last year I discovered that Janina Nemec is an absolutely pro in playing Set, a card deck game I've learned to love through the testing community. At first we didn't spot any deck so decided to opt for a round of Exploding Kittens (so much fun) - and finally discovered a Set deck in the end. Well, next year we're prepared to bring our own decks. Such a good way to close the day.

Open Space Day 2

New day, new schedule. Every day starts a tiny bit later, which is still early for me. And yet it was awesome again and worth the early start.

  • "Stop being a superhero!" by Janina Nemec. She did an early dry run of her upcoming talk to be presented at Agile Testing Days this year - which is your chance to catch this talk, you'll be in for a treat! Janina has vast experience of working in an ensemble full time for many years. In her talk she describes (superhero) behavioral patterns she's observed (like the architecture wise or the coding wizard - way too relatable) and the pain points that result from them (like lots of work in progress without things getting done, or the behavior not helping the team grow and work sustainably). There's a solution for this: working together as an ensemble, or team programming as she calls it - saving the world together.
  • "Build a minimal showcase app" by me. I have a recurring argument around a security topic and wanted to finally start building a minimal app to demonstrate good security practices. So what could be better than get this started together right at the conference? Nonetheless, I nearly didn't dare to suggest the session. And then I thought no one shows up (I didn't realize it was break time). Until people did show up and we formed a wonderful little ensemble helping me get started on a good way. Everyone else enjoyed setting things up and realizing that we all struggle in certain areas. When doing it together, however, we usually have the missing piece of knowledge in the round to avoid friction, solve problems without frustration, learn with fun, and get to value fast. A pleasant experience, would have loved to continue together.
  • "Security scanning in pipeline" by Raimo Radczewski & Chris Neuroth. Both wanted to try out security scanning tools like Trivy, Syft and Grype on a real case example and see how they work and what value they bring. A really interesting session that then also sparked a serendipitous hallway conversation on why attack trees might work better compared to threat models in order to get people to think like malicious actors and consider risk.
  • "Security games" by Claudius Link. He brought a whole bunch of games to teach security in a safe space with fun. We could try out a few of them and gain experience how they could be used for educational purposes. Games like Elevation of privilege and OWASP Cornucopia, yet also [d0x3d!] and a lot more I can't remember. Now I know there are a lot more out there to try out!

This was the last day for the main part of the conference, so we closed it with a retrospective. The best part here was that it didn't merely gather feedback for organizers and Juke as facilitator, it also was  intended to provide feedback for each other as participants. Over half of my group were here for the first time, and I just loved hearing their feedback and also input to make it even better for each other.

A gratitude round followed. The challenge was to thank five people we have not thanked yet. Honestly, this experience was a bit overwhelming, in all the best ways - and more than one feedback took me by surprise. I haven't mentioned yet, this conference goes big on hand-written kudos cards you can hand out any time for anything you appreciated the other person doing. The ones I received I will keep with me for long, they present such a dear memory.

Dinner followed with more great conversations. Then further sessions were hosted (remember, the evening schedule). I had a great time joining a code kata ensemble. We did the "Vending Machine Kata" which was an insightful exercise itself, yet my main takeaways were on the collaboration part. It was fascinating to see an ensemble start off quite free style and converge to more structure like having a dedicated navigator, using a timer for rotations, having the navigator stand up to be more prominent, etc. It just worked better with folks who have never worked with each other in this way. Also, we used the fish-bowl approach to ensembling, having outside observers and always open spots to join the ensemble. This exercise made me realize once again that this approach is simply not for me, despite having super kind and safe people around me it felt exclusive. I'm very sure it's actually more inclusive for other people to opt in. For me I much prefer the "all together just one ensemble without observers" format.

Last but not least, a retro gaming session! We played the old adventure game "Zak McKracken and the Alien Mindbenders" all together on a C64 system provided by Tobias Göschel - how awesome was that?

Workshop Day

Have I said the conference is over? There's still the additional workshop day! And what better format to have than asking people to bring their topics for hands-on sessions also on this day. It's also the time of the traditional code retreat, practicing the whole day solving the same kata in various ways. Originally, I was adamant to join the code retreat again, it was simply an amazing experience last year. And then Claudius Link came and suggested the only thing that could possibly lure me away: co-facilitating a security workshop. Well, what shall I say? I couldn't resist, this fit way too perfectly to my personal AskAppSec challenge this year.

We aligned on our main thoughts of what to aim for, pitched it to a fellow participant, incorporated the feedback, and came up with a workshop on "Painless Security". We crafted the agenda shortly before and went ahead with it, playing it by ear and experience in giving workshops. Having co-hosted a session the other day helped, too. Co-facilitation worked super well together, we often thought along the same lines and built on each others ideas. People shared many painful experiences and we gathered potential things to try out and what we related with most to bring back to work. I took a lot with me myself.

During lunch time, the security theme continued for me. First thinking about security conferences and ideas to contribute to the space together with Claudius Link and Susanne Neunes. Heading back towards the conference rooms, I noticed another table where Martin Schmidt and Philipp Zug, who also both participated in our security workshop, were scheming on a new security card deck. I loved the idea and they were so kind to invite me in. Well, it seems I got myself involved in two new topics - and yet I feel these are very much worth it.

I decided to check out what other sessions might still run that I could join. First, coffee though - and that's when Matthias Klass approached me and asked whether we might have another capture the flag session together, as a couple of people expressed their interest. Well, I didn't need to check the schedule anymore, security theme it was for the whole day! Of course I'll host another capture the flag session. What can I say, it was awesome! We spent the whole time until we officially had to leave the room and head for dinner. Nearly instantly, the idea popped up whether we could ask the hotel for opening the room once more during the evening. Asking was worth it, we got the keys and went all in after dinner. Many more rounds of capturing flags followed until everyone was so tired we couldn't think anymore - while being just super happy. Some of the challenges tackled were very familiar to us and hence solved a lot faster, others required us to piece together knowledge we usually don't need. All of them were very insightful and fun. So much fun. I loved that we all worked together so effectively as a big ensemble again. I really want to do more of this.

There's a traditional count of code katas done at SoCraTes, counting each exercise by everyone. I was really moved seeing so many people join me on practicing penetration testing, staying with me for so long and sharing my enthusiasm. It felt we just established a new counter at SoCraTes next to the code kata counter: the captured flags counter, aka the number of security secrets discovered. We collectively increased it to 8 overall! 

Why again next year?

Heading home, my heart was full, my brain energized, my body tired, and me super happy. I was certain that if I have any chance to be back next year I will take it.

Besides the obvious reasons of self-selected, very insightful content and just amazingly kind and inspiring people, there's a reason that is even more important. This conference is the best I've seen so far in intentionally designing welcoming and inclusive spaces. They continue to reduce friction and make it more accessible and safer to more and more people. Literally every year. There are lots of aspects of this to be found everywhere. Not only in the code of conduct that's actually being lived and enforced, or offering all gender toilets, or giving away tickets based on a lottery to level the playing field. It's in every little detail. Less noisy applause by waving hands. Exact food labelling and plenty of options for everyone. Child care service so lots of parents could join this year. Sharing Covid tests upfront for every day and encouraging masks.

The level of diversity already achieved has a huge positive impact on the quality of conversations and insights gained. A lot to learn and take with me to do better myself.