AskAppSec - Finding Closure

My personal challenge of the year, AskAppSec, came to an end and I finally found closure. Here I'm looking back to see what happened and what I can take with me for my next endeavors.

What I aimed for

The personal challenge I set out to in 2023 was to connect with security folks and related communities to grow my application security knowledge and skills. I've detailed things out in another blog post, so let me just re-share my original hypothesis here.

I believe that joining and actively participating in at least one security community for a period of six months will increase my understanding of practical application security in everyday work situations. I've proven the hypothesis when I have...

• solved five mobile application security challenges,
• explained how I solved them, and
• asked community members for their review and feedback to learn from.

 

What happened

Due to a lot of other things happening in life privately and at work, plus me taking up my conference speaking endeavors again more seriously since the pandemic broke out, I had a really late start with my personal challenge only in May this year.

I looked around for security communities to join and asked a lot of folks for recommendations. At first, I wanted to only join a choice selection to fully focus my engagement on those. Relatively soon though, I opted for a different path and joined as many communities as I found in order to figure out those where I found value for myself, that were open for newcomers, and that felt welcoming for me to participate.

Feeling overwhelmed by options, I started doing more of what I found valuable right now in the moment, and that gained me some dearly needed momentum. I also managed to secure a ticket to my first security conference, BSides Munich 2023, which in itself added to said momentum.

In my previous challenges, I used to take one action, work on one topic, instantly blog about it and then take the next step. This time around, I realized I did lots of things at the same time, overlapping with each other, and then wrote my blog posts rather at the end in a row. Here are the posts that matched the five main topics I chose to work on.

• AskAppSec - Input Validation
• AskAppSec - Painless Usable Security
• AskAppSec - Security Champions
• AskAppSec - Dependency Updates
• AskAppSec - Capturing Flags

Besides sharing these blog posts on my usual social media platforms, I also asked explicitly for input in the communities I've joined. Sometimes just in one of them, sometimes in multiple, depending on where it felt safe enough and if I've practiced asking strangers enough already to dare it. Sometimes I received feedback from folks on these posts, sometimes even feedback that added to what I already wrote so I edited my blog posts to reflect it there as well.

Through all this, I did make new connections to security folks. These new bridges between specialty roles and also fostering previously existing relationships really helped my own growth and offered opportunities for me to contribute back to community.

I've built up a new recommended resources page on all things security. It's still growing, yet hopefully already useful for others as well.

Finally, I'm ending my challenge later than planned and granted myself the freedom to do so - even though this broke my original constraint of ending it by end of October.

So, did all this increase my understanding of practical application security in everyday work situations as I believed in the start? Based on the conversations I ended up having at work and getting closer to our InfoSec team as well, I believe it did indeed.

Where I struggled

To be frank, I really struggled with this challenge. On the one hand, I'm supposed to struggle with my personal challenges, otherwise they wouldn't get me enough out of my comfort zone. On the other hand, this one felt particularly difficult to me.

• My late start really weighed on me, as usually I make use of the fresh energy of a new year to get things moving and then build on the momentum.
• I realized once more that it's only a subset of folks being engaged in communities. This is the same for all kinds of professions, something I've seen in testing and quality, development, architecture and so on just as well. It's a bubble in a bubble. This made getting recommendations harder than I thought it would be.
• Originally, I aimed to focus on mobile specific security. Sometimes this was the cases, yet mostly my topics were not super specific to mobile and instead applicable to other areas as well.
• Asking communities felt super daring. I am proud I managed to do so. While knowing I might not receive a lot of responses, of course I hoped for feedback. Well, I mostly didn't receive much input at all, which can be quite discouraging. Mostly it was either feedback through social media from communities I'm already in - after all, this seems like a natural thing. They know me, we're already well connected, and I still value the provided feedback a lot - I'm grateful. In other cases, it was feedback from new communities that merely stayed on the surface and unfortunately didn't add to what I already wrote or gave me new pointers. Sometimes, though, there was just brilliant feedback that really helped me and triggered new thoughts, so I'm trying to cling to that.
• I took on too many commitments next to my personal challenge and really struggled with my capacity. I neglected personal constraints I usually have on my personal challenges to keep some part reserved for self-care, and it drained my energy.

What I learned

This challenge taught me quite a few lessons. That fact in itself already really made it worth it. I'm definitely richer in experiences and knowledge than I was before. Here's what I'll take with me on top of the gained knowledge, skills, and connections.

• Sometimes life takes over and has unplanned demands, and that's okay. That could either lead to pivoting like I did with my challenge in 2020, or still pursuing it while being more flexible about like I did this year, and that's okay as well. I do need to take my own advice of good enough being actually good enough more often.
• It doesn't matter that I didn't do everything as I envisioned, as long as I learned valuable things - and I did. It's not about reaching everything; it's about taking actual steps instead of just wishing I would be the person who had taken those steps.
• As outcome of this challenge, I do have more connections to security folks now. Not a whole lot of them, yet valuable, deeper ones. Once again this shows me that quantity is not everything.
• Connections made face to face, be it remotely in a video call or in person, are way easier for me to make and they tend to hold longer. Therefore, I'm looking for joining more meetups and conferences that offer the opportunity to speak with one another.
• I really should not start too many new unrelated things at the same time overlapping the personal challenge I set out to take on. This year, I overdid it. I had unlearned how to enjoy myself for myself and overstepped my own boundaries which left me drained. The recent weeks where things finally got closer to the end really showed it to me: my body told me to stop and finally take the rest I need.

Conclusion

While I did really enjoy diving into security further again and this will definitely not be the end of this journey, I really needed closure on this specific challenge. There's always opportunity cost to consider and I need to free myself up for new things. I already took on topics reaching into the next year, and I am already looking forward to these next endeavors, so I want to make time for them without feeling overwhelmed. Therefore, one personal challenge a year is still a good thing to force myself out of my comfort zone while also keeping it within limits.

Having focused on my AskAppSec challenge during this year, there are still so many topics on my list of things I could do or write about. The good thing is, just closing this challenge doesn't mean I cannot pursue them anymore. It just means I'm leaving this open for myself, and I feel that's a good thing. And it also grants space in my life that's not just commitments to others, but commitment to myself.

Now, before revealing what I'm up to in 2024, I'll indeed take some time for myself. But looking back at AskAppSec, it's overall been a good challenge at a good time for me. So, let me close this post with a huge round of applause and gratitude for all those folks who talked security with me this year and hence helped me on this part of my journey - my sincere thanks to you!

AskAppSec - Capturing Flags

Deliberate practice proved being invaluable in my own career. The last months showed me once again that this applies to the field of security just as well.

As we can't practice security related skills on just any system without causing harm, we need dedicated spaces to practice safely. Fortunately, there are lots of options readily available out there, way too many to list them all. Hence, here are just a few sites that provide not only great starting points yet also the opportunity to go as deep as you can.

• OWASP Juice Shop: This is an intentionally vulnerable web app, mimicking a quite common e-commerce scenario. Based on this, you get a set of challenges presented that allow you to try out techniques to find and exploit the present vulnerabilities. It's been my own entry point into practice apps for security and the gamification behind this app in particular really drew me in further.
• OWASP WebGoat: Another commonly cited OWASP project that offers you a place to practice. In this case, you go through dedicated lessons to learn about vulnerabilities, to see how they work and how they can be mitigated.
• Hack The Box: This service offers you a huge amount of prepared virtual machines aka "boxes" to practice on safely. I really like their starting point machines that guide you towards the secret aka flag you're trying to find and introduce you to commonly used tools to identify and exploit vulnerabilities.
• TryHackMe: Another service offering lots and lots of machines to practice on. You have plenty of themed learning paths to learn on with a lot of detailed information to guide you on the way. Both Hack The Box and TryHackMe have big communities active on Discord offering a great support network.
• PortSwigger's Web Security Academy: The developers of BurpSuite provide a great resource with lots of challenges to solve in order to learn more about web security in general.
• PentesterLab: The courses offered on this platform include lots of explanation and guide you step by step to learn skills needed for penetration testing. My thanks go to Yianna Paris for introducing me to this service!

Besides these dedicated apps and labs available around the clock, you can also watch out for hosted public capture the flag (CTF) events. I've recently joined one from Huntress and I see several being announced for the upcoming holiday season, like TryHackMe's Advent of Cyber or the SANS Holiday Hack Challenge. Being in security-focused communities and following more security folks on social media really helps to learn about these CTFs. Alternatively, you can check dedicated sites like CTFtime to look out for the next ones coming up.

When practicing in these kinds of spaces on such kinds of challenges, I've experienced the following benefits.

• Reduce scariness. Dipping your toes into security can by scary indeed. You might not know where to even start, so having these kinds of practice spaces can serve as just the starting point you need. More often than not, they include challenges designed for beginners that offer further explanation and guidance to get you introduced into the space.
• Grow knowledge. Through these practice apps I usually got introduced to something I didn't know before, be it a concept, a tool, or anything. For example, they also provide a great reason to get to know security focused Unix systems like Kali Linux, Parrot Security or Mobexler and their respective tool boxes.
• Hone skills. The more we practice, the better our skills get, and the more we can make connections between things we know. More pieces to complete the puzzle, or in our case the next challenge. Creative problem solving is definitely a skill we're practicing here!
• Build confidence. The more touchpoints we get and the more we seize practice opportunities, the more we can grow our own confidence that we can also figure out the next challenge.
• Spread awareness. We can use the gained knowledge and skills to raise awareness about vulnerabilities with others. Even better, by practicing together we can increase awareness in real time. These kinds of challenges can help people see what's possible and why we need to defend our systems, protect value and keep harm away.
• Find joy. Security can be perceived as such a dour and tedious topic. Finding solutions to security challenges, however, can feel very rewarding in itself. Doing challenges together can further help with connecting security with fun and make it more interesting for people to engage with. It can also help to find community and like-minded people to learn and grow with.

All of these advantages I've experienced myself as I've been trying out various vulnerable apps, a bunch of labs offering dedicated challenges, as well as dipping my toes into my first public CTFs. I've also seen them over and over again with conference participants, joining me for many sessions of "capturing flags together" at SoCraTes, FroGS Conf and Agile Testing Days. It's been just the same when hosting practice sessions with colleagues in the past - something I'd like to pick up again in coordination with our current InfoSec team.

So, just practicing within these spaces gives us everything we need, right? Well, unfortunately that sounds too good to be true. There are also downsides to these kinds of challenges. Kudos to Dave van Stein for making me think more about this!

• Artificial challenges. All these spaces are crafted with a specific goal in mind, usually to educate and provide a safe place to practice. Therefore, challenges are inherently artificial and can't fully represent real-life scenarios.
• Mindsets differ. Attackers tend to think differently. I mean, they usually don't have the one clear flag to find in a constrained environment to announce their win. Instead, they might gather all kinds of information over a period of time, and based on that build their strategy on whether to exploit identified paths into a system, what to gain from it, and so on. It highly depends on their motivation and goals as well.
• Uncertainty instead of solutions. For labs and CTFs you'll know when you made the right move, you get a reward. In real life, there's no cheat sheet, there's no walkthrough. Just potential and ambiguity and never being completely sure that whatever you've found (if you've found any at all) is all there is to find.
• Overly focused on penetration testing. All the sites listed above are mainly offered to practice penetration testing. It's the one hot topic that probably attracts most people, but how often do you actually need those skills? How many jobs are in this area compared to all the others? There are so many more skills needed in the field of security. So where are the challenges on secure coding, or secret storing, or vulnerability evaluation, or threat identification, or incident response, and so on? Well, more practice apps are being built all the time, so some of these do exist already while not always being in the spotlight. And of course, there's official formal training to have as well (though it can come at high costs).

Weighing benefits and downsides against each other, I consider deliberate practice opportunities like the ones listed above still invaluable. We do have to be cautious, though, to put them into perspective and be clear about their goals and limitations.

That leaves me with yet another question: can we practice closer to reality? Here are a few approaches I think we could experiment with. I'd be happy to hear about further options to add to the list.

• Replay past real incidents. I'm thinking of actual security issues that your own company faced. We could replay these very real scenarios both from an attacker and a defender point of view and hence learn what we can do better - very concretely for our specific situation.
• Run open thinking exercises. Deliberately practice approaches like threat modelling, attack trees and similar to improve our thinking, within our actual work context to make it as applicable as possible.
• Host custom-tailored CTFs. Have one person hide a custom flag on your own system for people to find. It might still be an artificial scenario, yet placed in your very real context. This requires quite some preparation of course, like a dedicated environment to practice on and, as usual, explicit consent from all involved parties. The gained insights might still be worth the effort.

Personally, I'm sure I'll continue making good use of the various practice opportunities there are. I'm also considering joining or starting a CTF team to make practice even more deliberate. If anyone's interested or has recommendations for good places to find a welcoming, inclusive and diverse team please let me know.

Now let's bring this question back to the community: what do you do to practice your security skills?

AskAppSec - Dependency Updates

When one of my former managers commented on my blog post on Painless Usable Security, asking about our approach of keeping dependencies up to date, I realized that there's more to the topic and I should write a separate post about it. So here it is!

Keeping dependencies up to date has been a big topic in lots of the teams I've been part of. When I created our AppSec strategy for my current team beginning of the year, this topic once again stood out to me as the first one to tackle. This is based on our context. We have a whole bunch of services we own and most of them are around for quite a long time (and still valuable). We inherited a system that had degraded over time, and knowledge had been lost. Over the last two years, we learned to understand this system a lot better, as a whole team. We invested in several endeavors to get it in shape again, to preserve its value, and also ease extendibility.

Before going into details, let's first take a few steps back.

Why even have dependencies in the first place? Nowadays, most software is not built in an echo-chamber. It'll be based on lots and lots of other software that lots and lots of other people provided, so that we all can achieve more with less. There are specific contexts where dependencies will indeed be very constrained, usually in situations where stakes are very high. Yet even then you probably don't invent your own programming language, operating system or infrastructure. For the kinds of products that I've worked on so far, the system will have lots of dependencies to third-party frameworks, libraries, and more.

So, we need dependencies. Yet why should we update them and keep them updated? Can't we just keep everything as it is?

• Even if we don't change anything on our side, the world around us does not stop. Tech evolves every day. If we don't change anything, our system will naturally degrade from both business value and security perspective. More vulnerabilities in the dependency versions used will be found, and the system will be more and more at risk. Until suddenly it's very time-critical to have that risk mitigated. Remember Log4Shell, anyone?
• Besides security fixes, you'll also want to use new capabilities that dependencies offer in their newer versions. These can help preserve or even increase the value of your system and keeping it relevant to users, business and other stakeholders.

Fine. Yet why do people struggle so much with updates? Just do it, right?

• Dependency updates oftentimes come with the need to adapt your system to the new version. Sometimes that means completely re-architecting your solution; very painful. Sometimes dependencies had removed or replaced features in their newer versions, and you'll need to cater for that. Even if there's no obvious need to adapt the system, updates might break functionality in lots of surprising ways - so you better have suitable measures in place to mitigate that risk and detect regressions early.
• Updates can come in very fast, sometimes multiple times a day. Think about the JavaScript ecosystem for example. It can be overwhelming and feeling like a Sisyphean task: as soon as you got your system in an up-to-date state, there's a new version of at least one dependency released again!
• Dependencies have dependencies themselves. Oh my. Updating one dependency often means it's not compatible anymore with the rest, so you need to update a whole bunch of others as well; especially when updating frameworks. This also means, by the way, that dependencies have to keep on top of their own dependency updates. The struggle is real.
• Dependencies might be discontinued and not get any updates any longer due to a variety of reasons. They might get moved to other places, integrated into other projects, you name it. Sometimes you can migrate, sometimes you have to find similar tools from scratch again to meet your needs. For any newly introduced dependency, it needs evaluation and validation again (better do it regularly for existing ones as well). Licenses are crucial to consider, especially when including open source dependencies. Checking its usage, how many people contribute to the project, how many issues had been already identified or fixed, when the dependency was last updated, how much community support it has, and so on. It's good to have a guideline in your context what's considered suitable to choose and what not, and what's the reasoning behind it.
• You have more dependencies than you might think. The number of libraries directly used by your services alone is probably high. Yet there's also the infrastructure you're running on and its dependencies, like container images with their operating systems and - surprise - their respective dependencies. There are dependencies to data storage solutions and so on. It's a lot to keep up to date.
• Probably the biggest hurdle I've seen is getting buy-in to do this kind of maintenance "keep the light on" work as part of normal everyday business. It's an investment and there's opportunity cost - if you invest in one thing, you can't do another at the same time. This often leads to not investing in maintenance at all. A strategy which comes around to you soon enough, presenting you with even bigger investment needs to get back on track again, while already being hindered to follow other opportunities that you wanted to prioritize for business reasons. It can slow you down to a complete halt. Yet as that's usually only a future problem of a potential risk, it's really tempting for people to ignore it (guilty of that myself). It requires lots of experience, discipline, and good practices to still keep your system tidy and in shape, continuously. And keeping each other accountable, we're fallible human beings.
• Last but not least, context is crucial. In industrial cybersecurity settings, updates might have a hugely different impact, not only financially yet also when it comes to safety. Lesley Carhart describes it well in her CyberWork podcast episode. Context really matters, risks can differ heavily.

What about tools to help us keep dependencies in shape?

• There are Software Composition Analysis (SCA) tools to help you detect outdated dependencies, including their known vulnerabilities. They usually compare your dependencies' versions against a list of available versions and reported issues for each. Integrating tools into your source code hosting platform and delivery pipeline can help as well. You might have heard of OWASP Dependency-Check, Dependabot, Renovate, Snyk Open Source, and the like.
• Package managers often come with integrated checkers. Like NPMs dependency-check which also offers to update potentially straightforward ones automatically for you.
• Modern IDEs support you by indicating outdated or vulnerable dependencies, like IntelliJ IDEA Ultimate does. Check out Marit van Dijk's awesome talk Keep your dependencies in check to see it demonstrated along with lots of useful advice.
• There are repositories like Maven Central to get dependencies from in all available versions, which often also help if dependencies moved their artifacts, got renamed, deprecated and more.
• Tools like OpenCVE allow you to subscribe to updates for your technology stack to get alerts on potentially relevant security issues.
• Sometimes frameworks offer migration tools to help with bigger updates.
• More and more people talk about software bill of materials (SBOM) to keep inventory of all kinds of software including dependencies in use.

Lots of aspects come into play when updating dependencies, and there are probably a lot more factors to consider than listed here. The big question to answer is what's most helpful in your context to actually get the job done, get the dependencies updated and keep them updated.

Let's talk about strategies. How can we keep dependencies updated?

In a previous team, we worked with major updates once per quarter, going through everything. This worked okayish for the given context of an internal product with limited usage. In my current team's context, however, we have a customer-facing product with a hugely different attack surface. So far, the following worked for us to update our services' dependencies.

1. Establish, encourage and ensure 20% time for every team member and use it to drive tech initiatives. Like getting dependencies of our services in shape. Having dedicated time to improve certain areas like this was a massive cultural foundation for lots of good stuff happening.
2. Use tooling to support easier updates where feasible. Automated scanners to indicate outdated dependencies, utility tools to adapt required related documentation for compliance reasons, and automated checks to discover potential regressions. Tools like these were great in combination with our system knowledge, so we could quickly unveil more surprises where automation reached its limits.
3. Do the easiest, most straightforward, quick win updates first and get them out of the way. It'll reduce cognitive load and clear up headspace for the bigger challenges, like the ones requiring updates of several dependencies or even frameworks. The advice to "solve the smallest problems first" helped me massively with legacy systems like ours; as far as I remember the credit goes to Nat Bennett, yet I can't find the source anymore (if anyone does, let me know). Having said that, it doesn't mean you shouldn't prioritize updating your most critically vulnerable dependencies first. 
4. Small changes done frequently compound. The system will get better step by step, you'll get better at updating the system step by step. Always a bit better. It might not look like much today, yet a month from here it's already painting a different picture. We'll get there.
5. Build on existing energies and practices. This is one of my tools that helped me with lots of culture change initiatives. We also used this to keep dependencies in shape. We have regular tasks needed for each release, and updating dependencies simply became one of them.

All this, however, likely only worked due to the team culture we fostered where people are sharing everything; knowledge, skills, load, a common goal, and more. This made it clear from the start that keeping dependencies up to date is a team task as well and we're all responsible for it, together. I hope to share more once we've lived this approach for a longer time. I'm myself curious if we can manage to keep our system in shape this way or what other approaches will turn out to be more successful in the end.

A word of caution when it comes to tooling. Scanners are only as good as the team can respond and act on their results. The cry for more tools to make a problem go away, or the desire to just finding the one right tool to save us all is not going to fix the underlying issue. Just throwing more tools on a problem won't move anything towards better - most likely, the opposite will occur. All of this becomes noise. It's overwhelming. There's so much other work to do as well. People start shutting themselves off and ignoring alerts just in order to be able to deliver anything (most likely the thing that others put them under pressure for). Yes, alert fatigue is very real. We see the same overwhelm with observability tools, monitoring alerts, test findings, static analysis feedback, and more. Having yet another class of alerts you don't get time for to understand and fix just does not help get into a better place. Not to forget false positives! Alerts that are simply not alerting you on anything real or actionable or relevant for your context are like poison.

At BSides Munich 2023, Jasmin Mair talked about "My CI/CD pipeline contains all security tools available! Now what...?" (check out the recording, too). I loved the strategy she presented to add one tool at a time, train developers, set a baseline, and manage findings. Like eliminating one class of vulnerabilities at a time, and allowing people to follow. For the case of dependency updates, we decided to go service by service, starting with the most critical ones first, and only afterwards include more tools where needed. First make alerts be seen and worth responding to again. Once we live that, we can always improve further. By the way, that was also our approach to clean up other alerts, errors, log spam, and similar noise - enabling ourselves to see what's actually important again when it occurs.

Here's another side note. Getting things back in shape wasn't done with updating dependencies. It also included other topics like removing unused code. It's always been a struggle to clean up functionality that had been superseded by something better, was just not invested in anymore, or had never been used at all while being dragged along. In one of my previous teams, we decided to get rid of all that under one big theme that also could be sold to business easier: reducing complexity. It's also part of opportunity cost to keep maintaining things that are simply not worth it any more (if they ever were). We can't foretell the future and hence need space to try things out in our product to learn what actually solves the problems our users and business have. Yet if we want to make progress towards a shared goal, we do need to make clear decisions what to keep and what not, and follow through. When I joined my current team with our big legacy system, I was stunned how much unused code was still around from many years ago. This year, we finally had buy-in to clean up - oh how much I loved it! Not only did it indeed reduce complexity and increase maintainability, made things a lot clearer for new teammates, and so on - it also reduced our attack surface! A win on all sides.

A similar case can be made for services and features we want to keep, yet that are complex, hard to understand, inconvenient to modify, or people are afraid to touch them. Not a great starting point when things go awry and a security incident comes in. Actually, if any incident occurs. Even if nothing bad happens in that area, this part will tend to stay untouched until no one in the team knows about it anymore and it's even costlier and scarier to touch it.

All in all, when I came across the following statement in Tanya Janca's book "Alice and Bob Learn Application Security", it made so much sense to me: "technical debt is security debt". Big aha moment, this resonated heavily with me. Good for maintenance and extendibility can be really good for security, enabling us to adapt fast to an ever-changing world.

It's time to bring that original question back to the community: What do you do to make keeping dependencies up to date work?

AskAppSec - BSides Munich 2023

When I started out my AskAppSec challenge, I've asked around for recommendations on communities and conferences in the security space. Jay Harris encouraged me to look for a local BSides event as in his experience these were usually friendly, welcoming and a great opportunity to network. Now that I've attended my first one with BSides Munich, I can only confirm that impression! Just loved it.

Workshop Day

The advantage of local events is that no travel is required. The downside, however, that you have to commute to get there in the first place. This turned out quite tedious with public transport and its quirks, construction works, emergencies, and so on. Additionally, getting up very early compared to my usual working days meant I was already tired on arrival. And I was nervous! As with any new conference I'm at, I never know what awaits me and how I'll deal with that. It got better over time as I've been to tons of conferences. And yet, the anxious excitement keeps coming back, especially if I don't know anyone from the community yet who could offer a safe space. This time I got lucky as Claudius Link was coming. We met at SoCraTes, even gave sessions together this year, and we've just started planning an initiative for next year (stay tuned). In any case, it's really helpful to have a familiar face in the crowd!

On arriving, registration was smooth without any hassle, organizers and volunteers friendly and helpful, and I was positively surprised about the welcoming breakfast offered. In general, food was excellent and plenty throughout the day, including a variety to choose from for people having different needs and preferences. This makes an event already more inclusive and it's a detail I do pay attention to in order to gauge the overall spirit and atmosphere.

This first day was dedicated to workshops only. I noticed how few people were around. The workshop tickets were gone very quickly, yet there was lots of space. Super sad to see, especially considering the whole event being free. People reserving tickets yet not showing up meant they took away the opportunity from others who would have participated. Kudos to organizers who reminded folks frequently upfront to kindly give tickets back when realizing they couldn't come.

There were plenty of workshops to choose from, covering lots of interesting topics. For me, Claudia Ully's full-day workshop "The Hitchhacker's Guide to the Mobile Galaxy" was a clear winner as I want to dive deeper into mobile security and I can use the gained knowledge at work. I was not disappointed at all, this workshop was amazing! I loved how smooth the setup was, especially given that mobile has lots of requirements. It was awesome that while there is quite some theory needed to get everybody on a shared page to start from, the focus was on hands-on exercises. Claudia encouraged us to join forces, help each other and ask questions, which really made this a safe space to learn. The content was structured in a way that made it very accessible for people not having experience in the mobile space yet, while also providing lots of technical details valuable for people who came with prior knowledge. We went from mobile history and basics to Android specifics, static analysis, reverse engineering, to a discourse on iOS, to hooking into things with Frida and objection. And all that in the theme of Douglas Adam's "The Hitchhiker's Guide to the Galaxy"! Claudia even had a "42 - Don't Panic" towel with her, how cool is that? If you ever have the chance to catch one of her workshops, do it - fully recommended.

After such a full day of learning, I was pretty tired - and yet didn't want to miss the chance of socializing. Hence, I joined Claudius and a friend of his for drinks to conclude the day in great company.

Conference Day

The second day came, the main part of the conference with a program full of talks. And a whole lot more people! Same here, registration was quick, organization smooth, food was plenty and the venue a great choice, too. Lots of friendly and helpful organizers and volunteers around, and amazing speakers with a variety of topics to learn from.

The program consisted of two tracks which presented a difficult choice. Here's an overview of the talks I've picked.

• Keynote "The Seven Sins. And Virtues. Of IT Security. And how they affect our world." by Mario Heiderich. The conference theme was all around the 7 SYNs. What would the seven sins look like in cybersecurity, and what about the seven virtues? Mario's conclusion resonated with me: we cannot jump to the ideal state, yet we can take small steps and continue to learn.
  
• "(In)direct Syscalls: A journey from high to low" by Daniel Feichter. This talk dove right into the technicalities of Windows system calls and how red teamers can make use of them to bypass system controls. Packed full of details for a complex topic, this talk could only scratch the surface given the limited time. Daniel encouraged everyone to try it out and consume further material on the topic.
  
• "SOC Analyst’s Arsenal: Essential Tools, Tips and Tricks for Effective Investigations" by Samuel Kavaler. A talk full of hands-on advice and tool recommendations for the everyday work of a SOC Analyst. For people in different roles like me, it's been also interesting to learn which kinds of tools are used and for what reasons.
  
• "Bio-Lock The future and ethics around DNA Cryptography" by Tayla Sellschop. Cryptography is a whole topic in itself, yet what if we bring DNA into play? It offers a large storage space, while also not requiring as much computing power and hence power consumption, so it could become a sustainable solution in the future. On the other hand, there are a bunch of problems attached to using your own personal DNA - how would we feel about data breaches then? Yet as Tayla demonstrated, our DNA is already everywhere!
  
• "Secure containers - Do component reduction strategies fix your container security nightmares?" by Michael Wager and Michael Helwig. Really interesting overview of how we could tackle container security by using "distroless" images, only containing the application and its runtime dependencies without any other operating system programs. They are a lot more secure and less open to vulnerabilities, so why not make them the new default? At the same time, they also have disadvantages that might make them less attractive in their current state. Interesting topic to look into further.
  
• "Christmas Hancitor Campaign" by Artem Artemov. Loved this talk showcasing how proactive identification of threat actors and their victims can help prevent impact. Great storytelling of the investigation of a curious case and the actions taken to reveal more information until the puzzle pieces finally fell into their places and harm could be prevented. Incident response does not always have to happen in hindsight, it can start way earlier!
  
• "What We’ve Learned from Exposing Atlassian on the Internet: In-Depth Analysis from an Offensive Perspective" by Oleksandr Kazymyrov. A great story of "what would happen if..." and what you can learn from it to improve a system. Relevant for everyone having services publicly exposed to the internet behind SSO. Loved the testing mindset of always going a step further to identify what else can be accessed publicly and misused in an impactful way.
  
• "DevSecOps culture" by Ali Yazdani. This talk resonated a lot with me, from misconceptions shared to the cultural mindset shift required - I've seen this over and over again when working in testing and quality! Especially loved the emphasis on easing clear communication across roles as well as solving a problem together hands-on, no matter your role.
  
• "My CI/CD pipeline contains all security tools available! Now what...?" by Jasmin Mair. Another awesome talk where I just kept nodding! How many times have I heard some variation of "let's add some more tools" to solve a problem or satisfy a demand. Yet without the respective culture change nothing is solved just by having more tools. People need to learn the tooling, understand findings and figure out how to work towards a better outcome. Jasmin encouraged everyone to see it from a developer's perspective, being overwhelmed with hundreds of tools, each with their own interface and quirks, with every tool adding complexity and pain points. She made clear that proper tool evaluation and adoption is an investment and will take time, yet it's worth it.
  
• Keynote "Security by design" by Ana Oprea. The closing keynote draw a full circle to the opening one, also referring to the conference theme of 7 SYNS and how we can foster the virtues. Ana drew a connection between security and reliability and how designing for one of those aspects can help the other one and vice versa. I also liked that Ana emphasized risk assessment considerations and recommended techniques like threat modelling. She reminded us that people won't always realize they are a target or underestimate adversaries and their driving motivations.
  

By the way, slides can already be found on the website, and talk recordings will be published soon.

As I was taking sketchnotes, my biggest challenge was to switch rooms as there were often no breaks scheduled in between talks. It somehow worked out yet was more stressful than I hoped for, and it was strange to leave during questions, missing the answers. On the other hand, the breaks that had been scheduled worked out nicely. The program offered quick ones sufficient for bio breaks, and longer ones to digest what we've heard, refuel with nourishment, and connect with people.

The folks I talked with were really friendly and welcoming. Special thanks to Ben Wandel, Claudius Link, Sebastian Porst and Sergio A. Figueroa for our great lunch table! In general, I didn't notice much condescending behavior or being frowned upon due to aspects like my role or gender. I observed quite some diversity with this regard among the participants. Representation was even higher among the organizer and volunteer group, and it nicely showed in the conference concept and program.

Conclusion

This conference is driven by community and you can feel it. It was organized with care, ran smoothly, people appreciated the offer and seemed to have a good time. All this provided as a free event. Kudos to organizers and volunteers, thanks to sponsors for making this possible! 

I went home with my mind being full of all the things I've learned, my soul with all the new connections I've made, and my heart with the feeling that this is yet another place and community for me to become truly part of and belong to.

This definitely won't be my last BSides Munich and BSides event in general, I'm already looking forward to future ones. So, my first security conference was awesome - what are your recommendations for the next?

---

UPDATE: Fahri Korkmaz also wrote a blog post about BSides Munich. He shared lots of notes of talks I didn't attend, plus a lot more details on talks I did. Really worth checking it out and diving in deeper!

AskAppSec - Security Champions

The first time I heard about security champions programs was from Tanya Janca and the idea stuck with me ever since. If you haven't come across this concept yet, here are a few good resources on it.

• Building Security Champions by Tanya Janca
• Security Champions (OWASP)
• Security champions series by Snyk
• The Security Champion Framework by Chris Romeo, Izar Tarandach and Brook Schoenfield
• The Security Champions Podcast: Tanya Janca - A Recipe for Security Champions

For the first time, I'm in a company that not only has established a security champions program, it's also the first time I became a champion myself! Therefore, the topic grew even more relevant for me in the past months.

Recently, I came across several rather negative, or let's say frustrated viewpoints on security champions programs. People I met said it just never worked for them. Some shared they were not having real organizational buy-in and the program was merely a point on a checklist to tick off for the company to look good. Chris Romeo shared lots of security champion antipatterns in his Reasonable AppSec newsletter that made me think. "Why Security Champions Are Not the Silver Bullet" by Matthias Rohr is another thought-provoking piece pointing out that other initiatives might work better in certain contexts. What I don't hear much about in my bubble, though, are success stories from security champions programs. I do remember one person at Booster Conf talking about their program that managed to raise awareness and spread knowledge. Yet that's... basically it.

It's my first experience with such a program and I only see its current state after having run for quite some time. Therefore, I can't really tell how effective our program is and if it improved the situation compared to the one beforehand. From what I've observed, it does indeed seem to work quite well so far. It did manage to bring people together and scale the efforts of our InfoSec folks through having invested volunteers as contact persons and security advocates in each product development team, hence building bridges. There's clear guidance for lots of security topics and good practices. In the teams we have on demand support and feedback from InfoSec at any point from idea to production. At least from my personal perspective, collaboration works really well. We have buy-in and time set aside for security topics and can actively help drive security efforts for our products and the company. Huge shout-out to our awesome InfoSec folks at this point!

That being said, we recently also talked about how our program can be evolved. The conversation was initiated by an InfoSec person sharing Snyk's Security Champion Playbook and asking people for improvement ideas we could try. I did share my personal point of view of what I'm missing or what would help me benefit from the program even more. We're all working remotely and as of now asynchronously as security champions. It's not a secret how much I am a fan of synchronous collaboration, so that's what I would wish for more. Be it in the sense of regular calls with champions and InfoSec, or frequent pairing and ensembling sessions to work hands-on together. This could be on specific learning topics, general fun challenges like Capture the Flag (CTFs) sessions, on our regular security related tasks, or on solving current challenges in the teams - together. Joining the regular InfoSec call where folks exchange current news would be a great addition as well. 

We haven't decided yet what exactly to try out next. I'm curious what other ideas people have, what worked for them best so far and what not at all. More real experiences that we can all draw inspiration from. So, let me ask you: what makes security champions programs effective?

AskAppSec - Painless Usable Security

Imagine security being painless, easily usable and just the usual way we do things. Imagine this for both those who develop products and those who use these products. Wouldn't that be amazing? My optimism tells me it's possible, and yet we're often far from it.

In one way or another, I kept thinking about this for the last months. At SoCraTes and FroGS conf, I've facilitated sessions on the topic. We gathered lots of insights together with participants, hearing what struggles they faced and what opportunities they found. What we can do to change the narrative. Many thanks to everyone who contributed! Here are the points that came up repeatedly when asked what's painful.

• Fear. It's scary to ask questions, especially about security. Security teams (if you even have them) might be very detached from teams' everyday realities and not approachable, might even be condescending, or just wave around a policy without being helpful. There's a lot of secrecy and gatekeeping going on as well. And what if we make a mistake and people blame us? What if I see something yet have every incentive encouraging me to look the other way instead of reporting it?
• Fatigue. So many alerts, we're overwhelmed already. So many false positives reported by tools, which makes it even easier to ignore yet another scanner result and just bypass it so we can move forward, as we're pressured to do. Security theater is huge at play here as well, everyone talking how important security is without ever seeing real actions taken. Why not just tick those boxes in the easiest way so we can say we comply without actually fulfilling the spirit behind regulations.
• Future. Well, security is indeed a future problem, isn't it? Yeah, that risk exists, yet will it really ever happen? We'll rather cross the bridge when we come to it. We have so many other things to do after all. And as we can't invest in prevention now, let's put security last by default. Hence, we can ignore issues we see, as no real pain is perceived - until suddenly the pain is super high.
• Friction. I know this is the more secure way, yet I have to jump through ten hoops, get approval from hundred people and then sign this contract with my blood - or... I just do this one-liner change. Procedural problems are real. Poor experience is real. Difficult cross-team collaboration and dependencies are real. And they have very real impact on behavior. If something is way too much effort for what it's worth, we're usually not going this extra mile (or at least aren't rewarded for it).
• Futility. Security is just such a huge area, security work is never finished, we'll never know everything. The system is so complex. We lack knowledge and we have so much else to know already. We struggle to see the actual impact of vulnerabilities anyways. We can only know the system is insecure, not the other way around. All this feels really futile, so why invest at all.

This list resonates with me a lot, and I see these points in my own work context as well. Especially when there's a whole backlog of things that we know we need to improve, yet struggle with balancing competing priorities. Fatigue is a real challenge indeed, like fatigue of pointing out problems and proposing solutions that just don't cut the list of most valuable things to do right now.

I've also talked with several people in the communities I'm in, where security was sometimes perceived as painful due to other reasons. Like, why is security the only quality aspect that is considered and gets buy-in, what about all the other ways in which we can harm our users, our own people, our product and company? Why do we get external experts for security yet not for other topics (like accessibility)? Why do security policies just always make things harder? What is it that security slows us down while not achieving actually more secure outcomes?

Finally, there's the angle from friends and family not working in tech. Security? Well, that's often perceived as the thing that annoys you, that you skip. Oh my, another update, why do people have to change things all the time. Oh no, another factor to log in, why does everybody need to do this nowadays. My goodness, another popup to click away so I can do my job and go on with my life. I heard a lot of statements like this, usually accompanied with frustration and anger. Or with shrugging things off. I don't care if they have my data, what would they do with it anyways. Yeah, I know this company has proven to do bad things and yet they offer the best usable solution compared to more secure competitors.

So how can we reduce the pain and friction, increase usability and make security the easy route to take? In all my conversations, the following points came up repeatedly.

• Ease development experience. Anything that makes security easier and reduces friction and cognitive load from the start can help. Include thinking exercises like having evil personas you could use for user stories, and doing threat modelling to raise awareness before designing and building solutions. Provide good code examples. Have secure defaults, in frameworks, infrastructure, your own product. Keep things in shape and up to date. Enable folks to deliver fast and well, so we can respond fast and well to new threats. Planning for mistakes (that will inevitably happen) and recovery, and foster a culture where postmortems are considered an invaluable learning opportunity.
• Collaborate with security experts early on. No ivory towers, no jargon being thrown around. Instead, security folks being approachable and helpful, enabling team after team, pairing and ensembling hands-on. Security champion programs that actually build bridges and help scale good practices. Do this early on and continuously to make use of the best leverage. Then consider getting external persons to point out problems and receive advice, be it in the form of consultants, audits, bug bounty programs, or coordinated disclosure. 
• Be clear about risks to help prioritization. Not only do we need to assess risks, risks can also have vastly different impact depending on our specific context. Terrible consequences in one might be reasonably ignorable in another context. Learning what's most relevant in yours, and probing the risk appetite of the organization helps figure out priorities.

One thing that stuck with me is what both security and UX folks repeat over and over: security that's not usable is not security. Just as Jared Spool points out, "If it’s not usable, it’s not secure." Because people simply won't do it or find their way around. They have a task to accomplish, a goal to achieve, a job to do. If security blocks them instead of supports them, it might as well not be there in the first place.

• Insecure & Unintuitive: How We Need to Fix the UX of Security by Jared Spool
• Secure Design Concepts w/ Tanya Janca
• Security and usability: you CAN have it all!
• How good UX leads to great security by Josh Ben-David

The same applies to development teams. If practices leading to more security aren't usable, or don't fit in our everyday lives, they simply won't happen. We have to find ways to make it the easy and frictionless route, anything else is simply not sustainable.

This whole topic reminds me again of testing and quality, as so many things do in security. It's a lot about culture, it's a lot about advocacy and change. In the end it's about people. I'm wondering now: the team transformation tactics I found to help move towards holistic testing and quality, could I try them out for moving towards painless usable security as well? I probably should give it a try. Now that I'm thinking about it, I realize I literally just did apply them the last weeks. And before as well.

Let me give an example. One of our current focus topics is keeping our dependencies up to date. Updating them is one part, yet having a team reliably keep doing so is a whole different story. What I did was building on existing energies; in this case, building on existing practices that already worked for the team. And just last week for the first time it worked out well enough. People felt responsible and updated dependencies on their own without my nudging. It was clear, it was easy, it was part of everyday work. It didn't cause friction. Okay granted, I'll have to observe and evaluate this experiment further, yet on first glance it does look like less friction than before.

When it comes to security improvements for our product, I believe we need to work a lot more and a lot closer with UX folks and product designers. This expertise is invaluable and yet too often underrated. The resources listed above give lots of pointers why.

That leaves me wondering: why not also work together with UX and design to find more painless, usable, secure ways to build more painless, usable, secure solutions? There's a lot more for me to think about and try out.

I'm sure people made their own experience with this intersection of security and usability as well as respective pain points, be it for their team, organization, product or just generally in life. Therefore, let me ask you all: what's your approach to move towards painless, usable security?

---

UPDATE: This post didn't receive much response from the community yet. Really appreciated this person taking the time to share their thoughts and experiences!

AskAppSec - Input Validation

Input validation is a topic that's been following me around for years. I've came across countless resources speaking about the importance of input validation, or input filtering as it's called at times. What stuck with me is the recommendation to validate any input coming from any source, no matter if we're speaking about third parties, public interfaces we offer ourselves, internal services behind a firewall or accessible only from inside a private cluster. No matter if the input comes via clients, APIs, messages, data sources, or anything else. Based on my experience of working in testing and quality focused roles for over 14 years, I couldn't agree more. All that makes a lot of sense to me. Not only from a security point of view, yet also from a holistic quality perspective as input validation can help prevent errors, improve usability, increase observability, and more.

Here are a few interesting resources speaking about input validation from a security standpoint.

• Tanya Janca's book "Alice and Bob Learn Application Security" includes a great decision tree for input validation.
• OWASP's Input Validation Cheat Sheet offers a good introduction including examples.
• Kevin Smith's post on "Sanitize Your Inputs?" includes not only advice on input validation yet also touches on other topics to consider like sanitizing and escaping outputs.
• Cenk Kalpakoğlu's post on Defensive Programming Tips-1: Bad URL Handling Patterns showcases a specific example where validation is used as general mitigation approach.

Well. Seems convincing people to validate input is also a common challenge in AppSec; it definitely is in my bubble advocating for better quality outcomes. Most frequently I've seen these discussions when working with backend for frontend (BFF) services. This architectural pattern is often applied when you develop mobile applications, yet not limited to it. It is usually found along with having a bunch of downstream services that are all tasked with different duties. The BFF acts as the main entry point or proxy for a single client (e.g., the mobile app), hence only this API is public to the outside world. Any requests are routed through the BFF to the respective downstream backend services (which are usually protected further). Doing so, the BFF orchestrates incoming requests to different services; it can take care of authentication and authorization, as well as filter and aggregate data in order to respond with the needed information.

If you'd like to learn more about BFFs and see visual or code examples, I found the following resources useful.

• Pattern: Backends For Frontends by Sam Newman
• BFF @ SoundCloud by Lukasz Plotnicki
• BFF Pattern — Dos and Don’ts by Viduni Wickramarachchi
• Backend-for-Frontend: the demo by Nicolas Fränkel
• A Deep Dive into the Back-End for Front-End Pattern by Joydip Kanjilal

What about input validation for BFFs now? What I've heard frequently from colleagues can be summarized in the following statements.

• "The BFF is just an API gateway."
• "The BFF should not contain any logic, just pass through anything it receives to the backend services and vice versa."
• "Only the downstream backend service behind the BFF should validate input as it's their responsibility, otherwise we replicate the same logic everywhere."
• "Well, it's okay for the BFF to do sanitization, yet not validation."

Here's my viewpoint, and I'm very curious to hear further opinions.

Any modular component of our system needs to sanitize and validate input coming from outside in order to prevent falling into an unknown state. This is both causing poor user experience as well as presents an attractive situation for malicious actors looking for further insights that can be used for exploits. Components include frontend clients in favor of usability, even though malicious actors can easily circumvent them. As long as there is an interface accepting data from another source, there should be validation. Under that premise, I do think that also BFFs should validate incoming data, especially from the public facing side, yet also from internal backend services or other data sources. The BFF is one of the first layers of defense we have, hence if input is validated, we leave the door less wide open. We cannot rely on underlying backend services having foreseen anything that could enter the system from the outside and having sufficient mitigations in place; especially if they are developed by other teams in other contexts who might not even realize the impact of their local decisions. Also, lots of people tend to underestimate threats coming from within the company, like malicious insiders or mere human mistakes. I do understand that it's not always pragmatic or feasible to validate input on all boundaries. Yet if we have to decide, I choose validation at trust boundaries, like between the BFF as public interface and the outside world, as a minimum.

That being said: context is crucial, as always. Maybe the policy to only validate on the most downstream service works well in your situation. Maybe this is considered way too dangerous as you might be aware that specific services are not in good shape. Or maybe your product domain's nature means you're dealing with lots of confidential, sensitive data and you are more invested in keeping people out right at the door (aka the BFF API) without letting them any farther in. As usual, it depends on the risk appetite of the company, combined with your own ethics of what potential impact and harm you deem acceptable or not.

One thing I learned over and over again in my career is that arguments might convince rationally, yet they often don't reach people in a way that they change their behavior. They usually need to experience it, and usually a few times (and I'm not excluding myself in this equation). The trouble with security and similar quality aspects: I want to prevent the experience of harmful impact as much as possible. Speaking of the topic of whether it makes sense to validate input also for BFFs. I could of course invest in exploiting lack of validation, or showcasing a close to real situation, yet it's effort that still does not easily change the narrative and then behavior. If you have any further idea or tactic for these kinds of situations, your input is appreciated.

All in all, I do have a strong opinion on this topic, yet I hold it loosely enough to allow myself to be convinced by better ones. I did find posts that support my standpoint, like "Web App Security: Understanding The Meaning Of The BFF Pattern" by Syed Wahaj - yet that might be pure confirmation bias. So, I'd sincerely love to hear your thoughts and experience about this and learn more: should BFFs validate input?

---

UPDATE: I've shared this question with the wider community and received some validating feedback. My appreciation to everyone who offered their thoughts!

Great post, Lisi, as always. And I agree, every component should do at least some minimum of input validation to protect itself and the following systems. It adds to stability, observability, user experience, and more. I need to check with my team, when I’m back.

— Patrick Prill @testpappy.bsky.social (@TestPappy) September 13, 2023

What I found especially insightful was the following input from a We Hack Purple Community member, hence sharing it further with their permission so it can help more folks besides me. I think they nailed it, so I mostly maintained the original take with slight format editing from my side.

I think the answer to "Should BFFs validate input?" really depends on what it does with the data. The BFF will need to validate some input, but not necessarily all of it.

Generally, anything that looks at the data, parses it or interprets the data in any way will have to validate input.

A BFF will likely look at the HTTP request headers, so it has to validate those. It cannot assume that the request headers will be sensible, or not malicious. It may also have to decide how to deal with duplicate request headers, etc.

But maybe the BFF does not look at the request bodies, and just passes them through to the backend.

It probably would not make sense to duplicate lots of application logic in the BFF to perform application specific input validation on data the BFF itself does not process. Unless, maybe there are very common things that may make sense for a BFF to filter out before bothering the backend with it. But that would then be a bit more like a WAF that may do some general input validation, like looking for common SQL injection patterns. And this still does not absolve the backend from its input validation responsibilities.

The backend services will always have to validate the input they are handling, but even backend services may pass some data through to other downstream backend services. The important thing is that everything that looks at the data and processes it performs validation, whether that is a web server, an API gateway, a web API, or a BFF.

My deepest thanks go out to the person who took the time and energy to elaborate on this. They made the distinction I was looking for (without knowing I was): what exactly makes sense to validate where and why, given the specific context at hand. I think this is what I struggled with myself and hence struggled to convey more clearly to colleagues. Taking this explicit distinction, I feel enabled to map it to our context to make better informed decisions, and I also feel equipped to bring more clarity to the next conversation on input validation!

As a bonus, here's one more thoughtful response allowing us to weigh further aspects against each other.

SoCraTes 2023 - A Place Where I Belong

I nearly didn't go to SoCraTes this year, the "International Conference for Software Craft and Testing". My speaking budget was already strained, my schedule overbooked, and it would have meant going on vacation time. But then the organizers reached out and offered me yet another slot on the training day this year. They were even fine with me giving a workshop I already had prepared, and I changed my mind. I seized the opportunity and went to SoCraTes on vacation. What can I say, I don't regret it one bit! Granted, I'm super tired, and at the same time I'm super happy. It was so much worth it and convinced me to reserve this time of the year for 2024 as well!

Coming Back

It's always super exciting to join a conference for the first time. The second time around, a few things are already clear - you know the venue, more people, the procedures and other things that reduce your cognitive load. Still, the second time is curious as well - how will they welcome me this time? How easily can I reconnect to where we left things last year?

The first moments are awkward for me, at any conference. At SoCraTes, this very quickly vanished into a feeling of belonging. I felt welcomed, I was included, I had a right to be there. This is the foundation for everything coming afterwards.

My first contact after seeing familiar faces at registration was a person being there for the first time, Lydia Leifels - such a pleasure right from the start! At dinner, I met dear people I already know for a while, like Tobias Göschel, Thierry de Pauw (along with their daughter), Juke Trabold, and Woody Zuill. There were so many awesome folks I got to know or meet again over the course of the conference! Like Janina Nemec, Marc Kalmes, Stefan Scheidt, Claudius Link, Lea Rosema, Waldemar Tomme, Martin Schmidt, Markus Tacker - well, the list could go on and on.

Training Day

The second edition of the training day was even better than the first one. Trainers were amazing both times, yet for this year organizers listened well to the feedback and crafted a schedule of three tracks with each session having enough time to dive into the topic and generous breaks in between. Just awesome. Here's the choice of sessions I made.

• "Enforcing Architecture Using Tests" by Javiera Laso. This was the first time I heard about ArchUnit, a library to check for conventions like file names, structure, dependencies and more.  Writing tests was quite straightforward, and I can see how these could support maintainability and reduce friction by codifying agreements.
• "Modernize CI/CD Session" by Raimo Radczewski & Chris Neuroth. Very interesting talk sharing fundamental principles for a pipeline optimized for quick feedback while going small, safe steps. They demonstrated live how fast a change can be on production. A few statements I really related to were these: "minimize the time from code written to code on mainline, deployed to real users, running in a real environment, ready to be evaluated - there is no other way to develop software sustainably", "influence the loops you can", "the moment we stop slicing we deliver slower", "skip the review, pair with someone".
• "Ensemble Exploratory Testing" by me. Fun fact, I've given this workshop now for the 10th time so it became my most repeated conference session ever so far. Good thing is, it seems it doesn't get old! This time again, people were eager to join and seemed to enjoy the learning experience (which hopefully convinced them to try new approaches back at work). Well, I had fun observing people having lots of aha moments together.
  
• "Take a mess, make a mess, fix the mess" by Aki Salmi. A very interesting session on refactoring code that's untestable and contains hidden domain concepts, code that's still valuable yet needs to be modified. How? Together, we tried an approach many of us had not seen before: not even trying to understand the code in the first place. Instead, using the IDE's automated refactoring tools to slice it up first, turning hard to test code into easy to test code. Then we can document its behavior in tests, and hence gain our safety net to make the required changes. Check it out, you can follow every small commit Aki made. Nicolas Carlo also wrote a great post on this where you can follow an example: "Another way of refactoring untested code" (by the way, his newsletter is super insightful and a clear recommendation).

After the training day ended, the main part of the conference was opened with a world café. Find a group of people you don't know, take a question as starting point, see where the conversation leads you and doodle your insights on a shared canvas. After a given time box is over, everyone find themselves new groups besides one person staying at the table and getting the newcomers on the same page. Repeat until you finished three rounds. Really nice exercise, perfect to get to know first people ahead of the main conference part and have interesting conversations emerge. One main theme we had was on people, community, culture and how that's foundational. In case you're wondering where to start, I have a resource collection on inclusion that really helped me.

That's not the end of the day, of course - SoCraTes goes all in! Everyone is at the same place, we're together the whole time, so of course there's an evening / anytime during the night / morning schedule with bonus activities suggested by anyone. Such a good thing our hotel rooms are right there to retreat and rest any time. I love how Juke as facilitator and also organizers continuously emphasized the importance of caring for your needs and taking breaks.

Having learned from last year that joining all sessions can be quickly very exhausting, I decided to use the evening time for conversations and enjoying both atmosphere and company of wonderful people.

Open Space Day 1

Days start early at SoCraTes for me being a night owl, yet it's worth it. In its core it's an unconference offering an open space for everyone to bring their topics. Things they want to share, conversations they'd like to have, apps they want to build, tools to try out, skills to practice - and also challenges they'd like to get help on. With so many different people, there's a whole range of topics offered, covering a spectrum of deep dive tech topics to humans and culture as foundation for everything. No matter if it's very personal, related with our professions, challenges in society, or all combined, everything is represented. There are also sessions like doodling together, painting your nails, talking about sheep, whatever is most valuable to people right now. Be prepared to be surprised! This format of building your own schedule together on the fly works amazingly well. It definitely worked out well for me again this year.

• "Web accessibility - building beautiful web sites that don't make you puke" by njan Völker and Lina Sievering. I had hoped to learn more about accessibility at the conference, and already the first session was right on spot! Awesome workshop focused all around motion sickness induced by websites and apps. A topic close to my heart as I'm affected myself. Njan and Lina provided mindful and enlightening exercises to convey the problem and think of more accessible options together. Thanks to them, I also learned about the European Accessibility Act becoming effective as of 28 June 2025 - which means accessibility will be enforceable in Europe which hopefully gives us more leverage for accessible solutions when making product decisions.
  
• "Capture the flag together" by me. As part of my personal AskAppSec challenge, I recently tried out further services offering hacking labs like TryHackMe and Hack The Box to practice penetration testing. The first challenges were good fun to me, so I thought why not offer a session and do it together during the open space. I was positively surprised how many people came and joined me! I opted for Hack The Box and their starting point machines. It worked super well, people were engaged, shared lots of knowledge and we captured a flag together (the second we missed only due to my VPN interfering). A very insightful experience, validating my assumption that there's interest and these could be great sessions for more people to learn about security in a fun way.
• "Security for devs (& everyone)" by Claudius Link and me. The more security sessions the better, right? So why not host another one, together with Claudius who had the same idea. We had a great conversation with people bringing up all kinds of insights and common challenges. Nothing was immediately new for me, and yet it was validating to hear experienced people share similar views.
  
• "Documentation as Code" by Markus Decke. We work together at the same company, and we are transitioning to having more and more documentation as code, so I was curious about other people's ideas, struggles and in general experiences. We talked about benefits and use cases, shared tooling options and their limitations. One of the main themes was around what problem we're trying to solve and then aim for tackling exactly that, nothing else not to document only for the sake of documentation.

What about the evening? Well, after open space is before open space! As mentioned, there's always an evening schedule people are building up. Once more, I opted for dinner, conversations and more fun time! Last year I discovered that Janina Nemec is an absolutely pro in playing Set, a card deck game I've learned to love through the testing community. At first we didn't spot any deck so decided to opt for a round of Exploding Kittens (so much fun) - and finally discovered a Set deck in the end. Well, next year we're prepared to bring our own decks. Such a good way to close the day.

Open Space Day 2

New day, new schedule. Every day starts a tiny bit later, which is still early for me. And yet it was awesome again and worth the early start.

• "Stop being a superhero!" by Janina Nemec. She did an early dry run of her upcoming talk to be presented at Agile Testing Days this year - which is your chance to catch this talk, you'll be in for a treat! Janina has vast experience of working in an ensemble full time for many years. In her talk she describes (superhero) behavioral patterns she's observed (like the architecture wise or the coding wizard - way too relatable) and the pain points that result from them (like lots of work in progress without things getting done, or the behavior not helping the team grow and work sustainably). There's a solution for this: working together as an ensemble, or team programming as she calls it - saving the world together.
• "Build a minimal showcase app" by me. I have a recurring argument around a security topic and wanted to finally start building a minimal app to demonstrate good security practices. So what could be better than get this started together right at the conference? Nonetheless, I nearly didn't dare to suggest the session. And then I thought no one shows up (I didn't realize it was break time). Until people did show up and we formed a wonderful little ensemble helping me get started on a good way. Everyone else enjoyed setting things up and realizing that we all struggle in certain areas. When doing it together, however, we usually have the missing piece of knowledge in the round to avoid friction, solve problems without frustration, learn with fun, and get to value fast. A pleasant experience, would have loved to continue together.
• "Security scanning in pipeline" by Raimo Radczewski & Chris Neuroth. Both wanted to try out security scanning tools like Trivy, Syft and Grype on a real case example and see how they work and what value they bring. A really interesting session that then also sparked a serendipitous hallway conversation on why attack trees might work better compared to threat models in order to get people to think like malicious actors and consider risk.
• "Security games" by Claudius Link. He brought a whole bunch of games to teach security in a safe space with fun. We could try out a few of them and gain experience how they could be used for educational purposes. Games like Elevation of privilege and OWASP Cornucopia, yet also [d0x3d!] and a lot more I can't remember. Now I know there are a lot more out there to try out!

This was the last day for the main part of the conference, so we closed it with a retrospective. The best part here was that it didn't merely gather feedback for organizers and Juke as facilitator, it also was  intended to provide feedback for each other as participants. Over half of my group were here for the first time, and I just loved hearing their feedback and also input to make it even better for each other.

A gratitude round followed. The challenge was to thank five people we have not thanked yet. Honestly, this experience was a bit overwhelming, in all the best ways - and more than one feedback took me by surprise. I haven't mentioned yet, this conference goes big on hand-written kudos cards you can hand out any time for anything you appreciated the other person doing. The ones I received I will keep with me for long, they present such a dear memory.

Dinner followed with more great conversations. Then further sessions were hosted (remember, the evening schedule). I had a great time joining a code kata ensemble. We did the "Vending Machine Kata" which was an insightful exercise itself, yet my main takeaways were on the collaboration part. It was fascinating to see an ensemble start off quite free style and converge to more structure like having a dedicated navigator, using a timer for rotations, having the navigator stand up to be more prominent, etc. It just worked better with folks who have never worked with each other in this way. Also, we used the fish-bowl approach to ensembling, having outside observers and always open spots to join the ensemble. This exercise made me realize once again that this approach is simply not for me, despite having super kind and safe people around me it felt exclusive. I'm very sure it's actually more inclusive for other people to opt in. For me I much prefer the "all together just one ensemble without observers" format.

Last but not least, a retro gaming session! We played the old adventure game "Zak McKracken and the Alien Mindbenders" all together on a C64 system provided by Tobias Göschel - how awesome was that?

Workshop Day

Have I said the conference is over? There's still the additional workshop day! And what better format to have than asking people to bring their topics for hands-on sessions also on this day. It's also the time of the traditional code retreat, practicing the whole day solving the same kata in various ways. Originally, I was adamant to join the code retreat again, it was simply an amazing experience last year. And then Claudius Link came and suggested the only thing that could possibly lure me away: co-facilitating a security workshop. Well, what shall I say? I couldn't resist, this fit way too perfectly to my personal AskAppSec challenge this year.

We aligned on our main thoughts of what to aim for, pitched it to a fellow participant, incorporated the feedback, and came up with a workshop on "Painless Security". We crafted the agenda shortly before and went ahead with it, playing it by ear and experience in giving workshops. Having co-hosted a session the other day helped, too. Co-facilitation worked super well together, we often thought along the same lines and built on each others ideas. People shared many painful experiences and we gathered potential things to try out and what we related with most to bring back to work. I took a lot with me myself.

During lunch time, the security theme continued for me. First thinking about security conferences and ideas to contribute to the space together with Claudius Link and Susanne Neunes. Heading back towards the conference rooms, I noticed another table where Martin Schmidt and Philipp Zug, who also both participated in our security workshop, were scheming on a new security card deck. I loved the idea and they were so kind to invite me in. Well, it seems I got myself involved in two new topics - and yet I feel these are very much worth it.

I decided to check out what other sessions might still run that I could join. First, coffee though - and that's when Matthias Klass approached me and asked whether we might have another capture the flag session together, as a couple of people expressed their interest. Well, I didn't need to check the schedule anymore, security theme it was for the whole day! Of course I'll host another capture the flag session. What can I say, it was awesome! We spent the whole time until we officially had to leave the room and head for dinner. Nearly instantly, the idea popped up whether we could ask the hotel for opening the room once more during the evening. Asking was worth it, we got the keys and went all in after dinner. Many more rounds of capturing flags followed until everyone was so tired we couldn't think anymore - while being just super happy. Some of the challenges tackled were very familiar to us and hence solved a lot faster, others required us to piece together knowledge we usually don't need. All of them were very insightful and fun. So much fun. I loved that we all worked together so effectively as a big ensemble again. I really want to do more of this.

There's a traditional count of code katas done at SoCraTes, counting each exercise by everyone. I was really moved seeing so many people join me on practicing penetration testing, staying with me for so long and sharing my enthusiasm. It felt we just established a new counter at SoCraTes next to the code kata counter: the captured flags counter, aka the number of security secrets discovered. We collectively increased it to 8 overall! 

Why again next year?

Heading home, my heart was full, my brain energized, my body tired, and me super happy. I was certain that if I have any chance to be back next year I will take it.

Besides the obvious reasons of self-selected, very insightful content and just amazingly kind and inspiring people, there's a reason that is even more important. This conference is the best I've seen so far in intentionally designing welcoming and inclusive spaces. They continue to reduce friction and make it more accessible and safer to more and more people. Literally every year. There are lots of aspects of this to be found everywhere. Not only in the code of conduct that's actually being lived and enforced, or offering all gender toilets, or giving away tickets based on a lottery to level the playing field. It's in every little detail. Less noisy applause by waving hands. Exact food labelling and plenty of options for everyone. Child care service so lots of parents could join this year. Sharing Covid tests upfront for every day and encouraging masks.

The level of diversity already achieved has a huge positive impact on the quality of conversations and insights gained. A lot to learn and take with me to do better myself.