Sunday, August 18, 2019

#CodeConfident: Journey - Part 2

My Journey project made some progress! It's still far from where I'd like it to be, yet here's another part of my coding journal - including three more pairing sessions and a brief detour to my former Rest Practice project. Enjoy! :)

July 14

  • pairing session with Parveen Khan
  • when preparing for my pairing session, I removed all built directories and have the project build from scratch - which indeed proved my assumption that this would fix the profile service! :D
  • in the beginning I told her about the background of the app, did a short run through of the app, explained what had been generated and what I added or changed, and what are potential next things to do
  • Parveen wanted to explore the app and then do first changes together
  • when Parveen went through the app, we noticed several things, she raised lots of great questions and she triggered several ideas what to do next
    • TODO: the create / update challenge dialog showed the "influences" label in lowercase although it should be uppercase
    • TODO: the journal entry index page should sort entries by default by date in a descending way with the latest one on top (right now sorted by creation date)
    • TODO: for the index pages, the pagination thresholds should be defined, e.g. only showing 10 items per page
    • IDEA: Parveen asked me whether I wanted to use the app to instantly publish journal entries to my blog - and I thought of that indeed! Or at least to have a quick copy button for the description field so I could easily copy it over to my blog posts and make my own life easier
    • TODO: showing the challenge id for journal entries is not really useful, should rather show the tag instead (had that one on my list, yet it's great to hear confirmation from a different person that this should change)
    • TODO: the challenge delete dialog should not only ask for confirmation of the deletion but also inform what will happen with potential related journal entries
    • TODO: when trying to delete a challenge that still has journal entries, an unhelpful error 500 is thrown and the error message is not useful at all
  • then we went on with making small changes to the app and seeing their impact
  • for the challenge index view, we added a column for the hypothesis
  • for the journal entry index view, we wanted to do the same, yet decided not to dive deeper here for now and solve that puzzle yet
  • instead we changed the challenge detail view to not show the challenge id in the title yet the tag instead, and removed the tag from the rest of the detail page
  • IDEA: we found it would be great to see related journal entries also in the challenge detail view
  • Reflecting:
    • Lisi: having a practice project offers a great and safe learning environment to get your toes wet
    • Parveen: you need to have some idea of what app you want, then you start with small steps, then you get used to it, and you keep trying
    • Parveen: having me as trusted navigator gave her the confidence to do these changes herself; also, I gave her the confidence by telling her "it's fine, don't worry", "we can just delete it, we have it under version control" and letting her do it on her own and try things out; Lisi: it was super useful to practice explaining things I learned (some of them recently), especially in a simple way, really helps my own learning; this is also something such safe learning environments are perfect for
    • Parveen: was super great to both explore an app and make changes within just one hour! Now she understands why developers get into the zone, even forgetting to eat, doing things step by step; all this really helps to get into their shoes and create empathy
    • Lisi: was really cool, got a lot of ideas out of the session :D

July 14 (cont)


July 21


July 24


July 25


July 27


July 28

  • looking at examples from work
  • tried more things
  • really need to read more deeper first
  • nothing committed yet

August 11

  • pairing with Shivani Gaba
  • We planned to set out to pair on my Rest Practice project for which Shivani had feedback to share based on her experience with automated API tests.
  • Shortly before the call I realized that the framework used in this project was relying on Java 8 which I recently had removed from my computer so we could not run the tests. After spending some time back and forth how to install OpenJDK 8 so I would not need the Oracle version, we decided to go through the feedback first. (Later I found this site where I could download a pre-built version of OpenJDK 8 for Windows: https://adoptopenjdk.net/)
  • Instead of always using an explicit status code (e.g. 200), Shivani recommended to use predefined response status types so that the test becomes more readable and less error-prone.
    https://docs.oracle.com/javaee/7/api/javax/ws/rs/core/Response.Status.html
    https://www.programcreek.com/java-api-examples/javax.ws.rs.core.Response.Status
  • In the current tests I asserted for certain body properties. Shivani recommended to implement a JSON helper instead which would provide utility methods like comparing the actual response to an expected JSON. This comes in handy especially if you have a lot more fields to test for. Also, this helper could offer generic convenience methods for creating a JSON object with certain parameters, adding and removing properties, or merging JSONs. Shivani offered to provide a sample - thank you!!
  • Shivani recommended to store the complete response in a variable from which we can then extract headers, body, code whatever we need/want to use for each test only what's needed for the specific test.
  • All endpoints were repeated in each test. Shivani would rather extract them as global enums (e.g. something like "Employee.GET_ID"), or at least declare them as variable, and re-use them, which would also increase readability and maintainability. If they change, you only need to change them in one place.
  • One open question in the practice project was how to get a response without given/when/then when it's only about the setup. Shivani shared how she would wrap the setup in a custom method (e.g. a post method) to hide this implementation details from the rest of the test, which makes it more readable and more maintainable as well.
  • At this point we decided to switch to my Journey project and have a look at how integration tests are done there. I explained the background and purpose of the project, what JHipster already brought with it, and that most of the tests got auto-generated and need improvement.
  • We took the test for creating a journal entry as an example to start with. I shared that what surprised me with the generated tests was that they check the database directly instead of calling the API again to verify if the record really got created. This might be good as it's about testing the integration with the database, yet I was not so sure about depending on the database state here. Shivani did not work with Spring and its test context yet, yet this surprised her as well.
  • Shivani shared that oftentimes integration tests are only used for the happy path, yet they are perfect for negative testing as well. For example, what if a mandatory field is not there? How do we handle that? We decided to implement a test for that to learn how it's done with the Spring framework. We discovered we could simply set a mandatory field to null before creating the DTO out of the entity which led to a bad request we could assert for.
  • We learned more about how the Spring framework builds requests and uses matchers for assertions.
  • We did not yet find a way how to provide custom error messages this way, like we could do for AssertThat().
  • We found a more readable and re-usable way how to write the tests, extracting the expected response and the builder which made the test more lean. We could do this globally to reduce duplication.
    https://www.logicbig.com/how-to/code-snippets/jcode-spring-mvc-junit-test.html
  • Reflection:
    • I was really sorry for the rough start and the delay before we could work hands-on together. I was not as prepared for the session as I wanted to be, had a bad day. In the session I gained a lot new insights and good practices how to improve readability and maintainability of API tests. Thanks so much for sharing your experience Shivani!
    • Shivani shared she was happy to pair with me. It got her really motivated to work on her own project idea now; something she had in mind for longer yet did not start yet :)

August 15

  • Pairing session with Gem Hill on unit tests
  • Gem is starting her own testing tour and I felt super honored to be the first stop on her tour! :D For more details see https://letstalkabouttests.xyz/index.php/testing-tour/
  • We started with me explaining the background of my Journey project, what it's supposed to do and also the code generation part, showing Gem what's there.
  • We wanted to dive deeper into unit tests, and we had quite some generated tests to look into to understand them better and also critically question them.
  • We decided to dive into the frontend unit tests using Jest as Gem's team is now also working with TypeScript and Jest, so this was a perfect match; and also for me it was great as currently I am working on frontend unit tests as well at work in a similar setup.
  • I shortly introduced Jest and it's super useful watch feature where it runs all time in the background, running tests for changed files to quickly provide feedback while developing (my colleagues love that feature).
  • We decided to look at the test for the challenge detail component first as most simple starting point.
  • We went through the test from top til bottom, trying to formulate our understanding, making assumptions clear, voicing questions and parts we don't understand yet.
  • Then we had a look at the actual implementation to map it to what we saw in the test.
  • We noticed that the tested ngOnInit function only subscribed to the challenge, yet the subscription was not tested. This part is basic functionality provided from the framework - which is normally not what we want to test. So it's always the question where do we trust the framework implementation?
  • We had a closer look at the mocking part. Interesting to dive deeper into would be the ComponentFixture class which is provided by Angular as fixture for testing and debugging components. This is something we're not using at work so I'm curious to find out more.
  • The test is importing a test module which comes with a lot of mocks for services.
  • When looking at the test module we came across the same number used for ids again, "123" - it seems the JHipster developers used the same number for all kinds of ids. To make sure our test is independent we changed the id for the used challenge and the tests promptly failed. Here we could see how Jest points out differences. I am also using the Jest plugin for Visual Studio Code which even shows the error inline. That's a nice feature, yet for me it's easier and quicker to see what went wrong in the console output.
  • We decided to move on to another test and chose the assumed next smallest one, the one for the delete component.
  • Here we came across the fakeAsync() and tick() methods which caught our interest. Once again we enjoyed the Visual Studio Code feature to hover over a method to get its description :) We learned that tick() simulates the asynchronous passage of time.
  • We discussed spies which allow to mock return values of functions to stay independent from other parts of the app.
  • Gem suggested to interact with the locally running app and see what's actually happening. This triggered the idea that I should install the Redux DevTools so we could inspect even closer. For now, we deleted a challenge and checked the respective requests sent. This way we saw that opening the delete dialog sends a GET request for the respective challenge - why? We had a look at the implementation and saw the defined selector. Was this the reason? We checked all references yet it was only defined here. We could not yet clearly see how things connect here, something to find out later.
  • We had a look at the implementation for deletion and found a method to close the dialog on cancel. Once again we asked ourselves, is this worth writing a test for? Is it worth the effort of maintaining such a test? This should normally be code that's not touched very often so it's probably not worth it.
  • We discussed mutation testing, an approach my team is currently trying out and Gem used quite often in the past. This is a great way to see if our tests are actually valuable and would detect issues and where we are missing tests. In my team we plan to use this as better guideline for what to write tests for than mere coverage which is not as helpful.
  • Another part that caught our interest was that on deletion, an event gets broadcasted and the dialog closed. In the test, however, not the actual methods but respective spy methods had been used: dismissSpy() and broadcastSpy(). We assumed this makes the unit test even more "unit". Another point to dive into deeper!
  • We moved on to the next test, this time the more complex one for updating the component. Once again, we started from top, including the imports. Gem called out that it's interesting that a HttpResponse is imported, came unexpected.
  • We also once more saw that the test was named "Challenge Management Update Component", as it was generated - we both found it weird that the word "management" was included here.
  • We had a look at the test for saving. It's really great that the generated tests included comments for given / when / then which made them easier to read and understand. This test, interestingly, also showed a comment for the tick() method which the other test lacked!
  • We checked the differences for updating and creating a challenge, both on test and implementation side.
  • What caught our eye was a boolean called isSaving. We wondered what it would do? We decided to first interact with the app again to see what's actually happening. A GET request, a PUT request. Nothing unexpected. Then we had a look at the implementation for isSaving. And we found this boolean is set to true only during the time the actual saving process is taking place. The value is only used in the tests. We both found this very weird. We would not expect this kind of implementation just for the tests, we never saw something like this in the products we worked on. We would only expect to add this during debugging to see what's going on; yet for the test it felt strange and it didn't feel to provide any value. I took a note to remove this implementation later on.
  • Reflection:
    • Time was flying and we soon got to the end of our 90min timebox.
    • We both found it really nice to take time and dissect unit tests, challenging them, asking why, finding issues and questions while learning about the product. It proved again the point that you don't need to know code to the extent a developer does to be able to use your skills and strengths as a tester on code.
    • I absolutely enjoyed our pairing session. When pairing we both bring our skills, expertise and knowledge to the table, and Gem and I had both overlapping and differing knowledge which fit together nicely.
    • When pairing I realized how valuable it is to also look at the imports! I normally skipped them, yet we can already raise questions and improve our understanding here.
    • It was great that Gem reminded me to interact with the real product again. I tend to focus too narrowly on what's in front of me which in this case is the code base, so it's easy to get disconnected. I noticed this several times now already during my code-confident challenge, catching myself on things I should know better. Gem pointed out this needs a context switch which is not easy. I agree, and it also increased my empathy towards my developers a lot more.
    • Gem noticed me taking notes and realized she should also think about how to take notes better for her next stops on her testing tour.
    • Gem found JHipster really intriguing. All the weirdness and bloatiness aside it seems to be really valuable to quickly kick-start a project and be able to learn with it. She plans to generate her own project for further learning purpose.
    • We both found it super easy to pair with each other, even though we only met a few times in real life or virtually and paired only once before. Time flew by and we both were energized afterwards!
    • I'm super looking forward to hearing more about Gem's testing tour and the lessons she will learn on her way :)
  • TODO:
    • install the Redux DevTools
    • remove isSaving boolean

What else?

The next Ministry of Testing Power Hour is coming up! I'm excited to be the one answering all your questions about pairing and mobbing. Just ask them at the club, and on August 22nd I'll answer them from my experience.
Also, the call for collaboration for European Testing Conference is open! Seize the opportunity to participate in this amazing conference next year by proposing your session ideas. The submission alone is worth it as you will get invaluable feedback.

Sunday, July 14, 2019

#CodeConfident: Journey

As designed in my experiment for my #CodeConfident challenge, I have now five GitHub repositories of practice projects. As I had hoped, these five were already sufficient to feel confident enough to start my proof of concept project: to implement a new small product from scratch. May I present to you: my first steps of building Journey, a learning journal application.

Pre-implementation Thoughts

  • initial brainstorming
  • May 28
    • think about presentation already, maybe app about my challenge? reasons why, etc? people I collaborated with, etc. - basically built my presentation? or maybe as addition... and private repo in that case before I make it public
    • tech stack: Java, Spring, Hibernate & Angular, TypeScript, Redux flow & in memory or docker SQL database - basically what we have at work; cover with tests!!! make it especially testable
    • model the architecture!
    • have a rough goal in mind; maybe even fulfill a purpose I have myself, like keeping a list of good reads or bookmarks of things I want to do? these already exist.. does not need to be new, however!
    • then let the app evolve
    • MVP
    • did story mapping? 
    • Define a small scope first and start
    • Idea!! A coding journal app!
      • I know more what I would love to have
      • Would have free text fields, date picker etc.
      • Make it more general: a learning tracker! See your progress, visualize
      • Lots of potential features, can really be extended
      • Maybe also help define your challenge! :D
      • "You are awesome"
      • Growth
      • Add motivational cheers per day
    • Do story map, then architecture
      • Have it documented
    • For GitHub: use project this time to track progress
  • May 29
    • created mind map with all ideas around app
  • June 11
  • June 15
    • Journey is used for a blog engine, a migration tool, a router for rails, journey planner, game, MVC tool, native diary app
    • Journey
    • Ready to start :-)
    • Use the product to run all sorts of tests for, exploratory and automated, accessibility, security, static code analysis, etc.
    • Check out the security / vulnerability scanner snyk

June 17

  • created GitHub repo
  • started to initialize project with Spring Boot: https://start.spring.io/ adding the same dependencies as for https://spring.io/guides/tutorials/bookmarks/
  • decided to use JHipster instead to quickly generate the project with the complete desired tech stack: https://www.jhipster.tech/
  • found that there's also an online generator: https://start.jhipster.tech/#/
  • found they recommend https://adoptopenjdk.net/ for Java builds
  • still wanted to use the local installation
  • found that jhipster command is not recognized on Windows, same for yeoman; normally that's an issue with the path, could not figure out why yet; checked the path and it correctly pointed to node; found that this issue can occur with multiple node instances so the package install location might be different: https://github.com/yeoman/yeoman/issues/1659; learned that you can see where your package is installed: npm ls -g yo (https://stackoverflow.com/questions/24511032/yeoman-yo-command-not-found); added this to my path, and now the commands are recognized!
  • generated project: monolithic, no JHipster registry, JWT authentication, SQL, MySQL, MySQL, ehcache, Hibernate 2nd level cache, Gradle, API first development with generator, Angular, default JHipster theme, no internationalization, Protractor, no other generators
    https://www.jhipster.tech/creating-an-app/
    https://dev.to/keysh/spring-security-with-jwt-3j76

    added 1845 packages from 1540 contributors and audited 892246 packages in 472.857s
    found 1 moderate severity vulnerability
      run `npm audit fix` to fix them, or `npm audit` for details
    Application successfully committed to Git.

    If you find JHipster useful consider sponsoring the project https://www.jhipster.tech/sponsors/

    Server application generated successfully.

    Run your Spring Boot application:
    ./gradlew (gradlew if using Windows Command Prompt)

    Client application generated successfully.

    Start your Webpack development server with:
     npm start

    > journey@0.0.0 cleanup D:\Dev\journey
    > rimraf build/resources/main/static/ build/resources/main/aot

    INFO! Congratulations, JHipster execution is complete!
  • ran: npm audit fix
  • noticed: could not specify Java and Angular versions; got Java 8 & Angular 8
  • generated nice readme with instructions
  • ran: npm install
  • in 2 terminals:
    gradlew
    npm start
  • --> need to configure DB first, then go through everything and adapt, then commit

June 18

  • renamed package; IDE refactoring did not catch all cases, app startup told what to correct
  • learned how to start the DB in Docker (for Windows with a backslash instead):
    docker-compose -f src/main/docker/mysql.yml up
    (https://www.jhipster.tech/docker-compose/ but can also be found in readme file)
  • checked if app can now connect with DB: successful :)
  • updated readme
  • cleaned up for commit
  • TODO: follow further instructions to model entities etc.

June 20

  • GitHub & npm install informed about security vulnerability; still exists after upgrading Swagger UI
    https://www.npmjs.com/advisories/745/versions
    https://github.com/swagger-api/swagger-ui/issues/5152
    https://github.com/jonschlinkert/remarkable/issues/310
  • read more about how to get started with JHipster
    https://www.jhipster.tech/creating-an-entity/
    https://www.jhipster.tech/creating-a-spring-controller/
    https://www.jhipster.tech/creating-a-spring-service/
    https://www.jhipster.tech/using-dtos/
    https://www.jhipster.tech/entities-filtering/
  • started to generate first entity
  • thought about adding a date field, yet decided not to go this direction yet to avoid the whole complexity of dates and times; added only fields title and description in the end (id got generated automatically)
  • found Google calendar entries event descriptions allow up to 8192 characters, so decided to set the maximum to the 8000
  • decided to add filtering and pagination, went for links
    https://getbootstrap.com/docs/4.3/components/pagination/
    https://sroze.github.io/ngInfiniteScroll/
  • finished configuring the entity, was generated successfully (only a few warnings due to the upgraded Swagger UI were thrown)
  • checking the changes, JHipster indeed did most of the stuff already! Amazing what code generators can do for you already, bootstrapping whole apps; feeling I cheated here, yet I know that a tool is there to support and at work we also use generators, e.g. for Angular; it simply saves lots of time and also lets me have a base to extend on; might bereft me of some learnings, yet I can still do that later on anyway
  • generated tests are passing
  • ran the app; database connection failed; rebuilt the database container, didn't help; couldn't create the bean due to a missing table:
    Unable to build Hibernate SessionFactory; nested exception is org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation:
     missing table [jhi_date_time_wrapper]
  • connected to the database, saw that no tables had been created at all
  • stopped DB again and recreated it from scratch (without option -d)
  • startup still fails as tables are missing like authority, yet the DB already shows a few tables; saw Liquibase runs asynchronously while bean creation already failed
  • not making sense of reading the exceptions or internet searches; assumption: the previous sample implementation interferes with my new one
  • made liquibase configuration synchronous; had the same issue (only took longer); seems unrelated with data
  • back to the start: cloned the repo again, started up the app
  • old app successfully set up database; starting the new app, it complained about missing table jhi_date_time_wrapper; found the related liquibase changeset was marked to be run for a test context only; removed that, ran again; now the new app starts up just as expected
  • tried again with
    docker-compose -f src\main\docker\mysql.yml down
  • still not working, no matter if the changeset goes with or without the test context set
  • what I know: it runs when the db is set up properly; it fails when running the changesets
  • arrrgh, found a change in one of the data sets: had typed a character there by mistake ^^
  • nope, wasn't that.. :(
  • tried to run liquibase separately, but cannot find mysql connector
    https://stackoverflow.com/questions/51991209/liquibase-mysql-jdbc-driver-connectivity-error-using-liquibase-properties
  • don't have a properties file
    https://qxf2.com/blog/mysql-and-liquibase/
  • tried to add dependency like in https://discuss.gradle.org/t/how-do-i-include-a-classpath-in-my-task/12304, did not help
            classpath "mysql:mysql-connector-java:8.0.16"
  • feeling I'm going down a rabbit hole; focus again on app startup
  • interesting links:
    https://www.liquibase.org/documentation/update.html
    https://github.com/brettwooldridge/HikariCP/issues/1246
    https://github.com/brettwooldridge/HikariCP/issues/198
    https://stackoverflow.com/questions/54804275/hikaripool-1-connection-marked-as-broken-because-of-sqlstate08s01-errorcode
    https://www.baeldung.com/spring-boot-hikari
    https://github.com/brettwooldridge/HikariCP#configuration-knobs-baby
    https://www.gitmemory.com/issue/jhipster/generator-jhipster/9721/491481228
    https://github.com/jhipster/generator-jhipster/issues/9661
    https://www.liquibase.org/documentation/preconditions.html
  • tried commenting out fake csv data yet it failed as well
  • needed to remove the test context from the jhi_date_time_wrapper table creation
  • the first changeset is the culprit, it fails at different steps, afterwards it works; have to make it work in one run instead; seems to be a timeout until connection is cut off
  • tried again from scratch, including the jhi_date_time_wrapper changeset and having liquibase configured synchronously before app startup - and it worked!!!
  • tested it out carefully again, removing settings I tried out; found that it was indeed only the combination of having liquibase configured synchronously and to include the test changeset as somewhere it was expected (while it wasn't before), plus resetting the database completely or releasing the database changelog lock
  • lesson learned: read exceptions more carefully, search better, test out your assumptions one at a time
  • committing for now, will have to clean up all previous sample data later
  • interesting log messages and exceptions:
    2019-06-21 00:36:19.983 WARN 6040 --- [ journey-task-1] i.g.j.c.liquibase.AsyncSpringLiquibase : Starting Liquibase asynchronously, your database might not be ready at startup!

    2019-06-21 00:36:22.559 WARN 6040 --- [ journey-task-1] com.zaxxer.hikari.pool.ProxyConnection : Hikari - Connection com.mysql.cj.jdbc.ConnectionImpl@10549e00 marked as broken because of SQLSTATE(08S01), ErrorCode(0)

    com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
    The last packet successfully received from the server was 139 milliseconds ago. The last packet sent successfully to the server was 139 milliseconds ago.

    Caused by: org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation: missing table [jhi_authority]

    2019-06-21 00:36:22.542 WARN 6040 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.

    ApplicationContextException: Unable to start web server; nested exception is java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servletEndpointRegistrar' defined in class path resource [org/springframework/boot/actuate/autoconfigure/endpoint/web/ServletEndpointManagementContextConfiguration$WebMvcServletEndpointManagementContextConfiguration.class]: Bean instantiation via factory method failed;
    nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar]: Factory method 'servletEndpointRegistrar' threw exception;
     nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'auditEventsEndpoint' defined in class path resource [org/springframework/boot/actuate/autoconfigure/audit/AuditEventsEndpointAutoConfiguration.class]: Unsatisfied dependency expressed through method 'auditEventsEndpoint' parameter 0;
    ...
  • TODO: remove unused sample parts like user and registration (or keep it? nah, not the idea of the app, can re-implement in case direction changes, or still?) nah, reduce code where feasible! always a good idea, can look things up later again
  • TODO: cover with tests
  • TODO: adapt functionality where desired

June 21

  • explored more what the bootstrapped app already offers - and it's a lot! comes with user management, metrics and more; also: responsive!!
  • lots of directions to move on to, like adjusting pages, using nav bar instead of menu for journal, increasing the description field size, ... lots of feature ideas in my mind map as well, also for further entities
  • lots of things to test as well! e.g. lots of interesting output in the DevConsole (including errors ;) e.g. for health check, configuration, logs, metrics update... and the swagger warnings, have to solve them)
  • TODO: fix warnings and errors

June 22

  • pairing session with Amber Race
  • introduced her to the current state after bootstrapping the app and generating the first custom entity, journal entry
  • we agreed to switch who's at the keyboard from time to time, and did so
  • Amber was interested to explore frontend development, and so was I
  • I suggested a few things we could improve on frontend side for journal entries: hiding the id on the frontend, increasing the description input text box size to a multi-line element, adding a date input field
  • we started out with hiding the id for updating entries and could simply do it by removing the related html part
  • we wanted to change the description input to a textarea field, thinking this was an easy change as well - and we stumbled; it took us quite some trial and error and googling until we found out that textarea needed an explicit closing tag! Too easy to be true - seems you have to stumble here and then you know it; at least we got around implementing a custom component just for that
    https://www.w3schools.com/tags/tag_textarea.asp
    https://angular.io/guide/forms-overview
  • after implementing the textarea, we thought what happens if we press Enter in the field, does it create a new line or instantly submit the form? We found it does create new lines, yet we saw an error thrown in the console:
    Assertion failed: Input argument is not an HTMLInputElement
    Uncaught TypeError: Cannot read property 'type' of undefined
        at e.setFieldValue (onloadwff.js:71)
        at HTMLFormElement.formKeydownListener (onloadwff.js:71)

    I knew this error from work and even reported it as issue as it occurred quite frequently when using keyboard navigation; we googled the error and found out it's caused by the LastPass Chrome extension! o_o Tried it in FireFox, and indeed it didn't happen here. Guess I'm owing an apology to my fellow developers.. Lesson learned: google error messages before reporting them
    https://www.rockyourcode.com/assertion-failed-input-argument-is-not-an-html-input-element
    https://github.com/mui-org/material-ui/issues/14860
    https://github.com/KillerCodeMonkey/ngx-quill/issues/351
  • we wanted to add a new field when updating a journal entry, a date; we created a new form-group div with an input of type "datetime-local" and got errors; right, we're missing the field in the related TypeScript file as well, so added it here too; still we got errors; we found that the formControlName attribute is linked to the TypeScript field so they have to match; googling let us try the input type "date" instead - and we found it worked! It even automatically offered us a date picker :D of course the backend part needs to be implemented as well, so we can't save the value, but that's for another day
  • we still saw an error when loading the page, yet it seems to be one of these that the bootstrapped app offers yet did not implement: http://localhost:9000/management/info; have to dig into this later
  • Retrospective:
    • Amber: that was fun! Learned something about Angular and TypeScript, how it's wired together, has some picky things that you have to learn
    • Lisi: agree, all this is a test for me how far I can get puzzling pieces of knowledge and what I learn from googling together without doing formal tutorials; seems in the end I have to go back to tutorials, yet then I often end up only knowing how to do things within the clinical environment of a tutorial, not in a real app
    • Amber: sometimes you have to learn the hard way
    • Lisi: felt that bootstrapping the app to have it as practice playground is like having a legacy app where you have to find your way through, like when starting a new job

June 22 (cont)

  • received a pull request from my learning partner Toyer Mamoojee; clarified that the tests had been auto-generated; will look into the PR when I come to this part https://github.com/lisihocke/journey/pull/1/commits/8853f704d1e12cefa46cc9059594ff7ab9321755
  • fixed SwaggerUI warnings after upgrade to new version by adapting the webpack.common.js paths
    https://github.com/dancancro/great-big-example-application/issues/184
  • added new date field to the rest of frontend logic, adapted tests
    https://www.digitalocean.com/community/tutorials/understanding-date-and-time-in-javascript
  • copied backend part as far as I could; used Date in the beginning, stumbled with filters which used local dates; checked out the project one more and generated a test entity with a date field with JHipster to see how they would do it; using Java LocalDate and Moment in the frontend; adapted implementation accordingly
  • running liquibase for generating a changeset automatically failed; did it manually for now, to be investigated
  • nearly worked from the start! only had to make date nullable for the beginning as existing fields are not set yet
  • tried it out: date cannot be saved, bad request; no value had been sent and the field must not be null; sent API call directly via Postman, same result; compared again, found typo in DTO setter... o_O and that was it!
  • now working, even sorting (smoke test only ;) )
  • added not null constraint on database field; learned I need to specify the data type and a default value for those which are currently null
    https://www.liquibase.org/documentation/changes/add_not_null_constraint.html
  • running all kind of existing tests; integration tests for journal entry fail, all other pass; found I never ran the protractor tests; installed and ran them (found alternative to running "npm run e2e":)
    https://www.protractortest.org/#/
    npm install -g protractor
    webdriver-manager update
    webdriver-manager start
    protractor src\test\javascript\protractor.conf.js


    complained that mocha could not be found:
    E/launcher - Error: Error: Cannot find module 'mocha'

    npm install mocha

    did not help; installing globally and then running "npm install" again, however, worked! tests running now --> 2 tests fail, for logs and configuration
  • newly generated tests are passing; old ones fail, filter does indeed not work as expected, returns all existing entries; what did I forget to adapt?? copied over again criteria and query service from generated sources - now the tests are passing, seems there was a tiny little mistake somewhere I could not see
  • TODO: fix logs and configuration implementation

June 23


June 24

  • checking out project on Mac, new run
  • found that now profiles are working! but Swagger UI not; ran open api generation again, still not working; what else did I do??
  • tried again with existing project; decided to remove all generated parts like node modules and build etc.; after having everything regenerated the profile service is working again!! now Swagger does not work, same situation
  • logs show huge number of loggers but none displayed, filter does not do anything
  • Swagger errors:
    Refused to apply style from 'http://localhost:9000/swagger-ui/dist/css/typography.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
    swagger-ui.min.js:1 Failed to load resource: the server responded with a status of 404 (Not Found)
    GET http://localhost:9000/swagger-ui/dist/lib/swagger-oauth.js net::ERR_ABORTED 404 (Not Found)
    Uncaught ReferenceError: $ is not defined
        at index.html:33
  • tried to adapt the Swagger UI index.html to what is available; getting different errors now
    Uncaught ReferenceError: require is not defined
    :9000/swagger-ui/index.html:17 Uncaught ReferenceError: $ is not defined
    docs:1 Refused to display 'https://swagger.io/' in a frame because it set 'X-Frame-Options' to 'sameorigin'.
  • when clicking on Swagger, I get "swagger.io refused to connect"
  • https://swagger.io/docs/specification/about/
  • could not solve issue yet;
    Uncaught ReferenceError: require is not defined
        at Object.<anonymous> (swagger-ui.js:1)
  • TODO with Amitai: solve SwaggerUI issue, move journal to navbar, create new entity to define challenge and experiment, checklist for evaluation and progress tracking, re-brand from JHipster to journey

June 25

  • pairing with Amitai Schleier on journey project
  • Lisi presented what had been done, what got generated and what we changed, also options to move forward; also shared struggles upgrading SwaggerUI
  • Amitai: I have a learned behavior: whatever is annoying, your feeling is telling you the truth; it's annoying now it will be annoying later; so let's fix SwaggerUI
  • Amitai: can we go back to how it was before? --> Lisi showed him the changes in the package.json and the wepack common js, also addressed the changes in the index.html
  • Amitai: first thing I notice it's a major version bump
  • looked for an upgrade guide together, didn't find too promising results
    https://blog.readme.io/an-example-filled-guide-to-swagger-3-2/
  • tried to find official documentation
    https://swagger.io/docs/open-source-tools/swagger-ui/usage/installation/
  • decided to switch to figuring out what's actually going on, look at what we have
  • Amitai: I follow my curiosity until it seems like taking energy, always looking for the easy path; it's also fine to use all willpower and push through, sometimes does that as well, yet tend to follow his energy more
  • looked at the generated sources when the app was built; did not find anything related in the build directory, looked at the node module; found I missed to copy a JavaScript file in the Webpack configuration; added this, yet no difference
    { from: './node_modules/swagger-ui/dist/swagger-ui.js', to: 'swagger-ui/dist/swagger-ui.js' },
  • again googled for a tutorial, didn't find a promising thing
    https://idratherbewriting.com/learnapidoc/pubapis_openapi_tutorial_overview.html#migrating-from-openapi-20-to-30
  • Amitai: we haven't asked the right questions yet; we should look at what we have, it will tell us the right questions
  • adapted the index.html to call the SwaggerUI JavaScript only, so no 404 errors anymore; also checked that the script could be loaded successfully
  • now had two different errors:
    external "deep-extend":1 Uncaught ReferenceError: require is not defined
        at Object. (external "deep-extend":1)
        at __webpack_require__ (bootstrap 6cb5c7f86a6d6e7d420d:19)
        at Object. (index.js:1)
        at __webpack_require__ (bootstrap 6cb5c7f86a6d6e7d420d:19)
        at Object. (all.js:16)
        at __webpack_require__ (bootstrap 6cb5c7f86a6d6e7d420d:19)
        at bootstrap 6cb5c7f86a6d6e7d420d:62
        at bootstrap 6cb5c7f86a6d6e7d420d:62
        at webpackUniversalModuleDefinition (universalModuleDefinition:9)
        at universalModuleDefinition:1
    (anonymous) @ external "deep-extend":1
    __webpack_require__ @ bootstrap 6cb5c7f86a6d6e7d420d:19
    (anonymous) @ index.js:1
    __webpack_require__ @ bootstrap 6cb5c7f86a6d6e7d420d:19
    (anonymous) @ all.js:16
    __webpack_require__ @ bootstrap 6cb5c7f86a6d6e7d420d:19
    (anonymous) @ bootstrap 6cb5c7f86a6d6e7d420d:62
    (anonymous) @ bootstrap 6cb5c7f86a6d6e7d420d:62
    webpackUniversalModuleDefinition @ universalModuleDefinition:9
    (anonymous) @ universalModuleDefinition:1

    index.html:16 Uncaught ReferenceError: $ is not defined
        at index.html:16
  • Amitai: this message doesn't make me feel like I'm supposed to read this; Lisi: we're not the target audience
  • Amitai: idea: set up a throw-away project with Swagger 3 to see how it should work; partition the complexity of the problem
  • searched for example projects, did not find any suitable ones
    https://github.com/swagger-api/swagger-codegen/tree/master/samples/client/petstore/typescript-angular-v6/npm
    https://medium.com/@sean_bradley/add-swagger-ui-to-existing-nodejs-typescript-api-882ca7aded90
  • Amitai: code generation is a tricky bargain, it provides you lots of code you don't want to write, yet you don't want that much code; Lisi: really relate to that, had the feeling that I instantly entered a legacy project where I needed to understand first what's going on
  • looked at the script called in the index.html
  • what is Springfox?
    https://springfox.github.io/springfox/
  • googled the error message, yet did not find something meaningful
  • next question is what does the $ in JavaScript mean?
  • Reflecting:
    • Amitai: lots of things are interacting with each other, we need to know a lot more than we do; it's not clear where the boundaries are; I'm working at my best when the lack of knowledge is very contained
    • Lisi: what if I would remove Swagger completely from the project and start again? Amitai: probably the same; would rather pay someone; would look for someone who knows Swagger and pair with them
    • Amitai: if this were our job I'd come back rested and create a small project using SwaggerUI 3 from scratch, just to understand and throw away; then come back to this; the next thing would be what is going on with Webpack; but first let me write some code with my own hands
    • Lisi: feeling I really lack lots of knowledge, have these bits and pieces here and there and trying to puzzling them together, yet there's so much white space; Amitai: I need a certain amount of knowledge to be able to guess well
    • Amitai: this time I drove a lot of the questions, was that okay with you? Lisi: also noticed I stayed at the keyboard, hoped it was okay; for me it was really good to have someone else navigating as I tried already a lot and needed that fresh perspective and differences how to move on; Amitai: really comfortable in asking questions
    • Lisi: although we didn't have a happy end, this session was really valuable for me to see it really is okay that I could not fix the issue already, not that easy

June 25 (cont)


June 27

  • compared how index.html delivered with the npm package calls SwaggerUI; difference is that authentication is required for local app
  • ran new copy of the app with old version again to see the difference; npm package is highly different, also shows files that the new version does not have anymore; the API URLs you can check there are:
    /v2/admin-docs
    /v2/admin-docs?group=management
  • checked the JHipster sample app repos, all of them still show the old Swagger version
  • found that the script needs the jQuery scripts to run, otherwise I get the same error:
    Uncaught ReferenceError: $ is not defined
  • it's still a mystery; I learned the index.html needs to be re-written to work with the new version yet also allow authentication; also, that webpack might be another culprit; --> I put this on ice until I learned more about JavaScript to be able to fix this, or until JHipster releases a new version fixing the security vulnerability here
  • TODO: go on developing the app for now

July 1


July 2

  • decided to first create a new entity so the product takes more shape and later solve the other stuff (they will bite me, I'm sure of that; yet the end product also does not have to be perfect, it's about learning after all)
  • created new entity "Challenge" with fields that I used when defining mine
  • used the JournalEntry entity as a template, then copy & paste & adapt
  • generation of getters and setters is super useful (to call the generation popup in IntelliJ Windows edition: Alt+Insert)
  • noticed one more that IntelliJ used the asterisk import; searched again for the setting to always use explicit imports instead of wildcards: https://www.jetbrains.com/help/idea/creating-and-optimizing-imports.html#disable-wildcard-imports
  • got lots of unused imports, shortcut is really handy: for IntelliJ Windows it's Ctrl+Alt+O
  • finished backend part without test, tried to run the app
  • due to copy & paste I used varchar for the database update script although I needed a type that can store more data like text
    https://chartio.com/resources/tutorials/understanding-strorage-sizes-for-mysql-text-data-types/
  • also, caught copy & paste error in liquibase script; really need to follow-up to be able to generate the scripts automatically
  • as I ran the liquibase script halfway through before I got a checksum error; just deleted database and started from scratch, easiest thing without productive data
    https://www.exoplatform.com/blog/2017/10/18/how-to-modify-a-change-set-in-liquibase/
  • app is running! :D
  • smoke test for the new controller, called API, happy path was working
  • TODO: add relation of journalEntry to challenge
  • TODO: create tests for new controllers
  • TODO: create frontend part for challenge

July 3


July 6


July 8

  • checked existing UserMapper integration test
  • found that my new mappers for journal entry and challenge had a lot less logic, yet one part was the same; for user mapper a test had been generated, for the new entities not
  • implemented the same test for both new mappers
  • TODO NEXT: add implementation in frontend

July 9

  • tackling the frontend implementation for the challenge entity now
  • creating new challenge model, adapting the journal entry model
  • found that also the journal entry model showed attributes as optional although they shouldn't be from my perspective; learned later in the routing that this way you can call an empty constructor, so changed all to optional
  • learned shortcut in Visual Studio Code for Windows: Alt+Up/Down moves line
  • learned that optional parameters must follow all required ones
  • now trying to first make the existing part for journal entry work again before implementing the challenge frontend part
  • did not work at first try, didn't catch everything; had a look at the generated project to learn what's missing; missed quite a lot; learned they do the mapping via ids
  • seems I have to implement the rest of challenges as well to make journal entries work again; did that, still got one error in the journal update component ngOnInit; probably a copy & paste error; and it was, including a missing method
  • Visual Studio Code shortcut to optimize imports is similar to IntelliJ's: Shift+Alt+O
  • Visual Studio Code shortcut to duplicate the line down: Shift+Alt+Down
  • added all challenge fields in frontend, adapted to optional and textarea where desired
  • committed!! :D
  • thought: the copy & paste approach might be questionable, yet it gave me the working template needed to be confident enough to write the initial set, make it work, and then to do changes myself; what helped was the sheer repetitive practice of copying over, replacing all instances by hand so I really had to read the code line by line; sometimes used find and replace to reduce typos yet then I was really oblivious what the code was doing
  • lots of open todos, yet I feel it's really time for the first blog entry about this project
  • TODO: show tag instead of id for journal entry (all pages)
  • IDEA: don't show fields on detail page if they are empty?
  • TODO: adapt tests and cover the rest
  • TODO: show decent error message when you try to delete a journal entry that is linked to a challenge or vice versa
  • TODO: investigate error message on page load ("Error: Uncaught (in promise)")
  • TODO: test behavior a lot further, discover problems and risks and further ideas

July 13


To Be Continued...

This coding journal post already got way too long. Obviously, there are still lots of open todos, even for a first rough minimal application that does not need to claim perfection. Therefore my coding journal will proceed in upcoming posts.

On Another Note

I am super excited to announce that Gem Hill just started her own testing tour focused on security testing! We had a great session on my testing tour last year, and now she decided to kick off her own tour. Can't wait to learn more about it, and hope to make a guest appearance on her tour as well!