As designed in my
experiment for my
#CodeConfident challenge, I have now
five GitHub repositories of practice projects. As I had hoped, these five were already sufficient to feel confident enough to start my proof of concept project: to implement a new small product from scratch. May I present to you: my first steps of building
Journey, a learning journal application.
Pre-implementation Thoughts
- initial brainstorming
- May 28
- think about presentation already, maybe app about my challenge? reasons why, etc? people I collaborated with, etc. - basically built my presentation? or maybe as addition... and private repo in that case before I make it public
- tech stack: Java, Spring, Hibernate & Angular, TypeScript, Redux flow & in memory or docker SQL database - basically what we have at work; cover with tests!!! make it especially testable
- model the architecture!
- have a rough goal in mind; maybe even fulfill a purpose I have myself, like keeping a list of good reads or bookmarks of things I want to do? these already exist.. does not need to be new, however!
- then let the app evolve
- MVP
- did story mapping?
- Define a small scope first and start
- Idea!! A coding journal app!
- I know more what I would love to have
- Would have free text fields, date picker etc.
- Make it more general: a learning tracker! See your progress, visualize
- Lots of potential features, can really be extended
- Maybe also help define your challenge! :D
- "You are awesome"
- Growth
- Add motivational cheers per day
- Do story map, then architecture
- For GitHub: use project this time to track progress
- May 29
- created mind map with all ideas around app
- June 11
- June 15
- Journey is used for a blog engine, a migration tool, a router for rails, journey planner, game, MVC tool, native diary app
- Journey
- Ready to start :-)
- Use the product to run all sorts of tests for, exploratory and automated, accessibility, security, static code analysis, etc.
- Check out the security / vulnerability scanner snyk
June 17
- created GitHub repo
- started to initialize project with Spring Boot: https://start.spring.io/ adding the same dependencies as for https://spring.io/guides/tutorials/bookmarks/
- decided to use JHipster instead to quickly generate the project with the complete desired tech stack: https://www.jhipster.tech/
- found that there's also an online generator: https://start.jhipster.tech/#/
- found they recommend https://adoptopenjdk.net/ for Java builds
- still wanted to use the local installation
- found that jhipster command is not recognized on Windows, same for yeoman; normally that's an issue with the path, could not figure out why yet; checked the path and it correctly pointed to node; found that this issue can occur with multiple node instances so the package install location might be different: https://github.com/yeoman/yeoman/issues/1659; learned that you can see where your package is installed: npm ls -g yo (https://stackoverflow.com/questions/24511032/yeoman-yo-command-not-found); added this to my path, and now the commands are recognized!
- generated project: monolithic, no JHipster registry, JWT authentication, SQL, MySQL, MySQL, ehcache, Hibernate 2nd level cache, Gradle, API first development with generator, Angular, default JHipster theme, no internationalization, Protractor, no other generators
https://www.jhipster.tech/creating-an-app/
https://dev.to/keysh/spring-security-with-jwt-3j76
added 1845 packages from 1540 contributors and audited 892246 packages in 472.857s
found 1 moderate severity vulnerability
run `npm audit fix` to fix them, or `npm audit` for details
Application successfully committed to Git.
If you find JHipster useful consider sponsoring the project https://www.jhipster.tech/sponsors/
Server application generated successfully.
Run your Spring Boot application:
./gradlew (gradlew if using Windows Command Prompt)
Client application generated successfully.
Start your Webpack development server with:
npm start
> journey@0.0.0 cleanup D:\Dev\journey
> rimraf build/resources/main/static/ build/resources/main/aot
INFO! Congratulations, JHipster execution is complete!
- ran: npm audit fix
- noticed: could not specify Java and Angular versions; got Java 8 & Angular 8
- generated nice readme with instructions
- ran: npm install
- in 2 terminals:
gradlew
npm start
- --> need to configure DB first, then go through everything and adapt, then commit
June 18
- renamed package; IDE refactoring did not catch all cases, app startup told what to correct
- learned how to start the DB in Docker (for Windows with a backslash instead):
docker-compose -f src/main/docker/mysql.yml up
(https://www.jhipster.tech/docker-compose/ but can also be found in readme file)
- checked if app can now connect with DB: successful :)
- updated readme
- cleaned up for commit
- TODO: follow further instructions to model entities etc.
June 20
- GitHub & npm install informed about security vulnerability; still exists after upgrading Swagger UI
https://www.npmjs.com/advisories/745/versions
https://github.com/swagger-api/swagger-ui/issues/5152
https://github.com/jonschlinkert/remarkable/issues/310
- read more about how to get started with JHipster
https://www.jhipster.tech/creating-an-entity/
https://www.jhipster.tech/creating-a-spring-controller/
https://www.jhipster.tech/creating-a-spring-service/
https://www.jhipster.tech/using-dtos/
https://www.jhipster.tech/entities-filtering/
- started to generate first entity
- thought about adding a date field, yet decided not to go this direction yet to avoid the whole complexity of dates and times; added only fields title and description in the end (id got generated automatically)
- found Google calendar entries event descriptions allow up to 8192 characters, so decided to set the maximum to the 8000
- decided to add filtering and pagination, went for links
https://getbootstrap.com/docs/4.3/components/pagination/
https://sroze.github.io/ngInfiniteScroll/
- finished configuring the entity, was generated successfully (only a few warnings due to the upgraded Swagger UI were thrown)
- checking the changes, JHipster indeed did most of the stuff already! Amazing what code generators can do for you already, bootstrapping whole apps; feeling I cheated here, yet I know that a tool is there to support and at work we also use generators, e.g. for Angular; it simply saves lots of time and also lets me have a base to extend on; might bereft me of some learnings, yet I can still do that later on anyway
- generated tests are passing
- ran the app; database connection failed; rebuilt the database container, didn't help; couldn't create the bean due to a missing table:
Unable to build Hibernate SessionFactory; nested exception is org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation:
missing table [jhi_date_time_wrapper]
- connected to the database, saw that no tables had been created at all
- stopped DB again and recreated it from scratch (without option -d)
- startup still fails as tables are missing like authority, yet the DB already shows a few tables; saw Liquibase runs asynchronously while bean creation already failed
- not making sense of reading the exceptions or internet searches; assumption: the previous sample implementation interferes with my new one
- made liquibase configuration synchronous; had the same issue (only took longer); seems unrelated with data
- back to the start: cloned the repo again, started up the app
- old app successfully set up database; starting the new app, it complained about missing table jhi_date_time_wrapper; found the related liquibase changeset was marked to be run for a test context only; removed that, ran again; now the new app starts up just as expected
- tried again with
docker-compose -f src\main\docker\mysql.yml down
- still not working, no matter if the changeset goes with or without the test context set
- what I know: it runs when the db is set up properly; it fails when running the changesets
- arrrgh, found a change in one of the data sets: had typed a character there by mistake ^^
- nope, wasn't that.. :(
- tried to run liquibase separately, but cannot find mysql connector
https://stackoverflow.com/questions/51991209/liquibase-mysql-jdbc-driver-connectivity-error-using-liquibase-properties
- don't have a properties file
https://qxf2.com/blog/mysql-and-liquibase/
- tried to add dependency like in https://discuss.gradle.org/t/how-do-i-include-a-classpath-in-my-task/12304, did not help
classpath "mysql:mysql-connector-java:8.0.16"
- feeling I'm going down a rabbit hole; focus again on app startup
- interesting links:
https://www.liquibase.org/documentation/update.html
https://github.com/brettwooldridge/HikariCP/issues/1246
https://github.com/brettwooldridge/HikariCP/issues/198
https://stackoverflow.com/questions/54804275/hikaripool-1-connection-marked-as-broken-because-of-sqlstate08s01-errorcode
https://www.baeldung.com/spring-boot-hikari
https://github.com/brettwooldridge/HikariCP#configuration-knobs-baby
https://www.gitmemory.com/issue/jhipster/generator-jhipster/9721/491481228
https://github.com/jhipster/generator-jhipster/issues/9661
https://www.liquibase.org/documentation/preconditions.html
- tried commenting out fake csv data yet it failed as well
- needed to remove the test context from the jhi_date_time_wrapper table creation
- the first changeset is the culprit, it fails at different steps, afterwards it works; have to make it work in one run instead; seems to be a timeout until connection is cut off
- tried again from scratch, including the jhi_date_time_wrapper changeset and having liquibase configured synchronously before app startup - and it worked!!!
- tested it out carefully again, removing settings I tried out; found that it was indeed only the combination of having liquibase configured synchronously and to include the test changeset as somewhere it was expected (while it wasn't before), plus resetting the database completely or releasing the database changelog lock
- lesson learned: read exceptions more carefully, search better, test out your assumptions one at a time
- committing for now, will have to clean up all previous sample data later
- interesting log messages and exceptions:
2019-06-21 00:36:19.983 WARN 6040 --- [ journey-task-1] i.g.j.c.liquibase.AsyncSpringLiquibase : Starting Liquibase asynchronously, your database might not be ready at startup!
2019-06-21 00:36:22.559 WARN 6040 --- [ journey-task-1] com.zaxxer.hikari.pool.ProxyConnection : Hikari - Connection com.mysql.cj.jdbc.ConnectionImpl@10549e00 marked as broken because of SQLSTATE(08S01), ErrorCode(0)
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet successfully received from the server was 139 milliseconds ago. The last packet sent successfully to the server was 139 milliseconds ago.
Caused by: org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation: missing table [jhi_authority]
2019-06-21 00:36:22.542 WARN 6040 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.
ApplicationContextException: Unable to start web server; nested exception is java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servletEndpointRegistrar' defined in class path resource [org/springframework/boot/actuate/autoconfigure/endpoint/web/ServletEndpointManagementContextConfiguration$WebMvcServletEndpointManagementContextConfiguration.class]: Bean instantiation via factory method failed;
nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar]: Factory method 'servletEndpointRegistrar' threw exception;
nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'auditEventsEndpoint' defined in class path resource [org/springframework/boot/actuate/autoconfigure/audit/AuditEventsEndpointAutoConfiguration.class]: Unsatisfied dependency expressed through method 'auditEventsEndpoint' parameter 0;
...
- TODO: remove unused sample parts like user and registration (or keep it? nah, not the idea of the app, can re-implement in case direction changes, or still?) nah, reduce code where feasible! always a good idea, can look things up later again
- TODO: cover with tests
- TODO: adapt functionality where desired
June 21
- explored more what the bootstrapped app already offers - and it's a lot! comes with user management, metrics and more; also: responsive!!
- lots of directions to move on to, like adjusting pages, using nav bar instead of menu for journal, increasing the description field size, ... lots of feature ideas in my mind map as well, also for further entities
- lots of things to test as well! e.g. lots of interesting output in the DevConsole (including errors ;) e.g. for health check, configuration, logs, metrics update... and the swagger warnings, have to solve them)
- TODO: fix warnings and errors
June 22
- pairing session with Amber Race
- introduced her to the current state after bootstrapping the app and generating the first custom entity, journal entry
- we agreed to switch who's at the keyboard from time to time, and did so
- Amber was interested to explore frontend development, and so was I
- I suggested a few things we could improve on frontend side for journal entries: hiding the id on the frontend, increasing the description input text box size to a multi-line element, adding a date input field
- we started out with hiding the id for updating entries and could simply do it by removing the related html part
- we wanted to change the description input to a textarea field, thinking this was an easy change as well - and we stumbled; it took us quite some trial and error and googling until we found out that textarea needed an explicit closing tag! Too easy to be true - seems you have to stumble here and then you know it; at least we got around implementing a custom component just for that
https://www.w3schools.com/tags/tag_textarea.asp
https://angular.io/guide/forms-overview
- after implementing the textarea, we thought what happens if we press Enter in the field, does it create a new line or instantly submit the form? We found it does create new lines, yet we saw an error thrown in the console:
Assertion failed: Input argument is not an HTMLInputElement
Uncaught TypeError: Cannot read property 'type' of undefined
at e.setFieldValue (onloadwff.js:71)
at HTMLFormElement.formKeydownListener (onloadwff.js:71)
I knew this error from work and even reported it as issue as it occurred quite frequently when using keyboard navigation; we googled the error and found out it's caused by the LastPass Chrome extension! o_o Tried it in FireFox, and indeed it didn't happen here. Guess I'm owing an apology to my fellow developers.. Lesson learned: google error messages before reporting them
https://www.rockyourcode.com/assertion-failed-input-argument-is-not-an-html-input-element
https://github.com/mui-org/material-ui/issues/14860
https://github.com/KillerCodeMonkey/ngx-quill/issues/351
- we wanted to add a new field when updating a journal entry, a date; we created a new form-group div with an input of type "datetime-local" and got errors; right, we're missing the field in the related TypeScript file as well, so added it here too; still we got errors; we found that the formControlName attribute is linked to the TypeScript field so they have to match; googling let us try the input type "date" instead - and we found it worked! It even automatically offered us a date picker :D of course the backend part needs to be implemented as well, so we can't save the value, but that's for another day
- we still saw an error when loading the page, yet it seems to be one of these that the bootstrapped app offers yet did not implement: http://localhost:9000/management/info; have to dig into this later
- Retrospective:
- Amber: that was fun! Learned something about Angular and TypeScript, how it's wired together, has some picky things that you have to learn
- Lisi: agree, all this is a test for me how far I can get puzzling pieces of knowledge and what I learn from googling together without doing formal tutorials; seems in the end I have to go back to tutorials, yet then I often end up only knowing how to do things within the clinical environment of a tutorial, not in a real app
- Amber: sometimes you have to learn the hard way
- Lisi: felt that bootstrapping the app to have it as practice playground is like having a legacy app where you have to find your way through, like when starting a new job
June 22 (cont)
- received a pull request from my learning partner Toyer Mamoojee; clarified that the tests had been auto-generated; will look into the PR when I come to this part https://github.com/lisihocke/journey/pull/1/commits/8853f704d1e12cefa46cc9059594ff7ab9321755
- fixed SwaggerUI warnings after upgrade to new version by adapting the webpack.common.js paths
https://github.com/dancancro/great-big-example-application/issues/184
- added new date field to the rest of frontend logic, adapted tests
https://www.digitalocean.com/community/tutorials/understanding-date-and-time-in-javascript
- copied backend part as far as I could; used Date in the beginning, stumbled with filters which used local dates; checked out the project one more and generated a test entity with a date field with JHipster to see how they would do it; using Java LocalDate and Moment in the frontend; adapted implementation accordingly
- running liquibase for generating a changeset automatically failed; did it manually for now, to be investigated
- nearly worked from the start! only had to make date nullable for the beginning as existing fields are not set yet
- tried it out: date cannot be saved, bad request; no value had been sent and the field must not be null; sent API call directly via Postman, same result; compared again, found typo in DTO setter... o_O and that was it!
- now working, even sorting (smoke test only ;) )
- added not null constraint on database field; learned I need to specify the data type and a default value for those which are currently null
https://www.liquibase.org/documentation/changes/add_not_null_constraint.html
- running all kind of existing tests; integration tests for journal entry fail, all other pass; found I never ran the protractor tests; installed and ran them (found alternative to running "npm run e2e":)
https://www.protractortest.org/#/
npm install -g protractor
webdriver-manager update
webdriver-manager start
protractor src\test\javascript\protractor.conf.js
complained that mocha could not be found:
E/launcher - Error: Error: Cannot find module 'mocha'
npm install mocha
did not help; installing globally and then running "npm install" again, however, worked! tests running now --> 2 tests fail, for logs and configuration
- newly generated tests are passing; old ones fail, filter does indeed not work as expected, returns all existing entries; what did I forget to adapt?? copied over again criteria and query service from generated sources - now the tests are passing, seems there was a tiny little mistake somewhere I could not see
- TODO: fix logs and configuration implementation
June 23
- noticed through Jest plugin in Visual Studio Code that Jest tests failed due to syntax errors (unexpected token ; ), couldn't parse; ran them again, confirmed
- some Google results
https://github.com/facebook/jest/issues/6933
https://github.com/babel/babel/issues/9351
https://github.com/kulshekhar/ts-jest/issues/937
https://github.com/thymikee/jest-preset-angular
- interestingly, using "npm test" they all pass; seems to really be a linting issue
- Tried to lint D:/Dev/journey/src/test/javascript/jest.conf.js but found no valid enabled rules for this file type and file path in the resolved configuration.
[BABEL] Note: The code generator has deoptimised the styling of D:\Dev\journey\node_modules\@angular\core\bundles\core.umd.js as it exceeds the max of 500KB.
--> found that tsconfig.json showed allowJs set to true which expects jsRules to be configured in tslint.json then; added them, no warnings anymore
https://github.com/palantir/tslint/issues/3735
https://stackoverflow.com/questions/50518306/what-does-no-valid-rules-have-been-specified-for-javascript-files-mean
https://palantir.github.io/tslint/usage/configuration/
https://www.typescriptlang.org/docs/handbook/tsconfig-json.html
- jest is still failing; googled further and found that it's an issue with the Visual Code plugin and you have to set the path to jest in the IDE settings to e.g. "npm test --"
https://github.com/jest-community/vscode-jest/issues/408
https://github.com/jest-community/vscode-jest/issues/415
https://github.com/jest-community/vscode-jest/issues/417
interesting feature to have all tests run on IDE startup and if they pass enter watch mode; this way you get informed very quickly if you break something
- for running jest in command line you need to provide the location of the config file (as defined in the npm script)
- run jest in watch mode as configured as npm script:
npm run test:watch
- some interesting reading regarding npm:
https://corgibytes.com/blog/2017/04/18/npm-tips/
http://www.marcusoft.net/2015/08/npm-scripting-configs-and-arguments.html
- concerning the Babel warning: https://stackoverflow.com/questions/29576341/what-does-the-code-generator-has-deoptimised-the-styling-of-some-file-as-it-e
- Swagger API link not offered for admins; accessing URL /admin/docs throws errors; found that this came with the (necessary) upgrade, UI was working before; what wasn't working anymore was the http://localhost:8080/management/info, found it's coming from profile.service.ts which also added a ribbon to mark the version as develop; had that at the very beginning and then it stopped working
- learned about Swagger
https://swagger.io/tools/open-source/getting-started/
https://editor.swagger.io/
https://swagger.io/tools/swagger-inspector/
https://github.com/swagger-api/swagger-inflector
- found Swagger UI is actually available and working: http://localhost:9000/swagger-ui/index.html did not work before, did running openApiGenerator help here..? Yet did not define anything in the api.yml; strange; now also the endpoint http://localhost:9000/admin/docs is working as expected
- still the profile service is not working anymore and therefore other views depending on it; it was in the very beginning could check out that commit and compare, debug, add logging; next thing is to fix this part
- TODO: fix all current issues (/management/info, etc.)
- TODO: create other entities, move to nav bar
- TODO: brand to learning journal app
- IDEA: add footer with motivational cheers
- TODO: rewrite to ngrx syntax as at work
June 24
- checking out project on Mac, new run
- found that now profiles are working! but Swagger UI not; ran open api generation again, still not working; what else did I do??
- tried again with existing project; decided to remove all generated parts like node modules and build etc.; after having everything regenerated the profile service is working again!! now Swagger does not work, same situation
- logs show huge number of loggers but none displayed, filter does not do anything
- Swagger errors:
Refused to apply style from 'http://localhost:9000/swagger-ui/dist/css/typography.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
swagger-ui.min.js:1 Failed to load resource: the server responded with a status of 404 (Not Found)
GET http://localhost:9000/swagger-ui/dist/lib/swagger-oauth.js net::ERR_ABORTED 404 (Not Found)
Uncaught ReferenceError: $ is not defined
at index.html:33
- tried to adapt the Swagger UI index.html to what is available; getting different errors now
Uncaught ReferenceError: require is not defined
:9000/swagger-ui/index.html:17 Uncaught ReferenceError: $ is not defined
docs:1 Refused to display 'https://swagger.io/' in a frame because it set 'X-Frame-Options' to 'sameorigin'.
- when clicking on Swagger, I get "swagger.io refused to connect"
- https://swagger.io/docs/specification/about/
- could not solve issue yet;
Uncaught ReferenceError: require is not defined
at Object.<anonymous> (swagger-ui.js:1)
- TODO with Amitai: solve SwaggerUI issue, move journal to navbar, create new entity to define challenge and experiment, checklist for evaluation and progress tracking, re-brand from JHipster to journey
June 25
- pairing with Amitai Schleier on journey project
- Lisi presented what had been done, what got generated and what we changed, also options to move forward; also shared struggles upgrading SwaggerUI
- Amitai: I have a learned behavior: whatever is annoying, your feeling is telling you the truth; it's annoying now it will be annoying later; so let's fix SwaggerUI
- Amitai: can we go back to how it was before? --> Lisi showed him the changes in the package.json and the wepack common js, also addressed the changes in the index.html
- Amitai: first thing I notice it's a major version bump
- looked for an upgrade guide together, didn't find too promising results
https://blog.readme.io/an-example-filled-guide-to-swagger-3-2/
- tried to find official documentation
https://swagger.io/docs/open-source-tools/swagger-ui/usage/installation/
- decided to switch to figuring out what's actually going on, look at what we have
- Amitai: I follow my curiosity until it seems like taking energy, always looking for the easy path; it's also fine to use all willpower and push through, sometimes does that as well, yet tend to follow his energy more
- looked at the generated sources when the app was built; did not find anything related in the build directory, looked at the node module; found I missed to copy a JavaScript file in the Webpack configuration; added this, yet no difference
{ from: './node_modules/swagger-ui/dist/swagger-ui.js', to: 'swagger-ui/dist/swagger-ui.js' },
- again googled for a tutorial, didn't find a promising thing
https://idratherbewriting.com/learnapidoc/pubapis_openapi_tutorial_overview.html#migrating-from-openapi-20-to-30
- Amitai: we haven't asked the right questions yet; we should look at what we have, it will tell us the right questions
- adapted the index.html to call the SwaggerUI JavaScript only, so no 404 errors anymore; also checked that the script could be loaded successfully
- now had two different errors:
external "deep-extend":1 Uncaught ReferenceError: require is not defined
at Object. (external "deep-extend":1)
at __webpack_require__ (bootstrap 6cb5c7f86a6d6e7d420d:19)
at Object. (index.js:1)
at __webpack_require__ (bootstrap 6cb5c7f86a6d6e7d420d:19)
at Object. (all.js:16)
at __webpack_require__ (bootstrap 6cb5c7f86a6d6e7d420d:19)
at bootstrap 6cb5c7f86a6d6e7d420d:62
at bootstrap 6cb5c7f86a6d6e7d420d:62
at webpackUniversalModuleDefinition (universalModuleDefinition:9)
at universalModuleDefinition:1
(anonymous) @ external "deep-extend":1
__webpack_require__ @ bootstrap 6cb5c7f86a6d6e7d420d:19
(anonymous) @ index.js:1
__webpack_require__ @ bootstrap 6cb5c7f86a6d6e7d420d:19
(anonymous) @ all.js:16
__webpack_require__ @ bootstrap 6cb5c7f86a6d6e7d420d:19
(anonymous) @ bootstrap 6cb5c7f86a6d6e7d420d:62
(anonymous) @ bootstrap 6cb5c7f86a6d6e7d420d:62
webpackUniversalModuleDefinition @ universalModuleDefinition:9
(anonymous) @ universalModuleDefinition:1
index.html:16 Uncaught ReferenceError: $ is not defined
at index.html:16
- Amitai: this message doesn't make me feel like I'm supposed to read this; Lisi: we're not the target audience
- Amitai: idea: set up a throw-away project with Swagger 3 to see how it should work; partition the complexity of the problem
- searched for example projects, did not find any suitable ones
https://github.com/swagger-api/swagger-codegen/tree/master/samples/client/petstore/typescript-angular-v6/npm
https://medium.com/@sean_bradley/add-swagger-ui-to-existing-nodejs-typescript-api-882ca7aded90
- Amitai: code generation is a tricky bargain, it provides you lots of code you don't want to write, yet you don't want that much code; Lisi: really relate to that, had the feeling that I instantly entered a legacy project where I needed to understand first what's going on
- looked at the script called in the index.html
- what is Springfox?
https://springfox.github.io/springfox/
- googled the error message, yet did not find something meaningful
- next question is what does the $ in JavaScript mean?
- Reflecting:
- Amitai: lots of things are interacting with each other, we need to know a lot more than we do; it's not clear where the boundaries are; I'm working at my best when the lack of knowledge is very contained
- Lisi: what if I would remove Swagger completely from the project and start again? Amitai: probably the same; would rather pay someone; would look for someone who knows Swagger and pair with them
- Amitai: if this were our job I'd come back rested and create a small project using SwaggerUI 3 from scratch, just to understand and throw away; then come back to this; the next thing would be what is going on with Webpack; but first let me write some code with my own hands
- Lisi: feeling I really lack lots of knowledge, have these bits and pieces here and there and trying to puzzling them together, yet there's so much white space; Amitai: I need a certain amount of knowledge to be able to guess well
- Amitai: this time I drove a lot of the questions, was that okay with you? Lisi: also noticed I stayed at the keyboard, hoped it was okay; for me it was really good to have someone else navigating as I tried already a lot and needed that fresh perspective and differences how to move on; Amitai: really comfortable in asking questions
- Lisi: although we didn't have a happy end, this session was really valuable for me to see it really is okay that I could not fix the issue already, not that easy
June 25 (cont)
June 27
- compared how index.html delivered with the npm package calls SwaggerUI; difference is that authentication is required for local app
- ran new copy of the app with old version again to see the difference; npm package is highly different, also shows files that the new version does not have anymore; the API URLs you can check there are:
/v2/admin-docs
/v2/admin-docs?group=management
- checked the JHipster sample app repos, all of them still show the old Swagger version
- found that the script needs the jQuery scripts to run, otherwise I get the same error:
Uncaught ReferenceError: $ is not defined
- it's still a mystery; I learned the index.html needs to be re-written to work with the new version yet also allow authentication; also, that webpack might be another culprit; --> I put this on ice until I learned more about JavaScript to be able to fix this, or until JHipster releases a new version fixing the security vulnerability here
- TODO: go on developing the app for now
July 1
July 2
- decided to first create a new entity so the product takes more shape and later solve the other stuff (they will bite me, I'm sure of that; yet the end product also does not have to be perfect, it's about learning after all)
- created new entity "Challenge" with fields that I used when defining mine
- used the JournalEntry entity as a template, then copy & paste & adapt
- generation of getters and setters is super useful (to call the generation popup in IntelliJ Windows edition: Alt+Insert)
- noticed one more that IntelliJ used the asterisk import; searched again for the setting to always use explicit imports instead of wildcards: https://www.jetbrains.com/help/idea/creating-and-optimizing-imports.html#disable-wildcard-imports
- got lots of unused imports, shortcut is really handy: for IntelliJ Windows it's Ctrl+Alt+O
- finished backend part without test, tried to run the app
- due to copy & paste I used varchar for the database update script although I needed a type that can store more data like text
https://chartio.com/resources/tutorials/understanding-strorage-sizes-for-mysql-text-data-types/
- also, caught copy & paste error in liquibase script; really need to follow-up to be able to generate the scripts automatically
- as I ran the liquibase script halfway through before I got a checksum error; just deleted database and started from scratch, easiest thing without productive data
https://www.exoplatform.com/blog/2017/10/18/how-to-modify-a-change-set-in-liquibase/
- app is running! :D
- smoke test for the new controller, called API, happy path was working
- TODO: add relation of journalEntry to challenge
- TODO: create tests for new controllers
- TODO: create frontend part for challenge
July 3
July 6
- added field for journalEntry to refer to the challenge
- was unsure regarding the correct mapping, generated test entities in another project using JHipster to find more clues
- added field as foreign key constraint in the database
https://www.liquibase.org/documentation/changes/add_foreign_key_constraint.html
- got different errors; tried again to generate mapping, this time both one to many and many to one, now having different errors and cases
- now with bidirectional mapping and not extra database field I get
Can't map property "java.lang.Long challengeId" to "com.lisihocke.journey.domain.Challenge challenge". Consider to declare/implement a mapping method: "com.lisihocke.journey.domain.Challenge map(java.lang.Long value)".
- interesting resources:
https://www.baeldung.com/mapstruct
https://medium.com/@swathisprasad/automating-code-with-mapstruct-44b8c324a23e
https://sharing.luminis.eu/blog/object-mapping-magic-with-mapstruct/
https://github.com/TheDoctor-95/FuryViewer
https://stackoverflow.com/questions/34672216/cant-map-property-when-using-mapstruct
https://gitter.im/mapstruct/mapstruct-users/archives/2017/01/25
- found by trial & error that defining the "uses" parameter for both mappers worked!
@Mapper(componentModel = "spring", uses = {JournalEntryMapper.class})
- found I need to use @JoinColumn annotation on the field instead of a separate id field, and yet the column in the database; now the database was set up as expected
- tested via API, all seems to work
- committed state
- ran tests; unit tests still passing, yet all integration tests fail!
java.lang.IllegalStateException
Caused by: org.springframework.beans.factory.BeanCreationException
Caused by: javax.persistence.PersistenceException
Caused by: org.hibernate.tool.schema.spi.SchemaManagementException
- stopped app again, ran again - errors! seems I messed up my db when testing, looking for an entry that does not exist; started DB from scratch - and again it's looking for an entry that does not exist! Grrrrr..... and also a schema mapping seems to be wrong
- gradle clean did the trick, also used in the db now only text for long string fields, included "columnDefinition = "text"" in the entity field annotations
https://github.com/spring-projects/spring-boot/issues/7484
https://stackoverflow.com/questions/5952783/problems-with-hibernates-hbm2ddl-auto-validate-and-mysql-text-types
- ran integration tests again, still all are failing.... they fail when creating the Spring context, not related to a test; complain about schema
Caused by: org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation: wrong column type encountered in column [challenge_description] in table [challenge]; found [clob (Types#CLOB)], but expecting [text (Types#VARCHAR)]
- Problem is: H2 does not have text as data type..
http://www.h2database.com/html/datatypes.html
- if I change all to varchar it works with both databases, yet I again exceed the row size limit
- trying clob now, should convert correctly to MySql and H2
https://stackoverflow.com/questions/16890723/list-all-liquibase-sql-types
- also need to add the @lob annotation, then it worked for both databases!
https://stackoverflow.com/questions/11857309/changing-a-column-type-from-varchar-to-clob-with-jpa
- trying to understand how to mock the challenge of a journal entry
https://www.baeldung.com/java-spring-mockito-mock-mockbean
- googled a lot, then decided to generate the same scenario with JHipster again, this time including the NotNullable constraint; checked the tests, saw that they sometimes use the existing database; copied over what I missed - now the test passes!!
- adapted all tests, added by new, created Challenge integration tests
- cleaned up entities and the liquibase scripts according to JHipster proposal: in JournalEntry added "@ManyToOne(optional = false)" and removed "@JoinColumn(name = "challenge_id", nullable = false)"
- TODO FOR LATER: make integration tests completely independent, not using the existing database at all
- TODO FOR LATER: figure out why UserJWTControllerIT and UserResourceIT are passing when run in IDE, but fail when run via Gradle; with --debug or --info output they pass!! according to tests logs they never passed like this
- TODO: add tests for new mappers
- TODO: add implementation in frontend
July 8
- checked existing UserMapper integration test
- found that my new mappers for journal entry and challenge had a lot less logic, yet one part was the same; for user mapper a test had been generated, for the new entities not
- implemented the same test for both new mappers
- TODO NEXT: add implementation in frontend
July 9
- tackling the frontend implementation for the challenge entity now
- creating new challenge model, adapting the journal entry model
- found that also the journal entry model showed attributes as optional although they shouldn't be from my perspective; learned later in the routing that this way you can call an empty constructor, so changed all to optional
- learned shortcut in Visual Studio Code for Windows: Alt+Up/Down moves line
- learned that optional parameters must follow all required ones
- now trying to first make the existing part for journal entry work again before implementing the challenge frontend part
- did not work at first try, didn't catch everything; had a look at the generated project to learn what's missing; missed quite a lot; learned they do the mapping via ids
- seems I have to implement the rest of challenges as well to make journal entries work again; did that, still got one error in the journal update component ngOnInit; probably a copy & paste error; and it was, including a missing method
- Visual Studio Code shortcut to optimize imports is similar to IntelliJ's: Shift+Alt+O
- Visual Studio Code shortcut to duplicate the line down: Shift+Alt+Down
- added all challenge fields in frontend, adapted to optional and textarea where desired
- committed!! :D
- thought: the copy & paste approach might be questionable, yet it gave me the working template needed to be confident enough to write the initial set, make it work, and then to do changes myself; what helped was the sheer repetitive practice of copying over, replacing all instances by hand so I really had to read the code line by line; sometimes used find and replace to reduce typos yet then I was really oblivious what the code was doing
- lots of open todos, yet I feel it's really time for the first blog entry about this project
- TODO: show tag instead of id for journal entry (all pages)
- IDEA: don't show fields on detail page if they are empty?
- TODO: adapt tests and cover the rest
- TODO: show decent error message when you try to delete a journal entry that is linked to a challenge or vice versa
- TODO: investigate error message on page load ("Error: Uncaught (in promise)")
- TODO: test behavior a lot further, discover problems and risks and further ideas
July 13
To Be Continued...
This coding journal post already got way too long. Obviously, there are still lots of open todos, even for a first rough minimal application that does not need to claim perfection. Therefore my coding journal will proceed in upcoming posts.
On Another Note
I am super excited to announce that
Gem Hill just started
her own testing tour focused on security testing! We had a great session on
my testing tour last year, and now she decided to kick off her own tour. Can't wait to learn more about it, and hope to make a guest appearance on her tour as well!