Upload Sprint Recaps

_posts/2024-09-06-my-first-post.md → _posts/2024-08-06-my-first-post.md

@@ -13,7 +13,7 @@ override_image:
   path: assets/img/others/first-post-resized.jpg
   alt: My dog Eevee chilling in a field
 show_image_in_post: true
-date: 2024-09-06 20:25:00 +0800
+date: 2024-08-06 20:25:00 +0800
 pin: true
 published: true
 comments: true

_posts/2024-09-13-first-job.md → _posts/2024-08-13-first-job.md

@@ -4,7 +4,7 @@ title: "Starting my first SWE Job at Crédit Agricole CIB"
 description: >-
   My experience starting my first full-time software engineering job at Crédit Agricole CIB, the corporate and investment banking arm of Credit Agricole.
 author: ryo
-date: 2024-09-13 23:06:10 +0800
+date: 2024-08-13 23:06:10 +0800
 categories: [Crédit Agricole]
 tags: [first-job]
 image:

_posts/2024-09-29-cacib_sprint_2.md → _posts/2024-08-28-cacib_sprint_2.md

@@ -4,7 +4,7 @@ title: "Sprint 2 @ Crédit Agricole CIB"
 description: >-
   Sprint recap [12/08/2024 - 26/08/2024]. My first sprint in the team. Onboarding, orientations, and self-learning.
 author: ryo
-date: 2024-09-29 14:52:23 +0800
+date: 2024-08-29 14:52:23 +0800
 categories: [Crédit Agricole, Sprints]
 tags: [sprint]
 image:
@@ -17,6 +17,17 @@ pin: false
 published: true
 ---
 
+### Jira Tickets
+
+1. **Confluence New Joiner Onboarding Checklist**
+  - Building the team's documentation for future new joiners.
+
+2. **MVP 1 of the JSON-Avro tool (JSON to Avro conversion)**
+  - **Date** - Date strings need to be converted to days after unix epoch. Avro expects 'Integer' types as it's more efficient for storage and querying.
+  - **Decimal** - Decimal needs to be converted to encoded strings e.g. (\u00FF). Avro expects 'Byte' types as it's more efficient for storage and querying.
+  - **Namespace Specification** - JSON nodes with matching Avro schema type 'Union' must be nested with the full Namespace specification. This is to avoid ambiguity for which field-schema the Avro serialization engine should use for encoding, as 'Union' types can include multiple complex types.
+  - **Type Specification** - JSON nodes with matching Avro schema type 'Union' must be provided the fully qualified Type name. This is to avoid ambiguity for which Type the Avro serialization engine should expect within the 'Union', as 'Union' fields can allow multiple Avro Types.
+
 ### Main Activities
 
 - **Meetings:**
@@ -30,11 +41,11 @@ published: true
   - Setting up of local development environment.
   - Documenting team conventions, software installation guide, and new joiner checklist on Confluence.
   - Self-study on Avro, Java Spring Boot, and functional financing documentation.
-  - MVP 1 for the JSON-Avro tool (JSON to Avro conversion).
+  - MVP 1 for the JSON-Avro tool (JSON to Avro conversion) with documentation.
 
 ---
 
-### Key Challenges
+### Key Challenges/Blockers
 
 - Working with unfamiliar technology (Java Spring Boot, Avro).
 - Trying to grasp some functional financing know-how.
@@ -43,7 +54,7 @@ published: true
 
 ### Learning Outcomes
 
-- Gained familiarity with core Java programming.
+- Gained familiarity with core Java programming, as well as jackson by FasterXML.
 - Gained familiarity with Spring Boot, Maven, and Java dependency management.
 - Gained familiarity with the Apache Avro serialization format.
 - Gained insight into how a bank's core business operates, especially in the area of corporate financing, e.g.:

_posts/2024-10-06-cacib_sprint_3.md → _posts/2024-09-10-cacib_sprint_3.md

@@ -4,7 +4,7 @@ title: "Sprint 3 @ Crédit Agricole CIB"
 description: >-
   Sprint recap [26/08/2024 - 9/09/2024]. Working on the JSON-Avro tool's frontend and backend.
 author: ryo
-date: 2024-10-06 17:58:20 +0800
+date: 2024-09-10 17:58:20 +0800
 categories: [Crédit Agricole, Sprints]
 tags: [sprint]
 image:
@@ -17,21 +17,31 @@ pin: false
 published: true
 ---
 
+### Jira Tickets
+
+1. **MVP 1 of the JSON-Avro tool (JSON to Avro conversion)**
+  - Spillover from last sprint.
+
+2. **MVP 2 of the JSON-Avro tool (Avro to JSON conversion)**
+  - Involves reversing the data conversions performed in the first MVP. 
+
+3. **MVP 3 of the JSON-Avro tool (Thymeleaf UI)**
+  - Spring MVC controllers for the tool and a Thymeleaf UI (server-side rendering).
+
+
 ### Main Activities
 
-- **Merge Request (MVP 1)**: Submitted my first merge request for the JSON-Avro conversion tool.
-  - Reading/writing JSON and Avro files.
-  - Iterating over JSON files and Avro schemas with Jackson and building intermediary JSONs, performing certain data conversions including date, decimal, and nesting namespace and type specifications.
-  - Fixed MR comments regarding code quality.
-- **MVP 2 Work**: Started working on the Avro-to-JSON conversion.
 - **SonarQube Integration**: Integrated SonarQube into the project and addressed code smells.
-- **UI Development**: Developed the Thymeleaf UI for the JSON-Avro tool and created a merge request. The tool was dockerized and hosted on Kubernetes by a company architect.
+- **MVP 2 Work**: Started working on the Avro-to-JSON conversion.
+- **MVP 3 Work**: Developed the Thymeleaf UI for the JSON-Avro tool. The tool was dockerized and hosted on Kubernetes by a company architect.
+- **Merge Request (MVP 1, 2 and 3)**: Documented and created merge requests for the JSON-Avro conversion tool.
+  - Fixed MR comments regarding code quality.
 - **Support**: Provided user support for the JSON-Avro conversion tool.
 - **Demo**: Demo-ed the JSON-Avro conversion tool to internal stakeholders in Singapore and Paris as part of the Sprint demo.
 
 ---
 
-### Key Challenges
+### Key Challenges/Blockers
 
 - **Unfamiliar Technology**: Worked with Thymeleaf & Spring Web MVC for the first time, which took some ramp-up.
 - **JSON Un-nesting Complexity**: Dealing with un-nesting JSON nodes during the Avro-to-JSON conversion was more challenging than expected.

_posts/2024-09-22-cacib_sprint_4.md

@@ -0,0 +1,54 @@
+---
+layout: post
+title: "Sprint 4 @ Crédit Agricole CIB"
+description: >-
+  Sprint recap [09/09/2024 - 23/09/2024]. Working on the JSON-Avro tool's frontend and backend, as well as starting work on the main project.
+author: ryo
+date: 2024-09-22 17:58:20 +0800
+categories: [Crédit Agricole, Sprints]
+tags: [sprint]
+image:
+  path: assets/img/cacib/cacib-logo-2-modified.png
+  alt: CACIB Logo
+show_image_in_post: false
+toc: true
+comments: false
+pin: false
+published: true
+---
+
+### Jira Tickets
+
+1. **MVP 4 the JSON-Avro tool (demo feedback)**
+  - Support bulk file conversion, with a table to view conversion status for each file, as well as a 'download all as ZIP folder' functionality.
+  - Support a static directory of Avro schemas to be selected during conversion, with the option to override it with a user-uploaded schema.
+  - UI improvements.
+
+2. **Unit test coverage for a certain set of Java ETL drivers (in the main project)**
+  - More coverage was needed to pass Sonarqube checks.
+
+### Main Activities
+
+- **MVP 4 Work**: Worked on MVP 4 for the JSON-Avro tool, documented it, and created a merge request.
+- **Project Setup**: Setting up the main Java project modules locally with IntelliJ.
+- **Task Preparation**: Received more KTs for the main project, learned some JUnit, and familiarized myself with the overall codebase.
+- **Unit Test Coverage Work**: Wrote tests for my assigned ticket and created a merge request for it.
+- **Demo Was Cancelled**
+
+---
+
+### Key Challenges/Blockers
+
+- **Unfamiliar Technology**: With Java development, ETL (for batch) pipelines, and complex code architectures.
+- **Dependency Installation Problems**: Turns out my VM was wrongly configured with a different subnet and DNS server from the rest of the team.
+
+---
+
+### Learning Outcomes
+
+- **Java Dependency Management**: Learned about Java dependency management with regards to web proxies and using internal company artifact repositories.
+- **Familiarity with JUnit Testing**: JUnit and good testing practices.
+- **Familiarity with Spark Java**: A lot of the testing involves working with Java Spark Datasets.
+- **Familiarity with Java Developer Tools**: IntelliJ plugins, modules management, Maven tools, etc.
+
+---

_posts/2024-10-06-cacib_sprint_5.md

@@ -0,0 +1,49 @@
+---
+layout: post
+title: "Sprint 5 @ Crédit Agricole CIB"
+description: >-
+  Sprint recap [23/09/2024 - 07/10/2024]. Started working on frontend and backend components for another internal tool.
+author: ryo
+date: 2024-10-06 12:25:34 +0800
+categories: [Crédit Agricole, Sprints]
+tags: [sprint]
+image:
+  path: assets/img/cacib/cacib-logo-2-modified.png
+  alt: CACIB Logo
+show_image_in_post: false
+toc: true
+comments: false
+pin: false
+published: true
+---
+
+### Jira Tickets
+
+1. **Configuration Upload Utility Tool**
+  - Rebrand the JSON-Avro tool to be a generic utility web platform for the team, and include a new configuration upload module in it.
+  - The idea is that BAs should be able to upload Excel files, have the sheets validated, uploaded into S3, stored in an archive, and have the data updated in the corresponding tables (for each Excel sheet) in the PostgreSQL database. The data in these tables can contain configurations or parameters, and will be automatically pulled by the system during the running of ETL tasks.
+  - Every upload/insert to a location should have an audit trail stored in PostgreSQL.
+
+
+### Main Activities
+
+- **Fell Sick**: :+1:
+- **Trainings**: SQL, GitLab CI/CD, and Azure DevOps trainings as part of a department-wide new joiner training initiative.
+- **Gain Accesses**: Project S3, PostgreSQL, Certificates, etc.
+- **Utility Tool Work**: Started working on the frontend and backend components for the tool. While it's initially supposed to be for 1 type of configuration data, extra time was spent making the tool generic and user-friendly, and able to ingest any type of configuration or parameter data. Documented it and wrote a user guide.
+- **Demo Was Cancelled**
+
+---
+
+### Key Challenges/Blockers
+
+- **Certificates**: I was getting a `PKIX path build failure: unable to find valid certification path` error thrown by the AWS client when attempting to upload into AWS S3 from my Java application. Turns out I had to import a certificate into my java trust store with `keytool`. Also had to do this in my Dockerfile for the runtime used by the container running in Kubernetes (for the deployed tool).
+
+---
+
+### Learning Outcomes
+
+- **Security Infrastructure: Certificates**: Learned about the practical use of certificates in SSL/TLS communication to secure connections. Server certificates are often checked to see if it's issued by a trusted authority.
+- **Security Infrastructure: Trust Stores**: Learned about repositories called trust stores such as `cacerts`, the default java trust store that comes with the JDK and JRE. It's a file containing a collection of trusted certificates from well-known Certificate Authorities (CAs). `keytool` can be used to import root/intermediate certificates of servers into this keystore file.
+
+---

_posts/2024-10-20-cacib_sprint_6.md

@@ -0,0 +1,52 @@
+---
+layout: post
+title: "Sprint 6 @ Crédit Agricole CIB"
+description: >-
+  Sprint recap [07/10/2024 - 21/10/2024]. Working on UI analysis of other Angular projects in the bank, in preparation for UI development.
+author: ryo
+date: 2024-10-20 20:00:20 +0800
+categories: [Crédit Agricole, Sprints]
+tags: [sprint]
+image:
+  path: assets/img/cacib/cacib-logo-2-modified.png
+  alt: CACIB Logo
+show_image_in_post: false
+toc: true
+comments: false
+pin: false
+published: true
+---
+
+### Jira Tickets
+
+1. **UI Analysis for Angular Project A (UI Components)**
+  - The bank uses it's own in-house UI component library, which implements the bank's design language. Project A has been successful in implementing and extending this component library.
+  - My task is to identify and document components in project A that wrap these in-house UI components, including models used and inputs/outputs, and analyze what will be reusable for the upcoming UI development for my team's ETL-monitoring task.
+
+2. **UI Analysis for Angular Project B (Functionality)**
+  - While Angular project B is older and doesn't implement the bank's in-house UI component library, its functionalities are more in-line with the upcoming UI development task for my team (ETL-monitoring).
+  My task is to identify services and components that will be reusable with regards to ETL-monitoring, along with API requests/responses, and models used.
+
+### Main Activities
+
+- **Organizational Meetings**: Company Town Hall, updates on upcoming technologies to be introduced within CACIB in the near future, product presentations to department heads by department architects, etc.
+- **Trainings**: Angular trainings as part of a department-wide new joiner training initiative.
+- **Frontend JavaScript Environment Setup**: Setup of Node & NPM, including company proxy servers and Angular CLI.
+- **Gain Accesses**: Getting dev environment and source code access to both project A and B took a little time.
+- **Worked on UI Analysis and Documentation**: For both project A and B on confluence.
+
+---
+
+### Key Challenges/Blockers
+
+- **Unfamiliarity with Angular**: Most of my frontend experience at the time was either with React or React-based frameworks, and server-side rendering Java solutions (JSP, JSF, Thymeleaf). I've touched Angular before in school, but not in depth.
+- **Oof**: UI Documentation can get mind-numbing :skull:.
+
+---
+
+### Learning Outcomes
+
+- **Familiarity with Angular**: It turns out that spending 2 whole weeks reading, trying to understand, and documenting Angular code actually helps in learning how Angular works, including how large Angular application implement certain good Angular practices. 
+- **Gain Understanding on Team's Core ETL Project**: It turns out that many (or most) development operations in my department work with managing complex ETL chains, where each step in the chain can work with large amounts of data. Seeing and using the UI implementations of Project A and B firsthand helped me better understand my own team's core product.
+
+---