-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Thomas
committed
Jun 15, 2024
1 parent
73a148d
commit 5108c34
Showing
2 changed files
with
34 additions
and
32 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,53 +15,59 @@ I am Thomas and currently occupied as an (Azure) data engineer at [Capgemini Aus | |
Not very surpsingly, my main focus is on all things data. More specifically, that currently means: | ||
|
||
- Databricks, Spark & Delta | ||
- Synapse (Serverless & Dedicated[^dedic], Spark, Data Explorer/Kusto, ...) | ||
- Delta Live Tables & Spark Structured Streaming | ||
- Data Explorer/Kusto | ||
- Fabric | ||
- Data Pipelines (Synapse & Data Factory) | ||
- SQL-Servers (of any kind) | ||
- Datalakehouses | ||
- Streaming (data, not movies...) | ||
- Restful APIs & GraphQL | ||
- Azure Data Factory | ||
- Cloud Infrastructure as Code (Terraform, Bicep) | ||
|
||
After 18 years of data warehousing, i am now pushing lakehouse architectures and the idea of a unified data platform. Besides signing the [DataOps Manifesto](https://dataopsmanifesto.org/), i am also a supporter of the "treat data as a product" movement. | ||
And not so currently, meaning in the recent past i had a lot of: | ||
|
||
- Synapse (Serverless & Dedicated[^dedic]) | ||
- SQL-Server Warehouses | ||
- Restful APIs & GraphQL | ||
|
||
After 18 years of classical data warehousing, i finally got to dive into various (data-)lakes. And once in the lake(-house), there is no turning back, that's for sure. | ||
|
||
Besides signing the [DataOps Manifesto](https://dataopsmanifesto.org/), i am also a supporter of the "treat data as a product" movement. | ||
|
||
## Side-Shows | ||
|
||
Since one of my main personal treats is being lazy in combination with a deep hatred for repetitive tasks, i developed a natural interest in DevOps & automation. To make up for lack of concentration (and sleep...), i am usually not touching any code unless it's version controlled. That includes but is not limited to: cloud infrastructure (as code), database schemas, pipeline definitions, all kinds of scripts/notebooks and the grocery list for my next shopping tour. | ||
Since one of my main treats is lazyness when it comes to repetitive tasks, i developed a natural interest in DevOps & automation. And due to constant lack of sleep, I usually don't touch any code, unless it's version controlled. That includes but is not limited to: cloud infrastructure (as code), database schemas, pipeline definitions, all kinds of scripts, notebooks and the grocery list for my next shopping tour. | ||
|
||
As a result of the above, i somehow stumbled into the beautiful world of DevOps (although that name was not a thing back then). My personal project lifecycle these days often goes like this: Getting hired as data architect/engineer and then silently converting into the dedicated DevOps and Git guy. It was never my intention, but i think i got the DevOps Bingo card full now: | ||
|
||
As a result of the above, a topic that attracted my interest (and since then also became part of my professional portfolio) is the implementation/optimization of development & deployment workflows in data projects. Some of the tools/services that i have been using in that regard are: | ||
- Azure DevOps (Repos, Boards, Pipelines, etc.) | ||
- The Atlassian(s): Bitbucket, Jira, Confluence (and even Bamboo...). | ||
- Github (my teenage love) | ||
- And lately also a lot of: Gitlab | ||
- And if artefact registries also count on that list: Cloudsmith (great tool by the way) | ||
|
||
- Azure DevOps: Repos, Boards, Pipelines, Artifacts | ||
- Atlassian Cloud: Jira, Bitbucket, Confluence, Bamboo | ||
- Github: Repos, Actions, Issues, Projects, Codespaces | ||
- And the mother of them all: __Git__[^git] | ||
- _Gitlab: soon to be added to this list_ | ||
However, there can be only one god. And for me that is and always will be: __Git__[^git]. After using it for almost a decade now, i still feel like i don't know half of it and am constantly amazed how limitless the possibilities of this genious piece of software really are. | ||
|
||
Since __DevOps and automation__ usually require some sort of scripting, i am constantly trying to improve my Powershell, Python, Bash & Fish hacking skills. | ||
## Scripting | ||
|
||
Regarding __data manipulation & transformation__ i am most experienced in T-SQL, due to a strong SQL Server background. These days though, i mainly work with Python/Pyspark & Spark-SQL in (Jupyter or Databricks) notebooks, which is a lot of fun and a welcome change. | ||
Since __DevOps and automation__ usually require some sort of scripting, i also found myself dealing with the usual (scripting) suspects: Powershell, Bash, Python (and for fun and good looks also: Fish). | ||
|
||
My first IT related job as a web developer (end of last century...) still benefits me to date! Having basic knowledge about HTTP and OAUTH (flows) can be really helpful when using or providing REST-APIs. Not my favorite topic in general, but like in real life (relationships): some things just need time to grow :grin: | ||
Regarding __data manipulation & transformation__ i am most experienced in T-SQL, due to spending so many years implementing SQL warehouses. And for quite a while, writing data transformations in Python felt somehow "wrong" to me. But today there is no doubt about it in my mind: The added value that you get (for free) from the whole Python software engineering ecosystemn (formatting, linting, automated testing, packaging and so many more) cannot be missed out on. Implementing data projectss almost starts to feel like real software devleopment these days. | ||
|
||
In the past i also pretended to know a bit of: Lua, Tcl/Tk, VBA, Javascript, Actionscript and probably a few other languages which i forgot a long time ago. | ||
And a life without notebooks might be possible, but makes abosulety no sense to me :smirk:. | ||
|
||
## Professional Past | ||
|
||
Some technologies, software providers and topics that i spent time with in my (professional) past are: | ||
Way back in time, i also had to work a lot with (or in): | ||
|
||
- SQL-Server (on-Premise): Relational-DB, SSIS, SSAS, SSRS. | ||
- SAP: BW (Datamodeling, Processchains, Bex Queries, etc.), Business Objects (Analysis for Office, Lumira Designer, BI/BO Platform), ERP/R3 (mainly FI/CO/HCM/COPA), SEM-BCS (management consolidation). | ||
- IBM Planning Analytics (aka TM1), Palo/Jedox and other _real_ OLAP engines & databases including all sorts of Excel integrations that usually come along with those. | ||
- Classical Data Warehousing & Modeling. | ||
- Corporate Performance Management (CPM): Main focus being financial, sales, project and product planning & forecasting. | ||
- Financial Reporting: Implementation of financial reports for all kind of flavors and in different tools & frontends as well as supportive addins (like Graphomate). Also creation of notation concepts/standards for financial reporting (IBCS-like). | ||
- As an inhouse consultant, i also spent quite some time in improving the collaboration toolset and processes in the central Controlling department (40 people spread over various locations, countries and languages). Which basically means that i played around a lot with: Microsoft Teams (plus Addins/Apps), Sharepoint (on-premise and online), Planner/ToDo, OneDrive, O365, ...). | ||
- At the same time i also made some operational use of my educational/business background by conducting the monthly business consolidation (as well as providing financial reports) for over 100 legal entities in [Styria Media Group](https://www.styria.com). | ||
|
||
## Education & Certifications | ||
|
||
My educational background is a master's degree in "Management and International Business"[^mib] (yikes) and 35+ years of being a self-taught computer nerd. If you care about certifications (personally i don't), you can check my [Credly](https://www.credly.com/users/ttotter) or [MS-Learn](https://learn.microsoft.com/en-us/users/thomastotter-5644/) pages and might find out that (besides the obvious data engineering certs) i am also not a complete stranger to agile methods. | ||
My educational background is a master's degree in "Management and International Business"[^mib] (yikes) and 35+ years of being a self-taught computer nerd. If you care about certifications (personally, i don't), you can check my [Credly](https://credly.credly.ttotter.pw) or [MS-Learn](https://mslearn.totter.pw) pages and might find out that (besides the obvious data engineering certs) i am also not a complete stranger to agile methods. | ||
|
||
However, my opinion towards certifications is best explained by this picture (borrowed from the great Martin Fowler[^mf]): | ||
<br/> | ||
|
@@ -70,7 +76,7 @@ However, my opinion towards certifications is best explained by this picture (bo | |
|
||
## Creed | ||
|
||
Since i couldn't put my work ethics/beliefs to better words myself, i am _borrowing_ most of [Automattic's creed](https://automattic.com) here: | ||
Since i couldn't put my work ethics/beliefs to better words, i am _borrowing_ most of [Automattic's creed](https://automattic.com) here: | ||
|
||
[THE AUTOMATTIC CREED](https://automattic.com/creed/) | ||
|
||
|
@@ -90,15 +96,12 @@ And a personal addition: | |
|
||
## Get In Contact | ||
|
||
You can find, stalk, or contact me here: | ||
Besides this blog (that i update maybe once a year) you can also find (stalk) or contact me here: | ||
|
||
- [LinkedIn](https://www.linkedin.com/in/thomas-totter/) | ||
- [LinkedIn](https://linkedin.totter.pw) | ||
- [Github](https://github.com/brain246) | ||
- [Credly](https://www.credly.com/users/ttotter) | ||
- [MS-Learn](https://learn.microsoft.com/en-us/users/thomastotter-5644/) | ||
- [Mastodon.Social](https://mastodon.social/@brain246) | ||
- [Mastodon.Medium](https://mastodon.social/@brain246)[^mast] | ||
- [Email](mailto:[email protected]) | ||
- [Medium](https://medium.totter.pw) | ||
- [Credly](https://credly.totter.pw) | ||
|
||
[^blog]: Powered by [Github Pages](https://pages.github.com/) and [Chirpy](https://github.com/cotes2020/jekyll-theme-chirpy) | ||
|
||
|
@@ -113,5 +116,3 @@ You can find, stalk, or contact me here: | |
[^assig]: Greetings go out at to all my former POs, project leads and bosses :wave: | ||
|
||
[^corny]: Ok this one sounds a bit corny considering i am mainly working with Microsoft products and services. But it's the idea that counts! | ||
|
||
[^mast]: <a rel="me" href="https://mastodon.social/@brain246">Mastodon.Social</a> and <a rel="me" href="https://me.dm/@brain246">Mastodon.Medium</a> forced me to place these links here :imp: |