Hooks (DB Triggers) / Data Validation & Transformation

[MentalGear]

Hooks (DB Triggers) / Data Validation & Transformation

I'm sharing here some thoughts on how a data validation/transformation pipeline using Hooks could be designed.
I put them here to open a discussion around them and other concepts / patterns.

In general, I think validation (and transformation) can be separated into 3 stages:

• record level
• full-database level
• external-API service level

Together with migrations, which are also essential, the hooks mark-up could be structured like:

Pre-Insert Hook

Stage properties are sequential: if one fails, the next won't execute.

1. migration level

Input: ( inputData, collectionName, ctx= {..., schemaVersion } )
Checks incoming schema version and applies data migration transformations if necessary

2. record level

Input: ( inputData, collectionName, ctx )
Here we validate only the incoming data whether they align with our data validation schema. One can use their validation lib of choice.
Example:

const validData = dataValidationLib.parse( {data: inputData, schemaName:collectionName} )
return validData

3. full-database integration level

Input: ( prevValidInputData, collectionName, database, ctx )
Database query like syntax can be used to check against relevant data of the current collection or across all collections of the database, assessing whether the new data can be added.
Example:

if ( ('currentUser.$karmaPoints', '>', 10) && ('database.blockedUsers', 'nin', $id) )
return validData
else throw new Error()

4. external API service | async

Input: ( prevValidInputData, collectionName, database, ctx )
External API / web access to check against incoming data.

const emailIsValid = emailValidationService(validData.email)
if (emailIsValid) return validData
else throw new Error()

---

I think this is a quite flexible path while still providing enough structure so it be quickly setup (e.g. using default templates).