RFC - Typescript support for entities

[olivermrbl]

Hi all!

As some of you might know from looking in our roadmap, we are currently scoping how to approach Typescript support for Medusa. There are several things to which, we would like to get your input; folder structure, naming conventions, etc.. Though, to ease the process of migrating our project, we've chosen to take a "Divide and Conquer" approach.

This issue touches upon a single task related to how we should support Typescript for our entities / models. Allow me to outline, what we want to achieve starting from the end goal and an example entity and working our way out.

I want to stress, that this is very experimental, so any input is appreciated. We are currently investigating the best possible solution and this might as well not be a good foundation (and might not be possible). So feel free to throw any ideas in the mix.

Say we have an entity Item, that has the following properties:

@Entity()
export class Item {
  @PrimaryColumn()
  id: string 

  @Column()
  title: string

  @Column()
  cart_id: string

  @Column()
  unit_price: number
}

Medusa uses Typeorm

The important thing to note about this entity is, that id and cart_id are properties, that can only be set upon creating the entity (let's call this "Creatable"). Whereas unit_price and title are properties, that can be set on creation but also later updated (let's call this "Updateable").

Then we would have a service, ItemService, that handles the business logic around the entities. The service would have (among many others) the following methods:

async create(item: PartialEntity<Item, Creatable>): Promise<Item>

async update(itemId: string, update: PartialEntity<Item, Creatable & Updateable>): Promise<Item>

These two methods create and update users. We are using a type PartialEntity, which takes a generic type T (entity) and a Condition and gives back a partial entity with only the props that pass that condition.

The type PartialEntity is defined, alongside a couple of helpers, as such:

// type to get a subset of properties from T based on a Condition
export type PartialEntity<T, Condition> = Pick<T, FilteredActionables<T, Condition>>

// type to filter out properties that are of type never
export type FilteredActionables<T, Condition> = Actionable<T, Condition>[keyof T]

// E.g. Actionable<Item, string> will give us a Item with string props and rest being of type never
export type Actionable<T, Condition> = { [Key in keyof T]: T[Key] extends Condition ? Key : never }

The two methods alongside these three types should allow us to achieve the following when updating and creating items in our service layer:

async create(
    item: PartialEntity<Item, Creatable>
) {
  item.id // <- good
  item.title // <- good
  item.unit_price // <- good
  item.cart_id // <- good
}

async update(
    itemId: string,
    update: PartialEntity<Item, Updateable>
 ) {
  item.id // <- bad
  item.title // <- good
  item.unit_price // <- good
  item.cart_id // <- bad
}

This leaves us with the final step; defining the properties on the entities as Creatable and Updateable. And this is where it gets a bit tricky.

We would like to be able to say something along the lines of:

type Actionable<"somethings", "something else"> = "something third"

And then use that type when defining Entity properties:

@Entity()
export class Item {
  @PrimaryColumn()
  id: Actionable<string, Createable & Updateable> // <- id is of type string and can be included on update and creation

  @Column()
  cart_id: Actionable<string, Creatable> // <- cart id is of type string and can only be included on creation

  @Column()
  unit_price: Actionable<number, Creatable & Updateable> // <- unit_prive is of type number and can be included on creation and update
}

This concludes the outline for what we want to achieve. As mentioned earlier, we are still investigating possible solutions so this issue should be seen as more a foundation for discussion, and not so much a specific locked-in approach we want to take. There might be several other ways to go about this, that are a whole lot better, so please pitch in.

We would love to get some feedback and input ❤️

[sebastiannicolajsen]

I was kind of wondering about the complexity of Actionable as used for defining types on the columns. If we instead add something called an Action (of the form type Action<T, Condition> = T & Condition) and a couple of unique types to describe the column types (e.g. type Updatable = { uniqueIdentifier: "and_value" }). We can express column types in the format presented above. (Using Action instead of Actionable for giving columns types). If you are wondering about the weird type definition for Updatable see this thread.

[sebastiannicolajsen]

Also, the trouble here becomes if typeorm can recognise the base type element of the Action object. Of this I am unsure

[VilfredSikker]

Would it makes sense to create decorators for readonly properties and classes? With this you only have to annotate readonly properties and everything else is Updateable by default?
Here is an example of using class and property decorators https://stackblitz.com/edit/typescript-dpdrhg?file=index.ts

With this you don't have to create wrapper types for Actionable etc

Or maybe even just use the Typescript ReadOnly type https://www.typescriptlang.org/docs/handbook/utility-types.html#readonlytype

[olivermrbl]

This was also our initial thinking @VilfredSikker. Though, @zakariaelas pointed us to this StackOverflow, and it seems, that property decorators are not a valid approach. They do not mutate types, which means, that they do not provide us with any information at compile-time.

[VilfredSikker]

Did you look into the ReadOnly utility function?

[sebastiannicolajsen]

Also, the current suggestion wouldn't allow us to read let's say the id if we need that upon updating, right? There some ReadOnly decoration seems necessary?

[VilfredSikker]

Another take is just define this in typeorm: @Column({ readonly: true })

With typescript we just have to create interfaces for each type and define what is mutable

interface Product {
  id: number
  name: string
  quantity: number
}

type ProductDao = Pick<Product, "id">
type ProductUpdate = Omit<Product, "id">

So we only fetch with ProductDao and we can only update with ProductUpdate
This is a rough sketch. But it would make sure that we don't try and fetch/update product with properties we don't allow

[olivermrbl]

The following is a brief summary of how we will incrementally add Typescript to Medusa.

Motivation
We strive to build the best developer experience throughout our entire technology "ecosystem". In order to improve the development and implementation process, we want to add Typescript. This will not only make the software more robust but also provide the tools (type-checking, autocompletion, etc.) for a more efficient development process.

Decision
Our initial take on adding Typescript support can be found in the introduction of this discussion thread. Though in fear of over-engineering the feature, we've decided to go with a different and more low-level approach. Adding Typescript should be seen as an enhancement, that improves the developer experience and not a potential hurdle stemming from a complex typing implementation.

We want the following benefits from Typescript:

• Interfaces for structures
• Types for function arguments and return values
• Reusable types to share between all of our projects
• Clever utils from Typescript (Omit, Pick, etc.)
• Ability to incrementally add the support

Implementation
In the first iteration, we've decided to focus on adding types to our service layer and to describe the steps in the process, we will use our UserService as an example.

The process is as follows:

• Rename service from user.js -> user.ts
• Create interfaces in ./src/types/user.ts (the following are interfaces describing the properties allowed upon creating and updating a user, so the function argument types for UserService.create and UserService.update)

export interface CreateUserInput {
  id?: string
  email: string
  first_name?: string
  last_name?: string
  api_token?: string
  metadata?: JSON
}

export interface UpdateUserInput {
  readonly email?: string
  first_name?: string
  last_name?: string
  readonly password_hash: string
  api_token?: string
  metadata?: JSON
}

Note, that properties can be optional ? and readonly.

• Create interface for constructor

type UserServiceProps = {
  userRepository: typeof UserRepository
  eventBusService: EventBusService
  manager: EntityManager
}

class UserService extends BaseService {
  ...
  private userRepository_: typeof UserRepository
  private eventBus_: EventBusService
  private manager_: EntityManager
  private transactionManager_: EntityManager

  constructor({ userRepository, eventBusService, manager }: UserServiceProps) {
    super()

    /** @private @const {UserRepository} */
    this.userRepository_ = userRepository

    /** @private @const {EventBus} */
    this.eventBus_ = eventBusService

    /** @private @const {EntityManager} */
    this.manager_ = manager
  }
  ...
  ...
}

• Add function argument types and return types throughout the service

async update(userId: string, update: UpdateUserInput): Promise<User> {
  ...
}

This is a rough outline of the steps. Obviously, more tweaks and additions are needed, but for the sake of this summary, those are omitted. For a more detailed description of the implementation, we refer to branch typescript.

Tip: Compare the branch with develop to see changed files

Finally, it's important to once again stress, that we add Typescript to support the development process, so we prioritize readability, easy transition from JS, and an initial loosely typed approach.

All feedback and input is welcomed - thanks in advance!

[olivermrbl]

Following up with a brief description of how we could potentially add Typescript to our API validation layer.

Motivation
As mentioned in previous thread message, we are currently adding Typescript support and our current library for API validation Joi does not support types in an intuitive way.

(Potential) Decision
Decided to proceed with libraries class-validator and class-transformer, that when combined replaces the current Joi validation entirely. The libraries requires us to define classes for each request payload, transform the payload to classes, and finally validate the transformed payload. Additionally, @srindom has benchmarked the two solutions and class validation clearly performs better. See here.

Implementation (first take)
To showcase the implementation, we will use the admin route for creating a user.

Initially, our route looks something like this:

import _ from "lodash"

export default async (req, res) => {
  const schema = Validator.object().keys({
    email: Validator.string().email().required(),
    name: Validator.string().optional(),
    password: Validator.string().required(),
  })

  const { value, error } = schema.validate(req.body)
  if (error) {
    throw new MedusaError(MedusaError.Types.INVALID_DATA, error.details)
  }

  const userService = req.scope.resolve("userService")
  const data = _.pick(value, ["email", "name"])

  const user = await userService.create(data, value.password)

  res.status(200).json({ user })
}

Now, the steps to refactor the above route to use class validation instead of Joi can be boiled down to the following two steps:

• Transform the Joi schema to a class
• Replace Joi validation with a custom class validator

See below:

export default async (req, res) => {
  const validated = await validator(AdminCreateUserRequest, req.body)
  ...
  res.status(200).json({ user })
}

export class AdminCreateUserRequest {
  @IsEmail()
  email: string
  name?: string
  password: string
}

The validator function seen above will first transform the request payload into the class provided as first argument (here AdminCreateUserRequest). Then it will validate against the types defined in the class. If this validation fails, it throws an error with error messages included. The first take on the custom validator is as follows:

async function validator<T extends object, V>(
  typedClass: ClassConstructor<T>,
  plain: V
): Promise<T> {
  // transform payload to class
  const toValidate = plainToClass(typedClass, plain)

  // validate class
  const errors = await validate(toValidate)
  
  // throw if any errors
  if (errors?.length) {
    // push all errors to an array of error messages
    const errorMessages = errors.reduce((acc: string[], next) => {
      if (next.constraints) {
          for (const [_, msg] of Object.entries(next.constraints)) {
            acc.push(msg)
          }
      }
      return acc
    }, [])

    throw new MedusaError(
      MedusaError.Types.INVALID_DATA,
      errorMessages.join(",")
    )
  }

  return toValidate
}

Let me know, what you think :)