[Bug]: It is not possible to make a field "required" in a catalog schema.

[Jack-Burnett]

Singer SDK Version

0.6.1

Python Version

3.10

Bug scope

Taps (catalog, state, stream maps, etc.)

Operating System

MacOS

Description

I override the;

@property
    def schema(self) -> dict:

function in a tap stream. I return a json schema directly (e.g. I am not using the PropertiesList utility to help me).

Ideally I would like to make every field on my schema required (aka guaranteed) so downstreams can depend on them.
In fact, this is even something that is done in an example in the singer spec: https://hub.meltano.com/singer/spec#messages

However, when returning that schema, the required attribute is stripped from it internally, making it impossible to have fields be required.

I believe the root cause is that this class forbids it: https://github.com/transferwise/pipelinewise-singer-python/blob/da64a10cdbcad48ab373d4dab3d9e6dd6f58556b/singer/schema.py#L108.
as was the case with this ticket: #332

Code

No response

[edgarrmondragon]

Fix might be to subclass singer.schema.Schema in

sdk/singer_sdk/helpers/_singer.py

Lines 1 to 8 in eb024bd

	import logging
	from dataclasses import dataclass, fields
	from enum import Enum
	from typing import Any, Dict, Iterable, List, Optional, Tuple, Union, cast
	from singer.catalog import Catalog as BaseCatalog
	from singer.catalog import CatalogEntry as BaseCatalogEntry
	from singer.schema import Schema

and add the required field so SerDe respects it.

Related to #332, which suffers from the same problem.

[Jack-Burnett]

Yeah that seems like probably the best way 👍