Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify marking entities as deleted #263

Open
ashmanto opened this issue Oct 5, 2023 · 1 comment
Open

simplify marking entities as deleted #263

ashmanto opened this issue Oct 5, 2023 · 1 comment
Labels
enhancement New feature or request Priority:High

Comments

@ashmanto
Copy link
Contributor

ashmanto commented Oct 5, 2023

usually when filtering out entities in transforms we would want to mark it as deleted if it exists in the sink dataset.

currently doing:
(typescript)

if (type !== "foo") {
            return;
        }

will just igjore the entitiy that comes in, and not mark it as deleted if type changes.

we would need to do this everywhere:
(typescript)

if (type !== "foo") {
            let hasBeenEmitted = dh.FindById(dh.GetId(newEntity), ["<sink-dataset>"]);
            if (hasBeenEmitted) {
                dh.SetDeleted(<entity>, true);
            } else {
                return;
            }
        }

simplifying this with a function would be appreciated.

@ashmanto ashmanto added the enhancement New feature or request label Oct 5, 2023
@rompetroll
Copy link
Contributor

To make this specific beaviour a built in for typescript only, it could be added to the datahub typescript lib, similar to how SQuery here is done https://github.com/mimiro-io/datahub-tslib/blob/master/datahub.ts#L33.

In general, it could also be a good option to make it easy to import local typescript libs. So that you can have your local "common" file with typescript helper functions, which then can be imported into all local transform scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Priority:High
Projects
None yet
Development

No branches or pull requests

3 participants