Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import/export tags #3130

Merged
merged 36 commits into from
Jun 2, 2022
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
c903c52
Export and Import Workflow Tags
Lucaber Jan 7, 2022
983b775
clean up fe export
mutdmour Mar 22, 2022
6d2cb1c
remove usage count
mutdmour Mar 22, 2022
99671be
return updatedat, createdat
mutdmour Mar 23, 2022
6b3a90c
fix tags import
mutdmour Mar 23, 2022
10d4543
Merge branch 'master' of github.com:n8n-io/n8n into lucaber-import-tags
mutdmour Apr 13, 2022
6556170
move logic from workflow package
mutdmour Apr 14, 2022
f4309d8
refactor import
mutdmour Apr 14, 2022
40bca69
check for tags before import
mutdmour Apr 14, 2022
ab01d50
update checks on type
mutdmour Apr 14, 2022
e6305e8
fix on import
mutdmour Apr 14, 2022
01f8d20
resolve conflicts
mutdmour May 23, 2022
6562c02
fix build issues
mutdmour May 23, 2022
99c327c
fix type issue
mutdmour May 23, 2022
1c62ffb
remove unnessary ?
mutdmour May 23, 2022
5a6f465
update tag helpers so only name is required
mutdmour May 23, 2022
0ef2fe2
fix tag import
mutdmour May 23, 2022
53a23b0
add don't replace existing tags
mutdmour May 23, 2022
cc5731c
fix build issue
mutdmour May 23, 2022
c3dcce2
Merge branch 'master' of github.com:n8n-io/n8n into lucaber-import-tags
mutdmour May 24, 2022
dc26381
address comments
mutdmour May 24, 2022
33397fb
fix with promise.all
mutdmour May 24, 2022
2449908
update setting tags
mutdmour May 24, 2022
717a66b
update check
mutdmour May 24, 2022
bcf9ee6
fix existing check
mutdmour May 24, 2022
533aecf
add helper
mutdmour May 24, 2022
d55a6f2
Merge branch 'master' of github.com:n8n-io/n8n into lucaber-import-tags
mutdmour May 24, 2022
87f69ff
fix duplication
mutdmour May 24, 2022
f132e05
Merge branch 'master' of github.com:n8n-io/n8n into lucaber-import-tags
mutdmour May 25, 2022
d322d7a
fix multiple same tags bug
mutdmour May 25, 2022
f8ceff9
Merge branch 'master' of github.com:n8n-io/n8n into lucaber-import-tags
mutdmour May 30, 2022
44758c1
Merge branch 'master' of github.com:n8n-io/n8n into lucaber-import-tags
mutdmour May 30, 2022
135c0d9
fix db bugs
mutdmour May 30, 2022
15ae031
add more validation on workflow type
mutdmour May 31, 2022
52f8905
fix validation
mutdmour May 31, 2022
cae44ec
disable importing tags on copy paste
mutdmour Jun 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions packages/cli/commands/export/workflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,10 @@ export class ExportWorkflowsCommand extends Command {
findQuery.id = flags.id;
}

// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const workflows = await Db.collections.Workflow.find(findQuery);
const workflows = await Db.collections.Workflow.find({
where: findQuery,
relations: ['tags'],
});

if (workflows.length === 0) {
throw new Error('No workflows found with specified filters.');
Expand Down
26 changes: 20 additions & 6 deletions packages/cli/commands/import/workflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,24 @@ import glob from 'fast-glob';
import { UserSettings } from 'n8n-core';
import { EntityManager, getConnection } from 'typeorm';
import { getLogger } from '../../src/Logger';
import { Db, ICredentialsDb } from '../../src';
import { Db, ICredentialsDb, IWorkflowToImport } from '../../src';
import { SharedWorkflow } from '../../src/databases/entities/SharedWorkflow';
import { WorkflowEntity } from '../../src/databases/entities/WorkflowEntity';
import { Role } from '../../src/databases/entities/Role';
import { User } from '../../src/databases/entities/User';
import { setTagsForImport } from '../../src/TagHelpers';

const FIX_INSTRUCTION =
'Please fix the database by running ./packages/cli/bin/n8n user-management:reset';

function assertWorkflowsToImport(workflows: unknown): asserts workflows is IWorkflowToImport[] {
if (!Array.isArray(workflows)) {
throw new Error(
'File does not seem to contain workflows. Make sure the workflows are contained in an array.',
);
}
}

mutdmour marked this conversation as resolved.
Show resolved Hide resolved
export class ImportWorkflowsCommand extends Command {
static description = 'Import workflows';

Expand Down Expand Up @@ -83,6 +92,7 @@ export class ImportWorkflowsCommand extends Command {
// Make sure the settings exist
await UserSettings.prepareUserSettings();
const credentials = (await Db.collections.Credentials.find()) ?? [];
const tags = (await Db.collections.Tag.find()) ?? [];
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

let totalImported = 0;

Expand Down Expand Up @@ -111,6 +121,10 @@ export class ImportWorkflowsCommand extends Command {
});
}

if (Object.prototype.hasOwnProperty.call(workflow, 'tags')) {
await setTagsForImport(transactionManager, workflow, tags);
}
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

await this.storeWorkflow(workflow, user);
}
});
Expand All @@ -123,11 +137,7 @@ export class ImportWorkflowsCommand extends Command {

totalImported = workflows.length;

if (!Array.isArray(workflows)) {
throw new Error(
'File does not seem to contain workflows. Make sure the workflows are contained in an array.',
);
}
assertWorkflowsToImport(workflows);
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

await getConnection().transaction(async (transactionManager) => {
this.transactionManager = transactionManager;
Expand All @@ -139,6 +149,10 @@ export class ImportWorkflowsCommand extends Command {
});
}

if (Object.prototype.hasOwnProperty.call(workflow, 'tags')) {
await setTagsForImport(transactionManager, workflow, tags);
}

await this.storeWorkflow(workflow, user);
}
});
Expand Down
11 changes: 11 additions & 0 deletions packages/cli/src/Interfaces.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,13 @@ export interface ITagDb {
updatedAt: Date;
}

export interface ITagToImport {
id: string | number;
name: string;
createdAt?: string;
updatedAt?: string;
}

export type UsageCount = {
usageCount: number;
};
Expand All @@ -134,6 +141,10 @@ export interface IWorkflowDb extends IWorkflowBase {
tags: ITagDb[];
}

export interface IWorkflowToImport extends IWorkflowBase {
tags: ITagToImport[];
}

export interface IWorkflowResponse extends IWorkflowBase {
id: string;
}
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/Server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1241,7 +1241,7 @@ class App {
return TagHelpers.getTagsWithCountDb(tablePrefix);
}

return Db.collections.Tag.find({ select: ['id', 'name'] });
return Db.collections.Tag.find({ select: ['id', 'name', 'createdAt', 'updatedAt'] });
},
),
);
Expand Down
70 changes: 68 additions & 2 deletions packages/cli/src/TagHelpers.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
/* eslint-disable no-param-reassign */
/* eslint-disable @typescript-eslint/explicit-module-boundary-types */
/* eslint-disable import/no-cycle */
import { getConnection } from 'typeorm';
import { EntityManager, getConnection } from 'typeorm';

import { TagEntity } from './databases/entities/TagEntity';

import { ITagWithCountDb } from './Interfaces';
import { ITagToImport, ITagWithCountDb } from './Interfaces';

// ----------------------------------
// utils
Expand Down Expand Up @@ -38,6 +38,8 @@ export async function getTagsWithCountDb(tablePrefix: string): Promise<ITagWithC
.createQueryBuilder()
.select(`${tablePrefix}tag_entity.id`, 'id')
.addSelect(`${tablePrefix}tag_entity.name`, 'name')
.addSelect(`${tablePrefix}tag_entity.createdAt`, 'createdAt')
.addSelect(`${tablePrefix}tag_entity.updatedAt`, 'updatedAt')
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
.addSelect(`COUNT(${tablePrefix}workflows_tags.workflowId)`, 'usageCount')
.from(`${tablePrefix}tag_entity`, 'tag_entity')
.leftJoin(
Expand Down Expand Up @@ -86,3 +88,67 @@ export async function removeRelations(workflowId: string, tablePrefix: string) {
.where('workflowId = :id', { id: workflowId })
.execute();
}

const createTag = async (transactionManager: EntityManager, name: string): Promise<TagEntity> => {
const tag = new TagEntity();
tag.name = name;
return transactionManager.save<TagEntity>(tag);
};
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

const findOrCreateTag = async (
transactionManager: EntityManager,
importTag: ITagToImport,
tagsEntities: TagEntity[],
): Promise<TagEntity> => {
// Assume tag is identical if createdAt date is the same to preserve a changed tag name
const identicalMatch = tagsEntities.find(
(existingTag) =>
existingTag.id.toString() === importTag.id.toString() &&
existingTag.createdAt &&
importTag.createdAt &&
new Date(existingTag.createdAt) === new Date(importTag.createdAt),
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
);
if (identicalMatch) {
return identicalMatch;
}

// Find tag with identical name
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
const nameMatch = tagsEntities.find((existingTag) => existingTag.name === importTag.name);
if (nameMatch) {
return nameMatch;
}

// Create new Tag
const createdTag = await createTag(transactionManager, importTag.name);
return createdTag;
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
};

/**
* Set tag ids to use existing tags, creates a new tag if no matching tag could be found
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
*
* @param transactionManager
* @param workflow
* @param tagsEntities
* @returns
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
*/
export async function setTagsForImport(
transactionManager: EntityManager,
workflow: { tags: ITagToImport[] },
tags: TagEntity[],
): Promise<void> {
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
const workflowTags = workflow.tags;
if (!workflowTags || !Array.isArray(workflowTags) || workflowTags.length === 0) {
return;
}
for (let i = 0; i < workflowTags.length; i++) {
if (workflowTags[i] && typeof workflowTags[i].name !== 'undefined') {
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
// eslint-disable-next-line no-await-in-loop
const tag = await findOrCreateTag(transactionManager, workflowTags[i], tags);
workflowTags[i] = {
id: tag.id,
name: tag.name,
};
}
}
}
2 changes: 2 additions & 0 deletions packages/editor-ui/src/Interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -730,6 +730,8 @@ export interface ITag {
id: string;
name: string;
usageCount?: number;
createdAt?: string;
updatedAt?: string;
}

export interface ITagRow {
Expand Down
13 changes: 12 additions & 1 deletion packages/editor-ui/src/components/MainSidebar.vue
Original file line number Diff line number Diff line change
Expand Up @@ -511,10 +511,21 @@ export default mixins(
if (data.id && typeof data.id === 'string') {
data.id = parseInt(data.id, 10);
}
const blob = new Blob([JSON.stringify(data, null, 2)], {

const exportData: IWorkflowDataUpdate = {
...data,
tags: (tags || []).map(tagId => {
const {usageCount, ...tag} = this.$store.getters["tags/getTagById"](tagId);

return tag;
}),
};

const blob = new Blob([JSON.stringify(exportData, null, 2)], {
type: 'application/json;charset=utf-8',
});


let workflowName = this.$store.getters.workflowName || 'unsaved_workflow';

workflowName = workflowName.replace(/[^a-z0-9]/gi, '_');
Expand Down
12 changes: 6 additions & 6 deletions packages/editor-ui/src/modules/tags.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@ const module: Module<ITagsState, IRootState> = {
},
},
actions: {
fetchAll: async (context: ActionContext<ITagsState, IRootState>, params?: { force?: boolean, withUsageCount?: boolean }) => {
fetchAll: async (context: ActionContext<ITagsState, IRootState>, params?: { force?: boolean, withUsageCount?: boolean }): Promise<ITag[]> => {
const { force = false, withUsageCount = false } = params || {};
if (!force && context.state.fetchedAll && context.state.fetchedUsageCount === withUsageCount) {
return context.state.tags;
return Object.values(context.state.tags);
}

context.commit('setLoading', true);
Expand All @@ -77,7 +77,7 @@ const module: Module<ITagsState, IRootState> = {

return tags;
},
create: async (context: ActionContext<ITagsState, IRootState>, name: string) => {
create: async (context: ActionContext<ITagsState, IRootState>, name: string): Promise<ITag> => {
const tag = await createTag(context.rootGetters.getRestApiContext, { name });
context.commit('upsertTags', [tag]);

Expand All @@ -88,7 +88,7 @@ const module: Module<ITagsState, IRootState> = {
context.commit('upsertTags', [tag]);

return tag;
},
},
delete: async (context: ActionContext<ITagsState, IRootState>, id: string) => {
const deleted = await deleteTag(context.rootGetters.getRestApiContext, id);

Expand All @@ -98,8 +98,8 @@ const module: Module<ITagsState, IRootState> = {
}

return deleted;
},
},
},
};

export default module;
export default module;
4 changes: 4 additions & 0 deletions packages/editor-ui/src/store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -611,6 +611,10 @@ export const store = new Vuex.Store({
Vue.set(state.workflow, 'tags', tags);
},

addWorkflowTagIds (state, tags: string[]) {
Vue.set(state.workflow, 'tags', [...(state.workflow.tags || []), ...tags]);
},

removeWorkflowTagId (state, tagId: string) {
const tags = state.workflow.tags as string[];
const updated = tags.filter((id: string) => id !== tagId);
Expand Down
25 changes: 25 additions & 0 deletions packages/editor-ui/src/views/NodeView.vue
Original file line number Diff line number Diff line change
Expand Up @@ -1253,6 +1253,31 @@ export default mixins(
this.nodeSelectedByName(node.name);
});
});

const tagsEnabled = this.$store.getters['settings/areTagsEnabled'];
if (Array.isArray(workflowData.tags) && tagsEnabled) {
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
const allTags: ITag[] = await this.$store.dispatch('tags/fetchAll');
const tagNames = new Set(allTags.map((tag) => tag.name));
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
mutdmour marked this conversation as resolved.
Show resolved Hide resolved
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

const workflowTags = workflowData.tags as ITag[];
const notFound = workflowTags.filter((tag) => !tagNames.has(tag.name));
for (const tag of notFound) {
allTags.push(await this.$store.dispatch('tags/create', tag.name));
}
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

const tagIds = workflowTags.map((imported) => {
const tag = allTags.find(tag => tag.name === imported.name);
if (tag) {
return tag.id;
}

return undefined;
})
.filter((id) => !!id);
mutdmour marked this conversation as resolved.
Show resolved Hide resolved

this.$store.commit('addWorkflowTagIds', tagIds || []);
}

} catch (error) {
this.$showError(
error,
Expand Down