Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: taxonomy #1128

Merged
merged 71 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
ccea9fd
feat: taxonomies & terms support in export-to-csv
aman19K Sep 7, 2023
2bc8e33
fix bugs
aman19K Sep 11, 2023
cc51967
refactor: taxonomies
aman19K Sep 11, 2023
7fb4ceb
fix: correct prompt msg
aman19K Sep 12, 2023
b9144a4
fix: handle empty file in case of taxonomy and terms
aman19K Sep 12, 2023
bdde4ec
refactor: taxonomy uid optional flag and related changes
aman19K Sep 12, 2023
c71a112
fix: removed description from terms csv file
aman19K Sep 13, 2023
3f2d947
Merge pull request #1032 from contentstack/feat/CS-41088
aman19K Sep 13, 2023
7129825
feat: export-to-csv test cases using @oclif/test
aman19K Sep 18, 2023
0bfc7ac
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Sep 20, 2023
86804f5
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Sep 20, 2023
a634524
feat: export taxonomies & terms
aman19K Sep 20, 2023
afc55be
fix: remove unused package
aman19K Sep 21, 2023
fda1005
refactor: message
aman19K Sep 21, 2023
e1b9704
Merge pull request #1052 from contentstack/feat/CS-41090-export-taxonomy
aman19K Sep 22, 2023
6d98030
fix test cases
aman19K Sep 25, 2023
fdee37c
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Sep 25, 2023
62a6b96
fix: mkdirp issue
aman19K Sep 25, 2023
c5da4ab
feat: import taxonomies
aman19K Sep 25, 2023
aec02a3
fix: test cases
aman19K Sep 26, 2023
3344ff9
feat: import terms
aman19K Sep 26, 2023
db06e26
refactor: taxonomy export & import
aman19K Sep 26, 2023
64e76f8
refactor: removed limit from taxonomy and term types
aman19K Sep 26, 2023
7dda261
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Sep 26, 2023
eabc9b4
Merge branch 'feat/CS-38078-taxonomy' into feat/CS-41380
aman19K Sep 26, 2023
07d11f8
Merge pull request #1062 from contentstack/feat/CS-41385-import-taxonomy
aman19K Sep 27, 2023
b214b96
fix: entries test cases
aman19K Sep 27, 2023
ea93882
Merge branch 'feat/CS-38078-taxonomy' into feat/CS-41380
aman19K Sep 27, 2023
2ded3c0
fix: test cases
aman19K Sep 27, 2023
f0b8bab
refactor: updated package
aman19K Sep 27, 2023
1651cce
fix: directory creation in test cases
aman19K Sep 27, 2023
44b56c8
refactor: identical user api call
aman19K Sep 27, 2023
ecaf413
Merge pull request #1064 from contentstack/feat/CS-41380
aman19K Sep 27, 2023
e79f6f8
feat: content type module support taxonomy
aman19K Oct 3, 2023
f0f880a
feat: entries module supports taxonomy
aman19K Oct 4, 2023
9f32468
refactor: comment message
aman19K Oct 4, 2023
377e5ff
Merge pull request #1068 from contentstack/feat/CS-41092
aman19K Oct 4, 2023
e7bfed4
feat: sanitize taxonomies & terms against csv injection
aman19K Oct 5, 2023
c3b27c8
Merge branch 'feat/CS-38078-taxonomy' into feat/CS-41088-taxonomies-c…
aman19K Oct 5, 2023
9b16d62
refactor: sanitize taxonomies & term
aman19K Oct 5, 2023
d5d8ea3
Merge pull request #1076 from contentstack/feat/CS-41088-taxonomies-c…
aman19K Oct 5, 2023
53b0288
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Oct 5, 2023
c917c12
refactor: taxonomy UI text messages
aman19K Oct 6, 2023
d02321a
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Oct 9, 2023
a0c2336
feat: replace taxonomy & term http call with sdk
aman19K Oct 9, 2023
424f20d
feat: taxonomies & multiple field support in migration
aman19K Oct 10, 2023
cfc60e0
fix: create content type example
aman19K Oct 10, 2023
f1d3890
Revert "feat: replace taxonomy & term http call with sdk"
aman19K Oct 10, 2023
6e8e26c
refactor: removed console
aman19K Oct 10, 2023
4e21f17
fix: pagination & export to csv unit test cases
aman19K Oct 11, 2023
c2ef0fb
Merge pull request #1097 from contentstack/fix/CS-41094
aman19K Oct 11, 2023
19d9b92
Merge branch 'feat/CS-38078-taxonomy' into feat/CS-41655
aman19K Oct 11, 2023
710bfd1
feat: replace taxonomy & term http call with sdk
aman19K Oct 9, 2023
b1e7e08
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Oct 11, 2023
6a98419
Merge branch 'feat/CS-38078-taxonomy' into feat/CS-41655
aman19K Oct 11, 2023
4e87cbb
fix: taxonomies & term error handling
aman19K Oct 11, 2023
0fbd46a
fix: taxonomies export error
aman19K Oct 11, 2023
94cf81e
fix: handle parent_uid error in term creation
aman19K Oct 12, 2023
416cbc1
refactor: error message
aman19K Oct 12, 2023
b13bf0b
refactor: taxonomy export & import error msg
aman19K Oct 12, 2023
3b1d89e
Merge pull request #1095 from contentstack/feat/CS-41655
aman19K Oct 12, 2023
e42737b
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Oct 18, 2023
bae711b
refactor: log warning msg, update sdk version & handle taxonomies query
aman19K Oct 18, 2023
55a5852
refactor: replace spread operator with push
aman19K Oct 18, 2023
9c80757
Merge pull request #1111 from contentstack/feat/CS-42076-demo-feedback
aman19K Oct 18, 2023
dc57ad0
Merge branch 'development' into feat/CS-38078-taxonomy
aman19K Oct 20, 2023
9f46ac2
version bumped
aman19K Oct 26, 2023
f4a2d2c
Merge branch 'staging' into feat/CS-38078-taxonomy
aman19K Oct 26, 2023
7e5e95a
Merge pull request #1127 from contentstack/refactor/42306-version-bump
aman19K Oct 26, 2023
dd6120d
fix: syntax issue
aman19K Oct 27, 2023
6f32dd4
Merge pull request #1130 from contentstack/refactor/42306
aman19K Oct 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
420 changes: 354 additions & 66 deletions package-lock.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion packages/contentstack-bootstrap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ $ npm install -g @contentstack/cli-cm-bootstrap
$ csdx COMMAND
running command...
$ csdx (--version)
@contentstack/cli-cm-bootstrap/1.6.0 darwin-arm64 node-v20.8.0
@contentstack/cli-cm-bootstrap/1.6.1 darwin-arm64 node-v20.8.0
$ csdx --help [COMMAND]
USAGE
$ csdx COMMAND
Expand Down
4 changes: 2 additions & 2 deletions packages/contentstack-bootstrap/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@contentstack/cli-cm-bootstrap",
"description": "Bootstrap contentstack apps",
"version": "1.6.0",
"version": "1.6.1",
"author": "Contentstack",
"bugs": "https://github.com/contentstack/cli/issues",
"scripts": {
Expand All @@ -17,7 +17,7 @@
"test:report": "nyc --reporter=lcov mocha \"test/**/*.test.js\""
},
"dependencies": {
"@contentstack/cli-cm-seed": "~1.6.0",
"@contentstack/cli-cm-seed": "~1.6.1",
"@contentstack/cli-command": "~1.2.14",
"@contentstack/cli-utilities": "~1.5.4",
"inquirer": "8.2.4",
Expand Down
2 changes: 1 addition & 1 deletion packages/contentstack-clone/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ $ npm install -g @contentstack/cli-cm-clone
$ csdx COMMAND
running command...
$ csdx (--version)
@contentstack/cli-cm-clone/1.6.0 darwin-arm64 node-v20.8.0
@contentstack/cli-cm-clone/1.7.0 darwin-arm64 node-v20.8.0
$ csdx --help [COMMAND]
USAGE
$ csdx COMMAND
Expand Down
6 changes: 3 additions & 3 deletions packages/contentstack-clone/package.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"name": "@contentstack/cli-cm-clone",
"description": "Contentstack stack clone plugin",
"version": "1.6.0",
"version": "1.7.0",
"author": "Contentstack",
"bugs": "https://github.com/rohitmishra209/cli-cm-clone/issues",
"dependencies": {
"@contentstack/cli-cm-export": "~1.9.0",
"@contentstack/cli-cm-import": "~1.10.0",
"@contentstack/cli-cm-export": "~1.10.0",
"@contentstack/cli-cm-import": "~1.11.0",
"@contentstack/cli-command": "~1.2.14",
"@contentstack/cli-utilities": "~1.5.4",
"@colors/colors": "^1.5.0",
Expand Down
14 changes: 9 additions & 5 deletions packages/contentstack-export-to-csv/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@contentstack/cli-cm-export-to-csv",
"description": "Export entities to csv",
"version": "1.4.4",
"version": "1.5.0",
"author": "Abhinav Gupta @abhinav-from-contentstack",
"bugs": "https://github.com/contentstack/cli/issues",
"dependencies": {
Expand All @@ -15,12 +15,15 @@
},
"devDependencies": {
"@oclif/test": "^2.2.10",
"chai": "^4.2.0",
"@types/chai": "^4.3.6",
"@types/mocha": "^10.0.1",
"chai": "^4.3.8",
"debug": "^4.3.1",
"dotenv": "^16.3.1",
"eslint": "^7.32.0",
"eslint-config-oclif": "^4.0.0",
"globby": "^10.0.2",
"mocha": "^10.0.0",
"mocha": "^10.2.0",
"nyc": "^15.1.0",
"oclif": "^3.8.1"
},
Expand All @@ -44,7 +47,8 @@
"postpack": "rm -f oclif.manifest.json",
"prepack": "oclif manifest && oclif readme",
"test": "nyc mocha --forbid-only \"test/**/*.test.js\"",
"test:unit": "nyc mocha --timeout 10000 --forbid-only \"test/unit/**/*.test.js\"",
"test:unit": "mocha --timeout 10000 --forbid-only \"test/unit/**/*.test.js\" \"test/util/common-utils.test.js\"",
"test:unit:report": "nyc --extension .js mocha --forbid-only \"test/unit/**/*.test.js\" \"test/util/common-utils.test.js\"",
"version": "oclif readme && git add README.md",
"clean": "rm -rf ./node_modules tsconfig.build.tsbuildinfo"
},
Expand All @@ -61,4 +65,4 @@
}
},
"repository": "https://github.com/contentstack/cli"
}
}
234 changes: 173 additions & 61 deletions packages/contentstack-export-to-csv/src/commands/cm/export-to-csv.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class ExportToCsvCommand extends Command {
action: flags.string({
required: false,
multiple: false,
options: ['entries', 'users'],
options: ['entries', 'users', 'taxonomies'],
description: `Option to export data (entries, users)`,
}),
alias: flags.string({
Expand Down Expand Up @@ -59,6 +59,9 @@ class ExportToCsvCommand extends Command {
multiple: false,
required: false,
}),
'taxonomy-uid': flags.string({
description: 'Provide the taxonomy UID of the related terms you want to export',
}),
};

async run() {
Expand All @@ -75,6 +78,7 @@ class ExportToCsvCommand extends Command {
'content-type': contentTypesFlag,
alias: managementTokenAlias,
branch: branchUid,
'taxonomy-uid': taxonomyUID,
},
} = await this.parse(ExportToCsvCommand);

Expand All @@ -96,69 +100,17 @@ class ExportToCsvCommand extends Command {
let stackAPIClient;
let language;
let contentTypes = [];
let stackBranches;
const listOfTokens = configHandler.get('tokens');

if (managementTokenAlias && listOfTokens[managementTokenAlias]) {
managementAPIClient = await managementSDKClient({
host: this.cmaHost,
management_token: listOfTokens[managementTokenAlias].token,
});
stack = {
name: stackName || managementTokenAlias,
apiKey: listOfTokens[managementTokenAlias].apiKey,
token: listOfTokens[managementTokenAlias].token,
};
} else if (managementTokenAlias) {
this.error('Provided management token alias not found in your config.!');
if (managementTokenAlias) {
const { stackDetails, apiClient } = await this.getAliasDetails(managementTokenAlias, stackName);
managementAPIClient = apiClient;
stack = stackDetails;
} else {
let organization;

if (!isAuthenticated()) {
this.error(config.CLI_EXPORT_CSV_ENTRIES_ERROR, {
exit: 2,
suggestions: ['https://www.contentstack.com/docs/developers/cli/authentication/'],
});
}

if (org) {
organization = { uid: org };
} else {
organization = await util.chooseOrganization(managementAPIClient); // prompt for organization
}
if (!stackAPIKey) {
stack = await util.chooseStack(managementAPIClient, organization.uid); // prompt for stack
} else {
stack = await util.chooseStack(managementAPIClient, organization.uid, stackAPIKey);
}
stack = await this.getStackDetails(managementAPIClient, stackAPIKey, org);
}

stackAPIClient = this.getStackClient(managementAPIClient, stack);

if (branchUid) {
try {
const branchExists = await doesBranchExist(stackAPIClient, branchUid);
if (branchExists?.errorCode) {
throw new Error(branchExists.errorMessage);
}
stack.branch_uid = branchUid;
stackAPIClient = this.getStackClient(managementAPIClient, stack);
} catch (error) {
if (error.message || error.errorMessage) {
cliux.error(util.formatError(error));
this.exit();
}
}
} else {
stackBranches = await this.getStackBranches(stackAPIClient);
if (stackBranches === undefined) {
stackAPIClient = this.getStackClient(managementAPIClient, stack);
} else {
const { branch } = await util.chooseBranch(stackBranches);
stack.branch_uid = branch;
stackAPIClient = this.getStackClient(managementAPIClient, stack);
}
}
await this.checkAndUpdateBranchDetail(branchUid, stack, stackAPIClient, managementAPIClient);

const contentTypeCount = await util.getContentTypeCount(stackAPIClient);

Expand Down Expand Up @@ -258,6 +210,22 @@ class ExportToCsvCommand extends Command {
}
break;
}
case config.exportTaxonomies:
case 'taxonomies': {
let stack;
let stackAPIClient;
if (managementTokenAlias) {
const { stackDetails, apiClient } = await this.getAliasDetails(managementTokenAlias, stackName);
managementAPIClient = apiClient;
stack = stackDetails;
} else {
stack = await this.getStackDetails(managementAPIClient, stackAPIKey, org);
}

stackAPIClient = this.getStackClient(managementAPIClient, stack);
await this.createTaxonomyAndTermCsvFile(stackAPIClient, stackName, stack, taxonomyUID);
break;
}
}
} catch (error) {
if (error.message || error.errorMessage) {
Expand All @@ -273,8 +241,8 @@ class ExportToCsvCommand extends Command {
getStackClient(managementAPIClient, stack) {
const stackInit = {
api_key: stack.apiKey,
branch_uid: stack.branch_uid,
};
if(stack?.branch_uid) stackInit['branch_uid'] = stack.branch_uid;
if (stack.token) {
return managementAPIClient.stack({
...stackInit,
Expand All @@ -292,9 +260,149 @@ class ExportToCsvCommand extends Command {
.then(({ items }) => (items !== undefined ? items : []))
.catch((_err) => {});
}

/**
* check whether branch enabled org or not and update branch details
* @param {string} branchUid
* @param {object} stack
* @param {*} stackAPIClient
* @param {*} managementAPIClient
*/
async checkAndUpdateBranchDetail(branchUid, stack, stackAPIClient, managementAPIClient) {
if (branchUid) {
try {
const branchExists = await doesBranchExist(stackAPIClient, branchUid);
if (branchExists?.errorCode) {
throw new Error(branchExists.errorMessage);
}
stack.branch_uid = branchUid;
stackAPIClient = this.getStackClient(managementAPIClient, stack);
} catch (error) {
if (error?.message || error?.errorMessage) {
cliux.error(util.formatError(error));
this.exit();
}
}
} else {
const stackBranches = await this.getStackBranches(stackAPIClient);
if (stackBranches === undefined) {
stackAPIClient = this.getStackClient(managementAPIClient, stack);
} else {
const { branch } = await util.chooseBranch(stackBranches);
stack.branch_uid = branch;
stackAPIClient = this.getStackClient(managementAPIClient, stack);
}
}
}

/**
* fetch stack details from alias token
* @param {string} managementTokenAlias
* @param {string} stackName
* @returns
*/
async getAliasDetails(managementTokenAlias, stackName) {
let apiClient, stackDetails;
const listOfTokens = configHandler.get('tokens');
if (managementTokenAlias && listOfTokens[managementTokenAlias]) {
apiClient = await managementSDKClient({
host: this.cmaHost,
management_token: listOfTokens[managementTokenAlias].token,
});
stackDetails = {
name: stackName || managementTokenAlias,
apiKey: listOfTokens[managementTokenAlias].apiKey,
token: listOfTokens[managementTokenAlias].token,
};
} else if (managementTokenAlias) {
this.error('Provided management token alias not found in your config.!');
}
return {
apiClient,
stackDetails,
};
}

/**
* fetch stack details on basis of the selected org and stack
* @param {*} managementAPIClient
* @param {string} stackAPIKey
* @param {string} org
* @returns
*/
async getStackDetails(managementAPIClient, stackAPIKey, org) {
let organization, stackDetails;

if (!isAuthenticated()) {
this.error(config.CLI_EXPORT_CSV_ENTRIES_ERROR, {
exit: 2,
suggestions: ['https://www.contentstack.com/docs/developers/cli/authentication/'],
});
}

if (org) {
organization = { uid: org };
} else {
organization = await util.chooseOrganization(managementAPIClient); // prompt for organization
}
if (!stackAPIKey) {
stackDetails = await util.chooseStack(managementAPIClient, organization.uid); // prompt for stack
} else {
stackDetails = await util.chooseStack(managementAPIClient, organization.uid, stackAPIKey);
}
return stackDetails;
}

/**
* Create a taxonomies csv file for stack and a terms csv file for associated taxonomies
* @param {string} stackName
* @param {object} stack
* @param {string} taxUID
*/
async createTaxonomyAndTermCsvFile(stackAPIClient, stackName, stack, taxUID) {
const payload = {
stackAPIClient,
type: '',
limit: config.limit || 100
};
//check whether the taxonomy is valid or not
let taxonomies = [];
if (taxUID) {
payload['taxonomyUID'] = taxUID;
const taxonomy = await util.getTaxonomy(payload);
taxonomies.push(taxonomy);
} else {
taxonomies = await util.getAllTaxonomies(payload);
}

const formattedTaxonomiesData = util.formatTaxonomiesData(taxonomies);
if (formattedTaxonomiesData?.length) {
const fileName = `${stackName ? stackName : stack.name}_taxonomies.csv`;
util.write(this, formattedTaxonomiesData, fileName, 'taxonomies');
} else {
cliux.print('info: No taxonomies found! Please provide a valid stack.', { color: 'blue' });
}

for (let index = 0; index < taxonomies?.length; index++) {
const taxonomy = taxonomies[index];
const taxonomyUID = taxonomy?.uid;
if (taxonomyUID) {
payload['taxonomyUID'] = taxonomyUID;
const terms = await util.getAllTermsOfTaxonomy(payload);
const formattedTermsData = util.formatTermsOfTaxonomyData(terms, taxonomyUID);
const taxonomyName = taxonomy?.name ?? '';
const termFileName = `${stackName ?? stack.name}_${taxonomyName}_${taxonomyUID}_terms.csv`;
if (formattedTermsData?.length) {
util.write(this, formattedTermsData, termFileName, 'terms');
} else {
cliux.print(`info: No terms found for the taxonomy UID - '${taxonomyUID}'!`, { color: 'blue' });
}
}
}
}
}

ExportToCsvCommand.description = `Export entries or organization users to csv using this command`;
ExportToCsvCommand.description = `Export entries, taxonomies, terms or organization users to csv using this command`;

ExportToCsvCommand.examples = [
'csdx cm:export-to-csv',
Expand All @@ -310,6 +418,10 @@ ExportToCsvCommand.examples = [
'',
'Exporting organization users to csv with organization name provided',
'csdx cm:export-to-csv --action <users> --org <org-uid> --org-name <org-name>',
'Exporting taxonomies and related terms to a .CSV file with the provided taxonomy UID',
'csdx cm:export-to-csv --action <taxonomies> --alias <management-token-alias> --taxonomy-uid <taxonomy-uid>',
'Exporting taxonomies and respective terms to a .CSV file',
'csdx cm:export-to-csv --action <taxonomies> --alias <management-token-alias>',
];

module.exports = ExportToCsvCommand;
Loading
Loading