Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to CLI to support Axios Proxy, Large FHIR List Downloads, and Downloads of Layouts #80

Merged
merged 37 commits into from
May 29, 2019
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
0ae387c
Added support for proxy and added support for apps url for downloadin…
taylordeatri Jan 15, 2019
9b21842
Added unit test for get configuration_layouts command.
taylordeatri Jan 16, 2019
b1f2fbd
Updated list to show each bundle as it comes in.
taylordeatri Apr 19, 2019
6a88d6f
Merged Lifeomic latest.
taylordeatri Apr 19, 2019
51dccf3
Uncommented out lines in build.sh
taylordeatri Apr 19, 2019
3b27dc4
Fixed issues with listing large amounts of data.?
taylordeatri Apr 21, 2019
3d20c96
Added proxy to api.js and apps.js
taylordeatri May 5, 2019
1703616
Updated package-lock.json
taylordeatri May 6, 2019
00aa8a4
Merge branch 'master' into master
taylordeatri May 7, 2019
6dc9fae
Updated code based on failed build.
taylordeatri May 7, 2019
104c280
Added missing config for httpsProxy.
taylordeatri May 7, 2019
70a954e
Removed upload/download methods in apps.js and renamed configuration_…
taylordeatri May 7, 2019
805b7a6
Fixed lint errors.
taylordeatri May 7, 2019
342d0ad
Renamed and fixed layouts-get.test.js
taylordeatri May 7, 2019
8c7ee5c
Adjusted config.js and renamed test class.
taylordeatri May 7, 2019
93eba8a
Fixed missing brace.
taylordeatri May 7, 2019
69eb5a1
Fixed lint issue.
taylordeatri May 7, 2019
3a96497
PHD-417 Updated list to dump output per bundle rather than collect th…
taylordeatri May 8, 2019
0dd0778
PHD-417 Added --jsonLine to make line per resource output and --csv t…
taylordeatri May 9, 2019
f82c2c4
PHD-417 restored build.sh
taylordeatri May 9, 2019
09a0b25
Merge pull request #1 in PHI/phi_cli from feature/PHD-417-jmespath-cs…
taylordeatri May 14, 2019
dfe3b48
Merged LO master to update.
taylordeatri May 14, 2019
ac9cb99
Added changes from the Hackathon - including removing the logResults …
taylordeatri May 14, 2019
f3469a7
Added sample csv/tsv format files.
taylordeatri May 14, 2019
cdcc699
Fixed some lint errors.
taylordeatri May 14, 2019
701c3ad
Removed defaults from the config.js.
taylordeatri May 14, 2019
cc714dd
Changed configureProxy to be a member so proxyquire would work.
taylordeatri May 15, 2019
3c653ea
PHD-417 Fixed fhir-test.js tests.
taylordeatri May 28, 2019
f73d3b2
Merge branch 'master' into master
taylordeatri May 28, 2019
a2f78df
PHD-417 Fixed fhir-test.js tests.
taylordeatri May 28, 2019
9f7a8e2
Merge branch 'master' of github.com:taylordeatri/cli
taylordeatri May 28, 2019
43d1315
PHD-417 Fixed fhir-test.js tests.
taylordeatri May 28, 2019
363edbf
PHD-417 Removed package-lock.json and added it to the .gitignore.
taylordeatri May 28, 2019
438cd94
Fixed lint error.
taylordeatri May 28, 2019
48950a4
Removed the proxy code - still requires 1.19.0-beta.1 to work but it …
taylordeatri May 28, 2019
ef0fc1c
Removed proxy from fhir.test.js
taylordeatri May 28, 2019
60e7b2a
Merge branch 'master' into master
taylordeatri May 28, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions csv_medadmin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
mschroering marked this conversation as resolved.
Show resolved Hide resolved
"header": true,
"fieldDelimiter": ",",
"valueDelimiter": "'",
"fieldMaps": [
{
"columnName": "id",
"jpath": "id"
},
{
"columnName": "medication_code",
"jpath": "medicationCodeableConcept.coding[0].code"
},
{
"columnName": "medication_display",
"jpath": "medicationCodeableConcept.coding[0].display"
},
{
"columnName": "medication_text",
"jpath": "medicationCodeableConcept.coding[0].text"
},
{
"columnName": "dose_text",
"jpath": "dosage.text"
},
{
"columnName": "dose",
"jpath": "dosage.dose.value"
},
{
"columnName": "dose_unit",
"jpath": "dosage.dose.unit"
},
{
"columnName": "subject",
"jpath": "subject.reference"
},
{
"columnName": "date",
"jpath": "effectiveDateTime"
}
]
}
15 changes: 15 additions & 0 deletions csv_patient.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"header": true,
"fieldDelimiter": ",",
"valueDelimiter": "'",
"fieldMaps": [
{
"columnName": "id",
"jpath": "id"
},
{
"columnName": "dob",
"jpath": "birthDate"
}
]
}
11 changes: 11 additions & 0 deletions lib/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ const { dirname } = require('path');
const tokenProvider = require('./interceptor/tokenProvider');
const debug = require('debug')('lo:api');
const FileVerificationStream = require('./FileVerificationStream');
const configureProxy = require('./proxy');

axiosRetry(axios, {
retries: 3
Expand All @@ -31,8 +32,11 @@ function request (options, condition = isNetworkOrIdempotentRequestError) {
throw new Error(`Account needs to be set with 'lo defaults' or specified with the -a option.`);
}

const proxy = configureProxy();

const client = axios.create({
baseURL: config.get(`${environment}.apiUrl`),
proxy: proxy,
headers: {
'LifeOmic-Account': account
}
Expand Down Expand Up @@ -106,9 +110,12 @@ module.exports.download = async function (options, path, fileName) {

const bar = progress(fileName);

const proxy = configureProxy();

const res = await axios({
method: 'get',
url: response.data.downloadUrl,
proxy: proxy,
responseType: 'stream'
});

Expand Down Expand Up @@ -145,10 +152,12 @@ module.exports.getFileVerificationStream = async function (filePath, fileSize) {
};

module.exports.upload = async function (uploadUrl, fileSize, data, contentMD5) {
const proxy = configureProxy();
await axios({
method: 'put',
url: uploadUrl,
data,
proxy: proxy,
headers: {
'Content-Length': fileSize,
'Content-MD5': contentMD5
Expand Down Expand Up @@ -214,10 +223,12 @@ module.exports.multipartUpload = async function (options, uploadId, fileName, fi
try {
debug(`Uploading part ${part}, size: ${size}`);
const time = process.hrtime();
const proxy = configureProxy();
await axios({
method: 'put',
url: response.data.uploadUrl,
data: verifyStream.data,
proxy: proxy,
headers: {
'Content-Length': size,
'Content-MD5': verifyStream.contentMD5
Expand Down
53 changes: 53 additions & 0 deletions lib/apps.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
'use strict';

const axios = require('axios');
const axiosRetry = require('axios-retry');
// const tunnel = require('tunnel');
const config = require('./config');
const { name, version } = require('../package.json');
const tokenProvider = require('./interceptor/tokenProvider');
const configureProxy = require('./proxy');

axiosRetry(axios, {
retries: 3
});
axios.defaults.headers.common['User-Agent'] = `${name}/${version}`;

function request (options) {
const environment = config.getEnvironment();

const account = options.account || config.get(`${environment}.defaults.account`);
if (!account) {
throw new Error(`Account needs to be set with 'lo defaults' or specified with the -a option.`);
}

const proxy = configureProxy();
const baseURL = config.get(`${environment}.appsUrl`);

const client = axios.create({
baseURL: baseURL,
proxy: proxy,
headers: {
'LifeOmic-Account': account
}
});
client.interceptors.request.use(tokenProvider);
axiosRetry(client, { retries: 3 });
return client;
}

module.exports.get = function (options, path) {
return request(options).get(path);
};

module.exports.del = function (options, path) {
return request(options).delete(path);
};

module.exports.post = function (options, path, body) {
return request(options).post(path, body);
};

module.exports.put = function (options, path, body) {
return request(options).put(path, body);
};
66 changes: 59 additions & 7 deletions lib/cmds/fhir_cmds/list.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ const querystring = require('querystring');
const { post, getAccount } = require('../../fhir');
const print = require('../../print');
const url = require('url');
const jmespath = require('jmespath');
const fs = require('fs');

exports.command = 'list <type>';
exports.desc = 'List FHIR resources by type <type>.';
Expand All @@ -22,14 +24,31 @@ exports.builder = yargs => {
describe: 'Number of resources to return',
type: 'integer',
default: 1000
}).option('csv', {
describe: 'CSV Format Configuration file in json',
type: 'string'
});
};

async function search (type, query, options) {
const account = getAccount(options);
const result = [];
const limit = options.limit;

const csvConfigFile = options.csv;

const csvConfig = csvConfigFile != null ? JSON.parse(fs.readFileSync(csvConfigFile, 'UTF-8')) : null;

if (csvConfig && csvConfig.header) {
let header = '';
csvConfig.fieldMaps.forEach(fieldMap => {
if (header.length > 0) {
header = header + ',';
}
header = header + fieldMap.columnName;
});
console.log(header);
}

if (query) {
query.pageSize = limit;
} else {
Expand All @@ -46,22 +65,56 @@ async function search (type, query, options) {
);

const data = response.data;
result.push(...data.entry.map(x => x.resource));
const results = data.entry.map(x => x.resource);

if (result.length < limit && data.link) {
if (results.length > 0) {
if (csvConfig) {
const linesResult = formatResults(results, csvConfig, options);
linesResult.forEach(line => {
console.log(line);
});
} else {
results.forEach(result => print(result, options));
}
}

if (results.length < limit && data.link) {
const next = data.link.find(x => x.relation === 'next');
if (next) {
const parsedUrl = url.parse(next.url, true);
query = Object.assign(query, parsedUrl.query);
} else {
return result;
return;
}
} else {
return result;
return;
}
}
}

function formatResults (results, csvConfig, options) {
const lines = [];
results.forEach(result => {
const fldD = csvConfig.fieldDelimiter;
const valueD = csvConfig.valueDelimiter;
const fields = [];
csvConfig.fieldMaps.forEach(fieldMap => {
const value = jmespath.search(result, fieldMap.jpath);
if (value && value.length > 0) {
fields.push(valueD + value + valueD);
} else {
fields.push('');
}
});
let line = '';
fields.forEach(field => {
line = line + field + fldD;
});
lines.push(line.substring(0, line.length - 1));
});
return lines;
}

exports.handler = async argv => {
if (argv.query) {
argv.query = querystring.parse(argv.query);
Expand All @@ -77,6 +130,5 @@ exports.handler = async argv => {
}
}

const result = await search(argv.type, argv.query, argv);
print(result, argv);
await search(argv.type, argv.query, argv);
};
10 changes: 10 additions & 0 deletions lib/cmds/layouts.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
'use strict';

const options = require('../common-yargs');

exports.command = 'layouts <command>';
exports.desc = 'Perform operations on layouts.';
exports.builder = yargs => {
return options(yargs.commandDir('layouts_cmds'));
};
exports.handler = function (argv) {};
21 changes: 21 additions & 0 deletions lib/cmds/layouts_cmds/get.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
'use strict';

const { get } = require('../../apps');
const print = require('../../print');

exports.command = 'get <projectId> <type>';
exports.desc = 'Get Layouts information by <projectId> - <type> Patient|Insights';
exports.builder = yargs => {
yargs.positional('projectId', {
describe: 'The ID of the project which layouts to fetch.',
type: 'string'
}).positional('type', {
describe: 'The type of the layout of the project which layouts to fetch.',
type: 'string'
});
};

exports.handler = async argv => {
const response = await get(argv, `/configuration/layouts?project=${argv.projectId}&type=${argv.type}&format=json`);
print(response.data, argv);
};
26 changes: 26 additions & 0 deletions lib/cmds/setup.js
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,26 @@ exports.handler = async argv => {
mask: '*',
default: a => config.get(`${a.environment}.defaults.clientSecret`),
when: a => a.useClientCredentials && argv.apiKey === undefined && argv.authClientId === undefined && argv.clientId === undefined
},
{
type: 'confirm',
name: 'useHttpProxy',
message: 'Use an HTTP Proxy?',
default: a => config.get(`${a.environment}.defaults.useHttpProxy`) !== undefined
},
{
type: 'input',
name: 'httpProxyHost',
message: 'Http ProxyHost: ',
when: a => a.useHttpProxy,
default: a => config.get(`${a.environment}.defaults.httpProxyHost`)
},
{
type: 'input',
name: 'httpProxyPort',
message: 'Http ProxyPort: ',
when: a => a.useHttpProxy,
default: a => config.get(`${a.environment}.defaults.httpProxyPort`)
}
];

Expand Down Expand Up @@ -147,5 +167,11 @@ exports.handler = async argv => {
} else {
config.delete(`${environment}.defaults.apiKey`);
}

config.set(`${answers.environment}.defaults.useHttpProxy`, answers.useHttpProxy);
if (answers.useHttpProxy) {
config.set(`${answers.environment}.defaults.httpProxyHost`, answers.httpProxyHost);
config.set(`${answers.environment}.defaults.httpProxyPort`, answers.httpProxyPort);
}
console.log(chalk.green(`Default settings have been saved.`));
};
3 changes: 3 additions & 0 deletions lib/common-yargs.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ module.exports = (yargs) => {
.option('json', {
describe: 'List output as JSON',
type: 'boolean'
}).option('jsonLine', {
describe: 'List output as JSON LINE Format',
type: 'boolean'
}).option('account', {
describe: 'Override the account',
type: 'string'
Expand Down
7 changes: 6 additions & 1 deletion lib/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,13 @@ const conf = new Configstore('lifeomic-cli', {
userPooldId: 'us-east-2_ojOIXR4W7',
clientId: '2qt93qphnctjbs9ftdpjhvgkl1',
apiUrl: 'https://api.us.lifeomic.com',
appsUrl: 'https://apps.us.lifeomic.com/phc/api',
fhirUrl: 'https://fhir.us.lifeomic.com',
ga4ghUrl: 'https://ga4gh.us.lifeomic.com'
ga4ghUrl: 'https://ga4gh.us.lifeomic.com',
useHttpProxy: false,
httpProxyHost: '',
httpProxyPort: '',
httpProxyHttps: false
}
});

Expand Down
Loading