How to create a SAF CLI

This page provides the methodology of creating a SAF Command Line Interface (CLI). It provides the framework used for all future SAF CLI development.

Setup the development environment

• Create a new branch from the main in the SAF GitHub repository
• Clone the newly created branch git clone -b <new_branch_name> [email protected]:mitre/saf.git
• Install the necessary dependencies either via npm or brew - npm install or brew install
  • Execute the command from the root directory where the branch was cloned locally

SAF CLI directory structure

The SAF source code directory structure is comprised of multiple directories, each containing specific content (code, scripts, documents, tests, etc.)

• saf - this is the root directory
• .git - contains all the information that is necessary for the project in version control and all the information about commits, remote repository, etc
• .github - used to place GitHub related stuff inside it such workflows, formatting, etc
• .vscode - holds the VS Code editor configuration content
• bin - contains the runtime commands for node.js
• docs - contains eMASSer documentation
• lib - contains the compiled JavaScript files. This folder is created by the TypeScript Compiler (tsc) command. On the SAF CLI this command is scripted as npm run prepack command which will execute based on what OS it is running (win or mac)
• node_modules - contains all of the application supporting resources, created when the npn install command is executed
• src - this folder contains all of the SAF CLI commands, it is organized by capabilities
• test - contains all of the automated tests used to verify available capabilities

New SAF CLI Command

Any new SAF CLI command(s) should be added to the src -> commands directory inside a directory that indicates what the command is to accomplish. For example, if we are to add commands that connects to other systems like tenable.sc or splunk, we could create a sub-folder inside the src -> commands folder and call it interfaces, or a single directory for each interface, like tenable and splunk

Example:
  src/          or           src/          or          src/
  └── commands/              └── commands/             └── commands/
     └── tenable/                └── splunk/               └── interfaces/
         └── tenable.ts              └── splunk.ts             ├── tenable.ts 
                                                               └── splunk.ts

The objective is to keep the like commands grouped together.

OCLIF Configuration

The oclif behavior is configured inside the SAF CLI package.json under the oclif section. If the CLI being created does not belong to one of the available topics (oclif -> topics) a new topic needs to be added to the oclif section. See Topics for more information on how to.

SAF CLI Template

The following code can be used as a starter template

import path from 'path'
import {Flags} from '@oclif/core';
import {BaseCommand} from '../../utils/oclif/baseCommand'

export default class MYCLI extends BaseCommand<typeof MYCLI> {
  // Note: If the variable `usage` is not provided the default is used 
  // <%= command.id %> resolves to the command name
  static readonly usage = '<%= command.id %> -i <ckl-xml> -o <hdf-scan-results-json> [-r]'
    
  static readonly description = 'Describe what the CLI does - short and to the point'

  // Note: <%= config.bin %> resolves to the executable name (i.e., saf, emasser)
  static readonly examples = [
    '<%= config.bin %> <%= command.id %> -i the_input_file -o the_out_put_file',
    '<%= config.bin %> <%= command.id %> --interactive',
  ]

  // To describe multiple examples use:
  static readonly examples = [
    {
      description: '\x1B[93mInvoke the command using command line flags\x1B[0m',
      command: '<%= config.bin %> <%= command.id %> -i the_input_file -o the_out_put_file',
    },
    {
      description: '\x1B[93mInvoke the command interactively\x1B[0m',
      command: '<%= config.bin %> <%= command.id %> --interactive',
    },
  ]

  // Note: the BaseCommand abstract class implements the log level and interactive flags
  static flags = {
    input: Flags.string({
      char: 'i', required: false, exclusive: ['interactive'],
      description: '\x1B[31m(required if not --interactive)\x1B[34m The Input file',
    }),
    output: Flags.string({
      char: 'o', required: false, exclusive: ['interactive'],
      description: '\x1B[31m(required if not --interactive)\x1B[34m The Output file',
    }),
    includeRaw: Flags.boolean({
      char: 'r', required: false, description: 'Include raw input file in HDF JSON file',
    }),
  }

  async run(): Promise<any> {
    const {flags} = await this.parse(MYCLI)
      
    // Check if we are using the interactive flag
    let inputFile = ''
    let outputFile = ''
    if (flags.interactive) {
      const interactiveFlags = await getFlags() // see CLI Interactive Template
      inputFile = interactiveFlags.inputFile
      outputFile = path.join(interactiveFlags.outputDirectory, interactiveFlags.outputFileName)
    } else if (this.requiredFlagsProvided(flags)) { // see method template bellow
      inputFile = flags.input as string
      outputFile = flags.output as string
    } else {
      return
    }

    //*****************************//
    // Implement the CLI code here //
    //*****************************//
  }

  // Check for required fields template
  requiredFlagsProvided(flags: { input: any; output: any }): boolean {
    let missingFlags = false
    let strMsg = 'Warning: The following errors occurred:\n'

    if (!flags.input) {
      strMsg += colors.dim('  Missing required flag input file\n')
      missingFlags = true
    }

    if (!flags.output) {
      strMsg += colors.dim('  Missing required flag output (CSV file)\n')
      missingFlags = true 
    }

    if (missingFlags) {
      strMsg += 'See more help with -h or --help'
      this.warn(strMsg)
    }

    return !missingFlags
  }
}

CLI Interactive Template

The SAF CLI uses the inquirer.js module for interactively ask for the CLI flags, both required and optional flags. Currently the inquirer module being used is the legacy version of inquirere.js. Once the new inquirer is tested and proven to work with required plugins (like the file tree selection) we will change to the new @inquirer/prompts

To use the interactive mode create an asynchronous function that returns an object with the selected answers. The reason for the async is that inquirer returns a promise in the form of inquirer.prompt(questions, answers) -> promise

Note

If using the choices question object type, it may be necessary to increase the node default max listeners (defaults to 10) if more than 10 choices are required.
To increase the number of listeners use EventEmitter.defaultMaxListeners = [number_value]

Contiguous Question/Answer Template

Use the following code as a starter template

import inquirer from 'inquirer'
import {EventEmitter} from 'events'
import inquirerFileTreeSelection from 'inquirer-file-tree-selection-prompt'

async function getFlags(): Promise<any> {
  // Register the file tree selection plugin
  inquirer.registerPrompt('file-tree-selection', inquirerFileTreeSelection)

  // Create an array of question objects
  // This example asks for an input file and an output directory and
  // and filename to be generated in the selected output directory
  const questions = [
    {
      type: 'file-tree-selection',
      name: 'inputFile',
      message: 'Select the required JSON input file:',
      filters: 'json',
      pageSize: 15,
      require: true,
      enableGoUpperDirectory: true,
      transformer: (input: any) => {
        const name = input.split(path.sep).pop()
        const fileExtension =  name.split('.').slice(1).pop()
        if (name[0] === '.') {
          return colors.grey(name)
        }

        if (fileExtension === 'json') {
          return colors.green(name)
        }

        return name
      },
      validate: (input: any) => {
        const name = input.split(path.sep).pop()
        const fileExtension =  name.split('.').slice(1).pop()
        if (fileExtension !== 'json') {
          return 'Not a .json file, please select another file'
        }

        return true
      },
    },
    {
      type: 'file-tree-selection',
      name: 'outputDirectory',
      message: 'Select output directory for the generated file:',
      pageSize: 15,
      require: true,
      onlyShowDir: true,
      enableGoUpperDirectory: true,
      transformer: (input: any) => {
        const name = input.split(path.sep).pop()
        if (name[0] === '.') {
          return colors.grey(name)
        }

        return name
      },
    },
    {
      type: 'input',
      name: 'outputFileName',
      message: 'Specify the output filename (.csv). It will be saved to the previously selected directory:',
      require: true,
      default() {
        return 'default_filename.csv'
      },
    },
  ]

  // Variable used to store the prompts (question and answers)
  const interactiveValues: {[key: string]: any} = {}
  
  // Launch the prompt interface (inquiry session)
  const ask = inquirer.prompt(questions).then((answers: any) => {
    for (const envVar in answers) {
      if (answers[envVar] !== null) {
        // Save the responses to the return object
        interactiveValues[question] = answer[question]
      }
    }
  })

  await ask
  return interactiveValues
}

Dependent Question/Answer Template

Use the following code as a starter template

import inquirer from 'inquirer'
import {EventEmitter} from 'events'
import inquirerFileTreeSelection from 'inquirer-file-tree-selection-prompt'

async function getFlags(): Promise<any> {
  // Register the file tree selection plugin
  inquirer.registerPrompt('file-tree-selection', inquirerFileTreeSelection)

  // Create a question object
  // This example asks if user wants to use an input
  // file, if yes than ask for the file full path
  const addInputFilePrompt = {
    type: 'list',
    name: 'useInputFile',
    message: 'Include an input file:',
    choices: ['true', 'false'],
    default: false,
    filter(val: string) {
      return (val === 'true')
    },
  }
  const inputFilePrompt = {
    type: 'file-tree-selection',
    name: 'inputFilename',
    message: 'Select the input filename - in the form of .xml file:',
    filters: 'xml',
    pageSize: 15,
    require: true,
    enableGoUpperDirectory: true,
    transformer: (input: any) => {
      const name = input.split(path.sep).pop()
      const fileExtension =  name.split('.').slice(1).pop()
      if (name[0] === '.') {
        return colors.grey(name)
      }

      if (fileExtension === 'xml') {
        return colors.green(name)
      }

      return name
    },
    validate: (input: any) => {
      const name = input.split(path.sep).pop()
      const fileExtension =  name.split('.').slice(1).pop()
      if (fileExtension !== 'xml') {
        return 'Not a .xml file, please select another file'
      }

      return true
    },
  }
  
  // Variable used to store the prompts (question and answers)
  const interactiveValues: {[key: string]: any} = {}
  
  // Launch the prompt interface (inquiry session)
  let askInputFilename: any
  const askInputFilePrompt = inquirer.prompt(addOvalFilePrompt).then((addInputFilePrompt : any) => {
    if (addInputFilePrompt.useInputFile === true) {
      interactiveValues.useInputFile= true
      askInputFilename = inquirer.prompt(inputFilePrompt ).then((answer: any) => {
        for (const question in answer) {
          if (answer[question] !== null) {
            interactiveValues[question] = answer[question]
          }
        }
      })
    } else {
      interactiveValues.useInputFile= false
    }
  }).finally(async () => {
    await askInputFilename
  })
  await askInputFilePrompt
  return interactiveValues
}

CLI Supporting Functions

CLI helper functions are available in the cliHelper.ts TypeScript file.

Functions provided are:

• Colorize console log outputs (various colors and combinations)
• Data logging for every colorized output
• Initialize output log filename (setProcessLogFileName(fileName: string))
• Retrieve log output object (getProcessLogData(): Array<string>)
• Add content to the log data object (addToProcessLogData(str: string))
• Save the log output content (saveProcessLogData())