Add support for custom help groups

[smeijer]

I think it'd be helpful if we had a way to group commands and options, to create sections in --help.

There's a prior issue to this; #78, but that's over 9 years old. I'm also aware of addHelpText (#1296), but when using that to group commands, it adds boilerplate due to the need of needing to "hide" commands, and later formatting and properly indenting help ourselves.

What I wish to achieve is something like:

Usage: magicbell [options] [command]

Work with MagicBell from the command line

Options:
  -V, --version             output the version number
  -p, --profile <string>    Profile to use (default: "default")
  -h, --help                display help for command

Commands:
  broadcasts                Manage broadcasts
  imports                   Manage imports
  listen                    Listen to events for a users inbox
  metrics                   Manage metrics
  notification-preferences  Manage notification preferences
  notifications             Send and retrieve notifications
  push-subscriptions        Manage push subscriptions
  subscriptions             Manage subscriptions
  users                     Manage all known users

Other commands:
  config                    Display or change config values
  login                     Login to your account
  logout                    Logout of your account
  help                      Display help for a command

This separates 'config commands' from 'core commands'. If you'd run stripe --help or gh --help you can see similar command grouping.

At this moment, I achieve the above via:

for (const command of commands) {
  program.addCommand(command);
}

for (const command of otherCommands) {
  program.addCommand(command, { hidden: true });
}

program.addHelpCommand(false);
const padding = Object.values([...commands, ...otherCommands]).reduce((acc, command) => {
  return Math.max(acc, command.name().length);
}, 0);

program.addHelpText(
  'after',
  `
Other commands:
  ${otherCommands.map((c) => `${c.name().padEnd(padding, ' ')}  ${c.description()}`).join('\n  ')}
  ${'help'.padEnd(padding, ' ')}  Display help for a command
`,
);

That's doable, but I think it'd be nice to have native support for it. Something like:

program.addCommand(command, { group: 'Other commands' });
program.addOption(option, { group: 'Global flags' });

The group can both serve as group title, and grouping key.

[shadowspawn]

Thanks for the example, and creative partial work-around.

I see option and command groups in "big" utilities and have wondered about grouping or separator support in Commander, but have not progressed to ideas.

I had one related old comment saved:

• Request to support for -- parameter separator. #374 (comment)

[shadowspawn]

I did some research looking for prior art for adding options groups in the help. I found a few variations:

Python argparse has add_argument_group(), which creates an intermediate object, and then you add options to the group rather than the command: https://docs.python.org/dev/library/argparse.html#argument-groups

Yargs has cmd.group(option_keys) or using group property on option.

command-line-args has a group property when adding options: https://github.com/75lb/command-line-usage/wiki/How-to-break-the-option-list-up-into-groups

Oclif options have a helpGroup property:

• https://oclif.io/docs/flags
• feat: help improvements and customizability oclif/core#184

[smeijer]

Yargs still has an open issue to group commands. The groups only work for flags there I believe.

In the meanwhile, I've improved my work-around a bit, by adding a group method to my commands, and overriding the formatHelp method. I really like this approach of groups and the custom help formatter. If I were to add support for a callback, we could easily invoke the callback, and pass it the sections, so users can easily customise the help output. This would also be useful for customisation of section headings.

As you might see, I've customised the help quite a bit now. I wasn't looking for that, that's just a side effect of writing the custom help formatter like this 🙂.

Also, if there's any interest in this work, I'd be happy to submit a pull-request. If not, no hard feelings. I'm happy enough with my current solution.

createCommand

source: https://github.com/magicbell-io/magicbell-js/blob/e019d664762ab10386596a34c785658d5790bc02/packages/cli/src/lib/commands.ts#L5-L29

export function createCommand(name?: string): ExtendedCommand {
  const command = new Command(name) as ExtendedCommand;

  command.helpOption('-h, --help', 'Show help for command');
  command.showHelpAfterError(true);
  command.addHelpCommand(false);

  command.configureHelp({
    sortSubcommands: true,
    showGlobalOptions: true,
    subcommandTerm: (cmd) => cmd.name(),
    // with callback support, this could look like formatHelp((sections) => generateHelpString(sections))
    formatHelp: formatHelp,
  });

  command.group = (group) => {
    (command as any)._group = group;
    return command;
  };

  return command;
}

formatHelp

source: https://github.com/magicbell-io/magicbell-js/blob/01e5fd93c3de85a61ce70a7dac5d81f007cc5d63/packages/cli/src/lib/help.ts

import { Command, Help } from 'commander';
import kleur from 'kleur';

export function formatHelp(cmd: Command, helper: Help) {
  const termWidth = helper.padWidth(cmd, helper);
  const helpWidth = helper.helpWidth || 80;

  const itemSeparatorWidth = 2; // between term and description
  const indent = ' '.repeat(2);
  const moveOptions = !cmd.parent && cmd.commands.length;

  function formatItem(term, description) {
    if (description) {
      const fullText = `${term.padEnd(termWidth + itemSeparatorWidth)}${description}`;
      return helper.wrap(fullText, helpWidth - indent.length, termWidth + itemSeparatorWidth);
    }

    return term;
  }

  function formatList(textArray) {
    const list = textArray.join('\n').replace(/^/gm, indent).trim();
    return list ? indent + list : '';
  }

  const sections = {
    description: '',
    usage: '',
    arguments: '',
    options: '',
    commands: [] as { title: string; list: string }[],
    globalOptions: '',
  };

  sections.description = helper.commandDescription(cmd);
  sections.usage = helper.commandUsage(cmd);
  sections.arguments = formatList(
    helper.visibleArguments(cmd).map((argument) => {
      return formatItem(helper.argumentTerm(argument), helper.argumentDescription(argument));
    }),
  );

  // Note: options might benefit from similar grouping as commands below, I just didn't need that (yet)
  sections.options = formatList(
    helper
      .visibleOptions(cmd)
      .filter((option) => {
        // move --help to global options
        if (cmd.parent && option.long === '--help') {
          cmd.parent.addOption(option);
          return false;
        }

        return true;
      })
      .map((option) => {
        return formatItem(helper.optionTerm(option), helper.optionDescription(option));
      }),
  );

  // Commands
  const commands: Record<string, Command[]> = {};
  for (const command of helper.visibleCommands(cmd)) {
    const group = ((command as any)._group || 'Commands').trim();
    commands[group] = commands[group] || [];
    commands[group].push(command);
  }

  sections.commands = Object.entries(commands).map(([title, commands]) => ({
    title,
    list: formatList(commands.map((cmd) => formatItem(helper.subcommandTerm(cmd), helper.subcommandDescription(cmd)))),
  }));

  sections.globalOptions = this.showGlobalOptions
    ? formatList(
        helper
          .visibleGlobalOptions(cmd)
          .filter((option) => {
            // don't return --version on sub commands
            return option.long !== '--version';
          })
          .map((option) => {
            return formatItem(helper.optionTerm(option), helper.optionDescription(option));
          }),
      )
    : '';

  // -------
  //  Everything below here could be wrapped in a callback, so formatHelp(cb) can be used as formatter
  // -------

  const output = [];
  output.push(kleur.bold('Usage'), indent + sections.usage, '');

  if (sections.arguments) {
    output.push(kleur.bold('Arguments'), sections.arguments, '');
  }

  if (sections.options && !moveOptions) {
    output.push(kleur.bold('Options'), sections.options, '');
  }

  if (sections.commands.length) {
    sections.commands.forEach((section) => {
      output.push(kleur.bold(section.title), section.list, '');
    });
  }

  if (sections.options && moveOptions) {
    output.push(kleur.bold('Options'), sections.options, '');
  }

  if (sections.globalOptions) {
    output.push(kleur.bold('Global Options'), sections.globalOptions, '');
  }

  return output.join('\n');
}

usage

source: https://github.com/magicbell-io/magicbell-js/blob/e019d664762ab10386596a34c785658d5790bc02/packages/cli/src/index.ts#L42-L51

for (const command of commands) {
  program.addCommand(command.group('Resource commands'));
}

for (const command of otherCommands) {
  program.addCommand(command.group('Other commands'));
}

Result:

[shadowspawn]

Interesting, thanks.

I was wondering about the order the command groups will appear in the help list. I think that implementation will give varying results depending on the alphabetical ordering of the commands? Probably the order they were first used is good enough. So could perhaps create an empty commands[group] with a scan through the commands in original order, then populate the groups in a scan through the visibleCommands.

[smeijer]

Oh interesting. I haven't thought about the order. It just "naturally" worked. But yeah, when I'd drop the broadcasts command, "Other commands" move to the top 😅 . Not a direct issue for me now, but I'll have to think on how to solve that. Maybe track the calls to .group('...') and use the order of creation?

[shadowspawn]

I was suggesting could create empty groups using the raw commands (unsorted, so in creation order), then populate the groups in a scan through the visibleCommands. And ignore zero-length groups that turned out to be all hidden. So can still all be done in help generation.

[shadowspawn]

Playing with three different styles that fit into current API. The .helpGroup() chaining call on the command is not a solution on its own since it is not available with external commands.

// using command configuration options
program
  .command('a', 'external command a', { helpGroup: 'group 1' })
  .command('b', 'external command b', { helpGroup: 'group 1' });

program.command('c', { helpGroup: 'group 2' })
  .description('action command c')
  .action(() => {});
program.command('d', { helpGroup: 'group 2' })
  .description('action command d')
  .action(() => {});

// declare group before adding commands to group
program
  .defaultHelpGroup('group 3')
  .command('e', 'external command e')
  .command('f', 'external command f');

program.defaultHelpGroup('group 4');
program.command('g')
  .description('action command g')
  .action(() => {});
program.command('h')
  .description('action command h')
  .action(() => {});

// per command chaining style
program.command('i')
  .description('action command i')
  .helpGroup('group 5')
  .action(() => {});
program.command('j')
  .description('action command j')
  .helpGroup('group 5')
  .action(() => {});

---

Using with code sample from earlier comment:

for (const command of otherCommands6) {
  program.addCommand(command, { helpGroup: 'group 6' });
}

program.defaultHelpGroup('group 7');
for (const command of otherCommands7) {
  program.addCommand(command);
}

for (const command of otherCommands8) {
  command.helpGroup('group 8');
  program.addCommand(command);
}

Edit: .defaultHelpGroup() name is not clear since want groups for options as well as commands,

[shadowspawn]

Hmm, how to set the group for the built-in help command or help option?

(I see in your implementation you turn off the help command and handle the help option specially.)

[shadowspawn]

The help command could use similar syntax to .command() and .addCommand() to specify the group. And help option could do same, although that is a departure from what .option() allows. Slightly annoying for user that need to supply the name/flags and description but usable, author is doing extra work to specify groups anyway.

cmd.addHelpComand('help', 'display help for command', { group: 'Example:' });
cmd.helpOption('-h, --help', 'display help for command', { group: 'Example:' });

Commander largely avoids referring to commands and options by name, but a name style API does support help naturally:

cmd.commandGroup('Core Commands:', ['start', 'test', 'help']);

With support for "separator" style API, could perhaps allow:

cmd
   .startCommandGroup('Core:')
   .addCommand('foo', 'external foo description')
   .addHelpCommand();

[shadowspawn]

I noticed there is a request for untitled help groups "hidden" in the referenced PR:

• Expand help customisation with .addHelpText #1296 (comment)