Salt-based supportconfig #49480

isbm · 2018-09-01T16:37:19Z

What does this PR do?

Re-implementation of Support Config Tool in Salt.

What is this?

Over many years SUSE is using Support Config Tool [to collect system information] (https://www.suse.com/documentation/sled11/book_sle_admin/data/sec_admsupport_supportconfig.html) in order to provide detailed information of a running system to technical support. However, given on its age and therefore design, this tool has a number of limitations that would be nice to do not have.

Imagine, you want to know why one of your supported applications on some machine doesn't work. But you already have Salt running there. Would it be cool to run your crafted (or standard) state-like YAML scenario and then get all the needed precise data about broken component and ask exactly only what you need? No more poking "please try run this and that". Simply make your own YAML-based scenario what you would like to fetch, and say "run this scenario, send us the resulting archive". That's it.

It also works on whatever OS you run Salt.

Support Scenarios vs States

Support Scenario very much looks like orchestration. However, it is not an orchestration on its own. Salt state files (SLS) gets parsed when Salt is already running. That said, Salt is parsing, compiling, injecting required variables etc and then running them. Support Scenario, however, needs to be parsed and compiled beforehand Salt is even running. Additionally, Support Scenario runs a mix of runners and modules.

Overview

Salt Support is implemented on top of Salt as a platform and thus comes with its own CLI command, called salt-support. Once installed, you will find it along with salt, salt-key, salt-minion or salt-master etc commands. Naturally, it inherits most of the Salt typical features: debugging interface -l <level> and other that you already know:

$ salt-support --help

Salt Support is a program to collect all support data: logs, system
configuration etc.

Options:
  --version             show program's version number and exit
  -V, --versions-report
                        Show program's dependencies version number and exit.
  -h, --help            show this help message and exit
  -c CONFIG_DIR, --config-dir=CONFIG_DIR
                        Pass in an alternative configuration directory.
                        Default: '/etc/salt'.
  -t TIMEOUT, --timeout=TIMEOUT
                        Change the timeout, if applicable, for the running
                        command (in seconds). Default: 5.
  -P, --show-profiles   Show available profiles
  -p SUPPORT_PROFILE, --profile=SUPPORT_PROFILE
                        Specify support profile or comma-separated profiles,
                        e.g.: "salt,network"
  -a SUPPORT_ARCHIVE, --archive=SUPPORT_ARCHIVE
                        Specify name of the resulting support archive. Default
                        is "/tmp/zeus-support.tar.bz2".
  -u SUPPORT_UNIT, --unit=SUPPORT_UNIT
                        Specify examined unit (default "master").
  -U, --show-units      Show available units
  -f, --force           Force overwrite existing archive, if exists
  -o SUPPORT_OUTPUT_FORMAT, --out=SUPPORT_OUTPUT_FORMAT
                        Set the default output using the specified outputter,
                        unless profile does not overrides this. Default:
                        "yaml".

  Logging Options:
    Logging options which override any settings defined on the
    configuration files.

    -l LOG_LEVEL, --log-level=LOG_LEVEL
                        Console logging log level. One of 'all', 'garbage',
                        'trace', 'debug', 'profile', 'info', 'warning',
                        'error', 'critical', 'quiet'. Default: 'warning'.
    --log-file=LOG_FILE
                        Log file path. Default: '/var/log/salt/master'.
    --log-file-level=LOG_LEVEL_LOGFILE
                        Logfile logging log level. One of 'all', 'garbage',
                        'trace', 'debug', 'profile', 'info', 'warning',
                        'error', 'critical', 'quiet'. Default: 'warning'.

You can find additional help about salt-support issuing "man salt-support" or
on http://docs.saltstack.com

Typical Usage

By default, salt-support is acting as it being a Minion. So the usage is following:

Login to your broken machine:
ssh yourmachine
In case you want other profile than default, list available profiles:
salt-support -P

List of available profiles:
  1. default
  2. flt
  3. jobs-active
  4. jobs-last
  5. jobs-trace
  6. network
  7. postgres
  8. salt
  9. users

Run it:
salt-support
In case you running a custom scenario:
salt-support -p /path/to/your/own/profile.yml

At this point it will collect all the data of the machine, such as system info, its health, disks status, network layout, users, system logs, actual minion configuration, pillars, highstate etc and it will put into an archive in /tmp/yourhost-support.tar.bz2 file. Now you can send this archive to your technical support. 😎

What happens if I run it?

Essentially, Support Profile will just call any module or useful runner or useful utility in your battery-included Salt in any combination you want and record that data for you. Your support desk won't need to ask anymore "How looks output from the following command: .....?" It will do that for you and you will get a corresponding file inside the archive.

What is inside the archive?

It is a tree of directories: a date-time when the snapshot has been taken and full path to copied file tree, if you wanted so. Snapshot will contain many self-descriptive text files, such as systeminfo, system-health, network, packages etc. You can look into them and get an idea what is going on with the affected machine.

Jinja in Support Profile!?

Yes. Each profile is essentially a YAML Jinja2 template, just like SLS. However, difference is that it should be compiled beforehand. Currently there is only one way to call other modules within the template: {{ salt('cmd.run', 'uname') }} for example. If module returns a list or any structure, you can just iterate over it, just the same way as you would do in normal Jinja2.

Limitations

Current limitations of this implementation as of today (subject to change):

As of now, it cannot collect information from remote minions
It doesn't work over Salt SSH. You need manually "ssh" there and run it manually.
Profiles are still quite basic.

Tests written?

Yes

isbm · 2018-09-01T16:43:45Z

@gtmanfred @terminalmage @cachedout @rallytime Here. You have it. 😉

cachedout · 2018-09-02T17:54:44Z

This has earned the much-coveted Awesome label!

I'm really happy about this feature, @isbm. I think it's absolutely fantastic! Great work here.

cachedout · 2018-09-03T13:47:12Z

@isbm Can you start by addressing the lint errors? https://jenkinsci.saltstack.com/job/pr-lint/job/PR-49480/1/warnings52Result/new/

isbm · 2018-09-03T13:52:14Z

@cachedout I am exactly doing that right now! 😉

isbm · 2018-09-03T14:39:02Z

@cachedout hopefully this cleans it up.

isbm · 2018-09-04T10:30:02Z

@cachedout lints are now fixed. There is a case where I had to just mute W8470 (use with to immediately close file handler) but this is wrong since I have to pass not the content of the file but an opened file handler to the Tar archive, which then takes care of closing them on its own.

cachedout

This looks really nice. Well done, @isbm.

isbm · 2018-09-07T16:05:39Z

@cachedout @rallytime any updates here?

rallytime · 2018-09-07T17:59:00Z

Bah, sorry I meant to request a review from someone and I accidentally asked @cachedout to review again. My mistake!

isbm · 2018-09-10T09:59:03Z

@cachedout @terminalmage @rallytime any updates here?

rallytime · 2018-09-11T19:14:47Z

@isbm I think there will be a delay in reviews here because of SaltConf. In the mean time, I will restart the tests. The last run looks suspicious to me. :)

rallytime

I really like this feature @isbm!

meaksh

Found some minor issue here. Apart from that, we need to add salt-support script reference inside setup.py in order to get it installed while building the package.

scripts/salt-support

cachedout · 2018-09-16T23:26:06Z

salt/cli/support/__init__.py

+    return env.get_template(os.path.basename(path)).render(salt=caller, runners=runner).strip()
+
+
+def get_profile(profile, caller, runner):


This function is a little hard to read. Might be a good target to either break apart or add additional documentation about what's happening here.

cachedout · 2018-09-16T23:34:45Z

salt/cli/support/collector.py

+        self.__arch.close()
+        self.__arch = None
+
+    def _flush_content(self):


Here's another function that CodeClimate has marked as being difficult to read. Can we break this up into something easier to understand?

cachedout · 2018-09-16T23:35:24Z

salt/cli/support/collector.py

+
+        return conf
+
+    def _get_runner(self, conf):


Can we refactor to DRY between _get_runner and _get_caller?

cachedout · 2018-09-16T23:36:28Z

salt/cli/support/collector.py

+
+        return ret
+
+    def run(self):


Can we factor what's in here to call some smaller functions that are easier to understand?

salt/cli/support/collector.py

…port salt.ext.six as six'

Salt-based supportconfig. Re-implementation of Support Config Tool in Salt.

ghost requested review from a team September 1, 2018 16:37

cachedout added the Awesome label Sep 2, 2018

cachedout approved these changes Sep 4, 2018

View reviewed changes

rallytime requested review from cachedout, garethgreenaway and terminalmage September 7, 2018 17:58

rallytime approved these changes Sep 11, 2018

View reviewed changes

meaksh suggested changes Sep 11, 2018

View reviewed changes

scripts/salt-support Outdated Show resolved Hide resolved

isbm force-pushed the isbm-supportconfig branch from 41d2de3 to c333366 Compare September 12, 2018 09:48

cachedout reviewed Sep 16, 2018

View reviewed changes

cachedout reviewed Sep 25, 2018

View reviewed changes

salt/cli/support/collector.py Outdated Show resolved Hide resolved

terminalmage approved these changes Sep 27, 2018

View reviewed changes

Bo Maryniuk added 7 commits September 28, 2018 16:55

Add support option parser

b3c3e57

Add support script function

cef3963

Add salt-support starter

af94717

Initial support wrapper

866b967

Add data collector skeleton

5a8f062

Add default scenario of the support configuration

bd8db3d

Add main flow for the collector.

00ddf20

Bo Maryniuk added 11 commits September 28, 2018 16:55

Lintfix: PEP8 E302: expected 2 blank lines, found 0

158dd1b

Lintfix: use salt.utils.files.fopen instead of open

a7a4386

Lintfix: PEP8 E303: too many blank lines (3)

96b994a

Lintfix: Uses of an external blacklisted import 'six': Please use 'im…

f6cb329

…port salt.ext.six as six'

Lintfix: use salt.utils.files.fopen instead of open

8929bf2

Fix unit tests

b824fa6

Fix six import

1b02d13

Mute pylint: file handler explicitly needed

8920432

Lintfix: explicitly close filehandle

5345b47

Lintfix: mute fopen warning

2d86563

Remove development stub. Ughh...

870b382

isbm force-pushed the isbm-supportconfig branch from 3f0b2b9 to 870b382 Compare September 28, 2018 14:55

Removed blacklist of pkg_resources

f844bc9

cachedout merged commit 71531e3 into saltstack:develop Sep 29, 2018

isbm deleted the isbm-supportconfig branch October 21, 2018 13:52

sagetherage added code-jam Has Testcase labels Apr 15, 2020

DmitryKuzmenko mentioned this pull request Apr 22, 2020

Port #49480 to master #56830

Closed

DmitryKuzmenko added the has master-port port to master has been created label Apr 22, 2020

DmitryKuzmenko pushed a commit to DSRCorporation/salt that referenced this pull request Apr 28, 2020

Port saltstack#49480 to master

86b2b46

Salt-based supportconfig. Re-implementation of Support Config Tool in Salt.

terminalmage mentioned this pull request May 2, 2020

Port saltsupport to master #57054

Closed

terminalmage pushed a commit to terminalmage/salt that referenced this pull request Aug 31, 2020

Port saltstack#49480 to master

7cd8436

Salt-based supportconfig. Re-implementation of Support Config Tool in Salt.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Salt-based supportconfig #49480

Salt-based supportconfig #49480

isbm commented Sep 1, 2018 •

edited

Loading

isbm commented Sep 1, 2018

cachedout commented Sep 2, 2018

cachedout commented Sep 3, 2018

isbm commented Sep 3, 2018

isbm commented Sep 3, 2018

isbm commented Sep 4, 2018 •

edited

Loading

cachedout left a comment

isbm commented Sep 7, 2018

rallytime commented Sep 7, 2018

isbm commented Sep 10, 2018

rallytime commented Sep 11, 2018

rallytime left a comment

meaksh left a comment

cachedout Sep 16, 2018

cachedout Sep 16, 2018

cachedout Sep 16, 2018

cachedout Sep 16, 2018

		return env.get_template(os.path.basename(path)).render(salt=caller, runners=runner).strip()


		def get_profile(profile, caller, runner):

Salt-based supportconfig #49480

Salt-based supportconfig #49480

Conversation

isbm commented Sep 1, 2018 • edited Loading

What does this PR do?

What is this?

Support Scenarios vs States

Overview

Typical Usage

What happens if I run it?

What is inside the archive?

Jinja in Support Profile!?

Limitations

Tests written?

isbm commented Sep 1, 2018

cachedout commented Sep 2, 2018

cachedout commented Sep 3, 2018

isbm commented Sep 3, 2018

isbm commented Sep 3, 2018

isbm commented Sep 4, 2018 • edited Loading

cachedout left a comment

Choose a reason for hiding this comment

isbm commented Sep 7, 2018

rallytime commented Sep 7, 2018

isbm commented Sep 10, 2018

rallytime commented Sep 11, 2018

rallytime left a comment

Choose a reason for hiding this comment

meaksh left a comment

Choose a reason for hiding this comment

cachedout Sep 16, 2018

Choose a reason for hiding this comment

cachedout Sep 16, 2018

Choose a reason for hiding this comment

cachedout Sep 16, 2018

Choose a reason for hiding this comment

cachedout Sep 16, 2018

Choose a reason for hiding this comment

isbm commented Sep 1, 2018 •

edited

Loading

isbm commented Sep 4, 2018 •

edited

Loading