Allow user to force terminal size used by the CLI formatters by setting environment variable #4242

Kami · 2018-07-13T11:58:57Z

Right now, we try various approaches to obtain terminal size used by the CLI for table formatting purposes.

If none of that approaches work, we fall back to a default hard coded size (10 lines, 20 columns).

This pull request improves on that in two ways:

It uses a more reasonable default size. 20 columns for a total terminal width is simply a bad default. It means pretty much every table cell / column will wrap at couple of characters which will make output unreadable when we are unable to retrieve a terminal size.
~~Allows user to force terminal size using ST2_CLI_FORCE_TERMINAL_SIZE=<lines,columns> environment variable.~~

We have experienced this "issue" a lot in our end to end tests for a long time. Since those test commands don't run using shell, we fail to obtain a terminal size and we fall back to a really bad default which results in an unreadable output like that:

Before:

After:

Keep in mind that those actions run on a remote host which is later destroyed so we can't simply re-retrieve the output which means we are stuck with troubleshooting / debugging from this very hard to read output.

TODO

Changelog entry

This comes handy in scenarios where terminal size can't be obtained (e.g. when running tests outside tty).

arm4b · 2018-07-13T12:17:40Z

st2client/st2client/utils/terminal.py

    """
    :return: (lines, cols)
    """
+    # Allow user to force terminal size using a environment variables
+    # E.g. ST2_CLI_FORCE_TERMINAL_SIZE=80,200 # lines, columns


+1 for changing the default limits, but to force the terminal size we can rely on existing $COLUMNS and $LINES, without implementing new ST2_CLI_FORCE_TERMINAL_SIZE:

st2/st2client/st2client/utils/terminal.py

Lines 74 to 78 in 56f9cec

# try environment variables

try:

return tuple(int(os.getenv(var)) for var in ('LINES', 'COLUMNS'))

except:

pass

The problem is that this will only force it in case we fall back through all the other approaches to this very last one - aka it will only work if we can't retrieve the terminal size using other approaches.

User won't be able to force a different size in case we are able to retrieve the size using some other approach.

So that would be a different thing, more like ST2_CLI_FALLBACK_TERMINAL_SIZE or similar. But with the current approach, it can be overridden in any scenario since that environment variable has precedence over other approaches.

So ideally if st2 would rely on those vars.

For example, something like was typical while working with Docker:

kubectl exec -it ... env COLUMNS=80 LINES=200 TERM=xterm /bin/bash

Yes, it will still rely on those two environment variables if other method fails, but with this new environment variable there is also a way to override those values (e.g. in case some other method return a value, but user wants to use a different one without changing xterm settings, etc.).

variables first. This way it's more performant and consistent with upstream Python 3 implementation - https://github.com/python/cpython/blob/master/Lib/shutil.py#L1203. This way there is also no need for additional environment variables since those values are checked first.

Kami · 2018-07-13T13:45:16Z

@Arma and I discussed this some more and decided to go with this approach - 471b5ef.

Now we check LINES and COLUMNS environment variables first so there is no need for additional environment variables.

This way it's also consistent with upstream Python 3 implementation of get_terminal_size (https://github.com/python/cpython/blob/master/Lib/shutil.py#L1203).

Keep in mind that we need to maintain our own implementation because this function is not available in Python 2.

arm4b · 2018-07-13T13:58:20Z

st2client/st2client/utils/terminal.py

+    # checked those variables at the end as a fall back.
+    try:
+        lines = os.environ['LINES']
+        columns = os.environ['COLUMNS']


👍 on approach, but I think original implementation is safer on corner cases here:
https://github.com/python/cpython/blob/master/Lib/shutil.py#L1223-L1231

For example, when only LINES or only COLUMNS can be overridden by the user.

Yeah, agreed - our implementation requires both variables to be set (but we actually only use columns and ignore lines all together).

Original implementation allows you to specify those independently. I was thinking of making that change as well, but it's a lot more involved than just doing what I do right now (and previous code also requires both to be specified).

Setting only one var and expect it to take precedence is a valid behavior with LINES vs COLUMNS. It would be a surprise that only 2 alltogether would work.

We're following the established standards here. Ideally if we won't surprise user in a bad way :)

We can use a waterfall of fallbacks as in original .py implementation:
https://github.com/python/cpython/blob/master/Lib/shutil.py#L1233-L1234
https://github.com/python/cpython/blob/master/Lib/shutil.py#L1241-L1244

If we change the implementation to read columns only, get_terminal_size function wouldn't do what it's advertised/expected to do.

We only use columns value of terminal size so this allows us to simplify code for retrieving terminal size.

arm4b · 2018-07-13T15:55:41Z

st2client/st2client/utils/terminal.py

 import os
 import struct
 import subprocess
 import sys

 from st2client.utils.color import format_status

+DEFAULT_TERMINAL_SIZE_LINES = 80


Can remove as never needed/used

warrenvw · 2018-07-13T20:20:34Z

Thanks. I guess we won't need StackStorm/st2-docker#102 anymore... 👍

Kami added 4 commits July 13, 2018 13:48

Allow user to force terminal size used by the st2 CLI formattes.

99e4f62

This comes handy in scenarios where terminal size can't be obtained (e.g. when running tests outside tty).

Also use a more reasonable default terminal size.

df5afde

200 -> 150..

c9ad392

Make sure we cast it to int.

e59291d

Kami added this to the 2.9.0 milestone Jul 13, 2018

Kami added 3 commits July 13, 2018 14:01

Truncate extra whitespace.

921fe11

Add changelog entry.

208b593

Rewording.

56f9cec

arm4b reviewed Jul 13, 2018

View reviewed changes

arm4b suggested changes Jul 13, 2018

View reviewed changes

arm4b added the enhancement label Jul 13, 2018

Kami added 4 commits July 13, 2018 17:01

Add a note.

a034c06

Add tests for get_terminal_size.

cf50187

Number the various fallbacks.

62c61cc

Replace get_terminal_size with get_terminal_size_columns.

9530889

We only use columns value of terminal size so this allows us to simplify code for retrieving terminal size.

arm4b approved these changes Jul 13, 2018

View reviewed changes

arm4b reviewed Jul 13, 2018

View reviewed changes

Remove unused variable.

e2d2c8a

Kami merged commit a800b30 into master Jul 13, 2018

Kami deleted the st2_client_default_column_width branch July 13, 2018 17:52

Kami mentioned this pull request Jul 16, 2018

Changes for v2.8.1 release #4237

Merged

Kami modified the milestones: 2.9.0, 2.8.1 Jul 16, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow user to force terminal size used by the CLI formatters by setting environment variable #4242

Allow user to force terminal size used by the CLI formatters by setting environment variable #4242

Kami commented Jul 13, 2018 •

edited

Loading

arm4b Jul 13, 2018 •

edited

Loading

Kami Jul 13, 2018 •

edited

Loading

arm4b Jul 13, 2018

Kami Jul 13, 2018 •

edited

Loading

Kami commented Jul 13, 2018

arm4b Jul 13, 2018 •

edited

Loading

Kami Jul 13, 2018

arm4b Jul 13, 2018

arm4b Jul 13, 2018 •

edited

Loading

warrenvw commented Jul 13, 2018

	# try environment variables
	try:
	return tuple(int(os.getenv(var)) for var in ('LINES', 'COLUMNS'))
	except:
	pass

Allow user to force terminal size used by the CLI formatters by setting environment variable #4242

Allow user to force terminal size used by the CLI formatters by setting environment variable #4242

Conversation

Kami commented Jul 13, 2018 • edited Loading

TODO

arm4b Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

Kami Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

arm4b Jul 13, 2018

Choose a reason for hiding this comment

Kami Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

Kami commented Jul 13, 2018

arm4b Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

Kami Jul 13, 2018

Choose a reason for hiding this comment

arm4b Jul 13, 2018

Choose a reason for hiding this comment

arm4b Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

warrenvw commented Jul 13, 2018

Kami commented Jul 13, 2018 •

edited

Loading

arm4b Jul 13, 2018 •

edited

Loading

Kami Jul 13, 2018 •

edited

Loading

Kami Jul 13, 2018 •

edited

Loading

arm4b Jul 13, 2018 •

edited

Loading

arm4b Jul 13, 2018 •

edited

Loading