mate-terminal shows only 8 colors

[jmlich]

Expected behaviour

I believe mate-terminal should allow 16 colors instead of 8.

Actual behaviour

Second row of colors is not reachable

Steps to reproduce the behaviour

    for fg_color in {0..15}; do
        set_foreground=$(tput setaf $fg_color)
        for bg_color in {0..15}; do
            set_background=$(tput setab $bg_color)
            echo -n $set_background$set_foreground
            printf ' F:%s B:%s ' $fg_color $bg_color
        done
        echo $(tput sgr0)
    done

tput colors

MATE general version

1.19.1

$ dnf info mate-desktop
Installed Packages
Name         : mate-desktop
Version      : 1.19.1
Release      : 1.fc27
Arch         : x86_64
Size         : 179 k
Source       : mate-desktop-1.19.1-1.fc27.src.rpm
Repo         : @System
From repo    : fedora
Summary      : Shared code for mate-panel, mate-session, mate-file-manager, etc
URL          : http://mate-desktop.org
License      : GPLv2+ and LGPLv2+ and MIT
Description  : The mate-desktop package contains an internal library
             : (libmatedesktop) used to implement some portions of the MATE
             : desktop, and also some data files and other shared components of the
             : MATE user environment.

Package version

$ dnf info mate-terminal
Installed Packages
Name         : mate-terminal
Version      : 1.19.0
Release      : 1.fc27
Arch         : x86_64
Size         : 5.8 M
Source       : mate-terminal-1.19.0-1.fc27.src.rpm
Repo         : @System
From repo    : fedora
Summary      : Terminal emulator for MATE
URL          : http://mate-desktop.org
License      : GPLv3+
Description  : Mate-terminal is a terminal emulator for MATE. It supports translucent
             : backgrounds, opening multiple terminals in a single window (tabs) and
             : clickable URLs.

Linux Distribution

$ cat /etc/redhat-release 
Fedora release 27 (Twenty Seven)
$ uname -a
Linux dell 4.13.13-300.fc27.x86_64 #1 SMP Wed Nov 15 15:47:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Link to downstream report of your Distribution

https://bugzilla.redhat.com/show_bug.cgi?id=1517870

[alexarnaud]

Le 27/11/2017 à 16:51, Jozef Mlich a écrit :

I believe mate-terminal should allow 16 colors instead of 8.

Does gnome-terminal support 16 colors ? Best regards. -- Alex ARNAUD Visual-Impairment Project Manager Hypra - "Humanizing technology"

[jmlich]

gnome terminal says 256 colors.

$ tput colors
256

[egmontkob]

VTE and hence in turn gnome-terminal changed its default to TERM=xterm-256color 3 years ago. Apparently mate-terminal reverts it to xterm in src/terminal-screen.c get_child_environment(). I guess it shouldn't do that.

And while we're at it, you might want to revise the preceding line about COLORTERM too, see here, although VTE overrides that, no clue why.

[egmontkob]

Let's also note that the script isn't quite correct either. setaf/setab shouldn't be called with indices higher than or equal to the number of colors supported (as reported by tput colors), the behavior is undefined (side effects allowed) in that case.

For TERM=xterm, tput setab | cat -v outputs ^[[4%p1%dm. That is, the numeric argument it prints is the digit 4 followed by whatever the parameter is.

For parameters 0-7 it outputs 40-47 which are the 8 colors. For index 8 it outputs 48 which on its own forms an incorrect escape sequence (it'd require additional parameters for 256-color or truecolor codes), hence the background color remains unchanged (gray/white for that column). For index 9 it outputs 49 which resets the background color. For indices 10-15 it outputs the values... drumroll... 410-415 which are invalid.

(Same story with setaf and the initial digit being 3.)

[jmlich]

Should I add

export TERM=xterm-256color

into ~/.bashrc as a workaround?

[egmontkob]

I guess that should do as a workaround for the time being. Make it conditional to TERM being xterm though, you don't want to break the Linux console or screen/tmux etc. by altering their totally different TERM setting.

[monsta]

So, is it enough to drop two lines? It's an easy fix then.

https://git.gnome.org/browse/gnome-terminal/commit/?id=a0ecd018f070b96a6ca2ff778ddda0f7fde5b847
https://git.gnome.org/browse/gnome-terminal/commit/?id=1d5c1b6ca6373c1301494edbc9e43c3e6a9c9aaf

[egmontkob]

I guess it should be that easy, give it a try to see for sure.

[monsta]

@jmlich: I've created #221 for testing.

[jmlich]

@monsta I have tried to recompile e6ccc1f, but it looks like it doesn't fix the issue.

"tput colors" reports still 8 and "echo $TERM" says "xterm" (not "xterm-256color" as I expected)

I am not sure if I am doing everything correctly.

[egmontkob]

I've tested it and it works as expected for me.

@jmlich Perhaps you alter $TERM in your shell startup files (or your distribution does that for you). I do the same, just the other way around, from xterm to xterm-256color :-)

To test, I've switched to another shell under Profile Preferences -> Title and Command, and verified that this change indeed changes $TERM from xterm to xterm-256color. Then, as another round of testing, to be absolutely sure that no one has a chance to tamper with the variables (not even the other shell's startup files), I've even tried printenv as a Custom command, combined with When command exits: Hold the terminal open, and then used the terminal's Search feature.

[jmlich]

Maybe some script in /etc/profile.d/* changes it. I am not able to find it. )-; The colors works correctly on my computer only if I set TERM=xterm-256color variable in ~/.bashrc. That change was fixing the issue even before that commit. Honestly, I don't understand where the default value comes from. I will be happy, if someone can test it also on fedora.

[monsta]

For me it works:

monsta@asylum ~ $ tput colors
256
monsta@asylum ~ $ echo $TERM
xterm-256color
monsta@asylum ~ $ echo $COLORTERM

What's in your /etc/profile.d/ dir? I have three files, and only one is related (it's from VTE):

monsta@asylum ~ $ ls -l /etc/profile.d/
итого 12
-rw-r--r-- 1 root root  663 ноя 10  2015 bash_completion.sh
-rw-r--r-- 1 root root  294 окт 26 14:21 flatpak.sh
-rw-r--r-- 1 root root 1941 мар 16  2016 vte-2.91.sh
monsta@asylum ~ $ apt contains /etc/profile.d/vte-2.91.sh
libvte-2.91-common: /etc/profile.d/vte-2.91.sh

[raveit65]

Sadly, we have a heavy regression with the fix, see my comments in PR.

[jmlich]

[jmlich@jmlich-dell ~$ echo $TERM
xterm
[jmlich@jmlich-dell ~$ echo $COLORTERM
truecolor
[jmlich@jmlich-dell ~$ tput colors
8
[jmlich@jmlich-dell ~$ ls -l /etc/profile.d/
total 136
-rw-r--r--. 1 root root  664 Jul 26 05:51 bash_completion.sh
-rw-r--r--. 1 root root  987 Aug  2 20:47 ccache.csh
-rw-r--r--. 1 root root 1000 Aug  2 20:47 ccache.sh
-rw-r--r--. 1 root root  196 Aug  3 10:18 colorgrep.csh
-rw-r--r--. 1 root root  201 Aug  3 10:18 colorgrep.sh
-rw-r--r--. 1 root root 1741 Nov 10 18:53 colorls.csh
-rw-r--r--. 1 root root 1606 Nov 10 18:53 colorls.sh
-rw-r--r--. 1 root root   69 Aug  5 01:53 colorsysstat.csh
-rw-r--r--. 1 root root   56 Aug  5 01:53 colorsysstat.sh
-rw-r--r--. 1 root root  162 Aug  5 08:00 colorxzgrep.csh
-rw-r--r--. 1 root root  183 Aug  5 08:00 colorxzgrep.sh
-rw-r--r--. 1 root root  216 Aug  3 10:57 colorzgrep.csh
-rw-r--r--. 1 root root  220 Aug  3 10:57 colorzgrep.sh
-rw-r--r--. 1 root root   92 Aug 14 12:18 cvs.csh
-rw-r--r--. 1 root root   78 Aug 14 12:18 cvs.sh
-rw-r--r--. 1 root root  294 Dec 21 11:46 flatpak.sh
-rwxr-xr-x. 1 root root  249 Sep 21 09:40 kde.csh
-rwxr-xr-x. 1 root root  288 Sep 21 09:40 kde.sh
-rw-r--r--. 1 root root 1706 Jan  2 16:36 lang.csh
-rw-r--r--. 1 root root 2703 Jan  2 16:36 lang.sh
-rw-r--r--. 1 root root  500 Aug  3 17:02 less.csh
-rw-r--r--. 1 root root  253 Aug  3 17:02 less.sh
-rwxr-xr-x. 1 root root   49 Nov 27 16:14 mc.csh
-rwxr-xr-x. 1 root root  153 Nov 27 16:14 mc.sh
lrwxrwxrwx. 1 root root   29 Nov 27 09:51 modules.csh -> /etc/alternatives/modules.csh
lrwxrwxrwx. 1 root root   28 Nov 27 09:51 modules.sh -> /etc/alternatives/modules.sh
-rw-r--r--. 1 root root 1202 Jan  9 14:32 PackageKit.sh
-rw-r--r--. 1 root root  284 Aug 25 10:23 scl-init.csh
-rw-r--r--. 1 root root  637 Aug 25 10:23 scl-init.sh
-rw-r--r--. 1 root root  106 Jan  2 13:21 vim.csh
-rw-r--r--. 1 root root  248 Jan  2 13:21 vim.sh
-rw-r--r--. 1 root root 2092 Nov  2 15:21 vte.sh
-rw-r--r--. 1 root root  120 Aug  5 05:29 which2.csh
-rw-r--r--. 1 root root  157 Aug  5 05:29 which2.sh
-rw-r--r--. 1 root root  113 Jan  9 12:00 zz-modules-python-sphinx.csh
-rw-r--r--. 1 root root  113 Jan  9 12:00 zz-modules-python-sphinx.sh

[raveit65]

My results:

[rave@mother ~]$ $TERM
bash: xterm-256color: command not found
[rave@mother ~]$ echo $TERM
xterm-256color
[rave@mother ~]$ echo $COLORTERM
truecolor
[rave@mother ~]$ tput colors
256
[rave@mother ~]$ ls -l /etc/profile.d/
total 88
-rw-r--r--. 1 root root  771 May 26  2017 256term.csh
-rw-r--r--. 1 root root  841 May 26  2017 256term.sh
-rw-r--r--. 1 root root  664 Jun 27  2017 bash_completion.sh
-rw-r--r--. 1 root root  196 Jul  3  2017 colorgrep.csh
-rw-r--r--. 1 root root  201 Jul  3  2017 colorgrep.sh
-rw-r--r--. 1 root root 1741 Nov 10 18:57 colorls.csh
-rw-r--r--. 1 root root 1606 Nov 10 18:57 colorls.sh
-rw-r--r--. 1 root root  162 Feb 12  2017 colorxzgrep.csh
-rw-r--r--. 1 root root  183 Feb 12  2017 colorxzgrep.sh
-rw-r--r--. 1 root root  216 Feb 10  2017 colorzgrep.csh
-rw-r--r--. 1 root root  220 Feb 10  2017 colorzgrep.sh
-rw-r--r--. 1 root root   92 Aug 14 12:19 cvs.csh
-rw-r--r--. 1 root root   78 Aug 14 12:19 cvs.sh
-rw-r--r--. 1 root root 1706 May 26  2017 lang.csh
-rw-r--r--. 1 root root 2703 May 26  2017 lang.sh
-rw-r--r--. 1 root root  500 May 16  2017 less.csh
-rw-r--r--. 1 root root  253 May 16  2017 less.sh
-rw-r--r--. 1 root root  106 Jan  2 14:45 vim.csh
-rw-r--r--. 1 root root  248 Jan  2 14:45 vim.sh
-rw-r--r--. 1 root root 2092 Oct  5 14:36 vte.sh
-rw-r--r--. 1 root root  120 Feb 12  2017 which2.csh
-rw-r--r--. 1 root root  157 Feb 12  2017 which2.sh

[raveit65]

@egmontkob

What does tput colors output to you? Also, what do the commands TERM=xterm tput colors and TERM=xterm-256color tput colors print?

[rave@mother ~]$ tput colors
256

[rave@mother ~]$ TERM=xterm tput colors
8

[rave@mother ~]$ TERM=xterm-256color tput colors
256

[raveit65]

I haven't mentioned that in #209 explicitly, but the problem came with upgrade from fedora 26 to fedora 27.

Ok, i will test it later how it works with f27.
I am not sure at the moment if i ever will update f26 with mate-1.20, first fedora 27 will be updated from 1.19 to 1.20, but for me it isn't a problem to revert the commit for mate-terminal in fedora 26.

[raveit65]

@jmlich @egmontkob
So it depends on vte291 version?
We ship Mate for different distros with different vte versions.

[raveit65]

I see the same problems in f27 if i use that commit for mate-terminal.

[monsta]

Well, since it didn't fix the problem for @jmlich, we can revert it, I don't mind. Still, would be nice to find out why all that happens. I don't see these problems in Debian and Ubuntu.

[raveit65]

I found the culprit commit
https://git.gnome.org/browse/gnome-terminal/commit/?id=a0ecd018f070b96a6ca2ff778ddda0f7fde5b847
Removing this line in mate-terminal causes all mentioned problems.
The other commit from gnome works fine.
So i reverted only the commit from 2010.
8e3de22

[egmontkob]

Guys, at this point I absolutely do not understand anything about what and why you are doing.

You keep mentioning the change caused problems, while the only single problem you mentioned was git log not working correctly.

Not a single bit of worthwhile investigation happened on your side that would get to the bottom of the issue. I tried to help, but got answers only to some of my questions, and by the time I had a chance to continue to help suddenly the change was reverted and this issue was closed.

You keep ignoring the fact that most of the terminal emulators have advertised 256-color support for years now without known problems of such extent. Just do a random web search to see how popular request it is to use 256 colors (in fact, VTE supports 16 million colors a.k.a. true colors as well, and there's even a popular demand to advertise that too, unfortunately there's no official or widely accepted way to do that).

And after reverting the change, after making mate-terminal again advertise 8/16-color support only, you close the bug that requested the exact opposite?

[egmontkob]

TERM=dumb

Geez, I'm wondering where that comes from...

Could you please check the TERM value of mate-terminal itself, rather than the shells it launches? This should do it:

tr '\0' '\n' < /proc/$(pidof mate-terminal)/environ | grep ^TERM

This works nice

This could be a sufficient workaround then, until we find the actual problem (which might be in VTE, I'm really uncertain at this point.)

[egmontkob]

Just to make sure that we're still on the same page, could you please confirm this (merging your answers from two previous comments): In the same buggy mate-terminal and the same shell, these two commands give the answers as shown here:

$ printenv | grep ^TERM=
TERM=dumb
$ echo $TERM
xterm-256color

I have no idea how this could happen. In bash a variable may or may not be exported. If it's exported, both commands should report the same value. If it's not exported, printenv shouldn't show anything, only echo should print its value. According to my best knowledge, it's not possible to have different exported and non-exported values. I'm really puzzled. To double check: your shell is bash, isn't it?

[raveit65]

With mate-terminal-1.20.0-1.fc27.x86_64 (latest master, unreleased)

[rave@f27 ~]$ tr '\0' '\n' < /proc/$(pidof mate-terminal)/environ | grep ^TERM
TERM=dumb
[rave@f27 ~]$ printenv | grep ^TERM=
TERM=xterm
[rave@f27 ~]$ echo $TERM
xterm

with mate-terminal-1.20.0-2.fc27.x86_64 (broken version using https://github.com/raveit65/mate-terminal/commit/0ec221de105e753a6df56cfe276025e8d084ac3c)

[rave@f27 ~]$ tr '\0' '\n' < /proc/$(pidof mate-terminal)/environ | grep ^TERM                                
TERM=dumb
[rave@f27 ~]$ printenv | grep ^TERM=                                          
TERM=dumb
[rave@f27 ~]$ echo $TERM                                                      
dumb

with mate-terminal-1.20.0-3.fc27.x86_64 (using improved patch from #209 (comment))

[rave@f27 ~]$ tr '\0' '\n' < /proc/$(pidof mate-terminal)/environ | grep ^TERM
TERM=dumb
[rave@f27 ~]$ printenv | grep ^TERM=
TERM=xterm-256color
[rave@f27 ~]$ echo $TERM
xterm-256color

Hope this makes it clear to identify the issue.
Looks like 'dumb' comes from m-t itself ?

[egmontkob]

@raveit65 I still totally don't understand your observation from two weeks ago when echo $TERM reported xterm-256color and it was still broken. Let's put that aside as an unresolved mystery for now.

---

Your latest comment makes more sense to me. That dumb probably comes from the parent environment, maybe the window manager's or X startup script's default. Here gnome-terminal using dbus activation vs. mate-terminal starting up directly might also make a difference.

Seems to me that VTE setting up the default environment (including TERM=xterm-256color) is overridden by mate-terminal explicitly using its own env vars, which is a leftover from gnome-terminal. E.g. if I start up (from another terminal emulator) as TERM=foobar ./src/mate-terminal then inside that $TERM will still be foobar. VTE should take care of overriding it to xterm-256color (and does so in other terminal emulators), but in turn mate-terminal overrides it again, reverting VTE's fixes.

Could you please remove these lines, according to gnome-terminal commit e74c3e3fc? This fixes the issue for me, I'm curious to hear whether it works for you too. Thanks!

The next gnome-terminal commit aaa515ee6 might also be relevant, it's unclear to me what it does.

[raveit65]

@raveit65 I still totally don't understand your observation from two weeks ago when echo $TERM reported xterm-256color and it was still broken. Let's put that aside as an unresolved mystery for now.

Not my problem if you don't understand it....

Please open an PR and all is good.

[egmontkob]

Not my problem if you don't understand it....

This is not the first time your comments come across as somewhat offensive. I hope it's purely due to language barrier, neither of us being native English speaker, sometimes having a hard time expressing ourselves.

In your comments made about 2 weeks ago, you mentioned multiple times that you had TERM=xterm-256color and your system didn't work as expected. Even right now I have no idea how that could happen. If you can still reproduce this scenario, I'm happy to provide further help in order to track down the actual core issue.

It was 2 days ago that you first mentioned TERM being set to dumb, and in your posts made 1-2 days ago it's pretty consistent that things are broken with TERM=dumb while things work correctly with TERM=xterm-256color for you. That is, the situation has changed from 2 weeks ago. Things not working with TERM=dumb is not a surprise at all. Even though I have located the code for you that's responsible for reverting VTE's TERM=xterm-256color to dumb, I still have no idea about that behavior you faced 2 weeks ago, and I have no idea whether the suggested change fixes that or not.

Please open an PR and all is good.

I've never done PRs and it's not going to be now when I start doing them. Please go ahead and submit the patch on your own, if it indeed works for you. Thanks!

[monsta]

Ok, let's try #231 with the change suggested by @egmontkob. Just in case, I've made two commits, so you could test one or both.

[monsta]

Hmm, the test of #231 shows that this change isn't enough.
It's still the same in Fedora.

$ tr '\0' '\n' < /proc/$(pidof mate-terminal)/environ | grep ^TERM
TERM=dumb

For some reason I don't have TERM set in mate-terminal's environment in my Ubuntu 18.04 (and the above command returns nothing).

[egmontkob]

$ tr '\0' '\n' < /proc/$(pidof mate-terminal)/environ | grep ^TERM
TERM=dumb

This is okay and is irrelevant. There's no way mate-terminal could influence the environment mate-terminal itself is started up with. What matters is: is TERM set to xterm-256color in the shells it launches? For me it is (at 568ab78).

(To clarify: This information was useful for debugging. But it's not expected to see xterm-256color there.)

---

The remaining problem is: How come that "git log" still doesn't work for raveit65? In #231 he confirms that now TERM is set to xterm-256color for him. In his comments made on Feb 4 & 5 in this thread (including attached strace logs), faulty behavior was always tied to TERM=dumb whereas TERM=xterm-256color worked correctly. I don't know what's going on (I already made it clear on Feb 6 that I didn't know). Maybe TERM is set to xterm-256color initially, and later at some point somehow it changes to dumb?? What happens in other terminal emulators that also default to xterm-256color?? Is this problem specific to him, or does it affect a broader audicence? Maybe all Fedora users?? I'm happy to further investigate IF (and only IF!) Or can anyone else reproduce this problem and help me work together?

[monsta]

What matters is: is TERM set to xterm-256color in the shells it launches?

Yes, echo $TERM shows it my Debian and Ubuntu systems, and as you can see in #231, it's so in Fedora too. The only difference is that "dumb" value in mate-terminal's environment, which only happens in Fedora. That's why I asked that question.

If that varaible is irrelevant, there must be something else that causes git log to see the terminal as "not fully functional"...

[monsta]

@jmlich: can you also give #231 a try?

[egmontkob]

WARNING: terminal is not fully functional – this is not printed by git but by less. Reading the source of less-487, if I'm not mistaken it's printed if any of these terminal capabilities is missing or empty:

• ce
• cd
• cl
• both ho and cm
• both ll and cm
• all of da, al and sr

[raveit65]

The issue with using 'git log' was gone after a session restart.
But wordwrap is broken for me with latest PR.
First command is with copying the command in standard terminal size.
Second command is after resizing the terminal.

[egmontkob]

@raveit65 Wow, submitting offensive comment, then removing it along with the targeted person's response, true gentleman behavior. Go ahead and delete this one too, will you please? Lucky for you the MATE project doesn't seem to have a code of conduct, otherwise most likely your previous comment would be enough to kick you out from the developers. And it doesn't count that you've deleted it, it's still present in some of our mailboxes, and most importantly, has reached me too.

@monsta And others, if you remove raveit65 from the project I'm happy to further help you. Until then I'm out of here, and I don't mind to the slightest bit if mate-terminal is going to be the last to catch up with innovation in the terminal emulator world. I just don't see the point in spending any second developing it then, why not use gnome2 as it was back then?

[egmontkob]

@raveit65 Oh, and just one very last thing: When someone says (and not for the first time) that they were somewhat offended by your style, then the correct reaction is to go like "sorry, I didn't mean that, I'll try to take care" or something similar, but most definitely NEVER EVER to intentionally further offend them!!!

[lukefromdc]

I vote for keeping @raveit65 on the grounds that he has done good work. Also, we should all keep in mind: the competition is Microsoft and Apple, not each other. The enemy is patent trolls and those who try to impose government-mandated backdoors or lock bootloaders and chipsets against free software, not each other.

[sunweaver]

@raveit65: @egmontkob: I have just followed up on your "sort-of-heated debate" above and wonder if we could find a technical solution for the above issue.

It feels that the below commits might be helpful to achieve this goal...

Could you please remove these lines, according to gnome-terminal commit e74c3e3fc? This fixes the issue for me, I'm curious to hear whether it works for you too. Thanks!

The next gnome-terminal commit aaa515ee6 might also be relevant, it's unclear to me what it does.

THANKS!
Mike

[bwat47]

This is a pretty big usability issue IMO. Some common terminal applications such as htop look absolutely horrible under mate-terminal

[rubberbandaid]

@egmontkob Are you insane? @raveit65 has done an incredible amount for the MATE project, which I use and enjoy every single day, he absolutely must stay. To even suggest anything otherwise is unreal, do you even look at who is committing changes? He is a rockstar. Stop being a baby. I pray MATE never gets a code of conduct, so ungrateful users like you can't attack developers with it.

Your sense of entitlement to other people's time is uncomfortably embarrassing, even very early on in this thread. Good day.

[raveit65]

I am happy to forget everything what has been said here and start again from zero.
@monsta
Can you please handle the issue again?
Than i can test it again if you have an working solution.

[n3dst4]

Well this thread certainly was a rollercoaster. Not sure if this helps anyone but putting export TERM=xterm-256color in my .zshrc/.bashrc is a usable workaround for me, but if this was fixed at source I'd love MATE even more than I already do.

[lanefu]

another workaround is set this as your executing terminal command in your profile
env TERM=xterm-256color /bin/bash -i

[mfursov]

I had the same problem on one of my desktops and fixed it that way:

#export TERM=xterm-256color
export TERM=gnome-256color

now "tput colors" shows 256.