[bash-completion] Slow when entering make xxx commands when building kernel

[blackteahamburger]

ble version: 0.4.0-devel4+9641c3b
Bash version: 5.1.16(1)-release (x86_64-pc-linux-gnu)

Slow when entering kernel make xxx (e.g. make menuconfig, make install) commands when building kernel.

[akinomyoga]

Thank you for the report. I first would like to confirm the problem. The answer would largely depend on the situation, and I have to avoid preparing answers for every case because the answer would tend to be long in any case.

• What is slow? I guess that when you input make xxx into ble.sh in the source directory of the Linux kernel, the response of ble.sh becomes slow. Is that what you mean? For another possibility, maybe you mean that the command make xxx doesn't start soon when you run the command by hitting RET or C-m? Or, the execution time of the command make xxx becomes longer?
• What does "when building kernel" mean? Literally, it reads like you input make xxx into ble.sh while you are building the Linux kernel in the background. Another possibility is that you are just inside the source directory of the Linux kernel (without building the Linux kernel in the background)?
• BTW, it's not specified, but are you talking about the Linux kernel, but not any other kernel, right?

If you are running the build process in the background, it is somewhat expected. Building the Linux kernel consumes much computational resources. Every application can be slow while building the kernel. It's not specific to ble.sh. Maybe the plain Bash doesn't become as slow as ble.sh while building the kernel, but it is related to how much stuff ble.sh does more than the plain Bash. In particular, ble.sh touches filesystems to generate the candidates for autosuggestions while you are typing. This can be particularly slow since the I/O would likely be blocking while building the Linux kernel.

[blackteahamburger]

Thank you for the report. I first would like to confirm the problem. The answer would largely depend on the situation, and I have to avoid preparing answers for every case because the answer would tend to be long in any case.

• What is slow? I guess that when you input make xxx into ble.sh in the source directory of the Linux kernel, the response of ble.sh becomes slow. Is that what you mean? For another possibility, maybe you mean that the command make xxx doesn't start soon when you run the command by hitting RET or C-m? Or, the execution time of the command make xxx becomes longer?
• What does "when building kernel" mean? Literally, it reads like you input make xxx into ble.sh while you are building the Linux kernel in the background. Another possibility is that you are just inside the source directory of the Linux kernel (without building the Linux kernel in the background)?
• BTW, it's not specified, but are you talking about the Linux kernel, but not any other kernel, right?

If you are running the build process in the background, it is somewhat expected. Building the Linux kernel consumes much computational resources. Every application can be slow while building the kernel. It's not specific to ble.sh. Maybe the plain Bash doesn't become as slow as ble.sh while building the kernel, but it is related to how much stuff ble.sh does more than the plain Bash. In particular, ble.sh touches filesystems to generate the candidates for autosuggestions while you are typing. This can be particularly slow since the I/O would likely be blocking while building the Linux kernel.

I mean when I input make xxx into ble.sh in the source directory of the Linux kernel inside the source directory of the Linux kernel (without building the Linux kernel in the background), the response of ble.sh becomes slow.

Sorry I didn't make it clear.

[akinomyoga]

OK, thank you for your clarification! I'd first suspect the completion setting for the make command. I think it would be the one from bash-completion. This is merely a guess, so I'd like to know the answers to the following questions to confirm that:

• Q1: Does the response become faster when you turn off the autosuggestions? (Note that this is a temporary setting just for testing. I'm not requesting you to turn off the autosuggestions for daily use.)

$ bleopt complete_auto_complete=
$ make xxx # <-- Is the response faster here?

• Q2: What is the result of the following command?

$ complete -p make

[blackteahamburger]

Q1: Yes.
Q2: Before typing make xxx:

bash: complete: make: no completion specification

After typing make xxx:

complete -F _make make

[akinomyoga]

Thank you. The results you provided imply that the delay is caused by the processing of auto-complete (autosuggestion feature) and is particularly related to the make completion in the release version of bash-completion (<= 2.11).

• Q3: Could you check the results of the following two commands (edit: inside the source directory of the Linux kernel)? I'd like to check the execution time and the number of lines of each step.

$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT &>/dev/null | wc -l
$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT 2>/dev/null | sed -ne "$(_make_target_extract_script)" | wc -l

[blackteahamburger]

Q3:

$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT &>/dev/null | wc -l
0
# stuck here
$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT 2>/dev/null | sed -ne "$(_make_target_extract_script)" | wc -l
bash: _make_target_extract_script: command not found
# stuck here

[akinomyoga]

$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT &>/dev/null | wc -l
0
# stuck here

Thank you for checking.

• Q4: When you check that it is stuck, how long did you wait? Is it more than a second or more than one minute? I'd like to know the order of the time duration it takes.

$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT 2>/dev/null | sed -ne "$(_make_target_extract_script)" | wc -l
bash: _make_target_extract_script: command not found
# stuck here

This means that the bash-completion version seems different from mine.

• Q5: Could you provide the result of the following command?

$ ble/widget/display-shell-version

• Q6: Could you provide the output of the following command (after inputting make xxx)?

$ type _make

[blackteahamburger]

Q4: It seems to be stuck there forever.

But after inputting make xxx, Q3's answers are different:

$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT &>/dev/null | wc -l
0

real    0m0.736s
user    0m0.459s
sys     0m0.298s
$ time make -npq __BASH_MAKE_COMPLETION__=1 .DEFAULT 2>/dev/null | sed -ne "$(_make_target_extract_script)" | wc -l
0

real    0m0.649s
user    0m0.380s
sys     0m0.320s

Q5:

GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu) [Gentoo Linux]
ble.sh, version 0.4.0-devel4+9641c3b (noarch) [git 2.41.0, GNU Make 4.4.1, GNU Awk 5.3.0, API 4.0]
bash-completion, version 2.11 (hash:1a7f5634b4c9816716b69a7824e0b51cfde2bca3, 76332 bytes) (noarch)
oh-my-bash (agnoster), version 1.0.0+8c687e4e (noarch), aliases(general package-manager ls), completions(git composer ssh), plugins(git bashmarks)
locale: LANG=zh_CN.utf8 LC_CTYPE=zh_CN.utf8
terminal: TERM=xterm-256color wcwidth=15.0-west/15.1-2+ri, konsole:220380 (1;115;0)

Q6:

_make is a function
_make () 
{ 
    local cur prev words cword split;
    _init_completion -s || return;
    local makef makef_dir=("-C" ".") i;
    case $prev in 
        --file | --makefile | --old-file | --assume-old | --what-if | --new-file | --assume-new | -!(-*)[foW])
            _filedir;
            return
        ;;
        --include-dir | --directory | -!(-*)[ICm])
            _filedir -d;
            return
        ;;
        -!(-*)E)
            COMPREPLY=($(compgen -v -- "$cur"));
            return
        ;;
        --eval | -!(-*)[DVx])
            return
        ;;
        --jobs | -!(-*)j)
            COMPREPLY=($(compgen -W "{1..$(($(_ncpus) * 2))}" -- "$cur"));
            return
        ;;
    esac;
    $split && return;
    if [[ $cur == -* ]]; then
        local opts="$(_parse_help "$1")";
        COMPREPLY=($(compgen -W '${opts:-$(_parse_usage "$1")}' -- "$cur"));
        [[ ${COMPREPLY-} == *= ]] && compopt -o nospace;
    else
        if [[ $cur == *=* ]]; then
            prev=${cur%%=*};
            cur=${cur#*=};
            local diropt;
            [[ ${prev,,} == *dir?(ectory) ]] && diropt=-d;
            _filedir $diropt;
        else
            for ((i = 1; i < ${#words[@]}; i++))
            do
                if [[ ${words[i]} == -@(C|-directory) ]]; then
                    eval "makef_dir=( -C \"${words[i + 1]}\" )";
                    break;
                fi;
            done;
            for ((i = 1; i < ${#words[@]}; i++))
            do
                if [[ ${words[i]} == -@(f|-?(make)file) ]]; then
                    eval "makef=( -f \"${words[i + 1]}\" )";
                    break;
                fi;
            done;
            local mode=--;
            if ((COMP_TYPE != 9)); then
                mode=-d;
            fi;
            local IFS=' 
' script=$(_make_target_extract_script $mode "$cur");
            COMPREPLY=($(LC_ALL=C             $1 -npq __BASH_MAKE_COMPLETION__=1             ${makef+"${makef[@]}"} "${makef_dir[@]}" .DEFAULT 2>/dev/null |
            command sed -ne "$script"));
            if [[ $mode != -d ]]; then
                [[ ${COMPREPLY-} == */ ]] && compopt -o nospace;
            fi;
        fi;
    fi
}

[akinomyoga]

Q4: It seems to be stuck there forever.

But after inputting make xxx, Q3's answers are different:

Thanks for those results. A major delay seems to come from the execution time of make itself.

I made an experimental patch.

• Q7: Could you temporarily add the following code at the end of bashrc and see if the behavior in the source directory of the Linux kernel improves?

# End of ~/.bashrc

__load_completion make
function ble/complete/progcomp/patch:bash-completion/_comp_make.advice {
  ble/function#push "${ADVICE_WORDS[1]}" '
      local -a make_args; make_args=("${ADVICE_WORDS[1]}" "$@")
      ble/util/conditional-sync \
        '\''command "${make_args[@]}"'\'' \
        "! ble/complete/check-cancel <&$_ble_util_fd_stdin" 128 progressive-weight:killall'
  ble/function#advice/do
  ble/function#pop "${ADVICE_WORDS[1]}"
}
ble/function#advice around _make ble/complete/progcomp/patch:bash-completion/_comp_make.advice

[blackteahamburger]

Q7: Yes.

[akinomyoga]

Thank you!

Q7: Yes.

What does that mean? "You can add the setting and see it" or "The behavior improves"?

[blackteahamburger]

Thank you!

Q7: Yes.

What does that mean? "You can add the setting and see it" or "The behavior improves"?

The behavior improves.

[akinomyoga]

Thank you! Then, I'll later add these settings in the master version with some adjustments.

[akinomyoga]

I added those workarounds in commit 5f3a001. Could you check if the problem doesn't arise in the latest version (without having the adjustments in #389 (comment) in your own .bashrc)?

[blackteahamburger]

Solved, thank you!

[akinomyoga]

Thank you for checking! Also, thank you for your report and patience!