Failed to kill Runner: rpc error: code = Unavailable desc = connection error: desc

[asarkisian]

I am experiencing some issues which I cannot seem to resolve. I am constantly getting this error message and upon further inspection of logs/gauge.log, I can view the logs however it does not provide a stack trace. I'm hoping someone can shed some light on this matter. I will provide the console output as well as the log output. This is very difficult to debug as there is absolutely no stack trace. I don't see the stack trace in either the console or the log file.

Versions
Python Version: Python 3.7.0
Gauge version: 1.1.7
Commit Hash: 5d86b72

Plugins
flash (0.0.2)
html-report (4.0.12)
python (0.3.14)
reportserver (0.3.0)
screenshot (0.0.1)
spectacle (0.1.4)
xml-report (0.2.3)

Requirements.txt:
Appium-Python-Client==1.0.2
beautifulsoup4==4.9.1
browsermob-proxy==0.8.0
boto3==1.16.14
debugpy==1.2.1
decorator==4.4.2
getgauge==0.3.14
google==3.0.0
gTTs==2.2.1
psutil==5.8.0
python-vlc==3.0.11115
requests==2.24.0
requests-oauthlib==1.3.0
selenium==3.141.0
selenium-page-factory==2.1
tcp-latency==0.0.10
paramiko==2.7.2
fabric==2.6.0

CONSOLE OUTPUT
Executing Command: gauge run --install-plugins --verbose --log-level debug --env "default" --tags "about_page" --dir="." --fail-safe specs
gauge run specs
install-plugins : [true]
log-level : [debug]
tags : [about_page]
Plugin reportserver is already installed.
Parsing started.
Started concepts parsing.
172 specifications parsing completed.
Applying tags filter: about
The following specifications satisfy filter criteria:
specs/projects/web/test/about/about.spec
Parsing completed.
Checking updates...
Downloading https://downloads.gauge.org/plugin/flash?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
Downloading https://downloads.gauge.org/plugin/html-report?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
Downloading https://downloads.gauge.org/plugin/python?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
Downloading https://downloads.gauge.org/plugin/reportserver?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
Downloading https://downloads.gauge.org/plugin/screenshot?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
Downloading https://downloads.gauge.org/plugin/spectacle?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
Downloading https://downloads.gauge.org/plugin/xml-report?l=python&p=flash,html-report,python,reportserver,screenshot,spectacle,xml-report&o=darwin&a=amd64
You are using pip version 10.0.1, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Python: 3.7.0
Loading step implementations from /Users/testuser/Development/test/app/step_impl dirs.
Starting grpc server..
Attempting to connect to grpc server at port: 60022
Successfully made the connection with runner with port: 60022
Validation started.
Validation completed.
Successfully made the connection with plugin with port: 60043
Initialising suite data store.
Error occurred while waiting for runner process to finish.
Error : exit status 1
Error Message: transport is closing
Stacktrace:

Error Message: Runner is not Alive
Stacktrace:

Transformed SuiteResult to report structure
Successfully generated html-report to => /Users/testuser/Development/test/app/reports/html-report/index.html

Done generating HTML report using theme from /Users/testuser/.gauge/plugins/html-report/4.0.12/themes/default
Successfully generated xml-report to => /Users/testuser/Development/test/app/reports/xml-report

Origin report directory is ' /Users/testuser/Development/test/app/reports/html-report'
renaming index file to 'report.html'
Archive destination is ' /Users/testuser/Development/test/app/reports/html-report.zip'
removing archive '/Users/testuser/Development/test/app/reports/html-report.zip'
Origin report directory is ' /Users/testuser/Development/test/app/app/reports/html-report'
Archive destination is ' /Users/testuser/Development/test/app/reports/html-report.zip'
baseDir => 'app'
environment => 'default'
reportPath => ''
Could not find 'REPORTSERVER_HOST', setting to default 'http://localhost:8000'.
reportserverurl => 'http://localhost:8000/app/default'
Archive destination is ' /Users/testuser/Development/test/app/reports/html-report.zip'
Archive destination is '/Users/testuser/Development/test/app/reports/html-report.zip'
Could not send the archive from '/Users/testuser/Development/test/app/reports/html-report.zip' to 'http://localhost:8000/app/default'
Post http://localhost:8000/app/default: dial tcp [::1]:8000: connect: connection refused

Sending kill message to Flash plugin.
Sending kill message to Report Server plugin.
Sending kill message to Html Report plugin.
Sending kill message to Xml Report plugin.
Archive destination is ' /Users/testuser/Development/test/app/reports/html-report.zip'
removing archive ' /Users/testuser/Development/test/app/reports/html-report.zip'
Origin report directory is ' /Users/testuser/Development/test/app/reports/html-report'
Failed to kill Runner: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:60022: connect: connection refused"
renaming index file to 'index.html'
Specifications: 0 executed 0 passed 0 failed 0 skipped
Scenarios: 0 executed 0 passed 0 failed 0 skipped

Total time taken: 495ms

Terminating Automation Successfully
Terminated Stopwatch: [2021-03-10 | 21:33:06]
Overall Elapsed Time: "0:00:26.988521"

Waiting for logs to generate..

All processes completed

LOG OUTPUT:
10-03-2021 21:32:48.071 [python] [ERROR] You are using pip version 10.0.1, however version 21.0.1 is available.
10-03-2021 21:32:48.071 [python] [ERROR] You should consider upgrading via the 'pip install --upgrade pip' command.
10-03-2021 21:32:48.871 [python] [INFO] Python: 3.7.0
10-03-2021 21:32:48.871 [python] [DEBUG] Loading step implementations from /Users/testuser/Development/test/app/step_impl dirs.
10-03-2021 21:33:03.996 [python] [DEBUG] Starting grpc server..
10-03-2021 21:33:04.002 [Gauge] [DEBUG] Attempting to connect to grpc server at port: 60022
10-03-2021 21:33:04.004 [Gauge] [DEBUG] Successfully made the connection with runner with port: 60022
10-03-2021 21:33:04.004 [Gauge] [DEBUG] Validation started.
10-03-2021 21:33:04.008 [Gauge] [DEBUG] Validation completed.
10-03-2021 21:33:04.010 [Gauge] [DEBUG] Run started
10-03-2021 21:33:04.018 [Gauge] [DEBUG] Starting Html Report plugin
10-03-2021 21:33:04.112 [Gauge] [DEBUG] Attempting to connect to grpc server at port: 60027
10-03-2021 21:33:04.114 [Gauge] [DEBUG] Successfully made the connection with plugin with port: 60027
10-03-2021 21:33:04.115 [Gauge] [DEBUG] Starting Xml Report plugin
10-03-2021 21:33:04.190 [Gauge] [DEBUG] Attempting to connect to grpc server at port: 60032
10-03-2021 21:33:04.197 [Gauge] [DEBUG] Successfully made the connection with plugin with port: 60032
10-03-2021 21:33:04.205 [Gauge] [DEBUG] Starting Flash plugin
10-03-2021 21:33:04.349 [flash] [INFO] [Flash] Starting progress reporting at http://127.0.0.1:60037
10-03-2021 21:33:04.351 [Gauge] [DEBUG] Attempting to connect to grpc server at port: 60038
10-03-2021 21:33:04.354 [Gauge] [DEBUG] Successfully made the connection with plugin with port: 60038
10-03-2021 21:33:04.356 [Gauge] [DEBUG] Starting Report Server plugin
10-03-2021 21:33:04.428 [Gauge] [DEBUG] Attempting to connect to grpc server at port: 60043
10-03-2021 21:33:04.434 [Gauge] [DEBUG] Successfully made the connection with plugin with port: 60043
10-03-2021 21:33:04.434 [Gauge] [DEBUG] Initialising suite data store.
10-03-2021 21:33:04.479 [python] [CRITICAL] Exception occurred while loading step implementations from file: step_impl/global.py.
10-03-2021 21:33:04.496 [Gauge] [ERROR] Error occurred while waiting for runner process to finish.
Error : exit status 1
10-03-2021 21:33:04.506 [Gauge] [ERROR] Error Message: transport is closing
10-03-2021 21:33:04.506 [Gauge] [ERROR] Stacktrace:
10-03-2021 21:33:04.506 [Gauge] [ERROR] Error Message: Runner is not Alive
10-03-2021 21:33:04.506 [Gauge] [ERROR] Stacktrace:
10-03-2021 21:33:04.507 [html-report] [DEBUG] Transformed SuiteResult to report structure
10-03-2021 21:33:04.660 [html-report] [INFO] Successfully generated html-report to =>/Users/testuser/Development/test/app/reports/html-report/index.html
10-03-2021 21:33:04.660 [html-report] [DEBUG] Done generating HTML report using theme from /Users/armonds/.gauge/plugins/html-report/4.0.12/themes/default
10-03-2021 21:33:04.663 [xml-report] [INFO] Successfully generated xml-report to => /Users/testuser/Development/test/app/reports/xml-report
10-03-2021 21:33:05.670 [reportserver] [DEBUG] Origin report directory is '/Users/testuser/Development/test/app/reports/html-report'
10-03-2021 21:33:05.675 [reportserver] [DEBUG] renaming index file to 'report.html'
10-03-2021 21:33:05.675 [reportserver] [DEBUG] Archive destination is '/Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:05.675 [reportserver] [DEBUG] removing archive '/Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:05.675 [reportserver] [DEBUG] Origin report directory is '/Users/testuser/Development/test/app/reports/html-report'
10-03-2021 21:33:05.676 [reportserver] [DEBUG] Archive destination is '/Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:06.599 [reportserver] [DEBUG] baseDir => 'app'
10-03-2021 21:33:06.599 [reportserver] [DEBUG] environment => 'default'
10-03-2021 21:33:06.599 [reportserver] [DEBUG] reportPath => ''
10-03-2021 21:33:06.599 [reportserver] [DEBUG] Could not find 'REPORTSERVER_HOST', setting to default 'http://localhost:8000'.
10-03-2021 21:33:06.599 [reportserver] [DEBUG] reportserverurl => 'http://localhost:8000/app/default'
10-03-2021 21:33:06.599 [reportserver] [DEBUG] Archive destination is '/Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:06.615 [reportserver] [DEBUG] Archive destination is '/Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:06.615 [reportserver] [INFO] Could not send the archive from '/Users/testuser/Development/test/app/reports/html-report.zip' to 'http://localhost:8000/app/default'
Post http://localhost:8000/app/default: dial tcp [::1]:8000: connect: connection refused
10-03-2021 21:33:06.615 [Gauge] [DEBUG] Sending kill message to Flash plugin.
10-03-2021 21:33:06.615 [Gauge] [DEBUG] Sending kill message to Report Server plugin.
10-03-2021 21:33:06.615 [Gauge] [DEBUG] Sending kill message to Html Report plugin.
10-03-2021 21:33:06.615 [Gauge] [DEBUG] Sending kill message to Xml Report plugin.
10-03-2021 21:33:06.616 [reportserver] [DEBUG] Archive destination is '//Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:06.616 [reportserver] [DEBUG] removing archive '/Users/testuser/Development/test/app/reports/html-report.zip'
10-03-2021 21:33:06.637 [Gauge] [ERROR] Failed to kill Runner: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:60022: connect: connection refused"
10-03-2021 21:33:06.637 [reportserver] [DEBUG] Origin report directory is '/Users/testuser/Development/test/app/reports/html-report'
10-03-2021 21:33:06.637 [reportserver] [DEBUG] renaming index file to 'index.html'
10-03-2021 21:33:06.637 [Gauge] [INFO] Specifications: 0 executed 0 passed 0 failed 0 skipped
10-03-2021 21:33:06.637 [Gauge] [INFO] Scenarios: 0 executed 0 passed 0 failed 0 skipped
10-03-2021 21:33:06.637 [Gauge] [INFO]
Total time taken: 495ms

[sriv]

hi @asarkisian - I notice this line:

[CRITICAL] Exception occurred while loading step implementations from file: step_impl/global.py.

Which tells me that the gauge-python runner is unable to load this file due to some error, and this has caused gauge-python runner to crash. The error could be due to multiple possible reasons (ex either due to a missed requirement, or an unsupported python version).

There is an open issue that talks about improving the error reporting.

[asarkisian]

@sriv I have tried to use pyenv and downgraded my python ver to 3.8.0 and even lower than that all the way down to 3.7.0. It still doesn't work as expected. Is it possible I can send you a sample project so you can see on your end?

[asarkisian]

@sriv

I posted in a nother thread but here it is again:

Hi @sriv Absolutely. I removed all confidential content and only put in generic sample tests.

Repo: https://github.com/asarkisian/gauge_test_app

Here is some info on my system:

OS: Mac Catalina 10.15.7
Python3 Ver: 3.9.2
Python3 Path: /usr/bin/python3, /usr/local/bin/python3
Gauge version: 1.1.7
Plugins
html-report (4.0.12)
python (0.3.14)
screenshot (0.0.1)

OUTPUT:
`➜ test_app git:(master) gauge run specs
Python: 3.9.2
Error occurred while waiting for runner process to finish.
Error : exit status 1

Error Message: transport is closing
Stacktrace:

Error Message: Runner is not Alive
Stacktrace:

Successfully generated html-report to => /Users/armondsarkisian/Desktop/test_project/test_app/reports/html-report/index.html

Failed to kill Runner: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:51812: connect: connection refused"`

[sriv]

@asarkisian - thanks for the sample, it helped troubleshoot.

There are two aspects to this problem:

1. gauge-python does not log the stacktrace indicating the real cause of the failure, and hence you see a generic and unhelpful error message. This is reported in Exception while loading step implementations from file not logged gauge-python#228. I've raised a PR (log error and stacktrace when impl file load fails gauge-python#230) to fix this.

2. when I use the changes included above and run your sample, here's what I see in the gauge.log:

18-03-2021 07:54:10.937 [python] [CRITICAL] Exception occurred while loading step implementations from file: step_impl/global.py.
Traceback (most recent call last):
  File "/home/steam/.local/lib/python3.8/site-packages/getgauge/impl_loader.py", line 65, in _import_file
    m = importlib.import_module(module_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/gauge_test_app/step_impl/global.py", line 25, in <module>
    from framework import Framework
ImportError: cannot import name 'Framework' from 'framework' (/tmp/gauge_test_app/framework/__init__.py)
18-03-2021 07:54:10.939 [Gauge] [ERROR] Error occurred while waiting for runner process to finish.
Error : exit status 1

It seems that the project is unable to load because of an error when importing framework. I am not familiar with your setup, but it looks like some dependencies are missing from requirements.txt and hence not available to your framework. ex:

• reporting.py has import vlc, but python-vlc is missing in requirements.txt
• requests.py has from requests_oauthlib import OAuth1 but requests-oauthlib is missing in requirements.txt

There could be more, and I am not sure if these dependencies are somehow made available automatically in your environments.

Perhaps, once gauge-python 0.3.15 is out, you can run it and see the error in your system (alternatively, you could install gauge-python from source from https://github.com/getgauge/gauge-python/tree/228_log_impl_error).

[asarkisian]

Thanks for the update @sriv . I will wait for gauge-python 0.3.15 to be out. In the meantime, I will keep searching for the cause. It's especially difficult when the stack trace is blank

[sriv]

gauge-python 0.3.15 is released - https://github.com/getgauge/gauge-python/releases/tag/v0.3.15

It will take a few hours for the cache to get refreshed and then you can do a gauge install python to get the latest version. In the meanwhile you can download the zip file from the releases page and run gauge install python -f <path_to_zip>