GInga takes ages to load

[olebole]

(Python 3) When I start ginga, it shows a blank screen for ~30 seconds, before it is actually ready to go.

2018-03-15 10:25:15,976 | I | main.py:364 (main) | Chosen toolkit (qt5) family is 'qt'
2018-03-15 10:25:16,949 | D | Task.py:1012 (startall) | startall called
2018-03-15 10:25:16,949 | D | Task.py:1037 (startall) | starting threads in thread pool
2018-03-15 10:25:16,950 | D | Task.py:926 (taskloop) | Starting worker thread loop.
[...]
2018-03-15 10:25:52,975 | D | ImageView.py:1522 (get_rgb_object) | times: total=0.0047
2018-03-15 10:25:52,975 | D | ImageViewQt.py:172 (render_image) | redraw pixmap=<PyQt5.QtGui.QPixmap object 
2018-03-15 10:25:52,975 | D | ImageViewQt.py:175 (render_image) | drawing to pixmap
2018-03-15 10:25:52,975 | D | ImageViewQt.py:149 (_render_offscreen) | data shape is 664x664x4
2018-03-15 10:25:52,986 | D | ImageViewQt.py:252 (update_image) | updating window from pixmap
2018-03-15 10:25:52,986 | D | ImageView.py:1265 (redraw_now) | widget 'colorbar' redraw (whence=0) delta=0.0

The full log is attached.
ginga.log

[pllim]

How did you start it exactly? And which versions of Ginga and Astropy do you have? Are you doing something custom in your ginga_config.py on start-up? What data are you loading? I see some get_rgb_object calls and mention of a 664x664x4 data cube.

[olebole]

I started it just with

$ ginga --log-level=9 --log=ginga.log

(also happens without any command line arguments)

Versions:

• GInga 2.7.0
• astropy 3.0
• Python 3.6.4

No customization; not loading anything. There was a ~/.ginga when I recorded the log file, but the delay happens also when I delete it.

[pllim]

Using the following versions, mine started up in 3 seconds. Log:
ginga.log

• RHEL7 64-bit
• Python 3.6.3
• Numpy 1.14.1
• Scipy 1.0.0
• Astropy 3.1.dev21573
• Ginga 2.7.1.dev1877

And here my Qt info:

# Name                    Version                   Build  Channel
pyqt                      5.6.0            py36h0386399_5  
qt                        5.6.2                         7    conda-forge
qtconsole                 4.3.1            py36h8f73b5b_0  
qtpy                      1.3.1            py36h3691cc8_0  

p.s. That datacube-like dimension log entry apparently appears even when I don't start it with any datacube. Sorry for the noise.

[olebole]

I guess the delay comes from the modules loading: this happens serially, and each module takes about 2 seconds to load. The CPU is not used here, so each module seems to have a builtin wait somewhere.

[ejeschke]

There are no explicit waits in the modules. Is it possible for you to test in a different environment (e.g. miniconda)? For me startup is less than 3 sec with python 3 on Ubuntu 16.04 (native libs or miniconda) and similar on Mac OS X 10.11.6 with miniconda.

Is it possible that it is recompiling python 3 source each time? Even so, it should not be 30 sec.

[olebole]

Further investigation shows, that during startup, ginga tries to connect to some hosts:

• 130.167.180.4 (Port 80 and 443)
• 162.255.119.162 (Port 80)
• 151.101.113.147 (Port 80)

The first is an STSCI host (ssbwebv1.stsci.edu); but the other look weird to me. When I disable networking, ginga starts quickly, so the connection delay seems to be the cause.
Since calling home for an application destroys user privacy: Can you tell me what the purpose of the connections is and how they can be disabled?

[pllim]

Huh, Ginga should not be trying to connect to anything at STScI. @jhunkeler or @rendinam , do you know what ssbwebv1 is serving?

[rendinam]

I'm not able to detect any network activity from ginga during startup in a packet capture.

• Debian 9.2
• ginga 2.7.0
• astropy 3.0.1

As a sanity check, I am able to see network activity originating from ginga when I select "Help --> Documentation", as this loads a web page. Could the network activity you're seeing be originating from some other process?

I also noticed a lag of ~10s or so in startup on the first run of ginga. No network activity detected, but on subsequent starts ginga came up much faster (< 2s).

[olebole]

It is the ginga Python process:

$ netstat -ntp | grep python
tcp        0    517 141.33.60.119:37170     130.167.180.4:443       VERBUNDEN   24212/python3       
$ ps aux | grep 24212
ole     24212 16.4  1.2 3385052 201944 pts/1  Sl+  14:43   0:03 /usr/bin/python3 /usr/bin/ginga
ole     24274  0.0  0.0  12904  1076 pts/5    S+   14:43   0:00 grep -F --color=auto 24212

[rendinam]

Thanks for the output. The first version I ran was ginga as distributed as a conda package from the Astroconda channel. Then I noticed that you are running the system version, so I installed that. Sure enough, the ginga that ships as a Debian package creates network connections upon startup. Investigating further...

[rendinam]

In the network traffic produced during startup of the Debian version of ginga, some HTTP requests are made to a host data.astropy.org which resolves to an institute machine.
GET //usr/lib/python3/dist-packages/ginga/examples/configs/plugin_Operations.cfg HTTP/1.1
GET //usr/lib/python3/dist-packages/ginga/examples/configs/plugin_Toolbar.cfg HTTP/1.1
GET //usr/lib/python3/dist-packages/ginga/examples/configs/plugin_Pan.cfg HTTP/1.1
GET //usr/lib/python3/dist-packages/ginga/examples/configs/plugin_Thumbs.cfg HTTP/1.1
etc...

The conda environment for ginga 2.7.0 contains python3.5/site-packages/ginga/examples/configs/plugin_Operations.cfg and the other config files. In the Debian installation however, /usr/lib/python3/dist-packages/ginga exists, but the /examples subdirectory does not. It would appear that when ginga attempts to load these config files, it cannot find them on Debian and then somehow crafts a URL from the path information and attempts to get them via HTTP.

[pllim]

when ginga attempts to load these config files, it cannot find them on Debian and then somehow crafts a URL from the path information and attempts to get them via HTTP

Huh. AFAIK examples are just that, examples. If Ginga plugin does not detect a plugin_PluginName.cfg , it uses software default. Maybe @ejeschke can clarify.

[pllim]

Okay, I think I have a suspect! Plugin docstring (e.g., https://github.com/ejeschke/ginga/blob/master/ginga/rv/plugins/Mosaic.py#L636) tries to pull in those examples using https://github.com/ejeschke/ginga/blob/master/ginga/util/toolbox.py#L135 for documentation.

Would adding examples to MANIFEST solve this problem? It is weird that only Debian saw this but not astroconda; does the latter do a git clone and disregard the manifest?

[rendinam]

Copying the contents of ginga's examples directory into /usr/lib/python3/dist-packages/ginga/examples/ eliminates the network activity upon startup for the Debian version.

If the .deb build can be persuaded to include the package_data directories specified in setup.py with something like include_package_data=True, that might eliminate the symptoms. The fact that HTTP calls end up being made for local files when they happen to not be present is the core issue.

[ejeschke]

Great work everyone!

If the .deb build can be persuaded to include the package_data directories specified in setup.py with something like include_package_data=True, that might eliminate the symptoms.

@olebole, I just checked by doing a fresh install and the config files are installed to the package area. Can we do as @rendinam suggests above? That would seem to be the straightforward solution.

[ejeschke]

The fact that HTTP calls end up being made for local files when they happen to not be present is the core issue.

It would appear that when ginga attempts to load these config files, it cannot find them on Debian and then somehow crafts a URL from the path information and attempts to get them via HTTP.

This should probably also be addressed. @pllim, I assume we are using an astropy module helper function to retrieve these files. Is there a way to disable the automatic attempt to download the config files via http? Maybe we could just use regular python file system access to retrieve these, and put up an error message if they don't exist.

[pllim]

I assume we are using an astropy module helper function to retrieve these files

That is correct. get_pkg_data_filename does attempt to connect to data.astropy.org when it cannot find the files locally. I'll make a PR. My bad, sorry!

[ejeschke]

No, worries, @pllim.

@olebole, it is nice to have the examples installed somewhere when the package is installed; I had at least one user complain that they weren't installed, which is why I originally added them to the install.

[pllim]

When #645 is merged, you should not run into this issue anymore. However, if you try to generate doc manually, plugin docstring will be incomplete (minor inconvenience).

Also FWIW, the tarball distributed by PyPI does already include examples folder and all its contents.

[ejeschke]

However, if you try to generate doc manually, plugin docstring will be incomplete (minor inconvenience).

From the installed package? That's probably ok. I think we eventually want to install the documentation at install time, if possible.

[ejeschke]

Oops, didn't mean to close this yet.

[ejeschke]

Merged #645, so this should take care of the main issue reported.

[olebole]

In Debian, the examples are in /usr/share/doc/python3-ginga/examples. So, I can probably just change the default in generate_cfg_example(), right?

[ejeschke]

@pllim, I think you maybe should comment on this one because I didn't write that function. @olebole, you mean for a Debian-specific fork or branch?

[olebole]

Debian packages have usually a few minor changes; for GInga they are kept here. I update them with a new version, so from your side nothing is needed. As you can see, the changes are rather trivial and Debian specific.
Since the change of the default example path is package dependent (usr/share/doc/<<package>>/examples), I need different defaults here for the Python 2 and Python 3 packages. I solved this pragmatically with sed. So, I am happy with everything as it is, but open for improvements. 👍

[ejeschke]

@olebole, interesting link. Maybe @pllim can comment on whether generate_cfg_example() is the best place to patch that.

BTW, browsing that link, I'm led to a question that you might be able to answer. Is there a good generic function for locating scalable fonts in Linux (or at least Debian-based distros)?

[olebole]

@ejeschke Sorry, I don't know. I just hardcoded the same way as you did. System wide fonts are in /usr/share/fonts/. There is a Fonts wiki page on Debian; maybe it helps you more.

[pllim]

@olebole , what is the motivation for having the config files in usr/share/doc instead of using the ones that come with Ginga installation? They seem to be included by default when I check PyPI tarball.

Anyway, I don't see problem for you to sed it in a way that works for Debian. Even if you don't patch it, when generate_cfg_example() cannot find the file, it simply throws a warning and returns empty string, so you can simply comment out warnings.warn() in that function as well. The resultant incomplete docstring only affects offline help pages, which are never displayed unless you click "Help" when not connected to Internet.

[olebole]

These are not the config files, but the examples. In Debian, examples belong to the package documentation and therefore should be located in /usr/share/doc/<<package>>/examples.