Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Performance issue linked to jinja rendrer #64699

Closed
1 of 9 tasks
doc75 opened this issue Jul 22, 2023 · 1 comment
Closed
1 of 9 tasks

[BUG] Performance issue linked to jinja rendrer #64699

doc75 opened this issue Jul 22, 2023 · 1 comment
Labels
Bug broken, incorrect, or confusing behavior needs-triage

Comments

@doc75
Copy link

doc75 commented Jul 22, 2023

Description
Salt recently to end with timeout, without any change on my side (except certainly a salt version upgrade).
I guess that it might be the latest version upgrade, but I cannot be sure as I do not know how to get back to the previous version (if you indicate how I can do this with an apt installation on Ubuntu, I'll be happy to test)
After debugging a bit, it appears that the jinja renderer is now taking more than 60 second for each jinja file....

Setup
I installed salt from your python3 repo using apt. No specific configuration done execept the name of the master and also enable_fqdns_grains: False on the master in the file /etc/salt/minion.d/fqdns.conf file.

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service => VPS from Hetzner
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

Steps to Reproduce the behavior
Option #1 (On the master): salt machine-name state.highstate --state-output=changes
Option #2 (On a minion): salt-call state.apply

Extract of output of salt-call state.apply

[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/common/update.sls' using 'yaml' renderer: 0.0011582374572753906
[DEBUG   ] In saltenv 'base', looking at rel_path 'common/firewall.sls' to resolve 'salt://common/firewall.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/common/firewall.sls' to resolve 'salt://common/firewall.sls'
[DEBUG   ] compile template: /var/cache/salt/minion/files/base/common/firewall.sls
[DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
[DEBUG   ] In saltenv 'base', looking at rel_path 'common/firewall_macro.sls' to resolve 'salt://common/firewall_macro.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/common/firewall_macro.sls' to resolve 'salt://common/firewall_macro.sls'
[DEBUG   ] In saltenv 'base', looking at rel_path 'common/settings.sls' to resolve 'salt://common/settings.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/common/settings.sls' to resolve 'salt://common/settings.sls'
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/common/firewall.sls' using 'jinja' renderer: 60.518840074539185
[DEBUG   ] Rendered data from file: /var/cache/salt/minion/files/base/common/firewall.sls:

On the second PROFILE line, you can see that the jinja renderer took > 60 seconds.
This is the case for all .sls file (not specific to one or a subset of the .sls files).

Expected behavior
Get back to previous performances (between 25 and 60 seconds for a minion to apply its state).

Screenshots
Not applicable

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.7.3
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.6.1
        pygit2: Not Installed
        Python: 3.8.10 (default, May 26 2023, 14:05:08)
  python-gnupg: 0.4.5
        PyYAML: 5.3.1
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.2
 
System Versions:
          dist: ubuntu 20.04 focal
        locale: utf-8
       machine: x86_64
       release: 5.4.0-152-generic
        system: Linux
       version: Ubuntu 20.04 focal

The only difference on the minion is the release in System version: release: 5.13.0-52-generic

Additional context
Add any other context about the problem here.

@doc75 doc75 added Bug broken, incorrect, or confusing behavior needs-triage labels Jul 22, 2023
@doc75 doc75 changed the title [BUG] [BUG] Performance issue linked to jinja rendrer Jul 22, 2023
@doc75
Copy link
Author

doc75 commented Jul 23, 2023

After reinstalling master and minion, I do not have anymore the problem.
Might be due to some caching or similar topic.

@doc75 doc75 closed this as completed Jul 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior needs-triage
Projects
None yet
Development

No branches or pull requests

1 participant