Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

forumla's map.jinja not found in jinja-template-files on salt-ssh #31531

Open
sgeisbacher opened this issue Feb 28, 2016 · 32 comments
Open

forumla's map.jinja not found in jinja-template-files on salt-ssh #31531

sgeisbacher opened this issue Feb 28, 2016 · 32 comments
Labels
Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt Salt-SSH severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone

Comments

@sgeisbacher
Copy link

Expected Behavior

the forumla's file-generation by jinja-template should load map.jinja also on salt-ssh.
this formula works with salt minion state.highstate like a charm, but not with salt-ssh minion state.highstate

Actual Behavior

when performing a state.highstate via salt-ssh i get the following error-msg: (see bottom)

Steps to Reproduce Issue

  1. create a simple formula with map.jinja
  2. add file generation by jinja (file.managed: - source: salt://.../file.jinja - template: jinja) to this formula
  3. import the map.jinja to this file.jinja-template with {% from ... import ... %}
  4. try to provision this formula to a minion with salt-ssh --> you will get the error from above

Versions Report

Salt Version:
           Salt: 2015.8.7

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: Not Installed
           Mako: 0.9.1
         PyYAML: 3.10
          PyZMQ: 14.0.1
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: 0.8.2
        timelib: Not Installed

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.13.0-71-generic
         system: Ubuntu 14.04 trusty

Relevant Logs, Configs, or States

Error-Message: `Unable to manage file: Jinja error: jinjatest/map.jinja
              Traceback (most recent call last):
                File "/tmp/.root_09032a_salt/salt/utils/templates.py", line 366, in render_jinja_tmpl
                  output = template.render(**decoded_context)
                File "/tmp/.root_09032a_salt/jinja2/environment.py", line 969, in render
                  return self.environment.handle_exception(exc_info, True)
                File "/tmp/.root_09032a_salt/jinja2/environment.py", line 742, in handle_exception
                  reraise(exc_type, exc_value, tb)
                File "<template>", line 1, in top-level template code
                File "/tmp/.root_09032a_salt/salt/utils/jinja.py", line 144, in get_source
                  raise TemplateNotFound(template)
              TemplateNotFound: jinjatest/map.jinja

              ; line 1

              ---
              {% from "jinjatest/map.jinja" import jinjatest with context %}    <======================

              hello {{ jinjatest.name }}
              you are {{ jinjatest.age }} old
              Traceback (most recent call last):
                File "/tmp/.root_09032a_salt/salt/utils/templates.py", line 366, in render_jinja_tmpl
              [...]`

here is the simple-test-formula:
salt-jinjatest.zip

@jfindlay jfindlay added Bug broken, incorrect, or confusing behavior severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around Core relates to code central or existential to Salt Salt-SSH P3 Priority 3 labels Feb 29, 2016
@jfindlay jfindlay added this to the Approved milestone Feb 29, 2016
@jfindlay
Copy link
Contributor

@sgeisbacher, thanks for reporting.

@JubbaSmail
Copy link

@jfindlay can I work on this issue ?

@edgan
Copy link
Contributor

edgan commented Aug 10, 2016

The know workaround is using extra_filerefs. The salt-ssh code doesn't pick up files that aren't referenced by include or source. The import doesn't count. I have tried a system of having a state to source the map.jinja in cases of salt-ssh, but it proved to be more work than using the extra_filerefs method. See #9878 for history.

Saltfile:
extra_filerefs:
  - salt://jingatest/map.jinja

@rallytime
Copy link
Contributor

@Ismail-AlJubbah Certainly!

@awerner
Copy link
Contributor

awerner commented Nov 11, 2016

Another possible workaroung is to use the import statement in the init.sls and then pass the jinjatest variable as context to the file.managed state. This is also more clean IMO.
@Ismail-AlJubbah any progress?

@stale
Copy link

stale bot commented Jul 24, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Jul 24, 2018
@publicarray
Copy link

Any update?

@stale
Copy link

stale bot commented Jul 24, 2018

Thank you for updating this issue. It is no longer marked as stale.

@stale stale bot removed the stale label Jul 24, 2018
@awerner
Copy link
Contributor

awerner commented Sep 20, 2018

If using the context does not work for you, I have another workaround.

@stale
Copy link

stale bot commented Jan 8, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Jan 8, 2020
@max-arnold
Copy link
Contributor

Bump

@stale
Copy link

stale bot commented Jan 8, 2020

Thank you for updating this issue. It is no longer marked as stale.

@max-arnold
Copy link
Contributor

max-arnold commented Apr 17, 2020

I made an ugly patch that seemingly fixes this issue by packing the whole salt:// directory into the salt-ssh state tarball:

diff --git a/salt/fileclient.py b/salt/fileclient.py
index d606bea99e..64ae1b0a63 100644
--- a/salt/fileclient.py
+++ b/salt/fileclient.py
@@ -238,7 +238,7 @@ class Client(object):
         # the target directory and caching them
         for fn_ in self.file_list(saltenv):
             fn_ = salt.utils.data.decode(fn_)
-            if fn_.strip() and fn_.startswith(path):
+            if fn_.strip() and (fn_.startswith(path) or path == '/'):
                 if salt.utils.stringutils.check_include_exclude(
                     fn_, include_pat, exclude_pat
                 ):

To activate it add --extra-filerefs 'salt://' command line option to salt-ssh. Also it is possible to do the same through /etc/salt/master config snippet:

ssh_minion_opts:
  extra_filerefs: salt://

@baby-gnu
Copy link

Interestingly, I have it working for salt-ssh 3002.7 but not for 3003.3 as described in my comment saltstack-formulas/openvpn-formula#147 (comment).

I do not use extra_filerefs in both setup.

Regards.

@baby-gnu
Copy link

I did more tests with 6 machines with different versions of salt-ssh from 3002.7 to 3004.

Each machine has the following setup:

/root/.salt/Saltfile
# -*- mode: yaml; coding: utf-8 -*-

salt-ssh:
  config_dir: ~/.salt
/root/.salt/master
# -*- mode: yaml; coding: utf-8 -*-

####
#### Global parameters
####

# Unfortunately, salt does not support ~ expension
# We need to use absolute path for `root_dir`
root_dir: /root/.salt
pki_dir: pki
cachedir: cache
sock_dir: run
pidfile: pids

log_file: logs/master.log
key_logfile: logs/key.log

# Global level logged in file
log_level_logfile: debug

##
## file server
##
# I'm using formula repositories but override things from time to time
fileserver_backend:
  - roots

# Unfortunately, salt does not support ~ expension
# We need to use absolute path for `file_roots` and `pillar_roots`
file_roots:
  base:
    - /root/.salt/srv/salt/
    - /root/.salt/srv/formulas/*

pillar_roots:
  base:
    - /root/.salt/srv/pillar/
/root/.salt/roster
# -*- mode: yaml; coding: utf-8 -*-

testmachine6:
  host: 192.168.0.111
  user: root
/root/.salt/srv/salt/test-import/init.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
include:
  - .absolute
  - .relative
/root/.salt/srv/salt/test-import/absolute.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from "test-import/map.jinja" import mapdata %}

test-import/absolute/test.nop:
  test.nop:
    - name: {{ mapdata | json }}
/root/.salt/srv/salt/test-import/relative.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from "./map.jinja" import mapdata %}

test-import/relative/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

Then, I run the state.show_sls from all machines, I include a --versions-report to make things clear.

This works fine on 3002.7 and fails on all versions starting with 3003.

salt-ssh 'testmachine*' cmd.run 'salt-ssh --versions-report; salt-ssh -l critical testmachine6 state.show_sls test-import'
testmachine0:
    Salt Version:
              Salt: 3002.7
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.0
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: 3.9.7
            pygit2: Not Installed
            Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
      python-gnupg: Not Installed
            PyYAML: 5.3.1
             PyZMQ: 20.0.0
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.4
     
    System Versions:
              dist: debian 11 bullseye
            locale: utf-8
           machine: x86_64
           release: 5.10.0-8-amd64
            system: Linux
           version: Debian GNU/Linux 11 bullseye
     
    testmachine6:
        ----------
        test-import/absolute/test.nop:
            ----------
            __env__:
                base
            __sls__:
                test-import.absolute
            test:
                |_
                  ----------
                  name:
                      ----------
                      test:
                          foo
                - nop
                |_
                  ----------
                  order:
                      10000
        test-import/relative/test.nop:
            ----------
            __env__:
                base
            __sls__:
                test-import.relative
            test:
                |_
                  ----------
                  name:
                      ----------
                      test:
                          foo
                - nop
                |_
                  ----------
                  order:
                      10001
testmachine1:
    Salt Version:
              Salt: 3003
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.0
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: 3.9.7
            pygit2: Not Installed
            Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
      python-gnupg: Not Installed
            PyYAML: 5.3.1
             PyZMQ: 20.0.0
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.4
     
    System Versions:
              dist: debian 11 bullseye
            locale: utf-8
           machine: x86_64
           release: 5.10.0-8-amd64
            system: Linux
           version: Debian GNU/Linux 11 bullseye
     
    [CRITICAL] Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 497, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import/map.jinja" import mapdata %}    <======================
    
    test-import/absolute/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 497, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import/relative/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    testmachine6:
        - Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 497, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: test-import/map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "test-import/map.jinja" import mapdata %}    <======================
          
          test-import/absolute/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
        - Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 497, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: ./map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "./map.jinja" import mapdata %}    <======================
          
          test-import/relative/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
testmachine2:
    Salt Version:
              Salt: 3003.1
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.0
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: 3.9.7
            pygit2: Not Installed
            Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
      python-gnupg: Not Installed
            PyYAML: 5.3.1
             PyZMQ: 20.0.0
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.4
     
    System Versions:
              dist: debian 11 bullseye
            locale: utf-8
           machine: x86_64
           release: 5.10.0-8-amd64
            system: Linux
           version: Debian GNU/Linux 11 bullseye
     
    [CRITICAL] Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import/map.jinja" import mapdata %}    <======================
    
    test-import/absolute/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import/relative/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    testmachine6:
        - Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: test-import/map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "test-import/map.jinja" import mapdata %}    <======================
          
          test-import/absolute/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
        - Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: ./map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "./map.jinja" import mapdata %}    <======================
          
          test-import/relative/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
testmachine3:
    Salt Version:
              Salt: 3003.2
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.0
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: 3.9.7
            pygit2: Not Installed
            Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
      python-gnupg: Not Installed
            PyYAML: 5.3.1
             PyZMQ: 20.0.0
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.4
     
    System Versions:
              dist: debian 11 bullseye
            locale: utf-8
           machine: x86_64
           release: 5.10.0-8-amd64
            system: Linux
           version: Debian GNU/Linux 11 bullseye
     
    [CRITICAL] Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import/map.jinja" import mapdata %}    <======================
    
    test-import/absolute/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import/relative/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    testmachine6:
        - Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: test-import/map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "test-import/map.jinja" import mapdata %}    <======================
          
          test-import/absolute/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
        - Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: ./map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "./map.jinja" import mapdata %}    <======================
          
          test-import/relative/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
testmachine4:
    Salt Version:
              Salt: 3003.3
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.0
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: 3.9.7
            pygit2: Not Installed
            Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
      python-gnupg: Not Installed
            PyYAML: 5.3.1
             PyZMQ: 20.0.0
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.4
     
    System Versions:
              dist: debian 11 bullseye
            locale: utf-8
           machine: x86_64
           release: 5.10.0-8-amd64
            system: Linux
           version: Debian GNU/Linux 11 bullseye
     
    [CRITICAL] Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import/map.jinja" import mapdata %}    <======================
    
    test-import/absolute/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import/relative/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    testmachine6:
        - Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: test-import/map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "test-import/map.jinja" import mapdata %}    <======================
          
          test-import/absolute/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
        - Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 500, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: ./map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "./map.jinja" import mapdata %}    <======================
          
          test-import/relative/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
testmachine5:
    Salt Version:
              Salt: 3004
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.8.1
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.0
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: 3.9.7
            pygit2: Not Installed
            Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
      python-gnupg: Not Installed
            PyYAML: 5.3.1
             PyZMQ: 20.0.0
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.4
     
    System Versions:
              dist: debian 11 bullseye
            locale: utf-8
           machine: x86_64
           release: 5.10.0-8-amd64
            system: Linux
           version: Debian GNU/Linux 11 bullseye
     
    [CRITICAL] Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import/map.jinja" import mapdata %}    <======================
    
    test-import/absolute/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
        reraise(*rewrite_traceback_stack(source=source))
      File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
        raise value.with_traceback(tb)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import/relative/test.nop:
      test.nop:
        - name: {{ mapdata | json }}
    
    [...]
    ---
    testmachine6:
        - Rendering SLS 'base:test-import.absolute' failed: Jinja error: test-import/map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: test-import/map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "test-import/map.jinja" import mapdata %}    <======================
          
          test-import/absolute/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---
        - Rendering SLS 'base:test-import.relative' failed: Jinja error: ./map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
              reraise(*rewrite_traceback_stack(source=source))
            File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
              raise value.with_traceback(tb)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: ./map.jinja
          
          ; line 3
          
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "./map.jinja" import mapdata %}    <======================
          
          test-import/relative/test.nop:
            test.nop:
              - name: {{ mapdata | json }}
          
          [...]
          ---

@baby-gnu
Copy link

baby-gnu commented Oct 27, 2021

I ran a test with 3004 and the new slsutil.findup

/root/.salt/srv/salt/test-import/findup.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}

test-import/findup/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

but it does not work

salt-ssh testmachine6 slsutil.findup test-import map.jinja
[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
    return self[key]
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
    rv = self._mapping[key]
TypeError: unhashable type: 'dict'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 261, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 547, in render_jinja_tmpl
    raise SaltRenderError(
salt.exceptions.SaltRenderError: Jinja error: unhashable type: 'dict'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
    return self[key]
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
    rv = self._mapping[key]
TypeError: unhashable type: 'dict'

; line 3

---
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}    <======================

test-import/findup/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

[...]
---
[CRITICAL] Rendering SLS 'base:test-import.findup' failed: Jinja error: unhashable type: 'dict'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
    return self[key]
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
    rv = self._mapping[key]
TypeError: unhashable type: 'dict'

; line 3

---
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}    <======================

test-import/findup/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

[...]
---
testmachine6:
    - Rendering SLS 'base:test-import.findup' failed: Jinja error: unhashable type: 'dict'
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
          output = template.render(**decoded_context)
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
          self.environment.handle_exception()
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
          reraise(*rewrite_traceback_stack(source=source))
        File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
          raise value.with_traceback(tb)
        File "<template>", line 3, in top-level template code
        File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
          return self[key]
        File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
          rv = self._mapping[key]
      TypeError: unhashable type: 'dict'
      
      ; line 3
      
      ---
      # -*- mode: salt; coding: utf-8 -*-
      # vim: ft=sls
      {% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}    <======================
      
      test-import/findup/test.nop:
        test.nop:
          - name: {{ mapdata | json }}
      
      [...]
      ---

A direct run of slsutil.findup does not work

salt-ssh testmachine6 slsutil.findup test-import map.jinja
testmachine6:
    ----------
    retcode:
        1
    stderr:
        Passed invalid arguments: Starting path not found in the state tree: test-import.
        
        Usage:
    stdout:
        Find the first path matching a filename or list of filenames in a specified
            directory or the nearest ancestor directory. Returns the full path to the
            first file found.
        
            .. versionadded:: 3004
        
            :param str startpath: The fileserver path from which to begin the search.
                An empty string refers to the state tree root.
            :param filenames: A filename or list of filenames to search for. Searching for
                directory names is also supported.
            :param str saltenv: The fileserver environment to search. Default: ``base``
        
            Example: return the path to ``defaults.yaml``, walking up the tree from the
            state file currently being processed.
        
            .. code-block:: jinja
        
                {{ salt["slsutil.findup"](tplfile, "defaults.yaml") }}
        
            CLI Example:
        
            .. code-block:: bash
        
                salt '*' slsutil.findup formulas/shared/nginx map.jinja

@baby-gnu
Copy link

Retrying my tests, I have a strange behavior:

  1. it does not find map.jinja
    $ salt-ssh testmachine2 state.apply test-import-rootfs
    [ERROR   ] Rendering exception occurred
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import-rootfs/map.jinja
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 261, in render_tmpl
        output = render_str(tmplstr, context, tmplpath)
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 547, in render_jinja_tmpl
        raise SaltRenderError(
    salt.exceptions.SaltRenderError: Jinja error: test-import-rootfs/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import-rootfs/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import-rootfs/map.jinja" import mapdata %}    <======================
    
    test-import-rootfs/absolute/test.show_notification:
      test.show_notification:
        - text: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import-rootfs.absolute' failed: Jinja error: test-import-rootfs/map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: test-import-rootfs/map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "test-import-rootfs/map.jinja" import mapdata %}    <======================
    
    test-import-rootfs/absolute/test.show_notification:
      test.show_notification:
        - text: {{ mapdata | json }}
    
    [...]
    ---
    [ERROR   ] Rendering exception occurred
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 261, in render_tmpl
        output = render_str(tmplstr, context, tmplpath)
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 547, in render_jinja_tmpl
        raise SaltRenderError(
    salt.exceptions.SaltRenderError: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import-rootfs/relative/test.show_notification:
      test.show_notification:
        - text: {{ mapdata | json }}
    
    [...]
    ---
    [CRITICAL] Rendering SLS 'base:test-import-rootfs.relative' failed: Jinja error: ./map.jinja
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
        output = template.render(**decoded_context)
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
        self.environment.handle_exception()
      File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "<template>", line 3, in top-level template code
      File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
        raise TemplateNotFound(template)
    jinja2.exceptions.TemplateNotFound: ./map.jinja
    
    ; line 3
    
    ---
    # -*- mode: salt; coding: utf-8 -*-
    # vim: ft=sls
    {% from "./map.jinja" import mapdata %}    <======================
    
    test-import-rootfs/relative/test.show_notification:
      test.show_notification:
        - text: {{ mapdata | json }}
    
    [...]
    ---
    testmachine2:
        - Rendering SLS 'base:test-import-rootfs.absolute' failed: Jinja error: test-import-rootfs/map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
              raise rewrite_traceback_stack(source=source)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: test-import-rootfs/map.jinja
    
          ; line 3
    
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "test-import-rootfs/map.jinja" import mapdata %}    <======================
    
          test-import-rootfs/absolute/test.show_notification:
            test.show_notification:
              - text: {{ mapdata | json }}
    
          [...]
          ---
        - Rendering SLS 'base:test-import-rootfs.relative' failed: Jinja error: ./map.jinja
          Traceback (most recent call last):
            File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1291, in render
              self.environment.handle_exception()
            File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
              raise rewrite_traceback_stack(source=source)
            File "<template>", line 3, in top-level template code
            File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
              raise TemplateNotFound(template)
          jinja2.exceptions.TemplateNotFound: ./map.jinja
    
          ; line 3
    
          ---
          # -*- mode: salt; coding: utf-8 -*-
          # vim: ft=sls
          {% from "./map.jinja" import mapdata %}    <======================
    
          test-import-rootfs/relative/test.show_notification:
            test.show_notification:
              - text: {{ mapdata | json }}
    
          [...]
          ---
    
  2. Trying a sub-state is working
    $ salt-ssh testmachine2 state.apply test-import-rootfs.relative
    testmachine2:
    ----------
             ID: test-import-rootfs/relative/test.show_notification
       Function: test.show_notification
         Result: True
        Comment: {'test-import-rootfs': 'foo'}
        Started: 11:56:20.440018
       Duration: 0.714 ms
        Changes:
    
    Summary for testmachine2
    ------------
    Succeeded: 1
    Failed:    0
    ------------
    Total states run:     1
    Total run time:   0.714 ms
    
  3. Retrying the meta-state is now working
    salt-ssh testmachine2 state.apply test-import-rootfs
    testmachine2:
    ----------
             ID: test-import-rootfs/absolute/test.show_notification
       Function: test.show_notification
         Result: True
        Comment: {'test-import-rootfs': 'foo'}
        Started: 11:57:10.487955
       Duration: 0.693 ms
        Changes:
    ----------
             ID: test-import-rootfs/relative/test.show_notification
       Function: test.show_notification
         Result: True
        Comment: {'test-import-rootfs': 'foo'}
        Started: 11:57:10.488765
       Duration: 0.548 ms
        Changes:
    
    Summary for testmachine2
    ------------
    Succeeded: 2
    Failed:    0
    ------------
    Total states run:     2
    Total run time:   1.241 ms
    
/home/me/.salt/Saltfile
# -*- mode: yaml; coding: utf-8 -*-

salt-ssh:
  config_dir: ~/.salt
/home/me/.salt/master
# -*- mode: yaml; coding: utf-8 -*-

####
#### Global parameters
####

# Unfortunately, salt does not support ~ expension
# We need to use absolute path for `root_dir`
root_dir: /home/me/.salt
pki_dir: pki
cachedir: cache
sock_dir: run
pidfile: pids

log_file: logs/master.log
key_logfile: logs/key.log

# Global level logged in file
log_level_logfile: debug

##
## file server
##
# I'm using formula repositories but override things from time to time
fileserver_backend:
  - roots

# Unfortunately, salt does not support ~ expension
# We need to use absolute path for `file_roots` and `pillar_roots`
file_roots:
  base:
    - /home/me/.salt/srv/salt/

pillar_roots:
  base:
    - /home/me/.salt/srv/pillar/
/home/me/.salt/roster
# -*- mode: yaml; coding: utf-8 -*-

testmachine2:
  host: 192.168.0.102
  user: root
/home/me/.salt/srv/salt/test-import-rootfs/init.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
include:
  - .absolute
  - .relative
/home/me/.salt/srv/salt/test-import-rootfs/absolute.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from "test-import-rootfs/map.jinja" import mapdata %}

test-import-rootfs/absolute/test.show_notification:
  test.show_notification:
    - name: {{ mapdata | json }}
/home/me/.salt/srv/salt/test-import-rootfs/relative.sls
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from "./map.jinja" import mapdata %}

test-import-rootfs/relative/test.show_notification:
  test.show_notification:
    - name: {{ mapdata | json }}
/home/me/.salt/srv/salt/test-import-rootfs/map.jinja
{#- -*- mode: jinja; coding: utf-8 -*- #}
{#- vim: ft=sls #}
{%- set mapdata = {"test-import-rootfs": "foo"} %}

I think there is a caching issue.

If I change the content of /home/me/.salt/srv/salt/test-import-rootfs/map.jinja I still get the old content. Cleaning my ~/.salt/cache or the minion /var/tmp/.root_* directories does nothing.

@baby-gnu
Copy link

When I change map.jinja, the cache is updated only when I execute sub states like salt-ssh testmachine2 state.apply test-import-rootfs.absolute and not salt-ssh testmachine2 state.apply test-import-rootfs.

Weird.

@baby-gnu
Copy link

I have the exact same caching issue with gitfs.

@baby-gnu
Copy link

baby-gnu commented Apr 8, 2022

Hello, #61895 fixes my issues with either rootfs and gitfs.

@Ch3LL
Copy link
Contributor

Ch3LL commented Aug 1, 2023

Is extra_filerefs working on newer versions of Salt now?

@baby-gnu
Copy link

I found a workaround by explicitly caching the wanted files as I tried with collectd formula.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt Salt-SSH severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Projects
None yet
Development

No branches or pull requests