Jinja templates - what object properties are available?

[etfz]

Hi,

Just got started with Jinja templating with the config templates. I'm a little confused as to what I have to work with in terms of various objects. For example, I discovered that interface objects has ip_addresses, but I could find no reference to that in the model documentation, nor is the field normally included when looking an interface object up using the API.

Is there any reference for this?

[kkthxbye-code]

You can check the code:

netbox/netbox/dcim/models/device_components.py

Lines 670 to 675 in b7f028f

	ip_addresses = GenericRelation(
	to='ipam.IPAddress',
	content_type_field='assigned_object_type',
	object_id_field='assigned_object_id',
	related_query_name='interface'
	)

Alternatively you can print all the fields on the model you are working with in nbshell:

print([f"{f.name} - {type(f).__name__}" for f in Interface._meta.get_fields()])

['child_interfaces - ManyToOneRel',
 'bridge_interfaces - ManyToOneRel',
 'member_interfaces - ManyToOneRel',
 'id - BigAutoField',
 'created - DateTimeField',
 'last_updated - DateTimeField',
 'custom_field_data - JSONField',
 'device - ForeignKey',
 'name - CharField',
 'label - CharField',
 'description - CharField',
 'module - ForeignKey',
 'cable - ForeignKey',
 'cable_end - CharField',
 'mark_connected - BooleanField',
 '_path - ForeignKey',
 'enabled - BooleanField',
 'mac_address - MACAddressField',
 'mtu - PositiveIntegerField',
 'mode - CharField',
 'parent - ForeignKey',
 'bridge - ForeignKey',
 '_name - NaturalOrderingField',
 'lag - ForeignKey',
 'type - CharField',
 'mgmt_only - BooleanField',
 'speed - PositiveIntegerField',
 'duplex - CharField',
 'wwn - WWNField',
 'rf_role - CharField',
 'rf_channel - CharField',
 'rf_channel_frequency - DecimalField',
 'rf_channel_width - DecimalField',
 'tx_power - PositiveSmallIntegerField',
 'poe_mode - CharField',
 'poe_type - CharField',
 'wireless_link - ForeignKey',
 'untagged_vlan - ForeignKey',
 'vrf - ForeignKey',
 'tags - TaggableManager',
 'vdcs - ManyToManyField',
 'wireless_lans - ManyToManyField',
 'tagged_vlans - ManyToManyField',
 'ip_addresses - GenericRelation',
 'fhrp_group_assignments - GenericRelation',
 'l2vpn_terminations - GenericRelation',
 'inventory_items - GenericRelation',
 'journal_entries - GenericRelation',
 'cable_terminations - GenericRelation',
 'tagged_items - GenericRelation']

[etfz]

I'll check it out. Thank you!

[candlerb]

Also in nbshell, you can try help(Interface). You'll then get some help text for each one, although in many cases it's generic Djangoese.

 |  ip_addresses
 |      Accessor to the related objects manager on the one-to-many relation created
 |      by GenericRelation.
 |
 |      In the example::
 |
 |          class Post(Model):
 |              comments = GenericRelation(Comment)
 |
 |      ``post.comments`` is a ReverseGenericManyToOneDescriptor instance.

[robertmain]

Seconding this - I'm having a hard time writing my config templates. How does one get the IP address for an interface (for example)?

Is it:

{{ interface.address }}

or

{{ interface.ip_address.address }}

or

{{ interface.ip_address.value }}

or

{{ interface.ip_address.ip }}

or is it

{{ interface.ip_address | first }}

I ask in earnest because I genuinely can't tell (even after using nbshell, I'm a little unused to Django, sorry). If someone has any insight, I'd appreciate it.

[zndrr]

I too have just started out on serious template construction and seeking guidance.
Will check through the above for now, but definitely keeping an eye on this space.

[candlerb]

How does one get the IP address for an interface (for example)?

In general, an interface has multiple IP addresses, so you could either get the first one, or all of them.

Here's a custom link I use for getting the management IP address of a device (which will be an explicit attribute in NB 3.6):

# Link text
{% if object.interfaces.filter(mgmt_only=True).count() == 1 and object.interfaces.get(mgmt_only=True).ip_addresses.count() == 1 %}Open IPMI{% endif %}

# Link URL
https://{{ object.interfaces.get(mgmt_only=True).ip_addresses.first().address.ip }}/

In nbshell:

# /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py nbshell
...
>>> d = Device.objects.get(name="nuc1")
>>> i = d.interfaces.first()
>>> [a for a in dir(i) if "address" in a]
['count_ipaddresses', 'ip_addresses', 'mac_address']
>>> [a for a in i.ip_addresses.all()]
[<IPAddress: 10.12.255.11/24>, <IPAddress: 2001:db8::11/64>]
>>> [a.address for a in i.ip_addresses.all()]
[IPNetwork('10.12.255.11/24'), IPNetwork('2001:db8::11/64')]
>>> [a.address.ip for a in i.ip_addresses.all()]
[IPAddress('10.12.255.11'), IPAddress('2001:db8::11')]
>>> [str(a.address) for a in i.ip_addresses.all()]
['10.12.255.11/24', '2001:db8::11/64']
>>> [str(a.address.ip) for a in i.ip_addresses.all()]
['10.12.255.11', '2001:db8::11']

Here, a is a Netbox IPAddress object, a.address is the IPNetwork object with the prefix length, and a.address.ip is just the address without prefix.

[kkthxbye-code]

It's not a great answer, but it's the best we got. I made an earlier reply in this very topic explaining the example with IP addresses:

#12568 (comment)

Experimenting in nbshell is your best bet. Everything you can do there should be possible in jinja as long as it's possible in jinja syntax. If you find yourself doing complicated stuff often, it might be best to write a custom filter:

https://docs.netbox.dev/en/stable/configuration/system/#jinja2_filters

A playground for quickly testing configuration templates is a good idea, but not something we currently support. It should be easily doable in a plugin. Not sure if it's something we want built-in though, some of the other maintainers would have to weigh in. Making a feature request for it would be fine though.

What kind of wrong results do you mean? Like Undefined or something? Most of my interfaces (possibly all of them) have IPs, but I'll probably check for this as you suggested for completeness and robustness.

I think it just renders nothing if there's no IP, but maybe it actually throws an exception, not sure. You'll have to play with it. If you always have IPs on all physical interfaces it should be fine.

[robertmain]

Thanks for the help @kkthxbye-code - I ended up with this:

{%- set interfaces = device.interfaces.all() -%}
{%- set physical = interfaces | selectattr('type', 'ne', 'virtual') -%}
{%- for interface in physical -%}
{%- if interface.ip_addresses.first() %}
set interfaces ethernet {{ interface.name }} address {{ interface.ip_addresses.first().address.ip }}
{%- endif %}
{%- if interface.label %}
set interfaces ethernet {{ interface.name }} description {{ interface.label }}
{%- endif %}
set interfaces ethernet {{ interface.name }} duplex auto
set interfaces ethernet {{ interface.name }} speed {{ interface.speed | default('auto') }}
{% endfor %}

[zndrr]

This has actually been hugely educational! Made strides in templating via this discussion.
Would there be any possibility of adding some breadcrumbs like the above to general docos? Some of us just need to be guided in the right direction.

I'll mirror robertmain on the thanks. Much appreciated.

[kkthxbye-code]

Would there be any possibility of adding some breadcrumbs like the above to general docos?

I think we are always open to modify the docs, but it's hard with stuff like custom scripts, configuration templates and nbshell as you can do pretty much anything. They are inherently power user features, and require some sort of knowledge about the internal structure of the models in netbox and the django ORM.

In my opinion stuff like that is more suited for blog posts/how-to guides, but there might be room for some very limited pointers.

[zndrr]

@kkthxbye-code I think the pointers here would be sufficient to share? Particularly the discovery stuff.
That was enough for me at least, and it seems robertmain also; could intuit a lot from that.

Of course I'm not out of the woods yet, so a lot is left to trial-and-error. Juggling J2 whitespacing and readability is the same regardless XD.
For now, what I have mocked up should be sufficient as PoC for my job, since config rendering won't be relied upon in the short term.

[candlerb]

Interesting, thanks. How did you work this out?

Firstly, in nbshell there is info on the attributes on an object using help(MODEL) - it includes comments extracted from the source code and/or from Django generic documentation.

e.g. help(Interface) includes the following within the results:

 |  ip_addresses
 |      Accessor to the related objects manager on the one-to-many relation created
 |      by GenericRelation.
 |
 |      In the example::
 |
 |          class Post(Model):
 |              comments = GenericRelation(Comment)
 |
 |      ``post.comments`` is a ReverseGenericManyToOneDescriptor instance.

Secondly, in Python in general, you can get a list of all the property names and method names on an object using dir(obj). So for example:

>>> i = d.interfaces.first()
>>> dir(i)

will list everything on that particular interface object - which will be a huge list (it includes all the methods inherited from parent classes, and lots of private methods methods).

If you know roughly what you're looking for, you can filter this list. I showed:

>>> [a for a in dir(i) if "address" in a]

That's a list comprehension: it take objects from a list (for a in dir(i)), filters it (if "address" in a which in this case is a substring match) and then returns a new list comprising those elements (the first a after the opening bracket is the expression used to create each new element in the list).

In both ways you learn that ip_addresses is an available property or method on Interface, but address is not. Hence when you wrote

  set interfaces ethernet {{ interface.name }} address {{ interface.address }} 

you got a message telling you exactly that:

jinja2.exceptions.UndefinedError: 'dcim.models.device_components.Interface object' has no attribute 'address'

And finally, there's the source code for Interface and IPAddress. On Interface, this gives you:

    ip_addresses = GenericRelation(
        to='ipam.IPAddress',
        content_type_field='assigned_object_type',
        object_id_field='assigned_object_id',
        related_query_name='interface'
    )

But that involves getting more into Django internals.

[robertmain]

@candlerb very informative, thanks!

[candlerb]

I can't for the life of me figure out how to distinguish ip4 and ip6 address families

>>> help(IPAddress)
...
 |  ----------------------------------------------------------------------
 |  Readonly properties defined here:
 |
 |  family
 |
 |  ----------------------------------------------------------------------
...

>>> a = Device.objects.get(name="nuc1").interfaces.first().ip_addresses.first()
>>> a.address
IPNetwork('10.12.255.11/24')
>>> a.family
4

[zndrr]

Right, seems I didn't go deep enough -_-;; , and also missed that when checking the help previously...

I can see the info in nbshell, so it's just stringing it together in J2. first() and last() give you the choice but not all()... makes sense, but I just have to learn how to filter specifically. My lack of overall understanding of the language is showing.

Skill issue basically. Sorry and thanks, I can brute force my way there. For now need a to take a break.

[candlerb]

first() and last() give you a single concrete object, or else None - that's easy to handle.

all() gives you a proxy manager object. I think you should be able to iterate over it: {%- for interface in blah.all() %} - but if not, you can always turn it into a concrete list of objects using | list as @robertmain showed.

The Jinja2 filter foo | list does the same as list(foo) in Python.

It is indeed annoying that Jinja2 is kind-of Python-ish, but different :-(

[ross-cello]

Cool, I freaking got it. Thanks heaps @candlerb (I'm the same guy as above).
Abridged mock-up below. Subject to proper cleanup, whitespacing etc. Thought it might have value including here for others.

{% for interface in device.interfaces.all() %}
!
interface {{ interface.name }}
{% if interface.description %} description {{ interface.description }}{% else %} no description{% endif %}

{% for ip in interface.ip_addresses.all() %}
{%- if ip.family ==  4 %} ip address {{ ip.address.ip }} {{ ip.address.netmask }}{% endif %}
{%- if ip.family ==  6 %} ipv6 address {{ ip.address }}{% endif %}
{% endfor -%}
exit
{% endfor -%}

[robertmain]

I don't have a ton to contribute, but I wanted to check back in with my working template so far...hopefully it helps somebody...

{%- set vlans = device.site.vlans.all() | list -%}
{%- set interfaces = device.interfaces.all() -%}
{%- set physical = interfaces | selectattr('type', 'ne', 'virtual') | list -%}
{%- set virtual = interfaces | selectattr('type', 'eq', 'virtual') | list -%}
{%- set port_speeds = {
} -%}
{%- for interface in physical -%}
{%- if interface.ip_addresses.first() %}
set interfaces ethernet {{ interface.name }} address {{ interface.ip_addresses.first().address }}
{%- endif %}
{%- if interface.label %}
set interfaces ethernet {{ interface.name }} description {{ interface.label }}
{%- endif %}
{%- if not interface.enabled %}
set interfaces ethernet {{ interface.name }} disable
{%- endif %}
set interfaces ethernet {{ interface.name }} duplex auto
set interfaces ethernet {{ interface.name }} speed {{ port_speeds[interface.speed | string] or 'auto' }}
{%- if interface.mtu %}
set interfaces ethernet {{ interface.name }} mtu {{ interface.mtu }}
{%- endif %}
{%- for vif in virtual | selectattr('parent.id', 'eq', interface.id) | list %}
{% set vlan = vlans | selectattr('id', 'eq', vif.untagged_vlan_id) | first %}
{%- if vif.ip_addresses.first() %}
set interfaces ethernet {{ vif.parent.name }} vif {{ vlan.vid }} address {{ vif.ip_addresses.first().address }}
{%- endif %}
set interfaces ethernet {{ vif.parent.name }} vif {{ vlan.vid }} description {{ vlan.name }}
{%- if vif.mtu %}
set interfaces ethernet {{ vif.parent.name }} vif {{ vlan.vid }} mtu {{ vif.mtu }}
{%- endif %}
{%- endfor %}
{% endfor %}

It looks a bit of a wall of text so I'm sure there's a neater way to express that ^

Critique welcome.

[ross-cello]

I've played around a bit to no success, both with 'family' and 'version'. Mostly complains about iteration.
With loop.first, syntax can be somewhat shortened. But since it runs against ip4 and ip6 (inside ip_addresses.all() ), it'll only apply against first occurrence, which typically is ip4 when both families are present... that does make sense.
So far starting loops with ip4 and ip6 isolated has been unsuccessful. What a journey so far... XD

[candlerb]

I gave you the wrong syntax for selectattr. Here it is again, this time tested and working.

hostname {{ device.name }}
{%-for intf in device.interfaces.all() %}
!
interface {{ intf.name }}
{%- for ip in intf.ip_addresses.all() | selectattr("family","equalto",4) %}
 ip address {{ ip.address.ip }} {{ ip.address.netmask }}{% if not loop.first %} secondary{% endif %}
{%- else %}
 no ip address
{%- endfor %}
{%- for ip in intf.ip_addresses.all() | selectattr("family","equalto",6)  %}
 ipv6 address {{ ip.address }}
{%- else %}
 no ipv6 address
{%- endfor %}
{%- endfor %}

[kkthxbye-code]

Here's how you filter by family in a queryset:

>>> IPAddress.objects.filter(address__family=4)
<RestrictedQuerySet [<IPAddress: 10.0.0.1/24>]>
>>> IPAddress.objects.filter(address__family=6)
<RestrictedQuerySet [<IPAddress: 2001:4860:4860::8888/126>]>

[candlerb]

Thanks. Confirmed that works too:

hostname {{ device.name }}
{%-for intf in device.interfaces.all() %}
!
interface {{ intf.name }}
{%- for ip in intf.ip_addresses.filter(address__family=4) %}
 ip address {{ ip.address.ip }} {{ ip.address.netmask }}{% if not loop.first %} secondary{% endif %}
{%- else %}
 no ip address
{%- endfor %}
{%- for ip in intf.ip_addresses.filter(address__family=6) %}
 ipv6 address {{ ip.address }}
{%- else %}
 no ipv6 address
{%- endfor %}
{%- endfor %}

[ross-cello]

Ugh dammit, I tried permutations of both selectattr("family","equalto",6) and intf.ip_addresses.filter(address__family=4) but didn't get the syntax quite right, ending in failure! Still boils down to an issue of education. Thanks again for delivering.
I'll edit above once I refined. Needed to jig a bit to factor in first loop ip mtu and ip6 enable

{% for intf in device.interfaces.all() %}
!
interface {{ intf.name }}
{%- for ip in intf.ip_addresses.filter(address__family=4) %}
{%- if loop.first %}
 ip mtu 1500
 !--- add other single occurrence ip4 commands here
 ip address {{ ip.address.ip }} {{ ip.address.netmask }}{% else %}
 ip address {{ ip.address.ip }} {{ ip.address.netmask }} secondary{% endif %}
{%- else %}
 no ip address
{%- endfor %}
{%- for ip in intf.ip_addresses.filter(address__family=6) %}
{%- if loop.first %}
 ipv6 enable
 ipv6 mtu 1500
 !--- add other single occurrence ip6 commands here
 ipv6 address {{ ip.address }}{% else %}
 ipv6 address {{ ip.address }}{% endif %}
{%- else %}
 no ipv6 address
{%- endfor %}
 load-interval 30
{% endfor %}
!

I'm okay with that level of duplicity. It actually resembles typical syntax for secondaries. Please go easy on me if you see opportunities to refine futher. XD

[mtinberg]

Yeah, the Swagger docs are awesome for listing out every query parameter which is a great reference, but I don't know of an easy way to list all the Django object filters as they are used in Jinja2 templates (or graphql). There are some hints here https://demo.netbox.dev/static/docs/administration/netbox-shell/ https://docs.djangoproject.com/en/4.2/ref/models/querysets/#id4 which is great, but having a lookup table for reference would be easier instead of having to use my slow brain to find this doc and calculate what the filter should look like for a particular object type. — Mark Tinberg ***@***.***> Division of Information Technology-Network Services University of Wisconsin-Madison

…

________________________________ From: ross-cello ***@***.***> Sent: Monday, August 21, 2023 2:30 AM To: netbox-community/netbox ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [netbox-community/netbox] Jinja templates - what object properties are available? (Discussion #12568) Ugh dammit, I tried permutations of both selectattr("family","equalto",6) and intf.ip_addresses.filter(address__family=4) but didn't get the syntax quite right, ending in failure! Still boils down to an issue of education. Thanks again for delivering. I'll edit above once I refined. Needed to jig a bit to factor in first loop ip mtu and ip6 enable {% for intf in device.interfaces.all() %} ! interface {{ intf.name }} {%- for ip in intf.ip_addresses.filter(address__family=4) %} {%- if loop.first %} ip mtu 1500 ip address {{ ip.address.ip }} {{ ip.address.netmask }}{% else %} ip address {{ ip.address.ip }} {{ ip.address.netmask }} secondary{% endif %} {%- else %} no ip address {%- endfor %} {%- for ip in intf.ip_addresses.filter(address__family=6) %} {%- if loop.first %} ipv6 enable ipv6 mtu 1500 ipv6 address {{ ip.address }}{% else %} ipv6 address {{ ip.address }}{% endif %} {%- else %} no ipv6 address {%- endfor %} load-interval 30 {% endfor %} ! I'm okay with that level of duplicity. It actually resembles typical syntax for secondaries. Please go easy on me if you see opportunities to refine futher. XD — Reply to this email directly, view it on GitHub<#12568 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAS7UMYDPYDQFDD2NPABIJTXWMFBZANCNFSM6AAAAAAX7D5R7A>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ross-cello]

Dude this is awesome; the second link in particular (first doesn't load for me).
Didn't think to look for Django resources before. Now I can filter for dot1q etc:

>>> a.interfaces.filter(name__iregex=r'\.')
<RestrictedQuerySet [<Interface: GigabitEthernet0/0/0.10>, <Interface: GigabitEthernet0/0/1.20>]>

Edit: Constructed this quickly (subject to refinement):

{% for ifd1q in device.interfaces.filter(name__contains=".") %}
{% set dot1q = ifd1q.name.rsplit('.', 1) %}
!
interface {{ ifd1q.name }}
 encapsulation dot1q {{ dot1q[1] }}
{% endfor %}
!

Rendering this

!
interface GigabitEthernet0/0/0.10
 encapsulation dot1q 10
!
interface GigabitEthernet0/0/1.20
 encapsulation dot1q 20
!

[bitcollector1]

This is an amazing thread, thanks to all the fine folks who contributed to this discussion.

I was able to get my templates built completely a few weeks after the feature was released (with some help from slack forum) but I've learned a lot by just reading this thread.