NetBox: integrate into Malcolm for asset inventory/management

[mmguero]

From Malcolm created by robefernandez: cisagov#113

Congratulations for the project, it's really useful and easy to setup in just minutes using the scripts and docker compose.

I've just deployed the solution for testing it so I'm actually a newbie and I have to spend more time to discover all the features but I have a question that will be decisive to continue using it or not by the moment:
Does it have asset inventory capabilities to list all the devices on the network?

I set to true the property LOGSTASH_OUI_LOOKUP (Logstash will map MAC addresses to vendors for all source and destination MAC addresses when analyzing Zeek logs).

Is there any dashboard or any place where we can obtain a list of the network devices?

Best regards.

[mmguero]

The other way I could do it to preserve the mapping of MAC->OUI as I talked about in the prevoius comment would be to do this:

...
    "network": {
      "type": "ipv4",
      "mac_oui": {
        "00:10:db:ff:10:01": "Juniper Networks",
        "8c:85:90:65:85:8f": "Apple, Inc."
      }
    },
...

Basically network.mac_oui could be a hash of MAC addresses to OUIs... but I'm not sure how well this works in visualizations

[mmguero]

Currently MAC addresses and OUIs are stored in separate source/destination (or originator/responder if you prefer) fields:

srcMac/dstMac
srcOui/dstOui

There are visualizations in the Connections dashboard in Kibana, or by going to the SPIView in Moloch and selecting these fields you could CSV-export a list of these separately and combine them.

You are not the first person to ask about aggregated src/dst fields to do asset inventory, though. I am going to use this github issue as an enhancement request to create an aggregated field that contains both (probably using the Elastic Common Schema as that's where things will probably end up).

[mmguero]

Hmmm... at that point I Think you're better off just running two reports yourself and combining the results for source and destination. I don't know that it would be desirable to aggregate all of an events IPs into a single IP field. But you could create a kibana visualization that's a data table, with

srcIp, zeek.orig_hostname, srcMac, srcOui, and service

and then another one for destination IPs, then export and combine the results.

[mmguero]

Commit ae60cf2 creates network.oui and network.mac fields which are arrays made of source and dst OUI and source and dest mac, respectively. This will make it easier to get lists of MAC addresses for asset inventory purposes.

The only gotcha I can see here is is if you create like a table visualization with split rows, you'd end up with something like this:

Say you have a single record with ["Xerox", "Apple"] for network.oui and ["9c:93:4e:01:02:03", "d8:30:62:01:02:03"]. If you create a table visualization with split rows I guess you'd end up with something like this?

OUI            MAC Address
-----------     -------------------
Apple         d8:30:62:01:02:03
Apple         9c:93:4e:01:02:03
Xerox         d8:30:62:01:02:03 
Xerox         9c:93:4e:01:02:03

I think that's how it would work anyway, once they're merged into their arrays you lose the association between them like you have with srcMac/srcOui dstMac/dstOui.

[mmguero]

Thanks @mmguero for your quick response.

What do you think if IP address and hostname are also shown?

Let me give you an example:

Hostname	IP	MAC	OUI
Router	10.224.0.1	00:10:DB:FF:00:11	Juniper Networks
HMIfloor2	10.224.0.102	88:5D:90:7F:D3:67	Schmidt & Co.
HMIfloor3	10.224.0.103	1D:34:FE:9A:12:7A	Schmidt & Co.
PLC001	10.224.10.11	00:11:00:7F:5D:34	Schneider Electric
PLC002	10.224.10.12	00:11:00:7F:89:67	Schneider Electric
PLC003	10.224.10.13	00:11:00:7F:34:98	Schneider Electric
Scada001	10.224.10.10	1C:21:D1:5F:BA:76	B-Scada Inc.
Srv-GBC9	10.224.10.15	A0:AF:BD:FF:AB:13	Intel

And just to go one step ahead, what do you think if we can get what services are exposed?

Hostname	IP	MAC	OUI	Services
Srv-GBC9	10.224.10.15	A0:AF:BD:FF:AB:13	Intel	68/udp ( dhcpc )
				517/udp ( talk )
				518/udp ( ntalk )
				631/udp ( ipp )
				1719/udp ( h323gatestat )
				5353/udp ( zeroconf )

Thanks.

[mmguero]

After doing some more research, I think the direction we're going to go is to integrate NetBox into Malcolm, for several reasons:

• It's a very featureful asset management solution
• It's actively developed and maintained
• It's Apache License 2.0 license is compatible with Malcolm's
• I've worked out how to spin it up in Docker and integrate it with Malcolm's authentication mechanisms
• It's got an easy REST API that Malcolm (or whatever else) can interface with
• It provides a modern full-featured GUI a user could use to go beyond the stuff Malcolm could auto-populate

[mmguero]

I should check out the netbox device type library (see https://github.com/netbox-community/devicetype-library and https://github.com/minitriga/Netbox-Device-Type-Library-Import/blob/master/nb-dt-import.py) to flesh out device types upon startup.

[mmguero]

Closing this issue as the initial integration of netbox is done. But I will open another issue (and reference this one) that will be a meta-issue outlining the steps for data enrichment and other uses for the feature between netbox and the rest of the Malcolm stack.