Skip to content

API

Jump to bottom
Øyvind Hagberg edited this page Nov 7, 2022 · 27 revisions

API conventions

Versioning

The API is versioned. The version is in the url: /api/v1.
(We are aware that there's some debate about the "right way" to do API versioning, here's an article that discusses different methods.)

A consequence of Semantic Versioning is that the API is defined to be backwards compatible for anything but major version increments. Therefore, we only need to include the major version number in the API url.

When a new major version of the software is released, the old API can still be available through its old endpoint:
/api/v1
/api/v2
The goal is to support old API versions for a while, allowing users time to migrate to new versions.

Principles

  • GET requests never cause any changes or modify any data on the server side
  • Every function require that you specify which fields you want returned (the fields parameter)
  • The server is stateless as far as the API is concerned
  • The API returns data in JSON format

Authentication

Use the http header Authorization.

Authorization: <type> <credentials>

Use "APIKEY" as the type, and your key as credentials.

Here's an example, using curl:

curl -sSH "Authorization: APIKEY j3ZCWOzJA2vFmO6OhpdEIxYX3ZZR0FGi" \
"https://nivlheim.uio.no/api/v2/hostlist?fields=hostname"

API functions

file

Retrieve file contents and metadata.

Available fields: fileId, filename, isCommand, lastModified, received, content, certfp, hostname, versions, isNewestVersion, isDeleted

Parameters: Either a fileId or a combination of filename and (hostname or certfp).

  • You can also add "format=raw" to retrieve just the file content (as text/plain), as opposed to a JSON data structure.

If you specify a filename but no host, you'll get a JSON list with this file from all hosts.
In that case, you can specify a relative value for lastseen:

  • lastseen<2h last seen less that 2 hours ago
  • lastseen>5m last seen more than 5 minutes ago
    s = seconds, m = minutes, h = hours, d = days

NB! If a file has been deleted from a host, and Nivlheim still has an old copy, this API will return it. I.e. if you don't specify fileId it will return the latest version of a file even if the file has since been deleted. You can ask for the isDeleted field, which will tell you if a file still exists on the host or has been deleted.

Noteworthy http status codes
  • 404 (Not Found) if no file fits the criteria.
Examples

  • Retrieve details for the latest (newest) version of a file:

    • /api/v2/file?filename=/usr/sbin/dmidecode+-t+system&hostname=myhost.uio.no&fields=lastModified,content
    • /api/v2/file?filename=/etc/fedora-release&certfp=04E42F1F081827DA539FBEB961FAC952929C5ADA&fields=fileId,content

  • Retrieve details for a specific version of a file (use the fileId parameter):
    /api/v2/file?fileId=1234&fields=hostname,filename,content

Example response data 
{
    "fileId": 1234,
    "filename": "/usr/sbin/dmidecode -t system",
    "isCommand": true,
    "lastModified": "2018-02-08T13:00:22+01",
    "received": "2018-02-08T13:11:45+01",
    "content": "Line one\nLine two\nLine three",
    "certfp": "04E42F1F081827DA539FBEB961FAC952929C5ADA",
    "hostname": "bottleneck.example.no",
    "versions": [
        {
            "fileId": 1234,
            "lastModified": "2018-02-08T13:00:22+01"
        },
        {
            "fileId": 5678,
            "lastModified": "2018-01-02T03:04:05+01"
        }
    ]
}
Retrieve just the contents, as text/plain (no JSON)
  • /api/v2/file?fileId=41621&format=raw
  • /api/v2/file?hostname=bn.example.no&filename=/usr/bin/lscpu&format=raw
Example response data 
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Wed, 25 Jul 2018 14:00:26 GMT
Content-Length: 693

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             2
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 37
Model name:            Intel(R) Xeon(R) CPU E7- 2850  @ 2.00GHz
Stepping:              1
CPU MHz:               1995.000
BogoMIPS:              3990.00
Hypervisor vendor:     VMware
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              24576K
NUMA node0 CPU(s):     0,1

host

Retrieve metadata and/or a file list for a given host machine.

Available fields: ipAddress, hostname, lastseen, os, osEdition, osFamily, kernel, manufacturer, product, serialNo, certfp, clientVersion, files, support + any defined custom fields

Parameters: Append either the hostname or certfp to the url path

Noteworthy http status codes
  • 404 (not found) if no host fits the criteria.
Examples
  • List the files on a host, and also some details:
    /api/v2/host/bottleneck.example.no?fields=files,lastseen,kernel
Example response data (all fields included): 
{
    "ipAddress": "123.123.123.123",
    "hostname": "bottleneck.example.no",
    "lastseen": "2018-02-08T11:22:33+01",
    "os": "Fedora 27",
    "osEdition": "Workstation",
    "osFamily": "Linux",
    "kernel": "4.14.16-300.fc27.x86_64",
    "manufacturer": "Dell Inc.",
    "product": "OptiPlex 7010",
    "serialNo": "1YF2YF3",
    "certfp": "298376ACBDFCBAFDC589356864ABCDEF",
    "clientVersion": "0.2.0",
    "files": [
        {
            "filename": "/etc/redhat-release",
            "lastModified": "2018-02-08T13:00:22+01",
            "isCommand": false
        },
        {
            "filename": "/bin/uname -a",
            "lastModified": "2018-02-08T13:00:22+01",
            "isCommand": true
        },
        {
            "filename": "/usr/sbin/dmidecode -t system",
            "lastModified": "2018-02-08T13:00:22+01",
            "isCommand": true
        }
    ],
    "support": [
        {
            "start": "2018-01-02T11:22:33+01",
            "expires": "2018-02-03T11:22:33+01",
            "description": "On-site 4-hour premium support package",
            "hasExpired": true
        }
    ]
}
Removing hosts

To delete a host, use the http delete method:
DELETE /api/v2/host/hostname.example.com
DELETE /api/v2/host/5B13C90511EA751DB74932D3C23F2C0A63574ABE (certificate fingerprint)

Modifying hosts

To modify values for a host, use the http patch method. The body should be form/urlencoded, not json. Currently, the API only supports setting the value of overridehostname, which is used to specify what the hostname should be, instead of letting the system determine it.

PATCH /api/v2/host/hostname.example.com

overridehostname=newname.example.com

hostlist

Generate a list of hosts that fit certain criteria and specify which metadata fields you want in the list.

Available fields: ipAddress, hostname, lastseen, os, osEdition, osFamily, kernel, manufacturer, product, serialNo, certfp, clientVersion + any defined custom fields

Parameters: Any from the "available fields" list.

  • Wildcards: You can use * as a wildcard.
    os=Fedora*

  • Multiple values (logical OR), comma-separated: os=Debian+8,Debian+9

Operators:

  • Equals: hostname=myhost.mydomain.com
  • Not equals: manufacturer!=Dell*
  • Less/more: kernel<4.14
    hostname>a&hostname<b

Sorting:

  • Ascending: sort=hostname
  • Descending: sort=-lastseen (notice the minus sign)

Lastseen relative time:

  • lastseen<2h last seen less that 2 hours ago
  • lastseen>5m last seen more than 5 minutes ago
    s = seconds, m = minutes, h = hours, d = days

Limit the number of results:

  • limit=10

Jump to an offset in the result list (only makes sense if you also specify a sort order):

  • offset=20
Examples
  • List all hosts with names that start with "mech":
    /api/v2/hostlist?fields=hostname&hostname=mech*
  • Get a list of the last 10 hosts that reported in:
    /api/v2/hostlist?fields=hostname,certfp,lastseen&sort=-lastseen&limit=10
  • List all hosts that we haven't heard from in 3 hours:
    /api/v2/hostlist?fields=hostname,ipAddress,lastseen&lastseen>3h
Example response data: 
[
    {
        "ipAddress": "123.123.123.123",
        "hostname": "bottleneck.example.no"
    },
    {
        "124.124.124.124",
        "hostname": "other.example.no"
    }
]

Group and count

/api/v2/hostlist?fields=os&count=1

Example response data: 
[
    {
        "os": "Fedora 30",
        "count": 5
    },
    {
        "os": "Windows 2016",
        "count": 11
    },
    {
        "os": "macOS 10.13",
        "count": 62
    }
]

search

Search files for content and return the hits.

Parameters:

  • q: Text phrase to search for
  • filename: Optional. Restrict the search to files with this name.
  • fields: A comma-separated list of which fields to return per hit.

Available fields: filename, fileID, content, ipAddress, hostname, lastseen, os, osEdition, osFamily, kernel, manufacturer, product, serialNo, certfp, clientVersion + any defined custom fields

Example
  • /api/v2/search?q=Maipo&filename=/etc/redhat-release&fields=hostname,ipAddress,content
Example response data for this search 
[
    {
        "content": "Red Hat Enterprise Linux Workstation release 7.7 (Maipo)\n",
        "hostname": "tristan.example.com",
        "ipAddress": "110.120.130.140"
    },
    {
        "content": "Red Hat Enterprise Linux Server release 7.7 (Maipo)\n",
        "hostname": "isolde.example.com",
        "ipAddress": "150.160.170.180"
    }
]
Noteworthy http status codes
  • 503 Service Unavailable - The API will return this status code if the server process has recently (re)started and is loading data into memory. In that case, clients should look at the returned http header Retry-After and delay this many seconds before trying again.

msearch

Perform multiple searches for content and configure how to combine the results. Returns a list of the hosts that match.

Parameters:

  • q1..q<n>: Text phrase to search for
  • f1..f<n>: Optional. Restrict the search to files with this name.
  • op2..op<n>: Operation. How the result from this stage should be combined with the previous result. Only needed from stage 2 and onwards. Supported operations are:
    • and (intersection)
    • or (union)
    • sub (subtraction/difference)
  • fields: A comma-separated list of which fields to return per hit

Available fields: ipAddress, hostname, lastseen, os, osEdition, osFamily, kernel, manufacturer, product, serialNo, certfp, clientVersion + any defined custom fields

Example
  • /api/v2/msearch?q1=Maipo&f1=/etc/redhat-release&q2=mysql&f2=/etc/group&op2=and&fields=hostname,ipAddress
Example response data for this search 
[
    {
        "hostname": "tristan.example.com",
        "ipAddress": "110.120.130.140"
    },
    {
        "hostname": "isolde.example.com",
        "ipAddress": "150.160.170.180"
    }
]
Noteworthy http status codes
  • 503 Service Unavailable - The API will return this status code if the server process has recently (re)started and is loading data into memory. In that case, clients should look at the returned http header Retry-After and delay this many seconds before trying again.

grep

Searches all files and prints matching lines.

Available fields: You don't need to specify return fields for this function

Parameters:

  • q: Text phrase to search for
  • filename: Optional. Restrict the search to files with this name.
  • limit: (optional) Limit the number of returned lines

Returns: Colon-separated lines of result, on the format hostname:filename:line

Example
  • /api/v2/grep?q=wheel
Example response data 
log-receiver-test01.uio.no:/etc/sudoers:## allows people in group wheel to run all commands
log-receiver-test01.uio.no:/etc/sudoers:%wheel  all=(all)       all
Noteworthy http status codes
  • 503 Service Unavailable - The API will return this status code if the server process has recently (re)started and is loading data into memory. In that case, clients should look at the returned http header Retry-After and delay this many seconds before trying again.

status

Returns status information useful for monitoring.

Available fields: You don't need to specify return fields for this function

Parameters: none

Example

/api/v2/status

Example response data 
{
    "filesLastHour": 325,
    "numberOfMachines": 11,
    "numberOfFiles": 135,
    "reportingPercentage": 94,
    "parseQueueSize": 52,
    "taskQueueSize": 17,
    "failingTasks": 0,
    "ageOfNewestFile": 3.45678,
    "throughputPerSecond": 137,
    "lastExecutionTime": {
        "handleDNSchangesJob": 5.6169376,
        "parseFilesJob": 0.00103298,
        "pruneOldFilesJob": 77.20923,
        "scanQueueDirJob": 0.003824931
    },
    "errors": {
        "pruneOldFilesJob": "read unix @-\u003e/var/run/postgresql/.s.PGSQL.5432: read: connection reset by peer"
    }
}
Clone this wiki locally