Archiving configuration is ignored by Archiver Appliance

[kristjansoln]

Hey!

We've been setting up Channel Finder together with RecSync to pass archiving properties to the Archiver Appliance. The channels with the archive property were submitted to the Archiver, however, the exact configuration (i.e. the value of the archive property) seems to be ignored by Archiver. The PVs in the Archiver keep getting configured to monitor@1, even though a different configuration is specified in the archive property. After checking the logs and inspecting with Wireshark, it seems like the problem is in the archivePV request.

Here is a Wireshark capture of the archivePV request for an example PV.

...
Hypertext Transfer Protocol
    POST /mgmt/bpl/archivePV HTTP/1.1\r\n
    accept-encoding: gzip\r\n
    user-agent: ReactorNetty/1.0.22\r\n
    host: archiver.lan:17665\r\n
    accept: */*\r\n
    Content-Type: application/json\r\n
    content-length: 70\r\n
    \r\n
    [Full request URI: http://archiver.lan:17665/mgmt/bpl/archivePV]
    [HTTP request 4/5]
    [Prev request in frame: 3808]
    [Response in frame: 3927]
    [Next request in frame: 3931]
    File Data: 70 bytes
JavaScript Object Notation: application/json
    Array
        Object
            Member Key: pv
                String value: ioc1:aiExample
                Key: pv
            Member Key: samplingMethod
                String value: SCAN
                Key: samplingMethod
            Member Key: samplingPeriod
                String value: 5
                Key: samplingPeriod

The properties in the request are samplingMethod and samplingPeriod. However, in the Archiver Appliance documentation, these are defined as samplingmethod and samplingperiod.

Below are some tests using curl. Here, an archivePV request is performed, followed by a getPVStatus request for the same PV. You can see in the command output that the specified and the returned archiving configurations do not match.

$ curl -X POST -H "Content-Type: application/json" -d '[{"pv":"ioc1:test1:State-Sts","samplingMethod":"SCAN","samplingPeriod":"5"}]' http://archiver.lan:17665/mgmt/bpl/archivePV
...
$ curl "http://archive.lan:17665/mgmt/bpl/getPVStatus?pv=ioc1:test1:State-Sts"
[
   {
      "lastRotateLogs":"Never",
      "appliance":"archappl0",
      "pvName":"ioc1:test1:State-Sts",
      "pvNameOnly":"ioc1:test1:State-Sts",
      "connectionState":"false",
      "lastEvent":"Nov\/09\/2024 00:21:03 +01:00",
      "samplingPeriod":"1.0",
      "isMonitored":"true",
      "connectionLastRestablished":"Never",
      "connectionFirstEstablished":"Nov\/09\/2024 00:07:43 +01:00",
      "connectionLossRegainCount":"0",
      "status":"Being archived"
   }
]

Here is the same request, but using samplingmethod and samplingperiod instead.

$ curl -X POST -H "Content-Type: application/json" -d '[{"pv":"ioc1:test2:State-Sts","samplingmethod":"SCAN","samplingperiod":"5"}]' http://archiver.lan:17665/mgmt/bpl/archivePV
...
$ curl "http://archiver.lan:17665/mgmt/bpl/getPVStatus?pv=ioc1:test2:State-Sts"
[
   {
      "lastRotateLogs":"Never",
      "appliance":"archappl0",
      "pvName":"ioc1:test2:State-Sts",
      "pvNameOnly":"ioc1:test2:State-Sts",
      "connectionState":"true",
      "lastEvent":"Nov\/09\/2024 00:26:13 +01:00",
      "samplingPeriod":"5.0",
      "isMonitored":"false",
      "connectionLastRestablished":"Never",
      "connectionFirstEstablished":"Nov\/09\/2024 00:26:13 +01:00",
      "connectionLossRegainCount":"0",
      "status":"Being archived"
   }
]

For these tests, we are using Archiver Appliance v1.1.0 and a Channel Finder build from the latest commit on master (1e3c29f).

[shroffk]

Interesting...Has this been changed in the recent archiver release 2.0.5?
I have tested with newer versions of AA ( dec 2023 is what I use in production ) and I have not seen this issue.
@tynanford can you confirm if you are running into this issue with the camelCase request parameters not matching what the archiver expects

[tynanford]

We are still using CF version 4.7.2 and an old version of AA (dec 2022)

I tried to upgrade CF in September to the master branch and found the archiver wasn't being populated but I didn't have time to debug much more so we reverted. Maybe this is related...

[kristjansoln]

Well, when we were testing Channel Finder release 4.7.3-alpha, we encountered an issue where the AA processor would forward only the first PV to the Archiver, even though multiple PVs should have been forwarded. After some debugging, we concluded that the getPVStatus request to the Archiver used ampersands as separators, whereas the Archiver expected a comma-separated list, as stated in the documentation. Due to the incorrect separators, the Archiver processed and returned the status for only the first PV in the request.

This issue was fixed on the master branch, I believe a couple months ago, but if you experienced similar behavior, it might be related to your problem, @tynanford.

[kristjansoln]

I’ve tried to fix the problem on my own and have opened a PR, in case you decide this issue is to be fixed.

[shroffk]

I am just concerned with the versioning issues.
There has just been a new release of Archiver appliance, should we list the archiver version as a requirement? or try to support multiple versions?

[jacomago]

Sorry for the late reply on this one. From what I can tell all versions of the Archiver Appliance use "samplingperiod" and "samplingmethod" for submitting a PV to be archived. It's only in the response from that it uses "samplingMethod" and "samplingPeriod".

See: https://github.com/archiver-appliance/epicsarchiverap/blob/6d0077621379cb056ca7e1af69ca17bfd645d225/src/main/org/epics/archiverappliance/mgmt/bpl/ArchivePVAction.java#L109

and
https://github.com/archiver-appliance/epicsarchiverap/blob/6d0077621379cb056ca7e1af69ca17bfd645d225/src/main/org/epics/archiverappliance/config/PVTypeInfo.java#L132

And we do support multiple versions of the archiver appliance. See:

ChannelFinderService/src/main/java/org/phoebus/channelfinder/processors/aa/ArchiverClient.java

Line 29 in 8232cd4

private static final List<String> AA_STATUS_ENDPOINT_ONLY_SUPPORT_QUERY_VERSION = List.of("1.1.0",

For where we swap behavior between versions.