Report bugs to Variomedia dns api

[peterkelm]

This is the place to report bugs in the Variomedia DNS API.

If you experienced a bug, please report it in this issue.

Thanks!

[morlac]

Hi,
suddenly, when renewing my certs, for one of my subdomains I see the following response in the logs:

response='{"errors":[{"id":"invalid_api_data","status":"422","title":"Invalid API data.","detail":"Not ACE-encodable.","source":{"pointer":"/"}}],"links":{"self":"https://api.variomedia.de/domains/_acme-challenge.<SUBDOMAIN_HERE>"}}'
followed by:

response='{"errors":[{"id":"resource_not_found","status":"404","title":"Domain with ID '<SUBDOMAIN_HERE>' not found."}],"links":{"self":"https://api.variomedia.de/domains/<SUBDOMAIN_HERE>"}}'
I'm running on an all-updated 'Ubuntu 16.04.7 LTS'-Linode

[peterkelm]

@morlac, does that affect only one domain name of many? And if so, is there anything "special" about that particular one?

[morlac]

@peterkelm it happens on adding the TXT entry for the 2nd sub domain.
My config looks like:

Le_Domain='<DOMAIN>'
Le_Alt='*.<DOMAIN>,*.<SUB1>.<DOMAIN>,*.<SUB2>.<DOMAIN>'
[...]

there is nothing special with this domain at all.

Also any cleanup afterwards (removing of so far added TXT entries _acme-challange. and _acme-challange..) fails and must be done manually.

[peterkelm]

@morlac, I do not use the API with multiple subdomains, so there's a possibility that this isn't fully tested. However, your initial note seems to indicate that it worked okay at one point. Correct?

Writing the DNS entry happens in line 33 of dns_variomedia.sh - all other DNS API calls use only very simple data, so if something goes wrong this is most likely the place...

One of my certs renewed successfully yesterday... so it doesn't seem to be a general issue with the Variomedia API.
However, the Variomedia API has proven to be picky in the past when it comes to spaces in the JSON data.

Lastly, is there any further info in the .acme.sh/acme.sh.log logfile regarding this issue? It should contain a part like this:

[Sat Sep  5 00:23:11 CEST 2020] _sub_domain='_acme-challenge.subdomain'
[Sat Sep  5 00:23:11 CEST 2020] _domain='example.com'
[Sat Sep  5 00:23:11 CEST 2020] dns-records
[Sat Sep  5 00:23:11 CEST 2020] data='{"data": {"type": "dns-record", "attributes": {"record_type": "TXT", "name": "_acme-challenge.subdomain", "domain": "example.com", "data": "<some_key>", "ttl":300}}}'
[Sat Sep  5 00:23:11 CEST 2020] POST
[Sat Sep  5 00:23:11 CEST 2020] _post_url='https://api.variomedia.de/dns-records'

Does the "data=" line your log file look sane?

[morlac]

@peterkelm, yes, that's correct.
Until recently it worked more or less flawlessly - there once was a minor fault with a space in the response from the Variomedia API which was addressed with the last change in dns_variomedia.sh. (#3031 )

I took a closer look at the generated logs and to me it seems the error is rooted much earlier than I initially thougt.
I get errors at the very first beginning when talking to Variomedia-API (at the DOMAIN-Level):

command:

 acme.sh --cron --home "/root/.acme.sh/" --log-level 2 --debug 3

[....]
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4372                      d='morlac.de'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4387                      _d_alias
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4400                      txtdomain='_acme-challenge.morlac.de'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_base64:941                     base64 single line.
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4402                      txt='UhS6fEBZ-6DB9PVJ7tplNuTOiew2X4N9EiD4tx4uNHM'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4405                      d_api='/root/.acme.sh/dnsapi/dns_variomedia.sh'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4408                      dns_entry='morlac.de,_acme-challenge.morlac.de,,dns_variomedia,UhS6fEBZ-6DB9PVJ7tplNuTOiew2X4N9EiD4tx4uNHM,/root/.acme.sh/dnsapi/dns_variomedia.sh'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:issue:4410                      Found domain api file: /root/.acme.sh/dnsapi/dns_variomedia.sh
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_exists:514                     dns_variomedia_add exists=0
[Mon Sep  7 15:08:05 CEST 2020] Adding txt value: UhS6fEBZ-6DB9PVJ7tplNuTOiew2X4N9EiD4tx4uNHM for domain:  _acme-challenge.morlac.de
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:dns_variomedia_add:14 fulldomain='_acme-challenge.morlac.de'
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:dns_variomedia_add:15 txtvalue='UhS6fEBZ-6DB9PVJ7tplNuTOiew2X4N9EiD4tx4uNHM'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_setopt:2132                    OK
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_setopt:2150                    12:SAVED_VARIOMEDIA_API_TOKEN='<TOKEN>'
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:dns_variomedia_add:26 First detect the root zone
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:_get_root:100   h='_acme-challenge.morlac.de'
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:_variomedia_rest:127 ep='domains/_acme-challenge.morlac.de'
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:_variomedia_rest:133 _H1='Authorization: token <TOKEN>'
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:_variomedia_rest:134 _H2='Content-Type: application/vnd.api+json'
[Mon Sep  7 15:08:05 CEST 2020] dnsapi/dns_variomedia.sh:_variomedia_rest:135 _H3='Accept: application/vnd.variomedia.v1+json'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_get:1906                       GET
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_get:1910                       url='https://api.variomedia.de/domains/_acme-challenge.morlac.de'
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_get:1911                       timeout=
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_inithttp:1734                  Http already initialized.
[Mon Sep  7 15:08:05 CEST 2020] /acme.sh:_get:1923                       _CURL='curl -L --silent --dump-header /root/.acme.sh//http.header  --trace-ascii /tmp/tmp.qJDL30VlVL  -g '
[Mon Sep  7 15:08:06 CEST 2020] /acme.sh:_get:1963                       ret='0'
[Mon Sep  7 15:08:06 CEST 2020] dnsapi/dns_variomedia.sh:_variomedia_rest:149 response='{"errors":[{"id":"invalid_api_data","status":"422","title":"Invalid API data.","detail":"Not ACE-encodable.","source":{"pointer":"/"}}],"links":{"self":"https://api.variomedia.de/domains/_acme-challenge.morlac.de"}}'
[....]

(I added some debug2 statements in dns_variomedia.sh at 133, 134, 135)
What irritates me is the 1963: ret='0' combined the following line stating status 422 ..

Interstingly the TXT entry has been setup and I have to delete it amongst some later ones for manually in the Variomedia management console.
The token itself is unchanged since I got it from Variomedia support and somehow it seems to work?!

Edit: the data='{"data": {"type": "dns-record", "attributes": {"record_type": "TXT", "name": "_acme-challenge", "domain": "morlac.de", "data": "bIp7kc9_UAE2leu_MKj4-H_wYB0F57QL5-LQsVIzPy0", "ttl":300}}}' line for the domain and the first subdomain data='{"data": {"type": "dns-record", "attributes": {"record_type": "TXT", "name": "_acme-challenge.homecentral", "domain": "morlac.de", "data": "jTzdW-aOU--qW4khFVaab6ndBDy5cKGqaY1WHdWyJlk", "ttl":300}}}' look good I'd say ..

[hynek]

Hi, I'm responsible for the API at Variomedia.

The problem is that you used to rely on a (former) buggy behavior where the /domains endpoint ignored subdomains. That has been fixed at some point.

Your request must go against https://api.variomedia.de/domains/morlac.de

LMK if you need anything.

[peterkelm]

@hynek, thanks for that pointer. I think I understand what you referred to:

All calls to the /domains endpoint have to go to the "root domain" (as registered with you guys).

Will test this out over the next few days.

Out of interest, has this behavioral change been documented somewhere in the Variomedia API changelog? I didn't spot it anywhere...

[hynek]

All calls to the /domains endpoint have to go tho the "root domain" (as registered with you guys).

Exactly!

Will test this out over the next few days.

Today is my last day before my vacation but I'm happy to help y'all. We're big fans at Variomedia.

Out of interest, has this behavioral change been documented somewhere in the Variomedia API changelog? I didn't spot it anywhere...

TBH I didn't expect anyone having noticed that "glitch" but I guess Hyrum's Law strikes again. ¯\_(ツ)_/¯

[peterkelm]

Checked in PR #3244 to address the issue with subdomain handling in the /domains API call.
@morlac, would be great if you could let me know whether that change fixes the issue you saw (once the PR has been integrated here).

[morlac]

Hi, because of nessecity (renewal of my certs were due) I manually modified dnsapi/dns_variomedia.sh according to PR #3244 and the renewal process was successfull. :)
Thx @peterkelm

[IngoEF]

Hi,
after updating my pfSense-Routers from 2.6 to 2.7 LE-certificate renewal doesn't work anymore.
The SAN-certificate which I used to renew for about 3 years contains one wildcard an one domain certificate.
The initial error message in the log file seems to be this:
response='{"errors":[{"id":"resource_not_found","status":"404","title":"Domain with ID 'net' not found."}],"links":{"self":"https://api.variomedia.de/domains/net"}}'
After looking deeper, I think there goes something wrong in the "_get_root()" function in "dns_variomedia.sh".
It works through a loop ending in a domain name which is only the tld-part of my domain (net) and obviously this can't be the root.
Would you mind having a look at it and correct the issue?
Find attached the last log of unsuccessful renewal.
Thanks in advance.

acme_issuecert.log

[IngoEF]

After asking for it the Variomedia Support sent me a new script for their API with the comment:
Es gibt bei guthub bereits einen entsprehenden PullRequest welcher aber noch nicht bearbeitet ist. Vorab eine von uns angepasste Variante.
I attach it here for implementing it (e.g. in pfSense into the folder /usr/local/pkg/acme/dnsapi
It works!
dns_variomedia.zip
Of course you need to unpack it and overwrite the original one,

[sokai]

• I could fix this in my pfSense by using the dns_variomedia.sh script from the v3.0.7 branch (= last release)
  • The acme package of pfSense 2.7.1-RELEASE is v0.7.5, where the last acme.sh update was to 3.0.6 (in v0.7.4)
• I issued + renewed a LE wildcard cert for a subdomain (*.sub.doma.in)
• I think the main fix is/was (from September 2023) in the PR Fix Variomedia API #4787

1. SSH into your pfSense
   • ssh root@$PFSENSE
   • Select option 8) Shell
2. Backup current script
   • cd /usr/local/pkg/acme/dnsapi
   • mv dns_variomedia.sh dns_variomedia.dist.sh
3. Get script from dev branch
   • curl -L "https://github.com/acmesh-official/acme.sh/raw/dev/dnsapi/dns_variomedia.sh" --output dns_variomedia.sh
4. Logout
   • exit
   • Select option 0) Logout (SSH only)

Notice: I don't know how long that fix/workaround will last in the filesystem or if this will harm/cause other things.

Many thanks + KR