failed to send telegram notification, can't parse entities character '#' is reserved

[xshadowlegendx]

📣 Notification Service(s) Impacted

Telegram

🐞 Describe the bug

I am working on a script to send telegram notification after github pr merged webhook and sending the message without format=markdown works except it did not rendered the markdown, adding the option resulted in the error

Failed to send Telegram notification to -123123123: Bad Request: can't parse entities: Character '#' is reserved and must be escaped with the preceding '\', error=400

I used this command to send the message apprise -vvv -t 'Github PR Merged' -b "$MESSAGE_BODY" "tgram://$TELEGRAM_BOT_TOKEN/$TELEGRAM_CHAT_ID?format=markdown"

💡 Screenshots and Logs

2024-07-30 16:40:47,834 - DEBUG - Notification Plugin 108(s) and 154 Schema(s) loaded in 0.2239s
2024-07-30 16:40:47,836 - DEBUG - Loaded Telegram URL: tgram://7...Q/-123123123123/?image=False&detect=no&silent=no&preview=no&content=before&mdv=v2&format=markdown&overflow=upstream&rto=4.0&cto=4.0&verify=yes
2024-07-30 16:40:47,836 - DEBUG - Telegram POST URL: https://api.telegram.org/bottoken/sendMessage (cert_verify=True)
2024-07-30 16:40:47,836 - DEBUG - Telegram Payload: {'disable_notification': False, 'disable_web_page_preview': True, 'parse_mode': 'MarkdownV2', 'text': '# Github PR Merged\r\n\n[chore: added some stuff](https://httpbin.org)\n\nRepository: https://httpbin.org\n\nMerged onto: dev', 'chat_id': -123123123123}
2024-07-30 16:40:48,694 - WARNING - Failed to send Telegram notification to -123123123: Bad Request: can't parse entities: Character '#' is reserved and must be escaped with the preceding '\', error=400.
2024-07-30 16:40:48,694 - DEBUG - Response Details:
b'{"ok":false,"error_code":400,"description":"Bad Request: can\'t parse entities: Character \'#\' is reserved and must be escaped with the preceding \'\\\\\'"}'

💻 Your System Details:

• OS: macOS
• Python Version: 3.12.3

🔮 Additional context

I tried this apprise -vvv -i markdown -t 'Github PR Merged' -b "$MESSAGE_BODY" "tgram://$TELEGRAM_BOT_TOKEN/$TELEGRAM_CHAT_ID?format=markdown" it works only when I remove the URL in the repository line Repository: https://httpbin.org

2024-07-30 16:55:22,657 - DEBUG - Notification Plugin 108(s) and 154 Schema(s) loaded in 0.2276s
2024-07-30 16:55:22,658 - DEBUG - Loaded Telegram URL: tgram://7...Q/-123123123/?image=False&detect=no&silent=no&preview=no&content=before&mdv=v2&format=markdown&overflow=upstream&rto=4.0&cto=4.0&verify=yes
2024-07-30 16:55:22,658 - DEBUG - Telegram POST URL: https://api.telegram.org/bottoken/sendMessage (cert_verify=True)
2024-07-30 16:55:22,658 - DEBUG - Telegram Payload: {'disable_notification': False, 'disable_web_page_preview': True, 'parse_mode': 'MarkdownV2', 'text': 'Github PR Merged\r\n\n[chore: added some stuff](https://httpbin.org)\n\nRepository: xxxxx\n\nMerged onto: dev', 'chat_id': -123123123}
2024-07-30 16:55:23,973 - INFO - Sent Telegram notification.

when I include the repository line with URL, it shows this error instead

2024-07-30 16:57:56,750 - DEBUG - Notification Plugin 108(s) and 154 Schema(s) loaded in 0.2289s
2024-07-30 16:57:56,752 - DEBUG - Loaded Telegram URL: tgram://7...Q/-123123123/?image=False&detect=no&silent=no&preview=no&content=before&mdv=v2&format=markdown&overflow=upstream&rto=4.0&cto=4.0&verify=yes
2024-07-30 16:57:56,752 - DEBUG - Telegram POST URL: https://api.telegram.org/bottoken/sendMessage (cert_verify=True)
2024-07-30 16:57:56,752 - DEBUG - Telegram Payload: {'disable_notification': False, 'disable_web_page_preview': True, 'parse_mode': 'MarkdownV2', 'text': 'Github PR Merged\r\n\n[chore: added some stuff](https://httpbin.org)\n\nRepository: https://httpbin.org\n\nMerged onto: dev', 'chat_id': -123123123}
2024-07-30 16:57:57,529 - WARNING - Failed to send Telegram notification to -123123123: Bad Request: can't parse entities: Character '.' is reserved and must be escaped with the preceding '\', error=400.
2024-07-30 16:57:57,529 - DEBUG - Response Details:
b'{"ok":false,"error_code":400,"description":"Bad Request: can\'t parse entities: Character \'.\' is reserved and must be escaped with the preceding \'\\\\\'"}'

below is my webhook + apprise setup

FROM caronc/apprise:1.1

RUN curl -LO https://github.com/adnanh/webhook/releases/download/2.8.1/webhook-linux-amd64.tar.gz &&\
  tar -xzf webhook-linux-amd64.tar.gz &&\
  mv webhook-linux-amd64/webhook /usr/local/bin/ &&\
  rm -rf webhook-linux-amd64.tar.gz

CMD ["/usr/local/bin/webhook", "-hooks", "/etc/webhook/hooks.json", "-verbose"]

services:
  webhook_apprise:
    build: .
    ports:
      - 9000:9000
    volumes:
      - ./hooks.json:/etc/webhook/hooks.json:ro,Z
      - ./github-pr-merged-then-telegram-notify.sh:/github-pr-merged-then-telegram-notify.sh:ro,Z
    environment:
      TELEGRAM_BOT_TOKEN: chatbottoken

the github-pr-merged-then-telegram-notify.sh

#!/bin/bash

set -e

TELEGRAM_CHAT_ID=${1}

GITHUB_REPO=${2}
GITHUB_PR=${3}
GITHUB_PR_TO=${4}
GITHUB_PR_URL=${5}

MESSAGE_BODY=$(
  cat <<EOF

[${GITHUB_PR}]($GITHUB_PR_URL)

Repository: ${GITHUB_REPO}

Merged onto: ${GITHUB_PR_TO}
EOF
)

# this will error
apprise -vvv -t 'Github PR Merged' -b "$MESSAGE_BODY" "tgram://$TELEGRAM_BOT_TOKEN/$TELEGRAM_CHAT_ID?format=markdown"

# this works only if GITHUB_REPO is not a URL
# apprise -vvv -i markdown -t 'Github PR Merged' -b "$MESSAGE_BODY" "tgram://$TELEGRAM_BOT_TOKEN/$TELEGRAM_CHAT_ID?format=markdown"

the hooks.json

[
  {
    "id": "github-pr-merged-repo-x",
    "execute-command": "/github-pr-merged-then-telegram-notify.sh",
    "pass-arguments-to-command": [
      {
        "source": "string",
        "name": "-123123123"
      },
      {
        "source": "payload",
        "name": "repository"
      },
      {
        "source": "payload",
        "name": "pull_request.title"
      },
      {
        "source": "payload",
        "name": "pull_request.base.ref"
      },
      {
        "source": "payload",
        "name": "pull_request.url"
      }
    ],
    "trigger-rule": {
      "and": [
        {
          "match": {
            "type": "value",
            "value": "df8a9bb9-adc2-47f5-80cb-0274bc5658b1",
            "parameter": {
              "source": "header",
              "name": "x-api-key"
            }
          }
        },
        {
          "match": {
            "type": "value",
            "value": "true",
            "parameter": {
              "source": "payload",
              "name": "pull_request.merged"
            }
          }
        }
      ]
    }
  }
]

testing and simulate the github webhook

curl -X POST localhost:9000/hooks/github-pr-merged-repo-x -H 'x-api-key: df8a9bb9-adc2-47f5-80cb-0274bc5658b1' -H 'content-type: application/json' -d '{"pull_request":{"merged":true,"title":"chore: added some stuff","url":"https://httpbin.org","base":{"ref":"dev"}},"repository":"https://httpbin.org"}'

[caronc]

From a glance, it doesn't seem that you're doing anything wrong. Apprise appears to be passing a markdown payload I'd have expected. It seems weird that it's being rejected on characters that simply make sense (and we wouldn't want to escape).

What happens if you set the markdown version to v1?

[xshadowlegendx]

hello @caronc, so I just tested with mdv=v1 and it works!!! both with and without -i markdown, but without -i markdown the message sent with # sign I think it supposed to rendered as bold and big but it did not?

[caronc]

Seems strange that Telegram would have an issue with # in their new v2 markdown support. We both know it's a perfectly valid character to trigger a h1 heading. I'll have to check out their API documentation this weekend. Maybe they don't support heading now and only the **bold wrappers**.

Either way, the bug doesn't appear to be on the Apprise side, but maybe i can try to make Apprise more accommodating to bad characters we learn about like the one you're sharing here.

[caronc]

I switched the default markdown to v1 so others don't have this issue... a period isn't uncommon to be in a string, it shouldn't fial as much as it is for you or anyone. MarkdownV2 is a really custom version... very proprietary... it came from an earlier request, but the version 1 is definitley much better.

Not only with the default version of the Telegram Apprise Plugin switch back to 'v1, I also add/handle the extra character escaping IF you're not providing a body_format(source) ofmarkdownorNone` (not specified). Hopefull this makes sense.

Under all other circumstances, the other characters will be escaped.