Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MagentaTV IGMP Proxy #2

Closed
TheJulianJES opened this issue Apr 13, 2021 · 36 comments
Closed

MagentaTV IGMP Proxy #2

TheJulianJES opened this issue Apr 13, 2021 · 36 comments
Labels
kind/docs Improvements or additions to documentation

Comments

@TheJulianJES
Copy link

MagentaTV by Telekom comes in on VLAN 7 where the normal internet traffic also arrives. There is no longer two separated VLANs.
I guess I can skip the step "Creating VLAN and obtaining IP address" in https://github.com/fabianishere/udm-kernel-tools/blob/master/docs/iptv.md#creating-vlan-and-obtaining-ip-address?
This is the configuration that I'm currently using on my USG 3P successfully (Note: I'm doing the VLAN tagging (7) on my DrayTek Vigor 165 modem but it can also be done on the USG):

(The prefix-only thing is only there to prevent a high cpu usage on the USGs with IPv6 -- hopefully this is fixed on the UDM-Pro)

{
    "interfaces": {
        "ethernet": {
            "eth0": {
                "pppoe": {
                    "0": {
                        "dhcpv6-pd": {
                            "prefix-only": "''"
                        }
                    }
                }
            }
        }
    },
    "protocols": {
        "igmp-proxy": {
            "interface": {
                "eth1": {
                    "alt-subnet": [
                        "0.0.0.0/0"
                    ],
                    "role": "downstream",
                    "threshold": "1",
                    "whitelist": [
                        "239.35.0.0/16",
                        "232.0.0.0/16"
                    ]
                },
                "pppoe0": {
                    "alt-subnet": [
                        "0.0.0.0/0"
                    ],
                    "role": "upstream",
                    "threshold": "1"
                }
            }
        }
    }
}

Could /etc/igmpproxy.conf look something like this?

quickleave

##------------------------------------------------------
## Configuration for eth0 (Upstream Interface)
##------------------------------------------------------
phyint eth8 upstream  ratelimit 0  threshold 1
    altnet 192.168.0.0/16 # LAN
    altnet 239.35.0.0/16 # Multicast addresses used by MagentaTV
    altnet 232.0.0.0/16

##------------------------------------------------------
## Configuration for eth1 (Downstream Interface)
##------------------------------------------------------
phyint br0 downstream  ratelimit 0  threshold 1

# Disable other interfaces
phyint ppp0 disabled
phyint tun1 disabled

I'm getting my UDM-Pro in a couple of days (would have never bought one without this project and I'm not sure if it's good or bad that I bought one lol), so I can try if this works and I'll update this question if and how I got it to work.
(Again, thanks so much for doing this!!)

There's also some information in the following articles:
(although the article is in German, the addresses used by MagentaTV are specified): https://www.heise.de/ct/artikel/MagentaTV-auf-pfSense-Co-4698826.html
(Attention: this is the "old way" -- no longer done like this afaik): https://www.projectiwear.org/~plasmahh/t_home.html
(This also seems to be the "old way" with two separate VLANs since it's mentioned that it's a "non BNG" connection): https://telekomhilft.telekom.de/t5/Fernsehen/Brauche-Hilfe-fuer-Einrichtung-von-igmpproxy-auf-Linux-Router/td-p/3446240

@fabianishere
Copy link
Owner

Not having the IPTV on a separate VLAN simplifies the setup a whole bit. Basically, now you only need to configure igmpproxy and possibly NAT the address ranges from your ISP.

I expect the configuration of igmpproxy to work with one change. The upstream interface needs to be the VLAN7 interface of eth8, so the following:

quickleave

##------------------------------------------------------
## Configuration for eth0 (Upstream Interface)
##------------------------------------------------------
phyint eth8.7 upstream  ratelimit 0  threshold 1
    altnet 192.168.0.0/16 # LAN
    altnet 239.35.0.0/16 # Multicast addresses used by MagentaTV
    altnet 232.0.0.0/16

##------------------------------------------------------
## Configuration for eth1 (Downstream Interface)
##------------------------------------------------------
phyint br0 downstream  ratelimit 0  threshold 1

If IPTV does not work with only igmpproxy, you might need to NAT the address ranges of the ISP:

iptables -t nat -A POSTROUTING -d 239.35.0.0/16  -j MASQUERADE -o eth8.7
iptables -t nat -A POSTROUTING -d 232.0.0.0/16 -j MASQUERADE -o eth8.7

@TheJulianJES
Copy link
Author

Nice, thanks so much! I will report back in a couple of days if this works.

@TheJulianJES
Copy link
Author

TheJulianJES commented Apr 15, 2021

Update:
I'm still testing but the following configuration already seems to work:

quickleave

##------------------------------------------------------
## Configuration for upstream interface
##------------------------------------------------------
phyint ppp0 upstream  ratelimit 0  threshold 1
    altnet 192.168.0.0/16 # LAN
    altnet 239.35.0.0/16 # Multicast addresses used by MagentaTV
    altnet 232.0.0.0/16
    altnet 87.141.0.0/16

##------------------------------------------------------
## Configuration for downstream interface
##------------------------------------------------------
phyint br0 downstream  ratelimit 0  threshold 1

Notes:
altnet 239.35.0.0/16 is probably not needed anymore for MagentaTV.
I'm also doing VLAN-tagging on my modem directly. That's why I'm using ppp0 instead of eth8.7
I also didn't need to add any rules using iptables as that can be done (at least with MagentaTV) using the UniFi Controller. I'm currently still testing if these commands are needed: (Edit: looks like these two commands are needed -- not one with 87.141.0.0/16 though)

iptables -t nat -A POSTROUTING -d 239.35.0.0/16 -j MASQUERADE -o ppp0
Edit: looks like only this command is needed:

iptables -t nat -A POSTROUTING -d 232.0.0.0/16 -j MASQUERADE -o ppp0

Edit: Firewall rule also needed:
Internet In (maybe same also for Internet Local) -> Source: 87.141.215.251 + Destination (Group): 224.0.0.0/4 / 232.0.0.0/16 Port: 10000

(Thank you so much for creating this project and all the guides!!)

@TheJulianJES
Copy link
Author

I guess I can put the iptables command in the "on boot script" (if I can't get it to using the firewall rules).
I was just wondering, why the IPTV guide always mentions "on_data.d" instead of "on_boot.d" (https://github.com/fabianishere/udm-kernel-tools/blob/master/docs/iptv.md#creating-vlan-and-obtaining-ip-address)? Is that something separate from the on boot utility?

@fabianishere
Copy link
Owner

Nope, that’s a typo 😅

@J4s0nD4rk
Copy link

Thanks for summarizing the info for Magenta TV in this issue. Currently it seems to work with a receiver MR601 without stuttering.

I installed the edge2 kernel and igmpproxy as described and also had to create the Internet In rules mentioned above.
The rules for iptables didn't seem to be necessary, possibly because I set VLAN 7 in the UDM Pro for the Telekom network (WAN) and use a separate VLAN for the receivers.
As already written, the IP range 232.0.0.0/16 is for Magenta TV receivers of the second generation and thus the IP range 239.35.0.0/16 was not applicable.

There is still a problem after restarting the UDM Pro, as igmpproxy seems to be started too early and exits again immediately. A device does not seem to be available at the time.

@TheJulianJES
Copy link
Author

There is still a problem after restarting the UDM Pro, as igmpproxy seems to be started too early and exits again immediately. A device does not seem to be available at the time.

I can try again later but I can't recall having any problems with a restart. The UDMP restarts twice (when auto reboot into the custom kernel is enabled) (first without the custom kernel and then into the custom kernel). igmpproxy can only start with the custom kernel I guess?
This might not be related to your problem. At a later time, I can try to restart my UDMP again.

@J4s0nD4rk
Copy link

Also the magenta tv receiver MR201 as the second unit operates without stutter. However, the connection is broken on both devices after about 3-10 minutes and apparently has to be re-established. In udm traffic monitor you can see that the throughput goes down.

Under opnsense it was necessary to provide the firewall rules to allow options, is it possible that there is still something missing with the ip4 headers?

I think the reboot problem is related to my use of ppp0 as an upstream gateway. But igmpproxy can be restarted manually as workaround.

@TheJulianJES
Copy link
Author

I think I also experienced the same issue with a disconnect after 3 - 10 minutes. I just re-enabled all of the old firewall rules (coming from my USG): https://www.bjoerns-techblog.de/2017/06/deutsche-telekom-entertain-mit-dem-ubiquiti-unifi-security-gateway/

That seems to have fixed it (at least for me on 401/201(?))

@J4s0nD4rk
Copy link

I created the firewall rules, but the behavior has not changed.
Since I use Unifi switches in the network, I would probably have to set switch.igmp.header_checking = true. But unfortunately the old way of creating the configuration config.properties on the UDM Pro no longer works.
https://www.bjoerns-techblog.de/2017/05/ubiquiti-unifi-switch-us-8-60w-der-geht-auch-mit-entertain/

However, by another problem I can not reprovision the UDM Pro. Therefore, it will probably come down to a Factory Reset.

There was a problem applying the configuration on udmpro. Please try again or contact Ubiquiti Support if the problem persists

@TheJulianJES
Copy link
Author

IGMP Header Validation was disabled in some 5.x firmware I think. Try running the latest RC (5.43.36) on the switches and turn on IGMP Snooping.

@fabianishere fabianishere added the kind/docs Improvements or additions to documentation label May 2, 2021
@fabianishere
Copy link
Owner

Closing this issue now. If you find any issues again, I can re-open the issue.

@nicx
Copy link

nicx commented May 11, 2021

running this now for over a week with my udm pro and a 401 receiver without any stuttering and problems! thanks a lot to you guys for this solution. Now I can finally switch completely to the udm pro. 👍

My igmpproxy configuration running VDSL with an Draytek Vigor 165 and VLAN7 configured on the router:

##------------------------------------------------------
## Enable Quickleave mode (Sends Leave instantly)
##------------------------------------------------------
quickleave

phyint ppp0 upstream  ratelimit 0  threshold 1
    altnet 192.168.0.0/24 # LAN
    altnet 232.0.0.0/16
    altnet 87.141.0.0/16

phyint br0 downstream  ratelimit 0  threshold 1

In addition needed:

Open question I will check in a few days:

  • Is the upstream network "LAN" really needed in the igmpconfig even if the Receiver is located in another VLAN? --> Answer: YES :)
  • which firewall rules are obsolete from the USG-solution? -> NONE, so all rules are still needed.

@nicx
Copy link

nicx commented Jun 28, 2021

after a few more weeks its nearly working like a charm, but I noticed problems after a reboot /restart of the UDM Pro: The igmproxy service is started automatically, but my Media Receiver begins stuttering after a few seconds... I have to manually stop and restart the igmpproxy, then it works until to the next reboot.

@ALL any hints how to get this solved or which logfiles could help?
@TheJulianJES @J4s0nD4rk do you have the same problem?
@fabianishere shall I open a new issue or do you want to reopen this one?

@TheJulianJES
Copy link
Author

TheJulianJES commented Jun 28, 2021

Yeah. Same problem here. I need to restart it after a UDM-Pro reboot. Probably an issue regarding igmpproxy and not udm-kernel-tools though. (Maybe WAN isn't active yet?)

Perhaps it's also possible to have a delayed "restart" script for igmpproxy?

@nicx
Copy link

nicx commented Jun 30, 2021

any ideas for a solution?

@fabianishere
Copy link
Owner

fabianishere commented Jun 30, 2021

I haven't tested them myself, but I can think of two possible solutions:

  1. Edit the igmpproxy service and add a dependency on the UniFi service. Run systemctl edit igmpproxy and enter
    [Unit]
    Requires=unifi.service
    After=unifi.service
    
  2. Use the rescanvif option of igmpproxy (it is not documented anywhere, but from reading the source code, I gather it refreshes the interfaces every n seconds). Edit /etc/igmpproxy.conf to use this option:
    ...
    quickleave
    rescanvif
    

@nicx
Copy link

nicx commented Jun 30, 2021

thanks @fabianishere, I will test option 2 (because with this option everything regarding igmpproxy is configured in 1 place) and will give feedback.

@TheJulianJES
Copy link
Author

TheJulianJES commented Jul 1, 2021

Thanks for both options! The second option could potentially address #13(?)
The first option seems a bit "cleaner" to fix this problem. I'll probably try both.

I am unsure about one thing though. When the unifi service is stopped (UniFi Network Application/Controller), igmpproxy will also stop (if I'm reading this correctly).
Stopping and starting unifi.service can happen when updating the controller version. Will this also restart igmpproxy? Is it better to use Wants=unifi.service instead of Requires=unifi.service?
Or is it better to use Requires=unifi.service, After=unifi.service and PartOf=unifi.service? (Or WantedBy=unifi.service instead of PartOf, because PartOf only catches restarts (?) while WantedBy also catches stop/start?)

Also, I'm not sure if this is really an issue, but rescanvif seems to call rebuildIfVc() very often. Some people reported it to be resource extensive in one of their GitHub issue. (Just as a note in case the UDM-Pro runs hotter after setting this)

@nicx
Copy link

nicx commented Jul 1, 2021

thanks @fabianishere, I will test option 2 (because with this option everything regarding igmpproxy is configured in 1 place) and will give feedback.

ok seems that option 2 is not implemented:

root@ubnt:/# igmpproxy -vv -d /etc/igmpproxy.conf
Searching for config file at '/etc/igmpproxy.conf'
Config: Quick leave mode enabled.
Unknown token 'rescanvif' in configfile
Unable to load config file...
root@ubnt:/# 

I will try Option 1 and give feedback.

@nicx
Copy link

nicx commented Jul 1, 2021

A quick feedback regarding option 1: A first test was positive and the media receiver works without stuttering after a reboot of my UDM Pro. Looks good :) I will do a few more tests just to be sure.

And I think the questions of @TheJulianJES regarding the unifi.service dependency is absolutely reasonable, we need to clarify which option is the best.

@nicx
Copy link

nicx commented Jul 15, 2021

after a few more reboot-tests (including a live power outage) I would say option 1 is working without any problems. 👍
I am completely satisfied now :)

@thomas-baumeister
Copy link

thomas-baumeister commented Jul 24, 2021

thanks @fabianishere, I will test option 2 (because with this option everything regarding igmpproxy is configured in 1 place) and will give feedback.

ok seems that option 2 is not implemented:

root@ubnt:/# igmpproxy -vv -d /etc/igmpproxy.conf
Searching for config file at '/etc/igmpproxy.conf'
Config: Quick leave mode enabled.
Unknown token 'rescanvif' in configfile
Unable to load config file...
root@ubnt:/# 

I will try Option 1 and give feedback.

For anyone who wants to try the rescanvif option ...
This option seems to be available with igmpproxy 0.2.1. Since the Debian Stretch repository only has v0.1 you need to install it by hand. The package from Buster works for me.

@fabianishere
Copy link
Owner

fabianishere commented Jul 26, 2021

I have build an updated setup where the IPTV configuration happens entirely in a separate container. I haven't fully tested this approach yet, but it might be worth a try, since it drastically reduces the overall setup necessary (and does not lose the IGMP Proxy configuration when the UniFi container is reset).

The following script should replace most of the current guide:

# Run inside UbiOS
tee  /mnt/data/on_boot.d/15-iptv.sh <<EOF >/dev/null
if podman container exists iptv; then
  podman rm -f iptv
fi
podman run --network=host --privileged \
    --name iptv --rm -i -d \
    -v /run:/var/run/ -v /run:/run \
    -e IPTV_WAN_INTERFACE="eth8" \
    -e IPTV_WAN_RANGES="213.75.0.0/16 217.166.0.0/16" \
    -e IPTV_LAN_INTERFACES="br0" \
    fabianishere/udm-iptv:1.0

# NAT the IP-ranges to IPTV network
iptables -t nat -A POSTROUTING -d 213.75.112.0/21 -j MASQUERADE -o iptv
iptables -t nat -A POSTROUTING -d 217.166.0.0/16 -j MASQUERADE -o iptv
EOF
chmod +x /mnt/data/on_boot.d/15-iptv.sh

For the full description, see https://github.com/fabianishere/udm-kernel-tools/blob/container-setup/docs/iptv.md

@nicx
Copy link

nicx commented Jul 27, 2021

@fabianishere just tried your new approach (which is really great and clean) but with no luck. When executing the on_boot script the itv container is downloaded and started, but at the same time my internet connection drops.

this is my script (using DSL via ppp0 and br10 as my IOT Network where my IPTV devices are located) :

if podman container exists iptv; then
  podman rm -f iptv
fi

podman run --network=host --privileged \
    --name iptv --rm -i -d \
    -v /run:/var/run/ -v /run:/run \
    -e IPTV_WAN_INTERFACE="ppp0" \
    -e IPTV_WAN_RANGES="232.0.0.0/16 87.141.0.0/16" \
    -e IPTV_LAN_INTERFACES="br10" \
    fabianishere/udm-iptv:1.0
    
# NAT the IP-ranges to IPTV network
iptables -t nat -A POSTROUTING -d 232.0.0.0/16 -j MASQUERADE -o iptv
iptables -t nat -A POSTROUTING -d 87.141.0.0/16 -j MASQUERADE -o iptv

any idea how to get it running?

@fabianishere
Copy link
Owner

@nicx Currently, it tries to create a VLAN interface on ppp0 (for VLAN 4), but for your setup I guess that is not needed. Let me adjust the script to support that.

@nicx
Copy link

nicx commented Jul 27, 2021

@fabianishere great point. Any idea when you could do that? I am asking because if its not today then I have to implement the old solution again. I was just too fast here :)

@fabianishere
Copy link
Owner

@nicx Working on it at the moment.

@fabianishere
Copy link
Owner

@nicx Could you try the following:

if podman container exists iptv; then
  podman rm -f iptv
fi

podman run --network=host --privileged \
    --name iptv -i -d --restart always \
    -e IPTV_WAN_INTERFACE="ppp0" \
    -e IPTV_WAN_VLAN="0" \
    -e IPTV_WAN_RANGES="232.0.0.0/16 87.141.0.0/16" \
    -e IPTV_LAN_INTERFACES="br10" \
    fabianishere/udm-iptv:1.1 -d -v

@nicx
Copy link

nicx commented Jul 27, 2021

@fabianishere now its working but throws some warnings on start:

# /mnt/data/on_boot.d/15-start_container_iptv.sh 
c484ca5c9a0974ce0d49fcff10a6f401d6ea211c16c6c65e1ddfa69739401cf8
Trying to pull docker.io/fabianishere/udm-iptv:1.1...
Getting image source signatures
Copying blob 26b62bbe2487 done
Copying blob a5ff5152e9e1 done
Copying blob efef4ab222d8 done
Copying config 7c9405ef6d done
Writing manifest to image destination
Storing signatures
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
cf84b4083565560884d95f4f791b33394b7e34a65a30fae1c329a39e1c3a852a
# 
# podman container ls
CONTAINER ID  IMAGE                                 COMMAND               CREATED         STATUS             PORTS  NAMES
cf84b4083565  docker.io/fabianishere/udm-iptv:1.1   -d -v                 50 seconds ago  Up 49 seconds ago         iptv
424a8b479fe0  docker.io/adguard/adguardhome:latest  --no-check-update...  12 days ago     Up 5 hours ago            adguardhome
98d27ebcae40  localhost/unifi-os:current            /sbin/init            6 weeks ago     Up 5 hours ago            unifi-os
# 

how can I now debug inside the container? or run the command "systemctl status igmpproxy"?

@nicx
Copy link

nicx commented Jul 27, 2021

In addition here is the generated config, seems ok so far. The altnet 192.168.0.0/16is hard coded in the script, right? Is that the way it is meant to be? I am asking because my VLANs are subnetted like 192.168.0.0/24 192.168.1.0/24 192.168.2.0/24 etc.

# podman exec iptv cat /etc/igmpproxy.conf
quickleave
phyint ppp0 upstream  ratelimit 0  threshold 1
  altnet 192.168.0.0/16
  altnet 232.0.0.0/16
  altnet 87.141.0.0/16
phyint br0 disabled
phyint br0.mac disabled
phyint br10 downstream  ratelimit 0  threshold 1
phyint br20 disabled
phyint dummy0 disabled
phyint eth0 disabled
phyint eth1 disabled
phyint eth10 disabled
phyint eth10.10 disabled
phyint eth10.20 disabled
phyint eth2 disabled
phyint eth3 disabled
phyint eth4 disabled
phyint eth5 disabled
phyint eth6 disabled
phyint eth7 disabled
phyint eth8 disabled
phyint eth9 disabled
phyint gre0 disabled
phyint gretap0 disabled
phyint ifb0 disabled
phyint ifb1 disabled
phyint ip_vti0 disabled
phyint sit0 disabled
phyint switch0 disabled
phyint switch0.1 disabled
phyint switch0.10 disabled
phyint switch0.20 disabled
# 

@fabianishere
Copy link
Owner

To view the logs of igmpproxy, run the following command:

podman logs iptv

You can ignore the warnings you get while starting the container, they won't affect igmpproxy.

As for the altnets, that's currently hardcoded, but I could provide an option to also override this.

@nicx
Copy link

nicx commented Jul 28, 2021

@fabianishere after a few reboot tests I can say the new solution so far works rock solid for me :) Thanks a lot!

@phoenix-pnp
Copy link

Thanks @fabianishere and @TheJulianJES! Due to this issue discussion I managed to get Magenta TV working perfectly.

@Daniel1896HSV
Copy link

Hi,

first of all thank you for your work! I've come to a point where I can't go any further. I have nothing to do with IT myself and needed your help.
Can one of you help me with teamviewer or something similar?

I would be very thankful.

my setup

Vigor 160 - UDM Pro - PoE Switch - magenta receiver 401

@apeltzer
Copy link

apeltzer commented May 14, 2022

Just had quite a bit of time putting together all of the bits here, so figured I could jot things down and have all in a short howto:

Firewall Rules

https://www.bjoerns-techblog.de/2017/06/deutsche-telekom-entertain-mit-dem-ubiquiti-unifi-security-gateway/

Setup of stuff inside scripts

unifi-os shell
curl -L https://udm-boot.boostchicken.dev -o udm-boot_1.0.5_all.deb
dpkg -i udm-boot_1.0.5_all.deb
exit

Now create a startup script for running igmpproxy correctly:

vim /mnt/data/on_boot.d/15-iptv.sh
chmod +x /mnt/data/on_boot.d/15-iptv.sh

This needs to go into the container:

if podman container exists iptv; then
  podman rm -f iptv
fi

podman run --network=host --privileged \
    --name iptv -i -d --restart always \
    -e IPTV_WAN_INTERFACE="ppp0" \
    -e IPTV_WAN_VLAN="0" \
    -e IPTV_WAN_RANGES="232.0.0.0/16 87.141.0.0/16" \
    -e IPTV_WAN_DHCP_OPTIONS="-O staticroutes -V IPTV_RG" \
    -e IPTV_LAN_INTERFACES="br0" \
    fabianishere/udm-iptv:1.3 -d -v

After +- 5mins, still a short freeze. Will investigate what I have to do with this, likely some firewall rule still missing.

EDIT: Updated to use updm-iptv:1.3 and since then it seems to work stable without any further changes required 🤷🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/docs Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

8 participants