Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NET-10: Refactor Peer update Phase-1 #407

Merged
merged 14 commits into from
May 23, 2023
Merged

NET-10: Refactor Peer update Phase-1 #407

merged 14 commits into from
May 23, 2023

Conversation

abhishek9686
Copy link
Member

@abhishek9686 abhishek9686 commented May 18, 2023

Describe your changes

-> updates would involve only single peer when required

Provide Issue ticket number if applicable/not in title

Provide link to Netmaker PR if required

gravitl/netmaker#2303

Provide testing steps

  • Joining and leaving a network
  • Update Acls
  • connect/disconnect node from netclient
  • Add/remove the host from a network via UI
    For All the Above steps, please verify whether peers on the interface are updated correctly

Checklist before requesting a review

  • My changes affect only 10 files or less.
  • I have performed a self-review of my code and tested it.
  • If it is a new feature, I have added thorough tests, my code is <= 1450 lines.
  • If it is a bugfix, my code is <= 200 lines.
  • My functions are <= 80 lines.
  • I have had my code reviewed by a peer.
  • My unit tests pass locally.
  • Netclient & Netmaker are awesome.

Copy link
Contributor

@theguy951357 theguy951357 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pr steps pass
critical list skipped untill all stages are complete.

@afeiszli afeiszli merged commit 627b924 into refactor_peer_updates May 23, 2023
mattkasun added a commit that referenced this pull request May 24, 2023
abhishek9686 added a commit that referenced this pull request May 31, 2023
* NET-10: Refactor Peer update Phase-1 (#407)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* prevent server overwriting immutable host parameters

* Revert "NET-10: Refactor Peer update Phase-1 (#407)"

This reverts commit 627b924.

---------

Co-authored-by: Abhishek K <[email protected]>
afeiszli added a commit that referenced this pull request Jun 1, 2023
* Bump golang.org/x/term from 0.7.0 to 0.8.0

Bumps [golang.org/x/term](https://github.com/golang/term) from 0.7.0 to 0.8.0.
- [Commits](golang/term@v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* workflow refactor

* limit gui builds

* txeh updates

* Bump github.com/guumaster/hostctl from 1.1.3 to 1.1.4

Bumps [github.com/guumaster/hostctl](https://github.com/guumaster/hostctl) from 1.1.3 to 1.1.4.
- [Release notes](https://github.com/guumaster/hostctl/releases)
- [Changelog](https://github.com/guumaster/hostctl/blob/master/.goreleaser.yml)
- [Commits](guumaster/hostctl@v1.1.3...v1.1.4)

---
updated-dependencies:
- dependency-name: github.com/guumaster/hostctl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump golang.org/x/net from 0.9.0 to 0.10.0 (#402)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.9.0 to 0.10.0.
- [Commits](golang/net@v0.9.0...v0.10.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fixed data endcoding (#400)

* fixed data endcoding

* remove redundant casting

* Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#403)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.8.0 to 0.9.0.
- [Commits](golang/crypto@v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* NET-89:  TURN reactive Implementation (#398)

* initialize turn client always, irrespective of nat type

* run background thread to resolve peer connection using turn

* update go mod

* add read lock for turncfg

* use conn negotitation action at required locations

* use lasthandshake for connection status

* use a separate wait group for proxy threads

* comments fix

* typo fix

* add additional logging for errors

* update go mod

* update action var name

* update go mod

* use relevant func name

* pr comments

* Bump github.com/wailsapp/wails/v2 from 2.2.0 to 2.5.0 (#404)

Bumps [github.com/wailsapp/wails/v2](https://github.com/wailsapp/wails) from 2.2.0 to 2.5.0.
- [Release notes](https://github.com/wailsapp/wails/releases)
- [Commits](wailsapp/wails@v2.2.0...v2.5.0)

---
updated-dependencies:
- dependency-name: github.com/wailsapp/wails/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump alpine from 3.17.3 to 3.18.0 (#401)

Bumps alpine from 3.17.3 to 3.18.0.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* update versions (#406)

* Bump github.com/wailsapp/wails/v2 from 2.5.0 to 2.5.1 (#414)

Bumps [github.com/wailsapp/wails/v2](https://github.com/wailsapp/wails) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/wailsapp/wails/releases)
- [Commits](wailsapp/wails@v2.5.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/wailsapp/wails/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* discord message after branch test run (#419)

* discord message after branch test run

* add repo to discord message

* ripping out netmaker.conf (#415)

* add ip6tabels support to docker netclient (#421)

* prevent updates to host params that should not be changed (#417)

* NET-10: Refactor Peer update Phase-1 (#407)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* prevent server overwriting immutable host parameters

* Revert "NET-10: Refactor Peer update Phase-1 (#407)"

This reverts commit 627b924.

---------

Co-authored-by: Abhishek K <[email protected]>

* NET-186: hole punch wg port, start proxy on free port available (#420)

* report wg public listen port

* save the proxy port

* debug log

* checkin with servers on broker connection

* start proxy on freeport

* revert proxy autoswitch removal

* use natinfo for proxy port

* pr comments

* Net 202  (#422)

* join endpoint

* sso endpoint for gui

* review comments

* sso registration with gui (#389)

* Revert "fix(GRA-1387): remove join by sso/basic auth"

This reverts commit ec8630d.

* feat(NET-20): add join by basic auth for nc-gui

* fix(NET-20): wip - join by oauth/basic auth

* join endpoint

* sso endpoint for gui

* review comments

* fix(NET-20): wip - join by oauth/basic auth

* fix(NET-20):  join by oauth/basic auth

* Revert "join endpoint"

This reverts commit 5f2351f.

* fix(NET-20): only join via enrollment key option

---------

Co-authored-by: Matthew R Kasun <[email protected]>

* version and release note updates (#423)

* Net 12 (#427)

* discord message after branch test run

* add repo to discord message

* .goreleaser update

* update go mod for release

* Update go.mod

* fix go mod

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthew R Kasun <[email protected]>
Co-authored-by: Alex Feiszli <[email protected]>
Co-authored-by: Abhishek K <[email protected]>
Co-authored-by: Aceix <[email protected]>
Co-authored-by: Gravitl <[email protected]>
afeiszli added a commit that referenced this pull request Jun 7, 2023
* Bump golang.org/x/term from 0.7.0 to 0.8.0

Bumps [golang.org/x/term](https://github.com/golang/term) from 0.7.0 to 0.8.0.
- [Commits](golang/term@v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* workflow refactor

* limit gui builds

* txeh updates

* Bump github.com/guumaster/hostctl from 1.1.3 to 1.1.4

Bumps [github.com/guumaster/hostctl](https://github.com/guumaster/hostctl) from 1.1.3 to 1.1.4.
- [Release notes](https://github.com/guumaster/hostctl/releases)
- [Changelog](https://github.com/guumaster/hostctl/blob/master/.goreleaser.yml)
- [Commits](guumaster/hostctl@v1.1.3...v1.1.4)

---
updated-dependencies:
- dependency-name: github.com/guumaster/hostctl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump golang.org/x/net from 0.9.0 to 0.10.0 (#402)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.9.0 to 0.10.0.
- [Commits](golang/net@v0.9.0...v0.10.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fixed data endcoding (#400)

* fixed data endcoding

* remove redundant casting

* Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#403)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.8.0 to 0.9.0.
- [Commits](golang/crypto@v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* NET-89:  TURN reactive Implementation (#398)

* initialize turn client always, irrespective of nat type

* run background thread to resolve peer connection using turn

* update go mod

* add read lock for turncfg

* use conn negotitation action at required locations

* use lasthandshake for connection status

* use a separate wait group for proxy threads

* comments fix

* typo fix

* add additional logging for errors

* update go mod

* update action var name

* update go mod

* use relevant func name

* pr comments

* Bump github.com/wailsapp/wails/v2 from 2.2.0 to 2.5.0 (#404)

Bumps [github.com/wailsapp/wails/v2](https://github.com/wailsapp/wails) from 2.2.0 to 2.5.0.
- [Release notes](https://github.com/wailsapp/wails/releases)
- [Commits](wailsapp/wails@v2.2.0...v2.5.0)

---
updated-dependencies:
- dependency-name: github.com/wailsapp/wails/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump alpine from 3.17.3 to 3.18.0 (#401)

Bumps alpine from 3.17.3 to 3.18.0.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* update versions (#406)

* Bump github.com/wailsapp/wails/v2 from 2.5.0 to 2.5.1 (#414)

Bumps [github.com/wailsapp/wails/v2](https://github.com/wailsapp/wails) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/wailsapp/wails/releases)
- [Commits](wailsapp/wails@v2.5.0...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/wailsapp/wails/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* discord message after branch test run (#419)

* discord message after branch test run

* add repo to discord message

* ripping out netmaker.conf (#415)

* add ip6tabels support to docker netclient (#421)

* prevent updates to host params that should not be changed (#417)

* NET-10: Refactor Peer update Phase-1 (#407)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* prevent server overwriting immutable host parameters

* Revert "NET-10: Refactor Peer update Phase-1 (#407)"

This reverts commit 627b924.

---------

Co-authored-by: Abhishek K <[email protected]>

* NET-186: hole punch wg port, start proxy on free port available (#420)

* report wg public listen port

* save the proxy port

* debug log

* checkin with servers on broker connection

* start proxy on freeport

* revert proxy autoswitch removal

* use natinfo for proxy port

* pr comments

* Net 202  (#422)

* join endpoint

* sso endpoint for gui

* review comments

* sso registration with gui (#389)

* Revert "fix(GRA-1387): remove join by sso/basic auth"

This reverts commit ec8630d.

* feat(NET-20): add join by basic auth for nc-gui

* fix(NET-20): wip - join by oauth/basic auth

* join endpoint

* sso endpoint for gui

* review comments

* fix(NET-20): wip - join by oauth/basic auth

* fix(NET-20):  join by oauth/basic auth

* Revert "join endpoint"

This reverts commit 5f2351f.

* fix(NET-20): only join via enrollment key option

---------

Co-authored-by: Matthew R Kasun <[email protected]>

* version and release note updates (#423)

* Net 12 (#427)

* discord message after branch test run

* add repo to discord message

* .goreleaser update

* NET-185 Dont re-use the Config struct (#432)

* - dont re-use the Config struct- gitignore for idea

* test for yaml config merging

* NET-287: get host public IP using stun (#431)

* get host public IP using stun

* check if ip is not specified

* check if ip is not specified

* update go mod for release

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthew R Kasun <[email protected]>
Co-authored-by: Alex Feiszli <[email protected]>
Co-authored-by: Abhishek K <[email protected]>
Co-authored-by: Aceix <[email protected]>
Co-authored-by: Tobias Cudnik <[email protected]>
Co-authored-by: Gravitl <[email protected]>
abhishek9686 added a commit that referenced this pull request Jul 26, 2023
* NET-10: Refactor Peer update Phase-1 (#407)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* NET-193: Refactor Peer Updates - Firewall Manager (#418)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* detach firewall from proxy and peer updates

* rm unused funcs

* close firewall in daemon shutdown

* initialize firewall on start of the daemon

* rm config related to fw from proxy

* add netmaker forward rule on firewall init

* return if firewall is not initialized

* sync firewall rule on pull

* rm commented code

* update netclient dockerfile to support ip6tables

* rename pkg to firewall

* remove relay implementation from proxy

* remove relay implementation from proxy (#425)

* remove unwanted relay check

* remove relay config

* remove proxy server components

* remove proxy utilities

* rm unused proxy config

* remove multi-server support for turn

* remove NAT Auto Switch

* remove multi-server support for turn

* update go mod

* remove servers loop

* fix issues with merge conflicts

* NET-282: removed proxy-to-proxy comms, removed proxy udp server (#447)

* remove relay implementation from proxy

* remove unwanted relay check

* remove relay config

* remove proxy server components

* remove proxy utilities

* rm unused proxy config

* remove multi-server support for turn

* remove NAT Auto Switch

* remove multi-server support for turn

* update go mod

* remove servers loop

* fix issues with merge conflicts

* Net 282 proxy (#450)

* remove proxy manager

* use nat info for public IP

* remove server param

* rm ununsed proxy config

* statr proxy only if client is registered to a server

* rm unused func

* update go mod, remove usage of proxy listen port

* remove server map, network settings

* use tcp pinger to determine peer connection status and latency

* remove nat info usage

* nil pointer error fix,remove unused fields

* use icmp pinger for extclient

* NET-282: removed proxy dependency on peer updates, only use proxy for turn operations (#453)

* remove relay implementation from proxy

* remove unwanted relay check

* remove relay config

* remove proxy server components

* remove proxy utilities

* rm unused proxy config

* remove multi-server support for turn

* remove NAT Auto Switch

* remove multi-server support for turn

* update go mod

* remove servers loop

* fix issues with merge conflicts

* Net 282 proxy (#450)

* remove proxy manager

* use nat info for public IP

* remove server param

* rm ununsed proxy config

* statr proxy only if client is registered to a server

* rm unused func

* update go mod, remove usage of proxy listen port

* remove server map, network settings

* remove nat info usage

* nil pointer error fix,remove unused fields

* update go mod

* remove proxy functionalities

* fix errors

* rm unused field

* remove old changes

* remove peer conf struct usage

* simply nat detection,remove unused funcs

* rm unused func

* send host ack

* remove duplicate pull

* revert firewall changes

* fix linux build

* remove turn debug change

* return if remote conn fails
abhishek9686 added a commit that referenced this pull request Jul 28, 2023
* NET-10: Refactor Peer update Phase-1 (#407)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* NET-193: Refactor Peer Updates - Firewall Manager (#418)

* singleton peer update mq handler

* process adding and removing single peer

* avoid restarting daemon on join to network host update

* handle peer list on singleton update

* support multiple peer removal

* use map for peer data

* init peer map

* don't retrun if wg file update fails

* add debug comment

* add debug comment

* add func comments

* add comment

* rm ununsed funcs

* detach firewall from proxy and peer updates

* rm unused funcs

* close firewall in daemon shutdown

* initialize firewall on start of the daemon

* rm config related to fw from proxy

* add netmaker forward rule on firewall init

* return if firewall is not initialized

* sync firewall rule on pull

* rm commented code

* update netclient dockerfile to support ip6tables

* rename pkg to firewall

* remove relay implementation from proxy

* remove relay implementation from proxy (#425)

* remove unwanted relay check

* remove relay config

* remove proxy server components

* remove proxy utilities

* rm unused proxy config

* remove multi-server support for turn

* remove NAT Auto Switch

* remove multi-server support for turn

* update go mod

* remove servers loop

* fix issues with merge conflicts

* NET-282: removed proxy-to-proxy comms, removed proxy udp server (#447)

* remove relay implementation from proxy

* remove unwanted relay check

* remove relay config

* remove proxy server components

* remove proxy utilities

* rm unused proxy config

* remove multi-server support for turn

* remove NAT Auto Switch

* remove multi-server support for turn

* update go mod

* remove servers loop

* fix issues with merge conflicts

* Net 282 proxy (#450)

* remove proxy manager

* use nat info for public IP

* remove server param

* rm ununsed proxy config

* statr proxy only if client is registered to a server

* rm unused func

* update go mod, remove usage of proxy listen port

* remove server map, network settings

* use tcp pinger to determine peer connection status and latency

* remove nat info usage

* nil pointer error fix,remove unused fields

* use icmp pinger for extclient

* NET-282: removed proxy dependency on peer updates, only use proxy for turn operations (#453)

* remove relay implementation from proxy

* remove unwanted relay check

* remove relay config

* remove proxy server components

* remove proxy utilities

* rm unused proxy config

* remove multi-server support for turn

* remove NAT Auto Switch

* remove multi-server support for turn

* update go mod

* remove servers loop

* fix issues with merge conflicts

* Net 282 proxy (#450)

* remove proxy manager

* use nat info for public IP

* remove server param

* rm ununsed proxy config

* statr proxy only if client is registered to a server

* rm unused func

* update go mod, remove usage of proxy listen port

* remove server map, network settings

* remove nat info usage

* nil pointer error fix,remove unused fields

* update go mod

* remove proxy functionalities

* fix errors

* rm unused field

* remove old changes

* remove peer conf struct usage

* simply nat detection,remove unused funcs

* rm unused func

* send host ack

* remove duplicate pull

* revert firewall changes

* fix linux build

* remove turn debug change

* return if remote conn fails

* stop setting ingress routes

* add only NAT rules for egress

* rm ununsed funcs

* refresh egress range routes

* log for refresh routes

* continue on first time setup

* remove egress map

* rm unused func

* forward rules changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants