Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(protocol/activitypub): rebuild activitypub engine protocol and provider #600

Merged
merged 39 commits into from
Nov 14, 2024
Merged

feat(protocol/activitypub): rebuild activitypub engine protocol and provider #600

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
8d9a233
feat: re-implement the mastodon data-source section with mastodon rel…
FrankLi123 Oct 22, 2024
26f45dc
fix: modify followRelay logic with set relay url list
FrankLi123 Oct 22, 2024
42ec67c
fix: change port number
FrankLi123 Oct 22, 2024
bd8b5c5
refactor: declar global varible and fix minor typos
FrankLi123 Oct 22, 2024
f8530de
fix: use another request function to prevent hanging issue while maki…
FrankLi123 Oct 22, 2024
40017b7
feat: remove redundant code part and refactor code with simplified logic
FrankLi123 Oct 24, 2024
412bb3d
refactor: improve code format following effective-go guidelines
FrankLi123 Oct 24, 2024
af229ce
refactor: continue to revise code format and make structural adjustment
FrankLi123 Oct 25, 2024
b9fe03f
feat: add detailed comments to guide users on mastodon configuration
FrankLi123 Oct 27, 2024
8f310b2
fix: solve some typos and add more comments
FrankLi123 Oct 27, 2024
3173610
feat: add new mastodon monitor client and its test
FrankLi123 Oct 28, 2024
9dde4f9
refactor: change code structure for simpler client setting; add provi…
FrankLi123 Oct 28, 2024
8736020
feat: modify config test, config file and openapi doc with new mastod…
FrankLi123 Oct 28, 2024
7bc7b03
refactor: revise dependency versions and update with latest changes f…
FrankLi123 Oct 28, 2024
cdf0112
refactor: comment out mastodon client test and only enable local test…
FrankLi123 Oct 28, 2024
9b4f204
fix: solve lint typos
FrankLi123 Oct 28, 2024
9ef371b
fix: resolve deadlock issue to enable DB storage
FrankLi123 Oct 28, 2024
1bdd672
fix: resolve PR issues and improve code
FrankLi123 Oct 29, 2024
33307d5
feat: add port as a new parameter
FrankLi123 Oct 29, 2024
af05662
fix: clear the config_test issue by removing port param
FrankLi123 Oct 29, 2024
b93cd03
refactor: comment out the mastodon monitor test cases
FrankLi123 Oct 29, 2024
2e21b66
feat: add default relay url list
FrankLi123 Oct 29, 2024
707248a
fix: remove the use of the channel for signaling that the http server…
FrankLi123 Oct 29, 2024
971cc0c
feat: add more defauly relay URLs
FrankLi123 Oct 29, 2024
3f1345c
fix: remove the relay URL List for monitor client to resolve request …
FrankLi123 Oct 29, 2024
6970f4c
feat: rewrite mastodon monitor client
FrankLi123 Oct 30, 2024
c9dc6b3
feat: add retry mechanism for following relays
FrankLi123 Oct 30, 2024
db35a1d
fix: remove unused variables
FrankLi123 Oct 30, 2024
6ea267e
refactor: simplify the NewOption function logic
FrankLi123 Oct 30, 2024
4c5161e
feat: exclude messages with timestamps older than 3 months
FrankLi123 Oct 30, 2024
a27397e
chore: add ToDo comments for further discussion
FrankLi123 Oct 30, 2024
d1bac9e
fix: add netHTTPClient to mastodon client struct
FrankLi123 Oct 30, 2024
ecb96cc
fix: aggregate logs together
FrankLi123 Oct 31, 2024
313623b
fix: imporve code style
FrankLi123 Oct 31, 2024
e85e3aa
fix: remove mastodon client_test content and mastodon parameters from…
FrankLi123 Oct 31, 2024
b947809
Merge branch 'main' into feat/rebuild-activitypub-engine-protocol-and…
brucexc Nov 7, 2024
ed96287
fix: merge conflict
brucexc Nov 7, 2024
7adb758
typo
brucexc Nov 7, 2024
340ecef
chore: ap log
brucexc Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions config/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ endpoints:
http_headers:
user-agent: rss3-node
mastodon:
url: https://0.0.0.0:9092/
url: https://newdomain.ngrok.app
http_headers:
user-agent: rss3-node
database:
Expand Down Expand Up @@ -79,7 +79,8 @@ component:
worker: core
endpoint: mastodon
parameters:
kafka_topic: activitypub_events
relay_url_list: ["https://relay.fedi.buzz/instance/mastodon.social"]

decentralized:
- network: ethereum
worker: core
Expand All @@ -104,7 +105,7 @@ component:
}
},
"mastodon": {
"url": "https://0.0.0.0:9092/",
"url": "https://newdomain.ngrok.app",
"http_headers": {
"user-agent": "rss3-node"
}
Expand Down Expand Up @@ -172,7 +173,9 @@ component:
"worker": "core",
"endpoint": "mastodon",
"parameters": {
"kafka_topic": "activitypub_events"
"relay_url_list": [
"https://relay.fedi.buzz/instance/mastodon.social"
]
}
}
],
Expand Down Expand Up @@ -211,7 +214,7 @@ url = "https://rpc.ankr.com/eth"
user-agent = "rss3-node"

[endpoints.mastodon]
url = "https://0.0.0.0:9092/"
url = "https://newdomain.ngrok.app"

[endpoints.mastodon.http_headers]
user-agent = "rss3-node"
Expand Down Expand Up @@ -265,7 +268,7 @@ worker = "core"
endpoint = "mastodon"

[component.federated.parameters]
kafka_topic = "activitypub_events"
relay_url_list = ["https://relay.fedi.buzz/instance/mastodon.social"]

[[component.decentralized]]
network = "ethereum"
Expand Down Expand Up @@ -298,7 +301,7 @@ var configFileExpected = &File{
},
},
"mastodon": {
URL: "https://0.0.0.0:9092/",
URL: "https://newdomain.ngrok.app",
HTTPHeaders: map[string]string{
"user-agent": "rss3-node",
},
Expand Down Expand Up @@ -338,13 +341,14 @@ var configFileExpected = &File{
Worker: federated.Core,
EndpointID: "mastodon",
Endpoint: Endpoint{
URL: "https://0.0.0.0:9092/",
URL: "https://newdomain.ngrok.app",
HTTPHeaders: map[string]string{
"user-agent": "rss3-node",
},
},
Parameters: &Parameters{
"kafka_topic": "activitypub_events",
"relay_url_list": []interface{}{"https://relay.fedi.buzz/instance/mastodon.social"},
// "port": 8181, //ToDo: add port here and other places
},
},
},
Expand Down
5 changes: 3 additions & 2 deletions deploy/config.example.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ discovery:
# You MUST NOT share the signature with anyone.
signature: 0x...signature
server:
# `endpoint` is how how Global Indexers access your Node.
# `endpoint` is how Global Indexers access your Node.
# You can use a domain or an IP address.
endpoint: https://your.node.com
# If your Node is registered on the VSL Testnet, use https://gi.rss3.dev.
Expand Down Expand Up @@ -97,4 +97,5 @@ component:
worker: core
endpoint: mastodon
parameters:
kafka_topic: activitypub_events
relay_url_list: [ "https://relay.fedi.buzz/instance/mastodon.social"]
port: 8181
3 changes: 1 addition & 2 deletions deploy/min/config.min.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ redis:

endpoints:
mastodon:
url: your-kafka-ip:9092
url: https://your-public-url.app
farcaster:
url: https://nemes.farcaster.xyz:2281
ethereum:
Expand Down Expand Up @@ -100,4 +100,3 @@ component:
worker: core
endpoint: mastodon
parameters:
kafka_topic: activitypub_events
5 changes: 4 additions & 1 deletion docs/openapi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1515,7 +1515,10 @@
"authentication": {
"$ref": "#/components/schemas/Authentication"
},
"kafka_topic": {
"relay_url_list": {
"$ref": "#/components/schemas/ConfigDetail"
},
"port": {
"$ref": "#/components/schemas/ConfigDetail"
}
}
Expand Down
3 changes: 2 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@ require (
github.com/adrianbrad/psqldocker v1.2.1
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0
github.com/emirpasic/gods v1.18.1
github.com/go-fed/httpsig v1.1.0
github.com/go-redsync/redsync/v4 v4.13.0
github.com/google/uuid v1.6.0
github.com/grafana/pyroscope-go v1.2.0
github.com/jaytaylor/html2text v0.0.0-20230321000545-74c2419ad056
github.com/mitchellh/mapstructure v1.5.0
Expand Down Expand Up @@ -110,7 +112,6 @@ require (
github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
github.com/golang/snappy v0.0.5-0.20220116011046-fa5810519dcb // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/grafana/pyroscope-go/godeltaprof v0.1.8 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.22.0 // indirect
Expand Down
4 changes: 4 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff h1:tY80oXqG
github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff/go.mod h1:x7DCsMOv1taUwEWCzT4cmDeAkigA5/QCwUodaVOe8Ww=
github.com/gballet/go-verkle v0.1.1-0.20231031103413-a67434b50f46 h1:BAIP2GihuqhwdILrV+7GJel5lyPV3u1+PgzrWLc0TkE=
github.com/gballet/go-verkle v0.1.1-0.20231031103413-a67434b50f46/go.mod h1:QNpY22eby74jVhqH4WhDLDwxc/vqsern6pW+u2kbkpc=
github.com/go-fed/httpsig v1.1.0 h1:9M+hb0jkEICD8/cAiNqEB66R87tTINszBRTjwjQzWcI=
github.com/go-fed/httpsig v1.1.0/go.mod h1:RCMrTZvN1bJYtofsG4rd5NaO5obxQ5xBkdiS7xsT7bM=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
Expand Down Expand Up @@ -636,6 +638,8 @@ golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.19.0/go.mod h1:2CuTdWZ7KHSQwUzKva0cbMg6q2DMI3Mmxp+gKJbskEk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.25.0 h1:WtHI/ltw4NvSUig5KARz9h521QvRC8RmF/cuYqifU24=
golang.org/x/term v0.25.0/go.mod h1:RPyXicDX+6vLxogjjRxjgD2TKtmAO6NZBsBRfrOLu7M=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
Expand Down
Loading
Loading