Socket.io API endpoint for players of a game to start real-time communication #37

metalalive · 2023-11-22T15:03:36Z

API endpoint to `socket.io` server

localhost:8082 , path / ,GET method, no request body
Identity authentication and authorization (e.g. [verify JWT signature) will be postponed
- we can do that when we start JWT implementation for HTTP RESTful server

Interaction with `socket.io` server

Client (Player)

Possible events namespace to emit (or topics to publish)

/game/{game-id}/join <-- Not sure whether this is good design option
- For later comer who attempts to join a game which hasn't been started yet
- Possible arguments : e.g. player-id = abc999xyz
- Note we have not decided which binary format to apply to event message, but it is not JSON for sure.
/game/{game-id}/chat
- For player chatting with other players of the same existing game
- Possible arguments : e.g. player-id = abg007xyz, chat-message = Hello World

Possible events namespace to receive (topics to subscribe)

/game/{game-id}/config/difficulty
- refresh game difficulty
- Possible arguments : e.g. level = standard,
/game/{game-id}/config/investigators
- refresh the config, shows which player successfully selected which investigator
- Possible arguments : e.g. player-id = SamuraiBuck, investigator = magician

There should be many more topics to subscribe for each player

especially when the game started , each player completes any action or draw any card in his / her turn.
for now , I simply list a few we can try to implement in the beginning.

HTTP RESTful server

Possible events namespace to emit (or topics to publish)

/game/{game-id}/config/difficulty
/game/{game-id}/config/investigators

Possible events namespace to receive (topics to subscribe)

None

Question

event namespace /game/{game-id}/join , <-- is this good design option ?
- what about adding new REST endpoint /game/{game-id}/join ? to let the HTTP RESTful server know one more person would like to join , then re-use Game.add_player() in backend code.

TODO

implement minimal working socket.io server
infra setup (python-socketio and Kafka)

Reference

See previous discussion #29

@wingtkw @teds-lin , plz take a look at this,
feedback or suggestion are welcome , thanks.

The text was updated successfully, but these errors were encountered:

teds-lin · 2023-11-24T17:15:01Z

/game/{game-id}/join <-- Not sure whether this is good design option

For later comer who attempts to join a game which hasn't been started yet

設計方針是只與GaaS大平台串接還是會有考慮其他遊戲登錄平台？
這是指之後有其他玩家也想要加入遊戲房間，所以一同遊玩人數會變更的情境嗎？
如果是GaaS大平台串接，
因為是等大平台那邊玩家都組好隊，
才會送過來，當送過來後，
GaaS也沒辦法在新增玩家進來，
所以我認為先不考慮這個功能

/game/{game-id}/chat

不是很懂，看前面一條我以為是在討論REST endpoint?
可是下面解釋說是"player chatting with other players" 所以是在講socket.io?
我不認為玩家聊天API需要在REST endpoint也建置對應的，徒增複雜度
socket.io的話 namespace 官網圖有點難懂
其實就是像這篇文章介紹的
Client可以建立很多socket連線到不同的namespace，每個namespace又可以分很多room

在我們遊戲這個case，
我認為很簡單的用原本預設的/就好，
然後共同遊戲的玩家都在同一個game-id的room內

There should be many more topics to subscribe for each player

是在講Kafka?我還沒開始看，暫時不了解設計上需要什麼

如果是在講socket.io的event namespace
目前我想到兩個設計方案
(1)
因為是傳送和game相關的可能是命名為game_state或者直接叫game之類的
通常event namespace和上面剛剛提到的namespace命名上最大區別就是最前頭會不會加上/
如果是與玩家聊天相關的內容，
傳送資料結構內可能會用Message_Type來區分兩者。
(2)
另外一個設計方案就是
將玩家聊天相關內容獨立出來，event namespace命名就chat之類的名稱
這樣可以明確分隔傳送的是遊戲狀態更新(game)還是聊天訊息(chat)

加上之前討論我說

考慮只傳送差異更新而不是完整遊戲狀態

為了舉例方便，我用JSON示範
具體程式碼片段可能會像

# 已從Kafka接收到新的難度等級
new_difficulty_level = STANDARD

# 發布遊戲狀態更新到前端
sio.emit(
    'game', 
    {'event': CHANGE_DIFFICULTY_STATE,
      'difficulty': new_difficulty_level},
    to=game_id
)

metalalive · 2023-11-25T08:31:40Z

Hi, Ted, 我先回答部分問題

設計方針是只與GaaS大平台串接還是會有考慮其他遊戲登錄平台？

只考慮 GaaS 大平台

這是指之後有其他玩家也想要加入遊戲房間，所以一同遊玩人數會變更的情境嗎？
如果是GaaS大平台串接，
因為是等大平台那邊玩家都組好隊，
才會送過來，當送過來後，
GaaS也沒辦法在新增玩家進來，
所以我認為先不考慮這個功能

是的, 目前的 GaaS 大平台無法增加更多玩家到已存在的遊戲房間
我以為這部分是由每個遊戲應用程式處理,
所以我最初的問題是 :

我們是否應該添加 REST endpoint /game/{game-id}/join , 讓一些稍後過來的玩家能進入房間 ?
- 例如，一開始有2個人，然後遊戲開始之前 ,又有第三位玩家想加入
或者，讓玩家先連接到socket.io server , 然後 subscribe event , with the namespace /game/{game-id}/join ?

但ok , 這也不是關鍵功能，可以先不考慮

/game/{game-id}/chat

不是很懂，看前面一條我以為是在討論REST endpoint?
可是下面解釋說是"player chatting with other players" 所以是在講socket.io?

這不是 REST endpoint, 這是 event namespace , 這是官方文件中的範例

假設玩家 (client) 透過 https://localhost:8082/ 連接到 socket.io server
連線建立後, 玩家可以決定監聽哪個 event , with custom namespace such as :
- /game/{game-id}/chat , 接收同一遊戲房間其他玩家的訊息
- /game/{game-id}/config/investigators 如果任何玩家更改了 investigator ，這個事件應該反映到同一房間的其他玩家
- /game/{game-id}/config/difficulty , 如果任何玩家更改了 difficulty level ，這個事件應該反映到同一房間的其他玩家

所以我最初的comment 是問你們對 namespaces, topics 的命名規則有何偏好
與 REST endpoint 無關，抱歉造成混淆

其實就是像這篇文章介紹的
Client可以建立很多socket連線到不同的namespace，每個namespace又可以分很多room

OK

簡單的 topics (to subscribe) -->
- /game for 遊戲狀態更新
- /chat for 聊天訊息
- /something/else
比較複雜的 topics -->
- /game/{game-id}/chat 如果您希望聊天訊息僅在特定房間的玩家可以看到
- /game/{game-id}/config/investigators
- /game/{game-id}/state/card-draw
- , 其中 {game-id} 是代表遊戲 ID 的動態字串
- under the namespace game/{game-id} , there could be rooms such as chat, config , state ...etc
  - does it match the concept of rooms ?

在我們遊戲這個case，
我認為很簡單的用原本預設的 / 就好，
然後共同遊戲的玩家都在同一個game-id的room內

Where to place the game ID ? should game ID be part of topic or namespace ? like /game/{game-id}.
Or each room under the namespace /game is identified by game ID ?

metalalive · 2023-11-26T08:20:48Z

Hi, Ted , continue to reply rest of the questions

是在講Kafka?我還沒開始看，暫時不了解設計上需要什麼

please ignore the message , it is unrelated to Kafka ,
currently let's focus on socket.io namespaces in the game app.

如果是在講socket.io的event namespace
目前我想到兩個設計方案
(1)
...... omitted ........
(2)
另外一個設計方案就是
將玩家聊天相關內容獨立出來，event namespace命名就chat之類的名稱
這樣可以明確分隔傳送的是遊戲狀態更新(game)還是聊天訊息(chat)

I prefer the 2nd design option, but I am not sure whether I can set up the namespace such like /game/{game-id} in the backend app (where {game-id} is a variable can be changed dynamically) , I will try the library python-socketio in my local development environment & reply later with my experiment result .

wingtkw · 2023-12-02T05:10:37Z

想問一下
State card draw 是想用來做甚麽的
是回合抽卡完成
誰的回合
還是別的

我在想把卡有關的的endpoints
可以group 到 /game/{gameId}/card 之下
所以想了解一下這條endpoint 的用意是甚麽

metalalive · 2023-12-02T06:02:49Z

State card draw 是想用來做甚麽的
是回合抽卡完成
誰的回合
還是別的

抱歉，我只是舉個例子, 目前我還沒考慮實作細節
請忽略這個 card draw

Finally I got some time to work on the socket.io server, I will first start implementation which publish / subscribes message with the following topics -- 我目前正在做的 socket.io topics

/game/{game-id}/chat
/game/{game-id}/config/investigators

metalalive · 2023-12-05T09:46:34Z

To everyone following / interested with this issue, here is the experiment result,
feel free to ask question, I appreciate any feedback about the code and design,
If I don't get any reply by the end of Thursday afternoon (17:30:00 , 7th. Dec. 2023) , I will continue working on the real-time chat implementation and then quickly create a PR.

[1]
The comment in this python-socketio issue describes that socket.io client with http requires aiohttp , otherwise there will be timeout exception on the client side. In this game backend app , socket.io client is used ONLY for testing purpose, it is suggested to install aiohttp in test dependency of poetry config file.

[2]
the namespace in python-socketio seems more like a constant than a variable, it might NOT be good practice to generate namespace dynamically at runtime. Quote my previous comment :

I prefer the 2nd design option, but I am not sure whether I can set up the namespace such like /game/{game-id} in the backend app (where {game-id} is a variable) , I will try the library python-socketio in my local development environment & reply later with my experiment result .

After coding with python-socketio library, I decided to give up doing that , instead I set {game-id} to room with each event and namespace .

[3]
For testing

Initially, I tried AsyncClient , which monitors events by registering event-handling callbacks to the method on(...) , however the handler callback does not seem to provide any argument that represents which client is receiving which message , which is NOT friendly to tests.
Now I use AsyncSimpleClient in test case, it provides the method receive(...) for getting events from socket.io server, this is good for understanding which client is receiving which message.

[4]
I pushed my commits to the branch experiment/backend/socketio (it is only for experiment purpose) , this comment will be no longer edited .

Signed-off-by: T.H. <[email protected]>

Signed-off-by: Han Woo <[email protected]>

metalalive · 2023-12-06T10:39:43Z

To @wingtkw :
I have done the socket.io experiment (see the result above)

I am considering to add another API doc at doc/api for the socket.io real-time communication,
this document requires both of frontend and backend contributors working together,
can you check the test case in the branch /experiment/backend/socketio to see whether is it OK for chat event ?
currently there are fields like client ,gameID , and msg in the chat event, please take a look , thanks.

To anyone interested with the work,
you are also welcome to discuss with us ,
(I don't tag anyone else here, feel free to get involved), thanks

metalalive · 2023-12-06T10:52:21Z

TODO

the list is what I will do :

Portion 1

replace print message with logger in socket.io server
add fields like nickname to the data of the chat event (what other fields required by frontend app ?)
remove first forward slash character / from each event name, e.g. /chat renamed to chat

create PR, merge to main branch --- (done in 11th Dec 2023)

Portion 2

For detecting re-entrants , as well as the players who have never joined a game room
- socket.io server needs to know which game rooms are created, which players are currently in which room
- feasible implementation , how can socket.io server knows about it ? :
  - socket.io server maintains some kind of table map, and the table map records events emitted from REST server on creating game <-- (preferred, more de-coupled)
  - socket.io server directly read from game repository through the abstract interface.

Remind that :

GaaS Lobby platform creates a new game , by sending request to the REST endpoint /games , with POST method and list of player IDs

currently we (this game dev team) don't consider to let more players (or late comers) join an existing game room

create PR, merge to main branch --- (done in 26th Jan 2024)

Portion 3

REST HTTP server sends event for updating character selection and difficulty level of specific game, to socket.io server, the socket.io sevrer forwards the events to the players

create PR, merge to main branch --- (done in 1st Feb 2024)

Portion 4

for binary payload format , try switching to FlatBuffers or ProtoBuf (see previous discussion)

create PR, merge to main branch --- (done in 7th Feb 2024)

Tasks NOT included in this TODO list

ensure only HTTP RESTful server has privilege of emitting game-state-update events to socket.io server , it might be related to Kafka task below.
Kafka task is moved to the issue Apply Kafka to the socket.io backend server for real-time communication #56 , I need to figure out the appropriate usage context before applying Kafka.

teds-lin · 2023-12-06T13:36:48Z

通常event namespace和上面剛剛提到的namespace命名上最大區別就是最前頭會不會加上/

我注意到實驗分支 experiment/backend/socketio 實作中
event namespace 命名開頭都加上/, 例如 /chat, /init等
好奇這樣取名原因是？

一些相關討論：
Custom event naming convention
Is there any name convention for response event in socketIO?

補充說明：
namespace會當作URL的一部分，所以開頭第一個字元固定為/
如果預設連接的URL為http://xxx.example.com/
用預設的namespace="/"
當為socket.io時，URL實際連結會像http://xxx.example.com/socket.io/?EIO=4&transport=websocket
當namespace="/game"
實際上代表連接的URL為http://xxx.example.com/game
當為socket.io時，URL實際連結會像http://xxx.example.com/game/socket.io/?EIO=4&transport=websocket
至於event namespace 是socket.io內部通訊event實作，跟連接的URL無關

I am considering to add another API doc at doc/api for the socket.io real-time communication,

socket.io 文件通常用 AsyncAPI 撰寫, 但我不確定同樣放在doc/api這樣能不能相容, 或者放在doc/socketio？
Pt1
Pt2

metalalive · 2023-12-06T16:05:08Z

Hi , Ted

event namespace 命名開頭都加上/, 例如 /chat, /init等
好奇這樣取名原因是？

The naming convention of socket.io event really depends on development team (at least the name has to reflect the corresponding operation done by server or each client), it can be changed.
And yes, there is no reason to add extra slash symbol / at the beginning of event name, this can also be removed,
current events can be renamed to chat , init, deinit

socket.io 文件通常用 AsyncAPI 撰寫, 但我不確定同樣放在doc/api這樣能不能相容, 或者放在doc/socketio？

thanks, good to know about AsyncAPI and event-driven architecture API document
It is also OK to place it in different path such as doc/socketio ,

- remove unnecessary forward slash character in event name - check fields `msg` and `nickname` in the event data `chat` - logger to record server errors Signed-off-by: T.H. <[email protected]>

- validate event arguments with pydantic base model - rename the field `room` to game ID in `init` , `deinit` events - add the field `nickname` to the event data `init` and `deinit` - move constants , configurable parameters in appropriate modules Signed-off-by: T.H. <[email protected]>

Signed-off-by: T.H. <[email protected]>

- Socket.io server for real-time communication in the game - currently only some events `init` (join room), `deinit` (leave room), `chat` are implemented

metalalive · 2023-12-11T17:14:37Z

Let us focus on the 2nd portion of the TODO list in this GitHub Issue , discussion related to the design / implementation are appreciated.

the REST server emits event on creating new game
then socket.io server receives the create-game events to know which players should be in which game room.
- there will be adapter interface to data store, which simply records currently active players in which rooms
  - the abstract class name hasn't been determined yet, maybe something called RtcRoomRepository or what else.
- the dev team can switch between different storages e.g. in-memory data store or non-relational database if you like.
  - for simplicity, currently the socket.io server stores the events to in-memory data store

[update 15th Dec 2023]
I am fighting against tricky tech issues in other projects, will start working on this once I have time.

[update 11th Jan 2024]
I'll also check whether I can get this done without setting up Kafka

[update 23th Jan 2024]
http server can act as socketio client emitting events to socketio server without Kafka

currently there is only one http server, not a real distributed environment
I will figure out how it works internally in the future , reference doc

wingtkw · 2023-12-13T00:58:58Z

Let me review it on 15
Got AL on that day

… game - socketio server collects the events, to know all the ongoing games - http server maintains a shared socketio client sending all relavant events - rename `AbstractRepository` to `AbstractGameRepository` - TODO, implement room repository for real-time communication Signed-off-by: T.H. <[email protected]>

- implement (currently in-memory) room repository , so the socketio server knows all the ongoing games - check from the room repository when player attempts to initialize (or de-initialize) real-time communication (RTC) through the socketio event `INIT` (or `DEINIT`) - add new member to `GameFuncCodes`, for RTC endpoints (e.g. currently `socketio`) Signed-off-by: T.H. <[email protected]>

@wingtkw

…ackend/socketio/create-game Hi @wingtkw , As I described above I will merge the PR first , you are still welcome to leave comment at here, or create another issue / discussion thread for any question about learning my code at here

metalalive · 2024-01-31T16:05:24Z

I am working on the 3rd portion of the TODO list in this GitHub Issue , discussion related to the design / implementation are appreciated.

- http server emits corresponding event on completing the operations to socket.io server - socket.io server simply forwards the game-state messages to corresponding players of a given game Signed-off-by: T.H. <[email protected]>

…acter-selection-before-start Emit events on difficulty and character update

metalalive · 2024-02-04T17:09:54Z

I will work on the 4th (final) portion of the TODO list in this GitHub Issue.

I'll try FlatBuffers first with socket.io server and see how it goes

@wingtkw , @KalpasWang , @liamlin1102
Appreciate discussion / your feedback related to the design / implementation.

- integrate with FlatBuffers - binary payload generaated on switch-character event Signed-off-by: T.H. <[email protected]>

- integrate with FlatBuffers - binary payload generated on difficulty-config event Signed-off-by: T.H. <[email protected]>

…ckend/socketio/flatbuffer-payload - apply FlatBuffers payload to socket.io server on game config events - currently this affects the 2 events, such as switch character, set difficulty level - Other (future) events reflecting change of the game state can encode / decode payload in the same way for enhancing performance.

metalalive · 2024-02-07T04:06:26Z

Currently close this Issue,
any code / design contributor can reopen it whenever there is new event to add in the real-time communication.

c.c. @wingtkw , @KalpasWang

KalpasWang · 2024-02-07T05:28:18Z

@metalalive
it seems everyone disappear except us, what happened? 🤔

metalalive self-assigned this Dec 4, 2023

metalalive added a commit that referenced this issue Dec 6, 2023

[#37] feature: real-time chat in game with socket.io server

90183c6

Signed-off-by: T.H. <[email protected]>

metalalive added a commit that referenced this issue Dec 6, 2023

[#37] bugfix: modify build script for backend CI

d648e36

Signed-off-by: Han Woo <[email protected]>

metalalive mentioned this issue Dec 9, 2023

Socket.io server for real-time chat events in the game #47

Merged

metalalive added a commit that referenced this issue Dec 10, 2023

[#37] build: modify github-action script

38c4f56

Signed-off-by: T.H. <[email protected]>

metalalive added a commit that referenced this issue Dec 11, 2023

[#37] chore: merge pull request #47

45e3ec8

- Socket.io server for real-time communication in the game - currently only some events `init` (join room), `deinit` (leave room), `chat` are implemented

metalalive mentioned this issue Jan 25, 2024

Restrict players to the game room in socketio server, by http server emits event on creating game #54

Merged

This was referenced Feb 1, 2024

Emit events on difficulty and character update #55

Merged

Add API endpoint for starting a game #12

Closed

metalalive added a commit that referenced this issue Feb 1, 2024

[#37] chore: Merge pull request 55 from feature/backend/socketio/char…

2994db9

…acter-selection-before-start Emit events on difficulty and character update

metalalive mentioned this issue Feb 1, 2024

Frontend Status Store #53

Open

metalalive added a commit that referenced this issue Feb 6, 2024

[#37] feature: apply binary payload format to socket.io server

7a07047

- integrate with FlatBuffers - binary payload generaated on switch-character event Signed-off-by: T.H. <[email protected]>

metalalive added a commit that referenced this issue Feb 6, 2024

[#37] feature: apply binary payload format to socket.io server

bf61893

- integrate with FlatBuffers - binary payload generated on difficulty-config event Signed-off-by: T.H. <[email protected]>

metalalive mentioned this issue Feb 6, 2024

feature: apply FlatBuffers payload to socket.io server on game config events #57

Merged

metalalive closed this as completed Feb 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Socket.io API endpoint for players of a game to start real-time communication #37

Socket.io API endpoint for players of a game to start real-time communication #37

metalalive commented Nov 22, 2023 •

edited

Loading

teds-lin commented Nov 24, 2023 •

edited

Loading

metalalive commented Nov 25, 2023 •

edited

Loading

metalalive commented Nov 26, 2023 •

edited

Loading

wingtkw commented Dec 2, 2023 •

edited

Loading

metalalive commented Dec 2, 2023 •

edited

Loading

metalalive commented Dec 5, 2023 •

edited

Loading

metalalive commented Dec 6, 2023

metalalive commented Dec 6, 2023 •

edited

Loading

teds-lin commented Dec 6, 2023 •

edited

Loading

metalalive commented Dec 6, 2023 •

edited

Loading

metalalive commented Dec 11, 2023 •

edited

Loading

wingtkw commented Dec 13, 2023

metalalive commented Jan 31, 2024

metalalive commented Feb 4, 2024 •

edited

Loading

metalalive commented Feb 7, 2024 •

edited

Loading

KalpasWang commented Feb 7, 2024

Socket.io API endpoint for players of a game to start real-time communication #37

Socket.io API endpoint for players of a game to start real-time communication #37

Comments

metalalive commented Nov 22, 2023 • edited Loading

API endpoint to socket.io server

Interaction with socket.io server

Client (Player)

HTTP RESTful server

Question

TODO

Reference

teds-lin commented Nov 24, 2023 • edited Loading

metalalive commented Nov 25, 2023 • edited Loading

metalalive commented Nov 26, 2023 • edited Loading

wingtkw commented Dec 2, 2023 • edited Loading

metalalive commented Dec 2, 2023 • edited Loading

metalalive commented Dec 5, 2023 • edited Loading

metalalive commented Dec 6, 2023

metalalive commented Dec 6, 2023 • edited Loading

TODO

Portion 1

Portion 2

Portion 3

Portion 4

Tasks NOT included in this TODO list

teds-lin commented Dec 6, 2023 • edited Loading

metalalive commented Dec 6, 2023 • edited Loading

metalalive commented Dec 11, 2023 • edited Loading

wingtkw commented Dec 13, 2023

metalalive commented Jan 31, 2024

metalalive commented Feb 4, 2024 • edited Loading

metalalive commented Feb 7, 2024 • edited Loading

KalpasWang commented Feb 7, 2024

metalalive commented Nov 22, 2023 •

edited

Loading

API endpoint to `socket.io` server

Interaction with `socket.io` server

teds-lin commented Nov 24, 2023 •

edited

Loading

metalalive commented Nov 25, 2023 •

edited

Loading

metalalive commented Nov 26, 2023 •

edited

Loading

wingtkw commented Dec 2, 2023 •

edited

Loading

metalalive commented Dec 2, 2023 •

edited

Loading

metalalive commented Dec 5, 2023 •

edited

Loading

metalalive commented Dec 6, 2023 •

edited

Loading

teds-lin commented Dec 6, 2023 •

edited

Loading

metalalive commented Dec 6, 2023 •

edited

Loading

metalalive commented Dec 11, 2023 •

edited

Loading

metalalive commented Feb 4, 2024 •

edited

Loading

metalalive commented Feb 7, 2024 •

edited

Loading