From ad8c6cabab71cffcb1280be71a7cf5c5280158ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sebastian=20M=C3=BCller?= Date: Tue, 28 Nov 2023 12:18:35 +0100 Subject: [PATCH] Hyperdot M1 update --- evaluations/hyperdot_1_semuelle.md | 105 +++++++++++++++++++++++++++-- 1 file changed, 98 insertions(+), 7 deletions(-) diff --git a/evaluations/hyperdot_1_semuelle.md b/evaluations/hyperdot_1_semuelle.md index 9bfda6ec8..35b4265b8 100755 --- a/evaluations/hyperdot_1_semuelle.md +++ b/evaluations/hyperdot_1_semuelle.md @@ -10,15 +10,106 @@ | ------ | ----------- | :------: | ---- |----------------- | | **0a.** | License | | [LICENSE](https://github.com/Infra3-Network/hyperdot-node/blob/ef7ee82d333da7377eb0bb1740b240104a62bee1/LICENSE) | Apache 2.0 | | **0b.** | Documentation | | [README](https://github.com/Infra3-Network/hyperdot-node/blob/ef7ee82d333da7377eb0bb1740b240104a62bee1/README.md), [README.zh-cn](https://github.com/Infra3-Network/hyperdot-node/blob/ef7ee82d333da7377eb0bb1740b240104a62bee1/README.zh-cn.md) | — | -| **0c.** | Testing and Testing Guide | | link | Core functions will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | -| **0d.** | Docker | | link | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | -| **0e.** | Article | | link | We will publish an **article**/workshop that explains [...] (what was done/achieved as part of the grant). (Content, language and medium should reflect your target audience described above.) | -| **1.** | SQL Query API | | link | We will use the bigquery client to integrate substrate-etl from the data sources provided by substrate-etl and creating a data query api using the bigquery client and substrate-etl, including

1. We will integrate the table schema provided by substrate etl to create interfaces for different query capabilities we provide, such as chain information, transaction information, etc.
2. Show the table-scheme of the associated chain
3. Run and save sql queries | -| **2.** | Dashboard Editor | | link | The apis needed to implement the dashboard, including
1. Edit the visualization dashboard page by loading sql
2. Save the dashboard you've edited | -| **3.** | Discovery | | link | We will implement the apis required by the discovery feature for data analytics dashboards and data query sharing, including
1. The dashboards list api
2. dashboards item Detail api. Each dashboard item is clicked to show the user's previously edited dashboard
3. queries List api
4. queries item details api. After clicking each query item, it displays the sql data tables and other information that the user has saved and run before | -| X. | ... | | link | see [General Notes](#general-notes) | +| **0c.** | Testing and Testing Guide | | [README](https://github.com/Infra3-Network/hyperdot-node/tree/d3e7ac7d3d7fa29bf774f24847041d91556229d9#testing) | Tested using provided gcloud credentials. Inline docs incomplete. | +| **0d.** | Docker | | [README](https://github.com/Infra3-Network/hyperdot-node/tree/d3e7ac7d3d7fa29bf774f24847041d91556229d9#docker-installation-guide) | — | +| **0e.** | Article | | [link](https://medium.com/@jackson.libc.rs/the-data-analysis-powerhouse-for-the-polkadot-ecosystem-hyperdot-has-arrived-c44c16cc3539) | Good overview and walkthrough. | +| **1.** | SQL Query API | | [apis/service/query](https://github.com/Infra3-Network/hyperdot-node/tree/d3e7ac7d3d7fa29bf774f24847041d91556229d9/internal/apis/service/query) | — | +| **2.** | Dashboard Editor | | [apis/service/dashboard](https://github.com/Infra3-Network/hyperdot-node/tree/d3e7ac7d3d7fa29bf774f24847041d91556229d9/internal/apis/service/dashboard) | — | +| **3.** | Discovery | | [apis/service/query](https://github.com/Infra3-Network/hyperdot-node/blob/d3e7ac7d3d7fa29bf774f24847041d91556229d9/internal/apis/service/query/query.go) | — | ## General Notes - — + +### `make tests` output + +```sh +> make tests +Makefile:21: warning: overriding recipe for target 'build/docker' +Makefile:12: warning: ignoring old recipe for target 'build/docker' +go test -v ./tests/ --count=1 +2023/11/28 11:54:15 initDB +2023/11/28 11:54:15 initDB: [{"name":"Bigquery","datasets":{"Bigquery":{"id":"raw","title":"Raw","description":"Raw blockchain crypto data"}}}] +2023/11/28 11:54:21 SetBigQueryChainData: &{0xc00131c3c0} +[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. + +[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. + - using env: export GIN_MODE=release + - using code: gin.SetMode(gin.ReleaseMode) + +[GIN-debug] GET /apis/v1/system/engines --> infra-3.xyz/hyperdot-node/internal/apis/service/system.(*Service).RouteTables.(*Service).ListEnginesHandler.func1 (4 handlers) +[GIN-debug] GET /apis/v1/system/engines/:engineId --> infra-3.xyz/hyperdot-node/internal/apis/service/system.(*Service).RouteTables.(*Service).GetQueryEngineDatasetHandle.func2 (4 handlers) +[GIN-debug] POST /apis/v1/query/run --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).RunHandler.func1 (4 handlers) +[GIN-debug] GET /apis/v1/query/:id --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).GetQueryHandler.func2 (4 handlers) +[GIN-debug] GET /apis/v1/query --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).ListQueryHandler.func3 (4 handlers) +[GIN-debug] POST /apis/v1/query --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).CreateQueryHandler.func4 (4 handlers) +[GIN-debug] PUT /apis/v1/query --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).UpdateQueryHandler.func5 (4 handlers) +[GIN-debug] DELETE /apis/v1/query/:id --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).DeleteQueryHandler.func6 (4 handlers) +[GIN-debug] GET /apis/v1/query/favorite --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).ListFavoriteQueryHandler.func7 (4 handlers) +[GIN-debug] GET /apis/v1/query/browse --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).ListBrowseQueryHandler.func8 (4 handlers) +[GIN-debug] GET /apis/v1/query/charts --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).ListCurrentUserQueryChartHandler.func9 (4 handlers) +[GIN-debug] GET /apis/v1/query/charts/user/:userId --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).ListUserQueryChartHandler.func10 (4 handlers) +[GIN-debug] GET /apis/v1/query/chart/:id --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).GetCurrentUserQueryChartHandler.func11 (4 handlers) +[GIN-debug] GET /apis/v1/query/chart/:id/user/:userId --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).GetUserQueryChartHandler.func12 (4 handlers) +[GIN-debug] DELETE /apis/v1/query/chart/:id/ --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).DeleteQueryChartHandler.func13 (4 handlers) +[GIN-debug] PUT /apis/v1/query/favorite --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).QueryFavoriteHandler.func14 (4 handlers) +[GIN-debug] PUT /apis/v1/query/unfavorite --> infra-3.xyz/hyperdot-node/internal/apis/service/query.(*Service).RouteTables.(*Service).QueryUnfavoriteHandler.func15 (4 handlers) +[GIN-debug] GET /apis/v1/dashboard/:id --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).GetDashboardHandler.func1 (4 handlers) +[GIN-debug] GET /apis/v1/dashboard --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).ListDashboardHandler.func2 (4 handlers) +[GIN-debug] POST /apis/v1/dashboard --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).CreateDashboardHandler.func3 (4 handlers) +[GIN-debug] PUT /apis/v1/dashboard --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).UpdateDashboardHandler.func4 (4 handlers) +[GIN-debug] DELETE /apis/v1/dashboard/:id --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).DeleteDashboardHandler.func5 (4 handlers) +[GIN-debug] GET /apis/v1/dashboard/favorite --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).ListFavoriteDashboardHandler.func6 (4 handlers) +[GIN-debug] GET /apis/v1/dashboard/browse --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).ListBrowseUserDashboardHandler.func7 (4 handlers) +[GIN-debug] GET /apis/v1/dashboard/tag/populars --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).ListPopularDashboardTags.func8 (4 handlers) +[GIN-debug] PUT /apis/v1/dashboard/favorite --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).DashboardFavoriteHandler.func9 (4 handlers) +[GIN-debug] PUT /apis/v1/dashboard/unfavorite --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).DashboardUnfavoriteHandler.func10 (4 handlers) +[GIN-debug] DELETE /apis/v1/dashboard/panel/:panelId --> infra-3.xyz/hyperdot-node/internal/apis/service/dashboard.(*Service).RouteTables.(*Service).RemoveDashboardPanelHandler.func11 (4 handlers) +[GIN-debug] GET /apis/v1/user --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).GetCurrentUserHandler.func1 (4 handlers) +[GIN-debug] GET /apis/v1/user/:id --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).GetUserHandler.func2 (4 handlers) +[GIN-debug] PUT /apis/v1/user --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).UpdateUserHandler.func3 (4 handlers) +[GIN-debug] PUT /apis/v1/user/email --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).UpdateEmailHandler.func4 (4 handlers) +[GIN-debug] PUT /apis/v1/user/password --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).UpdatePasswordHandler.func5 (4 handlers) +[GIN-debug] POST /apis/v1/user/avatar/upload --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).UploadAvatarHandler.func6 (4 handlers) +[GIN-debug] GET /apis/v1/user/avatar --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).GetAvatarHandler.func7 (4 handlers) +[GIN-debug] POST /apis/v1/user/auth/createAccount --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).CreateAccountHandler.func8 (4 handlers) +[GIN-debug] POST /apis/v1/user/auth/login --> infra-3.xyz/hyperdot-node/internal/apis/service/user.(*Service).RouteTables.(*Service).LoginHandle.func9 (4 handlers) +[GIN-debug] GET /apis/v1/file --> infra-3.xyz/hyperdot-node/internal/apis/service/file.(*Service).RouteTables.(*Service).GetFileHandler.func1 (4 handlers) +[GIN-debug] GET /apis/v1/swager/*any --> github.com/swaggo/gin-swagger.CustomWrapHandler.func1 (4 handlers) +=== RUN TestDashboardCreate +[GIN] 2023/11/28 - 11:54:21 | 200 | 1.37953ms | | POST "/apis/v1/dashboard" +--- PASS: TestDashboardCreate (0.07s) +=== RUN TestQueryRun +2023/11/28 11:54:22 Start BuildBigQueryEngine Job +[GIN] 2023/11/28 - 11:54:24 | 200 | 2.060044546s | | POST "/apis/v1/query/run" +--- PASS: TestQueryRun (2.12s) +=== RUN TestCRUDQuery +[GIN] 2023/11/28 - 11:54:24 | 200 | 9.403502ms | | POST "/apis/v1/query" +[GIN] 2023/11/28 - 11:54:24 | 200 | 1.303269ms | | GET "/apis/v1/query/3" +[GIN] 2023/11/28 - 11:54:24 | 200 | 16.824092ms | | PUT "/apis/v1/query" +[GIN] 2023/11/28 - 11:54:24 | 200 | 906.584µs | | GET "/apis/v1/query/3" +[GIN] 2023/11/28 - 11:54:24 | 200 | 9.084341ms | | DELETE "/apis/v1/query/3" + +2023/11/28 11:54:24 /home/sebastian/work/w3f/code/grants/deliverables/hyperdot/hyperdot-node/internal/apis/service/query/query.go:248 record not found +[0.460ms] [rows:0] SELECT * FROM "hyperdot_queries" WHERE "hyperdot_queries"."id" = '3' ORDER BY "hyperdot_queries"."id" LIMIT 1 +[GIN] 2023/11/28 - 11:54:24 | 404 | 540.973µs | | GET "/apis/v1/query/3" +--- PASS: TestCRUDQuery (0.40s) +=== RUN TestUserCreateAccount +[GIN] 2023/11/28 - 11:54:24 | 400 | 832.505µs | | POST "/apis/v1/user/auth/createAccount" +--- PASS: TestUserCreateAccount (0.06s) +=== RUN TestUserLogin +[GIN] 2023/11/28 - 11:54:24 | 200 | 58.788743ms | | POST "/apis/v1/user/auth/login" +--- PASS: TestUserLogin (0.12s) +=== RUN TestUserGetCurrent +[GIN] 2023/11/28 - 11:54:24 | 200 | 1.156744ms | | GET "/apis/v1/user" +--- PASS: TestUserGetCurrent (0.06s) +=== RUN TestUserGet +[GIN] 2023/11/28 - 11:54:24 | 200 | 601.101µs | | GET "/apis/v1/user/1" +--- PASS: TestUserGet (0.07s) +=== RUN TestUserUpdate +[GIN] 2023/11/28 - 11:54:24 | 200 | 1.923897ms | | PUT "/apis/v1/user" +--- PASS: TestUserUpdate (0.06s) +PASS +ok infra-3.xyz/hyperdot-node/tests 9.225s +``` \ No newline at end of file