feat: DB driver and cache (#15)

Related to #11 PR to add a Postgres database driver using SQLC to generate Go code from SQL files: https://sqlc.dev/ This is the first prerequisite to allow associating a service request with a user for the purposes of metering, billing, rate limiting, user settings, etc. This PR adds the following: - `db/driver` package containing Postgres driver - `db` package containing local cache of user data - `user` package containing the `GatewayEndpoint` struct --------- Signed-off-by: Pascal van Leeuwen <[email protected]> Signed-off-by: commoddity <[email protected]> Co-authored-by: Daniel Olshansky <[email protected]>
buildwithgrove · Sep 19, 2024 · bd4e0cf · bd4e0cf
1 parent c7e5e7b
commit bd4e0cf
Show file tree

Hide file tree

Showing 34 changed files with 1,554 additions and 232 deletions.
diff --git a/Dockerfile b/Dockerfile
@@ -1,4 +1,4 @@
-FROM golang:1.22.3-alpine3.19 AS builder
+FROM golang:1.23-alpine3.19 AS builder
 RUN apk add --no-cache git
 
 WORKDIR /go/src/github.com/buildwithgrove/path

diff --git a/Makefile b/Makefile
@@ -13,16 +13,24 @@ help: ## Prints all the targets in all the Makefiles
 ### Run Path Make Targets ###
 #############################
 
+.PHONY: path_up_gateway
+path_up_gateway: ## Run just the PATH gateway without any dependencies
+	docker compose up -d --no-deps path_gateway
+
+.PHONY: path_up_build_gateway
+path_up_build_gateway: ## Run and build just the PATH gateway without any dependencies
+	docker compose up -d --build --no-deps path_gateway
+
 .PHONY: path_up
-path_up: ## Run docker compose up
+path_up: ## Run the PATH gateway and all related dependencies
 	docker compose up -d
 
 .PHONY: path_up_build
-path_up_build: ## Run docker compose up with build
+path_up_build: ## Run and build the PATH gateway and all related dependencies
 	docker compose up -d --build
 
 .PHONY: path_down
-path_down: ## Run docker compose down
+path_down: ## Stop the PATH gateway and all related dependencies
 	docker compose down
 
 #########################
@@ -59,3 +67,14 @@ copy_test_config: ## copies the example test configuration yaml file to .gitigno
 	else \
 		echo ".config.test.yaml already exists, not overwriting."; \
 	fi
+
+###############################
+### Generation Make Targets ###
+###############################
+
+.PHONY: sqlc_generate
+sqlc_generate: ## Generate SQLC code from db/driver/sqlc/*.sql files
+	sqlc generate -f ./db/driver/sqlc/sqlc.yaml
+
+# // TODO_TECHDEBT(@commoddity): move all mocks to a shared mocks package
+# // TODO_TECHDEBT(@commoddity): Add all other mock generation commands here
diff --git a/README.md b/README.md
@@ -17,25 +17,35 @@
 
 # Table of Contents <!-- omit in toc -->
 
-- [Introduction](#introduction)
-- [Releases](#path-releases)
-- [Quickstart (Shannon)](#quickstart-shannon)
-- [Configuration](#configuration)
-  - [Configuration File](#configuration-file)
-    - [Example Configuration Format](#example-configuration-format)
-- [Running PATH](#running-path)
-  - [Setup Config YAML](#setup-config-yaml)
-  - [Start the Container](#start-the-container)
-- [E2E Tests](#e2e-tests)
-  - [Running Tests](#running-tests)
-
-## Introduction
+- [1. Introduction](#1-introduction)
+  - [1.1. Prerequisites](#11-prerequisites)
+- [2. Path Releases](#2-path-releases)
+- [3. Quickstart (Shannon)](#3-quickstart-shannon)
+- [4. Configuration](#4-configuration)
+  - [4.1. Configuration File](#41-configuration-file)
+  - [4.2. Example Configuration Format](#42-example-configuration-format)
+- [5. Running PATH](#5-running-path)
+  - [5.1. Setup Config YAML](#51-setup-config-yaml)
+  - [5.2. Start the Container](#52-start-the-container)
+- [6. E2E Tests](#6-e2e-tests)
+  - [6.1. Running Tests](#61-running-tests)
+
+## 1. Introduction
 
 **PATH** (Path API & Toolkit Harness) is an open source framework for enabling access to a decentralized supply network.
 
 It provides various tools and libraries to streamline the integration and interaction with decentralized protocols.
 
-## Path Releases
+### 1.1. Prerequisites
+
+- [Docker](https://docs.docker.com/get-docker/)
+
+**Required For Development:**
+
+- [SQLC](https://docs.sqlc.dev/)
+- [Mockgen](https://github.com/uber-go/mock)
+
+## 2. Path Releases
 
 Path releases provide a Docker image you can start using right away to bootstrap your Path gateway without the need of building your own image. Images are available in our [Packages](https://github.com/buildwithgrove/path/pkgs/container/path) page. You can pull them directly using the following command:
 
@@ -47,7 +57,7 @@ Additionally, our releases contain additional information about what's up with u
 
 
 
-## Quickstart (Shannon)
+## 3. Quickstart (Shannon)
 
 1. Stake Apps and Gateway
 
@@ -77,9 +87,9 @@ Additionally, our releases contain additional information about what's up with u
 
    _For detailed instructions on running PATH, see the [Running PATH](#running-path) section._
 
-## Configuration
+## 4. Configuration
 
-### Configuration File
+### 4.1. Configuration File
 
 The configuration for PATH is defined in a YAML file, which should be named `.config.yaml`.
 
@@ -110,11 +120,7 @@ The configuration is divided into several sections:
    - _Optional. Default values will be used if not specified._
    - Configures router settings such as port and timeouts.
 
-5. **User Data Configuration (`user_data_config`)**:
-   - _Required only if the gateway operator wishes to associate user data with requests._
-   - Configures the PostgreSQL database connection string.
-
-#### Example Configuration Format
+### 4.2. Example Configuration Format
 
 ```yaml
 shannon_config:
@@ -138,9 +144,9 @@ services:
   - [Shannon](./cmd/config/testdata/shannon.example.yaml)
 - [Config YAML Schema](./config/config.schema.yaml)
 
-## Running PATH
+## 5. Running PATH
 
-#### Setup Config YAML
+### 5.1. Setup Config YAML
 
 - The PATH service requires the config YAML file to be populated.
 
@@ -156,7 +162,7 @@ services:
 
    **⚠️ IMPORTANT: The data required to populate the `.config.yaml` file is sensitive and the contents of this file must never be shared outside of your organization. ⚠️**
 
-#### Start the Container
+### 5.2. Start the Container
 
 1. Once the `.config.yaml` file is populated, to start the PATH service for a specific protocol, use the `make` target:
 
@@ -176,7 +182,7 @@ services:
    make path_down
    ```
 
-## E2E Tests
+## 6. E2E Tests
 
 This repository contains end-to-end (E2E) tests for the Shannon relay protocol. The tests ensure that the protocol behaves as expected under various conditions.
 
@@ -194,7 +200,7 @@ Currently, the E2E tests are configured to run against the Shannon testnet.
 
 Future work will include adding support for other protocols.
 
-#### Running Tests
+### 6.1. Running Tests
 
 To run the tests, use the following `make` targets:
 

diff --git a/cmd/main.go b/cmd/main.go
@@ -90,7 +90,7 @@ func main() {
 		log.Fatalf("failed to create API router: %v", err)
 	}
 
-	if err := apiRouter.Start(context.Background()); err != nil {
+	if err := apiRouter.Start(); err != nil {
 		log.Fatalf("failed to start API router: %v", err)
 	}
 }
diff --git a/config/config.go b/config/config.go
@@ -28,7 +28,6 @@ type (
 
 		Services map[relayer.ServiceID]ServiceConfig `yaml:"services"`
 		Router   RouterConfig                        `yaml:"router_config"`
-		UserData UserDataConfig                      `yaml:"user_data_config"`
 
 		// A map from human readable aliases (e.g. eth-mainnet) to service ID (e.g. 0021)
 		serviceAliases map[string]relayer.ServiceID
@@ -75,10 +74,6 @@ func (c GatewayConfig) GetRouterConfig() RouterConfig {
 	return c.Router
 }
 
-func (c GatewayConfig) GetUserDataConfig() UserDataConfig {
-	return c.UserData
-}
-
 // GetServiceIDFromAlias retrieves the ServiceID associated with a given service alias.
 //
 // This method allows for the use of a user-friendly string service alias in the

diff --git a/config/config_test.go b/config/config_test.go
@@ -67,9 +67,6 @@ func Test_LoadGatewayConfigFromYAML(t *testing.T) {
 					WriteTimeout:       5000 * time.Millisecond,
 					IdleTimeout:        5000 * time.Millisecond,
 				},
-				UserData: UserDataConfig{
-					DBConnectionString: "postgres://user:password@localhost:5432/database",
-				},
 				serviceAliases: map[string]relayer.ServiceID{
 					"eth-mainnet": "0021",
 				},
@@ -108,9 +105,6 @@ func Test_LoadGatewayConfigFromYAML(t *testing.T) {
 					WriteTimeout:       5000 * time.Millisecond,
 					IdleTimeout:        5000 * time.Millisecond,
 				},
-				UserData: UserDataConfig{
-					DBConnectionString: "postgres://user:password@localhost:5432/database",
-				},
 				serviceAliases: map[string]relayer.ServiceID{
 					"eth-mainnet": "0021",
 				},
@@ -320,7 +314,6 @@ func Test_GetServiceIDFromAlias(t *testing.T) {
 
 func compareConfigs(c *require.Assertions, want, got GatewayConfig) {
 	c.Equal(want.Router, got.Router)
-	c.Equal(want.UserData, got.UserData)
 	if want.MorseConfig != nil {
 		compareMorseFullNodeConfig(c, want.MorseConfig.FullNodeConfig, got.MorseConfig.FullNodeConfig)
 		c.Equal(want.MorseConfig.SignedAATs, got.MorseConfig.SignedAATs)

diff --git a/config/testdata/morse.example.yaml b/config/testdata/morse.example.yaml
@@ -34,7 +34,3 @@ router_config:
   read_timeout: "5000ms"
   write_timeout: "5000ms"
   idle_timeout: "5000ms"
-
-# user_data_config is required if user data is to be stored in a database.
-user_data_config:
-  db_connection_string: "postgres://user:password@localhost:5432/database"
diff --git a/config/testdata/shannon.example.yaml b/config/testdata/shannon.example.yaml
@@ -25,7 +25,3 @@ router_config:
   read_timeout: "5000ms"
   write_timeout: "5000ms"
   idle_timeout: "5000ms"
-
-# user_data_config is required if user data is to be stored in a database.
-user_data_config:
-  db_connection_string: "postgres://user:password@localhost:5432/database"
diff --git a/config/user_data.go b/config/user_data.go
diff --git a/config/user_data_test.go b/config/user_data_test.go
diff --git a/config/utils/utils.go b/config/utils/utils.go
@@ -30,10 +30,3 @@ func IsValidURL(s string) bool {
 	}
 	return u.Scheme == "http" || u.Scheme == "https"
 }
-
-// IsValidDBConnectionString checks if a string is a valid PostgreSQL connection string.
-func IsValidDBConnectionString(s string) bool {
-	// Regular expression to match a valid PostgreSQL connection string
-	var dbConnStringRegex = regexp.MustCompile(`^postgres://[^:]+:[^@]+@[^:]+:\d+/.+$`)
-	return dbConnStringRegex.MatchString(s)
-}