Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: DB driver and cache #15

Merged
merged 42 commits into from
Sep 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
9712372
feat: adds postgres driver and cache for user apps
commoddity Sep 7, 2024
6519d94
chore: add cache_refresh_interval to example yaml
commoddity Sep 7, 2024
297aa4c
fix: use whitelist type and value string types
commoddity Sep 7, 2024
76484a6
fix: update to use allowlists with enum
commoddity Sep 9, 2024
7b42346
fix: update throughput to rate limit and add capacity
commoddity Sep 9, 2024
c9e0955
fix: remove capacity limit field for now
commoddity Sep 9, 2024
41f5f08
fix: unit tests
commoddity Sep 9, 2024
2043664
chore: update README.md
commoddity Sep 9, 2024
1b821e5
fix: failing e2e test validate
commoddity Sep 9, 2024
2e2195a
fix: failing e2e test hydrate
commoddity Sep 9, 2024
9acc6b1
Merge branch 'main' into db-driver-and-cache
commoddity Sep 9, 2024
99c31a7
Apply suggestions from code review
commoddity Sep 10, 2024
26bd10d
fix: godoc comments in user app
commoddity Sep 10, 2024
a0f9af2
chore: plan type in driver.go
commoddity Sep 10, 2024
8ba5d98
fix: enforce db driver interface
commoddity Sep 10, 2024
aa968ef
fix: unit test for cache
commoddity Sep 10, 2024
b5e6dd4
fix: replace mockery with mockgen and create own mock file
commoddity Sep 10, 2024
581a18b
fix: improve comments for clarity
commoddity Sep 10, 2024
948bc82
chore: comments in docker-compose.yml
commoddity Sep 10, 2024
5ebcea6
chore: update README.md TOC
commoddity Sep 12, 2024
6b3d5b8
fix: temp remove allowlists
commoddity Sep 12, 2024
ba4f115
feat: rename user app to gateway endpoint
commoddity Sep 12, 2024
540dd26
feat: add getters to GatewayEndpoint
commoddity Sep 12, 2024
c7e6573
feat: add getters to GatewayEndpoint
commoddity Sep 12, 2024
8c8ac08
chore: add prerequisites section to readme
commoddity Sep 12, 2024
c675b73
chore: merge conflicts
commoddity Sep 12, 2024
0aa4d28
fix: package name in db/postrges
commoddity Sep 12, 2024
8051622
fix: move docker compose to own folder
commoddity Sep 12, 2024
27bd8a2
fix: address review comments
commoddity Sep 13, 2024
b4cd412
Apply suggestions from code review
commoddity Sep 13, 2024
63ff67b
chore: merge conflicts
commoddity Sep 13, 2024
95a6e7f
chore: move docker-compose.yml and add make targets
commoddity Sep 17, 2024
ff54f48
fix: update router test with mockgen and params
commoddity Sep 17, 2024
d04e9a5
chore: implement review suggestions
commoddity Sep 17, 2024
b903984
fix: remove user data from config
commoddity Sep 17, 2024
84ac602
chore: merge conflicts
commoddity Sep 18, 2024
38a7c95
chore: move helper func to postgres package
commoddity Sep 18, 2024
1bacf35
fix: address review comments
commoddity Sep 18, 2024
d5618cf
Merge branch 'main' into db-driver-and-cache
commoddity Sep 18, 2024
e64ad5a
chore: merge conflicts
commoddity Sep 19, 2024
6ebe8e0
chore: envoy folder structure
commoddity Sep 19, 2024
edfc376
fix: correct docker version in dockerfile
commoddity Sep 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.22.3-alpine3.19 AS builder
FROM golang:1.23-alpine3.19 AS builder
RUN apk add --no-cache git

WORKDIR /go/src/github.com/buildwithgrove/path
Expand Down
25 changes: 22 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,24 @@ help: ## Prints all the targets in all the Makefiles
### Run Path Make Targets ###
#############################

.PHONY: path_up_gateway
path_up_gateway: ## Run just the PATH gateway without any dependencies
docker compose up -d --no-deps path_gateway

.PHONY: path_up_build_gateway
path_up_build_gateway: ## Run and build just the PATH gateway without any dependencies
docker compose up -d --build --no-deps path_gateway

.PHONY: path_up
path_up: ## Run docker compose up
path_up: ## Run the PATH gateway and all related dependencies
docker compose up -d

.PHONY: path_up_build
path_up_build: ## Run docker compose up with build
path_up_build: ## Run and build the PATH gateway and all related dependencies
docker compose up -d --build

.PHONY: path_down
path_down: ## Run docker compose down
path_down: ## Stop the PATH gateway and all related dependencies
docker compose down

#########################
Expand Down Expand Up @@ -59,3 +67,14 @@ copy_test_config: ## copies the example test configuration yaml file to .gitigno
else \
echo ".config.test.yaml already exists, not overwriting."; \
fi

commoddity marked this conversation as resolved.
Show resolved Hide resolved
###############################
### Generation Make Targets ###
###############################

.PHONY: sqlc_generate
sqlc_generate: ## Generate SQLC code from db/driver/sqlc/*.sql files
sqlc generate -f ./db/driver/sqlc/sqlc.yaml
Olshansk marked this conversation as resolved.
Show resolved Hide resolved

# // TODO_TECHDEBT(@commoddity): move all mocks to a shared mocks package
# // TODO_TECHDEBT(@commoddity): Add all other mock generation commands here
60 changes: 33 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,35 @@

# Table of Contents <!-- omit in toc -->

- [Introduction](#introduction)
- [Releases](#path-releases)
- [Quickstart (Shannon)](#quickstart-shannon)
- [Configuration](#configuration)
- [Configuration File](#configuration-file)
- [Example Configuration Format](#example-configuration-format)
- [Running PATH](#running-path)
- [Setup Config YAML](#setup-config-yaml)
- [Start the Container](#start-the-container)
- [E2E Tests](#e2e-tests)
- [Running Tests](#running-tests)

## Introduction
- [1. Introduction](#1-introduction)
- [1.1. Prerequisites](#11-prerequisites)
- [2. Path Releases](#2-path-releases)
- [3. Quickstart (Shannon)](#3-quickstart-shannon)
- [4. Configuration](#4-configuration)
- [4.1. Configuration File](#41-configuration-file)
- [4.2. Example Configuration Format](#42-example-configuration-format)
- [5. Running PATH](#5-running-path)
- [5.1. Setup Config YAML](#51-setup-config-yaml)
- [5.2. Start the Container](#52-start-the-container)
- [6. E2E Tests](#6-e2e-tests)
- [6.1. Running Tests](#61-running-tests)

## 1. Introduction
Olshansk marked this conversation as resolved.
Show resolved Hide resolved

**PATH** (Path API & Toolkit Harness) is an open source framework for enabling access to a decentralized supply network.

It provides various tools and libraries to streamline the integration and interaction with decentralized protocols.

## Path Releases
### 1.1. Prerequisites

- [Docker](https://docs.docker.com/get-docker/)

**Required For Development:**

- [SQLC](https://docs.sqlc.dev/)
- [Mockgen](https://github.com/uber-go/mock)

## 2. Path Releases

Path releases provide a Docker image you can start using right away to bootstrap your Path gateway without the need of building your own image. Images are available in our [Packages](https://github.com/buildwithgrove/path/pkgs/container/path) page. You can pull them directly using the following command:

Expand All @@ -47,7 +57,7 @@ Additionally, our releases contain additional information about what's up with u



## Quickstart (Shannon)
## 3. Quickstart (Shannon)

1. Stake Apps and Gateway

Expand Down Expand Up @@ -77,9 +87,9 @@ Additionally, our releases contain additional information about what's up with u

_For detailed instructions on running PATH, see the [Running PATH](#running-path) section._

## Configuration
## 4. Configuration

### Configuration File
### 4.1. Configuration File

The configuration for PATH is defined in a YAML file, which should be named `.config.yaml`.

Expand Down Expand Up @@ -110,11 +120,7 @@ The configuration is divided into several sections:
- _Optional. Default values will be used if not specified._
- Configures router settings such as port and timeouts.

5. **User Data Configuration (`user_data_config`)**:
- _Required only if the gateway operator wishes to associate user data with requests._
- Configures the PostgreSQL database connection string.

#### Example Configuration Format
### 4.2. Example Configuration Format

```yaml
shannon_config:
Expand All @@ -138,9 +144,9 @@ services:
- [Shannon](./cmd/config/testdata/shannon.example.yaml)
- [Config YAML Schema](./config/config.schema.yaml)

## Running PATH
## 5. Running PATH

#### Setup Config YAML
### 5.1. Setup Config YAML

- The PATH service requires the config YAML file to be populated.

Expand All @@ -156,7 +162,7 @@ services:

**⚠️ IMPORTANT: The data required to populate the `.config.yaml` file is sensitive and the contents of this file must never be shared outside of your organization. ⚠️**

#### Start the Container
### 5.2. Start the Container

1. Once the `.config.yaml` file is populated, to start the PATH service for a specific protocol, use the `make` target:

Expand All @@ -176,7 +182,7 @@ services:
make path_down
```

## E2E Tests
## 6. E2E Tests

This repository contains end-to-end (E2E) tests for the Shannon relay protocol. The tests ensure that the protocol behaves as expected under various conditions.

Expand All @@ -194,7 +200,7 @@ Currently, the E2E tests are configured to run against the Shannon testnet.

Future work will include adding support for other protocols.

#### Running Tests
### 6.1. Running Tests

To run the tests, use the following `make` targets:

Expand Down
2 changes: 1 addition & 1 deletion cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ func main() {
log.Fatalf("failed to create API router: %v", err)
}

if err := apiRouter.Start(context.Background()); err != nil {
if err := apiRouter.Start(); err != nil {
log.Fatalf("failed to start API router: %v", err)
}
}
5 changes: 0 additions & 5 deletions config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ type (

Services map[relayer.ServiceID]ServiceConfig `yaml:"services"`
Router RouterConfig `yaml:"router_config"`
UserData UserDataConfig `yaml:"user_data_config"`

// A map from human readable aliases (e.g. eth-mainnet) to service ID (e.g. 0021)
serviceAliases map[string]relayer.ServiceID
Expand Down Expand Up @@ -75,10 +74,6 @@ func (c GatewayConfig) GetRouterConfig() RouterConfig {
return c.Router
}

func (c GatewayConfig) GetUserDataConfig() UserDataConfig {
return c.UserData
}

// GetServiceIDFromAlias retrieves the ServiceID associated with a given service alias.
//
// This method allows for the use of a user-friendly string service alias in the
Expand Down
7 changes: 0 additions & 7 deletions config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,6 @@ func Test_LoadGatewayConfigFromYAML(t *testing.T) {
WriteTimeout: 5000 * time.Millisecond,
IdleTimeout: 5000 * time.Millisecond,
},
UserData: UserDataConfig{
DBConnectionString: "postgres://user:password@localhost:5432/database",
},
serviceAliases: map[string]relayer.ServiceID{
"eth-mainnet": "0021",
},
Expand Down Expand Up @@ -108,9 +105,6 @@ func Test_LoadGatewayConfigFromYAML(t *testing.T) {
WriteTimeout: 5000 * time.Millisecond,
IdleTimeout: 5000 * time.Millisecond,
},
UserData: UserDataConfig{
DBConnectionString: "postgres://user:password@localhost:5432/database",
},
serviceAliases: map[string]relayer.ServiceID{
"eth-mainnet": "0021",
},
Expand Down Expand Up @@ -320,7 +314,6 @@ func Test_GetServiceIDFromAlias(t *testing.T) {

func compareConfigs(c *require.Assertions, want, got GatewayConfig) {
c.Equal(want.Router, got.Router)
c.Equal(want.UserData, got.UserData)
if want.MorseConfig != nil {
compareMorseFullNodeConfig(c, want.MorseConfig.FullNodeConfig, got.MorseConfig.FullNodeConfig)
c.Equal(want.MorseConfig.SignedAATs, got.MorseConfig.SignedAATs)
Expand Down
4 changes: 0 additions & 4 deletions config/testdata/morse.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,3 @@ router_config:
read_timeout: "5000ms"
write_timeout: "5000ms"
idle_timeout: "5000ms"

# user_data_config is required if user data is to be stored in a database.
user_data_config:
db_connection_string: "postgres://user:password@localhost:5432/database"
4 changes: 0 additions & 4 deletions config/testdata/shannon.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,3 @@ router_config:
read_timeout: "5000ms"
write_timeout: "5000ms"
idle_timeout: "5000ms"

# user_data_config is required if user data is to be stored in a database.
user_data_config:
db_connection_string: "postgres://user:password@localhost:5432/database"
39 changes: 0 additions & 39 deletions config/user_data.go

This file was deleted.

82 changes: 0 additions & 82 deletions config/user_data_test.go

This file was deleted.

7 changes: 0 additions & 7 deletions config/utils/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,3 @@ func IsValidURL(s string) bool {
}
return u.Scheme == "http" || u.Scheme == "https"
}

// IsValidDBConnectionString checks if a string is a valid PostgreSQL connection string.
func IsValidDBConnectionString(s string) bool {
// Regular expression to match a valid PostgreSQL connection string
var dbConnStringRegex = regexp.MustCompile(`^postgres://[^:]+:[^@]+@[^:]+:\d+/.+$`)
return dbConnStringRegex.MatchString(s)
}
Loading
Loading