Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support fifo queue #248

Merged
merged 3 commits into from
Nov 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ updates:
interval: "daily"
labels:
- "dependencies"
- "go"
- "go"
6 changes: 3 additions & 3 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ name: Build and Deploy

on:
schedule:
- cron: '0 05 * * *'
- cron: "0 05 * * *"
push:
branches:
- main
tags:
- 'v*.*.*'
- "v*.*.*"
pull_request:
branches:
- 'main'
- "main"

jobs:
docker:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/server.test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ jobs:
- uses: actions/checkout@v3
- name: test
working-directory: server
run: go test -v ./...
run: go test -v ./...
31 changes: 15 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,42 @@
# SQS-Admin #
# SQS-Admin

A minimal and lightweight UI for managing SQS-Queues for local development e.g. with [Localstack](https://localstack.cloud/).

![Sqs-Admin](screenshot.png)

## Why ##
## Why

There are already good UIs for SQS, but they are heavy with sizes >100 MB. Most likely because they ship with SQS itself.
If you choose e.g. Localstack for local development you don't need an additional local SQS setup, as it is already
provided by Localstack, unfortunately without UI.
This Alpine based image has a size ~15 MB. You can easily manage and create Queues.

## Usage ##
## Usage

The most common way to use SQS-Admin would be in conjunction with a ``docker-compose.yml``.
A working example can be found in the ``example`` directory.
The most common way to use SQS-Admin would be in conjunction with a `docker-compose.yml`.
A working example can be found in the `example` directory.

You probably need to have a SQS up and running somewhere to connect to, e.g. via Localstack.
To start SQS-Admin simply run:
``
docker run --rm -p 3999:3999 -e SQS_ENDPOINT_URL=<Endpoint-URL-of-our-SQS> -d pacovk/sqs-admin
``
`docker run --rm -p 3999:3999 -e SQS_ENDPOINT_URL=<Endpoint-URL-of-our-SQS> -d pacovk/sqs-admin`

## Configuration ##
## Configuration

You can easily configure the Docker Container via the following environment variables:

| ENV | Description | Default |
|------------------|----------------------------------------------------------------|-----------------------|
| ---------------- | -------------------------------------------------------------- | --------------------- |
| SQS_ENDPOINT_URL | **Endpoint where SQS is running, this one is mostly required** | http://localhost:4566 |
| SQS_AWS_REGION | AWS region the client internally uses to interact with SQS | eu-central-1 |

## Development ##
## Development

To configure the backend for local development you can set the following environment variable:

| ENV | Description | Default |
|------------------|----------------------------------------------------------------|-----------------------|
| HTTP_PORT | Port that the internal backend binds to and is serving | 3999 |
| ENV | Description | Default |
| --------- | ------------------------------------------------------ | ------- |
| HTTP_PORT | Port that the internal backend binds to and is serving | 3999 |

### Legal note ###
### Legal note

UI favicon by [John Sorrentino](https://favicon.io/emoji-favicons/cowboy-hat-face)
UI favicon by [John Sorrentino](https://favicon.io/emoji-favicons/cowboy-hat-face)
5 changes: 2 additions & 3 deletions example/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
version: '3.7'
version: "3.7"
services:

sqs-admin:
image: pacovk/sqs-admin
image: pacovk/sqs-admin
ports:
- "3999:3999"
environment:
Expand Down
43 changes: 39 additions & 4 deletions frontend/components/CreateQueueDialog.tsx
Original file line number Diff line number Diff line change
@@ -1,34 +1,57 @@
import React, { useState } from "react";
import {
Button,
Checkbox,
Dialog,
DialogActions,
DialogContent,
DialogContentText,
DialogTitle,
FormControlLabel,
FormGroup,
TextField,
} from "@mui/material";
import { CreateQueueDialogProps } from "../types";
import { CreateQueueDialogProps, Queue } from "../types";

const CreateQueueDialog = (props: CreateQueueDialogProps) => {
const [open, setOpen] = useState(false);
const [queueName, setQueueName] = useState("");
const [isFifoQueue, enableFifoQueue] = useState(false);

const handleClickOpen = () => {
setQueueName("");
enableFifoQueue(false);
setOpen(true);
};

const handleClose = () => {
setOpen(false);
};

const handleChange = (event: React.ChangeEvent<HTMLInputElement>) => {
const handleChangeQueueName = (
event: React.ChangeEvent<HTMLInputElement>
) => {
setQueueName(event.target.value);
};

const handleChangeFifoSwitch = (
event: React.ChangeEvent<HTMLInputElement>
) => {
enableFifoQueue(event.target.checked);
};

const submitCreateQueue = () => {
props.onSubmit(queueName);
let sanitizedQueueName = queueName;
if (isFifoQueue && !queueName.endsWith(".fifo")) {
sanitizedQueueName = `${sanitizedQueueName}.fifo`;
}
const newQueue: Queue = {
QueueName: sanitizedQueueName,
QueueAttributes: {
FifoQueue: `${isFifoQueue}`,
},
};
props.onSubmit(newQueue);
handleClose();
};

Expand All @@ -51,9 +74,21 @@ const CreateQueueDialog = (props: CreateQueueDialogProps) => {
type="text"
fullWidth
value={queueName}
onChange={handleChange}
onChange={handleChangeQueueName}
variant="standard"
/>
<FormGroup>
<FormControlLabel
control={
<Checkbox
checked={isFifoQueue}
onChange={handleChangeFifoSwitch}
inputProps={{ "aria-label": "controlled" }}
/>
}
label="FIFO Queue"
/>
</FormGroup>
</DialogContent>
<DialogActions>
<Button onClick={handleClose}>Cancel</Button>
Expand Down
23 changes: 23 additions & 0 deletions frontend/components/SendMessageDialog.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,12 @@ import { SendMessageDialogProps } from "../types";
const SendMessageDialog = (props: SendMessageDialogProps) => {
const [open, setOpen] = useState(false);
const [messageBody, setMessageBody] = useState("");
const [messageGroupId, setMessageGroupId] = useState("");

const handleClickOpen = () => {
setMessageBody("");
setOpen(true);
setMessageGroupId("");
};

const handleClose = () => {
Expand All @@ -27,9 +29,14 @@ const SendMessageDialog = (props: SendMessageDialogProps) => {
setMessageBody(event.target.value);
};

const handleGroupIdChange = (event: React.ChangeEvent<HTMLInputElement>) => {
setMessageGroupId(event.target.value);
};

const submitSendRequest = () => {
props.onSubmit({
messageBody: messageBody,
messageGroupId: messageGroupId === "" ? undefined : messageGroupId,
});
handleClose();
};
Expand Down Expand Up @@ -60,6 +67,22 @@ const SendMessageDialog = (props: SendMessageDialogProps) => {
onChange={handleChange}
variant="standard"
/>
<TextField
margin="dense"
id="messageGroupId"
label="Message-Group-Id"
type="text"
fullWidth
required={true}
value={messageGroupId}
onChange={handleGroupIdChange}
style={{
display: props.queue?.QueueName.endsWith(".fifo")
? "flex"
: "none",
}}
variant="standard"
/>
</DialogContent>
<DialogActions>
<Button onClick={handleClose}>Cancel</Button>
Expand Down
7 changes: 4 additions & 3 deletions frontend/types/index.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { AlertColor } from "@mui/material";
import { MessageAttributeValue } from "@aws-sdk/client-sqs";

export interface TabPanelProps {
children?: React.ReactNode;
Expand All @@ -8,12 +7,13 @@ export interface TabPanelProps {
}

export interface CreateQueueDialogProps {
onSubmit: (queueName: string) => void;
onSubmit: (queue: Queue) => void;
}

export interface SendMessageDialogProps {
onSubmit: (message: SqsMessage) => void;
disabled: boolean;
queue: Queue;
}

export interface AlertProps {
Expand All @@ -23,15 +23,16 @@ export interface AlertProps {
}

export interface Queue {
QueueUrl: string;
QueueName: string;
QueueUrl?: string;
QueueAttributes?: { [key: string]: string } | undefined;
}

export interface SqsMessage {
messageBody: string;
messageId?: string;
messageAttributes?: { [key: string]: string } | undefined;
messageGroupId?: string;
}

export interface ApiCall {
Expand Down
17 changes: 12 additions & 5 deletions frontend/views/Overview.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -78,14 +78,11 @@ const Overview = () => {
}
};

const createNewQueue = async (queueName: string) => {
const createNewQueue = async (queue: Queue) => {
await callApi({
method: "POST",
action: "CreateQueue",
queue: {
QueueUrl: "",
QueueName: queueName,
},
queue: queue,
onSuccess: () => {
setTimeout(() => {
triggerReload(!reload);
Expand Down Expand Up @@ -125,6 +122,15 @@ const Overview = () => {
const sendMessageToCurrentQueue = async (message: SqsMessage) => {
let queueUrl = queues[tabIndex]?.QueueUrl || null;
if (queueUrl !== null) {
if (
queues[tabIndex]?.QueueName.endsWith(".fifo") &&
!message.messageGroupId
) {
setError(
"You need to set a MessageGroupID when sending Messages to a FIFO queue"
);
return;
}
await callApi({
method: "POST",
action: "SendMessage",
Expand Down Expand Up @@ -163,6 +169,7 @@ const Overview = () => {
<SendMessageDialog
disabled={disabledStatus}
onSubmit={sendMessageToCurrentQueue}
queue={queues[tabIndex]}
/>
<Button
variant="contained"
Expand Down
16 changes: 12 additions & 4 deletions server/aws/sqsClient.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"github.com/aws/aws-sdk-go-v2/service/sqs"
awsTypes "github.com/aws/aws-sdk-go-v2/service/sqs/types"
"github.com/google/uuid"
"github.com/pacoVK/aws/types"
"log"
"strings"
Expand Down Expand Up @@ -81,15 +82,22 @@ func DeleteQueue(queueUrl string) (*sqs.DeleteQueueOutput, error) {
})
}

func CreateQueue(queueName string) (*sqs.CreateQueueOutput, error) {
func CreateQueue(queueName string, attributes *map[string]string) (*sqs.CreateQueueOutput, error) {
return sqsClient.CreateQueue(context.TODO(), &sqs.CreateQueueInput{
QueueName: &queueName,
QueueName: &queueName,
Attributes: *attributes,
})
}

func SendMessage(queueUrl string, sqsMessage types.SqsMessage) (*sqs.SendMessageOutput, error) {
deduplicationId := sqsMessage.MessageGroupId
if len(deduplicationId) > 0 {
deduplicationId = uuid.New().String()
}
return sqsClient.SendMessage(context.TODO(), &sqs.SendMessageInput{
QueueUrl: &queueUrl,
MessageBody: &sqsMessage.MessageBody,
QueueUrl: &queueUrl,
MessageBody: &sqsMessage.MessageBody,
MessageGroupId: &sqsMessage.MessageGroupId,
MessageDeduplicationId: &deduplicationId,
})
}
1 change: 1 addition & 0 deletions server/aws/types/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ type SqsMessage struct {
MessageId string `json:"messageId"`
MessageBody string `json:"messageBody"`
MessageAttributes map[string]string `json:"messageAttributes"`
MessageGroupId string `json:"messageGroupId,omitempty"`
}

type Request struct {
Expand Down
3 changes: 1 addition & 2 deletions server/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
version: '3.7'
version: "3.7"
services:

localstack:
image: localstack/localstack
ports:
Expand Down
1 change: 1 addition & 0 deletions server/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,5 @@ require (
github.com/aws/aws-sdk-go-v2/service/sts v1.17.1 // indirect
github.com/aws/smithy-go v1.13.4 // indirect
github.com/felixge/httpsnoop v1.0.1 // indirect
github.com/google/uuid v1.3.0 // indirect
)
2 changes: 2 additions & 0 deletions server/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ github.com/felixge/httpsnoop v1.0.1 h1:lvB5Jl89CsZtGIWuTcDM1E/vkVs49/Ml7JJe07l8S
github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/handlers v1.5.1 h1:9lRY6j8DEeeBT10CvO9hGW0gmky0BprnvDI5vfhUHH4=
github.com/gorilla/handlers v1.5.1/go.mod h1:t8XrUpc4KVXb7HGyJ4/cEnwQiaxrX/hz1Zv/4g96P1Q=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
Expand Down
4 changes: 2 additions & 2 deletions server/handler/sqsHandler.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ func SQSHandler() Handler {
payload := unpackRequestPayload(request.Body)
switch payload.Action {
case "CreateQueue":
log.Printf("Creating queue [%v]", payload.SqsQueue.QueueName)
_, err := aws.CreateQueue(payload.SqsQueue.QueueName)
log.Printf("Creating queue [%v] with attributes [%v]", payload.SqsQueue.QueueName, payload.SqsQueue.QueueAttributes)
_, err := aws.CreateQueue(payload.SqsQueue.QueueName, payload.SqsQueue.QueueAttributes)
checkForErrorAndRespondJSON(&writer, Response{
Payload: nil,
Error: err,
Expand Down
Loading