Skip to content

Commit

Permalink
feat(frontend): add gestion fichiers (#237)
Browse files Browse the repository at this point in the history
* feat(frontend): add gestion fichiers

* make fake key more explicit

* oups

* feat(frontend): much nicer code

* wip

* wip

* wip

* wip

* hum

* wip

* ci: update configmap container name

* ci: fix helm args

* fix(frontend): remove unused end

Co-authored-by: Julien Bouquillon <[email protected]>
Co-authored-by: LionelB <[email protected]>
  • Loading branch information
3 people authored Jan 4, 2021
1 parent c03e09f commit 881f2c1
Show file tree
Hide file tree
Showing 21 changed files with 844 additions and 9 deletions.
3 changes: 3 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ SMTP_URL=smtp.url
SMTP_EMAIL_USER=email
SMTP_EMAIL_PASSWORD=pass

AZURE_STORAGE_ACCOUNT_KEY=accountKey
AZURE_STORAGE_ACCOUNT_NAME=cdtnadmindev

##
## Shared secret between hasura and frontend
##
Expand Down
2 changes: 2 additions & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ Create namespace:
# Copy namespace default creds
- kubectl get secret cdtn-admin-secrets --namespace=cdtn-admin-secret --export -o yaml |
kubectl apply --namespace=${K8S_NAMESPACE} -f -
- kubectl get secret azure-cdtnadmindev-volume --namespace=cdtn-admin-secret --export -o yaml |
kubectl apply --namespace=${K8S_NAMESPACE} -f -

Create Azure DB (dev):
extends: .autodevops_create_azure_db_dev
Expand Down
3 changes: 3 additions & 0 deletions targets/frontend/.gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,11 @@ Deploy www (dev):
name: ${CI_COMMIT_REF_SLUG}-dev2
variables:
MANIFEST_FOLDER: targets/frontend/.k8s/environments/dev
# secrets have a different name in dev or prod...
HELM_RENDER_ARGS: >-
--set ingress.annotations.nginx\.ingress\.kubernetes\.io\/whitelist-source-range=$IP_ALLOWLIST
--set deployment.env[0].valueFrom.secretKeyRef.name=azure-cdtnadmindev-volume
--set deployment.env[1].valueFrom.secretKeyRef.name=azure-cdtnadmindev-volume
Deploy www (prod):
extends:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ data:
CI_COMMIT_SHORT_SHA: "${CI_COMMIT_SHORT_SHA}"
HASURA_GRAPHQL_ENDPOINT: "http://hasura-cdtn-admin/v1/graphql"
NEXT_PUBLIC_ACTIVATION_TOKEN_EXPIRES: "10080"
NEXT_PUBLIC_CONTAINER_NAME: "cdtn-dev"
NODE_ENV: "production"
REFRESH_TOKEN_EXPIRES: "43200"
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ data:
CI_COMMIT_SHORT_SHA: "${CI_COMMIT_SHORT_SHA}"
HASURA_GRAPHQL_ENDPOINT: "http://hasura-cdtn-admin/v1/graphql"
NEXT_PUBLIC_ACTIVATION_TOKEN_EXPIRES: "10080"
NEXT_PUBLIC_CONTAINER_NAME: "cdtn"
NODE_ENV: "production"
PRODUCTION: "true"
REFRESH_TOKEN_EXPIRES: "43200"
12 changes: 12 additions & 0 deletions targets/frontend/.k8s/www.values.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,18 @@ deployment:
path: /health
periodSeconds: 15

env:
- name: AZURE_STORAGE_ACCOUNT_KEY
valueFrom:
secretKeyRef:
name: azure-cdtnadminprod-volume
key: azurestorageaccountkey
- name: AZURE_STORAGE_ACCOUNT_NAME
valueFrom:
secretKeyRef:
name: azure-cdtnadminprod-volume
key: azurestorageaccountname

envFrom:
- configMapRef:
name: www-env
Expand Down
33 changes: 33 additions & 0 deletions targets/frontend/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Gestion des fichiers dans les différents environemments

Les url des fichiers sauvegardée en base sont relatives.
Pour chaque environnement, il faudra spécifier, l'url du serveur ainsi que le nom du container.

## Organisation

### cdtn

| déploiement | container | instance azure |
| ----------- | --------- | -------------- |
| branche | cdtn-dev | dev |
| master | cdtn | dev |
| preprod | cdtn | dev |
| prod | cdtn | prod |

### cdtn-admin

| déploiement | container | instance |
| ----------- | --------- | -------- |
| branche | cdtn-dev | dev |
| master | cdtn | dev |

## Synchronisation (job ci de copie)

### cdtn

**mep** : copie des fichiers cdtn (dev) › cdtn (prod)
**ingest (prod)** : copie les fichier de cdtn (dev) › cdtn (prod)

### cdtn-admin

**branche**: copie de cdtn (dev) › cdtn-dev (dev)
5 changes: 5 additions & 0 deletions targets/frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"name": "frontend",
"version": "1.0.0",
"dependencies": {
"@azure/abort-controller": "^1.0.1",
"@azure/storage-blob": "^12.3.0",
"@elastic/elasticsearch": "^7.10.0",
"@hapi/boom": "^9.1.1",
"@hapi/joi": "^17.1.1",
Expand All @@ -28,6 +30,7 @@
"d3-hierarchy": "^2.0.0",
"date-fns": "^2.16.1",
"diff": "^5.0.0",
"formidable": "^1.2.2",
"graphql": "^15.4.0",
"http-proxy-middleware": "^1.0.6",
"isomorphic-unfetch": "^3.1.0",
Expand All @@ -40,10 +43,12 @@
"next-urql": "^2.1.1",
"nodemailer": "^6.4.17",
"polished": "^4.0.5",
"pretty-bytes": "^5.4.1",
"react": "^16.14.0",
"react-ace": "^9.2.1",
"react-autosuggest": "^10.0.4",
"react-dom": "^16.14.0",
"react-dropzone": "^11.2.4",
"react-hook-form": "^6.14.0",
"react-icons": "^3.11.0",
"react-is": "^16.13.1",
Expand Down
62 changes: 62 additions & 0 deletions targets/frontend/src/components/button/CopyButton.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/** @jsx jsx */

import PropTypes from "prop-types";
import { useEffect, useState } from "react";
import { IoMdCheckmark, IoMdClipboard } from "react-icons/io";
import { Button } from "src/components/button";
import { jsx } from "theme-ui";

export const CopyButton = ({
onClip,
copied: optionalCopiedProp = false,
text,
...props
}) => {
const [copied, setCopied] = useState(optionalCopiedProp);
const [hasClipboardApi, setHasClipboardApi] = useState(false);
useEffect(() => {
setCopied(optionalCopiedProp);
}, [optionalCopiedProp, setCopied]);
useEffect(() => {
setHasClipboardApi(Boolean(navigator?.clipboard));
}, [setHasClipboardApi]);

return hasClipboardApi ? (
<Button
{...props}
disabled={copied}
onClick={(evt) => {
evt.preventDefault();
navigator.clipboard.writeText(text).then(() => {
setCopied(true);
if (onClip) {
onClip(text);
}
});
}}
>
{copied ? (
<>
<IoMdCheckmark sx={iconSx} /> Copié !
</>
) : (
<>
<IoMdClipboard sx={iconSx} />
Copier
</>
)}
</Button>
) : null;
};

const iconSx = {
height: "iconSmall",
mr: "xxsmall",
width: "iconSmall",
};

CopyButton.propTypes = {
copied: PropTypes.bool,
onClip: PropTypes.func,
text: PropTypes.string.isRequired,
};
7 changes: 7 additions & 0 deletions targets/frontend/src/components/layout/Nav.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import { useQuery } from "urql";

import { Li, List } from "../list";

const CONTAINER_NAME = process.env.NEXT_PUBLIC_CONTAINER_NAME || "cdtn-dev";

const getSourcesQuery = `
query getAlerts{
sources(order_by:{label:asc}) {
Expand Down Expand Up @@ -115,6 +117,11 @@ export function Nav() {
Blocs KALI
</ActiveLink>
</Li>
<Li>
<ActiveLink href={`/storage/${CONTAINER_NAME}`} passHref>
Fichiers
</ActiveLink>
</Li>
</List>
</Box>
</Box>
Expand Down
1 change: 1 addition & 0 deletions targets/frontend/src/components/layout/auth.layout.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,6 @@ export function Layout({ children, title }) {
}
Layout.propTypes = {
children: PropTypes.node,
noStack: PropTypes.bool,
title: PropTypes.string.isRequired,
};
51 changes: 51 additions & 0 deletions targets/frontend/src/components/storage/DropZone.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/** @jsx jsx */

import PropTypes from "prop-types";
import { useCallback } from "react";
import { useDropzone } from "react-dropzone";
import { jsx, Spinner } from "theme-ui";

const defaultStyles = {
border: "2px dotted silver",
borderRadius: "small",
p: "medium",
};

export function DropZone({ onDrop: onDropCallback, uploading, customStyles }) {
const onDrop = useCallback(
async (acceptedFiles) => {
const formData = new FormData();
for (const i in acceptedFiles) {
if (acceptedFiles[i] instanceof File) {
formData.append(acceptedFiles[i].path, acceptedFiles[i]);
}
}
onDropCallback(formData);
},
[onDropCallback]
);

const { getRootProps, getInputProps, isDragAccept } = useDropzone({
accept:
"image/jpeg, image/png, application/pdf, application/vnd.openxmlformats-officedocument.wordprocessingml.document",
onDrop,
});

if (isDragAccept) {
defaultStyles.backgroundColor = "dropZone";
} else {
delete defaultStyles.backgroundColor;
}
return (
<div {...getRootProps()} sx={{ ...defaultStyles, ...customStyles }}>
<input {...getInputProps()} />
{uploading ? <Spinner /> : <p>Glissez vos fichiers ici</p>}
</div>
);
}

DropZone.propTypes = {
customStyles: PropTypes.object,
onDrop: PropTypes.func.isRequired,
uploading: PropTypes.bool.isRequired,
};
59 changes: 59 additions & 0 deletions targets/frontend/src/lib/azure.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import { AbortController } from "@azure/abort-controller";
import {
BlobServiceClient,
StorageSharedKeyCredential,
} from "@azure/storage-blob";

const AZURE_STORAGE_ACCOUNT = {
key: process.env.AZURE_STORAGE_ACCOUNT_KEY || "",
name: process.env.AZURE_STORAGE_ACCOUNT_NAME || "",
};

export const getBlobContainer = (containerName) => {
const sharedKeyCredential = new StorageSharedKeyCredential(
AZURE_STORAGE_ACCOUNT.name,
AZURE_STORAGE_ACCOUNT.key
);

const service = new BlobServiceClient(
`https://${AZURE_STORAGE_ACCOUNT.name}.blob.core.windows.net`,
sharedKeyCredential
);

return service.getContainerClient(containerName);
};

export const getContainerBlobs = async (containerName) => {
const container = getBlobContainer(containerName);
const blobs = [];
for await (const blob of container.listBlobsFlat()) {
blobs.push({
contentLength: blob.properties.contentLength,
lastModified: blob.properties.lastModified,
name: blob.name,
url: `https://${AZURE_STORAGE_ACCOUNT.name}.blob.core.windows.net/${containerName}/${blob.name}`,
});
}
return blobs;
};

export const deleteBlob = async (containerName, blobName) => {
const container = getBlobContainer(containerName);
const client = container.getBlockBlobClient(blobName);
return client.delete();
};

export const uploadBlob = async (containerName, stream) => {
const container = getBlobContainer(containerName);
const client = container.getBlockBlobClient(stream.name);
const blockSize = 4 * 1024 * 1024;
const timeout = 30 * 60 * 1000;
const concurrency = 20;
const options = { abortSignal: AbortController.timeout(timeout) };
try {
return client.uploadStream(stream, blockSize, concurrency, options);
} catch (err) {
console.log("Error", err);
throw new Error("Error while uploading the file to azure");
}
};
6 changes: 6 additions & 0 deletions targets/frontend/src/lib/duration.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
import { formatDistanceToNow, parseISO } from "date-fns";
import frLocale from "date-fns/locale/fr";

export function toMs(minutes = 0) {
return toSecond(minutes) * 1000;
}
Expand All @@ -9,3 +12,6 @@ export function toSecond(minutes = 0) {
export function getExpiryDate(minutes = 0) {
return new Date(Date.now() + toMs(minutes));
}

export const timeSince = (date) =>
formatDistanceToNow(parseISO(date), { locale: frLocale });
8 changes: 7 additions & 1 deletion targets/frontend/src/lib/request.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,13 @@ export function request(endpoint, { body, ...customConfig } = {}) {
},
};
if (body) {
config.body = JSON.stringify(body);
if (typeof FormData !== "undefined" && body instanceof FormData) {
config.body = body;
// auto set by the browser with its specific multipart/form-data boundaries
delete config.headers["Content-Type"];
} else {
config.body = JSON.stringify(body);
}
}
return fetch(endpoint, config).then(async (response) => {
const data = await response.json();
Expand Down
Loading

0 comments on commit 881f2c1

Please sign in to comment.