Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(service/storage): rewrite structure #391

Merged
merged 43 commits into from
Nov 15, 2022
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
62489e3
refactor(service/storage): rewrite structure
alimd Nov 13, 2022
8073773
feat(service/storage): add requireToken to AlwatrConnection
njfamirm Nov 13, 2022
5b4160d
fix(service/storage): remove home route
njfamirm Nov 13, 2022
1eadd68
feat(nano-server): parse and validate query params in AlwatrConnection
njfamirm Nov 13, 2022
cae77a3
chore(.gitignore): ignore db folder
njfamirm Nov 13, 2022
d4452cd
feat(service/storage): add remove route
njfamirm Nov 13, 2022
b3c26c0
fix(storage): get data keys on keys
njfamirm Nov 13, 2022
cba790f
fix(nano-server): parse body on PATCH method
njfamirm Nov 13, 2022
b6454a5
feat(storage): add data method
njfamirm Nov 13, 2022
93d9418
refactor(service/storage): use requireQueryParams
njfamirm Nov 13, 2022
46fb970
feat(service/storage): add keys route
njfamirm Nov 13, 2022
a698e1d
feat(service/storage): add all route
njfamirm Nov 13, 2022
bf02e99
fix(service/storage): update demo request
njfamirm Nov 13, 2022
b2724f4
fix(service/storage): remove id type from requireQueryParams
njfamirm Nov 14, 2022
8e47296
Merge branch 'next' into feat/storage
njfamirm Nov 14, 2022
5b8ed75
Merge branch 'next' into feat/storage
njfamirm Nov 14, 2022
61b78ac
fix(service/storage): duplicate _data property
njfamirm Nov 14, 2022
b20afa1
fix(nano-server): method all
alimd Nov 14, 2022
5bb1b73
refactor(nano-server): AlwatrConnection methods
alimd Nov 14, 2022
d99eb74
feat(nano-server): validate content type header in requireJsonBody
njfamirm Nov 14, 2022
bd87dfc
refactor(nano-server): requireToken
alimd Nov 14, 2022
0cb9e34
refactor(nano-server): requireQueryParams
alimd Nov 14, 2022
f536b52
Merge branch 'feat/storage' of github.com:AliMD/alwatr into feat/storage
alimd Nov 14, 2022
1f67c07
fix(nano-server): remove ALL from Methods type
njfamirm Nov 14, 2022
46edbc3
feat(nano-server): add _sanitizeParam
njfamirm Nov 14, 2022
61d4a2a
fix(nano-server): set correct valitator type
njfamirm Nov 14, 2022
68d4593
refactor(service/storage): use new requireQueryParams
njfamirm Nov 14, 2022
38b5fa9
Merge branch 'next' into feat/storage
alimd Nov 15, 2022
a7e4bae
fix(nano-server): review require... methods and fix issues
alimd Nov 15, 2022
c8a7f02
Merge branch 'next' into feat/storage
alimd Nov 15, 2022
ca60e71
docs(nano-server): requireQueryParams, requireToken
njfamirm Nov 15, 2022
2ce8f2d
docs(nano-server): update readme from jsdoc
njfamirm Nov 15, 2022
45e04a2
docs(nano-server): demo.http
njfamirm Nov 15, 2022
97397f9
fix(storage): change remove to delete
njfamirm Nov 15, 2022
658426a
fix(ns/storage): review and fix issues
alimd Nov 15, 2022
f114104
refactor(ns/storage): rename route files
alimd Nov 15, 2022
73b287c
feat(ns/storage): home page
alimd Nov 15, 2022
1e46b80
docs(ns/storage): update readme
alimd Nov 15, 2022
393e542
docs(nano-server): getBody
njfamirm Nov 15, 2022
98aeb00
fix(ns/storage): home page issue
alimd Nov 15, 2022
e13f464
chore(ns/storage): update demo.http
alimd Nov 15, 2022
def1bf9
Merge branch 'feat/storage' of github.com:AliMD/alwatr into feat/storage
alimd Nov 15, 2022
ad5588c
fix(nano-server): no-non-null-assertion
alimd Nov 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 77 additions & 19 deletions packages/core/nano-server/src/nano-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -177,9 +177,9 @@ export class AlwatrNanoServer {

const middleware =
this.middlewareList[connection.method]?.[route] ||
this.middlewareList.all[route] ||
this.middlewareList.ALL[route] ||
this.middlewareList[connection.method]?.all ||
this.middlewareList.all.all;
this.middlewareList.ALL.all;

try {
if (typeof middleware === 'function') {
Expand Down Expand Up @@ -234,17 +234,7 @@ export class AlwatrConnection {
* Request method.
*/
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
readonly method = this.incomingMessage.method!.toUpperCase() as Methods;

/**
* The token placed in the request header.
*/
readonly token = this._getToken();

/**
* Request body for POST & PUT method.
*/
readonly bodyPromise = this._getRequestBody();
readonly method = this.incomingMessage.method!.toUpperCase() as Exclude<Methods, 'ALL'>;
Fixed Show fixed Hide fixed

protected _logger = createLogger(`alwatr-nano-server-connection`);

Expand Down Expand Up @@ -314,7 +304,10 @@ export class AlwatrConnection {
this.serverResponse.end();
}

protected _getToken(): string | null {
/**
* Get the token placed in the request header.
*/
getToken(): string | null {
const auth = this.incomingMessage.headers.authorization?.split(' ');

if (auth == null || auth[0] !== 'Bearer') {
Expand All @@ -324,9 +317,12 @@ export class AlwatrConnection {
return auth[1];
}

protected async _getRequestBody(): Promise<string | null> {
/**
* Get request body.
*/
async getBody(): Promise<string | null> {
// method must be POST or PUT
if (!(this.method === 'POST' || this.method === 'PUT')) {
if (!(this.method === 'POST' || this.method === 'PUT' || this.method === 'PATCH')) {
return null;
}

Expand All @@ -352,10 +348,19 @@ export class AlwatrConnection {
* ```
*/
async requireJsonBody<Type extends Record<string, unknown>>(): Promise<Type | null> {
// if request content type is json, parse the body
const body = await this.bodyPromise;
// if request content type is json
if (this.incomingMessage.headers['content-type'] !== 'application/json') {
this.reply({
ok: false,
statusCode: 400,
errorCode: 'require_body_json',
});
return null;
}

const body = await this.getBody();

if (body === null || body.length === 0) {
if (body == null || body.length === 0) {
this.reply({
ok: false,
statusCode: 400,
Expand All @@ -376,4 +381,57 @@ export class AlwatrConnection {
return null;
}
}

requireToken(validator?: (token: string) => boolean | Array<string> | string): string | null {
const token = this.getToken();

if (token == null) {
this.reply({
ok: false,
statusCode: 401,
errorCode: 'authorization_required',
});
return null;
}
else if (validator === undefined) return token;
else if (typeof validator === 'string') {
if (token === validator) return token;
}
else if (Array.isArray(validator)) {
if (validator.includes(token)) return token;
}
else if (typeof validator === 'function') {
if (validator(token) === true) return token;
}

this.reply({
ok: false,
statusCode: 403,
errorCode: 'access_denied',
});
return null;
}


requireQueryParams<T extends ParamsType = ParamsType>(
params: Record<string, 'string' | 'number' | 'boolean'>,
): T | null {
const parsedParams: Partial<T> = {};

for (const paramName in params) {
const paramType = params[paramName];
parsedParams[paramName] = this._sanitizeParam(paramName, paramType);
if (parsedParams[paramName] == null) {
this.reply({
ok: false,
statusCode: 406,
errorCode: `${param}_query_parameter_required`,
});
return null;
}
}
Fixed Show fixed Hide fixed

return parsedParams as T;
}
}
type ParamsType = Record<string, string | number | boolean>
2 changes: 1 addition & 1 deletion packages/core/nano-server/src/type.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ interface ReplySuccessContent {

export type ReplyContent = ReplyFailedContent | ReplySuccessContent;

export type Methods = '*' | 'GET' | 'HEAD' | 'POST' | 'PUT' | 'DELETE' | 'CONNECT' | 'TRACE' | 'OPTIONS' | 'PATCH';
export type Methods = 'ALL' | 'GET' | 'HEAD' | 'POST' | 'PUT' | 'DELETE' | 'CONNECT' | 'TRACE' | 'OPTIONS' | 'PATCH';

export interface Config {
/**
Expand Down
56 changes: 30 additions & 26 deletions packages/nanoservice/storage/demo.http
Original file line number Diff line number Diff line change
@@ -1,50 +1,49 @@
@apiUrl = http://localhost:80
@apiVersion = v0
@token = demo_123456789_123456789_123456789_123456789_123456789
@token = alwatr_110_313

### Get a public storage
GET {{apiUrl}}/{{apiVersion}}/comment-list/page-1.json
### Get a public storage over nginx serve
# GET {{apiUrl}}/{{apiVersion}}/public/comments/page1.json

### Get a document by storageName/docId
GET {{apiUrl}}/{{apiVersion}}/sample/doc-id-1
### Get a document by storageName and docId
GET {{apiUrl}}/{{apiVersion}}/?storage=comments/page1&id=c1
authorization: Bearer {{token}}

### Update/insert a document in storageName
POST {{apiUrl}}/{{apiVersion}}/sample
### Insert document
PATCH {{apiUrl}}/{{apiVersion}}/?storage=comments/page1
authorization: Bearer {{token}}
Content-Type: application/json

{
"_id": "doc-id-1",
"_id": "c1",
"_updatedBy": "demo",
"from": "Ali Mihandoost",
"message": "Salam ;)"
"message": "Salam"
}

### Insert a document to storageName
PUT {{apiUrl}}/{{apiVersion}}/sample
### Edit document
PATCH {{apiUrl}}/{{apiVersion}}/?storage=comments/page1
authorization: Bearer {{token}}
Content-Type: application/json

{
"_id": "c1",
"_updatedBy": "demo",
"from": "Ali Mihandoost",
"message": "Salam ;)"
}

### Insert a document to storageName with custom id
PUT {{apiUrl}}/{{apiVersion}}/sample
### Remove document
DELETE {{apiUrl}}/{{apiVersion}}/?storage=comments/page1&id=c1
authorization: Bearer {{token}}
Content-Type: application/json

{
"_id": "doc-id-2",
"from": "Ali Mihandoost",
"message": "Salam ;)"
}

### Insert a document to storageName with custom id
DELETE {{apiUrl}}/{{apiVersion}}/sample/doc-id-2
### Get keys
GET {{apiUrl}}/{{apiVersion}}/keys?storage=comments/page1
authorization: Bearer {{token}}

### Get all data
GET {{apiUrl}}/{{apiVersion}}/all?storage=comments/page1
authorization: Bearer {{token}}

### === Test other routes and errors ===

Expand All @@ -61,14 +60,19 @@ TRACE {{apiUrl}}/{{apiVersion}}
GET {{apiUrl}}/{{apiVersion}}/health

### empty body
POST {{apiUrl}}/{{apiVersion}}
### Insert document
PATCH {{apiUrl}}/{{apiVersion}}/?storage=comments/page1
authorization: Bearer {{token}}
Content-Type: application/json

### invalid json
POST {{apiUrl}}/{{apiVersion}}/echo
PATCH {{apiUrl}}/{{apiVersion}}/?storage=comments/page1
authorization: Bearer {{token}}
Content-Type: application/json

{
"a": 1,
b: 2,
"_id": "c1",
"_updatedBy": "demo",
from: "Ali Mihandoost",
"message": "Salam"
}
17 changes: 8 additions & 9 deletions packages/nanoservice/storage/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,31 +33,30 @@
"scripts": {
"l": "yarn lint",
"b": "yarn build",
"cb": "npm-run-all --sequential clean build",
"s": "npm-run-all --sequential clean build serve",
"cb": "run-s clean build",
"s": "run-s clean build serve",
"w": "yarn watch",
"f": "yarn format",
"fl": "yarn format:eslint",
"fp": "yarn format:prettier",
"start": "yarn serve",
"lint": "npm-run-all --sequential lint:*",
"lint": "run-s lint:*",
"lint:ts": "eslint **/*.ts",
"build": "npm-run-all --sequential build:*",
"build": "run-s build:*",
"build:ts": "tsc --build",
"format": "npm-run-all --sequential format:prettier format:eslint",
"format": "run-s format:prettier format:eslint",
"format:eslint": "eslint **/*.ts --fix",
"format:prettier": "prettier \"**/*.{html,json,md,ts}\" --ignore-path ./.eslintignore --write",
"clean": "rimraf dist/",
"serve": "node dist/index.js",
"serve:debug": "node --inspect dist/index.js",
"watch": "npm-run-all --parallel watch:ts watch:node",
"watch:node": "nodemon dist/index.js",
"watch:debug-node": "nodemon --inspect dist/index.js",
"watch": "run-s clean build && run-p watch:ts watch:node",
"watch:node": "nodemon -w dist/ dist/index.js",
"watch:debug-node": "nodemon -w dist/ --inspect dist/index.js",
"watch:ts": "yarn build:ts --watch --preserveWatchOutput"
},
"dependencies": {
"@alwatr/logger": "^0.21.0",
"@alwatr/math": "^0.21.0",
"@alwatr/nano-server": "^0.21.0",
"@alwatr/storage": "^0.21.0"
},
Expand Down
6 changes: 4 additions & 2 deletions packages/nanoservice/storage/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import './route/home.js';
import './route/update.js';
import './route/get.js';
import './route/update.js';
import './route/remove.js';
import './route/keys.js';
import './route/get-all.js';
import {logger} from './lib/config.js';

logger.logOther('..:: Alwatr Storage Nanoservice API ::..');
13 changes: 6 additions & 7 deletions packages/nanoservice/storage/src/lib/config.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
import {createLogger} from '@alwatr/logger';
import {isNumber} from '@alwatr/math';

export const logger = createLogger('service-storage');

export const config = {
/* eslint-disable @typescript-eslint/no-non-null-assertion */
port: isNumber(process.env.PORT) ? +process.env.PORT! : 80,
host: process.env.HOST ?? '0.0.0.0',
port: process.env.PORT != null ? +process.env.PORT : 80,
storagePath: process.env.STORAGE_PATH ?? 'db',
dataModelName: process.env.DATA_MODEL_NAME ?? 'data-model-list',
token: process.env.TOKEN ?? 'alwatr_110_313',
};

export const logger = createLogger('service-storage');

logger.logProperty('config', config);
logger.logProperty('config', {...config, token: '***'});
njfamirm marked this conversation as resolved.
Show resolved Hide resolved
31 changes: 0 additions & 31 deletions packages/nanoservice/storage/src/lib/storage-provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,35 +2,4 @@ import {AlwatrStorageProvider} from '@alwatr/storage/provider.js';

import {config} from './config.js';

import type {DataModel} from './type.js';

export const storageProvider = new AlwatrStorageProvider({path: config.storagePath});

const dataModelStorage = await storageProvider.get<DataModel>({
name: config.dataModelName,
path: `${config.storagePath}/private`,
});

if (dataModelStorage.length === 0) {
dataModelStorage.set({
_id: 'sample',
_updatedBy: 'system',
subFolder: 'public',
subStorage: false,
});
}

export function getDataModel(storageName: string): DataModel | null {
const splittedName = storageName.split('/');
let testStorageName = '';
for (let i = 0; i < splittedName.length; i++) {
testStorageName += splittedName[i];
const dataModel = dataModelStorage.get(testStorageName, true);
if (dataModel != null) {
if (dataModel.subStorage === false && dataModel._id !== storageName) continue;
return dataModel;
}
testStorageName += '/';
}
return null;
}
31 changes: 0 additions & 31 deletions packages/nanoservice/storage/src/lib/token.ts

This file was deleted.

13 changes: 0 additions & 13 deletions packages/nanoservice/storage/src/lib/type.ts

This file was deleted.

Loading