- Introduction
- Installation
- Normalizers
- Denormalizers
- Schemas
- Custom Schema Handlers
- Error Handling
- Usage Examples
- Configuration
- Testing
- Contributing
- License
DataNormTS facilitates the transformation of complex and nested data structures into normalized formats and vice versa. Normalization is essential for optimizing data management, especially when dealing with relational data or preparing data for state management libraries like Redux.
-
Flexible Schemas: Define customizable schemas to dictate how data should be normalized or denormalized.
-
Custom Handlers: Extend functionality by registering custom schema handlers for specialized data types.
-
Performance Optimizations: Utilize memoization and asynchronous operations to ensure efficient data processing.
-
Comprehensive Testing: Ensure reliability with extensive test suites covering various scenarios and edge cases.
You can install DataNormTS via npm or yarn:
npm install datanormtsyarn add datanormtsimport { normalize, denormalize } from 'datanormts';The Normalizers module is responsible for transforming input data into a normalized format based on predefined schemas. Normalization processes data structures to improve efficiency, avoid redundancy, and simplify data manipulation.
export function normalize(data: unknown, schema: Schema): NormalizedData;Description:
Transforms the provided data into a normalized format according to the specified schema.
Parameters:
data(unknown): The data to normalize.schema(Schema): The schema defining the normalization rules.
Returns:
NormalizedData: The normalized representation of the input data.
Throws:
NormalizationError: If the data is invalid or does not conform to the schema.
Example:
import { normalize } from 'datanormts';
const schema = {
user: {
type: 'object',
name: 'user',
properties: {
id: { type: 'string' },
name: { type: 'string' },
age: { type: 'number' },
isActive: { type: 'boolean' }
}
}
};
const data = {
id: '1',
name: 'John Doe',
age: 30,
isActive: true
};
const normalizedData = normalize(data, schema);
console.log(normalizedData);export async function safeNormalize(data: unknown, schema: Schema): Promise<NormalizedData>;Description:
Performs normalization in a thread-safe manner using locks to prevent concurrent modifications.
Parameters:
data(unknown): The data to normalize.schema(Schema): The schema defining the normalization rules.
Returns:
Promise<NormalizedData>: A promise that resolves to the normalized data.
Throws:
NormalizationError: If the data is invalid or does not conform to the schema.
Example:
import { safeNormalize } from 'datanormts';
const schema = { /* schema definition */ };
const data = { /* data to normalize */ };
safeNormalize(data, schema)
.then(normalizedData => {
console.log(normalizedData);
})
.catch(error => {
console.error(error);
});Handles normalization for custom schema types.
export function normalizeEntity(entity: unknown, schema: SchemaEntity, entities: NormalizedData['entities']): EntityID | EntityID[] | unknown;Description:
Normalizes entities based on custom schema definitions, allowing for specialized handling beyond standard types.
Key Functions:
normalizeEntity: Determines the normalization path based on schema type.normalizeCustomEntity: Handles custom schema normalization using registered handlers.
Handles normalization of object-type schemas.
export function normalizeObject(entity: Record<string, unknown>, schema: ObjectSchemaEntity, entities: NormalizedData['entities']): EntityID | Record<string, unknown>;Description:
Normalizes objects by extracting and storing entities with unique identifiers.
Example:
const schema = {
user: {
type: 'object',
name: 'user',
properties: {
id: { type: 'string' },
name: { type: 'string' },
age: { type: 'number' }
}
}
};
const data = { id: '1', name: 'John Doe', age: 30 };
const normalizedData = normalize(data, schema);Handles normalization of array-type schemas.
export function normalizeArray(array: unknown[], schema: ArraySchemaEntity, entities: NormalizedData['entities']): EntityID[];Description:
Normalizes arrays by processing each item according to the specified item schema.
Handles normalization of primitive data types.
export function normalizePrimitive(entity: unknown, schema: PrimitiveSchemaEntity): EntityID | number | string | boolean;Description:
Validates and returns primitive data types based on the schema.
Provides utility functions to assist with the normalization process.
export function isValidSchemaType(type: unknown): type is SchemaType;
export function isObjectSchemaEntity(schema: SchemaEntity): schema is ObjectSchemaEntity;
export function isArraySchemaEntity(schema: SchemaEntity): schema is ArraySchemaEntity;
export function isPrimitiveSchemaEntity(schema: SchemaEntity): schema is PrimitiveSchemaEntity;
export function isCustomSchemaEntity(schema: SchemaEntity): schema is CustomSchemaEntity;
export function isStringSchemaEntity(schema: PrimitiveSchemaEntity): schema is StringSchemaEntity;
export function isNumberSchemaEntity(schema: PrimitiveSchemaEntity): schema is NumberSchemaEntity;
export function isValidEntityIDArray(value: unknown): value is EntityID[];Description:
Includes type guard functions to determine the schema entity types and validate data.
The Denormalizers module is responsible for reconstructing the original data structure from its normalized form. Denormalization transforms normalized data back into its nested, relational format based on the provided schemas.
export function denormalize(normalizedData: NormalizedData, schema: Schema): unknown;Description:
Reconstructs the original data structure from normalized data using the specified schema.
Parameters:
normalizedData(NormalizedData): The data to denormalize.schema(Schema): The schema defining the denormalization rules.
Returns:
unknown: The denormalized data structure.
Throws:
DenormalizationError: If the normalized data is invalid or does not conform to the schema.
Example:
import { denormalize } from 'datanormts';
const schema = { /* schema definition */ };
const normalizedData = { /* normalized data */ };
const originalData = denormalize(normalizedData, schema);
console.log(originalData);Handles denormalization of individual entities.
export function denormalizeEntity(entityId: EntityID, schemaEntity: SchemaEntity, entities: NormalizedData['entities']): unknown;Description:
Denormalizes entities by retrieving them from the normalized entities collection based on their identifiers and schema definitions.
Handles denormalization of object-type schemas.
export function denormalizeObject(entity: unknown, schema: ObjectSchemaEntity, entities: NormalizedData['entities']): Record<string, unknown>;Description:
Reconstructs objects by denormalizing each property according to the schema.
Example:
const schema = { /* object schema */ };
const normalizedData = { /* normalized entities */ };
const denormalizedObject = denormalize(normalizedData, schema);
console.log(denormalizedObject);Handles denormalization of array-type schemas.
export function denormalizeArray(array: EntityID[], schema: ArraySchemaEntity, entities: NormalizedData['entities']): unknown[];Description:
Reconstructs arrays by denormalizing each element based on the item schema.
Handles denormalization for custom schema types.
export function denormalizeCustom(entity: unknown, schema: CustomSchemaEntity, entities: NormalizedData['entities']): unknown;Description:
Denormalizes entities based on custom schema handlers, allowing for specialized reconstruction beyond standard types.
Handles denormalization of primitive data types.
export function denormalizePrimitive(entityId: EntityID, schema: PrimitiveSchemaEntity): string | number | boolean;Description:
Validates and returns primitive data types based on the schema.
Schemas define how data should be normalized or denormalized. A schema is a TypeScript object that outlines the structure, types, and properties of the data entities.
Schemas can be of various types, each catering to different data structures and requirements.
Defines the structure of an object with specific properties.
Example:
const userSchema: Schema = {
user: {
type: 'object',
name: 'user',
properties: {
id: { type: 'string' },
name: { type: 'string' },
age: { type: 'number' },
isActive: { type: 'boolean' }
}
}
};Defines an array structure, specifying the schema of its items.
Example:
const usersSchema: Schema = {
users: {
type: 'array',
items: {
type: 'object',
name: 'user',
properties: {
id: { type: 'string' },
name: { type: 'string' }
}
}
}
};Defines primitive data types such as strings, numbers, and booleans.
Example:
const tagsSchema: Schema = {
tags: {
type: 'array',
items: { type: 'string' }
}
};Allows for custom data handling by registering specialized handlers.
Example:
const customEntitySchema: Schema = {
customEntity: {
type: 'custom',
name: 'customEntity'
}
};Custom schema handlers enable specialized processing of data types that go beyond the standard normalization and denormalization mechanisms. To utilize custom handlers, you must register them with the library.
Function: registerCustomSchemaHandler
export function registerCustomSchemaHandler(name: string, handler: CustomSchemaHandler): void;Parameters:
name(string): The identifier for the custom schema type.handler(CustomSchemaHandler): The function that handles normalization or denormalization for the custom schema.
Example:
import { registerCustomSchemaHandler } from 'datanormts';
const customHandler: CustomSchemaHandler = (entity, schema, entities) => {
// Custom normalization logic
return entity.id;
};
registerCustomSchemaHandler('customEntity', customHandler);Once registered, custom schema handlers are invoked automatically during normalization and denormalization processes based on the schema definitions.
Example:
import { normalize, denormalize } from 'datanormts';
// Register the custom handler
registerCustomSchemaHandler('customEntity', customHandler);
// Define the schema using the custom type
const schema: Schema = {
customEntity: {
type: 'custom',
name: 'customEntity'
}
};
// Normalize data
const data = { id: 'custom-id', some: 'data' };
const normalizedData = normalize(data, schema);
// Denormalize data
const originalData = denormalize(normalizedData, schema);
console.log(originalData);DataNormTS provides robust error handling mechanisms to ensure that developers receive clear and actionable feedback when issues arise during normalization or denormalization.
Class: NormalizationError
Description:
Thrown when normalization fails due to invalid input data, schema mismatches, or other related issues.
Common Scenarios:
- Invalid input data type.
- Schema validation failures.
- Missing required properties.
- Custom handler errors.
Example:
try {
const normalizedData = normalize(data, schema);
} catch (error) {
if (error instanceof NormalizationError) {
console.error('Normalization failed:', error.message);
}
}Class: DenormalizationError
Description:
Thrown when denormalization fails due to missing entities, invalid schemas, or other related issues.
Common Scenarios:
- Missing entities in normalized data.
- Invalid schema definitions.
- Circular references that cannot be resolved.
Example:
try {
const originalData = denormalize(normalizedData, schema);
} catch (error) {
if (error instanceof DenormalizationError) {
console.error('Denormalization failed:', error.message);
}
}import { normalize, denormalize } from 'datanormts';
// Define the schema
const schema = {
user: {
type: 'object',
name: 'user',
properties: {
id: { type: 'string' },
name: { type: 'string' },
age: { type: 'number' },
isActive: { type: 'boolean' }
}
}
};
// Original data
const data = {
id: '1',
name: 'John Doe',
age: 30,
isActive: true
};
// Normalize the data
const normalizedData = normalize(data, schema);
console.log('Normalized Data:', normalizedData);
/*
Output:
{
entities: {
user: {
'1': { id: '1', name: 'John Doe', age: 30, isActive: true }
}
},
result: '1'
}
*/
// Denormalize the data
const denormalizedData = denormalize(normalizedData, schema);
console.log('Denormalized Data:', denormalizedData);
/*
Output:
{
id: '1',
name: 'John Doe',
age: 30,
isActive: true
}
*/import { normalize, denormalize } from 'datanormts';
// Define the schema for users and posts
const schema = {
users: {
type: 'array',
items: {
type: 'object',
name: 'user',
properties: {
id: { type: 'string' },
name: { type: 'string' },
posts: {
type: 'array',
items: {
type: 'object',
name: 'post',
properties: {
id: { type: 'string' },
title: { type: 'string' }
}
}
}
}
}
}
};
// Original data with nested posts
const data = [
{ id: '1', name: 'John Doe', posts: [{ id: 'p1', title: 'Post 1' }] },
{ id: '2', name: 'Jane Doe', posts: [{ id: 'p2', title: 'Post 2' }] }
];
// Normalize the data
const normalizedData = normalize(data, schema);
console.log('Normalized Data:', normalizedData);
/*
Output:
{
entities: {
user: {
'1': { id: '1', name: 'John Doe', posts: ['p1'] },
'2': { id: '2', name: 'Jane Doe', posts: ['p2'] }
},
post: {
'p1': { id: 'p1', title: 'Post 1' },
'p2': { id: 'p2', title: 'Post 2' }
}
},
result: ['1', '2']
}
*/
// Denormalize the data
const denormalizedData = denormalize(normalizedData, schema);
console.log('Denormalized Data:', denormalizedData);
/*
Output:
[
{ id: '1', name: 'John Doe', posts: [{ id: 'p1', title: 'Post 1' }] },
{ id: '2', name: 'Jane Doe', posts: [{ id: 'p2', title: 'Post 2' }] }
]
*/import { normalize, denormalize, registerCustomSchemaHandler } from 'datanormts';
// Define a custom handler for 'customEntity'
const customHandler: CustomSchemaHandler = (entity, schema, entities) => {
if (typeof entity === 'string' || typeof entity === 'number') {
// During denormalization: entity is ID, return the entity ID
return entity;
} else {
// During normalization: entity is the object, return the object ID
return entity.id;
}
};
// Register the custom handler
registerCustomSchemaHandler('customEntity', customHandler);
// Define the custom schema
const schema = {
customEntity: {
type: 'custom',
name: 'customEntity'
}
};
// Original data
const data = { id: 'custom-id', some: 'data' };
// Normalize the data
const normalizedData = normalize(data, schema);
console.log('Normalized Data:', normalizedData);
/*
Output:
{
entities: {
customEntity: {
'custom-id': { id: 'custom-id', some: 'data' }
}
},
result: 'custom-id'
}
*/
// Denormalize the data
const denormalizedData = denormalize(normalizedData, schema);
console.log('Denormalized Data:', denormalizedData);
/*
Output:
{ id: 'custom-id', some: 'data' }
*/DataNormTS utilizes several configuration options to tailor its behavior according to project requirements.
Ensure your tsconfig.json is set up correctly to support DataNormTS.
{
"compilerOptions": {
"target": "es2018",
"module": "commonjs",
"lib": ["es2018"],
"declaration": true,
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"types": ["jest", "node"]
},
"include": ["src/**/*"],
"exclude": ["node_modules", "**/*.test.ts"]
}The library includes a configurable logger (src/logger.ts) that can be adjusted to set logging levels, output destinations, and other settings. Refer to the Logger Documentation for more details.
DataNormTS is built with reliability in mind, supported by comprehensive test suites covering diverse scenarios and edge cases.
Ensure you have Jest installed, then execute:
npx jestSample Test Output:
Test Suites: 8 passed, 8 total
Tests: 43 passed, 43 total
Snapshots: 0 total
Time: 2.883 s
Ran all test suites.
- Normalizers: Validates normalization logic across various data structures, including objects, arrays, primitives, and custom entities.
- Denormalizers: Ensures accurate reconstruction of original data from normalized forms, handling nested structures and error scenarios.
- Edge Cases: Tests deeply nested structures, large datasets, circular references, and invalid custom handlers to ensure robustness.
- Custom Handlers: Verifies the registration and functionality of custom schema handlers, including error handling for invalid scenarios.
Contributions are welcome! Help improve DataNormTS by submitting issues, feature requests, or pull requests.
-
Fork the Repository: Create a fork of the DataNormTS repository on GitHub.
-
Clone the Fork:
git clone https://github.com/your-username/DataNormTS.git
-
Create a Branch:
git checkout -b feature/your-feature-name
-
Make Changes: Implement your feature or fix.
-
Run Tests:
npx jest
-
Commit Changes:
git commit -m "Description of your changes" -
Push to Fork:
git push origin feature/your-feature-name
-
Create a Pull Request: Submit a pull request detailing your changes.
- Follow existing coding conventions and styles.
- Ensure all tests pass.
- Provide clear and concise commit messages.
- Update documentation as necessary.
DataNormTS is licensed under the MIT License. See the LICENSE file for more details.