Skip to content

Commit

Permalink
feat(WordsToNumber): Added fuzzy to fix wrong words
Browse files Browse the repository at this point in the history
  • Loading branch information
ali-master committed Dec 26, 2020
1 parent cfe7d20 commit 13282a9
Show file tree
Hide file tree
Showing 7 changed files with 78 additions and 51 deletions.
38 changes: 19 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,18 @@

## Features

- Convert Persian words to number and vice versa.
- Add and remove commas to numbers.
- Convert Persian numbers to Arabic or English numbers and vice versa.
- Validate Iranian national number(code-e Melli).
- Find city and province name by national code(code-e Melli).
- Bill calculator
- Check Iranian Sheba(IBAN) validation and recognize bank information by sheba code.
- Validate Bank card number.
- Find Bank's name by Card number.
- Validate the correctness of the text of the Persian language and clear the Arabic letters in the Persian text.
- Fix Persian characters in URL.
- Fix Persian zero-width non-joiner(Replace spaces by half-space)
- [Convert Persian words to number and vice versa](#convert-persian-words-to-the-number-and-vice-versa).
- [Add and remove commas to numbers](#add-and-remove-commas).
- [Convert Persian numbers to Arabic or English numbers and vice versa](#convert-persian-numbers-to-arabic-or-english-numbers-and-vice-versa).
- [Validate Iranian national number(code-e Melli)](#validate-iranian-national-numbercode-e-melli).
- [Find city and province name by national code(code-e Melli)](#find-city-and-province-name-by-national-codecode-e-melli).
- [Bill calculator](#bill-calculator).
- [Check Iranian Sheba(IBAN) validation and recognize bank information by sheba code](#iranian-shebaiban).
- [Validate Bank card number](#bank-number-validation-and-get-the-name-of-the-bank-by-bank-account-number).
- [Find Bank's name by Card number]((#bank-number-validation-and-get-the-name-of-the-bank-by-bank-account-number)).
- [Validate the correctness of the text of the Persian language and clear the Arabic letters in the Persian text](#validate-the-correctness-of-the-text-of-the-persian-language-and-clear-the-arabic-letters-in-the-persian-text).
- [Fix Persian characters in URL](#fix-persian-characters-in-url).
- [Fix Persian zero-width non-joiner(Replace spaces by half-space)](#fix-persian-zero-width-non-joinerreplace-spaces-by-half-space)

## Getting started

Expand Down Expand Up @@ -210,13 +210,13 @@ URLfix("Sample Text"); // "Sample Text"
### Bill calculator
| Method | Description | Return type
|--- |--- |---
| getResult | Result of bill calculated information | BillResult
| getAmount | Calculate Bill amount by payment id and bill id which entered by the Bill constructor | number
| getBillType | Get Bill provider type name | BillTypes
| getBarcode | Calculate and get Bill's barcode | string
| verificationBill | Validate entered both Bill id and payment id, and return true if bill id and payment id relation was true | boolean
| verificationBillId | Validate entered Bill id | boolean
| verificationBillPayment | Validate entered Bill payment id | boolean
| `getResult` | Result of bill calculated information | BillResult
| `getAmount` | Calculate Bill amount by payment id and bill id which entered by the Bill constructor | number
| `getBillType` | Get Bill provider type name | BillTypes
| `getBarcode` | Calculate and get Bill's barcode | `string`
| `verificationBill` | Validate entered both Bill id and payment id, and return true if bill id and payment id relation was true | boolean
| `verificationBillId` | Validate entered Bill id | `boolean`
| `verificationBillPayment` | Validate entered Bill payment id | `boolean`
```js
import { Bill } from "persian-tools2";

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
"rollup-plugin-uglify": "^6.0.4",
"standard-version": "^9.0.0",
"ts-jest": "^26.4.0",
"ts-node": "^9.0.0",
"typedoc": "^0.19.2",
"typescript": "^4.0.3",
"uglify-es": "^3.3.9"
Expand Down
2 changes: 1 addition & 1 deletion src/modules/wordsToNumber/fuzzy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { ALL_WORDS } from "./constants";

const Fuse = require("fuse.js");

interface FuzzyOptions {
export interface FuzzyOptions {
dataset?: string[];
threshold?: number;
}
Expand Down
35 changes: 23 additions & 12 deletions src/modules/wordsToNumber/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,16 @@ import { replaceArray } from "../../helpers";
import { digitsEnToFa, digitsFaToEn } from "../digits";
import removeOrdinalSuffix from "../removeOrdinalSuffix";
import { UNITS, TEN, MAGNITUDE, TYPO_LIST } from "./constants";
import { fuzzy, FuzzyOptions } from "./fuzzy";

// <Reference path='https://fa.wikipedia.org/wiki/الگو:عدد_به_حروف/توضیحات' />
// https://fa.wikipedia.org/wiki/۱۰۰۰۰۰۰۰۰۰_(عدد)

interface IOption {
digits?: string;
addCommas?: boolean;
fuzzy?: boolean;
fuzzyOptions?: FuzzyOptions;
}

class WordsToNumber {
Expand All @@ -20,20 +23,28 @@ class WordsToNumber {
* @param options
* @return Converted words to number. e.g: 350000
*/
public convert(words: string, options: IOption = {}): number | string | undefined {
public convert(
words: string,
{
digits = "en",
addCommas: shouldAddCommas = false,
fuzzy: fuzzyCleaner = false,
fuzzyOptions = {},
}: IOption = {},
): number | string | undefined {
if (!words) return;

const digits = options.digits ?? "en";
const shouldAddCommas = options.addCommas ?? false;
const prettify = fuzzyCleaner ? fuzzy(words, fuzzyOptions) : words;
if (fuzzyCleaner) {
console.log("prettify", prettify);
}
// @ts-ignore
let numbersConverted = this.compute(this.tokenize(words));
const computeNumbers = this.compute(this.tokenize(prettify));
const addCommasIfNeeded: string | number = shouldAddCommas
? (addCommas(computeNumbers) as string)
: (computeNumbers as number);

// @ts-ignore
numbersConverted = shouldAddCommas ? (addCommas(numbersConverted) as string) : (numbersConverted as number);
// @ts-ignore
numbersConverted = digits === "fa" ? (digitsEnToFa(numbersConverted as number) as string) : numbersConverted;

return numbersConverted;
return digits === "fa" ? (digitsEnToFa(addCommasIfNeeded as number) as string) : addCommasIfNeeded;
}
private tokenize(words: string): number[] {
let replacedWords = replaceArray(words, TYPO_LIST);
Expand All @@ -42,7 +53,7 @@ class WordsToNumber {

const result: number[] = [];
const slittedWords: string[] = replacedWords.split(" ");
slittedWords.forEach((word: string) =>
slittedWords.forEach((word) =>
// @ts-ignore
word === "و" ? "" : !isNaN(+word) ? result.push(+word) : result.push(word),
);
Expand All @@ -54,7 +65,7 @@ class WordsToNumber {
let sum = 0;
let isNegative = false;

tokens.forEach((token: string) => {
tokens.forEach((token) => {
// @ts-ignore
token = digitsFaToEn(token);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { fuzzy } from "../../src/modules/wordsToNumber/fuzzy";
import { ALL_WORDS } from "../../src/modules/wordsToNumber/constants";
import { fuzzy } from "../src/modules/wordsToNumber/fuzzy";
import { ALL_WORDS } from "../src/modules/wordsToNumber/constants";

describe("WordsToNumber - Fuzzy humanizer", () => {
it("Should clean the Texts and the result should be human-readable", () => {
Expand Down
32 changes: 32 additions & 0 deletions test/wordsToNumber.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import WordsToNumber from "../src/modules/wordsToNumber";

describe("WordsToNumber", () => {
it("Should works as well", () => {
expect(WordsToNumber.convert("منفی سه هزار")).toEqual(-3000);
expect(WordsToNumber.convert("سه هزار دویست و دوازده")).toEqual(3212);
expect(WordsToNumber.convert("دوازده هزار بیست دو")).toEqual(12022);
expect(WordsToNumber.convert("دوازده هزار بیست دو", { addCommas: true })).toEqual("12,022");
expect(WordsToNumber.convert("دوازده هزار و بیست و دو", { addCommas: true })).toEqual("12,022");
});

it("Ordinal words", () => {
expect(WordsToNumber.convert("منفی سه هزارمین", { digits: "fa", addCommas: true })).toEqual("-۳,۰۰۰");
expect(WordsToNumber.convert("منفی سه هزارمین", { digits: "fa" })).toEqual("-۳۰۰۰");
expect(WordsToNumber.convert("منفی سه هزارمین")).toEqual(-3000);
expect(WordsToNumber.convert("منفی سه هزارم")).toEqual(-3000);
expect(WordsToNumber.convert("منفی سه هزارمین")).not.toEqual("-3000");
expect(String(WordsToNumber.convert("منفی سه هزارمین"))).toHaveLength(5);
expect(WordsToNumber.convert("منفی سی اُم")).toEqual(-30);
expect(WordsToNumber.convert("سی و سوم")).toEqual(33);
});

it("Should return undefined", () => {
expect(WordsToNumber.convert("", { digits: "fa", addCommas: true })).toBeUndefined();
// @ts-ignore
expect(WordsToNumber.convert()).toBeUndefined();
});

it("Should works with fuzzy model", () => {
expect(WordsToNumber.convert("ضد و بنچاه و دو", { fuzzy: true })).toEqual(152);
});
});
17 changes: 0 additions & 17 deletions test/wordsToNumber/wordsToNumber.spec.ts

This file was deleted.

0 comments on commit 13282a9

Please sign in to comment.