Skip to content

Commit

Permalink
update the numbering
Browse files Browse the repository at this point in the history
  • Loading branch information
sahil11129 committed Mar 23, 2023
1 parent ed5a71d commit 1a1ca0c
Showing 1 changed file with 6 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ dev_iob_stream = prepare_train_from_json(dev_data, syntax_model)

* SIRE: Statistical Information and Relation Extraction (SIRE) is a technique used in natural language processing (NLP) to extract specific information and relationships from text. It involves using machine learning algorithms to identify and extract structured data such as entities, attributes, and relations from unstructured text. SIRE is used in a variety of applications, including information extraction, knowledge graph construction, and question answering. SIRE typically uses supervised learning approach, where a model is trained using annotated examples of text and the corresponding structured data. The model can then be used to extract the same information from new, unseen text.

## Step 2.1 Entity extraction function
## 1. Entity extraction function

Both the model are trained from labeled data, which require the syntax block to be executed first to generate the expected input for the entity-mention block. BiLSTM model requires Glove embedding for fine tuning. It allows for words to be represented as dense vectors in a high-dimensional space, where the distance between vectors reflects the semantic similarity between the corresponding words. We can use GloVe embedding to generate vector representations of the words in our data, which can then be utilized for further analysis or modeling." is a popular method for generating vector representations of words in natural language processing. It allows for words to be represented as dense vectors in a high-dimensional space, where the distance between vectors reflects the semantic similarity between the corresponding words. We can use GloVe embedding to generate vector representations of the words in our data, which can then be utilized for further analysis or modeling."

Expand All @@ -193,7 +193,7 @@ glove_model = watson_nlp.load(watson_nlp.download('embedding_glove_en_stock'))
```


## 2.2 Fine-Tuning the model
## 2. Fine-Tuning the model

Fine-tuning a Sire model for Entity extraction involves training the model on a labeled training dataset includes examples of Entities.

Expand All @@ -210,7 +210,7 @@ sire_custom = watson_nlp.workflows.entity_mentions.SIRE.train(syntax_model=synta
num_workers=5)
```

In the above Fine-tuning, `labeled_entity_mentions` is the path to a collection of labeled data (.json) or loaded DataStream of JSONs, which prepared above in Preparing Sample Data Set. `/home/wsuser/work/` is home directory which includes `train_iob_stream` is the training data that generate at beginning of the tutorial which includes 30,000 sentences, `en` is the language code for English, and `mentions_train_template` is the SIRE model entity mention template which we load in the beginning
In the above Fine-tuning, `labeled_entity_mentions` is the path to a collection of labeled data (.json) or loaded DataStream of JSONs, which prepared above in Preparing Sample Data Set. `/home/wsuser/work/` is home directory which includes `train_iob_stream` is the training data that generate at beginning of the tutorial which includes 30,000 sentences, `en` is the language code for English, and `mentions_train_template` is the SIRE model entity mention template which we load in the beginning, it is base training template for entity mentions SIRE block using the CRF algorithm.

```
#Save the Trained block model as a workflow model
Expand All @@ -223,7 +223,7 @@ project.save_data('sire_Entity_workflow_custom', data=sire_workflow.as_file_like

now save the model with Syntax model as workflow model so we can directly test on the input text.

## 2.3 Test the Fine-Tuned Model
## 3. Test the Fine-Tuned Model

Now let's run the trained models with testing data, Here testing data is a sentence from test data which we generate before.

Expand Down Expand Up @@ -322,7 +322,7 @@ The Watson NLP platform provides a fine-tune feature that allows for custom trai



## Step 3.1 Fine-Tuning the models
## 1. Fine-Tuning the models

Fine-tuning a BiLSTM model for Entity extraction involves training the model on a labeled training dataset includes examples of Entities.

Expand All @@ -347,7 +347,7 @@ project.save_data('Entity_workflow_bilstm_custom', data=mentions_workflow.as_fil
```
now save the model with Syntax model as workflow model so we can directly test on the input text.

## 3.2 Test the Fine-Tuned Model
## 2. Test the Fine-Tuned Model

Now let's run the trained models with testing data, Here testing data is a sentence from test data which we generate before. We can fetch single sentences : `text = pd.read_json('custom_entity_test_data.json')['text'][0]`

Expand Down

0 comments on commit 1a1ca0c

Please sign in to comment.