SPARKNLP-656 & SPARKNLP-657: Updated Documentation

[DevinTDHa]

Description

This PR updates the documentation so it is more clear how to use setTestDataset in *DLApproach annotators.

In addition installation instructions for M1 machines were also updated.

Resolves #13070 and #13079.

How Has This Been Tested?

No code changes.

[maziyarpanahi]

@DevinTDHa

Once you split (or use CoNLL() to have another DataFrame for test/dev), you need to transform it on the very same pipeline.

• example for NER: https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/75f443b1792ecc99575bfe217f30640acff7b55d/jupyter/training/english/dl-ner/ner_graph_builder.ipynb
• example for all the classfiers are here: (Train/Evaluation) https://github.com/JohnSnowLabs/spark-nlp-workshop/tree/master/jupyter/training/english/classification

So perhaps, we can have one example/doc for a normal training, and another one in case the needs to have testDataset param which usually doesn't use Pipeline (or everything up and incuding embeddings is in the Pipeline and the trainiable annotator is outside the pipeline)

Like

document = DocumentAssembler()\
    .setInputCol("description")\
    .setOutputCol("document")

use = UniversalSentenceEncoder.pretrained() \
 .setInputCols(["document"])\
 .setOutputCol("sentence_embeddings")

pipeline = Pipeline(stages = [document,use])

test_dataset = pipeline.fit(news_test_dataset).transform(news_test_dataset)

we transform and save the test/dev:

test_dataset.write.parquet("./test_news.parquet")

and we train:

classsifierdl = ClassifierDLApproach()\
  .setInputCols(["sentence_embeddings"])\
  .setOutputCol("class")\
  .setLabelColumn("category")\
  .setMaxEpochs(5)\
  .setEnableOutputLogs(True) \
  .setEvaluationLogExtended(True) \
  .setValidationSplit(0.2) \
  .setTestDataset("./test_news.parquet")

pipeline = Pipeline(
    stages = [
        document,
        use,
        classsifierdl
    ])

pipelineModel = pipeline.fit(trainDataset)

[DevinTDHa]

@maziyarpanahi Updated with the latest push to better examples