SPARKNLP 1034 implement starcoder2 for causal lm

[prabod]

This PR introduces StarCoder2

Description

This pull request adds support for the StarCoder2 model in the Spark NLP library. StarCoder2 is a Transformer model designed specifically for code generation and understanding. It features 3 billion parameters and is trained on a diverse dataset including multiple programming languages, making it highly versatile for various coding tasks. The new model provides enhanced functionality for code completion, generation, and understanding.

Motivation and Context

The inclusion of StarCoder2 addresses the need for a robust code-focused language model within Spark NLP. This model will significantly improve the library's capabilities in code generation and understanding tasks, offering developers a powerful tool for software development and data science projects.

How Has This Been Tested?

The StarCoder2 model has been tested using various unit and integration tests to ensure its proper functionality within the Spark NLP framework. Tests were conducted to verify the model's ability to perform code completion and generation tasks accurately. Additionally, performance benchmarks were compared to existing models to ensure its efficacy.

Screenshots (if appropriate):

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING page.
• I have added tests to cover my changes.
• All new and existing tests passed.