Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Langchain notebook 2 #2002

Merged
merged 25 commits into from
Jun 30, 2023
Merged

Langchain notebook 2 #2002

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
1336d7c
Langchain Transformation
sherylZhaoCode Apr 18, 2023
35085ca
Merge pull request #1 from sherylZhaoCode/sherylZhaoCode-langchainTra…
sherylZhaoCode Apr 18, 2023
1742478
added the notebook for demonstrating the usage of langchain transformer
sherylZhaoCode May 5, 2023
bfd810f
merged master
sherylZhaoCode May 5, 2023
43317af
modified ipython notebook
sherylZhaoCode May 9, 2023
be3fef9
small change to the notebook
sherylZhaoCode May 10, 2023
3109daa
black reformatted notebook
sherylZhaoCode May 15, 2023
dcdf39c
modified the pip installation part
sherylZhaoCode May 18, 2023
0bb36a6
package installation modification
sherylZhaoCode May 18, 2023
35d98f3
Merge branch 'master' into sherylZhao/add_notebook_for_langchaintrans…
dciborow May 18, 2023
f8abe10
added get ipython
sherylZhaoCode May 18, 2023
811ae14
Merge branch 'sherylZhao/add_notebook_for_langchaintransformer' of gi…
sherylZhaoCode May 18, 2023
47ad470
Merge branch 'master' into sherylZhao/add_notebook_for_langchaintrans…
dciborow May 19, 2023
39db577
removed ipython
sherylZhaoCode May 19, 2023
c5805db
Merge branch 'sherylZhao/add_notebook_for_langchaintransformer' of gi…
sherylZhaoCode May 19, 2023
e518c71
trial to make pip install working on synapse
sherylZhaoCode May 19, 2023
ec015c1
latest changes
sherylZhaoCode May 23, 2023
60daab3
chore: minor fixes
mhamilton723 May 23, 2023
88e50d1
docs: minor fixes for langchain notebook
mhamilton723 May 23, 2023
6910ed5
Update CognitiveServices - LangchainTransformer.ipynb
mhamilton723 May 24, 2023
30599ce
Update CognitiveServices - LangchainTransformer.ipynb
mhamilton723 May 24, 2023
14a4312
Merge branch 'master' into sherylZhao/add_notebook_for_langchaintrans…
sherylZhaoCode May 30, 2023
9bc4b83
Merge branch 'master' into sherylZhao/add_notebook_for_langchaintrans…
mhamilton723 Jun 29, 2023
20153ac
add restart cell
mhamilton723 Jun 30, 2023
6316470
Ignore tests on synapse
mhamilton723 Jun 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 21 additions & 12 deletions core/src/test/scala/com/microsoft/azure/synapse/ml/nbtest/DatabricksUtilities.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,17 +48,26 @@ object DatabricksUtilities {
val Folder = s"/SynapseMLBuild/build_${BuildInfo.version}"
val ScalaVersion: String = BuildInfo.scalaVersion.split(".".toCharArray).dropRight(1).mkString(".")

val Libraries: String = List(
Map("maven" -> Map("coordinates" -> PackageMavenCoordinate, "repo" -> PackageRepository)),
Map("pypi" -> Map("package" -> "nltk")),
Map("pypi" -> Map("package" -> "bs4")),
Map("pypi" -> Map("package" -> "plotly")),
Map("pypi" -> Map("package" -> "Pillow")),
Map("pypi" -> Map("package" -> "onnxmltools==1.7.0")),
Map("pypi" -> Map("package" -> "lightgbm")),
Map("pypi" -> Map("package" -> "mlflow")),
Map("pypi" -> Map("package" -> "openai"))
).toJson.compactPrint
val PipPackages: Seq[String] = Seq(
"nltk",
"bs4",
"plotly",
"Pillow",
"onnxmltools==1.7.0",
"lightgbm",
"mlflow",
"openai",
"langchain",
"pdf2image",
"pdfminer.six",
"pytesseract",
"unstructured"
)

val Libraries: String = (
List(Map("maven" -> Map("coordinates" -> PackageMavenCoordinate, "repo" -> PackageRepository))) ++
PipPackages.map(p => Map("pypi" -> Map("package" -> p)))
).toJson.compactPrint

// TODO: install synapse.ml.dl wheel package here
val GPULibraries: String = List(
Expand Down Expand Up @@ -285,7 +294,7 @@ object DatabricksUtilities {
val (url, nbName) = getRunUrlAndNBName(runId)
if (logLevel >= 1) println(s"Started Monitoring notebook $nbName, url: $url")

while (finalState.isEmpty & //scalastyle:ignore while
while (finalState.isEmpty & //scalastyle:ignore while
(System.currentTimeMillis() - startTime) < timeout &
lifeCycleState != "INTERNAL_ERROR"
) {
Expand Down
1 change: 1 addition & 0 deletions core/src/test/scala/com/microsoft/azure/synapse/ml/nbtest/SynapseTests.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ class SynapseTests extends TestBase {
.filterNot(_.getAbsolutePath.contains("DeepLearningDeepVisionClassification")) // Excluded by design task 1829306
.filterNot(_.getAbsolutePath.contains("VowpalWabbitClassificationusingVWnativeFormat"))
.filterNot(_.getAbsolutePath.contains("VowpalWabbitMulticlassclassification")) // Wait for Synpase fix
.filterNot(_.getAbsolutePath.contains("Langchain")) // Wait for Synpase fix
.sortBy(_.getAbsolutePath)

val expectedPoolCount: Int = selectedPythonFiles.length
Expand Down
Loading