[DOC] added basic examples to works with docsrv #59

mcarmonaa · 2017-09-27T12:39:14Z

I just added a few basic examples for spark, pyspark, and jupyter. I was trying to use classifyLanguage and extractUASTsfunctionality but they don't work right now, so I didn't add any example about. I'll try to make it work somehow since #57 doesn't work for me.

It would be nice if you could suggest other kind of examples to include, based on some usage cases.

erizocosmico · 2017-09-27T12:42:52Z

@mcarmonaa if you use the python wrapper branch the issue with the guava dependencies is fixed

codecov · 2017-09-27T12:43:35Z

Codecov Report

Merging #59 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #59   +/-   ##
=========================================
  Coverage     85.07%   85.07%           
  Complexity       66       66           
=========================================
  Files            18       18           
  Lines           563      563           
  Branches        100      100           
=========================================
  Hits            479      479           
  Misses           36       36           
  Partials         48       48

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fd7fe15...a331c28. Read the comment docs.

erizocosmico · 2017-09-27T12:45:20Z

_examples/notebooks/Basic+Example.ipynb

+   },
+   "outputs": [],
+   "source": [
+    "repositoriesDf = spark.read.format(\"tech.sourced.api\").option(\"table\",\"repositories\").load(\"/repositories\")\n",


there is an updated version of this notebook inside the python wrapper branch that uses the python wrapper

ajnavarro · 2017-09-27T14:44:16Z

_examples/notebooks/Basic+Example.ipynb

@@ -0,0 +1,178 @@
+{


I would prefer to have all the notebooks in one place. Notebooks are executables, so in this case it should be in the examples folder. Putting them all into the same place we avoid duplication and we can handle api changes better.

For the moment this file is duplicated here because of docsrv

@ajnavarro @mcarmonaa Could it be an alternative to put them wherever they should be, and then update the _examples/README.md link? I mean: point to the original one, instead of pointing to a copy.
(it should fix the problem of the duplication)

If you prefer to use a copy, I'd use a symbolic link to maintain the synchronicity

mcarmonaa · 2017-09-28T11:21:15Z

I've just added examples of classifyLanguages and extractUASTs

erizocosmico · 2017-10-03T15:33:27Z

_examples/pyspark/pyspark-shell-classifying-languages.md

@@ -0,0 +1,32 @@
+```bash


What do you think about adding title and a brief explanation of what is gonna happen in the snippet below at the start of each of these markdown files?

erizocosmico · 2017-10-04T07:42:09Z

_examples/pyspark/pyspark-shell-classifying-languages.md

@@ -0,0 +1,38 @@
+## Classifying languages example
+
+This example uses the pyspark-shell to show a basic usage of the UDF `classify_languages()`.


maybe we should change UDF, because to someone that's not familiar with Spark it sounds really alien

to show how to classify files by their language maybe?

erizocosmico · 2017-10-04T07:42:58Z

_examples/pyspark/pyspark-shell-lang-and-uast.md

@@ -0,0 +1,39 @@
+## Classifying languages and extracting UASTs example
+
+The combined usage of both UDFs `classify_languages()` and `extract_uasts()` has the advantage that doesn't rely the language detection task on the [bblfsh server](https://github.com/bblfsh/server) , so you can save some time.


same as above

erizocosmico · 2017-10-04T07:44:12Z

_examples/scala/spark-shell-classifying-languages.md

@@ -0,0 +1,41 @@
+## Classifying languages example
+
+This example uses the spark-shell to show a basic usage of the UDF `classifyLanguages`.


erizocosmico · 2017-10-04T07:44:20Z

_examples/scala/spark-shell-lang-and-uast.md

@@ -0,0 +1,42 @@
+## Classifying languages and extracting UASTs example
+
+The combined usage of both UDFs `classifyLanguages` and `extractUASTs` has the advantage that doesn't rely the language detection task on the [bblfsh server](https://github.com/bblfsh/server) , so you can save some time.


dpordomingo

I found problems with absolute links.
You may consider the following example, and apply it in the others.

dpordomingo · 2017-10-05T09:46:44Z

_examples/README.md

+
+### pyspark
+
+- [pyspark's shell  basic example](https://github.com/src-d/spark-api/blob/master/_examples/pyspark/pyspark-shell-basic.md)


I think the link is broken.

If you go to the original file:
https://github.com/mcarmonaa/spark-api/blob/feature/docsrv/_examples/README.md
And try it, it fails.

If you change it by the relative format, it seems to work:

- [pyspark's shell basic example](pyspark/pyspark-shell-basic.md)

Ok, I'll change the links to the relative format!

dpordomingo

Many thanks for the documentation and examples!

erizocosmico · 2017-10-10T15:54:22Z

Shall we merge this?

erizocosmico approved these changes Sep 27, 2017

View reviewed changes

erizocosmico reviewed Sep 27, 2017

View reviewed changes

ajnavarro reviewed Sep 27, 2017

View reviewed changes

erizocosmico suggested changes Oct 3, 2017

View reviewed changes

mcarmonaa added 5 commits October 4, 2017 08:22

added basic examples

6016e32

added uast extraction examples

363b09c

added classifyLanguages examples

0a9e115

added print schemas examples

b1cfe75

combined language classification and uast extraction examples

f034b50

erizocosmico suggested changes Oct 4, 2017

View reviewed changes

added title and brief explanations to the examples

9789aca

ajnavarro approved these changes Oct 4, 2017

View reviewed changes

dpordomingo suggested changes Oct 5, 2017

View reviewed changes

changed links to relative format

a331c28

ajnavarro approved these changes Oct 6, 2017

View reviewed changes

dpordomingo approved these changes Oct 6, 2017

View reviewed changes

erizocosmico approved these changes Oct 10, 2017

View reviewed changes

ajnavarro approved these changes Oct 10, 2017

View reviewed changes

ajnavarro merged commit fbd8f70 into src-d:master Oct 10, 2017

mcarmonaa deleted the feature/docsrv branch October 11, 2017 08:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DOC] added basic examples to works with docsrv #59

[DOC] added basic examples to works with docsrv #59

mcarmonaa commented Sep 27, 2017

erizocosmico commented Sep 27, 2017

codecov bot commented Sep 27, 2017 •

edited

Loading

erizocosmico Sep 27, 2017

ajnavarro Sep 27, 2017

mcarmonaa Sep 28, 2017

ajnavarro Sep 28, 2017

dpordomingo Oct 5, 2017 •

edited

Loading

mcarmonaa commented Sep 28, 2017

erizocosmico Oct 3, 2017

mcarmonaa Oct 4, 2017

erizocosmico Oct 4, 2017

erizocosmico Oct 4, 2017

erizocosmico Oct 4, 2017

erizocosmico Oct 4, 2017

dpordomingo left a comment

dpordomingo Oct 5, 2017

mcarmonaa Oct 5, 2017

dpordomingo left a comment

erizocosmico commented Oct 10, 2017

		@@ -0,0 +1,38 @@
		## Classifying languages example

		This example uses the pyspark-shell to show a basic usage of the UDF `classify_languages()`.

		@@ -0,0 +1,39 @@
		## Classifying languages and extracting UASTs example

		The combined usage of both UDFs `classify_languages()` and `extract_uasts()` has the advantage that doesn't rely the language detection task on the [bblfsh server](https://github.com/bblfsh/server) , so you can save some time.

		@@ -0,0 +1,41 @@
		## Classifying languages example

		This example uses the spark-shell to show a basic usage of the UDF `classifyLanguages`.

		@@ -0,0 +1,42 @@
		## Classifying languages and extracting UASTs example

		The combined usage of both UDFs `classifyLanguages` and `extractUASTs` has the advantage that doesn't rely the language detection task on the [bblfsh server](https://github.com/bblfsh/server) , so you can save some time.


		### pyspark

		- [pyspark's shell basic example](https://github.com/src-d/spark-api/blob/master/_examples/pyspark/pyspark-shell-basic.md)

[DOC] added basic examples to works with docsrv #59

[DOC] added basic examples to works with docsrv #59

Conversation

mcarmonaa commented Sep 27, 2017

erizocosmico commented Sep 27, 2017

codecov bot commented Sep 27, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dpordomingo Oct 5, 2017 • edited Loading

Choose a reason for hiding this comment

mcarmonaa commented Sep 28, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dpordomingo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dpordomingo left a comment

Choose a reason for hiding this comment

erizocosmico commented Oct 10, 2017

codecov bot commented Sep 27, 2017 •

edited

Loading

dpordomingo Oct 5, 2017 •

edited

Loading