Add a Community Projects page

[velvia]

This adds a new page to the docs listing community projects -- those created outside of Apache Spark that are of interest to the community of Spark users. Anybody can add to it just by submitting a PR.

There was a discussion thread about alternatives:

• Creating a Github organization for Spark projects - we could not find any sponsors for this, and it would be difficult to organize since many folks just create repos in their company organization or personal accounts
• Apache has some place for storing community projects, but it was deemed difficult to work with, and again would be some permissions issues -- not everyone could update it.

[SparkQA]

QA tests have started for PR 2219 at commit 613b021.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2219 at commit 613b021.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ExecutorClassLoader(conf: SparkConf, classUri: String, parent: ClassLoader,

[mateiz]

Good idea. Just to keep these up to date more easily, though, it might be better to put them on the wiki (https://cwiki.apache.org/confluence/display/SPARK/Wiki+Homepage) and link to the wiki from the official docs, like we do for Contributing to Spark. That way all versions of the docs will point to an up-to-date version of this page. Do you want to do that? If you sign up for an account on the wiki and tell me your username on it, I can give you edit access.

[velvia]

@mateiz Sure, my username is "velvia". I can create a new wiki page with the contents of the page in this PR. Thanks!

[mateiz]

@velvia okay, added you. When you do this, you might also want to move over the "Software Projects" list from the bottom of https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark (it's quite lost in there now).

[SparkQA]

QA tests have started for PR 2219 at commit 613b021.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2219 at commit 613b021.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class AddJar(path: String) extends LeafNode with Command

[pwendell]

Hey btw - I cleaned up the powered by page a bit to remove some older projects and I listed the JobServer first since it's one of the newer ones. We can also move the community projects to the top of the page so they get more visiblity.

[velvia]

Hi Patrick,

I was going to create a new community projects page, but if you think they
can stay on the powered by page but on top, that's fine too.

On Sun, Sep 7, 2014 at 10:23 PM, Patrick Wendell [email protected]
wrote:

Hey btw - I cleaned up the powered by page a bit to remove some older
projects and I listed the JobServer first since it's one of the newer ones.
We can also move the community projects to the top of the page so they get
more visiblity.

—
Reply to this email directly or view it on GitHub
#2219 (comment).

The fruit of silence is prayer;
the fruit of prayer is faith;
the fruit of faith is love;
the fruit of love is service;
the fruit of service is peace. -- Mother Teresa

[pwendell]

@velvia ah - I think either one is fine... we can also add another top level page.

[pwendell]

@velvia so is this subsumed by the wiki page then?

[velvia]

Let me modify this PR to point to the wiki page, sounds good? I think
having that menu item is still helpful....

On Sun, Sep 14, 2014 at 9:20 PM, Patrick Wendell [email protected]
wrote:

@velvia https://github.com/velvia so is this subsumed by the wiki page
then?

—
Reply to this email directly or view it on GitHub
#2219 (comment).

The fruit of silence is prayer;
the fruit of prayer is faith;
the fruit of faith is love;
the fruit of love is service;
the fruit of service is peace. -- Mother Teresa

[velvia]

@pwendell Ok, modified it, it's now just a tiny addition of a menu item to point to the wiki page.

[SparkQA]

QA tests have started for PR 2219 at commit 7316822.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2219 at commit 7316822.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class JavaSparkContext(val sc: SparkContext)
  • class TaskCompletionListenerException(errorMessages: Seq[String]) extends Exception
  • class RatingDeserializer(FramedSerializer):
  • class Encoder[T <: NativeType](columnType: NativeColumnType[T]) extends compression.Encoder[T]
  • class Encoder[T <: NativeType](columnType: NativeColumnType[T]) extends compression.Encoder[T]
  • class Encoder[T <: NativeType](columnType: NativeColumnType[T]) extends compression.Encoder[T]
  • class Encoder extends compression.Encoder[IntegerType.type]
  • class Decoder(buffer: ByteBuffer, columnType: NativeColumnType[IntegerType.type])
  • class Encoder extends compression.Encoder[LongType.type]
  • class Decoder(buffer: ByteBuffer, columnType: NativeColumnType[LongType.type])
  • class JavaStreamingContext(val ssc: StreamingContext) extends Closeable

[pwendell]

Okay thanks we can pull this in.