Skip to content

Commit

Permalink
Split off the "normal case" example from the "legacy case" one.
Browse files Browse the repository at this point in the history
* Split off the "normal case" example from the "legacy case" one.

(and create a "normal case" example, first)

* Correct damage from "helpful" IDE renaming help.

* Switch gcloud-java-examples to appassembler

This has the following benefits:

 * Stat example now works
 * Easier to see what is available (list the bin folder)
 * Easier to see what running an example entails (read the script)
 * Simpler command line (no more -Dexec.args)

* Rename example to examples and switch a Stat to ListFilesystems

* Clean up pom

Fix indent to 2, remove redundant groupId line that's inherited from the
parent.

* Reviewer comments and update_docs_version.sh

* Added blank line

* Sweating the small stuff

Adding a \n at the end of the last line.
  • Loading branch information
jean-philippe-martin authored and aozarov committed Apr 5, 2016
1 parent b584349 commit edd0f21
Show file tree
Hide file tree
Showing 10 changed files with 301 additions and 77 deletions.
6 changes: 6 additions & 0 deletions gcloud-java-contrib/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ Contents
--------

* [gcloud-java-nio](./gcloud-java-nio/): NIO Filesystem Provider for Google Cloud Storage.
* [gcloud-java-nio-examples](./gcloud-java-nio-examples/): How to add GCS NIO after the fact.

See also
--------

* [gcloud-java-examples](../gcloud-java-examples) for an example of how to use NIO normally.

Contributing
------------
Expand Down
41 changes: 41 additions & 0 deletions gcloud-java-contrib/gcloud-java-nio-examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
Example of adding the Google Cloud Storage NIO Provider to a legacy jar
=======================================================================

This project shows how to add GCS capabilities to a jar file for a Java 7 application
that uses Java NIO without the need to recompile.

Note that whenever possible, you instead want to recompile the app and use the normal
dependency mechanism to add a dependency to gcloud-java-nio. You can see examples of
this in the [gcloud-java-examples](../../gcloud-java-examples) project.

To run this example:

1. Before running the example, go to the [Google Developers Console][developers-console] to ensure that Google Cloud Storage API is enabled.

2. Log in using gcloud SDK (`gcloud auth login` in command line)

3. Compile the JAR with:
```
mvn package -DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true
```
4. Run the sample with:
```
java -cp gcloud-java-contrib/gcloud-java-nio/target/gcloud-java-nio-0.1.6-SNAPSHOT-shaded.jar:gcloud-java-contrib/gcloud-java-nio-examples/target/gcloud-java-nio-examples-0.1.6-SNAPSHOT.jar com.google.gcloud.nio.examples.ListFilesystems
```
Notice that it lists gcs, which it wouldn't if you ran it without the nio jar:
```
java -cp gcloud-java-contrib/gcloud-java-nio-examples/target/gcloud-java-nio-examples-0.1.6-SNAPSHOT.jar com.google.gcloud.nio.examples.ListFilesystems
```
The sample doesn't have anything about GCS in it. It gets that ability from the nio jar that
we're adding to the classpath. You can use the nio "fat shaded" jar for this purpose as it also
includes the dependencies for gcloud-java-nio.
The underlying mechanism is Java's standard [ServiceLoader](https://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html)
facility, the [standard way](http://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/filesystemprovider.html) to plug in NIO providers like this one.
If you have access to a project's source code you can also simply add gcloud-java-nio as
a dependency and let Maven pull in the required dependencies (this is what the nio unit tests do).
This approach is preferable as the fat jar approach may waste memory on multiple copies of dependencies.
91 changes: 91 additions & 0 deletions gcloud-java-contrib/gcloud-java-nio-examples/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>gcloud-java-nio-examples</artifactId>
<packaging>jar</packaging>
<name>GCloud Java NIO example</name>
<description>
Demonstrates how to use the gcloud-java-nio jar to add GCS functionality to legacy code.
</description>
<parent>
<groupId>com.google.gcloud</groupId>
<artifactId>gcloud-java-contrib</artifactId>
<version>0.1.6-SNAPSHOT</version>
</parent>
<properties>
<site.installationModule>nio</site.installationModule>
</properties>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>gcloud-java-storage</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>19.0</version>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>javax.inject</groupId>
<artifactId>javax.inject</artifactId>
<version>1</version>
</dependency>
<dependency>
<groupId>com.google.auto.service</groupId>
<artifactId>auto-service</artifactId>
<version>1.0-rc2</version>
<scope>provided</scope> <!-- to leave out of the all-deps jar -->
</dependency>
<dependency>
<groupId>com.google.auto.value</groupId>
<artifactId>auto-value</artifactId>
<version>1.1</version>
<scope>provided</scope> <!-- to leave out of the all-deps jar -->
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava-testlib</artifactId>
<version>19.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<version>0.27</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>1.9.5</version>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.4</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<configuration>
<skip>false</skip>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Copyright 2015 Google Inc. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.gcloud.nio.examples;

import java.io.IOException;
import java.net.URI;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.spi.FileSystemProvider;

/**
* ListFilesystems is a super-simple program that lists the available NIO filesystems.
*/
public class ListFilesystems {

/**
* See the class documentation.
*/
public static void main(String[] args) throws IOException {
listFilesystems();
}

private static void listFilesystems() {
System.out.println("Installed filesystem providers:");
for (FileSystemProvider p : FileSystemProvider.installedProviders()) {
System.out.println(" " + p.getScheme());
}
}

}
69 changes: 34 additions & 35 deletions gcloud-java-contrib/gcloud-java-nio/pom.xml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.gcloud</groupId>
<artifactId>gcloud-java-nio</artifactId>
<packaging>jar</packaging>
<name>GCloud Java NIO</name>
Expand Down Expand Up @@ -92,40 +91,40 @@
<compilerArgument>-Xlint:unchecked</compilerArgument>
</configuration>
</plugin>
<!-- shade the jar so we can demo adding the NIO jar to add functionality. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<relocations>
<relocation>
<pattern>com</pattern>
<shadedPattern>shaded.gcloud-nio.com</shadedPattern>
<excludes>
<exclude>com.google.gcloud.**</exclude>
</excludes>
</relocation>
<relocation>
<pattern>org</pattern>
<shadedPattern>shaded.gcloud-nio.org</shadedPattern>
</relocation>
<relocation>
<pattern>google</pattern>
<shadedPattern>shaded.gcloud-nio.google</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- shade the jar so we can demo adding the NIO jar to add functionality. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<relocations>
<relocation>
<pattern>com</pattern>
<shadedPattern>shaded.gcloud-nio.com</shadedPattern>
<excludes>
<exclude>com.google.gcloud.**</exclude>
</excludes>
</relocation>
<relocation>
<pattern>org</pattern>
<shadedPattern>shaded.gcloud-nio.org</shadedPattern>
</relocation>
<relocation>
<pattern>google</pattern>
<shadedPattern>shaded.gcloud-nio.google</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
1 change: 1 addition & 0 deletions gcloud-java-contrib/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
</properties>
<modules>
<module>gcloud-java-nio</module>
<module>gcloud-java-nio-examples</module>
</modules>
<build>
<plugins>
Expand Down
59 changes: 27 additions & 32 deletions gcloud-java-examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ To run examples from your command line:

2. Set your current project using `gcloud config set project PROJECT_ID`. This step is not necessary for `ResourceManagerExample`.

3. Compile using Maven (`mvn compile` in command line from your base project directory)
3. Compile using Maven: `cd gcloud-java-examples` in command line from your base project directory and then `mvn package appassembler:assemble -DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true`.

4. Run an example using Maven from command line.
4. Run an example from the command line using the Maven-generated scripts.

* Here's an example run of `BigQueryExample`.

Expand All @@ -55,62 +55,57 @@ To run examples from your command line:
```
Then you are ready to run the following example:
```
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.bigquery.BigQueryExample" -Dexec.args="create dataset new_dataset_id"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.bigquery.BigQueryExample" -Dexec.args="create table new_dataset_id new_table_id field_name:string"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.bigquery.BigQueryExample" -Dexec.args="list tables new_dataset_id"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.bigquery.BigQueryExample" -Dexec.args="load new_dataset_id new_table_id CSV gs://my_bucket/my_csv_file"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.bigquery.BigQueryExample" -Dexec.args="query 'select * from new_dataset_id.new_table_id'"
target/appassembler/bin/BigQueryExample create dataset new_dataset_id
target/appassembler/bin/BigQueryExample create table new_dataset_id new_table_id field_name:string
target/appassembler/bin/BigQueryExample list tables new_dataset_id
target/appassembler/bin/BigQueryExample load new_dataset_id new_table_id CSV gs://my_bucket/my_csv_file
target/appassembler/bin/BigQueryExample query 'select * from new_dataset_id.new_table_id'
```
* Here's an example run of `DatastoreExample`.
Be sure to change the placeholder project ID "your-project-id" with your own project ID. Also note that you have to enable the Google Cloud Datastore API on the [Google Developers Console][developers-console] before running the following commands.
```
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.datastore.DatastoreExample" -Dexec.args="your-project-id my_name add my\ comment"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.datastore.DatastoreExample" -Dexec.args="your-project-id my_name display"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.datastore.DatastoreExample" -Dexec.args="your-project-id my_name delete"
target/appassembler/bin/DatastoreExample your-project-id my_name add my\ comment
target/appassembler/bin/DatastoreExample your-project-id my_name display
target/appassembler/bin/DatastoreExample your-project-id my_name delete
```
* Here's an example run of `ResourceManagerExample`.
Be sure to change the placeholder project ID "your-project-id" with your own globally unique project ID.
```
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.resourcemanager.ResourceManagerExample" -Dexec.args="create your-project-id"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.resourcemanager.ResourceManagerExample" -Dexec.args="list"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.resourcemanager.ResourceManagerExample" -Dexec.args="get your-project-id"
target/appassembler/bin/ResourceManagerExample create your-project-id
target/appassembler/bin/ResourceManagerExample list
target/appassembler/bin/ResourceManagerExample get your-project-id
```
* Here's an example run of `StorageExample`.
Before running the example, go to the [Google Developers Console][developers-console] to ensure that Google Cloud Storage API is enabled and that you have a bucket. Also ensure that you have a test file (`test.txt` is chosen here) to upload to Cloud Storage stored locally on your machine.
```
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.storage.StorageExample" -Dexec.args="upload /path/to/test.txt <bucket_name>"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.storage.StorageExample" -Dexec.args="list <bucket_name>"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.storage.StorageExample" -Dexec.args="download <bucket_name> test.txt"
mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.storage.StorageExample" -Dexec.args="delete <bucket_name> test.txt"
target/appassembler/bin/StorageExample upload /path/to/test.txt <bucket_name>
target/appassembler/bin/StorageExample list <bucket_name>
target/appassembler/bin/StorageExample download <bucket_name> test.txt
target/appassembler/bin/StorageExample delete <bucket_name> test.txt
```
* Here's an example run of `Stat`, illustrating the use of the gcloud-java-nio jar.
Before running the example, go to the [Google Developers Console][developers-console] to ensure that Google Cloud Storage API is enabled and that you have a bucket with a file in it.
Compile the JAR with:
```
mvn package -DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true
```
Then run the sample with:
* Here's an example run of `Stat`, illustrating the use of gcloud-java-nio.
Before running the example, go to the [Google Developers Console][developers-console] to ensure that Google Cloud Storage API is enabled and that you have a bucket with a file in it.
Run the sample with (from the gcloud-java-examples folder):
```
java -cp gcloud-java-contrib/gcloud-java-nio/target/gcloud-java-nio-0.1.6-SNAPSHOT-shaded.jar:gcloud-java-examples/target/gcloud-java-examples-0.1.6-SNAPSHOT.jar com.google.gcloud.examples.nio.Stat --check
target/appassembler/bin/Stat --check
```
Or, if you have a file in `gs://mybucket/myfile.txt`, you can run:
```
java -cp gcloud-java-contrib/gcloud-java-nio/target/gcloud-java-nio-0.1.6-SNAPSHOT-shaded.jar:gcloud-java-examples/target/gcloud-java-examples-0.1.6-SNAPSHOT.jar com.google.gcloud.examples.nio.Stat gs://mybucket/myfile.txt
target/appassembler/bin/Stat gs://mybucket/myfile.txt
```
The sample doesn't have anything about GCS in it. It gets that ability from the nio jar that
we're adding to the classpath. You can use the nio "fat shaded" jar for this purpose as it also
includes the dependencies for gcloud-java-nio.
If you have access to a project's source code you can also simply add gcloud-java-nio as
a dependency and let Maven pull in the required dependencies (this is what the nio unit tests do).
This approach is preferable as the fat jar approach may waste memory on multiple copies of dependencies.
The sample doesn't have anything special about GCS in it, it just opens files via the NIO API.
It lists gcloud-java-nio as a dependency, and that enables it to interpret `gs://` paths.
Troubleshooting
---------------
Expand Down
Loading

0 comments on commit edd0f21

Please sign in to comment.