Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigtable emulator #3840

Merged
merged 30 commits into from
Nov 8, 2018
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
733584d
WIP: bigtable emulator
igorbernstein2 Oct 19, 2018
471f166
Tests & fixes
igorbernstein2 Oct 19, 2018
d84207f
Merge remote-tracking branch 'origin/master' into bigtable-emulator
igorbernstein2 Oct 19, 2018
51c61c8
visibility tweaks
igorbernstein2 Oct 19, 2018
c21a61e
fixes
igorbernstein2 Oct 31, 2018
b0b122d
Merge remote-tracking branch 'origin/master' into bigtable-emulator
igorbernstein2 Oct 31, 2018
a294c5a
readme + bom
igorbernstein2 Oct 31, 2018
5ef9eca
more readme
igorbernstein2 Oct 31, 2018
ee25573
Merge remote-tracking branch 'origin/master' into bigtable-emulator
igorbernstein2 Oct 31, 2018
5b5f708
Merge branch 'master' into bigtable-emulator
igorbernstein2 Nov 1, 2018
52e211b
create tmp manifest file in same dir as dest to avoid issues with cro…
igorbernstein2 Nov 1, 2018
558c78c
Merge branch 'bigtable-emulator' of github.com:igorbernstein2/google-…
igorbernstein2 Nov 1, 2018
870d3bb
Revert "create tmp manifest file in same dir as dest to avoid issues …
igorbernstein2 Nov 1, 2018
2c7b08f
Fix cross filesystem issues
igorbernstein2 Nov 1, 2018
fd2185e
java 7 fixes
igorbernstein2 Nov 2, 2018
fba8f50
Merge branch 'master' of github.com:GoogleCloudPlatform/google-cloud-…
igorbernstein2 Nov 2, 2018
368ce60
fix bom as well
igorbernstein2 Nov 2, 2018
42ee7e3
fix readme
igorbernstein2 Nov 2, 2018
7615628
still fixing java 7 issues
igorbernstein2 Nov 2, 2018
b9f8ba5
clean up
igorbernstein2 Nov 2, 2018
949b8c5
oops
igorbernstein2 Nov 2, 2018
5ee1ddb
fix windows build
igorbernstein2 Nov 6, 2018
b14326e
Merge branch 'master' of github.com:GoogleCloudPlatform/google-cloud-…
igorbernstein2 Nov 6, 2018
2ec37d8
Minor dep tweaks
igorbernstein2 Nov 6, 2018
1afa15a
tweaks
igorbernstein2 Nov 6, 2018
a9e7ae3
tweaks
igorbernstein2 Nov 6, 2018
2fbde43
Merge branch 'bigtable-emulator' of github.com:igorbernstein2/google-…
igorbernstein2 Nov 6, 2018
65f4c80
Merge branch 'master' of github.com:GoogleCloudPlatform/google-cloud-…
igorbernstein2 Nov 8, 2018
31769e9
tweak
igorbernstein2 Nov 8, 2018
9daec05
license
igorbernstein2 Nov 8, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions google-cloud-bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,11 @@
<artifactId>grpc-google-cloud-bigtable-admin-v2</artifactId>
<version>0.34.1-SNAPSHOT</version><!-- {x-version-update:grpc-google-cloud-bigtable-admin-v2:current} -->
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigtable-emulator</artifactId>
<version>0.69.1-alpha-SNAPSHOT</version><!-- {x-version-update:google-cloud-bigtable-emulator:current} -->
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigquery</artifactId>
Expand Down
181 changes: 181 additions & 0 deletions google-cloud-testing/google-cloud-bigtable-emulator/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Google Cloud Java Emulator for Bigtable

A Java wrapper for the [Cloud Bigtable][cloud-bigtable] emulator. This
wrapper bundles the native Bigtable emulator and exposes a simple Java
interface to ease writing tests. Please note that this wrapper is under
heavy development and APIs may change in the future.

[![Kokoro CI](http://storage.googleapis.com/cloud-devrel-public/java/badges/google-cloud-java/master.svg)](http://storage.googleapis.com/cloud-devrel-public/java/badges/google-cloud-java/master.html)
[![Maven](https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigtable-emulator.svg)](https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigtable-emulator.svg)
[![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/google-cloud-java)

## Quickstart

[//]: # ({x-version-update-start:google-cloud-bom:released})
If you are using Maven, add this to your pom.xml file
```xml
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bom</artifactId>
<version>0.69.0-alpha</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigtable</artifactId>
</dependency>

<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigtable-admin</artifactId>
</dependency>

<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigtable-emulator</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
</dependencies>```

If you are using Gradle, add this to your dependencies
```Groovy
compile 'com.google.cloud:google-cloud-bigtable:0.69.0-alpha'
compile 'com.google.cloud:google-cloud-bigtable-admin:0.69.0-alpha'
testCompile 'com.google.cloud:google-cloud-bigtable-emulator:0.69.0-alpha'
testCompile 'junit:junit:4.12'
```
If you are using SBT, add this to your dependencies
```Scala
libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "0.69.0-alpha"
libraryDependencies += "com.google.cloud" % "google-cloud-bigtable-admin" % "0.69.0-alpha"
libraryDependencies += "com.google.cloud" % "google-cloud-bigtable-emulator" % "0.69.0-alpha" % Test
libraryDependencies += "junit" % "junit" % "4.12" % Test
```
[//]: # ({x-version-update-end})

## Getting Started

Here is a code snippet showing a simple JUnit test. Add the following imports
at the top of your file:

```java
import com.google.api.core.ApiFuture;
import com.google.api.gax.core.NoCredentialsProvider;
import com.google.api.gax.grpc.GrpcTransportChannel;
import com.google.api.gax.rpc.FixedTransportChannelProvider;
import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient;
import com.google.cloud.bigtable.admin.v2.BigtableTableAdminSettings;
import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest;
import com.google.cloud.bigtable.data.v2.BigtableDataClient;
import com.google.cloud.bigtable.data.v2.BigtableDataSettings;
import com.google.cloud.bigtable.data.v2.models.Row;
import com.google.cloud.bigtable.data.v2.models.RowMutation;
import com.google.cloud.bigtable.emulator.v2.BigtableEmulatorRule;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
```

Then, to make a query to Bigtable, use the following code:
```java
@RunWith(JUnit4.class)
public class ExampleTest {
// Initialize the emulator Rule
@Rule
public final BigtableEmulatorRule bigtableEmulator = BigtableEmulatorRule.create();

// Clients that will be connected to the emulator
private BigtableTableAdminClient tableAdminClient;
private BigtableDataClient dataClient;

@Before
public void setUp() throws IOException {
// Initialize the clients to connect to the emulator
BigtableTableAdminSettings.Builder tableAdminSettings = BigtableTableAdminSettings.newBuilder()
.setInstanceName(com.google.bigtable.admin.v2.InstanceName.of("fake-project", "fake-instance"));
tableAdminSettings.stubSettings()
.setCredentialsProvider(NoCredentialsProvider.create())
.setTransportChannelProvider(
FixedTransportChannelProvider.create(
GrpcTransportChannel.create(bigtableEmulator.getAdminChannel())
)
);
tableAdminClient = BigtableTableAdminClient.create(tableAdminSettings.build());

BigtableDataSettings.Builder dataSettings = BigtableDataSettings.newBuilder()
.setInstanceName(com.google.cloud.bigtable.data.v2.models.InstanceName.of("fake-project", "fake-instance"))

This comment was marked as spam.

This comment was marked as spam.

.setCredentialsProvider(NoCredentialsProvider.create())
.setTransportChannelProvider(
FixedTransportChannelProvider.create(
GrpcTransportChannel.create(bigtableEmulator.getDataChannel())
)
);
dataClient = BigtableDataClient.create(dataSettings.build());

// Create a test table that can be used in tests
tableAdminClient.createTable(
CreateTableRequest.of("fake-table")
.addFamily("cf")
);
}

@Test
public void testWriteRead() throws ExecutionException, InterruptedException {
ApiFuture<Void> mutateFuture = dataClient.mutateRowAsync(
RowMutation.create("fake-table", "fake-key")
.setCell("cf", "col", "value")
);

mutateFuture.get();

ApiFuture<Row> rowFuture = dataClient.readRowAsync("fake-table", "fake-key");

Assert.assertEquals("value", rowFuture.get().getCells().get(0).getValue().toStringUtf8());
}
}
```

## Java Versions

Java 7 or above is required for using this client.

## Versioning

This library follows [Semantic Versioning](http://semver.org/).

It is currently in major version zero (`0.y.z`), which means that anything may
change at any time and the public API should not be considered stable.

## Contributing

Contributions to this library are always welcome and highly encouraged.

See [CONTRIBUTING] for more information on how to get started and [DEVELOPING] for a layout of the
codebase.

## License

Apache 2.0 - See [LICENSE] for more information.

[CONTRIBUTING]:https://github.com/googleapis/google-cloud-java/blob/master/CONTRIBUTING.md
[LICENSE]: https://github.com/googleapis/google-cloud-java/blob/master/LICENSE
[cloud-bigtable]: https://cloud.google.com/bigtable/

90 changes: 90 additions & 0 deletions google-cloud-testing/google-cloud-bigtable-emulator/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>google-cloud-testing</artifactId>
<groupId>com.google.cloud</groupId>
<version>0.69.1-alpha-SNAPSHOT</version><!-- {x-version-update:google-cloud-testing:current} -->
</parent>
<modelVersion>4.0.0</modelVersion>

<artifactId>google-cloud-bigtable-emulator</artifactId>
<version>0.69.1-alpha-SNAPSHOT</version><!-- {x-version-update:google-cloud-bigtable-emulator:current} -->

<build>
<plugins>
<plugin>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-gcloud-maven-plugin</artifactId>
<version>0.69.1-alpha-SNAPSHOT</version><!-- {x-version-update:google-cloud-gcloud-maven-plugin:current} -->

<executions>
<execution>
<id>gen-sources</id>
<phase>generate-resources</phase>
<goals>
<goal>download</goal>

This comment was marked as spam.

This comment was marked as spam.

</goals>
<configuration>
<componentNames>
<componentName>bigtable-darwin-x86</componentName>
<componentName>bigtable-darwin-x86_64</componentName>
<componentName>bigtable-linux-x86</componentName>
<componentName>bigtable-linux-x86_64</componentName>
<componentName>bigtable-windows-x86</componentName>
<componentName>bigtable-windows-x86_64</componentName>
</componentNames>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

<dependencies>
<!-- grpc deps are provided by the client -->
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-core</artifactId>
<version>1.13.1</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
<version>1.13.1</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>com.google.api</groupId>
<artifactId>api-common</artifactId>
</dependency>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>compile</scope>
<optional>true</optional>
</dependency>

<dependency>
<groupId>com.google.api.grpc</groupId>
<artifactId>grpc-google-cloud-bigtable-v2</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>com.google.api.grpc</groupId>
<artifactId>grpc-google-cloud-bigtable-admin-v2</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/*
* Copyright 2018 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.google.cloud.bigtable.emulator.v2;

import com.google.api.core.BetaApi;
import io.grpc.ManagedChannel;
import org.junit.rules.ExternalResource;

/**
* The BigtableEmulatorRule manages the lifecycle of the Bigtable {@link Emulator}. Before the
* start of a test, the emulator will be started on a random port and will be shutdown after the
* test finishes.
*
* <p>Example usage:
*
* <code>{@code
* {@literal @RunWith(JUnit4.class)}
* public class MyTest {
* {@literal @Rule}
* public final BigtableEmulatorRule bigtableEmulator = BigtableEmulatorRule.create();
*
* {@literal @Test}
* public void testUsingEmulator() {
* ManagedChannel adminChannel = bigtableEmulator.getAdminChannel();
* // Do something with channel
* }
* }
* }</code>
*/
@BetaApi("Surface for Bigtable emulator is not yet stable")
public class BigtableEmulatorRule extends ExternalResource {

This comment was marked as spam.

private Emulator emulator;

public static BigtableEmulatorRule create() {
return new BigtableEmulatorRule();
}

private BigtableEmulatorRule() { }

/** Initializes the Bigtable emulator before a test runs. */
@Override
protected void before() throws Throwable {
emulator = Emulator.createBundled();
emulator.start();
}

/** Stops the Bigtable emulator after a test finishes. */
@Override
protected void after() {
emulator.stop();
emulator = null;
}

/**
* Gets a {@link ManagedChannel} connected to the Emulator. The channel is configured for data
* operations.
*/
public ManagedChannel getDataChannel() {
return emulator.getDataChannel();
}

/**
* Gets a {@link ManagedChannel} connected to the Emulator. This channel should be used for admin
* operations.
*/
public ManagedChannel getAdminChannel() {
return emulator.getAdminChannel();
}

/**
* Gets the port of the emulator, allowing the caller to create their own {@link ManagedChannel}.
*/
public int getPort() {
return emulator.getPort();
}
}
Loading