From 844e5beb6230df6ca220935056aae7f6e73d2bc0 Mon Sep 17 00:00:00 2001 From: kolea2 <45548808+kolea2@users.noreply.github.com> Date: Tue, 15 Dec 2020 22:06:13 +0000 Subject: [PATCH 01/38] docs: use autogenerated readme functionality and regenerate (#568) * docs: use autogenerated readme functionality and regenerate * update version --- .readme-partials.yml | 325 +++++++++++++++++++++++++++++++++++++++++++ README.md | 154 ++++++++++++++------ synth.py | 1 - 3 files changed, 437 insertions(+), 43 deletions(-) create mode 100644 .readme-partials.yml diff --git a/.readme-partials.yml b/.readme-partials.yml new file mode 100644 index 0000000000..62730df47d --- /dev/null +++ b/.readme-partials.yml @@ -0,0 +1,325 @@ +custom_content: | + ## About Cloud Bigtable + + [Cloud Bigtable][cloud-bigtable] is Google's NoSQL Big Data database service. It's + the same database that powers many core Google services, including Search, Analytics, Maps, and + Gmail. + + Be sure to activate the Cloud Bigtable API and the Cloud Bigtable Admin API under APIs & Services in the GCP Console to use Cloud Bigtable from your project. + + See the Bigtable client library documentation ([Admin API](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/package-summary.html) and [Data API](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/package-summary.html)) to learn how to + interact with Cloud Bigtable using this Client Library. + + ## Concepts + + Cloud Bigtable is composed of instances, clusters, nodes and tables. + + ### Instances + Instances are containers for clusters. + + ### Clusters + Clusters represent the actual Cloud Bigtable service. Each cluster belongs to a single Cloud Bigtable instance, and an instance can have up to 4 clusters. When your application + sends requests to a Cloud Bigtable instance, those requests are actually handled by one of the clusters in the instance. + + ### Nodes + Each cluster in a production instance has 3 or more nodes, which are compute resources that Cloud Bigtable uses to manage your data. + + ### Tables + Tables contain the actual data and are replicated across all of the clusters in an instance. + + + ## Clients + The Cloud Bigtable API consists of: + + ### Data API + Allows callers to persist and query data in a table. It's exposed by [BigtableDataClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/BigtableDataClient.html). + + ### Admin API + Allows callers to create and manage instances, clusters, tables, and access permissions. This API is exposed by: [BigtableInstanceAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableInstanceAdminClient.html) for Instance and Cluster level resources. + + See [BigtableTableAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableTableAdminClient.html) for table management. + + See [BigtableDataClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/BigtableDataClient.html) for the data client. + + See [BigtableInstanceAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableInstanceAdminClient.html) for the instance admin client. + + See [BigtableTableAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableTableAdminClient.html) for the table admin client. + + #### Calling Cloud Bigtable + + The Cloud Bigtable API is split into 3 parts: Data API, Instance Admin API and Table Admin API. + + Here is a code snippet showing simple usage of the Data API. Add the following imports + at the top of your file: + + ```java + import com.google.cloud.bigtable.data.v2.BigtableDataClient; + import com.google.cloud.bigtable.data.v2.models.Query; + import com.google.cloud.bigtable.data.v2.models.Row; + + ``` + + Then, to make a query to Bigtable, use the following code: + ```java + // Instantiates a client + String projectId = "my-project"; + String instanceId = "my-instance"; + String tableId = "my-table"; + + // Create the client. + // Please note that creating the client is a very expensive operation + // and should only be done once and shared in an application. + BigtableDataClient dataClient = BigtableDataClient.create(projectId, instanceId); + + try { + // Query a table + Query query = Query.create(tableId) + .range("a", "z") + .limit(26); + + for (Row row : dataClient.readRows(query)) { + System.out.println(row.getKey()); + } + } finally { + dataClient.close(); + } + ``` + + The Admin APIs are similar. Here is a code snippet showing how to create a table. Add the following + imports at the top of your file: + + ```java + import static com.google.cloud.bigtable.admin.v2.models.GCRules.GCRULES; + import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient; + import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest; + import com.google.cloud.bigtable.admin.v2.models.Table; + ``` + + Then, to create a table, use the following code: + ```java + String projectId = "my-instance"; + String instanceId = "my-database"; + + BigtableTableAdminClient tableAdminClient = BigtableTableAdminClient + .create(projectId, instanceId); + + try { + tableAdminClient.createTable( + CreateTableRequest.of("my-table") + .addFamily("my-family") + ); + } finally { + tableAdminClient.close(); + } + ``` + + TIP: If you are experiencing version conflicts with gRPC, see [Version Conflicts](#version-conflicts). + + ## OpenCensus Tracing + + Cloud Bigtable client supports [OpenCensus Tracing](https://opencensus.io/tracing/), + which gives insight into the client internals and aids in debugging production issues. + By default, the functionality is disabled. For example to enable tracing using + [Google Stackdriver](https://cloud.google.com/trace/docs/): + + [//]: # (TODO: figure out how to keep opencensus version in sync with pom.xml) + + If you are using Maven, add this to your pom.xml file + ```xml + + io.opencensus + opencensus-impl + 0.24.0 + runtime + + + io.opencensus + opencensus-exporter-trace-stackdriver + 0.24.0 + + + io.grpc + * + + + com.google.auth + * + + + + ``` + If you are using Gradle, add this to your dependencies + ```Groovy + compile 'io.opencensus:opencensus-impl:0.24.0' + compile 'io.opencensus:opencensus-exporter-trace-stackdriver:0.24.0' + ``` + If you are using SBT, add this to your dependencies + ```Scala + libraryDependencies += "io.opencensus" % "opencensus-impl" % "0.24.0" + libraryDependencies += "io.opencensus" % "opencensus-exporter-trace-stackdriver" % "0.24.0" + ``` + + At the start of your application configure the exporter: + + ```java + import io.opencensus.exporter.trace.stackdriver.StackdriverTraceConfiguration; + import io.opencensus.exporter.trace.stackdriver.StackdriverTraceExporter; + + StackdriverTraceExporter.createAndRegister( + StackdriverTraceConfiguration.builder() + .setProjectId("YOUR_PROJECT_ID") + .build()); + ``` + + By default traces are [sampled](https://opencensus.io/tracing/sampling) at a rate of about 1/10,000. + You can configure a higher rate by updating the active tracing params: + + ```java + import io.opencensus.trace.Tracing; + import io.opencensus.trace.samplers.Samplers; + + Tracing.getTraceConfig().updateActiveTraceParams( + Tracing.getTraceConfig().getActiveTraceParams().toBuilder() + .setSampler(Samplers.probabilitySampler(0.01)) + .build() + ); + ``` + + ## OpenCensus Stats + + Cloud Bigtable client supports [Opencensus Metrics](https://opencensus.io/stats/), + which gives insight into the client internals and aids in debugging production issues. + All Cloud Bigtable Metrics are prefixed with `cloud.google.com/java/bigtable/`. The + metrics will be tagged with: + * `bigtable_project_id`: the project that contains the target Bigtable instance. + Please note that this id could be different from project that the client is running + in and different from the project where the metrics are exported to. + * `bigtable_instance_id`: the instance id of the target Bigtable instance + * `bigtable_app_profile_id`: the app profile id that is being used to access the target + Bigtable instance + + ### Available operation level metric views: + + * `cloud.google.com/java/bigtable/op_latency`: A distribution of latency of + each client method call, across all of it's RPC attempts. Tagged by + operation name and final response status. + + * `cloud.google.com/java/bigtable/completed_ops`: The total count of + method invocations. Tagged by operation name and final response status. + + * `cloud.google.com/java/bigtable/read_rows_first_row_latency`: A + distribution of the latency of receiving the first row in a ReadRows + operation. + + * `cloud.google.com/java/bigtable/attempt_latency`: A distribution of latency of + each client RPC, tagged by operation name and the attempt status. Under normal + circumstances, this will be identical to op_latency. However, when the client + receives transient errors, op_latency will be the sum of all attempt_latencies + and the exponential delays + + * `cloud.google.com/java/bigtable/attempts_per_op`: A distribution of attempts that + each operation required, tagged by operation name and final operation status. + Under normal circumstances, this will be 1. + + + By default, the functionality is disabled. For example to enable metrics using + [Google Stackdriver](https://cloud.google.com/monitoring/docs/): + + + [//]: # (TODO: figure out how to keep opencensus version in sync with pom.xml) + + If you are using Maven, add this to your pom.xml file + ```xml + + io.opencensus + opencensus-impl + 0.24.0 + runtime + + + io.opencensus + opencensus-exporter-stats-stackdriver + 0.24.0 + + + io.grpc + * + + + com.google.auth + * + + + + ``` + If you are using Gradle, add this to your dependencies + ```Groovy + compile 'io.opencensus:opencensus-impl:0.24.0' + compile 'io.opencensus:opencensus-exporter-stats-stackdriver:0.24.0' + ``` + If you are using SBT, add this to your dependencies + ```Scala + libraryDependencies += "io.opencensus" % "opencensus-impl" % "0.24.0" + libraryDependencies += "io.opencensus" % "opencensus-exporter-stats-stackdriver" % "0.24.0" + ``` + + At the start of your application configure the exporter and enable the Bigtable stats views: + + ```java + import io.opencensus.exporter.stats.stackdriver.StackdriverStatsConfiguration; + import io.opencensus.exporter.stats.stackdriver.StackdriverStatsExporter; + + StackdriverStatsExporter.createAndRegister( + StackdriverStatsConfiguration.builder() + .setProjectId("YOUR_PROJECT_ID") + .build() + ); + + BigtableDataSettings.enableOpenCensusStats(); + ``` + + ## Version Conflicts + + google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought + in by other libraries, for example Apache Beam. This happens because internal dependencies + between gRPC libraries are pinned to an exact version of grpc-core + (see [here](https://github.com/grpc/grpc-java/commit/90db93b990305aa5a8428cf391b55498c7993b6e)). + If both google-cloud-bigtable and the other library bring in two gRPC libraries that depend + on the different versions of grpc-core, then dependency resolution will fail. + The easiest way to fix this is to depend on the gRPC bom, which will force all the gRPC + transitive libraries to use the same version. + + Add the following to your project's pom.xml. + + ``` + + + + io.grpc + grpc-bom + 1.28.0 + pom + import + + + + ``` + + ## Container Deployment + + While deploying this client in [Google Kubernetes Engine(GKE)](https://cloud.google.com/kubernetes-engine) with [CoS](https://cloud.google.com/container-optimized-os/docs/). Please make sure to provide CPU configuration in your deployment file. With default configuration JVM detects only 1 CPU, which affects the number of channels with the client, resulting in performance repercussion. + + Also, The number of `grpc-nio-worker-ELG-1-#` thread is same as number of CPUs. These are managed by a single `grpc-default-executor-#` thread, which is shared among multiple client instances. + + For example: + ```yaml + appVersion: v1 + ... + spec: + ... + container: + resources: + requests: + cpu: "1" # Here 1 represents 100% of single node CPUs whereas other than 1 represents the number of CPU it would use from a node. + ``` + see [Assign CPU Resources to Containers](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/#specify-a-cpu-request-and-a-cpu-limit) for more information. diff --git a/README.md b/README.md index a0e24f99b2..b4178932a4 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,48 @@ # Google Cloud Bigtable Client for Java -Java idiomatic client for [Cloud Bigtable][cloud-bigtable]. +Java idiomatic client for [Cloud Bigtable][product-docs]. -[![Maven](https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigtable.svg)](https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigtable.svg) -[![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/google-cloud-java) +[![Maven][maven-version-image]][maven-version-link] +![Stability][stability-image] -- [Product Documentation][bigtable-product-docs] -- [Client Library Documentation](https://googleapis.dev/java/google-cloud-bigtable/latest/index.html) +- [Product Documentation][product-docs] +- [Client Library Documentation][javadocs] ## Quickstart -[//]: # ({x-version-update-start:google-cloud-bigtable:released}) -If you are using Maven, add this to your pom.xml file +If you are using Maven with [BOM][libraries-bom], add this to your pom.xml file +```xml + + + + com.google.cloud + libraries-bom + 16.1.0 + pom + import + + + + + + + com.google.cloud + google-cloud-bigtable + + +``` + +If you are using Maven without BOM, add this to your dependencies: + ```xml com.google.cloud google-cloud-bigtable 1.19.2 + ``` + If you are using Gradle, add this to your dependencies ```Groovy compile 'com.google.cloud:google-cloud-bigtable:1.19.2' @@ -27,28 +51,38 @@ If you are using SBT, add this to your dependencies ```Scala libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.19.2" ``` -[//]: # ({x-version-update-end}) ## Authentication -See the -[Authentication](https://github.com/googleapis/google-cloud-java#authentication) -section in the base directory's README. +See the [Authentication][authentication] section in the base directory's README. ## Getting Started -#### Prerequisites -For this tutorial, you will need a -[Google Cloud Platform Console](https://console.developers.google.com/) project with the Cloud Bigtable -API enabled. You will need to -[enable billing](https://support.google.com/cloud/answer/6158867?hl=en) to use Google Cloud Bigtable. -[Follow these instructions](https://cloud.google.com/resource-manager/docs/creating-managing-projects) to get your -project set up. You will also need to set up the local development environment by [installing the -Google Cloud SDK](https://cloud.google.com/sdk/) and running the following commands in command line: -`gcloud auth login`. + +### Prerequisites + +You will need a [Google Cloud Platform Console][developer-console] project with the Cloud Bigtable [API enabled][enable-api]. + +[Follow these instructions][create-project] to get your project set up. You will also need to set up the local development environment by +[installing the Google Cloud SDK][cloud-sdk] and running the following commands in command line: +`gcloud auth login` and `gcloud config set project [YOUR PROJECT ID]`. + +### Installation and setup + +You'll need to obtain the `google-cloud-bigtable` library. See the [Quickstart](#quickstart) section +to add `google-cloud-bigtable` as a dependency in your code. + +## About Cloud Bigtable + + +[Cloud Bigtable][product-docs] + +See the [Cloud Bigtable client library docs][javadocs] to learn how to +use this Cloud Bigtable Client Library. + ## About Cloud Bigtable -[Cloud Bigtable][cloud-bigtable] is Google's NoSQL Big Data database service. It's +[Cloud Bigtable][cloud-bigtable] is Google's NoSQL Big Data database service. It's the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail. @@ -60,10 +94,10 @@ interact with Cloud Bigtable using this Client Library. ## Concepts Cloud Bigtable is composed of instances, clusters, nodes and tables. - + ### Instances Instances are containers for clusters. - + ### Clusters Clusters represent the actual Cloud Bigtable service. Each cluster belongs to a single Cloud Bigtable instance, and an instance can have up to 4 clusters. When your application sends requests to a Cloud Bigtable instance, those requests are actually handled by one of the clusters in the instance. @@ -82,8 +116,8 @@ The Cloud Bigtable API consists of: Allows callers to persist and query data in a table. It's exposed by [BigtableDataClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/BigtableDataClient.html). ### Admin API -Allows callers to create and manage instances, clusters, tables, and access permissions. This API is exposed by: [BigtableInstanceAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableInstanceAdminClient.html) for Instance and Cluster level resources. - +Allows callers to create and manage instances, clusters, tables, and access permissions. This API is exposed by: [BigtableInstanceAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableInstanceAdminClient.html) for Instance and Cluster level resources. + See [BigtableTableAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableTableAdminClient.html) for table management. See [BigtableDataClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/BigtableDataClient.html) for the data client. @@ -243,7 +277,7 @@ metrics will be tagged with: in and different from the project where the metrics are exported to. * `bigtable_instance_id`: the instance id of the target Bigtable instance * `bigtable_app_profile_id`: the app profile id that is being used to access the target - Bigtable instance + Bigtable instance ### Available operation level metric views: @@ -371,14 +405,32 @@ spec: ``` see [Assign CPU Resources to Containers](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/#specify-a-cpu-request-and-a-cpu-limit) for more information. -## Troubleshooting -To get help, follow the instructions in the [shared Troubleshooting -document](https://github.com/googleapis/google-cloud-common/blob/master/troubleshooting/readme.md#troubleshooting). -Transport ---------- -Bigtable uses gRPC for the transport layer. + +## Samples + +Samples are in the [`samples/`](https://github.com/googleapis/java-bigtable/tree/master/samples) directory. The samples' `README.md` +has instructions for running the samples. + +| Sample | Source Code | Try it | +| --------------------------- | --------------------------------- | ------ | +| Filters | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/example/bigtable/Filters.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigtable/Filters.java) | +| Reads | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/example/bigtable/Reads.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigtable/Reads.java) | +| Write Batch | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/example/bigtable/WriteBatch.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigtable/WriteBatch.java) | +| Write Conditionally | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/example/bigtable/WriteConditionally.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigtable/WriteConditionally.java) | +| Write Increment | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/example/bigtable/WriteIncrement.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigtable/WriteIncrement.java) | +| Write Simple | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/example/bigtable/WriteSimple.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigtable/WriteSimple.java) | +| Hello World | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/m/examples/bigtable/HelloWorld.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/m/examples/bigtable/HelloWorld.java) | +| Instance Admin Example | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/m/examples/bigtable/InstanceAdminExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/m/examples/bigtable/InstanceAdminExample.java) | +| Quickstart | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/m/examples/bigtable/Quickstart.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/m/examples/bigtable/Quickstart.java) | +| Table Admin Example | [source code](https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/main/java/com/m/examples/bigtable/TableAdminExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigtable&page=editor&open_in_editor=samples/snippets/src/main/java/com/m/examples/bigtable/TableAdminExample.java) | + + + +## Troubleshooting + +To get help, follow the instructions in the [shared Troubleshooting document][troubleshooting]. ## Java Versions @@ -386,18 +438,24 @@ Java 7 or above is required for using this client. ## Versioning + This library follows [Semantic Versioning](http://semver.org/). + ## Contributing + Contributions to this library are always welcome and highly encouraged. -See [CONTRIBUTING] for more information on how to get started and [DEVELOPING] for a layout of the -codebase. +See [CONTRIBUTING][contributing] for more information how to get started. + +Please note that this project is released with a Contributor Code of Conduct. By participating in +this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more +information. ## License -Apache 2.0 - See [LICENSE] for more information. +Apache 2.0 - See [LICENSE][license] for more information. ## CI Status @@ -409,13 +467,10 @@ Java 8 OSX | [![Kokoro CI][kokoro-badge-image-3]][kokoro-badge-link-3] Java 8 Windows | [![Kokoro CI][kokoro-badge-image-4]][kokoro-badge-link-4] Java 11 | [![Kokoro CI][kokoro-badge-image-5]][kokoro-badge-link-5] -[CONTRIBUTING]:https://github.com/googleapis/google-cloud-java/blob/master/CONTRIBUTING.md -[DEVELOPING]:DEVELOPING.md -[LICENSE]: https://github.com/googleapis/google-cloud-java/blob/master/LICENSE -[cloud-platform]: https://cloud.google.com/ -[cloud-bigtable]: https://cloud.google.com/bigtable/ -[bigtable-product-docs]: https://cloud.google.com/bigtable/docs/ -[bigtable-client-lib-docs]: https://googleapis.dev/java/google-cloud-clients/latest/index.html?com/google/cloud/bigtable/package-summary.html +Java is a registered trademark of Oracle and/or its affiliates. + +[product-docs]: https://cloud.google.com/bigtable +[javadocs]: https://googleapis.dev/java/google-cloud-bigtable/latest/index.html [kokoro-badge-image-1]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigtable/java7.svg [kokoro-badge-link-1]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigtable/java7.html [kokoro-badge-image-2]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigtable/java8.svg @@ -426,3 +481,18 @@ Java 11 | [![Kokoro CI][kokoro-badge-image-5]][kokoro-badge-link-5] [kokoro-badge-link-4]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigtable/java8-win.html [kokoro-badge-image-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigtable/java11.svg [kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigtable/java11.html +[stability-image]: https://img.shields.io/badge/stability-ga-green +[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigtable.svg +[maven-version-link]: https://search.maven.org/search?q=g:com.google.cloud%20AND%20a:google-cloud-bigtable&core=gav +[authentication]: https://github.com/googleapis/google-cloud-java#authentication +[developer-console]: https://console.developers.google.com/ +[create-project]: https://cloud.google.com/resource-manager/docs/creating-managing-projects +[cloud-sdk]: https://cloud.google.com/sdk/ +[troubleshooting]: https://github.com/googleapis/google-cloud-common/blob/master/troubleshooting/readme.md#troubleshooting +[contributing]: https://github.com/googleapis/java-bigtable/blob/master/CONTRIBUTING.md +[code-of-conduct]: https://github.com/googleapis/java-bigtable/blob/master/CODE_OF_CONDUCT.md#contributor-code-of-conduct +[license]: https://github.com/googleapis/java-bigtable/blob/master/LICENSE + +[enable-api]: https://console.cloud.google.com/flows/enableapi?apiid=bigtable.googleapis.com +[libraries-bom]: https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/The-Google-Cloud-Platform-Libraries-BOM +[shell_img]: https://gstatic.com/cloudssh/images/open-btn.png diff --git a/synth.py b/synth.py index 799187531c..e12767cd8b 100644 --- a/synth.py +++ b/synth.py @@ -34,7 +34,6 @@ def main(): java.common_templates(excludes=[ '.gitignore', - 'README.md', '.kokoro/presubmit/integration.cfg', '.kokoro/nightly/integration.cfg', '.kokoro/presubmit/samples.cfg', From 958cf3b3a7df9655b19c540ad6d4351a4d42f8aa Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Tue, 15 Dec 2020 23:32:20 +0100 Subject: [PATCH 02/38] chore(deps): update dependency com.google.cloud:libraries-bom to v16.2.0 (#569) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [com.google.cloud:libraries-bom](https://togithub.com/GoogleCloudPlatform/cloud-opensource-java) | minor | `16.1.0` -> `16.2.0` | --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-bigtable). --- samples/snippets/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/samples/snippets/pom.xml b/samples/snippets/pom.xml index fd7b5d1503..045cef761b 100644 --- a/samples/snippets/pom.xml +++ b/samples/snippets/pom.xml @@ -30,7 +30,7 @@ com.google.cloud libraries-bom - 16.1.0 + 16.2.0 pom import From d83350ead0c5ed897505a603c9c5cd9b014f5545 Mon Sep 17 00:00:00 2001 From: "release-please[bot]" <55107282+release-please[bot]@users.noreply.github.com> Date: Tue, 15 Dec 2020 22:36:04 +0000 Subject: [PATCH 03/38] chore: release 1.19.3-SNAPSHOT (#570) :robot: I have created a release \*beep\* \*boop\* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). --- google-cloud-bigtable-bom/pom.xml | 14 +++++++------- google-cloud-bigtable-deps-bom/pom.xml | 2 +- google-cloud-bigtable-emulator/pom.xml | 8 ++++---- google-cloud-bigtable/pom.xml | 10 +++++----- grpc-google-cloud-bigtable-admin-v2/pom.xml | 8 ++++---- grpc-google-cloud-bigtable-v2/pom.xml | 8 ++++---- pom.xml | 2 +- proto-google-cloud-bigtable-admin-v2/pom.xml | 8 ++++---- proto-google-cloud-bigtable-v2/pom.xml | 8 ++++---- samples/snapshot/pom.xml | 2 +- versions.txt | 12 ++++++------ 11 files changed, 41 insertions(+), 41 deletions(-) diff --git a/google-cloud-bigtable-bom/pom.xml b/google-cloud-bigtable-bom/pom.xml index 9706f82c00..2777bda8f5 100644 --- a/google-cloud-bigtable-bom/pom.xml +++ b/google-cloud-bigtable-bom/pom.xml @@ -3,7 +3,7 @@ 4.0.0 com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom com.google.cloud @@ -72,32 +72,32 @@ com.google.cloud google-cloud-bigtable - 1.19.2 + 1.19.3-SNAPSHOT com.google.cloud google-cloud-bigtable-emulator - 0.128.2 + 0.128.3-SNAPSHOT com.google.api.grpc grpc-google-cloud-bigtable-admin-v2 - 1.19.2 + 1.19.3-SNAPSHOT com.google.api.grpc grpc-google-cloud-bigtable-v2 - 1.19.2 + 1.19.3-SNAPSHOT com.google.api.grpc proto-google-cloud-bigtable-admin-v2 - 1.19.2 + 1.19.3-SNAPSHOT com.google.api.grpc proto-google-cloud-bigtable-v2 - 1.19.2 + 1.19.3-SNAPSHOT diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index 99f051db52..d2e36fe8a7 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -12,7 +12,7 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom diff --git a/google-cloud-bigtable-emulator/pom.xml b/google-cloud-bigtable-emulator/pom.xml index 97a518e9e0..a90f63e942 100644 --- a/google-cloud-bigtable-emulator/pom.xml +++ b/google-cloud-bigtable-emulator/pom.xml @@ -5,7 +5,7 @@ 4.0.0 google-cloud-bigtable-emulator - 0.128.2 + 0.128.3-SNAPSHOT Google Cloud Java - Bigtable Emulator https://github.com/googleapis/java-bigtable @@ -14,7 +14,7 @@ com.google.cloud google-cloud-bigtable-parent - 1.19.2 + 1.19.3-SNAPSHOT scm:git:git@github.com:googleapis/java-bigtable.git @@ -80,14 +80,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import diff --git a/google-cloud-bigtable/pom.xml b/google-cloud-bigtable/pom.xml index 00940c6bee..59ea2b0a1e 100644 --- a/google-cloud-bigtable/pom.xml +++ b/google-cloud-bigtable/pom.xml @@ -2,7 +2,7 @@ 4.0.0 google-cloud-bigtable - 1.19.2 + 1.19.3-SNAPSHOT jar Google Cloud Bigtable https://github.com/googleapis/java-bigtable @@ -12,11 +12,11 @@ com.google.cloud google-cloud-bigtable-parent - 1.19.2 + 1.19.3-SNAPSHOT - 1.19.2 + 1.19.3-SNAPSHOT google-cloud-bigtable @@ -39,14 +39,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import diff --git a/grpc-google-cloud-bigtable-admin-v2/pom.xml b/grpc-google-cloud-bigtable-admin-v2/pom.xml index 3bf96ffda6..4ef1af15e7 100644 --- a/grpc-google-cloud-bigtable-admin-v2/pom.xml +++ b/grpc-google-cloud-bigtable-admin-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc grpc-google-cloud-bigtable-admin-v2 - 1.19.2 + 1.19.3-SNAPSHOT grpc-google-cloud-bigtable-admin-v2 GRPC library for grpc-google-cloud-bigtable-admin-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.2 + 1.19.3-SNAPSHOT @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import diff --git a/grpc-google-cloud-bigtable-v2/pom.xml b/grpc-google-cloud-bigtable-v2/pom.xml index 1fb3c130b9..a4f67619c0 100644 --- a/grpc-google-cloud-bigtable-v2/pom.xml +++ b/grpc-google-cloud-bigtable-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc grpc-google-cloud-bigtable-v2 - 1.19.2 + 1.19.3-SNAPSHOT grpc-google-cloud-bigtable-v2 GRPC library for grpc-google-cloud-bigtable-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.2 + 1.19.3-SNAPSHOT @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import diff --git a/pom.xml b/pom.xml index d4884896ec..c84b8af55c 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ google-cloud-bigtable-parent pom - 1.19.2 + 1.19.3-SNAPSHOT Google Cloud Bigtable Parent https://github.com/googleapis/java-bigtable diff --git a/proto-google-cloud-bigtable-admin-v2/pom.xml b/proto-google-cloud-bigtable-admin-v2/pom.xml index 96a8bd3f9a..097539eaf8 100644 --- a/proto-google-cloud-bigtable-admin-v2/pom.xml +++ b/proto-google-cloud-bigtable-admin-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc proto-google-cloud-bigtable-admin-v2 - 1.19.2 + 1.19.3-SNAPSHOT proto-google-cloud-bigtable-admin-v2 PROTO library for proto-google-cloud-bigtable-admin-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.2 + 1.19.3-SNAPSHOT @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import diff --git a/proto-google-cloud-bigtable-v2/pom.xml b/proto-google-cloud-bigtable-v2/pom.xml index c9c871ab78..b726220eb9 100644 --- a/proto-google-cloud-bigtable-v2/pom.xml +++ b/proto-google-cloud-bigtable-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc proto-google-cloud-bigtable-v2 - 1.19.2 + 1.19.3-SNAPSHOT proto-google-cloud-bigtable-v2 PROTO library for proto-google-cloud-bigtable-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.2 + 1.19.3-SNAPSHOT @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import com.google.cloud google-cloud-bigtable-bom - 1.19.2 + 1.19.3-SNAPSHOT pom import diff --git a/samples/snapshot/pom.xml b/samples/snapshot/pom.xml index c74086b84b..13af92c106 100644 --- a/samples/snapshot/pom.xml +++ b/samples/snapshot/pom.xml @@ -28,7 +28,7 @@ com.google.cloud google-cloud-bigtable - 1.19.2 + 1.19.3-SNAPSHOT diff --git a/versions.txt b/versions.txt index 3804ef0919..adb04530ee 100644 --- a/versions.txt +++ b/versions.txt @@ -1,9 +1,9 @@ # Format: # module:released-version:current-version -google-cloud-bigtable:1.19.2:1.19.2 -grpc-google-cloud-bigtable-admin-v2:1.19.2:1.19.2 -grpc-google-cloud-bigtable-v2:1.19.2:1.19.2 -proto-google-cloud-bigtable-admin-v2:1.19.2:1.19.2 -proto-google-cloud-bigtable-v2:1.19.2:1.19.2 -google-cloud-bigtable-emulator:0.128.2:0.128.2 +google-cloud-bigtable:1.19.2:1.19.3-SNAPSHOT +grpc-google-cloud-bigtable-admin-v2:1.19.2:1.19.3-SNAPSHOT +grpc-google-cloud-bigtable-v2:1.19.2:1.19.3-SNAPSHOT +proto-google-cloud-bigtable-admin-v2:1.19.2:1.19.3-SNAPSHOT +proto-google-cloud-bigtable-v2:1.19.2:1.19.3-SNAPSHOT +google-cloud-bigtable-emulator:0.128.2:0.128.3-SNAPSHOT From 27304431b86d38b6292651cf3ee6b463b13050fd Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Tue, 15 Dec 2020 15:28:07 -0800 Subject: [PATCH 04/38] chore: regenerate README (#572) This PR was generated using Autosynth. :rainbow:
Log from Synthtool ``` 2020-12-15 22:39:13,531 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2020-12-15 22:39:14,445 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
Full log will be available here: https://source.cloud.google.com/results/invocations/3f9ae92e-7ec9-4da2-95e0-49a01f5d0c1e/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 18 ++++++++++++++++++ README.md | 8 ++++---- 2 files changed, 22 insertions(+), 4 deletions(-) create mode 100644 .github/readme/synth.metadata/synth.metadata diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata new file mode 100644 index 0000000000..29915e03f7 --- /dev/null +++ b/.github/readme/synth.metadata/synth.metadata @@ -0,0 +1,18 @@ +{ + "sources": [ + { + "git": { + "name": ".", + "remote": "https://github.com/googleapis/java-bigtable.git", + "sha": "d83350ead0c5ed897505a603c9c5cd9b014f5545" + } + }, + { + "git": { + "name": "synthtool", + "remote": "https://github.com/googleapis/synthtool.git", + "sha": "996775eca5fd934edac3c2ae34b80ff0395b1717" + } + } + ] +} \ No newline at end of file diff --git a/README.md b/README.md index b4178932a4..a1c443cdc7 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ If you are using Maven with [BOM][libraries-bom], add this to your pom.xml file com.google.cloud libraries-bom - 16.1.0 + 16.2.0 pom import @@ -38,18 +38,18 @@ If you are using Maven without BOM, add this to your dependencies: com.google.cloud google-cloud-bigtable - 1.19.2 + 1.19.1 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.cloud:google-cloud-bigtable:1.19.2' +compile 'com.google.cloud:google-cloud-bigtable:1.19.1' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.19.2" +libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.19.1" ``` ## Authentication From 03fa1caedea753cf7e75a7219c25e5a4a6d72eb1 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Wed, 16 Dec 2020 08:44:08 -0800 Subject: [PATCH 05/38] ci(java): ignore bot users for generate-files-bot (#574) This PR was generated using Autosynth. :rainbow: Synth log will be available here: https://source.cloud.google.com/results/invocations/25c8457b-bca6-4764-82ce-8c920634111f/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: https://github.com/googleapis/synthtool/commit/3f67ceece7e797a5736a25488aae35405649b90b --- .github/generated-files-bot.yml | 4 ++++ synth.metadata | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/generated-files-bot.yml b/.github/generated-files-bot.yml index 20f3acc281..47c2ba132e 100644 --- a/.github/generated-files-bot.yml +++ b/.github/generated-files-bot.yml @@ -5,3 +5,7 @@ externalManifests: - type: json file: '.github/readme/synth.metadata/synth.metadata' jsonpath: '$.generatedFiles[*]' +ignoreAuthors: +- 'renovate-bot' +- 'yoshi-automation' +- 'release-please[bot]' diff --git a/synth.metadata b/synth.metadata index 13c1bc93c4..61bebbe9cf 100644 --- a/synth.metadata +++ b/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "2dea8ab1f56e92d42934fed64972a2898209330d" + "sha": "27304431b86d38b6292651cf3ee6b463b13050fd" } }, { @@ -19,7 +19,7 @@ "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "5d11bd2888c38ce1fb6fa6bf25494a4219a73928" + "sha": "3f67ceece7e797a5736a25488aae35405649b90b" } } ], From 091a8aca1f6669456168140c7e630b61e50fe32c Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Wed, 16 Dec 2020 09:22:02 -0800 Subject: [PATCH 06/38] chore: regenerate README (#576) This PR was generated using Autosynth. :rainbow:
Log from Synthtool ``` 2020-12-16 16:49:12,307 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2020-12-16 16:49:13,142 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
Full log will be available here: https://source.cloud.google.com/results/invocations/4a856a39-99d0-4d26-9382-e4c162c6cb16/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 4 ++-- README.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index 29915e03f7..5042ef665d 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,14 +4,14 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "d83350ead0c5ed897505a603c9c5cd9b014f5545" + "sha": "03fa1caedea753cf7e75a7219c25e5a4a6d72eb1" } }, { "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "996775eca5fd934edac3c2ae34b80ff0395b1717" + "sha": "3f67ceece7e797a5736a25488aae35405649b90b" } } ] diff --git a/README.md b/README.md index a1c443cdc7..efc0a69a8a 100644 --- a/README.md +++ b/README.md @@ -45,11 +45,11 @@ If you are using Maven without BOM, add this to your dependencies: If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.cloud:google-cloud-bigtable:1.19.1' +compile 'com.google.cloud:google-cloud-bigtable:1.19.2' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.19.1" +libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.19.2" ``` ## Authentication From fa8f5ef216fcbcd9f57343475f03a7353b0e9ddd Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Wed, 16 Dec 2020 18:41:13 +0100 Subject: [PATCH 07/38] chore(deps): update dependency com.google.cloud:google-cloud-bigtable to v1.19.2 (#575) --- samples/install-without-bom/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/samples/install-without-bom/pom.xml b/samples/install-without-bom/pom.xml index 02507562c9..96ef9380ac 100644 --- a/samples/install-without-bom/pom.xml +++ b/samples/install-without-bom/pom.xml @@ -29,7 +29,7 @@ com.google.cloud google-cloud-bigtable - 1.19.1 + 1.19.2 From 3f2892349d902b7defea98003e91a92fd1f4c037 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Wed, 16 Dec 2020 09:58:03 -0800 Subject: [PATCH 08/38] chore: regenerate README (#577) This PR was generated using Autosynth. :rainbow:
Log from Synthtool ``` 2020-12-16 17:43:35,394 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2020-12-16 17:43:36,275 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
Full log will be available here: https://source.cloud.google.com/results/invocations/fe91459f-12e6-4474-aa60-54eb8a65e66e/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 4 ++-- README.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index 5042ef665d..adbe5a4a5d 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,14 +4,14 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "03fa1caedea753cf7e75a7219c25e5a4a6d72eb1" + "sha": "fa8f5ef216fcbcd9f57343475f03a7353b0e9ddd" } }, { "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "3f67ceece7e797a5736a25488aae35405649b90b" + "sha": "d73e8dea90af1b463a7f9d4a33244cf4612ada7c" } } ] diff --git a/README.md b/README.md index efc0a69a8a..e9bb4b0f59 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ If you are using Maven without BOM, add this to your dependencies: com.google.cloud google-cloud-bigtable - 1.19.1 + 1.19.2 ``` From 8240779434a602dc8b2bf90dbe539c5d7470d850 Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Wed, 16 Dec 2020 19:49:58 -0500 Subject: [PATCH 09/38] feat: Surface the server-timing metric (#535) * Extract server-timing trailer and create metrics for gfe latency * Add more tests and refactor * Refactor comments and imports * reformatting * Clean up comments * Refactor, use GrpcMetadataResponse to get the trailer * Fix based on the code review * clean up HeaderTracerResponseObserver * Add more tests for all the ops * Improve documents, changes for directPath and more tests * Small fixes in the doc * small clean up --- README.md | 14 +- .../data/v2/BigtableDataSettings.java | 14 + .../data/v2/stub/EnhancedBigtableStub.java | 98 ++-- .../v2/stub/EnhancedBigtableStubSettings.java | 23 + .../data/v2/stub/metrics/HeaderTracer.java | 123 +++++ .../HeaderTracerStreamingCallable.java | 125 +++++ .../metrics/HeaderTracerUnaryCallable.java | 83 ++++ .../v2/stub/metrics/RpcMeasureConstants.java | 14 + .../v2/stub/metrics/RpcViewConstants.java | 22 + .../data/v2/stub/metrics/RpcViews.java | 34 ++ .../EnhancedBigtableStubSettingsTest.java | 36 +- .../metrics/HeaderTracerCallableTest.java | 428 ++++++++++++++++++ .../v2/stub/metrics/HeaderTracerTest.java | 77 ++++ .../v2/stub/metrics/MetricsTracerTest.java | 121 ++--- .../data/v2/stub/metrics/StatsTestUtils.java | 79 ++++ 15 files changed, 1161 insertions(+), 130 deletions(-) create mode 100644 google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracer.java create mode 100644 google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerStreamingCallable.java create mode 100644 google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerUnaryCallable.java create mode 100644 google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerCallableTest.java create mode 100644 google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerTest.java diff --git a/README.md b/README.md index e9bb4b0f59..bf339adc4a 100644 --- a/README.md +++ b/README.md @@ -296,12 +296,22 @@ metrics will be tagged with: each client RPC, tagged by operation name and the attempt status. Under normal circumstances, this will be identical to op_latency. However, when the client receives transient errors, op_latency will be the sum of all attempt_latencies - and the exponential delays + and the exponential delays. * `cloud.google.com/java/bigtable/attempts_per_op`: A distribution of attempts that each operation required, tagged by operation name and final operation status. Under normal circumstances, this will be 1. +### GFE metric views: + +* `cloud.google.com/java/bigtable/gfe_latency`: A distribution of the latency + between Google's network receives an RPC and reads back the first byte of + the response. + +* `cloud.google.com/java/bigtable/gfe_header_missing_count`: A counter of the + number of RPC responses received without the server-timing header, which + indicates that the request probably never reached Google's network. + By default, the functionality is disabled. For example to enable metrics using [Google Stackdriver](https://cloud.google.com/monitoring/docs/): @@ -357,6 +367,8 @@ StackdriverStatsExporter.createAndRegister( ); BigtableDataSettings.enableOpenCensusStats(); +// Enable GFE metric views +BigtableDataSettings.enableGfeOpenCensusStats(); ``` ## Version Conflicts diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/BigtableDataSettings.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/BigtableDataSettings.java index f4d63b5ec0..3002aa6113 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/BigtableDataSettings.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/BigtableDataSettings.java @@ -175,6 +175,20 @@ public static void enableOpenCensusStats() { // io.opencensus.contrib.grpc.metrics.RpcViews.registerClientGrpcBasicViews(); } + /** + * Enables OpenCensus GFE metric aggregations. + * + *

This will register views for gfe_latency and gfe_header_missing_count metrics. + * + *

gfe_latency measures the latency between Google's network receives an RPC and reads back the + * first byte of the response. gfe_header_missing_count is a counter of the number of RPC + * responses received without the server-timing header. + */ + @BetaApi("OpenCensus stats integration is currently unstable and may change in the future") + public static void enableGfeOpenCensusStats() { + com.google.cloud.bigtable.data.v2.stub.metrics.RpcViews.registerBigtableClientGfeViews(); + } + /** Returns the target project id. */ public String getProjectId() { return stubSettings.getProjectId(); diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java index d729d6244d..448390396f 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java @@ -66,6 +66,8 @@ import com.google.cloud.bigtable.data.v2.models.RowMutation; import com.google.cloud.bigtable.data.v2.models.RowMutationEntry; import com.google.cloud.bigtable.data.v2.stub.metrics.CompositeTracerFactory; +import com.google.cloud.bigtable.data.v2.stub.metrics.HeaderTracerStreamingCallable; +import com.google.cloud.bigtable.data.v2.stub.metrics.HeaderTracerUnaryCallable; import com.google.cloud.bigtable.data.v2.stub.metrics.MetricsTracerFactory; import com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants; import com.google.cloud.bigtable.data.v2.stub.mutaterows.BulkMutateRowsUserFacingCallable; @@ -162,6 +164,15 @@ public static EnhancedBigtableStubSettings finalizeSettings( .build()); } + ImmutableMap attributes = + ImmutableMap.builder() + .put(RpcMeasureConstants.BIGTABLE_PROJECT_ID, TagValue.create(settings.getProjectId())) + .put( + RpcMeasureConstants.BIGTABLE_INSTANCE_ID, TagValue.create(settings.getInstanceId())) + .put( + RpcMeasureConstants.BIGTABLE_APP_PROFILE_ID, + TagValue.create(settings.getAppProfileId())) + .build(); // Inject Opencensus instrumentation builder.setTracerFactory( new CompositeTracerFactory( @@ -187,23 +198,17 @@ public static EnhancedBigtableStubSettings finalizeSettings( GaxProperties.getLibraryVersion(EnhancedBigtableStubSettings.class)) .build()), // Add OpenCensus Metrics - MetricsTracerFactory.create( - tagger, - stats, - ImmutableMap.builder() - .put( - RpcMeasureConstants.BIGTABLE_PROJECT_ID, - TagValue.create(settings.getProjectId())) - .put( - RpcMeasureConstants.BIGTABLE_INSTANCE_ID, - TagValue.create(settings.getInstanceId())) - .put( - RpcMeasureConstants.BIGTABLE_APP_PROFILE_ID, - TagValue.create(settings.getAppProfileId())) - .build()), + MetricsTracerFactory.create(tagger, stats, attributes), // Add user configured tracer settings.getTracerFactory()))); - + builder.setHeaderTracer( + builder + .getHeaderTracer() + .toBuilder() + .setStats(stats) + .setTagger(tagger) + .setStatsAttributes(attributes) + .build()); return builder.build(); } @@ -268,11 +273,10 @@ public ServerStreamingCallable createReadRowsCallable( ServerStreamingCallable readRowsUserCallable = new ReadRowsUserCallable<>(readRowsCallable, requestContext); + SpanName span = getSpanName("ReadRows"); ServerStreamingCallable traced = new TracedServerStreamingCallable<>( - readRowsUserCallable, - clientContext.getTracerFactory(), - SpanName.of(CLIENT_NAME, "ReadRows")); + readRowsUserCallable, clientContext.getTracerFactory(), span); return traced.withDefaultCallContext(clientContext.getDefaultCallContext()); } @@ -315,6 +319,7 @@ public UnaryCallable createReadRowCallable(RowAdapter *

  • Upon receiving the response stream, it will merge the {@link * com.google.bigtable.v2.ReadRowsResponse.CellChunk}s in logical rows. The actual row * implementation can be configured by the {@code rowAdapter} parameter. + *
  • Add header tracer for tracking GFE metrics. *
  • Retry/resume on failure. *
  • Filter out marker rows. * @@ -356,10 +361,14 @@ public Map extract(ReadRowsRequest readRowsRequest) { ServerStreamingCallable watched = Callables.watched(merging, innerSettings, clientContext); + ServerStreamingCallable withHeaderTracer = + new HeaderTracerStreamingCallable<>( + watched, settings.getHeaderTracer(), getSpanName("ReadRows").toString()); + // Retry logic is split into 2 parts to workaround a rare edge case described in // ReadRowsRetryCompletedCallable ServerStreamingCallable retrying1 = - new ReadRowsRetryCompletedCallable<>(watched); + new ReadRowsRetryCompletedCallable<>(withHeaderTracer); ServerStreamingCallable retrying2 = Callables.retrying(retrying1, innerSettings, clientContext); @@ -380,6 +389,8 @@ public Map extract(ReadRowsRequest readRowsRequest) { * */ private UnaryCallable> createSampleRowKeysCallable() { + String methodName = "SampleRowKeys"; + ServerStreamingCallable base = GrpcRawCallableFactory.createServerStreamingCallable( GrpcCallSettings.newBuilder() @@ -399,11 +410,15 @@ public Map extract( UnaryCallable> spoolable = base.all(); + UnaryCallable> withHeaderTracer = + new HeaderTracerUnaryCallable<>( + spoolable, settings.getHeaderTracer(), getSpanName(methodName).toString()); + UnaryCallable> retryable = - Callables.retrying(spoolable, settings.sampleRowKeysSettings(), clientContext); + Callables.retrying(withHeaderTracer, settings.sampleRowKeysSettings(), clientContext); return createUserFacingUnaryCallable( - "SampleRowKeys", new SampleRowKeysCallable(retryable, requestContext)); + methodName, new SampleRowKeysCallable(retryable, requestContext)); } /** @@ -415,6 +430,7 @@ public Map extract( * */ private UnaryCallable createMutateRowCallable() { + String methodName = "MutateRow"; UnaryCallable base = GrpcRawCallableFactory.createUnaryCallable( GrpcCallSettings.newBuilder() @@ -431,11 +447,15 @@ public Map extract(MutateRowRequest mutateRowRequest) { .build(), settings.mutateRowSettings().getRetryableCodes()); + UnaryCallable withHeaderTracer = + new HeaderTracerUnaryCallable<>( + base, settings.getHeaderTracer(), getSpanName(methodName).toString()); + UnaryCallable retrying = - Callables.retrying(base, settings.mutateRowSettings(), clientContext); + Callables.retrying(withHeaderTracer, settings.mutateRowSettings(), clientContext); return createUserFacingUnaryCallable( - "MutateRow", new MutateRowCallable(retrying, requestContext)); + methodName, new MutateRowCallable(retrying, requestContext)); } /** @@ -459,11 +479,13 @@ private UnaryCallable createBulkMutateRowsCallable() { UnaryCallable userFacing = new BulkMutateRowsUserFacingCallable(baseCallable, requestContext); + SpanName spanName = getSpanName("MutateRows"); UnaryCallable traced = - new TracedUnaryCallable<>( - userFacing, clientContext.getTracerFactory(), SpanName.of(CLIENT_NAME, "MutateRows")); + new TracedUnaryCallable<>(userFacing, clientContext.getTracerFactory(), spanName); + UnaryCallable withHeaderTracer = + new HeaderTracerUnaryCallable<>(traced, settings.getHeaderTracer(), spanName.toString()); - return traced.withDefaultCallContext(clientContext.getDefaultCallContext()); + return withHeaderTracer.withDefaultCallContext(clientContext.getDefaultCallContext()); } /** @@ -569,6 +591,7 @@ public Map extract(MutateRowsRequest mutateRowsRequest) { * */ private UnaryCallable createCheckAndMutateRowCallable() { + String methodName = "CheckAndMutateRow"; UnaryCallable base = GrpcRawCallableFactory.createUnaryCallable( GrpcCallSettings.newBuilder() @@ -586,11 +609,15 @@ public Map extract( .build(), settings.checkAndMutateRowSettings().getRetryableCodes()); + UnaryCallable withHeaderTracer = + new HeaderTracerUnaryCallable<>( + base, settings.getHeaderTracer(), getSpanName(methodName).toString()); + UnaryCallable retrying = - Callables.retrying(base, settings.checkAndMutateRowSettings(), clientContext); + Callables.retrying(withHeaderTracer, settings.checkAndMutateRowSettings(), clientContext); return createUserFacingUnaryCallable( - "CheckAndMutateRow", new CheckAndMutateRowCallable(retrying, requestContext)); + methodName, new CheckAndMutateRowCallable(retrying, requestContext)); } /** @@ -619,12 +646,16 @@ public Map extract(ReadModifyWriteRowRequest request) { }) .build(), settings.readModifyWriteRowSettings().getRetryableCodes()); + String methodName = "ReadModifyWriteRow"; + UnaryCallable withHeaderTracer = + new HeaderTracerUnaryCallable<>( + base, settings.getHeaderTracer(), getSpanName(methodName).toString()); UnaryCallable retrying = - Callables.retrying(base, settings.readModifyWriteRowSettings(), clientContext); + Callables.retrying(withHeaderTracer, settings.readModifyWriteRowSettings(), clientContext); return createUserFacingUnaryCallable( - "ReadModifyWriteRow", new ReadModifyWriteRowCallable(retrying, requestContext)); + methodName, new ReadModifyWriteRowCallable(retrying, requestContext)); } /** @@ -635,8 +666,7 @@ private UnaryCallable createUserFacin String methodName, UnaryCallable inner) { UnaryCallable traced = - new TracedUnaryCallable<>( - inner, clientContext.getTracerFactory(), SpanName.of(CLIENT_NAME, methodName)); + new TracedUnaryCallable<>(inner, clientContext.getTracerFactory(), getSpanName(methodName)); return traced.withDefaultCallContext(clientContext.getDefaultCallContext()); } @@ -686,6 +716,10 @@ public UnaryCallable readModifyWriteRowCallable() { } // + private SpanName getSpanName(String methodName) { + return SpanName.of(CLIENT_NAME, methodName); + } + @Override public void close() { for (BackgroundResource backgroundResource : clientContext.getBackgroundResources()) { diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettings.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettings.java index 72b22c35e2..eaea47f4ef 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettings.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettings.java @@ -38,6 +38,7 @@ import com.google.cloud.bigtable.data.v2.models.ReadModifyWriteRow; import com.google.cloud.bigtable.data.v2.models.Row; import com.google.cloud.bigtable.data.v2.models.RowMutation; +import com.google.cloud.bigtable.data.v2.stub.metrics.HeaderTracer; import com.google.cloud.bigtable.data.v2.stub.mutaterows.MutateRowsBatchingDescriptor; import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsBatchingDescriptor; import com.google.common.base.MoreObjects; @@ -154,6 +155,7 @@ public class EnhancedBigtableStubSettings extends StubSettings primedTableIds; + private HeaderTracer headerTracer; private final ServerStreamingCallSettings readRowsSettings; private final UnaryCallSettings readRowSettings; @@ -187,6 +189,7 @@ private EnhancedBigtableStubSettings(Builder builder) { appProfileId = builder.appProfileId; isRefreshingChannel = builder.isRefreshingChannel; primedTableIds = builder.primedTableIds; + headerTracer = builder.headerTracer; // Per method settings. readRowsSettings = builder.readRowsSettings.build(); @@ -231,6 +234,11 @@ public List getPrimedTableIds() { return primedTableIds; } + /** Gets the tracer for capturing metrics in the header. */ + HeaderTracer getHeaderTracer() { + return headerTracer; + } + /** Returns a builder for the default ChannelProvider for this service. */ public static InstantiatingGrpcChannelProvider.Builder defaultGrpcTransportProviderBuilder() { return BigtableStubSettings.defaultGrpcTransportProviderBuilder() @@ -488,6 +496,7 @@ public static class Builder extends StubSettings.Builder primedTableIds; + private HeaderTracer headerTracer; private final ServerStreamingCallSettings.Builder readRowsSettings; private final UnaryCallSettings.Builder readRowSettings; @@ -511,6 +520,7 @@ private Builder() { this.appProfileId = SERVER_DEFAULT_APP_PROFILE_ID; this.isRefreshingChannel = false; primedTableIds = ImmutableList.of(); + headerTracer = HeaderTracer.newBuilder().build(); setCredentialsProvider(defaultCredentialsProviderBuilder().build()); // Defaults provider @@ -617,6 +627,7 @@ private Builder(EnhancedBigtableStubSettings settings) { appProfileId = settings.appProfileId; isRefreshingChannel = settings.isRefreshingChannel; primedTableIds = settings.primedTableIds; + headerTracer = settings.headerTracer; // Per method settings. readRowsSettings = settings.readRowsSettings.toBuilder(); @@ -739,6 +750,17 @@ public List getPrimedTableIds() { return primedTableIds; } + /** Configure the header tracer for surfacing metrics in the header. */ + Builder setHeaderTracer(HeaderTracer headerTracer) { + this.headerTracer = headerTracer; + return this; + } + + /** Gets the header tracer that'll be used to surface metrics in the header. */ + HeaderTracer getHeaderTracer() { + return headerTracer; + } + /** Returns the builder for the settings used for calls to readRows. */ public ServerStreamingCallSettings.Builder readRowsSettings() { return readRowsSettings; @@ -818,6 +840,7 @@ public String toString() { .add("appProfileId", appProfileId) .add("isRefreshingChannel", isRefreshingChannel) .add("primedTableIds", primedTableIds) + .add("headerTracer", headerTracer) .add("readRowsSettings", readRowsSettings) .add("readRowSettings", readRowSettings) .add("sampleRowKeysSettings", sampleRowKeysSettings) diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracer.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracer.java new file mode 100644 index 0000000000..f3eb0ef1e2 --- /dev/null +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracer.java @@ -0,0 +1,123 @@ +/* + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigtable.data.v2.stub.metrics; + +import com.google.api.core.InternalApi; +import com.google.auto.value.AutoValue; +import com.google.common.base.MoreObjects; +import io.grpc.Metadata; +import io.opencensus.stats.MeasureMap; +import io.opencensus.stats.Stats; +import io.opencensus.stats.StatsRecorder; +import io.opencensus.tags.TagContextBuilder; +import io.opencensus.tags.TagKey; +import io.opencensus.tags.TagValue; +import io.opencensus.tags.Tagger; +import io.opencensus.tags.Tags; +import java.util.Collections; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import javax.annotation.Nonnull; + +@InternalApi +@AutoValue +public abstract class HeaderTracer { + + private static final Metadata.Key SERVER_TIMING_HEADER_KEY = + Metadata.Key.of("server-timing", Metadata.ASCII_STRING_MARSHALLER); + private static final Pattern SERVER_TIMING_HEADER_PATTERN = Pattern.compile(".*dur=(?\\d+)"); + + @AutoValue.Builder + public abstract static class Builder { + // + public abstract Builder setTagger(@Nonnull Tagger tagger); + + public abstract Builder setStats(@Nonnull StatsRecorder stats); + + public abstract Builder setStatsAttributes(@Nonnull Map statsAttributes); + + abstract HeaderTracer autoBuild(); + + public HeaderTracer build() { + HeaderTracer headerTracer = autoBuild(); + return headerTracer; + } + // + } + + public abstract Tagger getTagger(); + + public abstract StatsRecorder getStats(); + + public abstract Map getStatsAttributes(); + + /** + * If the header has a server-timing field, extract the metric and publish it to OpenCensus. + * Otherwise increment the gfe header missing counter by 1. + */ + public void recordGfeMetadata(@Nonnull Metadata metadata, @Nonnull String spanName) { + MeasureMap measures = getStats().newMeasureMap(); + if (metadata.get(SERVER_TIMING_HEADER_KEY) != null) { + String serverTiming = metadata.get(SERVER_TIMING_HEADER_KEY); + Matcher matcher = SERVER_TIMING_HEADER_PATTERN.matcher(serverTiming); + measures.put(RpcMeasureConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT, 0L); + if (matcher.find()) { + long latency = Long.valueOf(matcher.group("dur")); + measures.put(RpcMeasureConstants.BIGTABLE_GFE_LATENCY, latency); + } + } else { + measures.put(RpcMeasureConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT, 1L); + } + measures.record(newTagCtxBuilder(spanName).build()); + } + + public void recordGfeMissingHeader(@Nonnull String spanName) { + MeasureMap measures = + getStats().newMeasureMap().put(RpcMeasureConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT, 1L); + measures.record(newTagCtxBuilder(spanName).build()); + } + + private TagContextBuilder newTagCtxBuilder(String span) { + TagContextBuilder tagContextBuilder = getTagger().currentBuilder(); + if (span != null) { + tagContextBuilder.putLocal(RpcMeasureConstants.BIGTABLE_OP, TagValue.create(span)); + } + // Copy client level tags in + for (Map.Entry entry : getStatsAttributes().entrySet()) { + tagContextBuilder.putLocal(entry.getKey(), entry.getValue()); + } + return tagContextBuilder; + } + + public static Builder newBuilder() { + return new AutoValue_HeaderTracer.Builder() + .setTagger(Tags.getTagger()) + .setStats(Stats.getStatsRecorder()) + .setStatsAttributes(Collections.emptyMap()); + } + + public abstract Builder toBuilder(); + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("stats", getStats()) + .add("tagger", getTagger()) + .add("statsAttributes", getStatsAttributes()) + .toString(); + } +} diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerStreamingCallable.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerStreamingCallable.java new file mode 100644 index 0000000000..fdca9297aa --- /dev/null +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerStreamingCallable.java @@ -0,0 +1,125 @@ +/* + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigtable.data.v2.stub.metrics; + +import com.google.api.core.InternalApi; +import com.google.api.gax.grpc.GrpcResponseMetadata; +import com.google.api.gax.rpc.ApiCallContext; +import com.google.api.gax.rpc.ResponseObserver; +import com.google.api.gax.rpc.ServerStreamingCallable; +import com.google.api.gax.rpc.StreamController; +import com.google.common.base.Preconditions; +import io.grpc.Metadata; +import javax.annotation.Nonnull; + +/** + * This callable will inject a {@link GrpcResponseMetadata} to access the headers and trailers + * returned by gRPC methods upon completion. The {@link HeaderTracer} will process metrics that were + * injected in the header/trailer and publish them to OpenCensus. If {@link + * GrpcResponseMetadata#getMetadata()} returned null, it probably means that the request has never + * reached GFE, and it'll increment the gfe_header_missing_counter in this case. + * + *

    If GFE metrics are not registered in {@link RpcViews}, skip injecting GrpcResponseMetadata. + * This is for the case where direct path is enabled, all the requests won't go through GFE and + * therefore won't have the server-timing header. + * + *

    This class is considered an internal implementation detail and not meant to be used by + * applications. + */ +@InternalApi +public class HeaderTracerStreamingCallable + extends ServerStreamingCallable { + + private final ServerStreamingCallable innerCallable; + private final HeaderTracer headerTracer; + private final String spanName; + + public HeaderTracerStreamingCallable( + @Nonnull ServerStreamingCallable callable, + @Nonnull HeaderTracer headerTracer, + @Nonnull String spanName) { + this.innerCallable = Preconditions.checkNotNull(callable, "Inner callable must be set"); + this.headerTracer = Preconditions.checkNotNull(headerTracer, "HeaderTracer must be set"); + this.spanName = Preconditions.checkNotNull(spanName, "Span name must be set"); + } + + @Override + public void call( + RequestT request, ResponseObserver responseObserver, ApiCallContext context) { + final GrpcResponseMetadata responseMetadata = new GrpcResponseMetadata(); + if (RpcViews.isGfeMetricsRegistered()) { + HeaderTracerResponseObserver innerObserver = + new HeaderTracerResponseObserver<>( + responseObserver, headerTracer, responseMetadata, spanName); + innerCallable.call(request, innerObserver, responseMetadata.addHandlers(context)); + } else { + innerCallable.call(request, responseObserver, context); + } + } + + private class HeaderTracerResponseObserver implements ResponseObserver { + + private ResponseObserver outerObserver; + private HeaderTracer headerTracer; + private GrpcResponseMetadata responseMetadata; + private String spanName; + + HeaderTracerResponseObserver( + ResponseObserver observer, + HeaderTracer headerTracer, + GrpcResponseMetadata metadata, + String spanName) { + this.outerObserver = observer; + this.headerTracer = headerTracer; + this.responseMetadata = metadata; + this.spanName = spanName; + } + + @Override + public void onStart(final StreamController controller) { + outerObserver.onStart(controller); + } + + @Override + public void onResponse(ResponseT response) { + outerObserver.onResponse(response); + } + + @Override + public void onError(Throwable t) { + // server-timing metric will be added through GrpcResponseMetadata#onHeaders(Metadata), + // so it's not checking trailing metadata here. + Metadata metadata = responseMetadata.getMetadata(); + if (metadata != null) { + headerTracer.recordGfeMetadata(metadata, spanName); + } else { + headerTracer.recordGfeMissingHeader(spanName); + } + outerObserver.onError(t); + } + + @Override + public void onComplete() { + Metadata metadata = responseMetadata.getMetadata(); + if (metadata != null) { + headerTracer.recordGfeMetadata(metadata, spanName); + } else { + headerTracer.recordGfeMissingHeader(spanName); + } + outerObserver.onComplete(); + } + } +} diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerUnaryCallable.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerUnaryCallable.java new file mode 100644 index 0000000000..17d84b2a2e --- /dev/null +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerUnaryCallable.java @@ -0,0 +1,83 @@ +/* + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigtable.data.v2.stub.metrics; + +import com.google.api.core.ApiFuture; +import com.google.api.core.InternalApi; +import com.google.api.gax.grpc.GrpcResponseMetadata; +import com.google.api.gax.rpc.ApiCallContext; +import com.google.api.gax.rpc.UnaryCallable; +import com.google.common.base.Preconditions; +import com.google.common.util.concurrent.MoreExecutors; +import io.grpc.Metadata; +import javax.annotation.Nonnull; + +/** + * This callable will inject a {@link GrpcResponseMetadata} to access the headers and trailers + * returned by gRPC methods upon completion. The {@link HeaderTracer} will process metrics that were + * injected in the header/trailer and publish them to OpenCensus. If {@link + * GrpcResponseMetadata#getMetadata()} returned null, it probably means that the request has never + * reached GFE, and it'll increment the gfe_header_missing_counter in this case. + * + *

    If GFE metrics are not registered in {@link RpcViews}, skip injecting GrpcResponseMetadata. + * This is for the case where direct path is enabled, all the requests won't go through GFE and + * therefore won't have the server-timing header. + * + *

    This class is considered an internal implementation detail and not meant to be used by + * applications. + */ +@InternalApi +public class HeaderTracerUnaryCallable + extends UnaryCallable { + + private final UnaryCallable innerCallable; + private final HeaderTracer headerTracer; + private final String spanName; + + public HeaderTracerUnaryCallable( + @Nonnull UnaryCallable innerCallable, + @Nonnull HeaderTracer headerTracer, + @Nonnull String spanName) { + this.innerCallable = Preconditions.checkNotNull(innerCallable, "Inner callable must be set"); + this.headerTracer = Preconditions.checkNotNull(headerTracer, "HeaderTracer must be set"); + this.spanName = Preconditions.checkNotNull(spanName, "Span name must be set"); + } + + @Override + public ApiFuture futureCall(RequestT request, ApiCallContext context) { + if (RpcViews.isGfeMetricsRegistered()) { + final GrpcResponseMetadata responseMetadata = new GrpcResponseMetadata(); + ApiFuture future = + innerCallable.futureCall(request, responseMetadata.addHandlers(context)); + future.addListener( + new Runnable() { + @Override + public void run() { + Metadata metadata = responseMetadata.getMetadata(); + if (metadata != null) { + headerTracer.recordGfeMetadata(metadata, spanName); + } else { + headerTracer.recordGfeMissingHeader(spanName); + } + } + }, + MoreExecutors.directExecutor()); + return future; + } else { + return innerCallable.futureCall(request, context); + } + } +} diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcMeasureConstants.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcMeasureConstants.java index 8c6e347a0f..e6e5c70db1 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcMeasureConstants.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcMeasureConstants.java @@ -74,4 +74,18 @@ public class RpcMeasureConstants { "cloud.google.com/java/bigtable/read_rows_first_row_latency", "Time between request being sent to the first row received", MILLISECOND); + + /** GFE t4t7 latency extracted from server-timing header. */ + public static final MeasureLong BIGTABLE_GFE_LATENCY = + MeasureLong.create( + "cloud.google.com/java/bigtable/gfe_latency", + "Latency between Google's network receives an RPC and reads back the first byte of the response", + MILLISECOND); + + /** Number of responses without the server-timing header. */ + public static final MeasureLong BIGTABLE_GFE_HEADER_MISSING_COUNT = + MeasureLong.create( + "cloud.google.com/java/bigtable/gfe_header_missing_count", + "Number of RPC responses received without the server-timing header, most likely means that the RPC never reached Google's network", + COUNT); } diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViewConstants.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViewConstants.java index d21060c4ac..8a14c01b13 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViewConstants.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViewConstants.java @@ -17,6 +17,8 @@ import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_APP_PROFILE_ID; import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_ATTEMPT_LATENCY; +import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT; +import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_GFE_LATENCY; import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_INSTANCE_ID; import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_OP; import static com.google.cloud.bigtable.data.v2.stub.metrics.RpcMeasureConstants.BIGTABLE_OP_ATTEMPT_COUNT; @@ -29,6 +31,7 @@ import io.opencensus.stats.Aggregation; import io.opencensus.stats.Aggregation.Count; import io.opencensus.stats.Aggregation.Distribution; +import io.opencensus.stats.Aggregation.Sum; import io.opencensus.stats.BucketBoundaries; import io.opencensus.stats.View; import java.util.Arrays; @@ -36,6 +39,7 @@ class RpcViewConstants { // Aggregations private static final Aggregation COUNT = Count.create(); + private static final Aggregation SUM = Sum.create(); private static final Aggregation AGGREGATION_WITH_MILLIS_HISTOGRAM = Distribution.create( @@ -124,4 +128,22 @@ class RpcViewConstants { BIGTABLE_APP_PROFILE_ID, BIGTABLE_OP, BIGTABLE_STATUS)); + + static final View BIGTABLE_GFE_LATENCY_VIEW = + View.create( + View.Name.create("cloud.google.com/java/bigtable/gfe_latency"), + "Latency between Google's network receives an RPC and reads back the first byte of the response", + BIGTABLE_GFE_LATENCY, + AGGREGATION_WITH_MILLIS_HISTOGRAM, + ImmutableList.of( + BIGTABLE_INSTANCE_ID, BIGTABLE_PROJECT_ID, BIGTABLE_APP_PROFILE_ID, BIGTABLE_OP)); + + static final View BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW = + View.create( + View.Name.create("cloud.google.com/java/bigtable/gfe_header_missing_count"), + "Number of RPC responses received without the server-timing header, most likely means that the RPC never reached Google's network", + BIGTABLE_GFE_HEADER_MISSING_COUNT, + SUM, + ImmutableList.of( + BIGTABLE_INSTANCE_ID, BIGTABLE_PROJECT_ID, BIGTABLE_APP_PROFILE_ID, BIGTABLE_OP)); } diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViews.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViews.java index cc31539496..9e8f6084a2 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViews.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/RpcViews.java @@ -33,15 +33,49 @@ public class RpcViews { RpcViewConstants.BIGTABLE_ATTEMPT_LATENCY_VIEW, RpcViewConstants.BIGTABLE_ATTEMPTS_PER_OP_VIEW); + private static final ImmutableSet GFE_VIEW_SET = + ImmutableSet.of( + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW); + + private static boolean gfeMetricsRegistered = false; + /** Registers all Bigtable specific views. */ public static void registerBigtableClientViews() { registerBigtableClientViews(Stats.getViewManager()); } + /** + * Register views for GFE metrics, including gfe_latency and gfe_header_missing_count. gfe_latency + * measures the latency between Google's network receives an RPC and reads back the first byte of + * the response. gfe_header_missing_count is a counter of the number of RPC responses without a + * server-timing header. + */ + public static void registerBigtableClientGfeViews() { + registerBigtableClientGfeViews(Stats.getViewManager()); + } + @VisibleForTesting static void registerBigtableClientViews(ViewManager viewManager) { for (View view : BIGTABLE_CLIENT_VIEWS_SET) { viewManager.registerView(view); } } + + @VisibleForTesting + static void registerBigtableClientGfeViews(ViewManager viewManager) { + for (View view : GFE_VIEW_SET) { + viewManager.registerView(view); + } + gfeMetricsRegistered = true; + } + + static boolean isGfeMetricsRegistered() { + return gfeMetricsRegistered; + } + + @VisibleForTesting + static void setGfeMetricsRegistered(boolean gfeMetricsRegistered) { + RpcViews.gfeMetricsRegistered = gfeMetricsRegistered; + } } diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettingsTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettingsTest.java index 2cd55a311c..10ba675826 100644 --- a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettingsTest.java +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettingsTest.java @@ -32,6 +32,7 @@ import com.google.cloud.bigtable.data.v2.models.Query; import com.google.cloud.bigtable.data.v2.models.Row; import com.google.cloud.bigtable.data.v2.models.RowMutation; +import com.google.cloud.bigtable.data.v2.stub.metrics.HeaderTracer; import com.google.common.collect.ImmutableMap; import com.google.common.collect.ImmutableSet; import com.google.common.collect.Range; @@ -75,6 +76,7 @@ public void settingsAreNotLostTest() { CredentialsProvider credentialsProvider = Mockito.mock(CredentialsProvider.class); WatchdogProvider watchdogProvider = Mockito.mock(WatchdogProvider.class); Duration watchdogInterval = Duration.ofSeconds(12); + HeaderTracer headerTracer = Mockito.mock(HeaderTracer.class); EnhancedBigtableStubSettings.Builder builder = EnhancedBigtableStubSettings.newBuilder() @@ -85,7 +87,8 @@ public void settingsAreNotLostTest() { .setEndpoint(endpoint) .setCredentialsProvider(credentialsProvider) .setStreamWatchdogProvider(watchdogProvider) - .setStreamWatchdogCheckInterval(watchdogInterval); + .setStreamWatchdogCheckInterval(watchdogInterval) + .setHeaderTracer(headerTracer); verifyBuilder( builder, @@ -96,7 +99,8 @@ public void settingsAreNotLostTest() { endpoint, credentialsProvider, watchdogProvider, - watchdogInterval); + watchdogInterval, + headerTracer); verifySettings( builder.build(), projectId, @@ -106,7 +110,8 @@ public void settingsAreNotLostTest() { endpoint, credentialsProvider, watchdogProvider, - watchdogInterval); + watchdogInterval, + headerTracer); verifyBuilder( builder.build().toBuilder(), projectId, @@ -116,7 +121,8 @@ public void settingsAreNotLostTest() { endpoint, credentialsProvider, watchdogProvider, - watchdogInterval); + watchdogInterval, + headerTracer); } private void verifyBuilder( @@ -128,7 +134,8 @@ private void verifyBuilder( String endpoint, CredentialsProvider credentialsProvider, WatchdogProvider watchdogProvider, - Duration watchdogInterval) { + Duration watchdogInterval, + HeaderTracer headerTracer) { assertThat(builder.getProjectId()).isEqualTo(projectId); assertThat(builder.getInstanceId()).isEqualTo(instanceId); assertThat(builder.getAppProfileId()).isEqualTo(appProfileId); @@ -137,6 +144,7 @@ private void verifyBuilder( assertThat(builder.getCredentialsProvider()).isEqualTo(credentialsProvider); assertThat(builder.getStreamWatchdogProvider()).isSameInstanceAs(watchdogProvider); assertThat(builder.getStreamWatchdogCheckInterval()).isEqualTo(watchdogInterval); + assertThat(builder.getHeaderTracer()).isEqualTo(headerTracer); } private void verifySettings( @@ -148,7 +156,8 @@ private void verifySettings( String endpoint, CredentialsProvider credentialsProvider, WatchdogProvider watchdogProvider, - Duration watchdogInterval) { + Duration watchdogInterval, + HeaderTracer headerTracer) { assertThat(settings.getProjectId()).isEqualTo(projectId); assertThat(settings.getInstanceId()).isEqualTo(instanceId); assertThat(settings.getAppProfileId()).isEqualTo(appProfileId); @@ -157,6 +166,7 @@ private void verifySettings( assertThat(settings.getCredentialsProvider()).isEqualTo(credentialsProvider); assertThat(settings.getStreamWatchdogProvider()).isSameInstanceAs(watchdogProvider); assertThat(settings.getStreamWatchdogCheckInterval()).isEqualTo(watchdogInterval); + assertThat(settings.getHeaderTracer()).isEqualTo(headerTracer); } @Test @@ -622,12 +632,26 @@ public void isRefreshingChannelFalseValueTest() { assertThat(builder.build().toBuilder().isRefreshingChannel()).isFalse(); } + @Test + public void verifyDefaultHeaderTracerNotNullTest() { + String dummyProjectId = "my-project"; + String dummyInstanceId = "my-instance"; + EnhancedBigtableStubSettings.Builder builder = + EnhancedBigtableStubSettings.newBuilder() + .setProjectId(dummyProjectId) + .setInstanceId(dummyInstanceId); + assertThat(builder.getHeaderTracer()).isNotNull(); + assertThat(builder.build().getHeaderTracer()).isNotNull(); + assertThat(builder.build().toBuilder().getHeaderTracer()).isNotNull(); + } + static final String[] SETTINGS_LIST = { "projectId", "instanceId", "appProfileId", "isRefreshingChannel", "primedTableIds", + "headerTracer", "readRowsSettings", "readRowSettings", "sampleRowKeysSettings", diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerCallableTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerCallableTest.java new file mode 100644 index 0000000000..9538b6a135 --- /dev/null +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerCallableTest.java @@ -0,0 +1,428 @@ +/* + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigtable.data.v2.stub.metrics; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.fail; + +import com.google.api.gax.rpc.ClientContext; +import com.google.api.gax.rpc.UnavailableException; +import com.google.bigtable.v2.BigtableGrpc.BigtableImplBase; +import com.google.bigtable.v2.CheckAndMutateRowRequest; +import com.google.bigtable.v2.CheckAndMutateRowResponse; +import com.google.bigtable.v2.MutateRowRequest; +import com.google.bigtable.v2.MutateRowResponse; +import com.google.bigtable.v2.MutateRowsRequest; +import com.google.bigtable.v2.MutateRowsResponse; +import com.google.bigtable.v2.ReadModifyWriteRowRequest; +import com.google.bigtable.v2.ReadModifyWriteRowResponse; +import com.google.bigtable.v2.ReadRowsRequest; +import com.google.bigtable.v2.ReadRowsResponse; +import com.google.bigtable.v2.SampleRowKeysRequest; +import com.google.bigtable.v2.SampleRowKeysResponse; +import com.google.cloud.bigtable.data.v2.BigtableDataSettings; +import com.google.cloud.bigtable.data.v2.FakeServiceHelper; +import com.google.cloud.bigtable.data.v2.internal.NameUtil; +import com.google.cloud.bigtable.data.v2.models.BulkMutation; +import com.google.cloud.bigtable.data.v2.models.ConditionalRowMutation; +import com.google.cloud.bigtable.data.v2.models.Mutation; +import com.google.cloud.bigtable.data.v2.models.Query; +import com.google.cloud.bigtable.data.v2.models.ReadModifyWriteRow; +import com.google.cloud.bigtable.data.v2.models.RowMutation; +import com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub; +import com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStubSettings; +import com.google.common.collect.ImmutableMap; +import io.grpc.ForwardingServerCall.SimpleForwardingServerCall; +import io.grpc.Metadata; +import io.grpc.ServerCall; +import io.grpc.ServerCallHandler; +import io.grpc.ServerInterceptor; +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import io.grpc.stub.StreamObserver; +import io.opencensus.impl.stats.StatsComponentImpl; +import io.opencensus.stats.StatsComponent; +import io.opencensus.stats.ViewData; +import io.opencensus.tags.TagKey; +import io.opencensus.tags.TagValue; +import io.opencensus.tags.Tags; +import java.util.Random; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; + +@RunWith(JUnit4.class) +public class HeaderTracerCallableTest { + private FakeServiceHelper serviceHelper; + private FakeServiceHelper serviceHelperNoHeader; + + private FakeService fakeService = new FakeService(); + + private final StatsComponent localStats = new StatsComponentImpl(); + private EnhancedBigtableStub stub; + private EnhancedBigtableStub noHeaderStub; + private int attempts; + + private static final String PROJECT_ID = "fake-project"; + private static final String INSTANCE_ID = "fake-instance"; + private static final String APP_PROFILE_ID = "default"; + private static final String TABLE_ID = "fake-table"; + + private static final long WAIT_FOR_METRICS_TIME_MS = 1_000; + + private AtomicInteger fakeServerTiming; + + @Before + public void setUp() throws Exception { + RpcViews.registerBigtableClientGfeViews(localStats.getViewManager()); + + // Create a server that'll inject a server-timing header with a random number and a stub that + // connects to this server. + fakeServerTiming = new AtomicInteger(new Random().nextInt(1000) + 1); + serviceHelper = + new FakeServiceHelper( + new ServerInterceptor() { + @Override + public ServerCall.Listener interceptCall( + ServerCall serverCall, + Metadata metadata, + ServerCallHandler serverCallHandler) { + return serverCallHandler.startCall( + new SimpleForwardingServerCall(serverCall) { + @Override + public void sendHeaders(Metadata headers) { + headers.put( + Metadata.Key.of("server-timing", Metadata.ASCII_STRING_MARSHALLER), + String.format("gfet4t7; dur=%d", fakeServerTiming.get())); + super.sendHeaders(headers); + } + }, + metadata); + } + }, + fakeService); + serviceHelper.start(); + + BigtableDataSettings settings = + BigtableDataSettings.newBuilderForEmulator(serviceHelper.getPort()) + .setProjectId(PROJECT_ID) + .setInstanceId(INSTANCE_ID) + .setAppProfileId(APP_PROFILE_ID) + .build(); + EnhancedBigtableStubSettings stubSettings = + EnhancedBigtableStub.finalizeSettings( + settings.getStubSettings(), Tags.getTagger(), localStats.getStatsRecorder()); + attempts = stubSettings.readRowsSettings().getRetrySettings().getMaxAttempts(); + stub = new EnhancedBigtableStub(stubSettings, ClientContext.create(stubSettings)); + + // Create another server without injecting the server-timing header and another stub that + // connects to it. + serviceHelperNoHeader = new FakeServiceHelper(fakeService); + serviceHelperNoHeader.start(); + + BigtableDataSettings noHeaderSettings = + BigtableDataSettings.newBuilderForEmulator(serviceHelperNoHeader.getPort()) + .setProjectId(PROJECT_ID) + .setInstanceId(INSTANCE_ID) + .setAppProfileId(APP_PROFILE_ID) + .build(); + EnhancedBigtableStubSettings noHeaderStubSettings = + EnhancedBigtableStub.finalizeSettings( + noHeaderSettings.getStubSettings(), Tags.getTagger(), localStats.getStatsRecorder()); + noHeaderStub = + new EnhancedBigtableStub(noHeaderStubSettings, ClientContext.create(noHeaderStubSettings)); + } + + @After + public void tearDown() { + stub.close(); + noHeaderStub.close(); + serviceHelper.shutdown(); + serviceHelperNoHeader.shutdown(); + } + + @Test + public void testGFELatencyMetricReadRows() throws InterruptedException { + stub.readRowsCallable().call(Query.create(TABLE_ID)); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + + long latency = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + + assertThat(latency).isEqualTo(fakeServerTiming.get()); + } + + @Test + public void testGFELatencyMetricMutateRow() throws InterruptedException { + stub.mutateRowCallable().call(RowMutation.create(TABLE_ID, "fake-key")); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + + long latency = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + ImmutableMap.of(RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.MutateRow")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + + assertThat(latency).isEqualTo(fakeServerTiming.get()); + } + + @Test + public void testGFELatencyMetricMutateRows() throws InterruptedException { + BulkMutation mutations = + BulkMutation.create(TABLE_ID) + .add("key", Mutation.create().setCell("fake-family", "fake-qualifier", "fake-value")); + stub.bulkMutateRowsCallable().call(mutations); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + + long latency = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.MutateRows")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + + assertThat(latency).isEqualTo(fakeServerTiming.get()); + } + + @Test + public void testGFELatencySampleRowKeys() throws InterruptedException { + stub.sampleRowKeysCallable().call(TABLE_ID); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + long latency = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.SampleRowKeys")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + assertThat(latency).isEqualTo(fakeServerTiming.get()); + } + + @Test + public void testGFELatencyCheckAndMutateRow() throws InterruptedException { + ConditionalRowMutation mutation = + ConditionalRowMutation.create(TABLE_ID, "fake-key") + .then(Mutation.create().setCell("fake-family", "fake-qualifier", "fake-value")); + stub.checkAndMutateRowCallable().call(mutation); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + long latency = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.CheckAndMutateRow")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + assertThat(latency).isEqualTo(fakeServerTiming.get()); + } + + @Test + public void testGFELatencyReadModifyWriteRow() throws InterruptedException { + ReadModifyWriteRow request = + ReadModifyWriteRow.create(TABLE_ID, "fake-key") + .append("fake-family", "fake-qualifier", "suffix"); + stub.readModifyWriteRowCallable().call(request); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + long latency = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadModifyWriteRow")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + assertThat(latency).isEqualTo(fakeServerTiming.get()); + } + + @Test + public void testGFEMissingHeaderMetric() throws InterruptedException { + // Make a few calls to the server which will inject the server-timing header and the counter + // should be 0. + stub.readRowsCallable().call(Query.create(TABLE_ID)); + stub.mutateRowCallable().call(RowMutation.create(TABLE_ID, "key")); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + long mutateRowMissingCount = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW, + ImmutableMap.of(RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.MutateRow")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + long readRowsMissingCount = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + + assertThat(mutateRowMissingCount).isEqualTo(0); + assertThat(readRowsMissingCount).isEqualTo(0); + + // Make a few more calls to the server which won't add the header and the counter should match + // the number of requests sent. + int readRowsCalls = new Random().nextInt(10) + 1; + int mutateRowCalls = new Random().nextInt(10) + 1; + for (int i = 0; i < mutateRowCalls; i++) { + noHeaderStub.mutateRowCallable().call(RowMutation.create(TABLE_ID, "fake-key" + i)); + } + for (int i = 0; i < readRowsCalls; i++) { + noHeaderStub.readRowsCallable().call(Query.create(TABLE_ID)); + } + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + + mutateRowMissingCount = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW, + ImmutableMap.of(RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.MutateRow")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + readRowsMissingCount = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + + assertThat(mutateRowMissingCount).isEqualTo(mutateRowCalls); + assertThat(readRowsMissingCount).isEqualTo(readRowsCalls); + } + + @Test + public void testMetricsWithErrorResponse() throws InterruptedException { + try { + stub.readRowsCallable().call(Query.create("random-table-id")).iterator().next(); + fail("readrows should throw exception"); + } catch (Exception e) { + assertThat(e).isInstanceOf(UnavailableException.class); + } + + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + long missingCount = + StatsTestUtils.getAggregationValueAsLong( + localStats, + RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW, + ImmutableMap.of(RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); + assertThat(missingCount).isEqualTo(attempts); + } + + @Test + public void testCallableBypassed() throws InterruptedException { + RpcViews.setGfeMetricsRegistered(false); + stub.readRowsCallable().call(Query.create(TABLE_ID)); + Thread.sleep(WAIT_FOR_METRICS_TIME_MS); + ViewData headerMissingView = + localStats + .getViewManager() + .getView(RpcViewConstants.BIGTABLE_GFE_HEADER_MISSING_COUNT_VIEW.getName()); + ViewData latencyView = + localStats.getViewManager().getView(RpcViewConstants.BIGTABLE_GFE_LATENCY_VIEW.getName()); + // Verify that the view is registered by it's not collecting metrics + assertThat(headerMissingView).isNotNull(); + assertThat(latencyView).isNotNull(); + assertThat(headerMissingView.getAggregationMap()).isEmpty(); + assertThat(latencyView.getAggregationMap()).isEmpty(); + } + + private class FakeService extends BigtableImplBase { + private final String defaultTableName = + NameUtil.formatTableName(PROJECT_ID, INSTANCE_ID, TABLE_ID); + + @Override + public void readRows(ReadRowsRequest request, StreamObserver observer) { + if (!request.getTableName().equals(defaultTableName)) { + observer.onError(new StatusRuntimeException(Status.UNAVAILABLE)); + return; + } + observer.onNext(ReadRowsResponse.getDefaultInstance()); + observer.onCompleted(); + } + + @Override + public void mutateRow(MutateRowRequest request, StreamObserver observer) { + observer.onNext(MutateRowResponse.getDefaultInstance()); + observer.onCompleted(); + } + + @Override + public void mutateRows(MutateRowsRequest request, StreamObserver observer) { + observer.onNext(MutateRowsResponse.getDefaultInstance()); + observer.onCompleted(); + } + + @Override + public void sampleRowKeys( + SampleRowKeysRequest request, StreamObserver observer) { + observer.onNext(SampleRowKeysResponse.getDefaultInstance()); + observer.onCompleted(); + } + + @Override + public void checkAndMutateRow( + CheckAndMutateRowRequest request, StreamObserver observer) { + observer.onNext(CheckAndMutateRowResponse.getDefaultInstance()); + observer.onCompleted(); + } + + @Override + public void readModifyWriteRow( + ReadModifyWriteRowRequest request, StreamObserver observer) { + observer.onNext(ReadModifyWriteRowResponse.getDefaultInstance()); + observer.onCompleted(); + } + } +} diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerTest.java new file mode 100644 index 0000000000..30e24dbfe3 --- /dev/null +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/HeaderTracerTest.java @@ -0,0 +1,77 @@ +/* + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigtable.data.v2.stub.metrics; + +import static com.google.common.truth.Truth.assertThat; + +import com.google.common.collect.ImmutableMap; +import io.opencensus.impl.stats.StatsComponentImpl; +import io.opencensus.stats.StatsComponent; +import io.opencensus.stats.StatsRecorder; +import io.opencensus.tags.TagKey; +import io.opencensus.tags.TagValue; +import io.opencensus.tags.Tagger; +import java.util.Map; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; +import org.mockito.Mockito; + +@RunWith(JUnit4.class) +public class HeaderTracerTest { + + private final StatsComponent localStats = new StatsComponentImpl(); + + @Test + public void testDefaultBuilder() { + HeaderTracer.Builder builder = HeaderTracer.newBuilder(); + HeaderTracer tracer = builder.build(); + assertThat(tracer.getStats()).isNotNull(); + assertThat(tracer.getTagger()).isNotNull(); + assertThat(tracer.getStatsAttributes()).isNotNull(); + assertThat(tracer.getStatsAttributes()).isEmpty(); + } + + @Test + public void testBuilder() { + HeaderTracer.Builder builder = HeaderTracer.newBuilder(); + Map attrs = + ImmutableMap.of(TagKey.create("fake-key"), TagValue.create("fake-value")); + Tagger tagger = Mockito.mock(Tagger.class); + StatsRecorder stats = localStats.getStatsRecorder(); + builder.setStats(stats).setStatsAttributes(attrs).setTagger(tagger); + HeaderTracer headerTracer = builder.build(); + assertThat(headerTracer.getStats()).isEqualTo(stats); + assertThat(headerTracer.getTagger()).isEqualTo(tagger); + assertThat(headerTracer.getStatsAttributes()).isEqualTo(attrs); + } + + @Test + public void testToBuilder() { + HeaderTracer.Builder builder = HeaderTracer.newBuilder(); + Map attrs = + ImmutableMap.of(TagKey.create("fake-key"), TagValue.create("fake-value")); + Tagger tagger = Mockito.mock(Tagger.class); + StatsRecorder stats = localStats.getStatsRecorder(); + builder.setStats(stats).setStatsAttributes(attrs).setTagger(tagger); + HeaderTracer headerTracer = builder.build(); + + HeaderTracer.Builder newBuilder = headerTracer.toBuilder(); + assertThat(newBuilder.build().getStats()).isEqualTo(stats); + assertThat(newBuilder.build().getTagger()).isEqualTo(tagger); + assertThat(newBuilder.build().getStatsAttributes()).isEqualTo(attrs); + } +} diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java index 4b025303e4..56d65f8298 100644 --- a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java @@ -39,24 +39,10 @@ import io.grpc.Status; import io.grpc.StatusRuntimeException; import io.grpc.stub.StreamObserver; -import io.opencensus.common.Function; import io.opencensus.impl.stats.StatsComponentImpl; -import io.opencensus.stats.AggregationData; -import io.opencensus.stats.AggregationData.CountData; -import io.opencensus.stats.AggregationData.DistributionData; -import io.opencensus.stats.AggregationData.LastValueDataDouble; -import io.opencensus.stats.AggregationData.LastValueDataLong; -import io.opencensus.stats.AggregationData.SumDataDouble; -import io.opencensus.stats.AggregationData.SumDataLong; -import io.opencensus.stats.View; -import io.opencensus.stats.ViewData; import io.opencensus.tags.TagKey; import io.opencensus.tags.TagValue; import io.opencensus.tags.Tags; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.Objects; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicInteger; import org.junit.After; @@ -117,7 +103,6 @@ public void setUp() throws Exception { EnhancedBigtableStubSettings stubSettings = EnhancedBigtableStub.finalizeSettings( settings.getStubSettings(), Tags.getTagger(), localStats.getStatsRecorder()); - stub = new EnhancedBigtableStub(stubSettings, ClientContext.create(stubSettings)); } @@ -155,11 +140,15 @@ public Object answer(InvocationOnMock invocation) throws Throwable { Thread.sleep(100); long opLatency = - getAggregationValueAsLong( + StatsTestUtils.getAggregationValueAsLong( + localStats, RpcViewConstants.BIGTABLE_OP_LATENCY_VIEW, ImmutableMap.of( RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows"), - RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK"))); + RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); assertThat(opLatency).isIn(Range.closed(sleepTime, elapsed)); } @@ -187,11 +176,15 @@ public Object answer(InvocationOnMock invocation) { Thread.sleep(100); long opLatency = - getAggregationValueAsLong( + StatsTestUtils.getAggregationValueAsLong( + localStats, RpcViewConstants.BIGTABLE_COMPLETED_OP_VIEW, ImmutableMap.of( RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows"), - RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK"))); + RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); assertThat(opLatency).isEqualTo(2); } @@ -225,9 +218,13 @@ public Object answer(InvocationOnMock invocation) throws Throwable { Thread.sleep(100); long firstRowLatency = - getAggregationValueAsLong( + StatsTestUtils.getAggregationValueAsLong( + localStats, RpcViewConstants.BIGTABLE_READ_ROWS_FIRST_ROW_LATENCY_VIEW, - ImmutableMap.of()); + ImmutableMap.of(), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); // adding buffer time to the upper range to allow for a race between the emulator and the client // recording the duration @@ -267,11 +264,15 @@ public Object answer(InvocationOnMock invocation) { Thread.sleep(100); long opLatency = - getAggregationValueAsLong( + StatsTestUtils.getAggregationValueAsLong( + localStats, RpcViewConstants.BIGTABLE_ATTEMPTS_PER_OP_VIEW, ImmutableMap.of( RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows"), - RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK"))); + RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); assertThat(opLatency).isEqualTo(2); } @@ -312,11 +313,15 @@ public Object answer(InvocationOnMock invocation) throws Throwable { Thread.sleep(100); long attemptLatency = - getAggregationValueAsLong( + StatsTestUtils.getAggregationValueAsLong( + localStats, RpcViewConstants.BIGTABLE_ATTEMPT_LATENCY_VIEW, ImmutableMap.of( RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.ReadRows"), - RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK"))); + RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("OK")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); // Average attempt latency will be just a single wait (as opposed to op latency which will be 2x // sleeptime) assertThat(attemptLatency).isIn(Range.closed(sleepTime, elapsed - sleepTime)); @@ -326,70 +331,4 @@ public Object answer(InvocationOnMock invocation) throws Throwable { private static StreamObserver anyObserver(Class returnType) { return (StreamObserver) any(returnType); } - - private long getAggregationValueAsLong(View view, ImmutableMap tags) { - ViewData viewData = localStats.getViewManager().getView(view.getName()); - Map, AggregationData> aggregationMap = - Objects.requireNonNull(viewData).getAggregationMap(); - - List tagValues = new ArrayList<>(); - - for (TagKey column : view.getColumns()) { - if (RpcMeasureConstants.BIGTABLE_PROJECT_ID == column) { - tagValues.add(TagValue.create(PROJECT_ID)); - } else if (RpcMeasureConstants.BIGTABLE_INSTANCE_ID == column) { - tagValues.add(TagValue.create(INSTANCE_ID)); - } else if (RpcMeasureConstants.BIGTABLE_APP_PROFILE_ID == column) { - tagValues.add(TagValue.create(APP_PROFILE_ID)); - } else { - tagValues.add(tags.get(column)); - } - } - - AggregationData aggregationData = aggregationMap.get(tagValues); - - return aggregationData.match( - new Function() { - @Override - public Long apply(SumDataDouble arg) { - return (long) arg.getSum(); - } - }, - new Function() { - @Override - public Long apply(SumDataLong arg) { - return arg.getSum(); - } - }, - new Function() { - @Override - public Long apply(CountData arg) { - return arg.getCount(); - } - }, - new Function() { - @Override - public Long apply(DistributionData arg) { - return (long) arg.getMean(); - } - }, - new Function() { - @Override - public Long apply(LastValueDataDouble arg) { - return (long) arg.getLastValue(); - } - }, - new Function() { - @Override - public Long apply(LastValueDataLong arg) { - return arg.getLastValue(); - } - }, - new Function() { - @Override - public Long apply(AggregationData arg) { - throw new UnsupportedOperationException(); - } - }); - } } diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/StatsTestUtils.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/StatsTestUtils.java index ff37e75a87..6aede96161 100644 --- a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/StatsTestUtils.java +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/StatsTestUtils.java @@ -23,9 +23,13 @@ import com.google.common.collect.Maps; import io.grpc.Context; import io.opencensus.common.Scope; +import io.opencensus.stats.AggregationData; import io.opencensus.stats.Measure; import io.opencensus.stats.MeasureMap; +import io.opencensus.stats.StatsComponent; import io.opencensus.stats.StatsRecorder; +import io.opencensus.stats.View; +import io.opencensus.stats.ViewData; import io.opencensus.tags.Tag; import io.opencensus.tags.TagContext; import io.opencensus.tags.TagContextBuilder; @@ -35,9 +39,12 @@ import io.opencensus.tags.TagValue; import io.opencensus.tags.Tagger; import io.opencensus.tags.unsafe.ContextUtils; +import java.util.ArrayList; import java.util.Iterator; +import java.util.List; import java.util.Map; import java.util.Map.Entry; +import java.util.Objects; import java.util.concurrent.BlockingQueue; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.TimeUnit; @@ -264,4 +271,76 @@ private static ImmutableMap getTags(TagContext tags) { ? ((FakeTagContext) tags).getTags() : ImmutableMap.of(); } + + public static long getAggregationValueAsLong( + StatsComponent stats, + View view, + ImmutableMap tags, + String projectId, + String instanceId, + String appProfileId) { + ViewData viewData = stats.getViewManager().getView(view.getName()); + Map, AggregationData> aggregationMap = + Objects.requireNonNull(viewData).getAggregationMap(); + + List tagValues = new ArrayList<>(); + + for (TagKey column : view.getColumns()) { + if (RpcMeasureConstants.BIGTABLE_PROJECT_ID == column) { + tagValues.add(TagValue.create(projectId)); + } else if (RpcMeasureConstants.BIGTABLE_INSTANCE_ID == column) { + tagValues.add(TagValue.create(instanceId)); + } else if (RpcMeasureConstants.BIGTABLE_APP_PROFILE_ID == column) { + tagValues.add(TagValue.create(appProfileId)); + } else { + tagValues.add(tags.get(column)); + } + } + + AggregationData aggregationData = aggregationMap.get(tagValues); + + return aggregationData.match( + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData.SumDataDouble arg) { + return (long) arg.getSum(); + } + }, + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData.SumDataLong arg) { + return arg.getSum(); + } + }, + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData.CountData arg) { + return arg.getCount(); + } + }, + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData.DistributionData arg) { + return (long) arg.getMean(); + } + }, + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData.LastValueDataDouble arg) { + return (long) arg.getLastValue(); + } + }, + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData.LastValueDataLong arg) { + return arg.getLastValue(); + } + }, + new io.opencensus.common.Function() { + @Override + public Long apply(AggregationData arg) { + throw new UnsupportedOperationException(); + } + }); + } } From 8d7102003b54757b64fd598290301d3b24fd9c29 Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Wed, 16 Dec 2020 19:54:32 -0500 Subject: [PATCH 10/38] fix: fix MutateRowsAttemptCallable to avoid NPE in MetricTracer (#557) --- .../mutaterows/MutateRowsAttemptCallable.java | 12 +++++++---- .../v2/stub/metrics/MetricsTracerTest.java | 20 +++++++++++++++++++ 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/mutaterows/MutateRowsAttemptCallable.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/mutaterows/MutateRowsAttemptCallable.java index 22266b6b81..e85270f619 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/mutaterows/MutateRowsAttemptCallable.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/mutaterows/MutateRowsAttemptCallable.java @@ -160,9 +160,17 @@ public void setExternalFuture(RetryingFuture externalFuture) { @Override public Void call() { try { + // externalFuture is set from MutateRowsRetryingCallable before invoking this method. It + // shouldn't be null unless the code changed Preconditions.checkNotNull( externalFuture, "External future must be set before starting an attempt"); + // attemptStared should be called at the very start of the operation. This will initialize + // variables in ApiTracer and avoid exceptions when the tracer marks the attempt as finished + callContext + .getTracer() + .attemptStarted(externalFuture.getAttemptSettings().getOverallAttemptCount()); + Preconditions.checkState( currentRequest.getEntriesCount() > 0, "Request doesn't have any mutations to send"); @@ -179,10 +187,6 @@ public Void call() { return null; } - callContext - .getTracer() - .attemptStarted(externalFuture.getAttemptSettings().getOverallAttemptCount()); - // Make the actual call ApiFuture> innerFuture = innerCallable.futureCall(currentRequest, currentCallContext); diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java index 56d65f8298..bc6166f79e 100644 --- a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java @@ -26,6 +26,7 @@ import com.google.bigtable.v2.ReadRowsResponse.CellChunk; import com.google.cloud.bigtable.data.v2.BigtableDataSettings; import com.google.cloud.bigtable.data.v2.FakeServiceHelper; +import com.google.cloud.bigtable.data.v2.models.BulkMutation; import com.google.cloud.bigtable.data.v2.models.Query; import com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub; import com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStubSettings; @@ -46,6 +47,7 @@ import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicInteger; import org.junit.After; +import org.junit.Assert; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -327,6 +329,24 @@ public Object answer(InvocationOnMock invocation) throws Throwable { assertThat(attemptLatency).isIn(Range.closed(sleepTime, elapsed - sleepTime)); } + @Test + public void testInvalidRequest() throws InterruptedException { + try { + stub.bulkMutateRowsCallable().call(BulkMutation.create(TABLE_ID)); + Assert.fail("Invalid request should throw exception"); + } catch (IllegalStateException e) { + Thread.sleep(100); + // Verify that the latency is recorded with an error code (in this case UNKNOWN) + long attemptLatency = + getAggregationValueAsLong( + RpcViewConstants.BIGTABLE_ATTEMPT_LATENCY_VIEW, + ImmutableMap.of( + RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.MutateRows"), + RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("UNKNOWN"))); + assertThat(attemptLatency).isAtLeast(0); + } + } + @SuppressWarnings("unchecked") private static StreamObserver anyObserver(Class returnType) { return (StreamObserver) any(returnType); From 23e97cb308403b35fbe972b08048d0e59423e694 Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Thu, 17 Dec 2020 13:02:35 -0500 Subject: [PATCH 11/38] fix: fix MetricTracerTest to rebase on head (#581) --- .../bigtable/data/v2/stub/metrics/MetricsTracerTest.java | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java index bc6166f79e..eb7bdaa998 100644 --- a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/MetricsTracerTest.java @@ -338,11 +338,15 @@ public void testInvalidRequest() throws InterruptedException { Thread.sleep(100); // Verify that the latency is recorded with an error code (in this case UNKNOWN) long attemptLatency = - getAggregationValueAsLong( + StatsTestUtils.getAggregationValueAsLong( + localStats, RpcViewConstants.BIGTABLE_ATTEMPT_LATENCY_VIEW, ImmutableMap.of( RpcMeasureConstants.BIGTABLE_OP, TagValue.create("Bigtable.MutateRows"), - RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("UNKNOWN"))); + RpcMeasureConstants.BIGTABLE_STATUS, TagValue.create("UNKNOWN")), + PROJECT_ID, + INSTANCE_ID, + APP_PROFILE_ID); assertThat(attemptLatency).isAtLeast(0); } } From 93353cfa3cb4bc30146f592bb1d6cdaac1fefebc Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Thu, 17 Dec 2020 13:22:48 -0500 Subject: [PATCH 12/38] chore: move gfe metrics readme (#580) --- .readme-partials.yml | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/.readme-partials.yml b/.readme-partials.yml index 62730df47d..3f62c8db5c 100644 --- a/.readme-partials.yml +++ b/.readme-partials.yml @@ -215,12 +215,20 @@ custom_content: | each client RPC, tagged by operation name and the attempt status. Under normal circumstances, this will be identical to op_latency. However, when the client receives transient errors, op_latency will be the sum of all attempt_latencies - and the exponential delays + and the exponential delays. * `cloud.google.com/java/bigtable/attempts_per_op`: A distribution of attempts that each operation required, tagged by operation name and final operation status. Under normal circumstances, this will be 1. + ### GFE metric views: + * `cloud.google.com/java/bigtable/gfe_latency`: A distribution of the latency + between Google's network receives an RPC and reads back the first byte of + the response. + + * `cloud.google.com/java/bigtable/gfe_header_missing_count`: A counter of the + number of RPC responses received without the server-timing header, which + indicates that the request probably never reached Google's network. By default, the functionality is disabled. For example to enable metrics using [Google Stackdriver](https://cloud.google.com/monitoring/docs/): @@ -276,6 +284,8 @@ custom_content: | ); BigtableDataSettings.enableOpenCensusStats(); + // Enable GFE metric views + BigtableDataSettings.enableGfeOpenCensusStats(); ``` ## Version Conflicts From 4dd763839e1bd6f350f5afaee578aac180322107 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Thu, 17 Dec 2020 10:40:04 -0800 Subject: [PATCH 13/38] chore: regenerate README (#582) This PR was generated using Autosynth. :rainbow:

    Log from Synthtool ``` 2020-12-17 18:27:00,044 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2020-12-17 18:27:00,999 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
    Full log will be available here: https://source.cloud.google.com/results/invocations/a685e710-846a-4969-a779-fee6f22b33bf/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 4 ++-- README.md | 10 ++++------ 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index adbe5a4a5d..328fb8e9d0 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,14 +4,14 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "fa8f5ef216fcbcd9f57343475f03a7353b0e9ddd" + "sha": "93353cfa3cb4bc30146f592bb1d6cdaac1fefebc" } }, { "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "d73e8dea90af1b463a7f9d4a33244cf4612ada7c" + "sha": "41e998d5afdc2c2143a23c9b044b9931936f7318" } } ] diff --git a/README.md b/README.md index bf339adc4a..ad5778e884 100644 --- a/README.md +++ b/README.md @@ -303,15 +303,13 @@ metrics will be tagged with: Under normal circumstances, this will be 1. ### GFE metric views: - * `cloud.google.com/java/bigtable/gfe_latency`: A distribution of the latency - between Google's network receives an RPC and reads back the first byte of - the response. +between Google's network receives an RPC and reads back the first byte of +the response. * `cloud.google.com/java/bigtable/gfe_header_missing_count`: A counter of the - number of RPC responses received without the server-timing header, which - indicates that the request probably never reached Google's network. - +number of RPC responses received without the server-timing header, which +indicates that the request probably never reached Google's network. By default, the functionality is disabled. For example to enable metrics using [Google Stackdriver](https://cloud.google.com/monitoring/docs/): From a9001a88f338fc2acf6bc48927765f29819124ee Mon Sep 17 00:00:00 2001 From: Dmitry <58846611+dmitry-fa@users.noreply.github.com> Date: Mon, 21 Dec 2020 23:08:39 +0300 Subject: [PATCH 14/38] docs: Expand hello world snippet to show how to access specific cells (#516) * samples: Expand hello world snippet to show how to access specific cells by family & qualifier * samples: Expand hello world snippet to show how to access specific cells by family & qualifier --- .../com/m/examples/bigtable/HelloWorld.java | 53 ++++++++++++++++--- .../m/examples/bigtable/HelloWorldTest.java | 25 ++++++--- 2 files changed, 62 insertions(+), 16 deletions(-) diff --git a/samples/snippets/src/main/java/com/m/examples/bigtable/HelloWorld.java b/samples/snippets/src/main/java/com/m/examples/bigtable/HelloWorld.java index 36e1fc08f9..9668cdd80d 100644 --- a/samples/snippets/src/main/java/com/m/examples/bigtable/HelloWorld.java +++ b/samples/snippets/src/main/java/com/m/examples/bigtable/HelloWorld.java @@ -12,7 +12,7 @@ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. -*/ + */ package com.m.examples.bigtable; @@ -29,6 +29,8 @@ import com.google.cloud.bigtable.data.v2.models.RowCell; import com.google.cloud.bigtable.data.v2.models.RowMutation; import java.io.IOException; +import java.util.ArrayList; +import java.util.List; // [END bigtable_hw_imports_veneer] @@ -48,7 +50,8 @@ public class HelloWorld { private static final String COLUMN_FAMILY = "cf1"; - private static final String COLUMN_QUALIFIER = "greeting"; + private static final String COLUMN_QUALIFIER_GREETING = "greeting"; + private static final String COLUMN_QUALIFIER_NAME = "name"; private static final String ROW_KEY_PREFIX = "rowKey"; private final String tableId; private final BigtableDataClient dataClient; @@ -94,8 +97,13 @@ public void run() throws Exception { createTable(); writeToTable(); readSingleRow(); + readSpecificCells(); readTable(); deleteTable(); + close(); + } + + public void close() { dataClient.close(); adminClient.close(); } @@ -119,13 +127,15 @@ public void writeToTable() { // [START bigtable_hw_write_rows_veneer] try { System.out.println("\nWriting some greetings to the table"); - String[] greetings = {"Hello World!", "Hello Bigtable!", "Hello Java!"}; - for (int i = 0; i < greetings.length; i++) { + String[] names = {"World", "Bigtable", "Java"}; + for (int i = 0; i < names.length; i++) { + String greeting = "Hello " + names[i] + "!"; RowMutation rowMutation = RowMutation.create(tableId, ROW_KEY_PREFIX + i) - .setCell(COLUMN_FAMILY, COLUMN_QUALIFIER, greetings[i]); + .setCell(COLUMN_FAMILY, COLUMN_QUALIFIER_NAME, names[i]) + .setCell(COLUMN_FAMILY, COLUMN_QUALIFIER_GREETING, greeting); dataClient.mutateRow(rowMutation); - System.out.println(greetings[i]); + System.out.println(greeting); } } catch (NotFoundException e) { System.err.println("Failed to write to non-existent table: " + e.getMessage()); @@ -134,7 +144,7 @@ public void writeToTable() { } /** Demonstrates how to read a single row from a table. */ - public void readSingleRow() { + public Row readSingleRow() { // [START bigtable_hw_get_by_key_veneer] try { System.out.println("\nReading a single row by row key"); @@ -145,29 +155,56 @@ public void readSingleRow() { "Family: %s Qualifier: %s Value: %s%n", cell.getFamily(), cell.getQualifier().toStringUtf8(), cell.getValue().toStringUtf8()); } + return row; + } catch (NotFoundException e) { + System.err.println("Failed to read from a non-existent table: " + e.getMessage()); + return null; + } + // [END bigtable_hw_get_by_key_veneer] + } + + /** Demonstrates how to access specific cells by family and qualifier. */ + public List readSpecificCells() { + // [START bigtable_hw_get_by_key_veneer] + try { + System.out.println("\nReading specific cells by family and qualifier"); + Row row = dataClient.readRow(tableId, ROW_KEY_PREFIX + 0); + System.out.println("Row: " + row.getKey().toStringUtf8()); + List cells = row.getCells(COLUMN_FAMILY, COLUMN_QUALIFIER_NAME); + for (RowCell cell : cells) { + System.out.printf( + "Family: %s Qualifier: %s Value: %s%n", + cell.getFamily(), cell.getQualifier().toStringUtf8(), cell.getValue().toStringUtf8()); + } + return cells; } catch (NotFoundException e) { System.err.println("Failed to read from a non-existent table: " + e.getMessage()); + return null; } // [END bigtable_hw_get_by_key_veneer] } /** Demonstrates how to read an entire table. */ - public void readTable() { + public List readTable() { // [START bigtable_hw_scan_all_veneer] try { System.out.println("\nReading the entire table"); Query query = Query.create(tableId); ServerStream rowStream = dataClient.readRows(query); + List tableRows = new ArrayList<>(); for (Row r : rowStream) { System.out.println("Row Key: " + r.getKey().toStringUtf8()); + tableRows.add(r); for (RowCell cell : r.getCells()) { System.out.printf( "Family: %s Qualifier: %s Value: %s%n", cell.getFamily(), cell.getQualifier().toStringUtf8(), cell.getValue().toStringUtf8()); } } + return tableRows; } catch (NotFoundException e) { System.err.println("Failed to read a non-existent table: " + e.getMessage()); + return null; } // [END bigtable_hw_scan_all_veneer] } diff --git a/samples/snippets/src/test/java/com/m/examples/bigtable/HelloWorldTest.java b/samples/snippets/src/test/java/com/m/examples/bigtable/HelloWorldTest.java index 985b37bbd7..6904161829 100644 --- a/samples/snippets/src/test/java/com/m/examples/bigtable/HelloWorldTest.java +++ b/samples/snippets/src/test/java/com/m/examples/bigtable/HelloWorldTest.java @@ -12,11 +12,13 @@ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. -*/ + */ package com.m.examples.bigtable; +import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient; @@ -24,7 +26,6 @@ import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest; import com.google.cloud.bigtable.data.v2.BigtableDataClient; import com.google.cloud.bigtable.data.v2.BigtableDataSettings; -import com.google.cloud.bigtable.data.v2.models.Row; import java.io.IOException; import java.util.Random; import java.util.concurrent.TimeUnit; @@ -32,7 +33,6 @@ import java.util.regex.Pattern; import org.junit.After; import org.junit.AfterClass; -import org.junit.AssumptionViolatedException; import org.junit.Before; import org.junit.BeforeClass; import org.junit.Test; @@ -86,10 +86,11 @@ public void setup() throws IOException { } @After - public void after() { + public void teardown() { if (adminClient.exists(tableId)) { adminClient.deleteTable(tableId); } + helloWorld.close(); } @Test @@ -103,18 +104,26 @@ public void testCreateAndDeleteTable() throws IOException { // Deletes a table. testHelloWorld.deleteTable(); assertTrue(!adminClient.exists(testTable)); + testHelloWorld.close(); } @Test public void testWriteToTable() { // Writes to a table. + assertNull(dataClient.readRow(tableId, "rowKey0")); helloWorld.writeToTable(); - Row row = dataClient.readRow(tableId, "rowKey0"); - assertNotNull(row); + assertNotNull(dataClient.readRow(tableId, "rowKey0")); } - // TODO: add test for helloWorld.readSingleRow() - // TODO: add test for helloWorld.readTable() + @Test + public void testReads() { + assertEquals(0, helloWorld.readTable().size()); + helloWorld.writeToTable(); + + assertEquals(2, helloWorld.readSingleRow().getCells().size()); + assertEquals(1, helloWorld.readSpecificCells().size()); + assertEquals(3, helloWorld.readTable().size()); + } @Test public void testRunDoesNotFail() throws Exception { From cfc33522fa37c906cf9a588c89181df9bdd2aabe Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Tue, 29 Dec 2020 11:53:48 -0800 Subject: [PATCH 15/38] chore(java): remove formatter action Source-Author: Jeff Ching Source-Date: Tue Dec 29 10:50:17 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 6133907dbb3ddab204a17a15d5c53ec0aae9b033 Source-Link: https://github.com/googleapis/synthtool/commit/6133907dbb3ddab204a17a15d5c53ec0aae9b033 --- .github/workflows/formatting.yaml | 25 ------------------------- synth.metadata | 5 ++--- 2 files changed, 2 insertions(+), 28 deletions(-) delete mode 100644 .github/workflows/formatting.yaml diff --git a/.github/workflows/formatting.yaml b/.github/workflows/formatting.yaml deleted file mode 100644 index 6844407b4d..0000000000 --- a/.github/workflows/formatting.yaml +++ /dev/null @@ -1,25 +0,0 @@ -on: - pull_request_target: - types: [opened, synchronize] - branches: - - master -name: format -jobs: - format-code: - runs-on: ubuntu-latest - env: - ACCESS_TOKEN: ${{ secrets.YOSHI_CODE_BOT_TOKEN }} - steps: - - uses: actions/checkout@v2 - with: - ref: ${{github.event.pull_request.head.ref}} - repository: ${{github.event.pull_request.head.repo.full_name}} - - uses: actions/setup-java@v1 - with: - java-version: 11 - - run: "mvn com.coveo:fmt-maven-plugin:format" - - uses: googleapis/code-suggester@v1 - with: - command: review - pull_number: ${{ github.event.pull_request.number }} - git_dir: '.' diff --git a/synth.metadata b/synth.metadata index 61bebbe9cf..76de2e57e9 100644 --- a/synth.metadata +++ b/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "27304431b86d38b6292651cf3ee6b463b13050fd" + "sha": "a9001a88f338fc2acf6bc48927765f29819124ee" } }, { @@ -19,7 +19,7 @@ "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "3f67ceece7e797a5736a25488aae35405649b90b" + "sha": "6133907dbb3ddab204a17a15d5c53ec0aae9b033" } } ], @@ -57,7 +57,6 @@ ".github/workflows/approve-readme.yaml", ".github/workflows/auto-release.yaml", ".github/workflows/ci.yaml", - ".github/workflows/formatting.yaml", ".github/workflows/samples.yaml", ".kokoro/build.bat", ".kokoro/build.sh", From 64bb85f620ec26a4d6f2bad2a4df3ffe7383be11 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Thu, 7 Jan 2021 16:35:54 +0100 Subject: [PATCH 16/38] chore(deps): update dependency com.google.cloud:libraries-bom to v16.2.1 (#587) --- samples/snippets/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/samples/snippets/pom.xml b/samples/snippets/pom.xml index 045cef761b..adaefc4374 100644 --- a/samples/snippets/pom.xml +++ b/samples/snippets/pom.xml @@ -30,7 +30,7 @@ com.google.cloud libraries-bom - 16.2.0 + 16.2.1 pom import From e89f3e778bbe29a0c221a38bd1785e1a78e23b46 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Thu, 7 Jan 2021 07:50:03 -0800 Subject: [PATCH 17/38] chore: regenerate README (#588) This PR was generated using Autosynth. :rainbow:
    Log from Synthtool ``` 2021-01-07 15:38:14,137 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2021-01-07 15:38:15,077 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
    Full log will be available here: https://source.cloud.google.com/results/invocations/07c27323-e217-46f5-bcf3-45e28b592a16/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 4 ++-- README.md | 11 +++++++++-- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index 328fb8e9d0..4f2698011e 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,14 +4,14 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "93353cfa3cb4bc30146f592bb1d6cdaac1fefebc" + "sha": "64bb85f620ec26a4d6f2bad2a4df3ffe7383be11" } }, { "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "41e998d5afdc2c2143a23c9b044b9931936f7318" + "sha": "a3e990f3545dc8ccd384a75d20ce9cb185ca6a28" } } ] diff --git a/README.md b/README.md index ad5778e884..6d887ff8d9 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ If you are using Maven with [BOM][libraries-bom], add this to your pom.xml file com.google.cloud libraries-bom - 16.2.0 + 16.2.1 pom import @@ -43,10 +43,17 @@ If you are using Maven without BOM, add this to your dependencies: ``` -If you are using Gradle, add this to your dependencies +If you are using Gradle 5.x or later, add this to your dependencies +```Groovy +implementation platform('com.google.cloud:libraries-bom:16.2.1') + +compile 'com.google.cloud:google-cloud-bigtable' +``` +If you are using Gradle without BOM, add this to your dependencies ```Groovy compile 'com.google.cloud:google-cloud-bigtable:1.19.2' ``` + If you are using SBT, add this to your dependencies ```Scala libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.19.2" From 5035ad0db01a9247634137050698c30da29722a6 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Tue, 12 Jan 2021 20:44:48 +0100 Subject: [PATCH 18/38] deps: update dependency com.google.cloud:google-cloud-shared-dependencies to v0.17.1 (#590) --- google-cloud-bigtable-deps-bom/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index d2e36fe8a7..dc6fa1a2ca 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -79,7 +79,7 @@ com.google.cloud google-cloud-shared-dependencies - 0.17.0 + 0.17.1 pom import From c58b73a7d70c8da1581ac06d77b5e362648a0868 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Thu, 14 Jan 2021 01:48:15 +0100 Subject: [PATCH 19/38] deps: update dependency com.google.cloud:google-cloud-shared-dependencies to v0.18.0 (#592) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [com.google.cloud:google-cloud-shared-dependencies](https://togithub.com/googleapis/java-shared-dependencies) | minor | `0.17.1` -> `0.18.0` | --- ### Release Notes
    googleapis/java-shared-dependencies ### [`v0.18.0`](https://togithub.com/googleapis/java-shared-dependencies/blob/master/CHANGELOG.md#​0180-httpswwwgithubcomgoogleapisjava-shared-dependenciescompare0171v0180-2021-01-13) [Compare Source](https://togithub.com/googleapis/java-shared-dependencies/compare/v0.17.1...v0.18.0) ##### Features - add commons-codec to dependencyManagement ([#​251](https://www.github.com/googleapis/java-shared-dependencies/issues/251)) ([4ee990d](https://www.github.com/googleapis/java-shared-dependencies/commit/4ee990d79c9207c81155f6ee9279308a2d4d0f9d)) ##### Dependencies - update dependency com.google.errorprone:error_prone_annotations to v2.5.0 ([#​247](https://www.github.com/googleapis/java-shared-dependencies/issues/247)) ([37c0861](https://www.github.com/googleapis/java-shared-dependencies/commit/37c0861cfb89f13a0682c98067c633b13b30b827)) ##### [0.17.1](https://www.github.com/googleapis/java-shared-dependencies/compare/0.17.0...v0.17.1) (2021-01-12) ##### Dependencies - update dependency com.fasterxml.jackson:jackson-bom to v2.12.1 ([#​245](https://www.github.com/googleapis/java-shared-dependencies/issues/245)) ([5ffc8a0](https://www.github.com/googleapis/java-shared-dependencies/commit/5ffc8a0d173ea0222ac9610ece0ac2aeb1d17f27))
    --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-bigtable). --- google-cloud-bigtable-deps-bom/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index dc6fa1a2ca..817a701484 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -79,7 +79,7 @@ com.google.cloud google-cloud-shared-dependencies - 0.17.1 + 0.18.0 pom import From dfa4da75e5ac81cc941177462326f7c38f18bacd Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Thu, 14 Jan 2021 17:07:37 +0100 Subject: [PATCH 20/38] deps: update dependency com.google.errorprone:error_prone_annotations to v2.5.0 (#591) --- google-cloud-bigtable-deps-bom/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index 817a701484..9564ce32ad 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -98,7 +98,7 @@ com.google.errorprone error_prone_annotations - 2.4.0 + 2.5.0 From d6a41329c70040ea543dd9e5e4935852a9970ac9 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Fri, 15 Jan 2021 17:16:01 +0100 Subject: [PATCH 21/38] chore(deps): update dependency com.google.cloud:libraries-bom to v16.3.0 (#593) --- samples/snippets/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/samples/snippets/pom.xml b/samples/snippets/pom.xml index adaefc4374..d3b306d130 100644 --- a/samples/snippets/pom.xml +++ b/samples/snippets/pom.xml @@ -30,7 +30,7 @@ com.google.cloud libraries-bom - 16.2.1 + 16.3.0 pom import From ea599a10e2e4fdbaf56c45b74fbb1ea5a708a7f2 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Fri, 15 Jan 2021 17:16:27 +0100 Subject: [PATCH 22/38] deps: update dependency com.google.errorprone:error_prone_annotations to v2.5.1 (#594) --- google-cloud-bigtable-deps-bom/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index 9564ce32ad..cffec3b6a7 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -98,7 +98,7 @@ com.google.errorprone error_prone_annotations - 2.5.0 + 2.5.1 From bcf2c4a2b124dcfd70401c3db5553a6bf837b623 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Fri, 15 Jan 2021 08:36:04 -0800 Subject: [PATCH 23/38] chore: regenerate README (#595) This PR was generated using Autosynth. :rainbow:
    Log from Synthtool ``` 2021-01-15 16:18:13,237 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2021-01-15 16:18:14,164 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
    Full log will be available here: https://source.cloud.google.com/results/invocations/ca64769c-46e8-432d-9a5f-a82d4d202847/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 4 ++-- README.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index 4f2698011e..6670adadb7 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,14 +4,14 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "64bb85f620ec26a4d6f2bad2a4df3ffe7383be11" + "sha": "ea599a10e2e4fdbaf56c45b74fbb1ea5a708a7f2" } }, { "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "a3e990f3545dc8ccd384a75d20ce9cb185ca6a28" + "sha": "16ec872dd898d7de6e1822badfac32484b5d9031" } } ] diff --git a/README.md b/README.md index 6d887ff8d9..3f58f94835 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ If you are using Maven with [BOM][libraries-bom], add this to your pom.xml file com.google.cloud libraries-bom - 16.2.1 + 16.3.0 pom import @@ -45,7 +45,7 @@ If you are using Maven without BOM, add this to your dependencies: If you are using Gradle 5.x or later, add this to your dependencies ```Groovy -implementation platform('com.google.cloud:libraries-bom:16.2.1') +implementation platform('com.google.cloud:libraries-bom:16.3.0') compile 'com.google.cloud:google-cloud-bigtable' ``` From e51d5ce4a87488b49c2c9a1d76c485a0d45475b1 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Fri, 22 Jan 2021 16:33:36 +0100 Subject: [PATCH 24/38] test(deps): update dependency com.google.truth:truth to v1.1.1 (#597) --- pom.xml | 2 +- samples/install-without-bom/pom.xml | 2 +- samples/snapshot/pom.xml | 2 +- samples/snippets/pom.xml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/pom.xml b/pom.xml index c84b8af55c..54f2411f83 100644 --- a/pom.xml +++ b/pom.xml @@ -181,7 +181,7 @@ com.google.truth truth - 1.1 + 1.1.1 junit diff --git a/samples/install-without-bom/pom.xml b/samples/install-without-bom/pom.xml index 96ef9380ac..fc7929e73b 100644 --- a/samples/install-without-bom/pom.xml +++ b/samples/install-without-bom/pom.xml @@ -42,7 +42,7 @@ com.google.truth truth - 1.1 + 1.1.1 test diff --git a/samples/snapshot/pom.xml b/samples/snapshot/pom.xml index 13af92c106..bdc0263f08 100644 --- a/samples/snapshot/pom.xml +++ b/samples/snapshot/pom.xml @@ -41,7 +41,7 @@ com.google.truth truth - 1.1 + 1.1.1 test diff --git a/samples/snippets/pom.xml b/samples/snippets/pom.xml index d3b306d130..fe3d5650d1 100644 --- a/samples/snippets/pom.xml +++ b/samples/snippets/pom.xml @@ -53,7 +53,7 @@ com.google.truth truth - 1.1 + 1.1.1 test From 527e897f08716460afd93e74cf50eefda4e8d935 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Mon, 25 Jan 2021 18:30:15 +0100 Subject: [PATCH 25/38] test(deps): update dependency com.google.truth:truth to v1.1.2 (#598) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.truth:truth](com/google/truth/truth) | `1.1.1` -> `1.1.2` | [![age](https://badges.renovateapi.com/packages/maven/com.google.truth:truth/1.1.2/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.truth:truth/1.1.2/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.truth:truth/1.1.2/compatibility-slim/1.1.1)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.truth:truth/1.1.2/confidence-slim/1.1.1)](https://docs.renovatebot.com/merge-confidence/) | --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-bigtable). --- pom.xml | 2 +- samples/install-without-bom/pom.xml | 2 +- samples/snapshot/pom.xml | 2 +- samples/snippets/pom.xml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/pom.xml b/pom.xml index 54f2411f83..457d7e19b0 100644 --- a/pom.xml +++ b/pom.xml @@ -181,7 +181,7 @@ com.google.truth truth - 1.1.1 + 1.1.2 junit diff --git a/samples/install-without-bom/pom.xml b/samples/install-without-bom/pom.xml index fc7929e73b..198bcb9442 100644 --- a/samples/install-without-bom/pom.xml +++ b/samples/install-without-bom/pom.xml @@ -42,7 +42,7 @@ com.google.truth truth - 1.1.1 + 1.1.2 test diff --git a/samples/snapshot/pom.xml b/samples/snapshot/pom.xml index bdc0263f08..de01bfaf5b 100644 --- a/samples/snapshot/pom.xml +++ b/samples/snapshot/pom.xml @@ -41,7 +41,7 @@ com.google.truth truth - 1.1.1 + 1.1.2 test diff --git a/samples/snippets/pom.xml b/samples/snippets/pom.xml index fe3d5650d1..155489dbbd 100644 --- a/samples/snippets/pom.xml +++ b/samples/snippets/pom.xml @@ -53,7 +53,7 @@ com.google.truth truth - 1.1.1 + 1.1.2 test From 8b2557a67afc68372d823db95305dad36cb6ed22 Mon Sep 17 00:00:00 2001 From: WhiteSource Renovate Date: Tue, 26 Jan 2021 19:30:27 +0100 Subject: [PATCH 26/38] build(deps): update dependency com.google.cloud:google-cloud-shared-config to v0.10.0 (#600) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-shared-config](https://togithub.com/googleapis/java-shared-config) | `0.9.4` -> `0.10.0` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/0.10.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/0.10.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/0.10.0/compatibility-slim/0.9.4)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/0.10.0/confidence-slim/0.9.4)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes
    googleapis/java-shared-config ### [`v0.10.0`](https://togithub.com/googleapis/java-shared-config/blob/master/CHANGELOG.md#​0100-httpswwwgithubcomgoogleapisjava-shared-configcomparev094v0100-2021-01-21) [Compare Source](https://togithub.com/googleapis/java-shared-config/compare/v0.9.4...v0.10.0) ##### Features - adding pom profile to generate docfx yml from javadoc ([#​213](https://www.github.com/googleapis/java-shared-config/issues/213)) ([3527c47](https://www.github.com/googleapis/java-shared-config/commit/3527c47ff413d415f87fccca84358da2c837841d)) ##### Dependencies - update dependency com.puppycrawl.tools:checkstyle to v8.39 ([#​209](https://www.github.com/googleapis/java-shared-config/issues/209)) ([fb53922](https://www.github.com/googleapis/java-shared-config/commit/fb539226d407001822a56c7fff792922cd85d1fe)) ##### [0.9.4](https://www.github.com/googleapis/java-shared-config/compare/v0.9.3...v0.9.4) (2020-10-21) ##### Documentation - Latest for Cloud-RAD ([#​199](https://www.github.com/googleapis/java-shared-config/issues/199)) ([34712af](https://www.github.com/googleapis/java-shared-config/commit/34712afac58aa0d148f0843026b3ff770ee030c2)) ##### [0.9.3](https://www.github.com/googleapis/java-shared-config/compare/v0.9.2...v0.9.3) (2020-10-15) ##### Dependencies - update auto-value-annotation.version to v1.7.4 ([#​157](https://www.github.com/googleapis/java-shared-config/issues/157)) ([5d7e394](https://www.github.com/googleapis/java-shared-config/commit/5d7e394d964010a3e32af492cec4be85aabc3ebf)) ##### [0.9.2](https://www.github.com/googleapis/java-shared-config/compare/v0.9.1...v0.9.2) (2020-07-02) ##### Dependencies - update dependency org.apache.maven.surefire:surefire-junit47 to v3.0.0-M5 ([#​180](https://www.github.com/googleapis/java-shared-config/issues/180)) ([802d9c5](https://www.github.com/googleapis/java-shared-config/commit/802d9c528d34b386face69ca75a014ce57fc3ac1)) ##### [0.9.1](https://www.github.com/googleapis/java-shared-config/compare/v0.9.0...v0.9.1) (2020-07-01) ##### Bug Fixes - maven-dependency-plugin configuration breaking downstream config ([#​174](https://www.github.com/googleapis/java-shared-config/issues/174)) ([507217f](https://www.github.com/googleapis/java-shared-config/commit/507217fe509cd4f16eb50c8075ab43229238e08d)) ##### Documentation - change Devsite output path to /java/docs/reference ([#​176](https://www.github.com/googleapis/java-shared-config/issues/176)) ([8b98af5](https://www.github.com/googleapis/java-shared-config/commit/8b98af54bf503d97bb86b6d02a5c4301b39384e1))
    --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-bigtable). --- google-cloud-bigtable-bom/pom.xml | 2 +- google-cloud-bigtable-deps-bom/pom.xml | 2 +- pom.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/google-cloud-bigtable-bom/pom.xml b/google-cloud-bigtable-bom/pom.xml index 2777bda8f5..a2fa9b6db1 100644 --- a/google-cloud-bigtable-bom/pom.xml +++ b/google-cloud-bigtable-bom/pom.xml @@ -8,7 +8,7 @@ com.google.cloud google-cloud-shared-config - 0.9.4 + 0.10.0 Google Cloud Bigtable BOM diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index cffec3b6a7..da2da4a2f9 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -7,7 +7,7 @@ com.google.cloud google-cloud-shared-config - 0.9.4 + 0.10.0 com.google.cloud diff --git a/pom.xml b/pom.xml index 457d7e19b0..cbe21eb4bd 100644 --- a/pom.xml +++ b/pom.xml @@ -14,7 +14,7 @@ com.google.cloud google-cloud-shared-config - 0.9.4 + 0.10.0 From 3d82093d24f77d32d10fb81a3b5e4d964f0c3e46 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Tue, 26 Jan 2021 11:14:14 -0800 Subject: [PATCH 27/38] chore: adding docfx doclet resource (#599) This PR was generated using Autosynth. :rainbow: Synth log will be available here: https://source.cloud.google.com/results/invocations/04e05d67-cc75-4021-a3f4-7b09ce5647ec/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: https://github.com/googleapis/synthtool/commit/3816b080296d4d52975079fd26c110dd26ba25af --- .kokoro/release/publish_javadoc.cfg | 3 +++ synth.metadata | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/.kokoro/release/publish_javadoc.cfg b/.kokoro/release/publish_javadoc.cfg index d597baebd7..a1da49fb81 100644 --- a/.kokoro/release/publish_javadoc.cfg +++ b/.kokoro/release/publish_javadoc.cfg @@ -27,3 +27,6 @@ before_action { } } } + +# Downloads docfx doclet resource. This will be in ${KOKORO_GFILE_DIR}/ +gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/docfx" \ No newline at end of file diff --git a/synth.metadata b/synth.metadata index 76de2e57e9..dfaf16ad92 100644 --- a/synth.metadata +++ b/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "a9001a88f338fc2acf6bc48927765f29819124ee" + "sha": "527e897f08716460afd93e74cf50eefda4e8d935" } }, { @@ -19,7 +19,7 @@ "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "6133907dbb3ddab204a17a15d5c53ec0aae9b033" + "sha": "3816b080296d4d52975079fd26c110dd26ba25af" } } ], From b09a21c1dd1a05b68bfd3a0134089ba32dca1774 Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Wed, 27 Jan 2021 12:45:18 -0500 Subject: [PATCH 28/38] fix: Retry "received rst stream" (#586) * fix: Retry "received rst stream" * Check if exception is InternalException instead of using fromThrowable --- .../data/v2/stub/EnhancedBigtableStub.java | 9 +- .../ReadRowsConvertExceptionCallable.java | 87 +++++++++++++++++++ .../v2/stub/readrows/ReadRowsRetryTest.java | 37 ++++++++ 3 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsConvertExceptionCallable.java diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java index 448390396f..8f2505c58f 100644 --- a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStub.java @@ -75,6 +75,7 @@ import com.google.cloud.bigtable.data.v2.stub.mutaterows.MutateRowsRetryingCallable; import com.google.cloud.bigtable.data.v2.stub.readrows.FilterMarkerRowsCallable; import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsBatchingDescriptor; +import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsConvertExceptionCallable; import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsResumptionStrategy; import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsRetryCompletedCallable; import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsUserCallable; @@ -345,8 +346,14 @@ public Map extract(ReadRowsRequest readRowsRequest) { .build(), readRowsSettings.getRetryableCodes()); + // Sometimes ReadRows connections are disconnected via an RST frame. This error is transient and + // should be treated similar to UNAVAILABLE. However, this exception has an INTERNAL error code + // which by default is not retryable. Convert the exception so it can be retried in the client. + ServerStreamingCallable convertException = + new ReadRowsConvertExceptionCallable<>(base); + ServerStreamingCallable merging = - new RowMergingCallable<>(base, rowAdapter); + new RowMergingCallable<>(convertException, rowAdapter); // Copy settings for the middle ReadRowsRequest -> RowT callable (as opposed to the inner // ReadRowsRequest -> ReadRowsResponse callable). diff --git a/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsConvertExceptionCallable.java b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsConvertExceptionCallable.java new file mode 100644 index 0000000000..69dd2b5b8a --- /dev/null +++ b/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsConvertExceptionCallable.java @@ -0,0 +1,87 @@ +/* + * Copyright 2021 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigtable.data.v2.stub.readrows; + +import com.google.api.core.InternalApi; +import com.google.api.gax.rpc.ApiCallContext; +import com.google.api.gax.rpc.ApiException; +import com.google.api.gax.rpc.InternalException; +import com.google.api.gax.rpc.ResponseObserver; +import com.google.api.gax.rpc.ServerStreamingCallable; +import com.google.api.gax.rpc.StreamController; + +/** + * This callable converts the "Received rst stream" exception into a retryable {@link ApiException}. + */ +@InternalApi +public final class ReadRowsConvertExceptionCallable + extends ServerStreamingCallable { + + private final ServerStreamingCallable innerCallable; + + public ReadRowsConvertExceptionCallable( + ServerStreamingCallable innerCallable) { + this.innerCallable = innerCallable; + } + + @Override + public void call( + ReadRowsRequest request, ResponseObserver responseObserver, ApiCallContext context) { + ReadRowsConvertExceptionResponseObserver observer = + new ReadRowsConvertExceptionResponseObserver<>(responseObserver); + innerCallable.call(request, observer, context); + } + + private class ReadRowsConvertExceptionResponseObserver implements ResponseObserver { + + private final ResponseObserver outerObserver; + + ReadRowsConvertExceptionResponseObserver(ResponseObserver outerObserver) { + this.outerObserver = outerObserver; + } + + @Override + public void onStart(StreamController controller) { + outerObserver.onStart(controller); + } + + @Override + public void onResponse(RowT response) { + outerObserver.onResponse(response); + } + + @Override + public void onError(Throwable t) { + outerObserver.onError(convertException(t)); + } + + @Override + public void onComplete() { + outerObserver.onComplete(); + } + } + + private Throwable convertException(Throwable t) { + // Long lived connections sometimes are disconnected via an RST frame. This error is + // transient and should be retried. + if (t instanceof InternalException) { + if (t.getMessage() != null && t.getMessage().contains("Received Rst stream")) { + return new InternalException(t, ((InternalException) t).getStatusCode(), true); + } + } + return t; + } +} diff --git a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsRetryTest.java b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsRetryTest.java index 96a22f5182..1536b01e01 100644 --- a/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsRetryTest.java +++ b/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/stub/readrows/ReadRowsRetryTest.java @@ -16,8 +16,11 @@ package com.google.cloud.bigtable.data.v2.stub.readrows; import com.google.api.gax.core.NoCredentialsProvider; +import com.google.api.gax.grpc.GrpcStatusCode; import com.google.api.gax.grpc.GrpcTransportChannel; +import com.google.api.gax.rpc.ApiException; import com.google.api.gax.rpc.FixedTransportChannelProvider; +import com.google.api.gax.rpc.InternalException; import com.google.api.gax.rpc.ServerStream; import com.google.bigtable.v2.BigtableGrpc; import com.google.bigtable.v2.ReadRowsRequest; @@ -39,6 +42,7 @@ import com.google.protobuf.StringValue; import io.grpc.Status; import io.grpc.Status.Code; +import io.grpc.StatusRuntimeException; import io.grpc.stub.StreamObserver; import io.grpc.testing.GrpcServerRule; import java.io.IOException; @@ -260,6 +264,30 @@ public void retryWithLastScannedKeyTest() { Truth.assertThat(actualResults).containsExactly("r7").inOrder(); } + @Test + public void retryRstStreamExceptionTest() { + ApiException exception = + new InternalException( + new StatusRuntimeException( + Status.INTERNAL.withDescription( + "HTTP/2 error code: INTERNAL_ERROR\nReceived Rst stream")), + GrpcStatusCode.of(Code.INTERNAL), + false); + service.expectations.add( + RpcExpectation.create() + .expectRequest("k1") + .expectRequest(Range.closedOpen("r1", "r3")) + .respondWithException(Code.INTERNAL, exception)); + service.expectations.add( + RpcExpectation.create() + .expectRequest("k1") + .expectRequest(Range.closedOpen("r1", "r3")) + .respondWith("k1", "r1", "r2")); + + List actualResults = getResults(Query.create(TABLE_ID).rowKey("k1").range("r1", "r3")); + Truth.assertThat(actualResults).containsExactly("k1", "r1", "r2").inOrder(); + } + private List getResults(Query query) { ServerStream actualRows = client.readRows(query); List actualValues = Lists.newArrayList(); @@ -292,6 +320,8 @@ public void readRows( } if (expectedRpc.statusCode.toStatus().isOk()) { responseObserver.onCompleted(); + } else if (expectedRpc.exception != null) { + responseObserver.onError(expectedRpc.exception); } else { responseObserver.onError(expectedRpc.statusCode.toStatus().asRuntimeException()); } @@ -301,6 +331,7 @@ public void readRows( private static class RpcExpectation { ReadRowsRequest.Builder requestBuilder; Status.Code statusCode; + ApiException exception; List responses; private RpcExpectation() { @@ -370,6 +401,12 @@ RpcExpectation respondWithStatus(Status.Code code) { return this; } + RpcExpectation respondWithException(Status.Code code, ApiException exception) { + this.statusCode = code; + this.exception = exception; + return this; + } + RpcExpectation respondWith(String... responses) { for (String response : responses) { this.responses.add( From ce1687a5beef12568bd651d31d980cdcec6cad49 Mon Sep 17 00:00:00 2001 From: Justin Beckwith Date: Thu, 28 Jan 2021 22:14:21 -0800 Subject: [PATCH 29/38] build: migrate to flakybot (#603) --- .kokoro/build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.kokoro/build.sh b/.kokoro/build.sh index 21b03455e3..9482a73aba 100755 --- a/.kokoro/build.sh +++ b/.kokoro/build.sh @@ -110,8 +110,8 @@ bash .kokoro/coerce_logs.sh if [[ "${ENABLE_BUILD_COP}" == "true" ]] then - chmod +x ${KOKORO_GFILE_DIR}/linux_amd64/buildcop - ${KOKORO_GFILE_DIR}/linux_amd64/buildcop -repo=googleapis/java-bigtable + chmod +x ${KOKORO_GFILE_DIR}/linux_amd64/flakybot + ${KOKORO_GFILE_DIR}/linux_amd64/flakybot -repo=googleapis/java-bigtable fi echo "exiting with ${RETURN_CODE}" From 0b139b5cc02ee69eca0bd5b943d97d8139023e9d Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Wed, 3 Feb 2021 09:00:06 -0800 Subject: [PATCH 30/38] chore: Re-generated to pick up changes from synthtool. (#609) * changes without context autosynth cannot find the source of changes triggered by earlier changes in this repository, or by version upgrades to tools such as linters. * build: migrate to flakybot Source-Author: Justin Beckwith Source-Date: Thu Jan 28 22:22:38 2021 -0800 Source-Repo: googleapis/synthtool Source-Sha: d1bb9173100f62c0cfc8f3138b62241e7f47ca6a Source-Link: https://github.com/googleapis/synthtool/commit/d1bb9173100f62c0cfc8f3138b62241e7f47ca6a * build(java): generate docfx yml on release * feat: generate docfx yml on release * fix: updates name variable * fix: remove non needed resource * fix: update date Source-Author: Emily Ball Source-Date: Mon Feb 1 15:24:59 2021 -0800 Source-Repo: googleapis/synthtool Source-Sha: 5de29e9434b63ea6d7e46dc348521c62969af1a1 Source-Link: https://github.com/googleapis/synthtool/commit/5de29e9434b63ea6d7e46dc348521c62969af1a1 * build(java): run linkage monitor as GitHub action Source-Author: Tomo Suzuki Source-Date: Tue Feb 2 16:20:26 2021 -0500 Source-Repo: googleapis/synthtool Source-Sha: e935c9ecb47da0f2e054f5f1845f7cf7c95fa625 Source-Link: https://github.com/googleapis/synthtool/commit/e935c9ecb47da0f2e054f5f1845f7cf7c95fa625 --- .github/workflows/ci.yaml | 13 ++++++- .kokoro/linkage-monitor.sh | 46 ---------------------- .kokoro/release/publish_javadoc.cfg | 5 +-- .kokoro/release/publish_javadoc11.cfg | 30 +++++++++++++++ .kokoro/release/publish_javadoc11.sh | 55 +++++++++++++++++++++++++++ synth.metadata | 7 ++-- 6 files changed, 102 insertions(+), 54 deletions(-) delete mode 100755 .kokoro/linkage-monitor.sh create mode 100644 .kokoro/release/publish_javadoc11.cfg create mode 100755 .kokoro/release/publish_javadoc11.sh diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 6830220756..98a472e132 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -54,7 +54,18 @@ jobs: with: java-version: 8 - run: java -version - - run: .kokoro/linkage-monitor.sh + - name: Install artifacts to local Maven repository + run: | + source .kokoro/common.sh + retry_with_backoff 3 10 \ + mvn install -B -V \ + -Dmaven.test.skip -DskipTests=true \ + -Dclirr.skip=true \ + -Denforcer.skip=true \ + -Dmaven.javadoc.skip=true \ + -Dgcloud.download.skip=true + shell: bash + - uses: GoogleCloudPlatform/cloud-opensource-java/linkage-monitor@v1-linkagemonitor lint: runs-on: ubuntu-latest steps: diff --git a/.kokoro/linkage-monitor.sh b/.kokoro/linkage-monitor.sh deleted file mode 100755 index 759ab4e2c2..0000000000 --- a/.kokoro/linkage-monitor.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/bash -# Copyright 2019 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -set -eo pipefail -# Display commands being run. -set -x - -## Get the directory of the build script -scriptDir=$(realpath $(dirname "${BASH_SOURCE[0]}")) -## cd to the parent directory, i.e. the root of the git repo -cd ${scriptDir}/.. - -# include common functions -source ${scriptDir}/common.sh - -# Print out Java version -java -version -echo ${JOB_TYPE} - -# attempt to install 3 times with exponential backoff (starting with 10 seconds) -retry_with_backoff 3 10 \ - mvn install -B -V \ - -DskipTests=true \ - -Dclirr.skip=true \ - -Denforcer.skip=true \ - -Dmaven.javadoc.skip=true \ - -Dgcloud.download.skip=true - -# Kokoro job cloud-opensource-java/ubuntu/linkage-monitor-gcs creates this JAR -JAR=linkage-monitor-latest-all-deps.jar -curl -v -O "https://storage.googleapis.com/cloud-opensource-java-linkage-monitor/${JAR}" - -# Fails if there's new linkage errors compared with baseline -java -jar ${JAR} com.google.cloud:libraries-bom diff --git a/.kokoro/release/publish_javadoc.cfg b/.kokoro/release/publish_javadoc.cfg index a1da49fb81..6558e0f39c 100644 --- a/.kokoro/release/publish_javadoc.cfg +++ b/.kokoro/release/publish_javadoc.cfg @@ -26,7 +26,4 @@ before_action { keyname: "docuploader_service_account" } } -} - -# Downloads docfx doclet resource. This will be in ${KOKORO_GFILE_DIR}/ -gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/docfx" \ No newline at end of file +} \ No newline at end of file diff --git a/.kokoro/release/publish_javadoc11.cfg b/.kokoro/release/publish_javadoc11.cfg new file mode 100644 index 0000000000..2ddd71ce63 --- /dev/null +++ b/.kokoro/release/publish_javadoc11.cfg @@ -0,0 +1,30 @@ +# Format: //devtools/kokoro/config/proto/build.proto + +env_vars: { + key: "STAGING_BUCKET_V2" + value: "docs-staging-v2" + # Production will be at: docs-staging-v2 +} + +# Configure the docker image for kokoro-trampoline +env_vars: { + key: "TRAMPOLINE_IMAGE" + value: "gcr.io/cloud-devrel-kokoro-resources/java11" +} + +env_vars: { + key: "TRAMPOLINE_BUILD_FILE" + value: "github/java-memcache/.kokoro/release/publish_javadoc11.sh" +} + +before_action { + fetch_keystore { + keystore_resource { + keystore_config_id: 73713 + keyname: "docuploader_service_account" + } + } +} + +# Downloads docfx doclet resource. This will be in ${KOKORO_GFILE_DIR}/ +gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/docfx" \ No newline at end of file diff --git a/.kokoro/release/publish_javadoc11.sh b/.kokoro/release/publish_javadoc11.sh new file mode 100755 index 0000000000..8dd5a24c3f --- /dev/null +++ b/.kokoro/release/publish_javadoc11.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# Copyright 2021 Google Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eo pipefail + +if [[ -z "${CREDENTIALS}" ]]; then + CREDENTIALS=${KOKORO_KEYSTORE_DIR}/73713_docuploader_service_account +fi + +if [[ -z "${STAGING_BUCKET_V2}" ]]; then + echo "Need to set STAGING_BUCKET_V2 environment variable" + exit 1 +fi + +# work from the git root directory +pushd $(dirname "$0")/../../ + +# install docuploader package +python3 -m pip install gcp-docuploader + +# compile all packages +mvn clean install -B -q -DskipTests=true + +export NAME=google-cloud-bigtable +export VERSION=$(grep ${NAME}: versions.txt | cut -d: -f3) + +# V3 generates docfx yml from javadoc +# generate yml +mvn clean site -B -q -P docFX + +pushd target/docfx-yml + +# create metadata +python3 -m docuploader create-metadata \ + --name ${NAME} \ + --version ${VERSION} \ + --language java + +# upload yml +python3 -m docuploader upload . \ + --credentials ${CREDENTIALS} \ + --staging-bucket ${STAGING_BUCKET_V2} \ + --destination-prefix docfx- diff --git a/synth.metadata b/synth.metadata index dfaf16ad92..a168a8aba1 100644 --- a/synth.metadata +++ b/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "527e897f08716460afd93e74cf50eefda4e8d935" + "sha": "ce1687a5beef12568bd651d31d980cdcec6cad49" } }, { @@ -19,7 +19,7 @@ "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "3816b080296d4d52975079fd26c110dd26ba25af" + "sha": "e935c9ecb47da0f2e054f5f1845f7cf7c95fa625" } } ], @@ -67,7 +67,6 @@ ".kokoro/continuous/java8.cfg", ".kokoro/continuous/readme.cfg", ".kokoro/dependencies.sh", - ".kokoro/linkage-monitor.sh", ".kokoro/nightly/common.cfg", ".kokoro/nightly/java11.cfg", ".kokoro/nightly/java7.cfg", @@ -95,6 +94,8 @@ ".kokoro/release/promote.sh", ".kokoro/release/publish_javadoc.cfg", ".kokoro/release/publish_javadoc.sh", + ".kokoro/release/publish_javadoc11.cfg", + ".kokoro/release/publish_javadoc11.sh", ".kokoro/release/snapshot.cfg", ".kokoro/release/snapshot.sh", ".kokoro/release/stage.cfg", From c560ac93a3c939cbc72cc9addb9ddeeeca98c5d3 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Thu, 4 Feb 2021 12:36:05 -0800 Subject: [PATCH 31/38] chore: Re-generated to pick up changes from synthtool (#611) This PR was generated using Autosynth. :rainbow: Synth log will be available here: https://source.cloud.google.com/results/invocations/f83a1787-8e7f-4e63-8cd7-669254653c34/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: https://github.com/googleapis/synthtool/commit/27b2d4f4674840628d0b75c5941e89c12af4764f Source-Link: https://github.com/googleapis/synthtool/commit/140ba24a136c63e7f10a998a63e7898aed63ea7d --- .github/workflows/auto-release.yaml | 2 +- LICENSE | 1 - synth.metadata | 4 ++-- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/.github/workflows/auto-release.yaml b/.github/workflows/auto-release.yaml index 2b6cdbc976..76e6404b4e 100644 --- a/.github/workflows/auto-release.yaml +++ b/.github/workflows/auto-release.yaml @@ -17,7 +17,7 @@ jobs: } // only approve PRs like "chore: release " - if ( !context.payload.pull_request.title.startsWith("chore: release") ) { + if ( !context.payload.pull_request.title.startsWith("chore(master): release") ) { return; } diff --git a/LICENSE b/LICENSE index d645695673..261eeb9e9f 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,3 @@ - Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ diff --git a/synth.metadata b/synth.metadata index a168a8aba1..aeb05d9e7f 100644 --- a/synth.metadata +++ b/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "ce1687a5beef12568bd651d31d980cdcec6cad49" + "sha": "0b139b5cc02ee69eca0bd5b943d97d8139023e9d" } }, { @@ -19,7 +19,7 @@ "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "e935c9ecb47da0f2e054f5f1845f7cf7c95fa625" + "sha": "27b2d4f4674840628d0b75c5941e89c12af4764f" } } ], From 4893b16460d9a7c943d82f0d679f5bfea9196319 Mon Sep 17 00:00:00 2001 From: kolea2 <45548808+kolea2@users.noreply.github.com> Date: Thu, 4 Feb 2021 16:02:09 -0500 Subject: [PATCH 32/38] chore: add remove_method api for latest microgen changes (#589) * chore: add remove_method api for latest microgen changes * fix spacing --- synth.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/synth.py b/synth.py index e12767cd8b..e26edaac07 100644 --- a/synth.py +++ b/synth.py @@ -124,6 +124,15 @@ def generate_admin_api(gapic): s.copy(library / f'grpc-google-cloud-bigtable-admin-v2-java/src', f'grpc-google-cloud-bigtable-admin-v2/src') s.copy(library / f'proto-google-cloud-bigtable-admin-v2-java/src', f'proto-google-cloud-bigtable-admin-v2/src') + #todo: fix in synthtool removing comments with method + java.remove_method(f'./grpc-google-cloud-bigtable-admin-v2/src/main/java/com/google/bigtable/admin/v2/BigtableInstanceAdminGrpc.java', "public UnaryCallSettings updateInstanceSettings()") + + java.remove_method(f'./google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/admin/v2/BaseBigtableInstanceAdminSettings.java', "/** Returns the builder for the settings used for calls to updateInstance. */") + java.remove_method(f'./google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/admin/v2/BaseBigtableInstanceAdminSettings.java', "public UnaryCallSettings.Builder updateInstanceSettings()") + + java.remove_method(f'./google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/admin/v2/BaseBigtableInstanceAdminSettings.java', "/** Returns the object with the settings used for calls to updateInstance. */") + java.remove_method(f'./google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/admin/v2/BaseBigtableInstanceAdminSettings.java', "public UnaryCallSettings updateInstanceSettings()") + make_internal_only(internal_only) java.format_code(f'./grpc-google-cloud-bigtable-admin-v2/src') From bb3ed6dcbadbd70dbd9c68152c8d93c4cefd4dcb Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Thu, 4 Feb 2021 21:28:15 -0800 Subject: [PATCH 33/38] fix: update repo name (#615) This PR was generated using Autosynth. :rainbow: Synth log will be available here: https://source.cloud.google.com/results/invocations/fb5ed929-07a6-4b9c-9c01-20c1d4125a75/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: https://github.com/googleapis/synthtool/commit/692715c0f23a7bb3bfbbaa300f7620ddfa8c47e5 --- .kokoro/release/publish_javadoc11.cfg | 4 ++-- synth.metadata | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.kokoro/release/publish_javadoc11.cfg b/.kokoro/release/publish_javadoc11.cfg index 2ddd71ce63..cd7ab7dda4 100644 --- a/.kokoro/release/publish_javadoc11.cfg +++ b/.kokoro/release/publish_javadoc11.cfg @@ -14,7 +14,7 @@ env_vars: { env_vars: { key: "TRAMPOLINE_BUILD_FILE" - value: "github/java-memcache/.kokoro/release/publish_javadoc11.sh" + value: "github/java-bigtable/.kokoro/release/publish_javadoc11.sh" } before_action { @@ -27,4 +27,4 @@ before_action { } # Downloads docfx doclet resource. This will be in ${KOKORO_GFILE_DIR}/ -gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/docfx" \ No newline at end of file +gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/docfx" diff --git a/synth.metadata b/synth.metadata index aeb05d9e7f..dc0eb7ac37 100644 --- a/synth.metadata +++ b/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "0b139b5cc02ee69eca0bd5b943d97d8139023e9d" + "sha": "4893b16460d9a7c943d82f0d679f5bfea9196319" } }, { @@ -19,7 +19,7 @@ "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "27b2d4f4674840628d0b75c5941e89c12af4764f" + "sha": "692715c0f23a7bb3bfbbaa300f7620ddfa8c47e5" } } ], From 3e8af747b329f6656a410160e8da14fd8227c8fc Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Fri, 5 Feb 2021 11:33:31 -0500 Subject: [PATCH 34/38] docs: Update stackdriver examples for tracing and stats (#613) * docs: Update stackdriver examples for tracing and stats * update based on review --- README.md | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3f58f94835..19fa0bf7e4 100644 --- a/README.md +++ b/README.md @@ -203,7 +203,7 @@ try { TIP: If you are experiencing version conflicts with gRPC, see [Version Conflicts](#version-conflicts). -## OpenCensus Tracing +## Client request tracing: OpenCensus Tracing Cloud Bigtable client supports [OpenCensus Tracing](https://opencensus.io/tracing/), which gives insight into the client internals and aids in debugging production issues. @@ -258,6 +258,8 @@ StackdriverTraceExporter.createAndRegister( .setProjectId("YOUR_PROJECT_ID") .build()); ``` +You can view the traces on the Google Cloud Platform Console +[Trace](https://console.cloud.google.com/traces) page. By default traces are [sampled](https://opencensus.io/tracing/sampling) at a rate of about 1/10,000. You can configure a higher rate by updating the active tracing params: @@ -273,7 +275,7 @@ Tracing.getTraceConfig().updateActiveTraceParams( ); ``` -## OpenCensus Stats +## Enabling Cloud Bigtable Metrics: OpenCensus Stats Cloud Bigtable client supports [Opencensus Metrics](https://opencensus.io/stats/), which gives insight into the client internals and aids in debugging production issues. @@ -376,6 +378,34 @@ BigtableDataSettings.enableOpenCensusStats(); BigtableDataSettings.enableGfeOpenCensusStats(); ``` +You can view the metrics on the Google Cloud Platform Console +[Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer) +page. + +You can configure how frequently metrics are pushed to StackDriver and the +[Monitored resource type](https://cloud.google.com/monitoring/api/resources) by +updating `StackdriverStatsConfiguration`: + +``` java +// Example: configuring export interval and monitored resource type +StackdriverStatsExporter.createAndRegister( + StackdriverStatsConfiguration.builder() + .setProjectId("YOUR_PROJECT_ID") + // Exporting metrics every 10 seconds + .setExportInterval(Duration.create(10, 0)) + // Configure monitored resource type. A common practice is to use the + // monitored resource objects that represent the physical resources + // where your application code is running. See the full list of + // monitored resource type here: + // https://cloud.google.com/monitoring/api/resources + .setMonitoredResource(MonitoredResource.newBuilder() + .setType("global") + .putLabels("project_id", "YOUR_PROJECT_ID") + .build()) + .build() +); +``` + ## Version Conflicts google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought From 4580cdff0b045d3861d1ab060e9a4d5aa617c982 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Fri, 5 Feb 2021 09:14:03 -0800 Subject: [PATCH 35/38] chore: regenerate README (#616) This PR was generated using Autosynth. :rainbow:
    Log from Synthtool ``` 2021-02-05 16:35:43,869 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2021-02-05 16:35:44,744 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
    Full log will be available here: https://source.cloud.google.com/results/invocations/6c71dbb7-cb82-4b8b-84bd-724ab1181866/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 4 +-- README.md | 34 ++------------------ 2 files changed, 4 insertions(+), 34 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index 6670adadb7..b49daf3d1e 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,14 +4,14 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "ea599a10e2e4fdbaf56c45b74fbb1ea5a708a7f2" + "sha": "3e8af747b329f6656a410160e8da14fd8227c8fc" } }, { "git": { "name": "synthtool", "remote": "https://github.com/googleapis/synthtool.git", - "sha": "16ec872dd898d7de6e1822badfac32484b5d9031" + "sha": "692715c0f23a7bb3bfbbaa300f7620ddfa8c47e5" } } ] diff --git a/README.md b/README.md index 19fa0bf7e4..3f58f94835 100644 --- a/README.md +++ b/README.md @@ -203,7 +203,7 @@ try { TIP: If you are experiencing version conflicts with gRPC, see [Version Conflicts](#version-conflicts). -## Client request tracing: OpenCensus Tracing +## OpenCensus Tracing Cloud Bigtable client supports [OpenCensus Tracing](https://opencensus.io/tracing/), which gives insight into the client internals and aids in debugging production issues. @@ -258,8 +258,6 @@ StackdriverTraceExporter.createAndRegister( .setProjectId("YOUR_PROJECT_ID") .build()); ``` -You can view the traces on the Google Cloud Platform Console -[Trace](https://console.cloud.google.com/traces) page. By default traces are [sampled](https://opencensus.io/tracing/sampling) at a rate of about 1/10,000. You can configure a higher rate by updating the active tracing params: @@ -275,7 +273,7 @@ Tracing.getTraceConfig().updateActiveTraceParams( ); ``` -## Enabling Cloud Bigtable Metrics: OpenCensus Stats +## OpenCensus Stats Cloud Bigtable client supports [Opencensus Metrics](https://opencensus.io/stats/), which gives insight into the client internals and aids in debugging production issues. @@ -378,34 +376,6 @@ BigtableDataSettings.enableOpenCensusStats(); BigtableDataSettings.enableGfeOpenCensusStats(); ``` -You can view the metrics on the Google Cloud Platform Console -[Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer) -page. - -You can configure how frequently metrics are pushed to StackDriver and the -[Monitored resource type](https://cloud.google.com/monitoring/api/resources) by -updating `StackdriverStatsConfiguration`: - -``` java -// Example: configuring export interval and monitored resource type -StackdriverStatsExporter.createAndRegister( - StackdriverStatsConfiguration.builder() - .setProjectId("YOUR_PROJECT_ID") - // Exporting metrics every 10 seconds - .setExportInterval(Duration.create(10, 0)) - // Configure monitored resource type. A common practice is to use the - // monitored resource objects that represent the physical resources - // where your application code is running. See the full list of - // monitored resource type here: - // https://cloud.google.com/monitoring/api/resources - .setMonitoredResource(MonitoredResource.newBuilder() - .setType("global") - .putLabels("project_id", "YOUR_PROJECT_ID") - .build()) - .build() -); -``` - ## Version Conflicts google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought From 7eb60875bb054bd25338325b465550411ebf6a4b Mon Sep 17 00:00:00 2001 From: Mattie Fu Date: Fri, 5 Feb 2021 15:44:45 -0500 Subject: [PATCH 36/38] chore: move readme changes on stackdriver examples (#617) --- .readme-partials.yml | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/.readme-partials.yml b/.readme-partials.yml index 3f62c8db5c..dd804349e4 100644 --- a/.readme-partials.yml +++ b/.readme-partials.yml @@ -115,7 +115,7 @@ custom_content: | TIP: If you are experiencing version conflicts with gRPC, see [Version Conflicts](#version-conflicts). - ## OpenCensus Tracing + ## Client request tracing: OpenCensus Tracing Cloud Bigtable client supports [OpenCensus Tracing](https://opencensus.io/tracing/), which gives insight into the client internals and aids in debugging production issues. @@ -171,6 +171,9 @@ custom_content: | .build()); ``` + You can view the traces on the Google Cloud Platform Console + [Trace](https://console.cloud.google.com/traces) page. + By default traces are [sampled](https://opencensus.io/tracing/sampling) at a rate of about 1/10,000. You can configure a higher rate by updating the active tracing params: @@ -185,7 +188,7 @@ custom_content: | ); ``` - ## OpenCensus Stats + ## Enabling Cloud Bigtable Metrics: OpenCensus Stats Cloud Bigtable client supports [Opencensus Metrics](https://opencensus.io/stats/), which gives insight into the client internals and aids in debugging production issues. @@ -288,6 +291,34 @@ custom_content: | BigtableDataSettings.enableGfeOpenCensusStats(); ``` + You can view the metrics on the Google Cloud Platform Console + [Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer) + page. + + You can configure how frequently metrics are pushed to StackDriver and the + [Monitored resource type](https://cloud.google.com/monitoring/api/resources) by + updating `StackdriverStatsConfiguration`: + + ``` java + // Example: configuring export interval and monitored resource type + StackdriverStatsExporter.createAndRegister( + StackdriverStatsConfiguration.builder() + .setProjectId("YOUR_PROJECT_ID") + // Exporting metrics every 10 seconds + .setExportInterval(Duration.create(10, 0)) + // Configure monitored resource type. A common practice is to use the + // monitored resource objects that represent the physical resources + // where your application code is running. See the full list of + // monitored resource type here: + // https://cloud.google.com/monitoring/api/resources + .setMonitoredResource(MonitoredResource.newBuilder() + .setType("global") + .putLabels("project_id", "YOUR_PROJECT_ID") + .build()) + .build() + ); + ``` + ## Version Conflicts google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought From f0e5d56a8e2fb1716f06a274e89a6ca368aa8172 Mon Sep 17 00:00:00 2001 From: Yoshi Automation Bot Date: Fri, 5 Feb 2021 12:58:05 -0800 Subject: [PATCH 37/38] chore: regenerate README (#619) This PR was generated using Autosynth. :rainbow:
    Log from Synthtool ``` 2021-02-05 20:47:02,858 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-bigtable/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2021-02-05 20:47:03,749 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ```
    Full log will be available here: https://source.cloud.google.com/results/invocations/697a0cf5-def6-45ec-a126-d13bcbd210de/targets - [ ] To automatically regenerate this PR, check this box. --- .github/readme/synth.metadata/synth.metadata | 2 +- README.md | 35 ++++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/.github/readme/synth.metadata/synth.metadata b/.github/readme/synth.metadata/synth.metadata index b49daf3d1e..431c5f6e36 100644 --- a/.github/readme/synth.metadata/synth.metadata +++ b/.github/readme/synth.metadata/synth.metadata @@ -4,7 +4,7 @@ "git": { "name": ".", "remote": "https://github.com/googleapis/java-bigtable.git", - "sha": "3e8af747b329f6656a410160e8da14fd8227c8fc" + "sha": "7eb60875bb054bd25338325b465550411ebf6a4b" } }, { diff --git a/README.md b/README.md index 3f58f94835..0807e969ed 100644 --- a/README.md +++ b/README.md @@ -203,7 +203,7 @@ try { TIP: If you are experiencing version conflicts with gRPC, see [Version Conflicts](#version-conflicts). -## OpenCensus Tracing +## Client request tracing: OpenCensus Tracing Cloud Bigtable client supports [OpenCensus Tracing](https://opencensus.io/tracing/), which gives insight into the client internals and aids in debugging production issues. @@ -259,6 +259,9 @@ StackdriverTraceExporter.createAndRegister( .build()); ``` +You can view the traces on the Google Cloud Platform Console +[Trace](https://console.cloud.google.com/traces) page. + By default traces are [sampled](https://opencensus.io/tracing/sampling) at a rate of about 1/10,000. You can configure a higher rate by updating the active tracing params: @@ -273,7 +276,7 @@ Tracing.getTraceConfig().updateActiveTraceParams( ); ``` -## OpenCensus Stats +## Enabling Cloud Bigtable Metrics: OpenCensus Stats Cloud Bigtable client supports [Opencensus Metrics](https://opencensus.io/stats/), which gives insight into the client internals and aids in debugging production issues. @@ -376,6 +379,34 @@ BigtableDataSettings.enableOpenCensusStats(); BigtableDataSettings.enableGfeOpenCensusStats(); ``` +You can view the metrics on the Google Cloud Platform Console +[Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer) +page. + +You can configure how frequently metrics are pushed to StackDriver and the +[Monitored resource type](https://cloud.google.com/monitoring/api/resources) by +updating `StackdriverStatsConfiguration`: + +``` java +// Example: configuring export interval and monitored resource type +StackdriverStatsExporter.createAndRegister( + StackdriverStatsConfiguration.builder() + .setProjectId("YOUR_PROJECT_ID") + // Exporting metrics every 10 seconds + .setExportInterval(Duration.create(10, 0)) + // Configure monitored resource type. A common practice is to use the + // monitored resource objects that represent the physical resources + // where your application code is running. See the full list of + // monitored resource type here: + // https://cloud.google.com/monitoring/api/resources + .setMonitoredResource(MonitoredResource.newBuilder() + .setType("global") + .putLabels("project_id", "YOUR_PROJECT_ID") + .build()) + .build() +); +``` + ## Version Conflicts google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought From 9cc505fbb491882edf9e81965e8ccde3a10ab4c4 Mon Sep 17 00:00:00 2001 From: "release-please[bot]" <55107282+release-please[bot]@users.noreply.github.com> Date: Mon, 8 Feb 2021 19:40:02 +0000 Subject: [PATCH 38/38] chore(master): release 1.20.0 (#610) :robot: I have created a release \*beep\* \*boop\* --- ## [1.20.0](https://www.github.com/googleapis/java-bigtable/compare/v1.19.2...v1.20.0) (2021-02-05) ### Features * Surface the server-timing metric ([#535](https://www.github.com/googleapis/java-bigtable/issues/535)) ([8240779](https://www.github.com/googleapis/java-bigtable/commit/8240779434a602dc8b2bf90dbe539c5d7470d850)) ### Bug Fixes * fix MetricTracerTest to rebase on head ([#581](https://www.github.com/googleapis/java-bigtable/issues/581)) ([23e97cb](https://www.github.com/googleapis/java-bigtable/commit/23e97cb308403b35fbe972b08048d0e59423e694)) * fix MutateRowsAttemptCallable to avoid NPE in MetricTracer ([#557](https://www.github.com/googleapis/java-bigtable/issues/557)) ([8d71020](https://www.github.com/googleapis/java-bigtable/commit/8d7102003b54757b64fd598290301d3b24fd9c29)) * Retry "received rst stream" ([#586](https://www.github.com/googleapis/java-bigtable/issues/586)) ([b09a21c](https://www.github.com/googleapis/java-bigtable/commit/b09a21c1dd1a05b68bfd3a0134089ba32dca1774)) * update repo name ([#615](https://www.github.com/googleapis/java-bigtable/issues/615)) ([bb3ed6d](https://www.github.com/googleapis/java-bigtable/commit/bb3ed6dcbadbd70dbd9c68152c8d93c4cefd4dcb)) ### Dependencies * update dependency com.google.cloud:google-cloud-shared-dependencies to v0.17.1 ([#590](https://www.github.com/googleapis/java-bigtable/issues/590)) ([5035ad0](https://www.github.com/googleapis/java-bigtable/commit/5035ad0db01a9247634137050698c30da29722a6)) * update dependency com.google.cloud:google-cloud-shared-dependencies to v0.18.0 ([#592](https://www.github.com/googleapis/java-bigtable/issues/592)) ([c58b73a](https://www.github.com/googleapis/java-bigtable/commit/c58b73a7d70c8da1581ac06d77b5e362648a0868)) * update dependency com.google.errorprone:error_prone_annotations to v2.5.0 ([#591](https://www.github.com/googleapis/java-bigtable/issues/591)) ([dfa4da7](https://www.github.com/googleapis/java-bigtable/commit/dfa4da75e5ac81cc941177462326f7c38f18bacd)) * update dependency com.google.errorprone:error_prone_annotations to v2.5.1 ([#594](https://www.github.com/googleapis/java-bigtable/issues/594)) ([ea599a1](https://www.github.com/googleapis/java-bigtable/commit/ea599a10e2e4fdbaf56c45b74fbb1ea5a708a7f2)) ### Documentation * Expand hello world snippet to show how to access specific cells ([#516](https://www.github.com/googleapis/java-bigtable/issues/516)) ([a9001a8](https://www.github.com/googleapis/java-bigtable/commit/a9001a88f338fc2acf6bc48927765f29819124ee)) * Update stackdriver examples for tracing and stats ([#613](https://www.github.com/googleapis/java-bigtable/issues/613)) ([3e8af74](https://www.github.com/googleapis/java-bigtable/commit/3e8af747b329f6656a410160e8da14fd8227c8fc)) * use autogenerated readme functionality and regenerate ([#568](https://www.github.com/googleapis/java-bigtable/issues/568)) ([844e5be](https://www.github.com/googleapis/java-bigtable/commit/844e5beb6230df6ca220935056aae7f6e73d2bc0)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --- CHANGELOG.md | 30 ++++++++++++++++++++ google-cloud-bigtable-bom/pom.xml | 14 ++++----- google-cloud-bigtable-deps-bom/pom.xml | 2 +- google-cloud-bigtable-emulator/pom.xml | 8 +++--- google-cloud-bigtable/pom.xml | 10 +++---- grpc-google-cloud-bigtable-admin-v2/pom.xml | 8 +++--- grpc-google-cloud-bigtable-v2/pom.xml | 8 +++--- pom.xml | 2 +- proto-google-cloud-bigtable-admin-v2/pom.xml | 8 +++--- proto-google-cloud-bigtable-v2/pom.xml | 8 +++--- samples/snapshot/pom.xml | 2 +- versions.txt | 12 ++++---- 12 files changed, 71 insertions(+), 41 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 48585112af..eb9c618931 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,35 @@ # Changelog +## [1.20.0](https://www.github.com/googleapis/java-bigtable/compare/v1.19.2...v1.20.0) (2021-02-05) + + +### Features + +* Surface the server-timing metric ([#535](https://www.github.com/googleapis/java-bigtable/issues/535)) ([8240779](https://www.github.com/googleapis/java-bigtable/commit/8240779434a602dc8b2bf90dbe539c5d7470d850)) + + +### Bug Fixes + +* fix MetricTracerTest to rebase on head ([#581](https://www.github.com/googleapis/java-bigtable/issues/581)) ([23e97cb](https://www.github.com/googleapis/java-bigtable/commit/23e97cb308403b35fbe972b08048d0e59423e694)) +* fix MutateRowsAttemptCallable to avoid NPE in MetricTracer ([#557](https://www.github.com/googleapis/java-bigtable/issues/557)) ([8d71020](https://www.github.com/googleapis/java-bigtable/commit/8d7102003b54757b64fd598290301d3b24fd9c29)) +* Retry "received rst stream" ([#586](https://www.github.com/googleapis/java-bigtable/issues/586)) ([b09a21c](https://www.github.com/googleapis/java-bigtable/commit/b09a21c1dd1a05b68bfd3a0134089ba32dca1774)) +* update repo name ([#615](https://www.github.com/googleapis/java-bigtable/issues/615)) ([bb3ed6d](https://www.github.com/googleapis/java-bigtable/commit/bb3ed6dcbadbd70dbd9c68152c8d93c4cefd4dcb)) + + +### Dependencies + +* update dependency com.google.cloud:google-cloud-shared-dependencies to v0.17.1 ([#590](https://www.github.com/googleapis/java-bigtable/issues/590)) ([5035ad0](https://www.github.com/googleapis/java-bigtable/commit/5035ad0db01a9247634137050698c30da29722a6)) +* update dependency com.google.cloud:google-cloud-shared-dependencies to v0.18.0 ([#592](https://www.github.com/googleapis/java-bigtable/issues/592)) ([c58b73a](https://www.github.com/googleapis/java-bigtable/commit/c58b73a7d70c8da1581ac06d77b5e362648a0868)) +* update dependency com.google.errorprone:error_prone_annotations to v2.5.0 ([#591](https://www.github.com/googleapis/java-bigtable/issues/591)) ([dfa4da7](https://www.github.com/googleapis/java-bigtable/commit/dfa4da75e5ac81cc941177462326f7c38f18bacd)) +* update dependency com.google.errorprone:error_prone_annotations to v2.5.1 ([#594](https://www.github.com/googleapis/java-bigtable/issues/594)) ([ea599a1](https://www.github.com/googleapis/java-bigtable/commit/ea599a10e2e4fdbaf56c45b74fbb1ea5a708a7f2)) + + +### Documentation + +* Expand hello world snippet to show how to access specific cells ([#516](https://www.github.com/googleapis/java-bigtable/issues/516)) ([a9001a8](https://www.github.com/googleapis/java-bigtable/commit/a9001a88f338fc2acf6bc48927765f29819124ee)) +* Update stackdriver examples for tracing and stats ([#613](https://www.github.com/googleapis/java-bigtable/issues/613)) ([3e8af74](https://www.github.com/googleapis/java-bigtable/commit/3e8af747b329f6656a410160e8da14fd8227c8fc)) +* use autogenerated readme functionality and regenerate ([#568](https://www.github.com/googleapis/java-bigtable/issues/568)) ([844e5be](https://www.github.com/googleapis/java-bigtable/commit/844e5beb6230df6ca220935056aae7f6e73d2bc0)) + ### [1.19.2](https://www.github.com/googleapis/java-bigtable/compare/v1.19.1...v1.19.2) (2020-12-15) diff --git a/google-cloud-bigtable-bom/pom.xml b/google-cloud-bigtable-bom/pom.xml index a2fa9b6db1..44950429aa 100644 --- a/google-cloud-bigtable-bom/pom.xml +++ b/google-cloud-bigtable-bom/pom.xml @@ -3,7 +3,7 @@ 4.0.0 com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom com.google.cloud @@ -72,32 +72,32 @@ com.google.cloud google-cloud-bigtable - 1.19.3-SNAPSHOT + 1.20.0 com.google.cloud google-cloud-bigtable-emulator - 0.128.3-SNAPSHOT + 0.129.0 com.google.api.grpc grpc-google-cloud-bigtable-admin-v2 - 1.19.3-SNAPSHOT + 1.20.0 com.google.api.grpc grpc-google-cloud-bigtable-v2 - 1.19.3-SNAPSHOT + 1.20.0 com.google.api.grpc proto-google-cloud-bigtable-admin-v2 - 1.19.3-SNAPSHOT + 1.20.0 com.google.api.grpc proto-google-cloud-bigtable-v2 - 1.19.3-SNAPSHOT + 1.20.0 diff --git a/google-cloud-bigtable-deps-bom/pom.xml b/google-cloud-bigtable-deps-bom/pom.xml index da2da4a2f9..2db2667bef 100644 --- a/google-cloud-bigtable-deps-bom/pom.xml +++ b/google-cloud-bigtable-deps-bom/pom.xml @@ -12,7 +12,7 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom diff --git a/google-cloud-bigtable-emulator/pom.xml b/google-cloud-bigtable-emulator/pom.xml index a90f63e942..a16eca7916 100644 --- a/google-cloud-bigtable-emulator/pom.xml +++ b/google-cloud-bigtable-emulator/pom.xml @@ -5,7 +5,7 @@ 4.0.0 google-cloud-bigtable-emulator - 0.128.3-SNAPSHOT + 0.129.0 Google Cloud Java - Bigtable Emulator https://github.com/googleapis/java-bigtable @@ -14,7 +14,7 @@ com.google.cloud google-cloud-bigtable-parent - 1.19.3-SNAPSHOT + 1.20.0 scm:git:git@github.com:googleapis/java-bigtable.git @@ -80,14 +80,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import diff --git a/google-cloud-bigtable/pom.xml b/google-cloud-bigtable/pom.xml index 59ea2b0a1e..dd5da594b9 100644 --- a/google-cloud-bigtable/pom.xml +++ b/google-cloud-bigtable/pom.xml @@ -2,7 +2,7 @@ 4.0.0 google-cloud-bigtable - 1.19.3-SNAPSHOT + 1.20.0 jar Google Cloud Bigtable https://github.com/googleapis/java-bigtable @@ -12,11 +12,11 @@ com.google.cloud google-cloud-bigtable-parent - 1.19.3-SNAPSHOT + 1.20.0 - 1.19.3-SNAPSHOT + 1.20.0 google-cloud-bigtable @@ -39,14 +39,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import diff --git a/grpc-google-cloud-bigtable-admin-v2/pom.xml b/grpc-google-cloud-bigtable-admin-v2/pom.xml index 4ef1af15e7..77b06ea876 100644 --- a/grpc-google-cloud-bigtable-admin-v2/pom.xml +++ b/grpc-google-cloud-bigtable-admin-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc grpc-google-cloud-bigtable-admin-v2 - 1.19.3-SNAPSHOT + 1.20.0 grpc-google-cloud-bigtable-admin-v2 GRPC library for grpc-google-cloud-bigtable-admin-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.3-SNAPSHOT + 1.20.0 @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import diff --git a/grpc-google-cloud-bigtable-v2/pom.xml b/grpc-google-cloud-bigtable-v2/pom.xml index a4f67619c0..1d9174e712 100644 --- a/grpc-google-cloud-bigtable-v2/pom.xml +++ b/grpc-google-cloud-bigtable-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc grpc-google-cloud-bigtable-v2 - 1.19.3-SNAPSHOT + 1.20.0 grpc-google-cloud-bigtable-v2 GRPC library for grpc-google-cloud-bigtable-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.3-SNAPSHOT + 1.20.0 @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import diff --git a/pom.xml b/pom.xml index cbe21eb4bd..c416025ca9 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ google-cloud-bigtable-parent pom - 1.19.3-SNAPSHOT + 1.20.0 Google Cloud Bigtable Parent https://github.com/googleapis/java-bigtable diff --git a/proto-google-cloud-bigtable-admin-v2/pom.xml b/proto-google-cloud-bigtable-admin-v2/pom.xml index 097539eaf8..0016c9b99f 100644 --- a/proto-google-cloud-bigtable-admin-v2/pom.xml +++ b/proto-google-cloud-bigtable-admin-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc proto-google-cloud-bigtable-admin-v2 - 1.19.3-SNAPSHOT + 1.20.0 proto-google-cloud-bigtable-admin-v2 PROTO library for proto-google-cloud-bigtable-admin-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.3-SNAPSHOT + 1.20.0 @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import diff --git a/proto-google-cloud-bigtable-v2/pom.xml b/proto-google-cloud-bigtable-v2/pom.xml index b726220eb9..4fed7e2809 100644 --- a/proto-google-cloud-bigtable-v2/pom.xml +++ b/proto-google-cloud-bigtable-v2/pom.xml @@ -4,13 +4,13 @@ 4.0.0 com.google.api.grpc proto-google-cloud-bigtable-v2 - 1.19.3-SNAPSHOT + 1.20.0 proto-google-cloud-bigtable-v2 PROTO library for proto-google-cloud-bigtable-v2 com.google.cloud google-cloud-bigtable-parent - 1.19.3-SNAPSHOT + 1.20.0 @@ -18,14 +18,14 @@ com.google.cloud google-cloud-bigtable-deps-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import com.google.cloud google-cloud-bigtable-bom - 1.19.3-SNAPSHOT + 1.20.0 pom import diff --git a/samples/snapshot/pom.xml b/samples/snapshot/pom.xml index de01bfaf5b..ca5e4b9820 100644 --- a/samples/snapshot/pom.xml +++ b/samples/snapshot/pom.xml @@ -28,7 +28,7 @@ com.google.cloud google-cloud-bigtable - 1.19.3-SNAPSHOT + 1.20.0 diff --git a/versions.txt b/versions.txt index adb04530ee..39792207da 100644 --- a/versions.txt +++ b/versions.txt @@ -1,9 +1,9 @@ # Format: # module:released-version:current-version -google-cloud-bigtable:1.19.2:1.19.3-SNAPSHOT -grpc-google-cloud-bigtable-admin-v2:1.19.2:1.19.3-SNAPSHOT -grpc-google-cloud-bigtable-v2:1.19.2:1.19.3-SNAPSHOT -proto-google-cloud-bigtable-admin-v2:1.19.2:1.19.3-SNAPSHOT -proto-google-cloud-bigtable-v2:1.19.2:1.19.3-SNAPSHOT -google-cloud-bigtable-emulator:0.128.2:0.128.3-SNAPSHOT +google-cloud-bigtable:1.20.0:1.20.0 +grpc-google-cloud-bigtable-admin-v2:1.20.0:1.20.0 +grpc-google-cloud-bigtable-v2:1.20.0:1.20.0 +proto-google-cloud-bigtable-admin-v2:1.20.0:1.20.0 +proto-google-cloud-bigtable-v2:1.20.0:1.20.0 +google-cloud-bigtable-emulator:0.129.0:0.129.0