Skip to content
This repository was archived by the owner on Sep 16, 2023. It is now read-only.

Commit ac18c87

Browse files
yoshi-automationchingor13
yoshi-automation
and
chingor13
authored
feat(v1): add Jobs.SubmitJobAsOperation, SparkR and Presto job types to WorkflowTemplates (#137)
* fix Dataproc: add missing `REQUIRED` annotation. googleapis/googleapis@7504771 commit 75047719f704d61f405cac6d7439637ab36c1232 Author: Google APIs <[email protected]> Date: Wed Apr 1 18:40:05 2020 -0700 fix Dataproc: add missing `REQUIRED` annotation. PiperOrigin-RevId: 304305096 * Add v1beta2 SubmitJobAsOperation RPC to Dataproc. googleapis/googleapis@5378173 commit 5378173a889f9c7d83e36e52d38a6267190de692 Author: Google APIs <[email protected]> Date: Fri Apr 3 05:15:57 2020 -0700 Add v1beta2 SubmitJobAsOperation RPC to Dataproc. PiperOrigin-RevId: 304594381 * Updates to Dataproc v1beta2 API: - Adds SparkR and Presto job types to WorkflowTemplates - Adds new Optional Components - Clarifies usage of some APIs googleapis/googleapis@cad0f51 commit cad0f5137a70d0d14a8d9acbfcee98e4cd3e9662 Author: Google APIs <[email protected]> Date: Mon Apr 6 09:55:41 2020 -0700 Updates to Dataproc v1beta2 API: - Adds SparkR and Presto job types to WorkflowTemplates - Adds new Optional Components - Clarifies usage of some APIs PiperOrigin-RevId: 305053062 * Updates Dataproc v1 API: - Adds Dataproc Jobs.SubmitJobAsOperation RPC - Adds SparkR and Presto job types to WorkflowTemplates - Adds new Optional Components - Clarifies usage of some APIs googleapis/googleapis@9ff6fd3 commit 9ff6fd3b22f99167827e89aae7778408b5e82425 Author: Google APIs <[email protected]> Date: Mon Apr 6 09:57:56 2020 -0700 Updates Dataproc v1 API: - Adds Dataproc Jobs.SubmitJobAsOperation RPC - Adds SparkR and Presto job types to WorkflowTemplates - Adds new Optional Components - Clarifies usage of some APIs PiperOrigin-RevId: 305053617 * docs: change relative URLs to absolute URLs to fix broken links. googleapis/googleapis@7445911 commit 744591190e828440f72745aef217f883afd1fd71 Author: Google APIs <[email protected]> Date: Tue Apr 7 13:06:04 2020 -0700 docs: change relative URLs to absolute URLs to fix broken links. PiperOrigin-RevId: 305323909 * chore: allow proto interface changes Co-authored-by: Jeff Ching <[email protected]>
1 parent 043ab3a commit ac18c87

File tree

97 files changed

+14403
-1913
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

97 files changed

+14403
-1913
lines changed

‎google-cloud-dataproc-bom/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
<version>0.5.0</version>
1212
</parent>
1313

14-
<name>Google Cloud dataproc BOM</name>
14+
<name>Google Cloud Dataproc BOM</name>
1515
<url>https://github.com/googleapis/java-dataproc</url>
1616
<description>
1717
BOM for Google Cloud Dataproc

‎google-cloud-dataproc/src/main/java/com/google/cloud/dataproc/v1/AutoscalingPolicyServiceClient.java

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -251,7 +251,7 @@ public final AutoscalingPolicy updateAutoscalingPolicy(UpdateAutoscalingPolicyRe
251251
* has the following format: `projects/{project_id}/regions/{region}`
252252
* <p>&#42; For `projects.locations.autoscalingPolicies.create`, the resource name of the
253253
* location has the following format: `projects/{project_id}/locations/{location}`
254-
* @param policy The autoscaling policy to create.
254+
* @param policy Required. The autoscaling policy to create.
255255
* @throws com.google.api.gax.rpc.ApiException if the remote call fails
256256
*/
257257
public final AutoscalingPolicy createAutoscalingPolicy(
@@ -284,7 +284,7 @@ public final AutoscalingPolicy createAutoscalingPolicy(
284284
* has the following format: `projects/{project_id}/regions/{region}`
285285
* <p>&#42; For `projects.locations.autoscalingPolicies.create`, the resource name of the
286286
* location has the following format: `projects/{project_id}/locations/{location}`
287-
* @param policy The autoscaling policy to create.
287+
* @param policy Required. The autoscaling policy to create.
288288
* @throws com.google.api.gax.rpc.ApiException if the remote call fails
289289
*/
290290
public final AutoscalingPolicy createAutoscalingPolicy(
@@ -317,7 +317,7 @@ public final AutoscalingPolicy createAutoscalingPolicy(
317317
* has the following format: `projects/{project_id}/regions/{region}`
318318
* <p>&#42; For `projects.locations.autoscalingPolicies.create`, the resource name of the
319319
* location has the following format: `projects/{project_id}/locations/{location}`
320-
* @param policy The autoscaling policy to create.
320+
* @param policy Required. The autoscaling policy to create.
321321
* @throws com.google.api.gax.rpc.ApiException if the remote call fails
322322
*/
323323
public final AutoscalingPolicy createAutoscalingPolicy(String parent, AutoscalingPolicy policy) {
@@ -335,8 +335,10 @@ public final AutoscalingPolicy createAutoscalingPolicy(String parent, Autoscalin
335335
* <pre><code>
336336
* try (AutoscalingPolicyServiceClient autoscalingPolicyServiceClient = AutoscalingPolicyServiceClient.create()) {
337337
* RegionName parent = RegionName.of("[PROJECT]", "[REGION]");
338+
* AutoscalingPolicy policy = AutoscalingPolicy.newBuilder().build();
338339
* CreateAutoscalingPolicyRequest request = CreateAutoscalingPolicyRequest.newBuilder()
339340
* .setParent(parent.toString())
341+
* .setPolicy(policy)
340342
* .build();
341343
* AutoscalingPolicy response = autoscalingPolicyServiceClient.createAutoscalingPolicy(request);
342344
* }
@@ -358,8 +360,10 @@ public final AutoscalingPolicy createAutoscalingPolicy(CreateAutoscalingPolicyRe
358360
* <pre><code>
359361
* try (AutoscalingPolicyServiceClient autoscalingPolicyServiceClient = AutoscalingPolicyServiceClient.create()) {
360362
* RegionName parent = RegionName.of("[PROJECT]", "[REGION]");
363+
* AutoscalingPolicy policy = AutoscalingPolicy.newBuilder().build();
361364
* CreateAutoscalingPolicyRequest request = CreateAutoscalingPolicyRequest.newBuilder()
362365
* .setParent(parent.toString())
366+
* .setPolicy(policy)
363367
* .build();
364368
* ApiFuture&lt;AutoscalingPolicy&gt; future = autoscalingPolicyServiceClient.createAutoscalingPolicyCallable().futureCall(request);
365369
* // Do something

‎google-cloud-dataproc/src/main/java/com/google/cloud/dataproc/v1/ClusterControllerClient.java

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -677,7 +677,7 @@ public final UnaryCallable<GetClusterRequest, Cluster> getClusterCallable() {
677677

678678
// AUTO-GENERATED DOCUMENTATION AND METHOD
679679
/**
680-
* Lists all regions/{region}/clusters in a project.
680+
* Lists all regions/{region}/clusters in a project alphabetically.
681681
*
682682
* <p>Sample code:
683683
*
@@ -704,7 +704,7 @@ public final ListClustersPagedResponse listClusters(String projectId, String reg
704704

705705
// AUTO-GENERATED DOCUMENTATION AND METHOD
706706
/**
707-
* Lists all regions/{region}/clusters in a project.
707+
* Lists all regions/{region}/clusters in a project alphabetically.
708708
*
709709
* <p>Sample code:
710710
*
@@ -751,7 +751,7 @@ public final ListClustersPagedResponse listClusters(
751751

752752
// AUTO-GENERATED DOCUMENTATION AND METHOD
753753
/**
754-
* Lists all regions/{region}/clusters in a project.
754+
* Lists all regions/{region}/clusters in a project alphabetically.
755755
*
756756
* <p>Sample code:
757757
*
@@ -778,7 +778,7 @@ public final ListClustersPagedResponse listClusters(ListClustersRequest request)
778778

779779
// AUTO-GENERATED DOCUMENTATION AND METHOD
780780
/**
781-
* Lists all regions/{region}/clusters in a project.
781+
* Lists all regions/{region}/clusters in a project alphabetically.
782782
*
783783
* <p>Sample code:
784784
*
@@ -805,7 +805,7 @@ public final ListClustersPagedResponse listClusters(ListClustersRequest request)
805805

806806
// AUTO-GENERATED DOCUMENTATION AND METHOD
807807
/**
808-
* Lists all regions/{region}/clusters in a project.
808+
* Lists all regions/{region}/clusters in a project alphabetically.
809809
*
810810
* <p>Sample code:
811811
*

‎google-cloud-dataproc/src/main/java/com/google/cloud/dataproc/v1/JobControllerClient.java

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,14 +20,18 @@
2020
import com.google.api.core.ApiFutures;
2121
import com.google.api.core.BetaApi;
2222
import com.google.api.gax.core.BackgroundResource;
23+
import com.google.api.gax.longrunning.OperationFuture;
2324
import com.google.api.gax.paging.AbstractFixedSizeCollection;
2425
import com.google.api.gax.paging.AbstractPage;
2526
import com.google.api.gax.paging.AbstractPagedListResponse;
27+
import com.google.api.gax.rpc.OperationCallable;
2628
import com.google.api.gax.rpc.PageContext;
2729
import com.google.api.gax.rpc.UnaryCallable;
2830
import com.google.cloud.dataproc.v1.stub.JobControllerStub;
2931
import com.google.cloud.dataproc.v1.stub.JobControllerStubSettings;
3032
import com.google.common.util.concurrent.MoreExecutors;
33+
import com.google.longrunning.Operation;
34+
import com.google.longrunning.OperationsClient;
3135
import com.google.protobuf.Empty;
3236
import java.io.IOException;
3337
import java.util.List;
@@ -107,6 +111,7 @@
107111
public class JobControllerClient implements BackgroundResource {
108112
private final JobControllerSettings settings;
109113
private final JobControllerStub stub;
114+
private final OperationsClient operationsClient;
110115

111116
/** Constructs an instance of JobControllerClient with default settings. */
112117
public static final JobControllerClient create() throws IOException {
@@ -139,12 +144,14 @@ public static final JobControllerClient create(JobControllerStub stub) {
139144
protected JobControllerClient(JobControllerSettings settings) throws IOException {
140145
this.settings = settings;
141146
this.stub = ((JobControllerStubSettings) settings.getStubSettings()).createStub();
147+
this.operationsClient = OperationsClient.create(this.stub.getOperationsStub());
142148
}
143149

144150
@BetaApi("A restructuring of stub classes is planned, so this may break in the future")
145151
protected JobControllerClient(JobControllerStub stub) {
146152
this.settings = null;
147153
this.stub = stub;
154+
this.operationsClient = OperationsClient.create(this.stub.getOperationsStub());
148155
}
149156

150157
public final JobControllerSettings getSettings() {
@@ -156,6 +163,16 @@ public JobControllerStub getStub() {
156163
return stub;
157164
}
158165

166+
/**
167+
* Returns the OperationsClient that can be used to query the status of a long-running operation
168+
* returned by another API method call.
169+
*/
170+
@BetaApi(
171+
"The surface for long-running operations is not stable yet and may change in the future.")
172+
public final OperationsClient getOperationsClient() {
173+
return operationsClient;
174+
}
175+
159176
// AUTO-GENERATED DOCUMENTATION AND METHOD
160177
/**
161178
* Submits a job to a cluster.
@@ -709,6 +726,119 @@ public final UnaryCallable<DeleteJobRequest, Empty> deleteJobCallable() {
709726
return stub.deleteJobCallable();
710727
}
711728

729+
// AUTO-GENERATED DOCUMENTATION AND METHOD
730+
/**
731+
* Submits job to a cluster.
732+
*
733+
* <p>Sample code:
734+
*
735+
* <pre><code>
736+
* try (JobControllerClient jobControllerClient = JobControllerClient.create()) {
737+
* String projectId = "";
738+
* String region = "";
739+
* Job job = Job.newBuilder().build();
740+
* Job response = jobControllerClient.submitJobAsOperationAsync(projectId, region, job).get();
741+
* }
742+
* </code></pre>
743+
*
744+
* @param projectId Required. The ID of the Google Cloud Platform project that the job belongs to.
745+
* @param region Required. The Dataproc region in which to handle the request.
746+
* @param job Required. The job resource.
747+
* @throws com.google.api.gax.rpc.ApiException if the remote call fails
748+
*/
749+
@BetaApi(
750+
"The surface for long-running operations is not stable yet and may change in the future.")
751+
public final OperationFuture<Job, JobMetadata> submitJobAsOperationAsync(
752+
String projectId, String region, Job job) {
753+
SubmitJobRequest request =
754+
SubmitJobRequest.newBuilder().setProjectId(projectId).setRegion(region).setJob(job).build();
755+
return submitJobAsOperationAsync(request);
756+
}
757+
758+
// AUTO-GENERATED DOCUMENTATION AND METHOD
759+
/**
760+
* Submits job to a cluster.
761+
*
762+
* <p>Sample code:
763+
*
764+
* <pre><code>
765+
* try (JobControllerClient jobControllerClient = JobControllerClient.create()) {
766+
* String projectId = "";
767+
* String region = "";
768+
* Job job = Job.newBuilder().build();
769+
* SubmitJobRequest request = SubmitJobRequest.newBuilder()
770+
* .setProjectId(projectId)
771+
* .setRegion(region)
772+
* .setJob(job)
773+
* .build();
774+
* Job response = jobControllerClient.submitJobAsOperationAsync(request).get();
775+
* }
776+
* </code></pre>
777+
*
778+
* @param request The request object containing all of the parameters for the API call.
779+
* @throws com.google.api.gax.rpc.ApiException if the remote call fails
780+
*/
781+
@BetaApi(
782+
"The surface for long-running operations is not stable yet and may change in the future.")
783+
public final OperationFuture<Job, JobMetadata> submitJobAsOperationAsync(
784+
SubmitJobRequest request) {
785+
return submitJobAsOperationOperationCallable().futureCall(request);
786+
}
787+
788+
// AUTO-GENERATED DOCUMENTATION AND METHOD
789+
/**
790+
* Submits job to a cluster.
791+
*
792+
* <p>Sample code:
793+
*
794+
* <pre><code>
795+
* try (JobControllerClient jobControllerClient = JobControllerClient.create()) {
796+
* String projectId = "";
797+
* String region = "";
798+
* Job job = Job.newBuilder().build();
799+
* SubmitJobRequest request = SubmitJobRequest.newBuilder()
800+
* .setProjectId(projectId)
801+
* .setRegion(region)
802+
* .setJob(job)
803+
* .build();
804+
* OperationFuture&lt;Job, JobMetadata&gt; future = jobControllerClient.submitJobAsOperationOperationCallable().futureCall(request);
805+
* // Do something
806+
* Job response = future.get();
807+
* }
808+
* </code></pre>
809+
*/
810+
@BetaApi("The surface for use by generated code is not stable yet and may change in the future.")
811+
public final OperationCallable<SubmitJobRequest, Job, JobMetadata>
812+
submitJobAsOperationOperationCallable() {
813+
return stub.submitJobAsOperationOperationCallable();
814+
}
815+
816+
// AUTO-GENERATED DOCUMENTATION AND METHOD
817+
/**
818+
* Submits job to a cluster.
819+
*
820+
* <p>Sample code:
821+
*
822+
* <pre><code>
823+
* try (JobControllerClient jobControllerClient = JobControllerClient.create()) {
824+
* String projectId = "";
825+
* String region = "";
826+
* Job job = Job.newBuilder().build();
827+
* SubmitJobRequest request = SubmitJobRequest.newBuilder()
828+
* .setProjectId(projectId)
829+
* .setRegion(region)
830+
* .setJob(job)
831+
* .build();
832+
* ApiFuture&lt;Operation&gt; future = jobControllerClient.submitJobAsOperationCallable().futureCall(request);
833+
* // Do something
834+
* Operation response = future.get();
835+
* }
836+
* </code></pre>
837+
*/
838+
public final UnaryCallable<SubmitJobRequest, Operation> submitJobAsOperationCallable() {
839+
return stub.submitJobAsOperationCallable();
840+
}
841+
712842
@Override
713843
public final void close() {
714844
stub.close();

‎google-cloud-dataproc/src/main/java/com/google/cloud/dataproc/v1/JobControllerSettings.java

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,12 @@
2525
import com.google.api.gax.rpc.ApiClientHeaderProvider;
2626
import com.google.api.gax.rpc.ClientContext;
2727
import com.google.api.gax.rpc.ClientSettings;
28+
import com.google.api.gax.rpc.OperationCallSettings;
2829
import com.google.api.gax.rpc.PagedCallSettings;
2930
import com.google.api.gax.rpc.TransportChannelProvider;
3031
import com.google.api.gax.rpc.UnaryCallSettings;
3132
import com.google.cloud.dataproc.v1.stub.JobControllerStubSettings;
33+
import com.google.longrunning.Operation;
3234
import com.google.protobuf.Empty;
3335
import java.io.IOException;
3436
import java.util.List;
@@ -99,6 +101,19 @@ public UnaryCallSettings<DeleteJobRequest, Empty> deleteJobSettings() {
99101
return ((JobControllerStubSettings) getStubSettings()).deleteJobSettings();
100102
}
101103

104+
/** Returns the object with the settings used for calls to submitJobAsOperation. */
105+
public UnaryCallSettings<SubmitJobRequest, Operation> submitJobAsOperationSettings() {
106+
return ((JobControllerStubSettings) getStubSettings()).submitJobAsOperationSettings();
107+
}
108+
109+
/** Returns the object with the settings used for calls to submitJobAsOperation. */
110+
@BetaApi(
111+
"The surface for long-running operations is not stable yet and may change in the future.")
112+
public OperationCallSettings<SubmitJobRequest, Job, JobMetadata>
113+
submitJobAsOperationOperationSettings() {
114+
return ((JobControllerStubSettings) getStubSettings()).submitJobAsOperationOperationSettings();
115+
}
116+
102117
public static final JobControllerSettings create(JobControllerStubSettings stub)
103118
throws IOException {
104119
return new JobControllerSettings.Builder(stub.toBuilder()).build();
@@ -227,6 +242,19 @@ public UnaryCallSettings.Builder<DeleteJobRequest, Empty> deleteJobSettings() {
227242
return getStubSettingsBuilder().deleteJobSettings();
228243
}
229244

245+
/** Returns the builder for the settings used for calls to submitJobAsOperation. */
246+
public UnaryCallSettings.Builder<SubmitJobRequest, Operation> submitJobAsOperationSettings() {
247+
return getStubSettingsBuilder().submitJobAsOperationSettings();
248+
}
249+
250+
/** Returns the builder for the settings used for calls to submitJobAsOperation. */
251+
@BetaApi(
252+
"The surface for long-running operations is not stable yet and may change in the future.")
253+
public OperationCallSettings.Builder<SubmitJobRequest, Job, JobMetadata>
254+
submitJobAsOperationOperationSettings() {
255+
return getStubSettingsBuilder().submitJobAsOperationOperationSettings();
256+
}
257+
230258
@Override
231259
public JobControllerSettings build() throws IOException {
232260
return new JobControllerSettings(this);

0 commit comments

Comments
 (0)