Skip to content

feat: Support for Directed Reads #1966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
**/node_modules
/.coverage
/coverage
/.idea
/.vscode
/.nyc_output
/docs/
/out/
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-spanner/tre
| Updates the default leader of an existing database | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/database-update-default-leader.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/database-update-default-leader.js,samples/README.md) |
| Updates a Cloud Spanner Database. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/database-update.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/database-update.js,samples/README.md) |
| Datatypes | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/datatypes.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/datatypes.js,samples/README.md) |
| Runs an execute sql request with directed read options | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/directed-reads.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/directed-reads.js,samples/README.md) |
| Delete using DML returning. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-delete.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/dml-returning-delete.js,samples/README.md) |
| Insert using DML returning. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-insert.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/dml-returning-insert.js,samples/README.md) |
| Update using DML returning. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-update.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/dml-returning-update.js,samples/README.md) |
Expand Down
2 changes: 1 addition & 1 deletion protos/protos.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion protos/protos.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ and automatic, synchronous replication for high availability.
* [Updates the default leader of an existing database](#updates-the-default-leader-of-an-existing-database)
* [Updates a Cloud Spanner Database.](#updates-a-cloud-spanner-database.)
* [Datatypes](#datatypes)
* [Runs an execute sql request with directed read options](#runs-an-execute-sql-request-with-directed-read-options)
* [Delete using DML returning.](#delete-using-dml-returning.)
* [Insert using DML returning.](#insert-using-dml-returning.)
* [Update using DML returning.](#update-using-dml-returning.)
Expand Down Expand Up @@ -517,6 +518,23 @@ __Usage:__



### Runs an execute sql request with directed read options

View the [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/directed-reads.js).

[![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/directed-reads.js,samples/README.md)

__Usage:__


`node directed-reads.js <INSTANCE_ID> <DATABASE_ID> <PROJECT_ID>`


-----




### Delete using DML returning.

View the [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-delete.js).
Expand Down
117 changes: 117 additions & 0 deletions samples/directed-reads.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// sample-metadata:
// title: Runs an execute sql request with directed read options
// usage: node directed-reads.js <INSTANCE_ID> <DATABASE_ID> <PROJECT_ID>

'use strict';

function main(
instanceId = 'my-instance',
databaseId = 'my-database',
projectId = 'my-project-id'
) {
// [START spanner_directed_read]
// Imports the Google Cloud Spanner client library
const {Spanner, protos} = require('@google-cloud/spanner');

// Only one of excludeReplicas or includeReplicas can be set
// Each accepts a list of replicaSelections which contains location and type
// * `location` - The location must be one of the regions within the
// multi-region configuration of your database.
// * `type` - The type of the replica
// Some examples of using replicaSelectors are:
// * `location:us-east1` --> The "us-east1" replica(s) of any available type
// will be used to process the request.
// * `type:READ_ONLY` --> The "READ_ONLY" type replica(s) in nearest
//. available location will be used to process the
// request.
// * `location:us-east1 type:READ_ONLY` --> The "READ_ONLY" type replica(s)
// in location "us-east1" will be used to process
// the request.
// includeReplicas also contains an option for autoFailover which when set
// Spanner will not route requests to a replica outside the
// includeReplicas list when all the specified replicas are unavailable
// or unhealthy. The default value is `false`
const directedReadOptionsForClient = {
excludeReplicas: {
replicaSelections: [
{
location: 'us-east4',
},
],
},
};

// Instantiates a client with directedReadOptions
const spanner = new Spanner({
projectId: projectId,
directedReadOptions: directedReadOptionsForClient,
});

async function spannerDirectedReads() {
// Gets a reference to a Cloud Spanner instance and backup
const instance = spanner.instance(instanceId);
const database = instance.database(databaseId);
const directedReadOptionsForRequest = {
includeReplicas: {
replicaSelections: [
{
type: protos.google.spanner.v1.DirectedReadOptions.ReplicaSelection
.Type.READ_ONLY,
},
],
autoFailoverDisabled: true,
},
};

await database.getSnapshot(async (err, transaction) => {
if (err) {
console.error(err);
return;
}
try {
// Read rows while passing directedReadOptions directly to the query.
// These will override the options passed at Client level.
const [rows] = await transaction.run({
sql: 'SELECT SingerId, AlbumId, AlbumTitle FROM Albums',
directedReadOptions: directedReadOptionsForRequest,
});
rows.forEach(row => {
const json = row.toJSON();
console.log(
`SingerId: ${json.SingerId}, AlbumId: ${json.AlbumId}, AlbumTitle: ${json.AlbumTitle}`
);
});
console.log(
'Successfully executed read-only transaction with directedReadOptions'
);
} catch (err) {
console.error('ERROR:', err);
} finally {
transaction.end();
// Close the database when finished.
await database.close();
}
});
}
spannerDirectedReads();
// [END spanner_directed_read]
}
process.on('unhandledRejection', err => {
console.error(err.message);
process.exitCode = 1;
});
main(...process.argv.slice(2));
20 changes: 20 additions & 0 deletions samples/system-test/spanner.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -1985,5 +1985,25 @@ describe('Spanner', () => {
)
);
});

// directed_read_options
it('should run read-only transaction with directed read options set', async () => {
const output = execSync(
`node directed-reads.js ${INSTANCE_ID} ${DATABASE_ID} ${PROJECT_ID}`
);
console.log(output);
assert.match(
output,
new RegExp(
'SingerId: 2, AlbumId: 2, AlbumTitle: Forever Hold your Peace'
)
);
assert.match(
output,
new RegExp(
'Successfully executed read-only transaction with directedReadOptions'
)
);
});
});
});
2 changes: 1 addition & 1 deletion src/batch-transaction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import * as extend from 'extend';
import * as is from 'is';
import {Snapshot} from './transaction';
import {google} from '../protos/protos';
import {Session, Database, Spanner} from '.';
import {Session, Database} from '.';
import {
CLOUD_RESOURCE_HEADER,
addLeaderAwareRoutingHeader,
Expand Down
12 changes: 12 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,13 +110,17 @@ export type GetInstanceConfigOperationsCallback = PagedCallback<
* Session pool configuration options.
* @property {boolean} [routeToLeaderEnabled=True] If set to false leader aware routing will be disabled.
* Disabling leader aware routing would route all requests in RW/PDML transactions to any region.
* @property {google.spanner.v1.IDirectedReadOptions} [directedReadOptions] Sets the DirectedReadOptions for all ReadRequests and ExecuteSqlRequests for the Client.
* Indicates which replicas or regions should be used for non-transactional reads or queries.
* DirectedReadOptions won't be set for readWrite transactions"
*/
export interface SpannerOptions extends GrpcClientOptions {
apiEndpoint?: string;
servicePath?: string;
port?: number;
sslCreds?: grpc.ChannelCredentials;
routeToLeaderEnabled?: boolean;
directedReadOptions?: google.spanner.v1.IDirectedReadOptions | null;
}
export interface RequestConfig {
client: string;
Expand Down Expand Up @@ -217,6 +221,7 @@ class Spanner extends GrpcService {
projectFormattedName_: string;
resourceHeader_: {[k: string]: string};
routeToLeaderEnabled = true;
directedReadOptions: google.spanner.v1.IDirectedReadOptions | null;

/**
* Placeholder used to auto populate a column with the commit timestamp.
Expand Down Expand Up @@ -291,6 +296,12 @@ class Spanner extends GrpcService {
},
options || {}
) as {} as SpannerOptions;

const directedReadOptions = options.directedReadOptions
? options.directedReadOptions
: null;
delete options.directedReadOptions;

const emulatorHost = Spanner.getSpannerEmulatorHost();
if (
emulatorHost &&
Expand Down Expand Up @@ -332,6 +343,7 @@ class Spanner extends GrpcService {
this.resourceHeader_ = {
[CLOUD_RESOURCE_HEADER]: this.projectFormattedName_,
};
this.directedReadOptions = directedReadOptions;
}

/** Closes this Spanner client and cleans up all resources used by it. */
Expand Down
2 changes: 2 additions & 0 deletions src/session-pool.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1081,7 +1081,9 @@ export class SessionPool extends EventEmitter implements SessionPoolInterface {
* @private
*/
_stopHouseKeeping(): void {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
clearInterval(this._pingHandle as any);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
clearInterval(this._evictHandle as any);
}
}
40 changes: 40 additions & 0 deletions src/transaction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ export interface ExecuteSqlRequest extends Statement, RequestOptions {
queryOptions?: IQueryOptions;
requestOptions?: Omit<IRequestOptions, 'transactionTag'>;
dataBoostEnabled?: boolean | null;
directedReadOptions?: google.spanner.v1.IDirectedReadOptions;
}

export interface KeyRange {
Expand All @@ -107,6 +108,7 @@ export interface ReadRequest extends RequestOptions {
partitionToken?: Uint8Array | null;
requestOptions?: Omit<IRequestOptions, 'transactionTag'>;
dataBoostEnabled?: boolean | null;
directedReadOptions?: google.spanner.v1.IDirectedReadOptions;
}

export interface BatchUpdateError extends grpc.ServiceError {
Expand Down Expand Up @@ -457,6 +459,8 @@ export class Snapshot extends EventEmitter {
* PartitionReadRequest message used to create this partition_token.
* @property {google.spanner.v1.RequestOptions} [requestOptions]
* Common options for this request.
* @property {google.spanner.v1.IDirectedReadOptions} [directedReadOptions]
* Indicates which replicas or regions should be used for non-transactional reads or queries.
* @property {object} [gaxOptions]
* Call options. See {@link https://googleapis.dev/nodejs/google-gax/latest/interfaces/CallOptions.html|CallOptions}
* for more details.
Expand Down Expand Up @@ -591,6 +595,10 @@ export class Snapshot extends EventEmitter {
transaction.singleUse = this._options;
}

const directedReadOptions = this._getDirectedReadOptions(
request.directedReadOptions
);

request = Object.assign({}, request);

delete request.gaxOptions;
Expand All @@ -600,6 +608,7 @@ export class Snapshot extends EventEmitter {
delete request.keys;
delete request.ranges;
delete request.requestOptions;
delete request.directedReadOptions;

const reqOpts: spannerClient.spanner.v1.IReadRequest = Object.assign(
request,
Expand All @@ -610,6 +619,7 @@ export class Snapshot extends EventEmitter {
this.requestOptions?.transactionTag ?? undefined,
requestOptions
),
directedReadOptions: directedReadOptions,
transaction,
table,
keySet,
Expand Down Expand Up @@ -993,6 +1003,8 @@ export class Snapshot extends EventEmitter {
* that it is not ready for any more data. Increase this value if you
* experience 'Stream is still not ready to receive data' errors as a
* result of a slow writer in your receiving stream.
* @property {object} [directedReadOptions]
* Indicates which replicas or regions should be used for non-transactional reads or queries.
*/
/**
* Create a readable object stream to receive resulting rows from a SQL
Expand Down Expand Up @@ -1068,6 +1080,10 @@ export class Snapshot extends EventEmitter {
query;
let reqOpts;

const directedReadOptions = this._getDirectedReadOptions(
query.directedReadOptions
);

const sanitizeRequest = () => {
query = query as ExecuteSqlRequest;
const {params, paramTypes} = Snapshot.encodeParams(query);
Expand All @@ -1085,6 +1101,7 @@ export class Snapshot extends EventEmitter {
delete query.maxResumeRetries;
delete query.requestOptions;
delete query.types;
delete query.directedReadOptions;

reqOpts = Object.assign(query, {
session: this.session.formattedName_!,
Expand All @@ -1094,6 +1111,7 @@ export class Snapshot extends EventEmitter {
this.requestOptions?.transactionTag ?? undefined,
requestOptions
),
directedReadOptions: directedReadOptions,
transaction,
params,
paramTypes,
Expand Down Expand Up @@ -1288,6 +1306,28 @@ export class Snapshot extends EventEmitter {
return {params, paramTypes};
}

/**
* Get directed read options
* @private
* @param {google.spanner.v1.IDirectedReadOptions} directedReadOptions Request directedReadOptions object.
*/
protected _getDirectedReadOptions(
directedReadOptions:
| google.spanner.v1.IDirectedReadOptions
| null
| undefined
) {
if (
!directedReadOptions &&
this._getSpanner().directedReadOptions &&
this._options.readOnly
) {
return this._getSpanner().directedReadOptions;
}

return directedReadOptions;
}

/**
* Update transaction properties from the response.
*
Expand Down
Loading