Skip to content

feat(spanner): add support for snapshot isolation #2245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 20, 2025

Conversation

alkatrivedi
Copy link
Contributor

@alkatrivedi alkatrivedi commented Mar 6, 2025

This PR contains code changes to add support for option IsolationLevel at the client level and at the transaction level.
supported methods are(RW and Blind Write):

- writeAtLeastOnce
- runTransactionAsync
- runTransaction
- getTransaction
- async getTransaction(from transaction runner class)

@alkatrivedi alkatrivedi requested review from a team as code owners March 6, 2025 13:51
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: spanner Issues related to the googleapis/nodejs-spanner API. labels Mar 6, 2025
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch 2 times, most recently from 7b2f36b to 854c1ee Compare March 7, 2025 05:24
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from 854c1ee to 0913aec Compare March 7, 2025 05:28
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from 0913aec to d574252 Compare March 7, 2025 05:30
@alkatrivedi alkatrivedi requested a review from surbhigarg92 March 7, 2025 06:22
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch 2 times, most recently from c73a9b9 to 1a2b964 Compare March 10, 2025 18:57
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Mar 10, 2025
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from 1a2b964 to 780461d Compare March 10, 2025 19:21
/**
* Use option isolationLevel to add the isolation level in the transaction.
*/
setIsolationLevel(isolationLevel: any): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use any here. Use protos.google.spanner.v1.TransactionOptions.IsolationLevel

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, thanks for the suggestion!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this method used anywhere now ?

src/database.ts Outdated
@@ -316,6 +316,10 @@ export interface RestoreOptions {
gaxOptions?: CallOptions;
}

export interface WriteAtLeastOnceOptions extends CallOptions {
defaultTransactionOptions?: Pick<RunTransactionOptions, 'isolationLevel'>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why didn't we have change_stream here, it creates a RW transaction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will add that option in a separate PR, it got skipped earlier

src/database.ts Outdated
Comment on lines 2238 to 2244
if (options.isolationLevel) {
transaction!.setIsolationLevel(options.isolationLevel);
} else if (defaultTransactionOptions) {
transaction!.setIsolationLevel(
defaultTransactionOptions.isolationLevel
);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code along with below code is getting repeated. Create a method in transaction.ts as setReadWriteTransactionOptions and move the code there

if (options.optimisticLock) {
          transaction!.useOptimisticLock();
        }
        if (options.excludeTxnFromChangeStreams) {
          transaction!.excludeTxnFromChangeStreams();
        }```

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified

src/table.ts Outdated
@@ -52,7 +54,10 @@ export type DropTableCallback = UpdateSchemaCallback;

interface MutateRowsOptions extends CommitOptions {
requestOptions?: Omit<IRequestOptions, 'requestTag'>;
excludeTxnFromChangeStreams?: boolean;
defaultTransactionOptions?: Pick<
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from 850b7ae to 780461d Compare March 12, 2025 12:53
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch 2 times, most recently from 5ee608e to 68edb46 Compare March 13, 2025 10:27
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from 68edb46 to 8633c2c Compare March 13, 2025 10:54
src/database.ts Outdated
transaction!.excludeTxnFromChangeStreams();
}
transaction?.setReadWriteTransactionOptions(
options && Object.keys(options).length
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would this handle the case, when TransactionOptions only has excludeTxnFromChangeStreams set and isolation_level needs to be taken from defaultTransactionOptions

@@ -26,6 +26,7 @@ import {isSessionNotFoundError} from './session-pool';
import {Database} from './database';
import {google} from '../protos/protos';
import IRequestOptions = google.spanner.v1.IRequestOptions;
import {protos} from '.';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import {google} from '../protos/protos' is already imported.

You could import IsolationLevel on top like IRequestOptions

import IsolationLevel = google.spanner.v1.TransactionOptions.IsolationLevel

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thanks for the suggestion!

@@ -46,6 +47,7 @@ export interface RunTransactionOptions {
requestOptions?: Pick<IRequestOptions, 'transactionTag'>;
optimisticLock?: boolean;
excludeTxnFromChangeStreams?: boolean;
isolationLevel?: protos.google.spanner.v1.TransactionOptions.IsolationLevel;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the IsolationLevel import instead of the full syntax.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

const defaults = {
timeout: 3600000,
isolationLevel:
protos.google.spanner.v1.TransactionOptions.IsolationLevel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change it here and all other places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

transaction!.setReadWriteTransactionOptions(
this.options && Object.keys(this.options).length
? this.options
: this.session.parent._getSpanner().defaultTransactionOptions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not use _getSpanner() private method . Instead create a private method _getSpanner in this class and use it. Refer this method in all other classes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have refactored the code, please check

@@ -1822,9 +1823,33 @@ export class Transaction extends Dml {

this._queuedMutations = [];
this._options = {readWrite: options};
this._options.isolationLevel =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly change the import and protos full path

/**
* Set isolation level .
*/
if (options?.isolationLevel) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how would this use the default isolation level if available ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the code
in the Transaction class constructor we are assigning the default isolationLevel value as this._options.isolationLevel = IsolationLevel.ISOLATION_LEVEL_UNSPECIFIED;

now here in this case this._options.isolationLevel = options?.isolationLevel? options?.isolationLevel:this._getSpanner().defaultTransactionOptions.isolationLevel;

if the isolationLevel will get pass at the transaction level it will assign that value to the this._options.isolationLevel otherwise this value this._getSpanner().defaultTransactionOptions.isolationLevel will get assign which if being passed from spanner options will use that value otherwise it will also be containing default value of unspecified

/**
* Use option isolationLevel to add the isolation level in the transaction.
*/
setIsolationLevel(isolationLevel: any): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this method used anywhere now ?

test/database.ts Outdated
assert.strictEqual(options, fakeOptions);
});

it('should optionally accept `option` isolationLevel when passed with Spanner options', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add test cases for below

  1. Should override isolationLevel from runTransactionOptions, when passed in both defaultTransactionOptions and runTransactionOptions

  2. Should use isolation level from defaultTransactionOptions when runTransactionOptions are passed but does not consists of isolation level and contains other properties. The resulting options should be a combination of defaultTransactionOptions and runTransactionOptions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If defaultTransactionOptions has more properties except, isolation level, then other properties should be ignored.

Copy link
Contributor Author

@alkatrivedi alkatrivedi Mar 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have added the below tests in spanner.ts

  1. 'should be able to use isolationLevel from Spanner Option when other options are passed at transaction level'
  2. 'should override isolationLevel from Spanner Option when passed at transaction level'

@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from 56cfbe1 to b628ba6 Compare March 19, 2025 08:36
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch 4 times, most recently from 8cd74dc to ae96078 Compare March 19, 2025 11:18
@alkatrivedi alkatrivedi added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Mar 19, 2025
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 19, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 19, 2025
@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from ae96078 to aaaf522 Compare March 19, 2025 11:25
src/database.ts Outdated
@@ -3683,6 +3674,7 @@ class Database extends common.GrpcServiceObject {
span.addEvent('Using Session', {'session.id': session?.id});
this._releaseOnEnd(session!, transaction!, span);
try {
transaction!.setReadWriteTransactionOptions(options);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
transaction!.setReadWriteTransactionOptions(options);
transaction!.setReadWriteTransactionOptions(options as RunTransactionOptions);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do this for all other places

@alkatrivedi alkatrivedi force-pushed the isolation-level-option branch from aaaf522 to 440f020 Compare March 20, 2025 09:17
@alkatrivedi alkatrivedi added kokoro:force-run Add this label to force Kokoro to re-run the tests. automerge Merge the pull request once unit tests and other checks pass. labels Mar 20, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 20, 2025
@gcf-merge-on-green gcf-merge-on-green bot merged commit b60a683 into main Mar 20, 2025
21 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 20, 2025
@gcf-merge-on-green gcf-merge-on-green bot deleted the isolation-level-option branch March 20, 2025 10:14
alkatrivedi added a commit that referenced this pull request Mar 20, 2025
This PR contains code changes to add support for option IsolationLevel at the client level and at the transaction level.
supported methods are(RW and Blind Write):

```
- writeAtLeastOnce
- runTransactionAsync
- runTransaction
- getTransaction
- async getTransaction(from transaction runner class)
```
alkatrivedi added a commit that referenced this pull request Mar 27, 2025
* feat(spanner): support for Multiplexed Session Partitioned Ops

* feat(spanner): add support for snapshot isolation (#2245)

This PR contains code changes to add support for option IsolationLevel at the client level and at the transaction level.
supported methods are(RW and Blind Write):

```
- writeAtLeastOnce
- runTransactionAsync
- runTransaction
- getTransaction
- async getTransaction(from transaction runner class)
```

* refactor

* refactor
alkatrivedi added a commit that referenced this pull request Apr 14, 2025
* feat(spanner): support for Multiplexed Session Partitioned Ops

* feat(spanner): add support for snapshot isolation (#2245)

This PR contains code changes to add support for option IsolationLevel at the client level and at the transaction level.
supported methods are(RW and Blind Write):

```
- writeAtLeastOnce
- runTransactionAsync
- runTransaction
- getTransaction
- async getTransaction(from transaction runner class)
```

* refactor

* refactor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/nodejs-spanner API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants