Skip to content

feat: Retry by default for uploads, blob deletes, metadata updates #1400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 24, 2025

Conversation

andrewsg
Copy link
Contributor

No description provided.

@andrewsg andrewsg requested review from a team as code owners December 21, 2024 02:02
@product-auto-label product-auto-label bot added the size: l Pull request size is large. label Dec 21, 2024
@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/python-storage API. label Dec 21, 2024
Copy link
Contributor

@cojenco cojenco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks for enabling retry by default woohoo

@andrewsg
Copy link
Contributor Author

PTAL

Copy link
Contributor

@cojenco cojenco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@andrewsg andrewsg merged commit 0426005 into main Jan 24, 2025
13 checks passed
@andrewsg andrewsg deleted the retry-by-default branch January 24, 2025 18:28
@ishowta
Copy link

ishowta commented Feb 6, 2025

@andrewsg I've noticed that other SDKs (at least Node.js and Go) seem to apply retries only to idempotent operations, as was previously the case in Python. If you don't mind, could you share a bit of insight into the rationale behind this change?

https://github.com/googleapis/nodejs-storage/blob/75c309c0761e4029dcd13024e748d8957052f766/src/storage.ts#L293

https://github.com/googleapis/google-cloud-go/blob/b1e3ce67d97275c7883e2b23617d070981d46bf7/storage/storage.go#L2355

@andrewsg
Copy link
Contributor Author

andrewsg commented Feb 6, 2025

@ishowta The C++ library currently has enabled retries by default on these operations. We are changing our policy around these specific operations to match the C++ library's precedent, starting with Python, but assuming there are no issues, later to include the other languages. We've determined that the potential negative effect of a double request for these listed operations is practically nil and the actual negative effect of having retries off by default is very large, as most people don't use generations or if_generation_match. However, if you do experience an unexpected issue with this change, please do let us know about it.

@ishowta
Copy link

ishowta commented Feb 6, 2025

Thank you for the explanation! In some of our projects, we've been configuring the Cloud Storage SDK to always retry, so I'll use this as a reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/python-storage API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants