-
Notifications
You must be signed in to change notification settings - Fork 84
Comparing changes
Open a pull request
base repository: googleapis/java-storage
base: v2.24.0
head repository: googleapis/java-storage
compare: v2.25.0
- 20 commits
- 190 files changed
- 7 contributors
Commits on Jul 13, 2023
-
chore(main): release 2.24.1-SNAPSHOT (#2113)
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Configuration menu - View commit details
-
Copy full SHA for d54b9cd - Browse repository at this point
Copy the full SHA d54b9cdView commit details -
feat: update Storage.createFrom(BlobInfo, Path) to have 150% higher t…
…hroughput (#2059) When uploading a file where we are able to rewind to an arbitrary offset, we can be more optimistic in the way we send requests to GCS. Add new code middleware to allow PUTing an entire file to GCS in a single request, and using query resumable session to recover from the specific offset in the case of retryable error. ### Benchmark Results #### Methodology Generate a random file on disk of size `128KiB..2GiB` from `/dev/urandom`, then upload the generated file using `Storage.createFrom(BlobInfo, Path)`. Perform each 4096 times. Run on a c2-standard-60 instance is us-central1 against a regional bucket located in us-central1. #### Results The following summary of throughput in MiB/s as observed between the existing implementation, and the new implementation proposed in this PR. ``` count mean std min 50% 75% 90% 99% max runId ApiName createFrom - existing JSON 4096.0 66.754 10.988 3.249 67.317 73.476 78.961 91.197 107.247 createFrom - new JSON 4096.0 158.769 67.105 4.600 170.680 218.618 240.992 266.297 305.205 ``` #### Comparison When comparing the new implementation to the existing implementation we get the following improvement to throughput (higher is better): ``` stat pct mean 137.841 50% 153.547 90% 205.204 99% 192.003 ```
Configuration menu - View commit details
-
Copy full SHA for 4c2f44e - Browse repository at this point
Copy the full SHA 4c2f44eView commit details -
test(deps): update dependency com.google.api.grpc:proto-google-cloud-…
…kms-v1 to v0.115.0 (#2116) * test(deps): update dependency com.google.api.grpc:proto-google-cloud-kms-v1 to v0.115.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 29feeaf - Browse repository at this point
Copy the full SHA 29feeafView commit details -
feat: BlobWriteChannelV2 - same throughput less GC (#2110)
Use stable buffer allocation with laziness. Leverage new JsonResumableSession to provide more robustness and easier separation of concerns compared to BlobWriteChannel * rename blobWriteChannel.ser.properties to the correct blobReadChannel.ser.properties ### Runtime improvments Throughput is on par with the existing v1 implementation, however GC impact has been lightened with the new implementation. Below is the summary of the GC improvement between v1 and v2. These GC numbers were collected while uploading 4096 randomly sized objects, from 128KiB..2GiB across 16 concurrent threads, using a default chunkSize of 16MiB. | metric | unit | v1 | v2 | % decrease | |---------------------------------|--------|-------------:|-------------:|-----------:| | gc.alloc.rate | MB/sec | 2240.056 | 1457.731 | 34.924 | | gc.alloc.rate.norm | B/op | 955796726217 | 638403730507 | 33.207 | | gc.churn.G1_Eden_Space | MB/sec | 1597.009 | 1454.304 | 8.936 | | gc.churn.G1_Eden_Space.norm | B/op | 681418424320 | 636902965248 | 6.533 | | gc.churn.G1_Old_Gen | MB/sec | 691.877 | 11.316 | 98.364 | | gc.churn.G1_Old_Gen.norm | B/op | 295213237398 | 4955944331 | 98.321 | | gc.churn.G1_Survivor_Space | MB/sec | 0.004 | 0.002 | 50.000 | | gc.churn.G1_Survivor_Space.norm | B/op | 1572864 | 786432 | 50.000 | | gc.count | counts | 1670 | 1319 | 21.018 | | gc.time | ms | 15936 | 9527 | 40.217 | Overall allocation rate is decreased, while Old_Gen use is almost entirely eliminated. ``` openjdk version "11.0.18" 2023-01-17 OpenJDK Runtime Environment (build 11.0.18+10-post-Debian-1deb11u1) OpenJDK 64-Bit Server VM (build 11.0.18+10-post-Debian-1deb11u1, mixed mode, sharing) -Xms12g -Xmx12g ``` All other java parameters are defaults.
Configuration menu - View commit details
-
Copy full SHA for 1b52a10 - Browse repository at this point
Copy the full SHA 1b52a10View commit details
Commits on Jul 14, 2023
-
Configuration menu - View commit details
-
Copy full SHA for a1a09f6 - Browse repository at this point
Copy the full SHA a1a09f6View commit details -
Configuration menu - View commit details
-
Copy full SHA for ccd19e3 - Browse repository at this point
Copy the full SHA ccd19e3View commit details -
chore(deps): update dependency com.google.cloud:google-cloud-storage …
…to v2.24.0 (#2115) * chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.24.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 889e433 - Browse repository at this point
Copy the full SHA 889e433View commit details -
fix: update GrpcStorageImpl.createFrom(BlobInfo, Path) to use Rewinda…
…bleContent (#2112) With the introduction of RewindableContent, we can now upload an entire file in a single WriteObjectRequest stream. * chore: rename RewindableHttpContent -> RewindableContent
Configuration menu - View commit details
-
Copy full SHA for c805051 - Browse repository at this point
Copy the full SHA c805051View commit details
Commits on Jul 18, 2023
-
fix: update BlobWriteChannelV2 to properly carry forward offset after…
… incremental flush (#2125) The tests in ITObjectTest did not perform a flush before capturing the write channel and this flow was not previously validated.
Configuration menu - View commit details
-
Copy full SHA for c099a2f - Browse repository at this point
Copy the full SHA c099a2fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 31ecab7 - Browse repository at this point
Copy the full SHA 31ecab7View commit details
Commits on Jul 19, 2023
-
build(deps): update dependency com.google.cloud:google-cloud-shared-c…
…onfig to v1.5.7 (#2128) * build(deps): update dependency com.google.cloud:google-cloud-shared-config to v1.5.7 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for de80b66 - Browse repository at this point
Copy the full SHA de80b66View commit details
Commits on Jul 20, 2023
-
Configuration menu - View commit details
-
Copy full SHA for fe9662d - Browse repository at this point
Copy the full SHA fe9662dView commit details -
chore(deps): update dependency com.google.cloud:libraries-bom to v26.…
…19.0 (#2127) * chore(deps): update dependency com.google.cloud:libraries-bom to v26.19.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for e16595e - Browse repository at this point
Copy the full SHA e16595eView commit details -
deps(test): update gcr.io/cloud-devrel-public-resources/storage-testb…
…ench docker tag to v0.37.0 (#2130) * test: re-enable tests Fixes #2114 --------- Co-authored-by: BenWhitehead <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9e8b6d3 - Browse repository at this point
Copy the full SHA 9e8b6d3View commit details -
deps: update dependency com.google.cloud:google-cloud-shared-dependen…
…cies to v3.13.1 (#2129)
Configuration menu - View commit details
-
Copy full SHA for a7e854e - Browse repository at this point
Copy the full SHA a7e854eView commit details
Commits on Jul 21, 2023
-
chore: Update the Java code generator (gapic-generator-java) to 2.23.1 (
#2131) * chore: Update the Java code generator (gapic-generator-java) to 2.23.1 PiperOrigin-RevId: 549674836 Source-Link: googleapis/googleapis@3b7a121 Source-Link: https://github.com/googleapis/googleapis-gen/commit/2d643a4eff8c8ef4c9d5a2e59d2a0ca7c991eab6 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMmQ2NDNhNGVmZjhjOGVmNGM5ZDVhMmU1OWQyYTBjYTdjOTkxZWFiNiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 9483222 - Browse repository at this point
Copy the full SHA 9483222View commit details
Commits on Jul 24, 2023
-
chore: update requirements.txt to address dependabot security alerts (#…
…1829) (#2134) * chore: update dependencies in requirements.txt to address dependabot security alerts * update dependencies within gcp/templates --------- Source-Link: googleapis/synthtool@af76aa5 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:46d2d262cd285c638656c8bde468011b723dc0c7ffd6a5ecc2650fe639c82e8f Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for b6efb50 - Browse repository at this point
Copy the full SHA b6efb50View commit details -
Configuration menu - View commit details
-
Copy full SHA for f5477e4 - Browse repository at this point
Copy the full SHA f5477e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3d22686 - Browse repository at this point
Copy the full SHA 3d22686View commit details -
chore(main): release 2.25.0 (#2117)
* chore(main): release 2.25.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for aebe26f - Browse repository at this point
Copy the full SHA aebe26fView commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v2.24.0...v2.25.0