Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: googleapis/java-storage
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v2.24.0
Choose a base ref
...
head repository: googleapis/java-storage
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v2.25.0
Choose a head ref
  • 20 commits
  • 190 files changed
  • 7 contributors

Commits on Jul 13, 2023

  1. chore(main): release 2.24.1-SNAPSHOT (#2113)

    🤖 I have created a release *beep* *boop*
    ---
    
    
    ### Updating meta-information for bleeding-edge SNAPSHOT release.
    
    ---
    This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
    release-please[bot] authored Jul 13, 2023
    Configuration menu
    Copy the full SHA
    d54b9cd View commit details
    Browse the repository at this point in the history
  2. feat: update Storage.createFrom(BlobInfo, Path) to have 150% higher t…

    …hroughput (#2059)
    
    When uploading a file where we are able to rewind to an arbitrary offset, we can be more optimistic in the way we send requests to GCS.
    
    Add new code middleware to allow PUTing an entire file to GCS in a single request, and using query resumable session to recover from the specific offset in the case of retryable error.
    
    
    ### Benchmark Results
    
    #### Methodology
    
    Generate a random file on disk of size `128KiB..2GiB` from `/dev/urandom`, then upload the generated file using `Storage.createFrom(BlobInfo, Path)`.
    
    Perform each 4096 times.
    
    Run on a c2-standard-60 instance is us-central1 against a regional bucket located in us-central1.
    
    #### Results
    
    The following summary of throughput in MiB/s as observed between the existing implementation, and the new implementation proposed in this PR.
    
    ```
                                    count     mean     std    min      50%      75%      90%      99%      max
    runId                 ApiName                                                                             
    createFrom - existing JSON     4096.0   66.754  10.988  3.249   67.317   73.476   78.961   91.197  107.247
    createFrom - new      JSON     4096.0  158.769  67.105  4.600  170.680  218.618  240.992  266.297  305.205
    ```
    
    #### Comparison
    
    When comparing the new implementation to the existing implementation we get the following improvement to throughput (higher is better):
    ```
    stat      pct
    mean  137.841
     50%  153.547
     90%  205.204
     99%  192.003
    ```
    BenWhitehead authored Jul 13, 2023
    Configuration menu
    Copy the full SHA
    4c2f44e View commit details
    Browse the repository at this point in the history
  3. test(deps): update dependency com.google.api.grpc:proto-google-cloud-…

    …kms-v1 to v0.115.0 (#2116)
    
    * test(deps): update dependency com.google.api.grpc:proto-google-cloud-kms-v1 to v0.115.0
    
    * 🦉 Updates from OwlBot post-processor
    
    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    
    ---------
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    renovate-bot and gcf-owl-bot[bot] authored Jul 13, 2023
    Configuration menu
    Copy the full SHA
    29feeaf View commit details
    Browse the repository at this point in the history
  4. feat: BlobWriteChannelV2 - same throughput less GC (#2110)

    Use stable buffer allocation with laziness.
    
    Leverage new JsonResumableSession to provide more robustness and easier
    separation of concerns compared to BlobWriteChannel
    
    * rename blobWriteChannel.ser.properties to the correct blobReadChannel.ser.properties
    
    ### Runtime improvments
    
    Throughput is on par with the existing v1 implementation, however GC
    impact has been lightened with the new implementation.
    
    Below is the summary of the GC improvement between v1 and v2.
    
    These GC numbers were collected while uploading 4096 randomly sized
    objects, from 128KiB..2GiB across 16 concurrent threads, using a default
    chunkSize of 16MiB.
    
    | metric                          | unit   |           v1 |           v2 | % decrease |
    |---------------------------------|--------|-------------:|-------------:|-----------:|
    | gc.alloc.rate                   | MB/sec |     2240.056 |     1457.731 |     34.924 |
    | gc.alloc.rate.norm              | B/op   | 955796726217 | 638403730507 |     33.207 |
    | gc.churn.G1_Eden_Space          | MB/sec |     1597.009 |     1454.304 |      8.936 |
    | gc.churn.G1_Eden_Space.norm     | B/op   | 681418424320 | 636902965248 |      6.533 |
    | gc.churn.G1_Old_Gen             | MB/sec |      691.877 |       11.316 |     98.364 |
    | gc.churn.G1_Old_Gen.norm        | B/op   | 295213237398 |   4955944331 |     98.321 |
    | gc.churn.G1_Survivor_Space      | MB/sec |        0.004 |        0.002 |     50.000 |
    | gc.churn.G1_Survivor_Space.norm | B/op   |      1572864 |       786432 |     50.000 |
    | gc.count                        | counts |         1670 |         1319 |     21.018 |
    | gc.time                         | ms     |        15936 |         9527 |     40.217 |
    
    Overall allocation rate is decreased, while Old_Gen use is almost entirely eliminated.
    
    ```
    openjdk version "11.0.18" 2023-01-17
    OpenJDK Runtime Environment (build 11.0.18+10-post-Debian-1deb11u1)
    OpenJDK 64-Bit Server VM (build 11.0.18+10-post-Debian-1deb11u1, mixed
    mode, sharing)
    
    -Xms12g -Xmx12g
    ```
    
    All other java parameters are defaults.
    BenWhitehead authored Jul 13, 2023
    Configuration menu
    Copy the full SHA
    1b52a10 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2023

  1. Configuration menu
    Copy the full SHA
    a1a09f6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ccd19e3 View commit details
    Browse the repository at this point in the history
  3. chore(deps): update dependency com.google.cloud:google-cloud-storage …

    …to v2.24.0 (#2115)
    
    * chore(deps): update dependency com.google.cloud:google-cloud-storage to v2.24.0
    
    * 🦉 Updates from OwlBot post-processor
    
    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    
    ---------
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    renovate-bot and gcf-owl-bot[bot] authored Jul 14, 2023
    Configuration menu
    Copy the full SHA
    889e433 View commit details
    Browse the repository at this point in the history
  4. fix: update GrpcStorageImpl.createFrom(BlobInfo, Path) to use Rewinda…

    …bleContent (#2112)
    
    With the introduction of RewindableContent, we can now upload an entire file in a single WriteObjectRequest stream.
    
    * chore: rename RewindableHttpContent -> RewindableContent
    BenWhitehead authored Jul 14, 2023
    Configuration menu
    Copy the full SHA
    c805051 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2023

  1. fix: update BlobWriteChannelV2 to properly carry forward offset after…

    … incremental flush (#2125)
    
    The tests in ITObjectTest did not perform a flush before capturing the write channel and this flow was not previously validated.
    BenWhitehead authored Jul 18, 2023
    Configuration menu
    Copy the full SHA
    c099a2f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    31ecab7 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. build(deps): update dependency com.google.cloud:google-cloud-shared-c…

    …onfig to v1.5.7 (#2128)
    
    * build(deps): update dependency com.google.cloud:google-cloud-shared-config to v1.5.7
    
    * 🦉 Updates from OwlBot post-processor
    
    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    
    ---------
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    renovate-bot and gcf-owl-bot[bot] authored Jul 19, 2023
    Configuration menu
    Copy the full SHA
    de80b66 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2023

  1. docs: fix broken link for TESTING.md (#2126)

    Fixes b/290932427
    BenWhitehead authored Jul 20, 2023
    Configuration menu
    Copy the full SHA
    fe9662d View commit details
    Browse the repository at this point in the history
  2. chore(deps): update dependency com.google.cloud:libraries-bom to v26.…

    …19.0 (#2127)
    
    * chore(deps): update dependency com.google.cloud:libraries-bom to v26.19.0
    
    * 🦉 Updates from OwlBot post-processor
    
    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    
    ---------
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    renovate-bot and gcf-owl-bot[bot] authored Jul 20, 2023
    Configuration menu
    Copy the full SHA
    e16595e View commit details
    Browse the repository at this point in the history
  3. deps(test): update gcr.io/cloud-devrel-public-resources/storage-testb…

    …ench docker tag to v0.37.0 (#2130)
    
    * test: re-enable tests
    
    Fixes #2114
    
    ---------
    
    Co-authored-by: BenWhitehead <[email protected]>
    renovate-bot and BenWhitehead authored Jul 20, 2023
    Configuration menu
    Copy the full SHA
    9e8b6d3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    a7e854e View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. chore: Update the Java code generator (gapic-generator-java) to 2.23.1 (

    #2131)
    
    * chore: Update the Java code generator (gapic-generator-java) to 2.23.1
    
    PiperOrigin-RevId: 549674836
    
    Source-Link: googleapis/googleapis@3b7a121
    
    Source-Link: https://github.com/googleapis/googleapis-gen/commit/2d643a4eff8c8ef4c9d5a2e59d2a0ca7c991eab6
    Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMmQ2NDNhNGVmZjhjOGVmNGM5ZDVhMmU1OWQyYTBjYTdjOTkxZWFiNiJ9
    
    * 🦉 Updates from OwlBot post-processor
    
    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    
    ---------
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    gcf-owl-bot[bot] and gcf-owl-bot[bot] authored Jul 21, 2023
    Configuration menu
    Copy the full SHA
    9483222 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. chore: update requirements.txt to address dependabot security alerts (#…

    …1829) (#2134)
    
    * chore: update dependencies in requirements.txt to address dependabot security alerts
    
    * update dependencies within gcp/templates
    
    ---------
    
    
    Source-Link: googleapis/synthtool@af76aa5
    Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:46d2d262cd285c638656c8bde468011b723dc0c7ffd6a5ecc2650fe639c82e8f
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    Co-authored-by: Anthonios Partheniou <[email protected]>
    3 people authored Jul 24, 2023
    Configuration menu
    Copy the full SHA
    b6efb50 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f5477e4 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3d22686 View commit details
    Browse the repository at this point in the history
  4. chore(main): release 2.25.0 (#2117)

    * chore(main): release 2.25.0
    
    * 🦉 Updates from OwlBot post-processor
    
    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    
    ---------
    
    Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    release-please[bot] and gcf-owl-bot[bot] authored Jul 24, 2023
    Configuration menu
    Copy the full SHA
    aebe26f View commit details
    Browse the repository at this point in the history
Loading