Skip to content

feat: add DataFrame.to_arrow to create Arrow Table from DataFrame #807

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 27, 2024

Conversation

tswast
Copy link
Collaborator

@tswast tswast commented Jun 26, 2024

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Towards #735 🦕

@tswast tswast requested review from a team as code owners June 26, 2024 21:18
Copy link

snippet-bot bot commented Jun 26, 2024

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@product-auto-label product-auto-label bot added size: l Pull request size is large. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. samples Issues that are directly related to samples. labels Jun 26, 2024
@tswast tswast mentioned this pull request Jun 26, 2024
@tswast tswast requested review from chelsea-lin and removed request for milkshakeiii June 27, 2024 15:41
@tswast tswast assigned chelsea-lin and unassigned junyazhang Jun 27, 2024
Copy link
Contributor

@chelsea-lin chelsea-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -0,0 +1,292 @@
# Copyright 2019 Google LLC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file auto generated from a template? If not, can we update the copyright year to 2024? Also check other new files too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is templated, so not much we can do.

polars_df = polars.from_arrow(arrow_table)
# [END bigquery_dataframes_to_polars]

assert polars_df is not None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we assert polars_df is equal to bf_df? Or maybe there are some diverse?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could possibly compare shapes or some aggregate values. I'll give it a try.

@tswast tswast enabled auto-merge (squash) June 27, 2024 22:08
@tswast tswast merged commit 1e3feda into main Jun 27, 2024
23 checks passed
@tswast tswast deleted the issue735-to_arrow-for-polars branch June 27, 2024 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. samples Issues that are directly related to samples. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants