Skip to content

feat(bigframes): Transpiler supports more string ops#17693

Draft
TrevorBergeron wants to merge 2 commits into
mainfrom
tbergeron_py_udf_fstrings
Draft

feat(bigframes): Transpiler supports more string ops#17693
TrevorBergeron wants to merge 2 commits into
mainfrom
tbergeron_py_udf_fstrings

Conversation

@TrevorBergeron

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for compiling f-strings, nullity checks (such as 'is None' and 'is not None'), and various string operations (e.g., capitalize, islower, isupper) in Python UDFs. Feedback is provided regarding the Polars compilation of 'IsLowerOp' and 'IsUpperOp', where the current regex patterns incorrectly require the entire string to consist only of letters, failing to match Python's native behavior which allows non-cased characters.

Comment on lines +495 to +503
@compile_op.register(string_ops.IsLowerOp)
def _(self, op: ops.ScalarOp, input: pl.Expr) -> pl.Expr:
assert isinstance(op, string_ops.IsLowerOp)
return input.str.contains(r"^[a-z]+$")

@compile_op.register(string_ops.IsUpperOp)
def _(self, op: ops.ScalarOp, input: pl.Expr) -> pl.Expr:
assert isinstance(op, string_ops.IsUpperOp)
return input.str.contains(r"^[A-Z]+$")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In Python, str.islower() and str.isupper() return True if there is at least one cased character and all cased characters are lowercase/uppercase respectively. Non-cased characters (such as spaces, punctuation, and digits) are allowed.

The current regex patterns ^[a-z]+$ and ^[A-Z]+$ require the entire string to consist only of letters of that case, which incorrectly returns False for valid Python strings like "hello world", "hello!", or "abc123".

To match Python's behavior, use regex lookaheads to ensure there is at least one cased character of the correct case, and no cased characters of the opposite case.

Suggested change
@compile_op.register(string_ops.IsLowerOp)
def _(self, op: ops.ScalarOp, input: pl.Expr) -> pl.Expr:
assert isinstance(op, string_ops.IsLowerOp)
return input.str.contains(r"^[a-z]+$")
@compile_op.register(string_ops.IsUpperOp)
def _(self, op: ops.ScalarOp, input: pl.Expr) -> pl.Expr:
assert isinstance(op, string_ops.IsUpperOp)
return input.str.contains(r"^[A-Z]+$")
@compile_op.register(string_ops.IsLowerOp)
def _(self, op: ops.ScalarOp, input: pl.Expr) -> pl.Expr:
assert isinstance(op, string_ops.IsLowerOp)
return input.str.contains(r"^(?=[^a-z]*[a-z])[^A-Z]+$")
@compile_op.register(string_ops.IsUpperOp)
def _(self, op: ops.ScalarOp, input: pl.Expr) -> pl.Expr:
assert isinstance(op, string_ops.IsUpperOp)
return input.str.contains(r"^(?=[^A-Z]*[A-Z])[^a-z]+$")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant