Skip to content

chore(bigtable): use ruff for generated sync files #16713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
daniel-sanche wants to merge 5 commits into
base: main
Choose a base branch
from
Open

chore(bigtable): use ruff for generated sync files #16713

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
123dcfc
use ruff for cross_sync format
daniel-sanche Apr 17, 2026
f107b02
strip out unused imports
daniel-sanche Apr 17, 2026
a46a791
regenerated files
daniel-sanche Apr 17, 2026
15a38a6
update ruff execution
daniel-sanche Apr 17, 2026
42575bd
fixed unused imports
daniel-sanche Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 45 additions & 8 deletions packages/google-cloud-bigtable/.cross_sync/generate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,18 +51,55 @@ def render(self, with_formatter=True, save_to_disk: bool = True) -> str:
Render the file to a string, and optionally save to disk

Args:
with_formatter: whether to run the output through black before returning
with_formatter: whether to run the output through ruff before returning
save_to_disk: whether to write the output to the file path
"""
full_str = self.header + ast.unparse(self.tree)
if with_formatter:
import black # type: ignore
import autoflake # type: ignore

full_str = black.format_str(
autoflake.fix_code(full_str, remove_all_unused_imports=True),
mode=black.FileMode(),
)
import subprocess
import sys

try:
# Run ruff check
result = subprocess.run(
[
sys.executable,
"-m",
"ruff",
"check",
"--select",
"I,F401",
"--fix",
"--line-length=88",
"-",
],
input=full_str,
text=True,
capture_output=True,
check=True,
)
Comment on lines +64 to +80
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When running ruff check on code provided via stdin (-), it is highly recommended to provide a filename using --stdin-filename. This allows ruff to correctly identify the file's location, which is essential for determining if imports are first-party or third-party (the I rule) and for applying any path-specific configuration rules from pyproject.toml.

Suggested change
result = subprocess.run(
[
sys.executable,
"-m",
"ruff",
"check",
"--select",
"I,F401",
"--fix",
"--line-length=88",
"-",
],
input=full_str,
text=True,
capture_output=True,
check=True,
)
result = subprocess.run(
[
sys.executable,
"-m",
"ruff",
"check",
"--select",
"I,F401",
"--fix",
"--line-length=88",
"--stdin-filename",
self.output_path,
"-",
],
input=full_str,
text=True,
capture_output=True,
check=True,
)

full_str = result.stdout

# Run ruff format
result = subprocess.run(
[
sys.executable,
"-m",
"ruff",
"format",
"--line-length=88",
"--config",
"format.skip-magic-trailing-comma=true",
"-",
],
input=full_str,
text=True,
capture_output=True,
check=True,
)
Comment on lines +84 to +99
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the ruff check call, ruff format should also be provided with --stdin-filename to ensure it respects any file-specific formatting configurations defined in the repository's configuration files.

Suggested change
result = subprocess.run(
[
sys.executable,
"-m",
"ruff",
"format",
"--line-length=88",
"--config",
"format.skip-magic-trailing-comma=true",
"-",
],
input=full_str,
text=True,
capture_output=True,
check=True,
)
result = subprocess.run(
[
sys.executable,
"-m",
"ruff",
"format",
"--line-length=88",
"--config",
"format.skip-magic-trailing-comma=true",
"--stdin-filename",
self.output_path,
"-",
],
input=full_str,
text=True,
capture_output=True,
check=True,
)

full_str = result.stdout
except subprocess.CalledProcessError as e:
print(f"ruff formatting failed: {e.stderr}")
Comment on lines +101 to +102
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In a code generation script, it is generally better to raise an exception when formatting fails rather than just printing to stdout. This ensures that the failure is caught by CI/CD pipelines (e.g., via nox) and prevents the script from silently returning unformatted or partially formatted code. Additionally, including e.stdout in the error message can be helpful as ruff sometimes outputs error details there.

Suggested change
except subprocess.CalledProcessError as e:
print(f"ruff formatting failed: {e.stderr}")
except subprocess.CalledProcessError as e:
raise RuntimeError(
f"ruff formatting failed for {self.output_path}: {e.stderr or e.stdout}"
) from e

if save_to_disk:
import os
os.makedirs(os.path.dirname(self.output_path), exist_ok=True)
Expand Down
5 changes: 4 additions & 1 deletion packages/google-cloud-bigtable/google/cloud/bigtable/data/_sync_autogen/_mutate_rows.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,12 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

from typing import TYPE_CHECKING, Sequence

from google.api_core import exceptions as core_exceptions
from google.api_core import retry as retries

import google.cloud.bigtable.data.exceptions as bt_exceptions
import google.cloud.bigtable_v2.types.bigtable as types_pb
from google.cloud.bigtable.data._cross_sync import CrossSync
Expand All @@ -32,10 +35,10 @@
)

if TYPE_CHECKING:
from google.cloud.bigtable.data.mutations import RowMutationEntry
from google.cloud.bigtable.data._sync_autogen.client import (
_DataApiTarget as TargetType,
)
from google.cloud.bigtable.data.mutations import RowMutationEntry
from google.cloud.bigtable_v2.services.bigtable.client import (
BigtableClient as GapicClientType,
)
Expand Down
3 changes: 3 additions & 0 deletions packages/google-cloud-bigtable/google/cloud/bigtable/data/_sync_autogen/_read_rows.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,12 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

from typing import TYPE_CHECKING, Sequence

from google.api_core import retry as retries
from google.api_core.retry import exponential_sleep_generator

from google.cloud.bigtable.data._cross_sync import CrossSync
from google.cloud.bigtable.data._helpers import (
_attempt_timeout_generator,
Expand Down
5 changes: 3 additions & 2 deletions ...ages/google-cloud-bigtable/google/cloud/bigtable/data/_sync_autogen/_swappable_channel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

from typing import Callable
from grpc import ChannelConnectivity
from grpc import Channel

from grpc import Channel, ChannelConnectivity


class _WrappedChannel(Channel):
Expand Down
55 changes: 24 additions & 31 deletions packages/google-cloud-bigtable/google/cloud/bigtable/data/_sync_autogen/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,16 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

import abc
import concurrent.futures
import os
import random
import time
import warnings
from functools import partial
from typing import TYPE_CHECKING, Any, Callable, Optional, Sequence, Set, cast
from typing import TYPE_CHECKING, Any, Callable, Iterable, Optional, Sequence, Set, cast

import google.auth._default
import google.auth.credentials
from google.api_core import client_options as client_options_lib
Expand All @@ -39,7 +41,8 @@
from google.cloud.environment_vars import BIGTABLE_EMULATOR
from google.protobuf.internal.enum_type_wrapper import EnumTypeWrapper
from google.protobuf.message import Message
from grpc import Channel
from grpc import Channel, insecure_channel, intercept_channel

from google.cloud.bigtable.client import _DEFAULT_BIGTABLE_EMULATOR_CLIENT
from google.cloud.bigtable.data._cross_sync import CrossSync
from google.cloud.bigtable.data._helpers import (
Expand All @@ -55,6 +58,13 @@
_WarmedInstanceKey,
)
from google.cloud.bigtable.data._metrics import BigtableClientSideMetricsController
from google.cloud.bigtable.data._sync_autogen._swappable_channel import (
SwappableChannel as SwappableChannelType,
)
from google.cloud.bigtable.data._sync_autogen.metrics_interceptor import (
BigtableMetricsInterceptor as MetricsInterceptorType,
)
from google.cloud.bigtable.data._sync_autogen.mutations_batcher import _MB_SIZE
from google.cloud.bigtable.data.exceptions import (
FailedQueryShardError,
ShardedReadRowsExceptionGroup,
Expand All @@ -78,6 +88,10 @@
RowFilterChain,
StripValueTransformerFilter,
)
from google.cloud.bigtable_v2.services.bigtable import BigtableClient as GapicClient
from google.cloud.bigtable_v2.services.bigtable.transports import (
BigtableGrpcTransport as TransportType,
)
from google.cloud.bigtable_v2.services.bigtable.transports.base import (
DEFAULT_CLIENT_INFO,
)
Expand All @@ -88,19 +102,6 @@
ReadModifyWriteRowRequest,
SampleRowKeysRequest,
)
from typing import Iterable
from grpc import insecure_channel, intercept_channel
from google.cloud.bigtable.data._sync_autogen._swappable_channel import (
SwappableChannel as SwappableChannelType,
)
from google.cloud.bigtable.data._sync_autogen.metrics_interceptor import (
BigtableMetricsInterceptor as MetricsInterceptorType,
)
from google.cloud.bigtable.data._sync_autogen.mutations_batcher import _MB_SIZE
from google.cloud.bigtable_v2.services.bigtable import BigtableClient as GapicClient
from google.cloud.bigtable_v2.services.bigtable.transports import (
BigtableGrpcTransport as TransportType,
)

if TYPE_CHECKING:
from google.cloud.bigtable.data._helpers import RowKeySamples, ShardedQuery
Expand Down Expand Up @@ -297,8 +298,7 @@ def _ping_and_warm_instances(
instance_key: if provided, only warm the instance associated with the key
channel: grpc channel to warm. If none, warms `self.transport.grpc_channel`
Returns:
list[BaseException | None]: sequence of results or exceptions from the ping requests
"""
list[BaseException | None]: sequence of results or exceptions from the ping requests"""
channel = channel or self.transport.grpc_channel
instance_list = (
[instance_key] if instance_key is not None else self._active_instances
Expand Down Expand Up @@ -654,8 +654,7 @@ def execute_query(
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error
google.cloud.bigtable.data.exceptions.ParameterTypeInferenceFailed: Raised if
a parameter is passed without an explicit type, and the type cannot be infered
google.protobuf.message.DecodeError: raised if the deserialization of a PROTO/ENUM value fails.
"""
google.protobuf.message.DecodeError: raised if the deserialization of a PROTO/ENUM value fails."""
instance_name = self._gapic_client.instance_path(self.project, instance_id)
converted_param_types = _to_param_types(parameters, parameter_types)
prepare_request = {
Expand Down Expand Up @@ -893,8 +892,7 @@ def read_rows_stream(
google.api_core.exceptions.DeadlineExceeded: raised after operation timeout
will be chained with a RetryExceptionGroup containing GoogleAPIError exceptions
from any retries that failed
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error
"""
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error"""
operation_timeout, attempt_timeout = _get_timeouts(
operation_timeout, attempt_timeout, self
)
Expand Down Expand Up @@ -944,8 +942,7 @@ def read_rows(
google.api_core.exceptions.DeadlineExceeded: raised after operation timeout
will be chained with a RetryExceptionGroup containing GoogleAPIError exceptions
from any retries that failed
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error
"""
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error"""
row_generator = self.read_rows_stream(
query,
operation_timeout=operation_timeout,
Expand Down Expand Up @@ -987,8 +984,7 @@ def read_row(
google.api_core.exceptions.DeadlineExceeded: raised after operation timeout
will be chained with a RetryExceptionGroup containing GoogleAPIError exceptions
from any retries that failed
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error
"""
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error"""
if row_key is None:
raise ValueError("row_key must be string or bytes")
query = ReadRowsQuery(row_keys=row_key, row_filter=row_filter, limit=1)
Expand Down Expand Up @@ -1121,8 +1117,7 @@ def row_exists(
google.api_core.exceptions.DeadlineExceeded: raised after operation timeout
will be chained with a RetryExceptionGroup containing GoogleAPIError exceptions
from any retries that failed
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error
"""
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error"""
if row_key is None:
raise ValueError("row_key must be string or bytes")
strip_filter = StripValueTransformerFilter(flag=True)
Expand Down Expand Up @@ -1172,8 +1167,7 @@ def sample_row_keys(
google.api_core.exceptions.DeadlineExceeded: raised after operation timeout
will be chained with a RetryExceptionGroup containing GoogleAPIError exceptions
from any retries that failed
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error
"""
google.api_core.exceptions.GoogleAPIError: raised if the request encounters an unrecoverable error"""
operation_timeout, attempt_timeout = _get_timeouts(
operation_timeout, attempt_timeout, self
)
Expand Down Expand Up @@ -1236,8 +1230,7 @@ def mutations_batcher(
batch_retryable_errors: a list of errors that will be retried if encountered.
Defaults to the Table's default_mutate_rows_retryable_errors.
Returns:
MutationsBatcher: a MutationsBatcher context manager that can batch requests
"""
MutationsBatcher: a MutationsBatcher context manager that can batch requests"""
return CrossSync._Sync_Impl.MutationsBatcher(
self,
flush_interval=flush_interval,
Expand Down
5 changes: 4 additions & 1 deletion ...ges/google-cloud-bigtable/google/cloud/bigtable/data/_sync_autogen/metrics_interceptor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,18 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

import time
from functools import wraps
from typing import Sequence

from grpc import UnaryStreamClientInterceptor, UnaryUnaryClientInterceptor

from google.cloud.bigtable.data._metrics.data_model import (
ActiveOperationMetric,
OperationState,
OperationType,
)
from grpc import UnaryStreamClientInterceptor, UnaryUnaryClientInterceptor


def _with_active_operation(func):
Expand Down
10 changes: 5 additions & 5 deletions packages/google-cloud-bigtable/google/cloud/bigtable/data/_sync_autogen/mutations_batcher.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,13 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

import atexit
import concurrent.futures
import warnings
from collections import deque
from typing import TYPE_CHECKING, Sequence, cast

from google.cloud.bigtable.data._cross_sync import CrossSync
from google.cloud.bigtable.data._helpers import (
TABLE_DEFAULT,
Expand All @@ -37,10 +39,10 @@
)

if TYPE_CHECKING:
from google.cloud.bigtable.data.mutations import RowMutationEntry
from google.cloud.bigtable.data._sync_autogen.client import (
_DataApiTarget as TargetType,
)
from google.cloud.bigtable.data.mutations import RowMutationEntry
_MB_SIZE = 1024 * 1024


Expand Down Expand Up @@ -86,8 +88,7 @@ def _has_capacity(self, additional_count: int, additional_size: int) -> bool:
additional_count: number of mutations in the pending entry
additional_size: size of the pending entry
Returns:
bool: True if there is capacity to send the pending entry, False otherwise
"""
bool: True if there is capacity to send the pending entry, False otherwise"""
acceptable_size = max(self._max_mutation_bytes, additional_size)
acceptable_count = max(self._max_mutation_count, additional_count)
new_size = self._in_flight_mutation_bytes + additional_size
Expand Down Expand Up @@ -440,8 +441,7 @@ def _wait_for_batch_results(
list of Exceptions encountered by any of the tasks. Errors are expected
to be FailedMutationEntryError, representing a failed mutation operation.
If a task fails with a different exception, it will be included in the
output list. Successful tasks will not be represented in the output list.
"""
output list. Successful tasks will not be represented in the output list."""
if not tasks:
return []
exceptions: list[Exception] = []
Expand Down
12 changes: 6 additions & 6 deletions ...bigtable/google/cloud/bigtable/data/execute_query/_sync_autogen/execute_query_iterator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,13 @@
# This file is automatically generated by CrossSync. Do not edit manually.

from __future__ import annotations

from typing import TYPE_CHECKING, Any, Dict, Optional, Sequence, Tuple

from google.api_core import retry as retries
from google.protobuf.internal.enum_type_wrapper import EnumTypeWrapper
from google.protobuf.message import Message

from google.cloud.bigtable.data._cross_sync import CrossSync
from google.cloud.bigtable.data._helpers import (
_attempt_timeout_generator,
Expand Down Expand Up @@ -89,8 +92,7 @@ def __init__(
for protobuf deserialization.
Raises:
None
:class:`ValueError <exceptions.ValueError>` as a safeguard if data is processed in an unexpected state
"""
:class:`ValueError <exceptions.ValueError>` as a safeguard if data is processed in an unexpected state"""
self._table_name = None
self._app_profile_id = app_profile_id
self._client = client
Expand Down Expand Up @@ -193,8 +195,7 @@ def _next_impl(self) -> CrossSync._Sync_Impl.Iterator[QueryResultRow]:
def __next__(self) -> QueryResultRow:
"""Yields QueryResultRows representing the results of the query.

:raises: :class:`ValueError <exceptions.ValueError>` as a safeguard if data is processed in an unexpected state
"""
:raises: :class:`ValueError <exceptions.ValueError>` as a safeguard if data is processed in an unexpected state"""
if self._is_closed:
raise CrossSync._Sync_Impl.StopIteration
return self._result_generator.__next__()
Expand Down Expand Up @@ -242,8 +243,7 @@ def close(self) -> None:

Called automatically by iterator

:raises: :class:`ValueError <exceptions.ValueError>` if called in an invalid state
"""
:raises: :class:`ValueError <exceptions.ValueError>` if called in an invalid state"""
self._close_internal()

def _close_internal(self) -> None:
Expand Down
6 changes: 2 additions & 4 deletions packages/google-cloud-bigtable/noxfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,7 @@
"pytest",
"pytest-cov",
"pytest-asyncio",
BLACK_VERSION,
"autoflake",
RUFF_VERSION,
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The variable RUFF_VERSION is used here, but its definition is not visible in the provided diff. If it was intended to replace BLACK_VERSION, please ensure that RUFF_VERSION is defined as a constant (e.g., RUFF_VERSION = "ruff==0.4.4") earlier in the file and that the now-unused BLACK_VERSION definition is removed.

]
UNIT_TEST_EXTERNAL_DEPENDENCIES: List[str] = []
UNIT_TEST_LOCAL_DEPENDENCIES: List[str] = []
Expand Down Expand Up @@ -553,8 +552,7 @@ def generate_sync(session):
"""
Re-generate sync files for the library from CrossSync-annotated async source
"""
session.install(BLACK_VERSION)
session.install("autoflake")
session.install(RUFF_VERSION)
session.run("python", ".cross_sync/generate.py", ".")


Expand Down
Loading
Loading