Skip to content

Commit 04a11a6

Browse files
lszinvrahul2393
andauthored
chore: Add Custom OpenTelemetry Exporter in for Service Metrics (#1273)
* chore: Add Custom OpenTelemetry Exporter in for Service Metrics * Updated copyright dates to 2025 --------- Co-authored-by: rahul2393 <[email protected]>
1 parent 959bb9c commit 04a11a6

12 files changed

+970
-0
lines changed
+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Custom Metric Exporter
2+
The custom metric exporter, as defined in [metrics_exporter.py](./metrics_exporter.py), is designed to work in conjunction with OpenTelemetry and the Spanner client. It converts data into its protobuf equivalent and sends it to Google Cloud Monitoring.
3+
4+
## Filtering Criteria
5+
The exporter filters metrics based on the following conditions, utilizing values defined in [constants.py](./constants.py):
6+
7+
* Metrics with a scope set to `gax-python`.
8+
* Metrics with one of the following predefined names:
9+
* `attempt_latencies`
10+
* `attempt_count`
11+
* `operation_latencies`
12+
* `operation_count`
13+
* `gfe_latency`
14+
* `gfe_missing_header_count`
15+
16+
## Service Endpoint
17+
The exporter sends metrics to the Google Cloud Monitoring [service endpoint](https://cloud.google.com/python/docs/reference/monitoring/latest/google.cloud.monitoring_v3.services.metric_service.MetricServiceClient#google_cloud_monitoring_v3_services_metric_service_MetricServiceClient_create_service_time_series), distinct from the regular client endpoint. This service endpoint operates under a different quota limit than the user endpoint and features an additional server-side filter that only permits a predefined set of metrics to pass through.
18+
19+
When introducing new service metrics, it is essential to ensure they are allowed through by the server-side filter as well.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
BUILT_IN_METRICS_METER_NAME = "gax-python"
16+
NATIVE_METRICS_PREFIX = "spanner.googleapis.com/internal/client"
17+
SPANNER_RESOURCE_TYPE = "spanner_instance_client"
18+
19+
# Monitored resource labels
20+
MONITORED_RES_LABEL_KEY_PROJECT = "project_id"
21+
MONITORED_RES_LABEL_KEY_INSTANCE = "instance_id"
22+
MONITORED_RES_LABEL_KEY_INSTANCE_CONFIG = "instance_config"
23+
MONITORED_RES_LABEL_KEY_LOCATION = "location"
24+
MONITORED_RES_LABEL_KEY_CLIENT_HASH = "client_hash"
25+
MONITORED_RESOURCE_LABELS = [
26+
MONITORED_RES_LABEL_KEY_PROJECT,
27+
MONITORED_RES_LABEL_KEY_INSTANCE,
28+
MONITORED_RES_LABEL_KEY_INSTANCE_CONFIG,
29+
MONITORED_RES_LABEL_KEY_LOCATION,
30+
MONITORED_RES_LABEL_KEY_CLIENT_HASH,
31+
]
32+
33+
# Metric labels
34+
METRIC_LABEL_KEY_CLIENT_UID = "client_uid"
35+
METRIC_LABEL_KEY_CLIENT_NAME = "client_name"
36+
METRIC_LABEL_KEY_DATABASE = "database"
37+
METRIC_LABEL_KEY_METHOD = "method"
38+
METRIC_LABEL_KEY_STATUS = "status"
39+
METRIC_LABEL_KEY_DIRECT_PATH_ENABLED = "directpath_enabled"
40+
METRIC_LABEL_KEY_DIRECT_PATH_USED = "directpath_used"
41+
METRIC_LABELS = [
42+
METRIC_LABEL_KEY_CLIENT_UID,
43+
METRIC_LABEL_KEY_CLIENT_NAME,
44+
METRIC_LABEL_KEY_DATABASE,
45+
METRIC_LABEL_KEY_METHOD,
46+
METRIC_LABEL_KEY_STATUS,
47+
METRIC_LABEL_KEY_DIRECT_PATH_ENABLED,
48+
METRIC_LABEL_KEY_DIRECT_PATH_USED,
49+
]
50+
51+
# Metric names
52+
METRIC_NAME_OPERATION_LATENCIES = "operation_latencies"
53+
METRIC_NAME_ATTEMPT_LATENCIES = "attempt_latencies"
54+
METRIC_NAME_OPERATION_COUNT = "operation_count"
55+
METRIC_NAME_ATTEMPT_COUNT = "attempt_count"
56+
METRIC_NAME_GFE_LATENCY = "gfe_latency"
57+
METRIC_NAME_GFE_MISSING_HEADER_COUNT = "gfe_missing_header_count"
58+
METRIC_NAMES = [
59+
METRIC_NAME_OPERATION_LATENCIES,
60+
METRIC_NAME_ATTEMPT_LATENCIES,
61+
METRIC_NAME_OPERATION_COUNT,
62+
METRIC_NAME_ATTEMPT_COUNT,
63+
]

0 commit comments

Comments
 (0)